2024-09-11T23:02:16.065 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T23:02:16.092 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T23:02:16.190 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901461 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901461' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request - SLOW_OPS - slow request sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEVMjnznru40KOvpc3ZmdhPIFCgPNN1I+CtDWl8r/KSjiaJExJiXxHR9kbiX0ndVLfwI70pBwD5Nf9hHs7tIWUs= smithi090.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGDotrA3rOVqcEMiVqvrtMF0FjenZ9wnqU18Jsg8J9lm4g9qhEDW+zjGly1HVA+fkBawltG7SZuQn3a1YC1TZIY= smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIXlvNBVEgZCUfG+58H+rOnINhZ7ZWVdxlfD/y1mm5DGArI+cQQQxsppcGXLfuDizfYiKDTF6OZ5fctO4W6kanw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T23:02:16.191 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-11T23:02:16.191 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-11T23:02:16.192 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T23:02:16.193 INFO:teuthology.task.internal:Checking packages... 2024-09-11T23:02:16.216 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-11T23:02:16.216 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T23:02:16.216 INFO:teuthology.packaging:ref: None 2024-09-11T23:02:16.216 INFO:teuthology.packaging:tag: None 2024-09-11T23:02:16.216 INFO:teuthology.packaging:branch: squid 2024-09-11T23:02:16.216 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:02:16.216 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-11T23:02:16.462 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-11T23:02:16.464 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T23:02:16.470 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T23:02:16.470 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T23:02:16.520 INFO:teuthology.task.internal:Saving configuration 2024-09-11T23:02:16.541 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T23:02:16.546 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T23:02:16.578 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901461', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 22:56:35.888697', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEVMjnznru40KOvpc3ZmdhPIFCgPNN1I+CtDWl8r/KSjiaJExJiXxHR9kbiX0ndVLfwI70pBwD5Nf9hHs7tIWUs='} 2024-09-11T23:02:16.611 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901461', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 22:56:35.891259', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGDotrA3rOVqcEMiVqvrtMF0FjenZ9wnqU18Jsg8J9lm4g9qhEDW+zjGly1HVA+fkBawltG7SZuQn3a1YC1TZIY='} 2024-09-11T23:02:16.643 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901461', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 22:56:35.886611', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIXlvNBVEgZCUfG+58H+rOnINhZ7ZWVdxlfD/y1mm5DGArI+cQQQxsppcGXLfuDizfYiKDTF6OZ5fctO4W6kanw='} 2024-09-11T23:02:16.644 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T23:02:16.650 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-11T23:02:16.650 INFO:teuthology.task.internal:roles: ubuntu@smithi090.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-11T23:02:16.650 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-11T23:02:16.650 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T23:02:16.834 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f13c57af5b0>, signals=[15]) 2024-09-11T23:02:16.834 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T23:02:17.605 INFO:teuthology.task.internal:Opening connections... 2024-09-11T23:02:17.605 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2024-09-11T23:02:17.606 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:02:17.706 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi090.front.sepia.ceph.com 2024-09-11T23:02:17.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:02:17.789 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2024-09-11T23:02:17.790 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:02:17.895 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T23:02:17.904 DEBUG:teuthology.orchestra.run.smithi043:> uname -m 2024-09-11T23:02:17.922 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2024-09-11T23:02:17.923 DEBUG:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2024-09-11T23:02:17.980 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Stream" 2024-09-11T23:02:17.981 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="9" 2024-09-11T23:02:17.981 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos" 2024-09-11T23:02:17.981 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora" 2024-09-11T23:02:17.981 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="9" 2024-09-11T23:02:17.981 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el9" 2024-09-11T23:02:17.981 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T23:02:17.981 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2024-09-11T23:02:17.981 INFO:teuthology.orchestra.run.smithi043.stdout:LOGO="fedora-logo-icon" 2024-09-11T23:02:17.981 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T23:02:17.981 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://centos.org/" 2024-09-11T23:02:17.981 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T23:02:17.981 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T23:02:17.981 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T23:02:17.982 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2024-09-11T23:02:18.016 DEBUG:teuthology.orchestra.run.smithi090:> uname -m 2024-09-11T23:02:18.034 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64 2024-09-11T23:02:18.035 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2024-09-11T23:02:18.093 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream" 2024-09-11T23:02:18.093 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9" 2024-09-11T23:02:18.094 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos" 2024-09-11T23:02:18.094 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora" 2024-09-11T23:02:18.094 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9" 2024-09-11T23:02:18.094 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9" 2024-09-11T23:02:18.094 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T23:02:18.094 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31" 2024-09-11T23:02:18.094 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon" 2024-09-11T23:02:18.094 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T23:02:18.094 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/" 2024-09-11T23:02:18.094 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T23:02:18.094 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T23:02:18.094 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T23:02:18.095 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2024-09-11T23:02:18.124 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-09-11T23:02:18.143 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-09-11T23:02:18.143 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-09-11T23:02:18.201 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="CentOS Stream" 2024-09-11T23:02:18.202 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="9" 2024-09-11T23:02:18.202 INFO:teuthology.orchestra.run.smithi097.stdout:ID="centos" 2024-09-11T23:02:18.202 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="rhel fedora" 2024-09-11T23:02:18.202 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="9" 2024-09-11T23:02:18.202 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el9" 2024-09-11T23:02:18.202 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T23:02:18.202 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2024-09-11T23:02:18.202 INFO:teuthology.orchestra.run.smithi097.stdout:LOGO="fedora-logo-icon" 2024-09-11T23:02:18.202 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T23:02:18.202 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://centos.org/" 2024-09-11T23:02:18.202 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T23:02:18.202 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T23:02:18.202 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T23:02:18.203 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-09-11T23:02:18.240 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T23:02:18.252 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T23:02:18.271 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T23:02:18.271 DEBUG:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2024-09-11T23:02:18.273 DEBUG:teuthology.orchestra.run.smithi090:> test '!' -e /home/ubuntu/cephtest 2024-09-11T23:02:18.276 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2024-09-11T23:02:18.293 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T23:02:18.299 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T23:02:18.299 DEBUG:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2024-09-11T23:02:18.331 DEBUG:teuthology.orchestra.run.smithi090:> test -z $(ls -A /var/lib/ceph) 2024-09-11T23:02:18.333 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2024-09-11T23:02:18.421 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T23:02:18.523 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T23:02:18.537 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-11T23:02:18.537 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-11T23:02:18.537 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-11T23:02:18.538 DEBUG:teuthology.orchestra.run.smithi043:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T23:02:18.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:02:18.556 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-09-11T23:02:18.614 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-503.el9.x86_64 2024-09-11T23:02:18.615 INFO:teuthology.task.kernel:Running kernel on smithi043: 5.14.0-503.el9.x86_64 2024-09-11T23:02:18.615 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum install -y kernel 2024-09-11T23:02:20.991 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-11T23:02:22.834 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-11T23:02:27.409 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-11T23:02:29.110 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-11T23:02:29.898 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-11T23:02:35.553 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-11T23:02:37.652 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T23:02:37.652 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T23:02:37.749 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T23:02:37.753 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T23:02:37.753 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-11T23:02:37.753 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T23:02:37.753 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-11T23:02:37.754 INFO:teuthology.orchestra.run.smithi043.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T23:02:37.754 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-11T23:02:37.754 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T23:02:37.754 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T23:02:37.754 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T23:02:37.754 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:02:37.754 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-11T23:02:37.754 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-11T23:02:37.754 INFO:teuthology.orchestra.run.smithi043.stdout:Install 4 Packages 2024-09-11T23:02:37.754 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:02:37.755 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 79 M 2024-09-11T23:02:37.755 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 126 M 2024-09-11T23:02:37.755 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-11T23:02:38.580 INFO:teuthology.orchestra.run.smithi043.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 42 kB/s | 23 kB 00:00 2024-09-11T23:02:39.455 INFO:teuthology.orchestra.run.smithi043.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-11T23:02:40.389 INFO:teuthology.orchestra.run.smithi043.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 15 MB/s | 35 MB 00:02 2024-09-11T23:02:41.089 INFO:teuthology.orchestra.run.smithi043.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 11 MB/s | 29 MB 00:02 2024-09-11T23:02:41.090 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T23:02:41.090 INFO:teuthology.orchestra.run.smithi043.stdout:Total 24 MB/s | 79 MB 00:03 2024-09-11T23:02:41.801 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-11T23:02:42.160 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-11T23:02:42.160 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-11T23:02:43.146 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-11T23:02:43.146 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-11T23:02:45.720 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-11T23:02:46.403 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T23:02:46.492 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T23:02:49.294 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T23:02:49.418 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T23:02:54.548 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T23:02:54.586 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:02:59.838 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:03:38.922 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:03:38.965 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:03:39.362 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:03:39.362 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T23:03:39.362 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T23:03:39.362 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T23:03:40.244 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T23:03:40.245 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:03:40.245 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-11T23:03:40.245 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T23:03:40.245 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T23:03:40.245 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T23:03:40.245 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T23:03:40.245 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:03:40.245 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T23:03:40.642 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T23:03:40.729 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T23:03:40.730 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T23:03:40.730 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T23:03:40.730 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T23:03:40.730 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T23:03:40.730 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-11T23:03:40.730 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi043.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T23:03:40.730 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T23:03:40.789 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T23:03:40.790 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T23:03:40.790 DEBUG:teuthology.orchestra.run.smithi043:> sudo rpm -qi grub2-tools 2024-09-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi043.stdout:Name : grub2-tools 2024-09-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi043.stdout:Epoch : 1 2024-09-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi043.stdout:Version : 2.06 2024-09-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi043.stdout:Release : 82.el9 2024-09-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi043.stdout:Architecture: x86_64 2024-09-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi043.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi043.stdout:Group : Unspecified 2024-09-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi043.stdout:Size : 8274670 2024-09-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi043.stdout:License : GPLv3+ 2024-09-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi043.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi043.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi043.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi043.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi043.stdout:Packager : builder@centos.org 2024-09-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi043.stdout:Vendor : CentOS 2024-09-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi043.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi043.stdout:Summary : Support tools for GRUB. 2024-09-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi043.stdout:Description : 2024-09-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi043.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi043.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi043.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T23:03:40.829 INFO:teuthology.orchestra.run.smithi043.stdout:hardware devices. 2024-09-11T23:03:40.829 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:03:40.829 INFO:teuthology.orchestra.run.smithi043.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T23:03:40.830 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T23:03:40.830 INFO:teuthology.task.kernel:Updating grub on smithi043 to boot 5.14.0-505.el9.x86_64 2024-09-11T23:03:40.830 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T23:03:41.452 INFO:teuthology.orchestra.run.smithi043.stderr:Generating grub configuration file ... 2024-09-11T23:03:42.797 INFO:teuthology.orchestra.run.smithi043.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T23:03:42.823 INFO:teuthology.orchestra.run.smithi043.stderr:done 2024-09-11T23:03:42.832 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-09-11T23:03:42.848 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.RTJy5PXlfO 2024-09-11T23:03:42.848 DEBUG:teuthology.orchestra.run.smithi043:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.RTJy5PXlfO 2024-09-11T23:03:42.923 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.RTJy5PXlfO 2024-09-11T23:03:43.068 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.RTJy5PXlfO is 6KB 2024-09-11T23:03:43.116 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.RTJy5PXlfO 2024-09-11T23:03:43.131 DEBUG:teuthology.orchestra.run.smithi043:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T23:03:43.194 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T23:03:43.194 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T23:03:43.194 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T23:03:43.194 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T23:03:43.195 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T23:03:43.352 DEBUG:teuthology.orchestra.run.smithi043:> sudo shutdown -r now 2024-09-11T23:04:13.356 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-11T23:04:13.357 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-11T23:04:13.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:05:13.369 DEBUG:teuthology.orchestra.remote:timed out 2024-09-11T23:05:22.371 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-11T23:05:22.372 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:05:22.864 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-09-11T23:05:23.233 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-11T23:05:23.233 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-11T23:05:23.233 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-09-11T23:05:23.248 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-505.el9.x86_64 2024-09-11T23:05:23.248 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T23:05:23.248 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T23:05:23.249 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T23:05:24.249 DEBUG:teuthology.parallel:result is None 2024-09-11T23:05:24.249 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T23:05:24.257 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T23:05:24.257 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T23:05:24.259 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T23:05:24.262 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T23:05:24.280 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T23:05:24.293 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T23:05:24.327 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T23:05:24.327 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T23:05:24.330 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T23:05:24.332 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T23:05:24.388 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T23:05:24.400 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T23:05:24.401 DEBUG:teuthology.orchestra.run.smithi090:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T23:05:24.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:05:24.418 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T23:05:24.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:05:24.435 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T23:05:24.437 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T23:05:24.461 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T23:05:24.494 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:05:24.510 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:05:24.523 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:05:24.531 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:05:24.553 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:05:24.593 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T23:05:24.594 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T23:05:24.602 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T23:05:24.602 DEBUG:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T23:05:24.604 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T23:05:24.607 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T23:05:24.636 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T23:05:24.663 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T23:05:24.663 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T23:05:24.671 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T23:05:24.675 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T23:05:24.692 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:05:24.775 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:05:24.818 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:05:24.903 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:05:24.940 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T23:05:24.940 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T23:05:25.005 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:05:25.060 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:05:25.100 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:05:25.194 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:05:25.227 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-11T23:05:25.228 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T23:05:25.294 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:05:25.349 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T23:05:25.397 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:05:25.486 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T23:05:25.525 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-11T23:05:25.525 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T23:05:25.592 DEBUG:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2024-09-11T23:05:25.598 DEBUG:teuthology.orchestra.run.smithi090:> sudo service rsyslog restart 2024-09-11T23:05:25.600 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2024-09-11T23:05:25.634 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:05:25.667 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:05:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:05:26.087 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T23:05:26.094 INFO:teuthology.task.internal:Starting timer... 2024-09-11T23:05:26.094 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T23:05:26.140 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T23:05:26.171 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-11T23:05:26.171 DEBUG:teuthology.orchestra.run.smithi043:> sudo service auditd rotate 2024-09-11T23:05:26.300 INFO:teuthology.orchestra.run.smithi043.stdout:Rotating logs: 2024-09-11T23:05:26.302 DEBUG:teuthology.orchestra.run.smithi090:> sudo service auditd rotate 2024-09-11T23:05:26.360 INFO:teuthology.orchestra.run.smithi090.stdout:Rotating logs: 2024-09-11T23:05:26.362 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2024-09-11T23:05:26.418 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2024-09-11T23:05:26.419 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T23:05:26.420 DEBUG:teuthology.orchestra.run.smithi043:> /usr/sbin/getenforce 2024-09-11T23:05:26.486 INFO:teuthology.orchestra.run.smithi043.stdout:Permissive 2024-09-11T23:05:26.487 DEBUG:teuthology.orchestra.run.smithi090:> /usr/sbin/getenforce 2024-09-11T23:05:26.515 INFO:teuthology.orchestra.run.smithi090.stdout:Permissive 2024-09-11T23:05:26.516 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2024-09-11T23:05:26.535 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2024-09-11T23:05:26.536 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi043.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi090.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi097.front.sepia.ceph.com': 'permissive'} 2024-09-11T23:05:26.536 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:05:26.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:05:26.565 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:05:26.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:05:26.597 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:05:26.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:05:26.623 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T23:05:26.623 DEBUG:teuthology.orchestra.run.smithi043:> sudo /usr/sbin/setenforce permissive 2024-09-11T23:05:26.659 DEBUG:teuthology.orchestra.run.smithi090:> sudo /usr/sbin/setenforce permissive 2024-09-11T23:05:26.696 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2024-09-11T23:05:26.737 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T23:05:26.747 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-11T23:05:26.750 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-11T23:05:26.871 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T23:05:26.885 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T23:05:26.887 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi043.front.sepia.ceph.com,smithi090.front.sepia.ceph.com,smithi097.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T23:09:28.304 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi043.front.sepia.ceph.com'), Remote(name='ubuntu@smithi090.front.sepia.ceph.com'), Remote(name='ubuntu@smithi097.front.sepia.ceph.com')] 2024-09-11T23:09:28.306 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-11T23:09:28.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:09:28.393 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-09-11T23:09:28.477 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-11T23:09:28.477 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-09-11T23:09:28.478 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:09:28.560 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-09-11T23:09:28.645 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-09-11T23:09:28.646 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-11T23:09:28.646 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T23:09:28.728 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-09-11T23:09:28.808 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-11T23:09:28.808 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T23:09:28.818 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T23:09:28.818 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T23:09:28.819 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:09:28.821 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T23:09:28.821 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:09:28.823 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T23:09:28.824 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:09:28.850 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T23:09:28.854 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T23:09:28.865 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T23:09:28.869 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T23:09:28.878 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T23:09:28.890 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: ntpd: command not found 2024-09-11T23:09:28.892 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T23:09:28.895 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: ntpd: command not found 2024-09-11T23:09:28.902 INFO:teuthology.orchestra.run.smithi043.stdout:506 Cannot talk to daemon 2024-09-11T23:09:28.907 INFO:teuthology.orchestra.run.smithi090.stdout:506 Cannot talk to daemon 2024-09-11T23:09:28.915 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2024-09-11T23:09:28.917 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T23:09:28.921 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T23:09:28.927 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2024-09-11T23:09:28.932 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T23:09:28.936 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T23:09:28.941 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T23:09:28.955 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T23:09:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:09:28.990 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:09:28.992 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:09:28.992 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-09-11T23:09:28.992 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:09:28.993 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:09:28.993 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:09:28.993 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:09:28.994 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:09:28.994 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-09-11T23:09:28.995 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:09:28.995 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:09:28.995 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:09:28.995 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:09:29.012 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:09:29.016 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:09:29.016 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-09-11T23:09:29.016 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:09:29.016 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:09:29.016 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:09:29.016 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:09:29.016 INFO:teuthology.run_tasks:Running task install... 2024-09-11T23:09:29.028 DEBUG:teuthology.task.install:project ceph 2024-09-11T23:09:29.028 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-11T23:09:29.028 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-11T23:09:29.028 INFO:teuthology.task.install:Using flavor: default 2024-09-11T23:09:29.039 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T23:09:29.039 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-11T23:09:29.039 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T23:09:29.040 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:09:29.043 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T23:09:29.043 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:09:29.044 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T23:09:29.044 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:09:29.202 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T23:09:29.203 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T23:09:29.219 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T23:09:29.220 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T23:09:29.221 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T23:09:29.221 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T23:09:29.327 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T23:09:29.328 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T23:09:29.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T23:09:29.336 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T23:09:29.336 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-11T23:09:29.336 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T23:09:29.345 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T23:09:29.345 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-11T23:09:29.345 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T23:09:29.355 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T23:09:29.355 DEBUG:teuthology.orchestra.run.smithi043:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T23:09:29.361 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T23:09:29.362 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T23:09:29.372 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T23:09:29.372 DEBUG:teuthology.orchestra.run.smithi090:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T23:09:29.421 DEBUG:teuthology.orchestra.run.smithi043:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T23:09:29.428 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T23:09:29.438 DEBUG:teuthology.orchestra.run.smithi090:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T23:09:29.498 DEBUG:teuthology.orchestra.run.smithi043:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T23:09:29.502 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T23:09:29.513 DEBUG:teuthology.orchestra.run.smithi090:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T23:09:29.563 INFO:teuthology.orchestra.run.smithi043.stdout:check_obsoletes = 1 2024-09-11T23:09:29.565 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum clean all 2024-09-11T23:09:29.566 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2024-09-11T23:09:29.567 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2024-09-11T23:09:29.577 INFO:teuthology.orchestra.run.smithi090.stdout:check_obsoletes = 1 2024-09-11T23:09:29.578 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum clean all 2024-09-11T23:09:29.792 INFO:teuthology.orchestra.run.smithi097.stdout:45 files removed 2024-09-11T23:09:29.805 INFO:teuthology.orchestra.run.smithi090.stdout:45 files removed 2024-09-11T23:09:29.820 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T23:09:29.828 INFO:teuthology.orchestra.run.smithi043.stdout:45 files removed 2024-09-11T23:09:29.830 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T23:09:29.855 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T23:09:30.355 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 321 kB/s | 84 kB 00:00 2024-09-11T23:09:30.378 INFO:teuthology.orchestra.run.smithi090.stdout:ceph packages for x86_64 308 kB/s | 84 kB 00:00 2024-09-11T23:09:30.434 INFO:teuthology.orchestra.run.smithi043.stdout:ceph packages for x86_64 298 kB/s | 84 kB 00:00 2024-09-11T23:09:30.639 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-11T23:09:30.662 INFO:teuthology.orchestra.run.smithi090.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-11T23:09:30.726 INFO:teuthology.orchestra.run.smithi043.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-11T23:09:30.898 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-11T23:09:30.938 INFO:teuthology.orchestra.run.smithi090.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-11T23:09:31.135 INFO:teuthology.orchestra.run.smithi043.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-09-11T23:09:32.407 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 8.2 MB 00:01 2024-09-11T23:09:32.627 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 5.7 MB/s | 8.2 MB 00:01 2024-09-11T23:09:34.731 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 2.2 MB/s | 8.2 MB 00:03 2024-09-11T23:09:35.804 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:02 2024-09-11T23:09:38.441 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 4.1 MB/s | 20 MB 00:04 2024-09-11T23:09:40.587 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 7.8 MB/s | 6.5 MB 00:00 2024-09-11T23:09:41.225 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 3.8 MB/s | 20 MB 00:05 2024-09-11T23:09:42.296 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 19 kB 00:00 2024-09-11T23:09:42.884 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-11T23:09:43.525 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-11T23:09:44.376 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 19 kB 00:00 2024-09-11T23:09:45.139 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-11T23:09:46.601 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.5 MB 00:01 2024-09-11T23:09:48.119 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 19 kB 00:00 2024-09-11T23:09:49.032 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-11T23:09:49.223 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-11T23:09:50.629 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-11T23:09:51.636 INFO:teuthology.orchestra.run.smithi043.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:09:51.637 INFO:teuthology.orchestra.run.smithi043.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:09:51.651 INFO:teuthology.orchestra.run.smithi043.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T23:09:51.652 INFO:teuthology.orchestra.run.smithi043.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T23:09:51.653 INFO:teuthology.orchestra.run.smithi043.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T23:09:51.654 INFO:teuthology.orchestra.run.smithi043.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T23:09:51.656 INFO:teuthology.orchestra.run.smithi043.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T23:09:51.659 INFO:teuthology.orchestra.run.smithi043.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T23:09:51.660 INFO:teuthology.orchestra.run.smithi043.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T23:09:51.728 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-11T23:09:51.746 INFO:teuthology.orchestra.run.smithi043.stdout:====================================================================================== 2024-09-11T23:09:51.747 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-11T23:09:51.747 INFO:teuthology.orchestra.run.smithi043.stdout:====================================================================================== 2024-09-11T23:09:51.747 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-11T23:09:51.747 INFO:teuthology.orchestra.run.smithi043.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T23:09:51.747 INFO:teuthology.orchestra.run.smithi043.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T23:09:51.748 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T23:09:51.748 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T23:09:51.748 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T23:09:51.748 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T23:09:51.748 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T23:09:51.749 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T23:09:51.749 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T23:09:51.749 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T23:09:51.749 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T23:09:51.749 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T23:09:51.750 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T23:09:51.750 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T23:09:51.750 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T23:09:51.750 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T23:09:51.750 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T23:09:51.750 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T23:09:51.750 INFO:teuthology.orchestra.run.smithi043.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T23:09:51.751 INFO:teuthology.orchestra.run.smithi043.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T23:09:51.751 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T23:09:51.751 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T23:09:51.751 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T23:09:51.751 INFO:teuthology.orchestra.run.smithi043.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T23:09:51.752 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T23:09:51.752 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T23:09:51.752 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T23:09:51.752 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T23:09:51.752 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T23:09:51.753 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T23:09:51.753 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T23:09:51.753 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T23:09:51.753 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T23:09:51.753 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T23:09:51.754 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T23:09:51.754 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T23:09:51.754 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T23:09:51.754 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrading: 2024-09-11T23:09:51.754 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T23:09:51.755 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T23:09:51.755 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T23:09:51.755 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T23:09:51.755 INFO:teuthology.orchestra.run.smithi043.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T23:09:51.756 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T23:09:51.756 INFO:teuthology.orchestra.run.smithi043.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T23:09:51.756 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T23:09:51.756 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-11T23:09:51.757 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T23:09:51.757 INFO:teuthology.orchestra.run.smithi043.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T23:09:51.757 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T23:09:51.757 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T23:09:51.757 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T23:09:51.758 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T23:09:51.758 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T23:09:51.758 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T23:09:51.758 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T23:09:51.758 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T23:09:51.758 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T23:09:51.759 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T23:09:51.759 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T23:09:51.759 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T23:09:51.759 INFO:teuthology.orchestra.run.smithi043.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T23:09:51.759 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T23:09:51.760 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T23:09:51.760 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T23:09:51.760 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T23:09:51.760 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T23:09:51.761 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T23:09:51.761 INFO:teuthology.orchestra.run.smithi043.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T23:09:51.761 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T23:09:51.761 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T23:09:51.761 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T23:09:51.762 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T23:09:51.762 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T23:09:51.762 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T23:09:51.762 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T23:09:51.762 INFO:teuthology.orchestra.run.smithi043.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T23:09:51.763 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T23:09:51.763 INFO:teuthology.orchestra.run.smithi043.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T23:09:51.763 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T23:09:51.763 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T23:09:51.763 INFO:teuthology.orchestra.run.smithi043.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T23:09:51.763 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T23:09:51.763 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T23:09:51.763 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T23:09:51.763 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T23:09:51.764 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T23:09:51.764 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T23:09:51.764 INFO:teuthology.orchestra.run.smithi043.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T23:09:51.764 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T23:09:51.764 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T23:09:51.764 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T23:09:51.765 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T23:09:51.765 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T23:09:51.765 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T23:09:51.765 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T23:09:51.765 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T23:09:51.765 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T23:09:51.765 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T23:09:51.765 INFO:teuthology.orchestra.run.smithi043.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T23:09:51.766 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T23:09:51.766 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T23:09:51.766 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T23:09:51.766 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T23:09:51.766 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T23:09:51.766 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T23:09:51.766 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T23:09:51.766 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T23:09:51.766 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T23:09:51.767 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T23:09:51.767 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T23:09:51.767 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T23:09:51.767 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T23:09:51.767 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T23:09:51.768 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T23:09:51.768 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T23:09:51.768 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T23:09:51.768 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T23:09:51.768 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T23:09:51.768 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T23:09:51.769 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T23:09:51.769 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T23:09:51.769 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T23:09:51.769 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T23:09:51.769 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T23:09:51.770 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T23:09:51.770 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T23:09:51.770 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T23:09:51.770 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T23:09:51.771 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T23:09:51.771 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T23:09:51.771 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T23:09:51.771 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T23:09:51.771 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T23:09:51.771 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T23:09:51.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T23:09:51.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T23:09:51.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T23:09:51.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T23:09:51.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T23:09:51.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T23:09:51.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T23:09:51.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T23:09:51.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T23:09:51.772 INFO:teuthology.orchestra.run.smithi043.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T23:09:51.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T23:09:51.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T23:09:51.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T23:09:51.773 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T23:09:51.773 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T23:09:51.774 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T23:09:51.774 INFO:teuthology.orchestra.run.smithi043.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T23:09:51.774 INFO:teuthology.orchestra.run.smithi043.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T23:09:51.774 INFO:teuthology.orchestra.run.smithi043.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T23:09:51.774 INFO:teuthology.orchestra.run.smithi043.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T23:09:51.774 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T23:09:51.775 INFO:teuthology.orchestra.run.smithi043.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T23:09:51.775 INFO:teuthology.orchestra.run.smithi043.stdout:Installing weak dependencies: 2024-09-11T23:09:51.775 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T23:09:51.775 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:09:51.775 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-11T23:09:51.776 INFO:teuthology.orchestra.run.smithi043.stdout:====================================================================================== 2024-09-11T23:09:51.776 INFO:teuthology.orchestra.run.smithi043.stdout:Install 148 Packages 2024-09-11T23:09:51.776 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade 8 Packages 2024-09-11T23:09:51.776 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:09:51.777 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 226 M 2024-09-11T23:09:51.777 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-11T23:09:52.582 INFO:teuthology.orchestra.run.smithi043.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 52 kB/s | 6.5 kB 00:00 2024-09-11T23:09:52.791 INFO:teuthology.orchestra.run.smithi043.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 4.2 MB/s | 897 kB 00:00 2024-09-11T23:09:52.893 INFO:teuthology.orchestra.run.smithi043.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.4 MB/s | 145 kB 00:00 2024-09-11T23:09:52.907 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:09:52.908 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:09:52.922 INFO:teuthology.orchestra.run.smithi097.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T23:09:52.923 INFO:teuthology.orchestra.run.smithi097.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T23:09:52.924 INFO:teuthology.orchestra.run.smithi097.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T23:09:52.925 INFO:teuthology.orchestra.run.smithi097.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T23:09:52.927 INFO:teuthology.orchestra.run.smithi097.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T23:09:52.930 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T23:09:52.931 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T23:09:52.967 INFO:teuthology.orchestra.run.smithi043.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-11T23:09:52.993 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-11T23:09:53.009 INFO:teuthology.orchestra.run.smithi097.stdout:====================================================================================== 2024-09-11T23:09:53.010 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-09-11T23:09:53.010 INFO:teuthology.orchestra.run.smithi097.stdout:====================================================================================== 2024-09-11T23:09:53.010 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-09-11T23:09:53.010 INFO:teuthology.orchestra.run.smithi097.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T23:09:53.010 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T23:09:53.010 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T23:09:53.010 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T23:09:53.010 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T23:09:53.010 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T23:09:53.010 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T23:09:53.010 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T23:09:53.010 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T23:09:53.011 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T23:09:53.011 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T23:09:53.011 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T23:09:53.011 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T23:09:53.011 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T23:09:53.011 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T23:09:53.011 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T23:09:53.011 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T23:09:53.011 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T23:09:53.011 INFO:teuthology.orchestra.run.smithi097.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T23:09:53.011 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T23:09:53.011 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T23:09:53.012 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T23:09:53.012 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T23:09:53.012 INFO:teuthology.orchestra.run.smithi097.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T23:09:53.012 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T23:09:53.012 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T23:09:53.012 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T23:09:53.012 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T23:09:53.012 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T23:09:53.012 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T23:09:53.012 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T23:09:53.012 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T23:09:53.012 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T23:09:53.013 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T23:09:53.013 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T23:09:53.013 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T23:09:53.013 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T23:09:53.013 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2024-09-11T23:09:53.013 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T23:09:53.013 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T23:09:53.013 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T23:09:53.013 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T23:09:53.013 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T23:09:53.013 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T23:09:53.013 INFO:teuthology.orchestra.run.smithi097.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T23:09:53.013 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T23:09:53.014 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-09-11T23:09:53.014 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T23:09:53.014 INFO:teuthology.orchestra.run.smithi097.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T23:09:53.014 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T23:09:53.014 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T23:09:53.014 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T23:09:53.014 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T23:09:53.014 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T23:09:53.014 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T23:09:53.014 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T23:09:53.014 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T23:09:53.014 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T23:09:53.014 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T23:09:53.015 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T23:09:53.015 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T23:09:53.015 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T23:09:53.015 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T23:09:53.015 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T23:09:53.015 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T23:09:53.015 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T23:09:53.015 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T23:09:53.015 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T23:09:53.015 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T23:09:53.015 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T23:09:53.015 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T23:09:53.015 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T23:09:53.016 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T23:09:53.016 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T23:09:53.016 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T23:09:53.016 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T23:09:53.016 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T23:09:53.016 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T23:09:53.016 INFO:teuthology.orchestra.run.smithi097.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T23:09:53.016 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T23:09:53.016 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T23:09:53.016 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T23:09:53.016 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T23:09:53.016 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T23:09:53.016 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T23:09:53.017 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T23:09:53.017 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T23:09:53.017 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T23:09:53.017 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T23:09:53.017 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T23:09:53.017 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T23:09:53.017 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T23:09:53.017 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T23:09:53.017 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T23:09:53.017 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T23:09:53.017 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T23:09:53.017 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T23:09:53.018 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T23:09:53.018 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T23:09:53.018 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T23:09:53.018 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T23:09:53.018 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T23:09:53.018 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T23:09:53.018 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T23:09:53.018 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T23:09:53.018 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T23:09:53.018 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T23:09:53.018 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T23:09:53.018 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T23:09:53.019 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T23:09:53.019 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T23:09:53.019 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T23:09:53.019 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T23:09:53.019 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T23:09:53.019 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T23:09:53.019 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T23:09:53.019 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T23:09:53.019 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T23:09:53.019 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T23:09:53.019 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T23:09:53.019 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T23:09:53.020 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T23:09:53.020 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T23:09:53.020 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T23:09:53.020 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T23:09:53.020 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T23:09:53.020 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T23:09:53.020 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T23:09:53.020 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T23:09:53.020 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T23:09:53.020 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T23:09:53.020 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T23:09:53.020 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T23:09:53.020 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T23:09:53.021 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T23:09:53.021 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T23:09:53.021 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T23:09:53.021 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T23:09:53.021 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T23:09:53.021 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T23:09:53.021 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T23:09:53.021 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T23:09:53.021 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T23:09:53.021 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T23:09:53.021 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T23:09:53.022 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T23:09:53.022 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T23:09:53.022 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T23:09:53.022 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T23:09:53.022 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T23:09:53.022 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T23:09:53.022 INFO:teuthology.orchestra.run.smithi097.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T23:09:53.022 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T23:09:53.022 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T23:09:53.022 INFO:teuthology.orchestra.run.smithi097.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T23:09:53.022 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T23:09:53.022 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T23:09:53.022 INFO:teuthology.orchestra.run.smithi097.stdout:Installing weak dependencies: 2024-09-11T23:09:53.023 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T23:09:53.023 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:09:53.023 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-09-11T23:09:53.023 INFO:teuthology.orchestra.run.smithi097.stdout:====================================================================================== 2024-09-11T23:09:53.023 INFO:teuthology.orchestra.run.smithi097.stdout:Install 148 Packages 2024-09-11T23:09:53.023 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 8 Packages 2024-09-11T23:09:53.023 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:09:53.023 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 226 M 2024-09-11T23:09:53.024 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-09-11T23:09:53.067 INFO:teuthology.orchestra.run.smithi043.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 15 MB/s | 1.5 MB 00:00 2024-09-11T23:09:53.134 INFO:teuthology.orchestra.run.smithi043.stdout:(6/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 9.8 MB/s | 2.4 MB 00:00 2024-09-11T23:09:53.301 INFO:teuthology.orchestra.run.smithi043.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 20 MB/s | 4.7 MB 00:00 2024-09-11T23:09:53.751 INFO:teuthology.orchestra.run.smithi043.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 16 MB/s | 21 MB 00:01 2024-09-11T23:09:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 506 kB/s | 25 kB 00:00 2024-09-11T23:09:53.855 INFO:teuthology.orchestra.run.smithi097.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 49 kB/s | 6.5 kB 00:00 2024-09-11T23:09:53.918 INFO:teuthology.orchestra.run.smithi043.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 17 MB/s | 11 MB 00:00 2024-09-11T23:09:53.985 INFO:teuthology.orchestra.run.smithi043.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.4 MB/s | 228 kB 00:00 2024-09-11T23:09:54.044 INFO:teuthology.orchestra.run.smithi043.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 463 kB/s | 27 kB 00:00 2024-09-11T23:09:54.111 INFO:teuthology.orchestra.run.smithi043.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 477 kB/s | 32 kB 00:00 2024-09-11T23:09:54.255 INFO:teuthology.orchestra.run.smithi097.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.2 MB/s | 897 kB 00:00 2024-09-11T23:09:54.303 INFO:teuthology.orchestra.run.smithi043.stdout:(14/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 14 MB/s | 17 MB 00:01 2024-09-11T23:09:54.331 INFO:teuthology.orchestra.run.smithi097.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.9 MB/s | 145 kB 00:00 2024-09-11T23:09:54.336 INFO:teuthology.orchestra.run.smithi043.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 3.3 MB/s | 752 kB 00:00 2024-09-11T23:09:54.361 INFO:teuthology.orchestra.run.smithi043.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.7 MB/s | 163 kB 00:00 2024-09-11T23:09:54.372 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-11T23:09:54.387 INFO:teuthology.orchestra.run.smithi043.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.5 MB/s | 127 kB 00:00 2024-09-11T23:09:54.429 INFO:teuthology.orchestra.run.smithi043.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 7.4 MB/s | 506 kB 00:00 2024-09-11T23:09:54.496 INFO:teuthology.orchestra.run.smithi043.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 678 kB/s | 45 kB 00:00 2024-09-11T23:09:54.579 INFO:teuthology.orchestra.run.smithi043.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.6 MB/s | 136 kB 00:00 2024-09-11T23:09:54.696 INFO:teuthology.orchestra.run.smithi043.stdout:(21/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 17 MB/s | 5.3 MB 00:00 2024-09-11T23:09:54.722 INFO:teuthology.orchestra.run.smithi043.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.1 MB/s | 162 kB 00:00 2024-09-11T23:09:54.772 INFO:teuthology.orchestra.run.smithi043.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 4.2 MB/s | 321 kB 00:00 2024-09-11T23:09:54.797 INFO:teuthology.orchestra.run.smithi043.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 3.9 MB/s | 301 kB 00:00 2024-09-11T23:09:54.839 INFO:teuthology.orchestra.run.smithi043.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.5 MB/s | 100 kB 00:00 2024-09-11T23:09:54.873 INFO:teuthology.orchestra.run.smithi043.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.1 MB/s | 85 kB 00:00 2024-09-11T23:09:54.906 INFO:teuthology.orchestra.run.smithi097.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 4.6 MB/s | 5.4 MB 00:01 2024-09-11T23:09:54.940 INFO:teuthology.orchestra.run.smithi043.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.5 MB/s | 171 kB 00:00 2024-09-11T23:09:54.956 INFO:teuthology.orchestra.run.smithi097.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 3.8 MB/s | 2.4 MB 00:00 2024-09-11T23:09:55.040 INFO:teuthology.orchestra.run.smithi043.stdout:(28/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 15 MB/s | 3.1 MB 00:00 2024-09-11T23:09:55.065 INFO:teuthology.orchestra.run.smithi043.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 254 kB/s | 32 kB 00:00 2024-09-11T23:09:55.116 INFO:teuthology.orchestra.run.smithi043.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.9 MB/s | 146 kB 00:00 2024-09-11T23:09:55.190 INFO:teuthology.orchestra.run.smithi097.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 5.4 MB/s | 1.5 MB 00:00 2024-09-11T23:09:55.508 INFO:teuthology.orchestra.run.smithi043.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 8.1 MB/s | 3.6 MB 00:00 2024-09-11T23:09:55.583 INFO:teuthology.orchestra.run.smithi043.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 3.3 MB/s | 249 kB 00:00 2024-09-11T23:09:55.683 INFO:teuthology.orchestra.run.smithi043.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 13 MB/s | 7.4 MB 00:00 2024-09-11T23:09:55.709 INFO:teuthology.orchestra.run.smithi043.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 394 kB/s | 49 kB 00:00 2024-09-11T23:09:55.715 INFO:teuthology.orchestra.run.smithi097.stdout:(7/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 21 MB 00:01 2024-09-11T23:09:55.742 INFO:teuthology.orchestra.run.smithi043.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 281 kB/s | 16 kB 00:00 2024-09-11T23:09:55.767 INFO:teuthology.orchestra.run.smithi043.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.8 MB/s | 285 kB 00:00 2024-09-11T23:09:55.818 INFO:teuthology.orchestra.run.smithi043.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 9.9 MB/s | 765 kB 00:00 2024-09-11T23:09:55.852 INFO:teuthology.orchestra.run.smithi043.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 218 kB/s | 18 kB 00:00 2024-09-11T23:09:55.995 INFO:teuthology.orchestra.run.smithi043.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 455 kB/s | 80 kB 00:00 2024-09-11T23:09:56.033 INFO:teuthology.orchestra.run.smithi097.stdout:(8/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 4.4 MB/s | 4.7 MB 00:01 2024-09-11T23:09:56.091 INFO:teuthology.orchestra.run.smithi097.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 435 kB/s | 25 kB 00:00 2024-09-11T23:09:56.102 INFO:teuthology.orchestra.run.smithi043.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.7 MB/s | 191 kB 00:00 2024-09-11T23:09:56.136 INFO:teuthology.orchestra.run.smithi043.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 801 kB 00:00 2024-09-11T23:09:56.194 INFO:teuthology.orchestra.run.smithi043.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-09-11T23:09:56.245 INFO:teuthology.orchestra.run.smithi043.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-11T23:09:56.287 INFO:teuthology.orchestra.run.smithi043.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 8.1 MB/s | 1.2 MB 00:00 2024-09-11T23:09:56.312 INFO:teuthology.orchestra.run.smithi043.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-11T23:09:56.383 INFO:teuthology.orchestra.run.smithi097.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 16 MB/s | 11 MB 00:00 2024-09-11T23:09:56.442 INFO:teuthology.orchestra.run.smithi097.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.8 MB/s | 228 kB 00:00 2024-09-11T23:09:56.492 INFO:teuthology.orchestra.run.smithi097.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 540 kB/s | 27 kB 00:00 2024-09-11T23:09:56.542 INFO:teuthology.orchestra.run.smithi097.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 634 kB/s | 32 kB 00:00 2024-09-11T23:09:56.609 INFO:teuthology.orchestra.run.smithi097.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 752 kB 00:00 2024-09-11T23:09:56.658 INFO:teuthology.orchestra.run.smithi090.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:09:56.658 INFO:teuthology.orchestra.run.smithi090.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T23:09:56.659 INFO:teuthology.orchestra.run.smithi097.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 3.2 MB/s | 163 kB 00:00 2024-09-11T23:09:56.670 INFO:teuthology.orchestra.run.smithi090.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T23:09:56.670 INFO:teuthology.orchestra.run.smithi090.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T23:09:56.671 INFO:teuthology.orchestra.run.smithi090.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T23:09:56.672 INFO:teuthology.orchestra.run.smithi090.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T23:09:56.674 INFO:teuthology.orchestra.run.smithi090.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T23:09:56.676 INFO:teuthology.orchestra.run.smithi090.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T23:09:56.677 INFO:teuthology.orchestra.run.smithi090.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T23:09:56.709 INFO:teuthology.orchestra.run.smithi097.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.5 MB/s | 127 kB 00:00 2024-09-11T23:09:56.738 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-09-11T23:09:56.761 INFO:teuthology.orchestra.run.smithi090.stdout:====================================================================================== 2024-09-11T23:09:56.761 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-09-11T23:09:56.761 INFO:teuthology.orchestra.run.smithi090.stdout:====================================================================================== 2024-09-11T23:09:56.761 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-09-11T23:09:56.761 INFO:teuthology.orchestra.run.smithi090.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T23:09:56.761 INFO:teuthology.orchestra.run.smithi090.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T23:09:56.762 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T23:09:56.762 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T23:09:56.762 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T23:09:56.762 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T23:09:56.762 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T23:09:56.762 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T23:09:56.762 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T23:09:56.762 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T23:09:56.762 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T23:09:56.762 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T23:09:56.762 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T23:09:56.762 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T23:09:56.763 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T23:09:56.763 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T23:09:56.763 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T23:09:56.763 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T23:09:56.763 INFO:teuthology.orchestra.run.smithi090.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T23:09:56.763 INFO:teuthology.orchestra.run.smithi090.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T23:09:56.763 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T23:09:56.763 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T23:09:56.763 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T23:09:56.763 INFO:teuthology.orchestra.run.smithi090.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T23:09:56.763 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T23:09:56.763 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T23:09:56.764 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T23:09:56.764 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T23:09:56.764 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T23:09:56.764 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T23:09:56.764 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T23:09:56.764 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T23:09:56.764 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T23:09:56.764 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T23:09:56.764 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T23:09:56.764 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T23:09:56.764 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T23:09:56.764 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrading: 2024-09-11T23:09:56.764 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T23:09:56.765 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T23:09:56.765 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T23:09:56.765 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T23:09:56.765 INFO:teuthology.orchestra.run.smithi090.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T23:09:56.765 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T23:09:56.765 INFO:teuthology.orchestra.run.smithi090.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T23:09:56.765 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T23:09:56.765 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-09-11T23:09:56.765 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T23:09:56.765 INFO:teuthology.orchestra.run.smithi090.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T23:09:56.765 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T23:09:56.765 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T23:09:56.765 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T23:09:56.766 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T23:09:56.766 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T23:09:56.766 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T23:09:56.766 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T23:09:56.766 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T23:09:56.766 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T23:09:56.766 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T23:09:56.766 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T23:09:56.766 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T23:09:56.766 INFO:teuthology.orchestra.run.smithi090.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T23:09:56.766 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T23:09:56.766 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T23:09:56.766 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T23:09:56.767 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T23:09:56.767 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T23:09:56.767 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T23:09:56.767 INFO:teuthology.orchestra.run.smithi090.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T23:09:56.767 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T23:09:56.767 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T23:09:56.767 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T23:09:56.767 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T23:09:56.767 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T23:09:56.767 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T23:09:56.767 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T23:09:56.767 INFO:teuthology.orchestra.run.smithi090.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T23:09:56.768 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T23:09:56.768 INFO:teuthology.orchestra.run.smithi090.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T23:09:56.768 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T23:09:56.768 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T23:09:56.768 INFO:teuthology.orchestra.run.smithi090.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T23:09:56.768 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T23:09:56.768 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T23:09:56.768 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T23:09:56.768 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T23:09:56.768 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T23:09:56.768 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T23:09:56.768 INFO:teuthology.orchestra.run.smithi090.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T23:09:56.768 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T23:09:56.769 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T23:09:56.769 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T23:09:56.770 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T23:09:56.770 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T23:09:56.770 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T23:09:56.770 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T23:09:56.770 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T23:09:56.770 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T23:09:56.770 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T23:09:56.770 INFO:teuthology.orchestra.run.smithi090.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T23:09:56.770 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T23:09:56.770 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T23:09:56.770 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T23:09:56.770 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T23:09:56.771 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T23:09:56.771 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T23:09:56.771 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T23:09:56.771 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T23:09:56.771 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T23:09:56.771 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T23:09:56.771 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T23:09:56.771 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T23:09:56.771 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T23:09:56.771 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T23:09:56.771 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T23:09:56.771 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T23:09:56.771 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T23:09:56.772 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T23:09:56.772 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T23:09:56.772 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T23:09:56.772 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T23:09:56.772 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T23:09:56.772 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T23:09:56.772 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T23:09:56.772 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T23:09:56.772 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T23:09:56.772 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T23:09:56.772 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T23:09:56.772 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T23:09:56.772 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T23:09:56.773 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T23:09:56.773 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T23:09:56.773 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T23:09:56.773 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T23:09:56.773 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T23:09:56.773 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T23:09:56.773 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T23:09:56.773 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T23:09:56.773 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T23:09:56.773 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T23:09:56.773 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T23:09:56.773 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T23:09:56.773 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T23:09:56.774 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T23:09:56.774 INFO:teuthology.orchestra.run.smithi090.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T23:09:56.774 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T23:09:56.774 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T23:09:56.774 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T23:09:56.774 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T23:09:56.774 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T23:09:56.774 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T23:09:56.774 INFO:teuthology.orchestra.run.smithi090.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T23:09:56.774 INFO:teuthology.orchestra.run.smithi090.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T23:09:56.774 INFO:teuthology.orchestra.run.smithi090.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T23:09:56.774 INFO:teuthology.orchestra.run.smithi090.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T23:09:56.774 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T23:09:56.775 INFO:teuthology.orchestra.run.smithi090.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T23:09:56.775 INFO:teuthology.orchestra.run.smithi090.stdout:Installing weak dependencies: 2024-09-11T23:09:56.775 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T23:09:56.775 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-11T23:09:56.775 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-09-11T23:09:56.775 INFO:teuthology.orchestra.run.smithi090.stdout:====================================================================================== 2024-09-11T23:09:56.775 INFO:teuthology.orchestra.run.smithi090.stdout:Install 148 Packages 2024-09-11T23:09:56.775 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrade 8 Packages 2024-09-11T23:09:56.775 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-11T23:09:56.776 INFO:teuthology.orchestra.run.smithi097.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 8.5 MB/s | 506 kB 00:00 2024-09-11T23:09:56.776 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 226 M 2024-09-11T23:09:56.776 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-09-11T23:09:56.896 INFO:teuthology.orchestra.run.smithi043.stdout:(46/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 16 MB/s | 49 MB 00:03 2024-09-11T23:09:56.921 INFO:teuthology.orchestra.run.smithi043.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 197 kB/s | 125 kB 00:00 2024-09-11T23:09:56.946 INFO:teuthology.orchestra.run.smithi043.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 339 kB/s | 215 kB 00:00 2024-09-11T23:09:56.996 INFO:teuthology.orchestra.run.smithi043.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-11T23:09:57.018 INFO:teuthology.orchestra.run.smithi097.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 21 MB/s | 5.3 MB 00:00 2024-09-11T23:09:57.038 INFO:teuthology.orchestra.run.smithi043.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 7.9 MB/s | 943 kB 00:00 2024-09-11T23:09:57.060 INFO:teuthology.orchestra.run.smithi097.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 1.1 MB/s | 45 kB 00:00 2024-09-11T23:09:57.063 INFO:teuthology.orchestra.run.smithi043.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 325 kB/s | 22 kB 00:00 2024-09-11T23:09:57.097 INFO:teuthology.orchestra.run.smithi043.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-11T23:09:57.123 INFO:teuthology.orchestra.run.smithi043.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-09-11T23:09:57.181 INFO:teuthology.orchestra.run.smithi043.stdout:(54/156): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-09-11T23:09:57.206 INFO:teuthology.orchestra.run.smithi043.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 218 kB/s | 18 kB 00:00 2024-09-11T23:09:57.240 INFO:teuthology.orchestra.run.smithi043.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-09-11T23:09:57.242 INFO:teuthology.orchestra.run.smithi090.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 52 kB/s | 6.5 kB 00:00 2024-09-11T23:09:57.244 INFO:teuthology.orchestra.run.smithi097.stdout:(20/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 8.2 MB/s | 17 MB 00:02 2024-09-11T23:09:57.268 INFO:teuthology.orchestra.run.smithi043.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-11T23:09:57.269 INFO:teuthology.orchestra.run.smithi097.stdout:(21/156): python3-ceph-common-19.1.1-298.gb53b6 650 kB/s | 136 kB 00:00 2024-09-11T23:09:57.295 INFO:teuthology.orchestra.run.smithi097.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 3.2 MB/s | 162 kB 00:00 2024-09-11T23:09:57.299 INFO:teuthology.orchestra.run.smithi043.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 770 kB/s | 45 kB 00:00 2024-09-11T23:09:57.320 INFO:teuthology.orchestra.run.smithi097.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 6.2 MB/s | 321 kB 00:00 2024-09-11T23:09:57.354 INFO:teuthology.orchestra.run.smithi097.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.0 MB/s | 301 kB 00:00 2024-09-11T23:09:57.358 INFO:teuthology.orchestra.run.smithi043.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-09-11T23:09:57.379 INFO:teuthology.orchestra.run.smithi097.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.7 MB/s | 100 kB 00:00 2024-09-11T23:09:57.391 INFO:teuthology.orchestra.run.smithi043.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 6.2 MB/s | 581 kB 00:00 2024-09-11T23:09:57.404 INFO:teuthology.orchestra.run.smithi097.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-11T23:09:57.417 INFO:teuthology.orchestra.run.smithi043.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-11T23:09:57.450 INFO:teuthology.orchestra.run.smithi043.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-09-11T23:09:57.484 INFO:teuthology.orchestra.run.smithi090.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 3.6 MB/s | 897 kB 00:00 2024-09-11T23:09:57.509 INFO:teuthology.orchestra.run.smithi043.stdout:(63/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.3 MB/s | 3.0 MB 00:00 2024-09-11T23:09:57.513 INFO:teuthology.orchestra.run.smithi097.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 1.5 MB/s | 171 kB 00:00 2024-09-11T23:09:57.559 INFO:teuthology.orchestra.run.smithi043.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-11T23:09:57.584 INFO:teuthology.orchestra.run.smithi043.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 511 kB/s | 38 kB 00:00 2024-09-11T23:09:57.593 INFO:teuthology.orchestra.run.smithi090.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.3 MB/s | 145 kB 00:00 2024-09-11T23:09:57.609 INFO:teuthology.orchestra.run.smithi043.stdout:(66/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.5 MB/s | 292 kB 00:00 2024-09-11T23:09:57.613 INFO:teuthology.orchestra.run.smithi097.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 319 kB/s | 32 kB 00:00 2024-09-11T23:09:57.718 INFO:teuthology.orchestra.run.smithi043.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-11T23:09:57.730 INFO:teuthology.orchestra.run.smithi097.stdout:(29/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 8.8 MB/s | 3.1 MB 00:00 2024-09-11T23:09:57.756 INFO:teuthology.orchestra.run.smithi097.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.0 MB/s | 146 kB 00:00 2024-09-11T23:09:57.802 INFO:teuthology.orchestra.run.smithi043.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 11 MB/s | 910 kB 00:00 2024-09-11T23:09:57.826 INFO:teuthology.orchestra.run.smithi090.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 7.6 MB/s | 5.4 MB 00:00 2024-09-11T23:09:57.861 INFO:teuthology.orchestra.run.smithi043.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-11T23:09:57.961 INFO:teuthology.orchestra.run.smithi043.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 12 MB/s | 4.9 MB 00:00 2024-09-11T23:09:58.019 INFO:teuthology.orchestra.run.smithi090.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 7.9 MB/s | 1.5 MB 00:00 2024-09-11T23:09:58.036 INFO:teuthology.orchestra.run.smithi043.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-11T23:09:58.086 INFO:teuthology.orchestra.run.smithi090.stdout:(6/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 4.8 MB/s | 2.4 MB 00:00 2024-09-11T23:09:58.095 INFO:teuthology.orchestra.run.smithi043.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-11T23:09:58.106 INFO:teuthology.orchestra.run.smithi097.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 9.5 MB/s | 3.6 MB 00:00 2024-09-11T23:09:58.173 INFO:teuthology.orchestra.run.smithi097.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 3.7 MB/s | 249 kB 00:00 2024-09-11T23:09:58.203 INFO:teuthology.orchestra.run.smithi043.stdout:(73/156): postgresql-server-13.16-1.el9.x86_64. 17 MB/s | 5.8 MB 00:00 2024-09-11T23:09:58.223 INFO:teuthology.orchestra.run.smithi097.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 986 kB/s | 49 kB 00:00 2024-09-11T23:09:58.262 INFO:teuthology.orchestra.run.smithi043.stdout:(74/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 6.5 MB/s | 4.4 MB 00:00 2024-09-11T23:09:58.273 INFO:teuthology.orchestra.run.smithi097.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 328 kB/s | 16 kB 00:00 2024-09-11T23:09:58.289 INFO:teuthology.orchestra.run.smithi043.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 734 kB/s | 61 kB 00:00 2024-09-11T23:09:58.321 INFO:teuthology.orchestra.run.smithi043.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-11T23:09:58.323 INFO:teuthology.orchestra.run.smithi097.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 5.6 MB/s | 285 kB 00:00 2024-09-11T23:09:58.354 INFO:teuthology.orchestra.run.smithi043.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-09-11T23:09:58.378 INFO:teuthology.orchestra.run.smithi090.stdout:(7/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 17 MB/s | 21 MB 00:01 2024-09-11T23:09:58.379 INFO:teuthology.orchestra.run.smithi043.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 439 kB/s | 26 kB 00:00 2024-09-11T23:09:58.405 INFO:teuthology.orchestra.run.smithi043.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-11T23:09:58.424 INFO:teuthology.orchestra.run.smithi097.stdout:(36/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 7.5 MB/s | 765 kB 00:00 2024-09-11T23:09:58.439 INFO:teuthology.orchestra.run.smithi043.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-11T23:09:58.464 INFO:teuthology.orchestra.run.smithi043.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-09-11T23:09:58.466 INFO:teuthology.orchestra.run.smithi097.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 438 kB/s | 18 kB 00:00 2024-09-11T23:09:58.470 INFO:teuthology.orchestra.run.smithi090.stdout:(8/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 10 MB/s | 4.7 MB 00:00 2024-09-11T23:09:58.545 INFO:teuthology.orchestra.run.smithi090.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 337 kB/s | 25 kB 00:00 2024-09-11T23:09:58.547 INFO:teuthology.orchestra.run.smithi043.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-09-11T23:09:58.573 INFO:teuthology.orchestra.run.smithi043.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-11T23:09:58.600 INFO:teuthology.orchestra.run.smithi097.stdout:(38/156): fuse-2.9.9-16.el9.x86_64.rpm 598 kB/s | 80 kB 00:00 2024-09-11T23:09:58.606 INFO:teuthology.orchestra.run.smithi043.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-11T23:09:58.640 INFO:teuthology.orchestra.run.smithi043.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-09-11T23:09:58.665 INFO:teuthology.orchestra.run.smithi043.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 719 kB/s | 42 kB 00:00 2024-09-11T23:09:58.699 INFO:teuthology.orchestra.run.smithi043.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-11T23:09:58.724 INFO:teuthology.orchestra.run.smithi043.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-11T23:09:58.758 INFO:teuthology.orchestra.run.smithi043.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-09-11T23:09:58.783 INFO:teuthology.orchestra.run.smithi043.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 869 kB/s | 51 kB 00:00 2024-09-11T23:09:58.784 INFO:teuthology.orchestra.run.smithi097.stdout:(39/156): libgfortran-11.5.0-2.el9.x86_64.rpm 4.3 MB/s | 801 kB 00:00 2024-09-11T23:09:58.817 INFO:teuthology.orchestra.run.smithi043.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 916 kB/s | 54 kB 00:00 2024-09-11T23:09:58.834 INFO:teuthology.orchestra.run.smithi097.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-11T23:09:58.875 INFO:teuthology.orchestra.run.smithi043.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 715 kB/s | 42 kB 00:00 2024-09-11T23:09:58.895 INFO:teuthology.orchestra.run.smithi090.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 20 MB/s | 11 MB 00:00 2024-09-11T23:09:58.917 INFO:teuthology.orchestra.run.smithi097.stdout:(41/156): ceph-mgr-diskprediction-local-19.1.1- 6.4 MB/s | 7.4 MB 00:01 2024-09-11T23:09:58.926 INFO:teuthology.orchestra.run.smithi043.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 884 kB/s | 44 kB 00:00 2024-09-11T23:09:58.962 INFO:teuthology.orchestra.run.smithi090.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.4 MB/s | 228 kB 00:00 2024-09-11T23:09:58.968 INFO:teuthology.orchestra.run.smithi097.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.8 MB/s | 253 kB 00:00 2024-09-11T23:09:58.992 INFO:teuthology.orchestra.run.smithi043.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-11T23:09:59.021 INFO:teuthology.orchestra.run.smithi090.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 463 kB/s | 27 kB 00:00 2024-09-11T23:09:59.026 INFO:teuthology.orchestra.run.smithi097.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-11T23:09:59.051 INFO:teuthology.orchestra.run.smithi043.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 615 kB/s | 36 kB 00:00 2024-09-11T23:09:59.071 INFO:teuthology.orchestra.run.smithi090.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 636 kB/s | 32 kB 00:00 2024-09-11T23:09:59.077 INFO:teuthology.orchestra.run.smithi097.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-11T23:09:59.126 INFO:teuthology.orchestra.run.smithi043.stdout:(96/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.4 MB/s | 5.1 MB 00:00 2024-09-11T23:09:59.127 INFO:teuthology.orchestra.run.smithi097.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-11T23:09:59.147 INFO:teuthology.orchestra.run.smithi090.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 9.8 MB/s | 752 kB 00:00 2024-09-11T23:09:59.152 INFO:teuthology.orchestra.run.smithi043.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 474 kB/s | 47 kB 00:00 2024-09-11T23:09:59.186 INFO:teuthology.orchestra.run.smithi097.stdout:(46/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-11T23:09:59.214 INFO:teuthology.orchestra.run.smithi090.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.4 MB/s | 163 kB 00:00 2024-09-11T23:09:59.244 INFO:teuthology.orchestra.run.smithi097.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 3.8 MB/s | 1.2 MB 00:00 2024-09-11T23:09:59.281 INFO:teuthology.orchestra.run.smithi090.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.9 MB/s | 127 kB 00:00 2024-09-11T23:09:59.327 INFO:teuthology.orchestra.run.smithi043.stdout:(98/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-09-11T23:09:59.352 INFO:teuthology.orchestra.run.smithi043.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 288 kB/s | 58 kB 00:00 2024-09-11T23:09:59.365 INFO:teuthology.orchestra.run.smithi090.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 5.9 MB/s | 506 kB 00:00 2024-09-11T23:09:59.386 INFO:teuthology.orchestra.run.smithi043.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 1.1 MB/s | 304 kB 00:00 2024-09-11T23:09:59.411 INFO:teuthology.orchestra.run.smithi043.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-11T23:09:59.437 INFO:teuthology.orchestra.run.smithi043.stdout:(102/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 879 kB/s | 96 kB 00:00 2024-09-11T23:09:59.456 INFO:teuthology.orchestra.run.smithi097.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-09-11T23:09:59.462 INFO:teuthology.orchestra.run.smithi043.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 594 kB/s | 45 kB 00:00 2024-09-11T23:09:59.514 INFO:teuthology.orchestra.run.smithi097.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-11T23:09:59.520 INFO:teuthology.orchestra.run.smithi043.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-11T23:09:59.540 INFO:teuthology.orchestra.run.smithi097.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-09-11T23:09:59.554 INFO:teuthology.orchestra.run.smithi043.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-11T23:09:59.565 INFO:teuthology.orchestra.run.smithi097.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 431 kB/s | 22 kB 00:00 2024-09-11T23:09:59.579 INFO:teuthology.orchestra.run.smithi043.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 788 kB/s | 19 kB 00:00 2024-09-11T23:09:59.599 INFO:teuthology.orchestra.run.smithi097.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-11T23:09:59.624 INFO:teuthology.orchestra.run.smithi097.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 556 kB/s | 33 kB 00:00 2024-09-11T23:09:59.640 INFO:teuthology.orchestra.run.smithi090.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 19 MB/s | 5.3 MB 00:00 2024-09-11T23:09:59.675 INFO:teuthology.orchestra.run.smithi097.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-09-11T23:09:59.688 INFO:teuthology.orchestra.run.smithi043.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-11T23:09:59.699 INFO:teuthology.orchestra.run.smithi090.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 773 kB/s | 45 kB 00:00 2024-09-11T23:09:59.730 INFO:teuthology.orchestra.run.smithi043.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-11T23:09:59.750 INFO:teuthology.orchestra.run.smithi097.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-09-11T23:09:59.755 INFO:teuthology.orchestra.run.smithi043.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-11T23:09:59.766 INFO:teuthology.orchestra.run.smithi090.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 2.0 MB/s | 136 kB 00:00 2024-09-11T23:09:59.775 INFO:teuthology.orchestra.run.smithi097.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-09-11T23:09:59.781 INFO:teuthology.orchestra.run.smithi043.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-11T23:09:59.806 INFO:teuthology.orchestra.run.smithi043.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.3 MB/s | 151 kB 00:00 2024-09-11T23:09:59.808 INFO:teuthology.orchestra.run.smithi097.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-11T23:09:59.833 INFO:teuthology.orchestra.run.smithi090.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.4 MB/s | 162 kB 00:00 2024-09-11T23:09:59.834 INFO:teuthology.orchestra.run.smithi097.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-09-11T23:09:59.847 INFO:teuthology.orchestra.run.smithi043.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-11T23:09:59.868 INFO:teuthology.orchestra.run.smithi097.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-11T23:09:59.881 INFO:teuthology.orchestra.run.smithi043.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-11T23:09:59.901 INFO:teuthology.orchestra.run.smithi097.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-11T23:09:59.906 INFO:teuthology.orchestra.run.smithi043.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-11T23:09:59.908 INFO:teuthology.orchestra.run.smithi090.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 4.2 MB/s | 321 kB 00:00 2024-09-11T23:09:59.927 INFO:teuthology.orchestra.run.smithi097.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 417 kB/s | 24 kB 00:00 2024-09-11T23:09:59.932 INFO:teuthology.orchestra.run.smithi043.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-11T23:09:59.952 INFO:teuthology.orchestra.run.smithi097.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-11T23:09:59.957 INFO:teuthology.orchestra.run.smithi043.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 562 kB/s | 14 kB 00:00 2024-09-11T23:09:59.982 INFO:teuthology.orchestra.run.smithi043.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.8 MB/s | 172 kB 00:00 2024-09-11T23:09:59.985 INFO:teuthology.orchestra.run.smithi097.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-11T23:10:00.011 INFO:teuthology.orchestra.run.smithi097.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-09-11T23:10:00.025 INFO:teuthology.orchestra.run.smithi043.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 8.4 MB/s | 358 kB 00:00 2024-09-11T23:10:00.036 INFO:teuthology.orchestra.run.smithi097.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-11T23:10:00.050 INFO:teuthology.orchestra.run.smithi043.stdout:(119/156): lua-devel-5.4.4-4.el9.x86_64.rpm 35 kB/s | 22 kB 00:00 2024-09-11T23:10:00.092 INFO:teuthology.orchestra.run.smithi043.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.5 MB/s | 303 kB 00:00 2024-09-11T23:10:00.134 INFO:teuthology.orchestra.run.smithi043.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 2.6 MB/s | 220 kB 00:00 2024-09-11T23:10:00.143 INFO:teuthology.orchestra.run.smithi090.stdout:(23/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 8.2 MB/s | 17 MB 00:02 2024-09-11T23:10:00.162 INFO:teuthology.orchestra.run.smithi097.stdout:(66/156): openblas-openmp-0.3.26-2.el9.x86_64.r 33 MB/s | 4.9 MB 00:00 2024-09-11T23:10:00.167 INFO:teuthology.orchestra.run.smithi043.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-11T23:10:00.168 INFO:teuthology.orchestra.run.smithi090.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 1.1 MB/s | 301 kB 00:00 2024-09-11T23:10:00.193 INFO:teuthology.orchestra.run.smithi043.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-11T23:10:00.193 INFO:teuthology.orchestra.run.smithi090.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 100 kB 00:00 2024-09-11T23:10:00.227 INFO:teuthology.orchestra.run.smithi090.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-11T23:10:00.237 INFO:teuthology.orchestra.run.smithi097.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-09-11T23:10:00.243 INFO:teuthology.orchestra.run.smithi043.stdout:(124/156): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-09-11T23:10:00.268 INFO:teuthology.orchestra.run.smithi043.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-09-11T23:10:00.278 INFO:teuthology.orchestra.run.smithi090.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 3.3 MB/s | 171 kB 00:00 2024-09-11T23:10:00.279 INFO:teuthology.orchestra.run.smithi097.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-11T23:10:00.294 INFO:teuthology.orchestra.run.smithi043.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-11T23:10:00.319 INFO:teuthology.orchestra.run.smithi043.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-11T23:10:00.321 INFO:teuthology.orchestra.run.smithi097.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 11 MB/s | 910 kB 00:00 2024-09-11T23:10:00.337 INFO:teuthology.orchestra.run.smithi090.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 545 kB/s | 32 kB 00:00 2024-09-11T23:10:00.344 INFO:teuthology.orchestra.run.smithi043.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-11T23:10:00.346 INFO:teuthology.orchestra.run.smithi097.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-11T23:10:00.370 INFO:teuthology.orchestra.run.smithi043.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T23:10:00.395 INFO:teuthology.orchestra.run.smithi043.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-11T23:10:00.404 INFO:teuthology.orchestra.run.smithi090.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.1 MB/s | 146 kB 00:00 2024-09-11T23:10:00.422 INFO:teuthology.orchestra.run.smithi097.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-11T23:10:00.437 INFO:teuthology.orchestra.run.smithi043.stdout:(131/156): protobuf-compiler-3.14.0-13.el9.x86_ 863 kB/s | 863 kB 00:00 2024-09-11T23:10:00.462 INFO:teuthology.orchestra.run.smithi043.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 862 kB/s | 58 kB 00:00 2024-09-11T23:10:00.496 INFO:teuthology.orchestra.run.smithi043.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-11T23:10:00.505 INFO:teuthology.orchestra.run.smithi097.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 31 MB/s | 5.8 MB 00:00 2024-09-11T23:10:00.530 INFO:teuthology.orchestra.run.smithi097.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 805 kB/s | 87 kB 00:00 2024-09-11T23:10:00.554 INFO:teuthology.orchestra.run.smithi090.stdout:(30/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 8.6 MB/s | 3.1 MB 00:00 2024-09-11T23:10:00.555 INFO:teuthology.orchestra.run.smithi043.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 4.3 MB/s | 1.0 MB 00:00 2024-09-11T23:10:00.580 INFO:teuthology.orchestra.run.smithi043.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-09-11T23:10:00.589 INFO:teuthology.orchestra.run.smithi097.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-11T23:10:00.605 INFO:teuthology.orchestra.run.smithi043.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-11T23:10:00.630 INFO:teuthology.orchestra.run.smithi043.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 615 kB/s | 31 kB 00:00 2024-09-11T23:10:00.637 INFO:teuthology.orchestra.run.smithi090.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 15 MB/s | 3.6 MB 00:00 2024-09-11T23:10:00.656 INFO:teuthology.orchestra.run.smithi043.stdout:(138/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.2 MB/s | 272 kB 00:00 2024-09-11T23:10:00.673 INFO:teuthology.orchestra.run.smithi097.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 6.0 MB 00:00 2024-09-11T23:10:00.681 INFO:teuthology.orchestra.run.smithi043.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-11T23:10:00.697 INFO:teuthology.orchestra.run.smithi090.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 4.2 MB/s | 249 kB 00:00 2024-09-11T23:10:00.698 INFO:teuthology.orchestra.run.smithi097.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-11T23:10:00.706 INFO:teuthology.orchestra.run.smithi043.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-11T23:10:00.731 INFO:teuthology.orchestra.run.smithi043.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-09-11T23:10:00.732 INFO:teuthology.orchestra.run.smithi097.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-09-11T23:10:00.757 INFO:teuthology.orchestra.run.smithi043.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 988 kB/s | 74 kB 00:00 2024-09-11T23:10:00.757 INFO:teuthology.orchestra.run.smithi097.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 437 kB/s | 26 kB 00:00 2024-09-11T23:10:00.782 INFO:teuthology.orchestra.run.smithi043.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-11T23:10:00.782 INFO:teuthology.orchestra.run.smithi097.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-11T23:10:00.808 INFO:teuthology.orchestra.run.smithi097.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-11T23:10:00.815 INFO:teuthology.orchestra.run.smithi043.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-11T23:10:00.833 INFO:teuthology.orchestra.run.smithi097.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 688 kB/s | 35 kB 00:00 2024-09-11T23:10:00.841 INFO:teuthology.orchestra.run.smithi043.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-09-11T23:10:00.866 INFO:teuthology.orchestra.run.smithi043.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-11T23:10:00.891 INFO:teuthology.orchestra.run.smithi043.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-11T23:10:00.908 INFO:teuthology.orchestra.run.smithi097.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-09-11T23:10:00.950 INFO:teuthology.orchestra.run.smithi043.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-11T23:10:00.984 INFO:teuthology.orchestra.run.smithi097.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-09-11T23:10:01.009 INFO:teuthology.orchestra.run.smithi097.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-11T23:10:01.051 INFO:teuthology.orchestra.run.smithi097.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-09-11T23:10:01.083 INFO:teuthology.orchestra.run.smithi097.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 630 kB/s | 42 kB 00:00 2024-09-11T23:10:01.110 INFO:teuthology.orchestra.run.smithi097.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-11T23:10:01.135 INFO:teuthology.orchestra.run.smithi097.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-11T23:10:01.169 INFO:teuthology.orchestra.run.smithi097.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-09-11T23:10:01.175 INFO:teuthology.orchestra.run.smithi043.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 10 MB/s | 2.3 MB 00:00 2024-09-11T23:10:01.194 INFO:teuthology.orchestra.run.smithi097.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 867 kB/s | 51 kB 00:00 2024-09-11T23:10:01.220 INFO:teuthology.orchestra.run.smithi097.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-11T23:10:01.243 INFO:teuthology.orchestra.run.smithi043.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 9.0 MB/s | 3.2 MB 00:00 2024-09-11T23:10:01.247 INFO:teuthology.orchestra.run.smithi090.stdout:(33/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 18 MB/s | 49 MB 00:02 2024-09-11T23:10:01.272 INFO:teuthology.orchestra.run.smithi090.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 86 kB/s | 49 kB 00:00 2024-09-11T23:10:01.278 INFO:teuthology.orchestra.run.smithi097.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 715 kB/s | 42 kB 00:00 2024-09-11T23:10:01.297 INFO:teuthology.orchestra.run.smithi090.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 329 kB/s | 16 kB 00:00 2024-09-11T23:10:01.301 INFO:teuthology.orchestra.run.smithi043.stdout:(151/156): librados2-19.1.1-298.gb53b6d3e.el9.x 7.9 MB/s | 3.4 MB 00:00 2024-09-11T23:10:01.326 INFO:teuthology.orchestra.run.smithi043.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 489 kB/s | 41 kB 00:00 2024-09-11T23:10:01.331 INFO:teuthology.orchestra.run.smithi090.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.8 MB/s | 285 kB 00:00 2024-09-11T23:10:01.337 INFO:teuthology.orchestra.run.smithi097.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 758 kB/s | 44 kB 00:00 2024-09-11T23:10:01.368 INFO:teuthology.orchestra.run.smithi043.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 9.4 MB/s | 1.8 MB 00:00 2024-09-11T23:10:01.373 INFO:teuthology.orchestra.run.smithi090.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 440 kB/s | 18 kB 00:00 2024-09-11T23:10:01.395 INFO:teuthology.orchestra.run.smithi097.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-11T23:10:01.407 INFO:teuthology.orchestra.run.smithi090.stdout:(38/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 6.9 MB/s | 765 kB 00:00 2024-09-11T23:10:01.463 INFO:teuthology.orchestra.run.smithi097.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 537 kB/s | 36 kB 00:00 2024-09-11T23:10:01.477 INFO:teuthology.orchestra.run.smithi043.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-09-11T23:10:01.549 INFO:teuthology.orchestra.run.smithi090.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 7.4 MB/s | 7.4 MB 00:00 2024-09-11T23:10:01.560 INFO:teuthology.orchestra.run.smithi043.stdout:(155/156): NetworkManager-tui-1.49.90-1.el9.x86 960 kB/s | 249 kB 00:00 2024-09-11T23:10:01.621 INFO:teuthology.orchestra.run.smithi097.stdout:(96/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-09-11T23:10:01.647 INFO:teuthology.orchestra.run.smithi097.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 258 kB/s | 47 kB 00:00 2024-09-11T23:10:01.660 INFO:teuthology.orchestra.run.smithi043.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 4.1 MB/s | 1.4 MB 00:00 2024-09-11T23:10:01.666 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T23:10:01.666 INFO:teuthology.orchestra.run.smithi043.stdout:Total 23 MB/s | 226 MB 00:09 2024-09-11T23:10:01.680 INFO:teuthology.orchestra.run.smithi097.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-11T23:10:01.706 INFO:teuthology.orchestra.run.smithi097.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 985 kB/s | 58 kB 00:00 2024-09-11T23:10:01.731 INFO:teuthology.orchestra.run.smithi097.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-11T23:10:01.756 INFO:teuthology.orchestra.run.smithi097.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-11T23:10:01.782 INFO:teuthology.orchestra.run.smithi097.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 888 kB/s | 45 kB 00:00 2024-09-11T23:10:01.857 INFO:teuthology.orchestra.run.smithi097.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-09-11T23:10:01.949 INFO:teuthology.orchestra.run.smithi097.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.8 MB/s | 548 kB 00:00 2024-09-11T23:10:01.991 INFO:teuthology.orchestra.run.smithi097.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 7.3 MB/s | 309 kB 00:00 2024-09-11T23:10:02.016 INFO:teuthology.orchestra.run.smithi097.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 774 kB/s | 19 kB 00:00 2024-09-11T23:10:02.041 INFO:teuthology.orchestra.run.smithi097.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.3 MB/s | 863 kB 00:00 2024-09-11T23:10:02.092 INFO:teuthology.orchestra.run.smithi097.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 519 kB/s | 26 kB 00:00 2024-09-11T23:10:02.125 INFO:teuthology.orchestra.run.smithi097.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-11T23:10:02.151 INFO:teuthology.orchestra.run.smithi097.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-11T23:10:02.184 INFO:teuthology.orchestra.run.smithi097.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-11T23:10:02.199 INFO:teuthology.orchestra.run.smithi090.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 97 kB/s | 80 kB 00:00 2024-09-11T23:10:02.350 INFO:teuthology.orchestra.run.smithi090.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 239 kB/s | 191 kB 00:00 2024-09-11T23:10:02.484 INFO:teuthology.orchestra.run.smithi090.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 744 kB/s | 801 kB 00:01 2024-09-11T23:10:02.602 INFO:teuthology.orchestra.run.smithi097.stdout:(112/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x 7.5 MB/s | 49 MB 00:06 2024-09-11T23:10:02.652 INFO:teuthology.orchestra.run.smithi097.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.8 MB/s | 839 kB 00:00 2024-09-11T23:10:02.685 INFO:teuthology.orchestra.run.smithi097.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.4 MB/s | 43 kB 00:00 2024-09-11T23:10:02.711 INFO:teuthology.orchestra.run.smithi097.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-11T23:10:02.736 INFO:teuthology.orchestra.run.smithi097.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 568 kB/s | 14 kB 00:00 2024-09-11T23:10:02.761 INFO:teuthology.orchestra.run.smithi097.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-09-11T23:10:02.786 INFO:teuthology.orchestra.run.smithi097.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-11T23:10:02.820 INFO:teuthology.orchestra.run.smithi097.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-11T23:10:02.887 INFO:teuthology.orchestra.run.smithi097.stdout:(120/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.1 MB/s | 4.4 MB 00:00 2024-09-11T23:10:02.902 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-11T23:10:02.913 INFO:teuthology.orchestra.run.smithi097.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-09-11T23:10:02.938 INFO:teuthology.orchestra.run.smithi097.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 1.8 MB/s | 220 kB 00:00 2024-09-11T23:10:02.963 INFO:teuthology.orchestra.run.smithi097.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-11T23:10:02.989 INFO:teuthology.orchestra.run.smithi097.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-09-11T23:10:03.014 INFO:teuthology.orchestra.run.smithi097.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 349 kB/s | 18 kB 00:00 2024-09-11T23:10:03.039 INFO:teuthology.orchestra.run.smithi097.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-09-11T23:10:03.059 INFO:teuthology.orchestra.run.smithi090.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 295 kB/s | 253 kB 00:00 2024-09-11T23:10:03.069 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-11T23:10:03.069 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-11T23:10:03.073 INFO:teuthology.orchestra.run.smithi097.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 327 kB/s | 19 kB 00:00 2024-09-11T23:10:03.098 INFO:teuthology.orchestra.run.smithi097.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 336 kB/s | 20 kB 00:00 2024-09-11T23:10:03.132 INFO:teuthology.orchestra.run.smithi097.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-11T23:10:03.165 INFO:teuthology.orchestra.run.smithi097.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-11T23:10:03.199 INFO:teuthology.orchestra.run.smithi097.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-09-11T23:10:03.224 INFO:teuthology.orchestra.run.smithi097.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 983 kB/s | 58 kB 00:00 2024-09-11T23:10:03.227 INFO:teuthology.orchestra.run.smithi090.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 143 kB/s | 106 kB 00:00 2024-09-11T23:10:03.249 INFO:teuthology.orchestra.run.smithi097.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-09-11T23:10:03.275 INFO:teuthology.orchestra.run.smithi097.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-11T23:10:03.300 INFO:teuthology.orchestra.run.smithi097.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-11T23:10:03.326 INFO:teuthology.orchestra.run.smithi097.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-11T23:10:03.328 INFO:teuthology.orchestra.run.smithi090.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-11T23:10:03.351 INFO:teuthology.orchestra.run.smithi097.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 615 kB/s | 31 kB 00:00 2024-09-11T23:10:03.393 INFO:teuthology.orchestra.run.smithi097.stdout:(138/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.0 MB/s | 2.0 MB 00:00 2024-09-11T23:10:03.418 INFO:teuthology.orchestra.run.smithi097.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-09-11T23:10:03.435 INFO:teuthology.orchestra.run.smithi097.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 704 kB/s | 59 kB 00:00 2024-09-11T23:10:03.452 INFO:teuthology.orchestra.run.smithi097.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 609 kB/s | 36 kB 00:00 2024-09-11T23:10:03.477 INFO:teuthology.orchestra.run.smithi097.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-09-11T23:10:03.502 INFO:teuthology.orchestra.run.smithi097.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-11T23:10:03.528 INFO:teuthology.orchestra.run.smithi097.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-11T23:10:03.553 INFO:teuthology.orchestra.run.smithi097.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-11T23:10:03.578 INFO:teuthology.orchestra.run.smithi097.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-11T23:10:03.604 INFO:teuthology.orchestra.run.smithi097.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-11T23:10:03.662 INFO:teuthology.orchestra.run.smithi097.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-11T23:10:03.837 INFO:teuthology.orchestra.run.smithi090.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 174 kB/s | 135 kB 00:00 2024-09-11T23:10:03.929 INFO:teuthology.orchestra.run.smithi097.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 9.8 MB/s | 3.4 MB 00:00 2024-09-11T23:10:03.962 INFO:teuthology.orchestra.run.smithi090.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 170 kB/s | 125 kB 00:00 2024-09-11T23:10:04.004 INFO:teuthology.orchestra.run.smithi097.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 7.9 MB/s | 3.2 MB 00:00 2024-09-11T23:10:04.055 INFO:teuthology.orchestra.run.smithi097.stdout:(151/156): NetworkManager-1.49.90-1.el9.x86_64. 5.9 MB/s | 2.3 MB 00:00 2024-09-11T23:10:04.096 INFO:teuthology.orchestra.run.smithi090.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 283 kB/s | 215 kB 00:00 2024-09-11T23:10:04.129 INFO:teuthology.orchestra.run.smithi090.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-09-11T23:10:04.138 INFO:teuthology.orchestra.run.smithi097.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 304 kB/s | 41 kB 00:00 2024-09-11T23:10:04.164 INFO:teuthology.orchestra.run.smithi097.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 2.2 MB/s | 249 kB 00:00 2024-09-11T23:10:04.188 INFO:teuthology.orchestra.run.smithi090.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 373 kB/s | 22 kB 00:00 2024-09-11T23:10:04.230 INFO:teuthology.orchestra.run.smithi090.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 943 kB 00:00 2024-09-11T23:10:04.264 INFO:teuthology.orchestra.run.smithi090.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-09-11T23:10:04.298 INFO:teuthology.orchestra.run.smithi097.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 4.9 MB/s | 1.8 MB 00:00 2024-09-11T23:10:04.298 INFO:teuthology.orchestra.run.smithi090.stdout:(53/156): boost-program-options-1.75.0-8.el9.x8 531 kB/s | 107 kB 00:00 2024-09-11T23:10:04.314 INFO:teuthology.orchestra.run.smithi090.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 389 kB/s | 33 kB 00:00 2024-09-11T23:10:04.365 INFO:teuthology.orchestra.run.smithi090.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 224 kB/s | 11 kB 00:00 2024-09-11T23:10:04.390 INFO:teuthology.orchestra.run.smithi090.stdout:(56/156): flexiblas-openblas-openmp-3.0.4-8.el9 242 kB/s | 18 kB 00:00 2024-09-11T23:10:04.424 INFO:teuthology.orchestra.run.smithi090.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-11T23:10:04.449 INFO:teuthology.orchestra.run.smithi090.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-09-11T23:10:04.464 INFO:teuthology.orchestra.run.smithi097.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 4.2 MB/s | 1.4 MB 00:00 2024-09-11T23:10:04.516 INFO:teuthology.orchestra.run.smithi090.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-09-11T23:10:04.550 INFO:teuthology.orchestra.run.smithi090.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.1 MB/s | 662 kB 00:00 2024-09-11T23:10:04.575 INFO:teuthology.orchestra.run.smithi090.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-11T23:10:04.617 INFO:teuthology.orchestra.run.smithi090.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 3.4 MB/s | 581 kB 00:00 2024-09-11T23:10:04.643 INFO:teuthology.orchestra.run.smithi090.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 513 kB/s | 47 kB 00:00 2024-09-11T23:10:04.676 INFO:teuthology.orchestra.run.smithi090.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-11T23:10:04.703 INFO:teuthology.orchestra.run.smithi090.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 642 kB/s | 38 kB 00:00 2024-09-11T23:10:04.736 INFO:teuthology.orchestra.run.smithi090.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-09-11T23:10:04.895 INFO:teuthology.orchestra.run.smithi090.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-09-11T23:10:04.940 INFO:teuthology.orchestra.run.smithi097.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.1 MB/s | 2.4 MB 00:00 2024-09-11T23:10:04.945 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-09-11T23:10:04.946 INFO:teuthology.orchestra.run.smithi097.stdout:Total 19 MB/s | 226 MB 00:11 2024-09-11T23:10:04.970 INFO:teuthology.orchestra.run.smithi090.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 17 MB/s | 4.9 MB 00:00 2024-09-11T23:10:05.013 INFO:teuthology.orchestra.run.smithi090.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 7.6 MB/s | 910 kB 00:00 2024-09-11T23:10:05.063 INFO:teuthology.orchestra.run.smithi090.stdout:(70/156): postgresql-13.16-1.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-09-11T23:10:05.096 INFO:teuthology.orchestra.run.smithi090.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-11T23:10:05.139 INFO:teuthology.orchestra.run.smithi090.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-11T23:10:05.164 INFO:teuthology.orchestra.run.smithi090.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-11T23:10:05.223 INFO:teuthology.orchestra.run.smithi090.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-11T23:10:05.315 INFO:teuthology.orchestra.run.smithi090.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-09-11T23:10:05.361 INFO:teuthology.orchestra.run.smithi090.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-11T23:10:05.390 INFO:teuthology.orchestra.run.smithi090.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-09-11T23:10:05.407 INFO:teuthology.orchestra.run.smithi090.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 554 kB/s | 26 kB 00:00 2024-09-11T23:10:05.445 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-11T23:10:05.445 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-11T23:10:05.500 INFO:teuthology.orchestra.run.smithi090.stdout:(79/156): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-11T23:10:05.525 INFO:teuthology.orchestra.run.smithi090.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 143 kB/s | 19 kB 00:00 2024-09-11T23:10:05.558 INFO:teuthology.orchestra.run.smithi090.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-11T23:10:05.584 INFO:teuthology.orchestra.run.smithi090.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 414 kB/s | 35 kB 00:00 2024-09-11T23:10:05.617 INFO:teuthology.orchestra.run.smithi090.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-11T23:10:05.651 INFO:teuthology.orchestra.run.smithi090.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-09-11T23:10:05.677 INFO:teuthology.orchestra.run.smithi090.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-11T23:10:05.702 INFO:teuthology.orchestra.run.smithi090.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 839 kB/s | 42 kB 00:00 2024-09-11T23:10:05.744 INFO:teuthology.orchestra.run.smithi090.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-09-11T23:10:05.771 INFO:teuthology.orchestra.run.smithi090.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-11T23:10:05.846 INFO:teuthology.orchestra.run.smithi090.stdout:(89/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-09-11T23:10:05.871 INFO:teuthology.orchestra.run.smithi090.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 509 kB/s | 51 kB 00:00 2024-09-11T23:10:05.905 INFO:teuthology.orchestra.run.smithi090.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.7 MB/s | 279 kB 00:00 2024-09-11T23:10:05.930 INFO:teuthology.orchestra.run.smithi090.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 640 kB/s | 54 kB 00:00 2024-09-11T23:10:05.964 INFO:teuthology.orchestra.run.smithi090.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 709 kB/s | 42 kB 00:00 2024-09-11T23:10:05.989 INFO:teuthology.orchestra.run.smithi090.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 752 kB/s | 44 kB 00:00 2024-09-11T23:10:06.023 INFO:teuthology.orchestra.run.smithi090.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-11T23:10:06.049 INFO:teuthology.orchestra.run.smithi090.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 612 kB/s | 36 kB 00:00 2024-09-11T23:10:06.074 INFO:teuthology.orchestra.run.smithi090.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 939 kB/s | 47 kB 00:00 2024-09-11T23:10:06.116 INFO:teuthology.orchestra.run.smithi090.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 4.4 MB/s | 304 kB 00:00 2024-09-11T23:10:06.141 INFO:teuthology.orchestra.run.smithi090.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 862 kB/s | 58 kB 00:00 2024-09-11T23:10:06.175 INFO:teuthology.orchestra.run.smithi090.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-11T23:10:06.187 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-09-11T23:10:06.200 INFO:teuthology.orchestra.run.smithi090.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-11T23:10:06.226 INFO:teuthology.orchestra.run.smithi090.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 888 kB/s | 45 kB 00:00 2024-09-11T23:10:06.292 INFO:teuthology.orchestra.run.smithi090.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-09-11T23:10:06.339 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-09-11T23:10:06.339 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-09-11T23:10:06.510 INFO:teuthology.orchestra.run.smithi090.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-09-11T23:10:06.560 INFO:teuthology.orchestra.run.smithi090.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.0 MB/s | 548 kB 00:00 2024-09-11T23:10:06.585 INFO:teuthology.orchestra.run.smithi090.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 849 kB/s | 19 kB 00:00 2024-09-11T23:10:06.627 INFO:teuthology.orchestra.run.smithi090.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 2.6 MB/s | 309 kB 00:00 2024-09-11T23:10:06.670 INFO:teuthology.orchestra.run.smithi090.stdout:(108/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-09-11T23:10:06.695 INFO:teuthology.orchestra.run.smithi090.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 388 kB/s | 26 kB 00:00 2024-09-11T23:10:06.728 INFO:teuthology.orchestra.run.smithi090.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.5 MB/s | 67 kB 00:00 2024-09-11T23:10:06.770 INFO:teuthology.orchestra.run.smithi090.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 3.7 MB/s | 151 kB 00:00 2024-09-11T23:10:06.797 INFO:teuthology.orchestra.run.smithi090.stdout:(112/156): liboath-2.6.7-2.el9.x86_64.rpm 382 kB/s | 49 kB 00:00 2024-09-11T23:10:06.847 INFO:teuthology.orchestra.run.smithi090.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-11T23:10:06.873 INFO:teuthology.orchestra.run.smithi090.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-11T23:10:06.898 INFO:teuthology.orchestra.run.smithi090.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-11T23:10:06.923 INFO:teuthology.orchestra.run.smithi090.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 568 kB/s | 14 kB 00:00 2024-09-11T23:10:06.957 INFO:teuthology.orchestra.run.smithi090.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-09-11T23:10:06.991 INFO:teuthology.orchestra.run.smithi090.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-11T23:10:07.024 INFO:teuthology.orchestra.run.smithi090.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-11T23:10:07.058 INFO:teuthology.orchestra.run.smithi090.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-09-11T23:10:07.084 INFO:teuthology.orchestra.run.smithi090.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-11T23:10:07.159 INFO:teuthology.orchestra.run.smithi090.stdout:(122/156): libarrow-9.0.0-11.el9.x86_64.rpm 7.7 MB/s | 4.4 MB 00:00 2024-09-11T23:10:07.184 INFO:teuthology.orchestra.run.smithi090.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-09-11T23:10:07.210 INFO:teuthology.orchestra.run.smithi090.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-09-11T23:10:07.235 INFO:teuthology.orchestra.run.smithi090.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-09-11T23:10:07.260 INFO:teuthology.orchestra.run.smithi090.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-11T23:10:07.285 INFO:teuthology.orchestra.run.smithi090.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-11T23:10:07.311 INFO:teuthology.orchestra.run.smithi090.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-11T23:10:07.336 INFO:teuthology.orchestra.run.smithi090.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T23:10:07.370 INFO:teuthology.orchestra.run.smithi090.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-11T23:10:07.404 INFO:teuthology.orchestra.run.smithi090.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-09-11T23:10:07.446 INFO:teuthology.orchestra.run.smithi090.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-09-11T23:10:07.471 INFO:teuthology.orchestra.run.smithi090.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-11T23:10:07.505 INFO:teuthology.orchestra.run.smithi090.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-11T23:10:07.531 INFO:teuthology.orchestra.run.smithi090.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-11T23:10:07.557 INFO:teuthology.orchestra.run.smithi090.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-09-11T23:10:07.582 INFO:teuthology.orchestra.run.smithi090.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-09-11T23:10:07.607 INFO:teuthology.orchestra.run.smithi090.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-11T23:10:07.641 INFO:teuthology.orchestra.run.smithi090.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-09-11T23:10:07.666 INFO:teuthology.orchestra.run.smithi090.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 611 kB/s | 36 kB 00:00 2024-09-11T23:10:07.700 INFO:teuthology.orchestra.run.smithi090.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-09-11T23:10:07.750 INFO:teuthology.orchestra.run.smithi090.stdout:(142/156): python3-grpcio-1.46.7-10.el9.x86_64. 2.9 MB/s | 2.0 MB 00:00 2024-09-11T23:10:07.776 INFO:teuthology.orchestra.run.smithi090.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-09-11T23:10:07.809 INFO:teuthology.orchestra.run.smithi090.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-09-11T23:10:07.835 INFO:teuthology.orchestra.run.smithi090.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-09-11T23:10:07.860 INFO:teuthology.orchestra.run.smithi090.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-11T23:10:07.894 INFO:teuthology.orchestra.run.smithi090.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-11T23:10:07.953 INFO:teuthology.orchestra.run.smithi090.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-11T23:10:08.135 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-09-11T23:10:08.136 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-09-11T23:10:08.186 INFO:teuthology.orchestra.run.smithi090.stdout:(149/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-11T23:10:08.262 INFO:teuthology.orchestra.run.smithi090.stdout:(150/156): librados2-19.1.1-298.gb53b6d3e.el9.x 8.5 MB/s | 3.4 MB 00:00 2024-09-11T23:10:08.887 INFO:teuthology.orchestra.run.smithi090.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 65 kB/s | 41 kB 00:00 2024-09-11T23:10:09.104 INFO:teuthology.orchestra.run.smithi090.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 2.0 MB/s | 2.3 MB 00:01 2024-09-11T23:10:09.204 INFO:teuthology.orchestra.run.smithi090.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 1.8 MB/s | 1.8 MB 00:01 2024-09-11T23:10:09.746 INFO:teuthology.orchestra.run.smithi090.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 290 kB/s | 249 kB 00:00 2024-09-11T23:10:10.080 INFO:teuthology.orchestra.run.smithi090.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 1.4 MB/s | 1.4 MB 00:00 2024-09-11T23:10:10.247 INFO:teuthology.orchestra.run.smithi090.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 2.4 MB 00:01 2024-09-11T23:10:10.254 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-09-11T23:10:10.254 INFO:teuthology.orchestra.run.smithi090.stdout:Total 17 MB/s | 226 MB 00:13 2024-09-11T23:10:11.486 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-09-11T23:10:11.636 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-09-11T23:10:11.636 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-09-11T23:10:12.008 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-11T23:10:12.148 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T23:10:12.296 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T23:10:12.494 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T23:10:12.541 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T23:10:12.566 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T23:10:12.629 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T23:10:12.975 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T23:10:13.083 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:10:13.208 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:10:13.234 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:10:13.327 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:10:13.356 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:10:13.470 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-09-11T23:10:13.471 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-09-11T23:10:13.534 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T23:10:13.594 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T23:10:13.622 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:10:13.727 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:10:13.756 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:10:13.819 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:10:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:10:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:10:14.077 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:10:14.262 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:10:14.366 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-09-11T23:10:14.389 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T23:10:14.485 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T23:10:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:10:14.590 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:10:14.616 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T23:10:14.804 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T23:10:14.821 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:10:14.845 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T23:10:14.870 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T23:10:14.922 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T23:10:14.931 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T23:10:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T23:10:15.071 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T23:10:15.106 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T23:10:15.181 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T23:10:15.232 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T23:10:15.261 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T23:10:15.273 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T23:10:15.316 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T23:10:15.357 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T23:10:15.384 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T23:10:15.401 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:10:15.465 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T23:10:15.519 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:10:15.524 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T23:10:15.548 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:10:15.555 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T23:10:15.598 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T23:10:15.638 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:10:15.669 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:10:15.677 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T23:10:15.708 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T23:10:15.752 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T23:10:15.780 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T23:10:15.816 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T23:10:15.853 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T23:10:15.873 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T23:10:15.902 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:10:15.920 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T23:10:16.006 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:10:16.006 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T23:10:16.036 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:10:16.099 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:10:16.121 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:10:16.146 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:10:16.152 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T23:10:16.207 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T23:10:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:10:16.477 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:10:16.602 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T23:10:16.715 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:10:16.761 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T23:10:16.787 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:10:16.846 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T23:10:16.880 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T23:10:16.924 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T23:10:16.981 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T23:10:17.030 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T23:10:17.158 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:10:17.205 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T23:10:17.265 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T23:10:17.321 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T23:10:17.396 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T23:10:17.412 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T23:10:17.433 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T23:10:17.505 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T23:10:17.543 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T23:10:17.545 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T23:10:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T23:10:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T23:10:17.695 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T23:10:17.697 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T23:10:17.727 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T23:10:17.730 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T23:10:17.771 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T23:10:17.774 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T23:10:17.807 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T23:10:17.817 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T23:10:17.835 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T23:10:17.902 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T23:10:17.958 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T23:10:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T23:10:17.997 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T23:10:18.009 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T23:10:18.040 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T23:10:18.050 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T23:10:18.116 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T23:10:18.151 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T23:10:18.157 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T23:10:18.192 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T23:10:18.194 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T23:10:18.223 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T23:10:18.232 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T23:10:18.265 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T23:10:18.290 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T23:10:18.292 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T23:10:18.337 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T23:10:18.393 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T23:10:18.411 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T23:10:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T23:10:18.542 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T23:10:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T23:10:18.598 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T23:10:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T23:10:18.656 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T23:10:18.700 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T23:10:18.739 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T23:10:18.769 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T23:10:18.804 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T23:10:18.837 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T23:10:18.879 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T23:10:18.921 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T23:10:19.012 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T23:10:19.061 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T23:10:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T23:10:19.114 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T23:10:19.193 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T23:10:19.207 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T23:10:19.224 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T23:10:19.261 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T23:10:19.265 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T23:10:19.312 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T23:10:19.354 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T23:10:19.512 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T23:10:19.574 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T23:10:19.628 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T23:10:19.704 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T23:10:19.736 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T23:10:19.780 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T23:10:19.827 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T23:10:19.863 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T23:10:19.908 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T23:10:19.928 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-09-11T23:10:19.950 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T23:10:20.021 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T23:10:20.053 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T23:10:20.072 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T23:10:20.101 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T23:10:20.106 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T23:10:20.162 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T23:10:20.205 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T23:10:20.220 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T23:10:20.239 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T23:10:20.245 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T23:10:20.279 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T23:10:20.281 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T23:10:20.294 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T23:10:20.324 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T23:10:20.344 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T23:10:20.415 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T23:10:20.454 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T23:10:20.459 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T23:10:20.486 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T23:10:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T23:10:20.551 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T23:10:20.577 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T23:10:20.629 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T23:10:20.650 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T23:10:20.651 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T23:10:20.690 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T23:10:20.767 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:10:20.770 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T23:10:20.777 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T23:10:20.817 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T23:10:20.845 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T23:10:20.888 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T23:10:20.890 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:10:20.926 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:10:20.935 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T23:10:20.967 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T23:10:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T23:10:21.018 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:10:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T23:10:21.056 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:10:21.065 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T23:10:21.106 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T23:10:21.141 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T23:10:21.216 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T23:10:21.227 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T23:10:21.271 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T23:10:21.278 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T23:10:21.307 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:10:21.311 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T23:10:21.411 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:10:21.412 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T23:10:21.441 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:10:21.462 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T23:10:21.471 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T23:10:21.488 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T23:10:21.518 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:10:21.544 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T23:10:21.600 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T23:10:21.737 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:10:21.911 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T23:10:22.101 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T23:10:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T23:10:22.195 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:10:22.242 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T23:10:22.276 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T23:10:22.287 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:10:22.374 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T23:10:22.439 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T23:10:22.484 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T23:10:22.492 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T23:10:22.572 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T23:10:22.600 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T23:10:22.604 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T23:10:22.696 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T23:10:22.735 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T23:10:22.778 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T23:10:22.787 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T23:10:22.818 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T23:10:22.851 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T23:10:22.896 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T23:10:22.904 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T23:10:22.928 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T23:10:22.946 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T23:10:22.990 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T23:10:23.003 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T23:10:23.022 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T23:10:23.031 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T23:10:23.054 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T23:10:23.064 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T23:10:23.131 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T23:10:23.185 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T23:10:23.225 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T23:10:23.260 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T23:10:23.336 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T23:10:23.372 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T23:10:23.415 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T23:10:23.444 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T23:10:23.488 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T23:10:23.523 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T23:10:23.527 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:10:23.532 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T23:10:23.567 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T23:10:23.642 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T23:10:23.644 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:10:23.649 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T23:10:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T23:10:23.711 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T23:10:23.783 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T23:10:23.836 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T23:10:24.216 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T23:10:24.255 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:10:24.327 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:10:24.357 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T23:10:24.359 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T23:10:24.459 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T23:10:24.497 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T23:10:24.540 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T23:10:24.578 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T23:10:24.620 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T23:10:24.839 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T23:10:24.857 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T23:10:24.892 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T23:10:24.948 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T23:10:24.955 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T23:10:24.976 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T23:10:25.009 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T23:10:25.028 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T23:10:25.068 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T23:10:25.113 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T23:10:25.160 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T23:10:25.204 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T23:10:25.257 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T23:10:25.308 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T23:10:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T23:10:25.364 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:10:25.450 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:10:25.451 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T23:10:25.478 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T23:10:25.493 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T23:10:25.513 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T23:10:25.536 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T23:10:25.637 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T23:10:25.677 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T23:10:25.717 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T23:10:25.751 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T23:10:25.777 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T23:10:25.862 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T23:10:26.013 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T23:10:26.050 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:10:26.122 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:10:26.153 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T23:10:26.345 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T23:10:26.386 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T23:10:26.423 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T23:10:26.458 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T23:10:26.498 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T23:10:26.538 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T23:10:26.568 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T23:10:26.611 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T23:10:26.645 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T23:10:26.686 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T23:10:26.721 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T23:10:26.795 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:10:26.808 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T23:10:26.828 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:10:26.859 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T23:10:26.901 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T23:10:27.028 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T23:10:27.085 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T23:10:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:10:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:10:27.523 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:10:27.523 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T23:10:27.523 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T23:10:27.523 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:10:27.566 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:10:27.801 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T23:10:27.858 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T23:10:27.902 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T23:10:28.009 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T23:10:28.065 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T23:10:28.119 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T23:10:28.205 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T23:10:28.247 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T23:10:28.422 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T23:10:28.454 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T23:10:28.468 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:10:28.500 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:10:28.539 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T23:10:29.041 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:10:29.087 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:10:29.181 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T23:10:29.182 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:10:29.182 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T23:10:29.183 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T23:10:29.183 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:10:29.214 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:10:29.295 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T23:10:30.511 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T23:10:30.599 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T23:10:30.627 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T23:10:30.661 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T23:10:30.971 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T23:10:31.057 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:10:31.150 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T23:10:31.188 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T23:10:31.242 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T23:10:31.719 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T23:10:31.754 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:10:31.825 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T23:10:31.856 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T23:10:34.135 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:10:34.179 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:10:34.721 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T23:10:34.776 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:10:34.871 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T23:10:34.871 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T23:10:34.871 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T23:10:34.871 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-11T23:10:34.912 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:10:39.905 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:10:39.906 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /sys 2024-09-11T23:10:39.906 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /proc 2024-09-11T23:10:39.906 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /mnt 2024-09-11T23:10:39.906 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /var/tmp 2024-09-11T23:10:39.906 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /home 2024-09-11T23:10:39.906 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /root 2024-09-11T23:10:39.906 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /tmp 2024-09-11T23:10:39.906 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:10:40.084 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:10:40.101 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:10:40.101 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:40.101 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T23:10:40.101 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:10:40.992 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:10:40.993 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2024-09-11T23:10:40.993 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2024-09-11T23:10:40.993 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2024-09-11T23:10:40.993 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2024-09-11T23:10:40.993 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2024-09-11T23:10:40.993 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /root 2024-09-11T23:10:40.993 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2024-09-11T23:10:40.993 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:10:41.169 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:10:41.188 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:10:41.188 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:41.188 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T23:10:41.188 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:10:41.645 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:10:41.674 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:10:41.674 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:41.674 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T23:10:41.674 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:10:41.674 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:10:41.674 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:10:41.913 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:10:41.937 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:10:41.938 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:41.938 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T23:10:41.938 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:10:41.938 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:10:41.938 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:10:42.398 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:10:42.423 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:10:42.423 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:42.423 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T23:10:42.423 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:10:42.423 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:10:42.423 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:10:42.529 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T23:10:42.596 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T23:10:42.659 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T23:10:42.664 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:10:42.691 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:10:42.691 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:42.691 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T23:10:42.691 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:10:42.691 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:10:42.691 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:10:42.852 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T23:10:42.914 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:10:42.916 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T23:10:42.938 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:10:42.938 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:42.938 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T23:10:42.938 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:10:42.938 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:10:42.938 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:10:43.021 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T23:10:43.122 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T23:10:43.263 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T23:10:43.330 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T23:10:43.364 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T23:10:43.373 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:10:43.392 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T23:10:43.397 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:10:43.397 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:43.397 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T23:10:43.397 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:10:43.397 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:10:43.397 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:10:43.504 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T23:10:43.569 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T23:10:43.624 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T23:10:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T23:10:43.858 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T23:10:43.872 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:10:43.899 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:10:43.928 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T23:10:43.931 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T23:10:43.995 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T23:10:44.022 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:10:44.024 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T23:10:44.172 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T23:10:44.225 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T23:10:44.257 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T23:10:44.285 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T23:10:44.727 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:10:44.766 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:10:44.790 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T23:10:44.852 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T23:10:44.882 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:10:45.127 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:10:45.166 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:10:45.929 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:10:45.966 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:10:46.180 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:10:46.207 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:10:46.316 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:10:46.484 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T23:10:46.509 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:10:46.534 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:10:46.534 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:46.534 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T23:10:46.534 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:10:46.534 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:10:46.534 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:10:46.585 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:10:46.596 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:10:46.698 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T23:10:46.754 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T23:10:46.931 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:10:46.967 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:10:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:10:47.265 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T23:10:47.294 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:10:47.318 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:10:47.318 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:47.319 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T23:10:47.319 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:10:47.319 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:10:47.319 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:10:47.370 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:10:47.382 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:10:47.480 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T23:10:47.535 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T23:10:47.744 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T23:10:47.771 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:10:47.795 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:10:47.795 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:47.795 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T23:10:47.795 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:10:47.795 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:10:47.796 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:10:47.838 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:10:47.862 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:10:47.863 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:47.863 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T23:10:47.863 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:10:47.913 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:10:47.938 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:10:47.938 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:47.938 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T23:10:47.939 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:10:47.939 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:10:47.939 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:10:48.256 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:10:48.281 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:10:48.281 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:48.281 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T23:10:48.281 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:10:48.281 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:10:48.281 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:10:48.457 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T23:10:48.489 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:10:48.513 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:10:48.513 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:48.513 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T23:10:48.513 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:10:48.513 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:10:48.513 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:10:48.549 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T23:10:48.549 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /sys 2024-09-11T23:10:48.549 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /proc 2024-09-11T23:10:48.549 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /mnt 2024-09-11T23:10:48.549 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /var/tmp 2024-09-11T23:10:48.549 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /home 2024-09-11T23:10:48.549 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /root 2024-09-11T23:10:48.549 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /tmp 2024-09-11T23:10:48.549 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-11T23:10:48.556 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:10:48.579 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:10:48.580 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:48.580 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T23:10:48.580 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:10:48.632 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:10:48.655 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:10:48.655 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:48.656 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T23:10:48.656 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:10:48.656 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:10:48.656 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:10:48.722 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:10:48.738 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T23:10:48.738 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:48.738 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T23:10:48.739 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-11T23:10:48.958 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:10:48.981 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:10:48.982 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:48.982 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T23:10:48.982 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:10:48.982 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:10:48.982 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:10:50.269 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:10:50.296 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T23:10:50.296 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:50.296 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T23:10:50.296 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:10:50.296 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T23:10:50.296 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-11T23:10:50.528 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:10:50.551 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T23:10:50.551 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:50.552 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T23:10:50.552 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:10:50.552 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T23:10:50.552 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-11T23:10:50.987 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:10:51.011 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T23:10:51.011 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:51.011 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T23:10:51.011 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:10:51.011 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T23:10:51.011 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-11T23:10:51.126 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T23:10:51.183 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T23:10:51.246 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T23:10:51.431 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T23:10:51.490 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T23:10:51.567 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T23:10:51.656 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T23:10:51.840 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T23:10:51.883 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T23:10:51.915 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T23:10:51.943 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T23:10:52.437 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:10:52.469 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T23:10:52.509 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T23:10:52.570 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T23:10:52.600 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:10:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T23:10:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T23:10:53.194 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T23:10:53.225 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T23:10:53.243 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T23:10:53.442 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T23:10:53.482 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T23:10:53.512 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T23:10:53.537 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T23:10:53.542 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T23:10:53.578 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T23:10:53.588 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T23:10:53.618 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T23:10:53.638 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T23:10:53.651 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T23:10:53.678 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:10:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T23:10:53.858 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T23:10:53.908 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T23:10:53.939 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T23:10:53.968 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T23:10:54.006 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T23:10:54.231 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:10:54.269 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:10:54.291 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:10:54.635 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T23:10:54.635 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:10:54.671 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:10:54.692 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:10:54.696 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:10:54.799 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T23:10:54.959 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T23:10:54.991 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:10:55.015 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T23:10:55.015 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:55.015 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T23:10:55.015 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:10:55.015 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T23:10:55.015 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-11T23:10:55.066 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:10:55.079 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T23:10:55.176 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T23:10:55.213 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T23:10:55.223 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T23:10:55.299 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T23:10:55.348 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T23:10:55.447 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T23:10:55.476 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T23:10:55.502 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T23:10:55.530 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T23:10:55.536 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T23:10:55.536 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:10:55.626 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T23:10:55.628 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:10:55.629 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:10:55.667 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T23:10:55.713 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:10:55.738 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T23:10:55.744 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:10:55.744 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:10:55.747 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T23:10:55.780 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T23:10:55.805 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T23:10:55.831 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T23:10:55.831 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:10:55.915 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:10:55.915 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:10:55.931 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:10:55.931 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:10:55.999 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:10:56.016 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:10:56.024 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T23:10:56.030 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:10:56.030 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:10:56.084 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:10:56.144 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T23:10:56.178 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:10:56.201 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T23:10:56.202 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:56.202 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T23:10:56.202 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:10:56.202 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T23:10:56.202 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-11T23:10:56.226 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:10:56.226 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:10:56.245 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:10:56.268 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T23:10:56.268 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:56.268 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T23:10:56.268 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-11T23:10:56.310 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:10:56.322 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:10:56.346 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T23:10:56.346 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:56.346 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T23:10:56.346 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:10:56.346 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T23:10:56.346 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-11T23:10:56.383 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:10:56.648 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:10:56.672 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T23:10:56.672 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T23:10:56.672 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T23:10:56.672 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:10:56.672 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T23:10:56.672 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-11T23:11:01.249 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T23:11:01.286 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T23:11:01.336 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T23:11:01.369 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T23:11:01.405 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T23:11:01.585 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T23:11:01.624 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:11:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T23:11:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T23:11:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T23:11:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T23:11:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T23:11:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T23:11:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T23:11:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:11:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:11:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:11:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T23:11:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T23:11:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:11:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T23:11:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T23:11:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T23:11:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T23:11:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T23:11:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T23:11:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T23:11:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T23:11:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T23:11:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T23:11:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T23:11:01.627 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T23:11:01.630 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T23:11:01.630 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T23:11:01.630 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T23:11:01.630 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T23:11:01.630 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T23:11:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T23:11:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T23:11:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T23:11:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T23:11:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T23:11:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T23:11:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T23:11:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T23:11:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T23:11:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T23:11:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T23:11:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T23:11:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T23:11:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T23:11:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T23:11:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T23:11:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T23:11:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T23:11:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T23:11:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T23:11:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T23:11:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T23:11:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T23:11:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T23:11:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T23:11:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T23:11:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T23:11:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T23:11:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T23:11:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T23:11:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T23:11:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T23:11:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T23:11:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T23:11:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T23:11:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T23:11:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T23:11:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T23:11:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T23:11:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T23:11:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T23:11:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T23:11:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T23:11:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T23:11:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T23:11:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T23:11:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T23:11:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T23:11:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T23:11:01.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T23:11:01.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T23:11:01.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T23:11:01.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T23:11:01.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T23:11:01.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T23:11:01.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T23:11:01.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T23:11:01.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T23:11:01.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T23:11:01.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T23:11:01.646 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T23:11:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T23:11:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T23:11:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T23:11:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T23:11:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T23:11:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T23:11:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T23:11:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T23:11:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T23:11:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T23:11:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T23:11:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T23:11:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T23:11:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T23:11:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T23:11:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T23:11:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T23:11:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T23:11:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T23:11:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T23:11:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T23:11:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T23:11:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T23:11:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T23:11:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T23:11:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T23:11:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T23:11:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T23:11:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T23:11:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T23:11:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T23:11:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T23:11:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T23:11:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T23:11:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T23:11:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T23:11:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T23:11:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T23:11:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T23:11:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T23:11:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T23:11:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T23:11:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T23:11:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T23:11:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T23:11:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T23:11:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T23:11:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T23:11:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T23:11:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T23:11:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T23:11:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T23:11:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T23:11:01.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T23:11:01.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T23:11:01.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T23:11:01.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T23:11:01.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T23:11:01.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T23:11:01.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T23:11:01.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T23:11:01.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T23:11:01.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T23:11:01.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T23:11:01.651 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T23:11:01.652 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T23:11:01.652 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T23:11:01.652 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T23:11:01.652 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T23:11:01.652 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T23:11:01.652 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T23:11:01.652 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T23:11:01.652 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T23:11:01.674 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T23:11:01.706 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T23:11:01.752 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T23:11:02.135 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:11:02.135 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T23:11:02.135 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T23:11:02.135 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T23:11:02.135 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T23:11:02.135 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T23:11:02.135 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T23:11:02.136 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T23:11:02.136 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:11:02.136 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:11:02.136 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:11:02.136 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T23:11:02.136 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T23:11:02.136 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:11:02.136 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T23:11:02.137 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T23:11:02.137 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T23:11:02.137 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T23:11:02.137 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T23:11:02.137 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T23:11:02.137 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T23:11:02.137 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T23:11:02.137 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T23:11:02.137 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T23:11:02.137 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T23:11:02.137 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T23:11:02.138 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T23:11:02.138 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T23:11:02.138 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T23:11:02.138 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T23:11:02.138 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T23:11:02.138 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T23:11:02.138 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T23:11:02.138 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T23:11:02.138 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T23:11:02.138 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T23:11:02.138 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T23:11:02.138 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T23:11:02.139 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T23:11:02.139 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T23:11:02.139 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T23:11:02.139 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T23:11:02.139 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T23:11:02.139 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T23:11:02.139 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T23:11:02.139 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T23:11:02.139 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T23:11:02.139 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T23:11:02.139 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T23:11:02.139 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T23:11:02.139 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T23:11:02.140 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T23:11:02.140 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T23:11:02.140 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T23:11:02.140 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T23:11:02.140 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T23:11:02.140 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T23:11:02.140 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T23:11:02.140 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T23:11:02.140 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T23:11:02.140 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T23:11:02.140 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T23:11:02.140 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T23:11:02.140 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T23:11:02.141 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T23:11:02.141 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T23:11:02.141 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T23:11:02.141 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T23:11:02.141 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T23:11:02.141 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T23:11:02.141 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T23:11:02.141 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T23:11:02.141 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T23:11:02.141 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T23:11:02.141 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T23:11:02.141 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T23:11:02.142 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T23:11:02.142 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T23:11:02.142 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T23:11:02.142 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T23:11:02.142 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T23:11:02.142 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T23:11:02.142 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T23:11:02.142 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T23:11:02.142 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T23:11:02.142 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T23:11:02.142 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T23:11:02.142 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T23:11:02.142 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T23:11:02.143 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T23:11:02.143 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T23:11:02.143 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T23:11:02.143 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T23:11:02.143 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T23:11:02.143 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T23:11:02.143 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T23:11:02.143 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T23:11:02.143 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T23:11:02.143 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T23:11:02.143 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T23:11:02.143 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T23:11:02.143 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T23:11:02.144 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T23:11:02.144 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T23:11:02.144 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T23:11:02.144 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T23:11:02.144 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T23:11:02.144 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T23:11:02.144 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T23:11:02.144 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T23:11:02.144 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T23:11:02.144 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T23:11:02.144 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T23:11:02.144 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T23:11:02.145 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T23:11:02.145 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T23:11:02.145 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T23:11:02.145 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T23:11:02.145 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T23:11:02.145 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T23:11:02.145 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T23:11:02.145 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T23:11:02.145 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T23:11:02.145 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T23:11:02.145 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T23:11:02.145 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T23:11:02.145 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T23:11:02.145 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T23:11:02.146 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T23:11:02.146 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T23:11:02.146 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T23:11:02.146 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T23:11:02.146 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T23:11:02.146 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T23:11:02.146 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T23:11:02.146 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T23:11:02.146 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T23:11:02.146 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T23:11:02.146 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T23:11:02.146 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T23:11:02.146 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T23:11:02.147 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T23:11:02.147 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T23:11:02.147 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T23:11:02.147 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T23:11:02.147 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T23:11:02.147 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T23:11:02.147 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T23:11:02.147 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T23:11:02.147 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T23:11:02.147 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T23:11:02.147 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T23:11:02.147 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T23:11:02.147 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T23:11:02.148 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T23:11:02.148 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T23:11:02.148 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T23:11:02.148 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T23:11:02.148 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T23:11:02.148 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T23:11:02.148 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T23:11:02.148 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T23:11:02.148 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T23:11:02.148 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T23:11:02.355 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:11:02.403 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:11:02.423 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T23:11:03.250 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T23:11:03.395 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T23:11:03.436 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T23:11:03.518 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T23:11:03.550 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T23:11:03.584 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T23:11:03.618 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T23:11:03.619 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:11:03.719 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T23:11:03.719 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:11:03.820 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T23:11:03.845 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T23:11:03.851 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:11:03.851 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:11:04.040 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T23:11:04.040 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:11:04.133 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T23:11:04.205 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:11:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:11:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:11:04.249 INFO:teuthology.orchestra.run.smithi043.stdout:Upgraded: 2024-09-11T23:11:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T23:11:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T23:11:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T23:11:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T23:11:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T23:11:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T23:11:04.250 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-11T23:11:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T23:11:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T23:11:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T23:11:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T23:11:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T23:11:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T23:11:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T23:11:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T23:11:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T23:11:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T23:11:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T23:11:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T23:11:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T23:11:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T23:11:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T23:11:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T23:11:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T23:11:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T23:11:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T23:11:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T23:11:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T23:11:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T23:11:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T23:11:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T23:11:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T23:11:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T23:11:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T23:11:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T23:11:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T23:11:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T23:11:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T23:11:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T23:11:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T23:11:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T23:11:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T23:11:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T23:11:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T23:11:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T23:11:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T23:11:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T23:11:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T23:11:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T23:11:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T23:11:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T23:11:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T23:11:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T23:11:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T23:11:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T23:11:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T23:11:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T23:11:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T23:11:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T23:11:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T23:11:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T23:11:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T23:11:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T23:11:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T23:11:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T23:11:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T23:11:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T23:11:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T23:11:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T23:11:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T23:11:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T23:11:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T23:11:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T23:11:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T23:11:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T23:11:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T23:11:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T23:11:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T23:11:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T23:11:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T23:11:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T23:11:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T23:11:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T23:11:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T23:11:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T23:11:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T23:11:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T23:11:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T23:11:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T23:11:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T23:11:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T23:11:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T23:11:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T23:11:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T23:11:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T23:11:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T23:11:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T23:11:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T23:11:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T23:11:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T23:11:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T23:11:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T23:11:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T23:11:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T23:11:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T23:11:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T23:11:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T23:11:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T23:11:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T23:11:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T23:11:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T23:11:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T23:11:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T23:11:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T23:11:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T23:11:04.262 INFO:teuthology.orchestra.run.smithi043.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T23:11:04.262 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:11:04.262 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-11T23:11:04.392 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:11:04.393 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:11:04.393 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2024-09-11T23:11:04.393 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T23:11:04.393 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T23:11:04.393 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T23:11:04.393 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T23:11:04.393 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.393 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.393 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T23:11:04.393 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T23:11:04.393 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-09-11T23:11:04.393 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T23:11:04.394 INFO:teuthology.orchestra.run.smithi097.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T23:11:04.394 INFO:teuthology.orchestra.run.smithi097.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T23:11:04.394 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T23:11:04.394 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.394 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.394 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.394 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.394 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:04.394 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.394 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.394 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.394 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:04.395 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:04.395 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:04.395 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:04.395 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:04.395 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.395 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.395 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:04.395 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.395 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.395 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.395 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:04.395 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:04.396 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.396 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.396 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:04.396 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T23:11:04.396 INFO:teuthology.orchestra.run.smithi097.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T23:11:04.396 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T23:11:04.396 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T23:11:04.396 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T23:11:04.396 INFO:teuthology.orchestra.run.smithi097.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T23:11:04.396 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T23:11:04.396 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T23:11:04.396 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T23:11:04.397 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T23:11:04.397 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T23:11:04.397 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.397 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.397 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.397 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T23:11:04.397 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T23:11:04.397 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T23:11:04.397 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T23:11:04.397 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T23:11:04.397 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.397 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.397 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T23:11:04.398 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.398 INFO:teuthology.orchestra.run.smithi097.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T23:11:04.398 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T23:11:04.398 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T23:11:04.398 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T23:11:04.398 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T23:11:04.398 INFO:teuthology.orchestra.run.smithi097.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T23:11:04.398 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T23:11:04.398 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T23:11:04.398 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T23:11:04.398 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T23:11:04.398 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T23:11:04.398 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T23:11:04.399 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T23:11:04.399 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T23:11:04.399 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T23:11:04.399 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T23:11:04.399 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T23:11:04.399 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T23:11:04.399 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T23:11:04.399 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T23:11:04.399 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T23:11:04.399 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T23:11:04.399 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T23:11:04.399 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.399 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.400 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.400 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T23:11:04.400 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T23:11:04.400 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T23:11:04.400 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T23:11:04.400 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T23:11:04.400 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T23:11:04.400 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T23:11:04.400 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T23:11:04.400 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T23:11:04.400 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T23:11:04.400 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T23:11:04.401 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T23:11:04.401 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T23:11:04.401 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T23:11:04.401 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T23:11:04.401 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T23:11:04.401 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T23:11:04.401 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T23:11:04.401 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T23:11:04.401 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T23:11:04.401 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T23:11:04.401 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T23:11:04.401 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T23:11:04.401 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T23:11:04.402 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T23:11:04.402 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T23:11:04.402 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T23:11:04.402 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T23:11:04.402 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T23:11:04.402 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T23:11:04.402 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T23:11:04.402 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T23:11:04.402 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T23:11:04.402 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T23:11:04.402 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T23:11:04.402 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T23:11:04.402 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T23:11:04.402 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T23:11:04.403 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T23:11:04.403 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T23:11:04.403 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.403 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.403 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T23:11:04.403 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T23:11:04.403 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T23:11:04.403 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.403 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T23:11:04.403 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T23:11:04.403 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T23:11:04.403 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T23:11:04.403 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T23:11:04.404 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T23:11:04.404 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T23:11:04.404 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T23:11:04.404 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T23:11:04.404 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T23:11:04.404 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T23:11:04.404 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T23:11:04.404 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T23:11:04.404 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T23:11:04.404 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T23:11:04.404 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.404 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.404 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:04.405 INFO:teuthology.orchestra.run.smithi097.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T23:11:04.405 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T23:11:04.405 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T23:11:04.405 INFO:teuthology.orchestra.run.smithi097.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T23:11:04.405 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T23:11:04.405 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T23:11:04.405 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T23:11:04.405 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:11:04.406 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-11T23:11:04.673 DEBUG:teuthology.parallel:result is None 2024-09-11T23:11:04.806 DEBUG:teuthology.parallel:result is None 2024-09-11T23:11:12.059 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:11:12.059 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T23:11:12.060 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T23:11:12.060 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T23:11:12.060 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T23:11:12.060 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T23:11:12.060 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T23:11:12.060 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T23:11:12.060 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T23:11:12.060 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T23:11:12.060 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T23:11:12.060 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T23:11:12.060 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T23:11:12.061 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T23:11:12.061 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T23:11:12.061 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T23:11:12.061 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T23:11:12.061 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T23:11:12.061 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T23:11:12.061 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T23:11:12.061 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T23:11:12.061 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T23:11:12.061 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T23:11:12.061 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T23:11:12.061 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T23:11:12.061 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T23:11:12.062 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T23:11:12.062 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T23:11:12.062 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T23:11:12.062 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T23:11:12.074 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T23:11:12.074 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T23:11:12.074 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T23:11:12.074 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T23:11:12.074 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T23:11:12.074 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T23:11:12.075 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T23:11:12.075 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T23:11:12.075 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T23:11:12.075 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T23:11:12.075 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T23:11:12.075 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T23:11:12.075 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T23:11:12.075 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T23:11:12.075 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T23:11:12.075 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T23:11:12.075 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T23:11:12.075 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T23:11:12.076 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T23:11:12.076 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T23:11:12.076 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T23:11:12.076 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T23:11:12.076 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T23:11:12.076 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T23:11:12.076 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T23:11:12.076 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T23:11:12.076 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T23:11:12.076 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T23:11:12.076 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T23:11:12.076 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T23:11:12.077 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T23:11:12.077 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T23:11:12.077 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T23:11:12.077 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T23:11:12.077 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T23:11:12.077 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T23:11:12.077 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T23:11:12.077 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T23:11:12.077 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T23:11:12.077 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T23:11:12.077 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T23:11:12.077 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T23:11:12.078 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T23:11:12.078 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T23:11:12.078 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T23:11:12.078 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T23:11:12.078 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T23:11:12.078 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T23:11:12.078 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T23:11:12.078 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T23:11:12.078 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T23:11:12.078 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T23:11:12.078 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T23:11:12.078 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T23:11:12.079 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T23:11:12.079 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T23:11:12.079 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T23:11:12.079 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T23:11:12.079 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T23:11:12.079 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T23:11:12.079 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T23:11:12.079 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T23:11:12.079 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T23:11:12.079 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T23:11:12.079 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T23:11:12.079 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T23:11:12.079 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T23:11:12.080 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T23:11:12.080 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T23:11:12.080 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T23:11:12.080 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T23:11:12.080 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T23:11:12.080 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T23:11:12.080 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T23:11:12.080 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T23:11:12.080 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T23:11:12.080 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T23:11:12.080 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T23:11:12.080 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T23:11:12.081 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T23:11:12.081 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T23:11:12.081 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T23:11:12.081 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T23:11:12.081 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T23:11:12.081 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T23:11:12.081 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T23:11:12.081 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T23:11:12.081 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T23:11:12.081 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T23:11:12.081 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T23:11:12.081 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T23:11:12.081 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T23:11:12.082 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T23:11:12.082 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T23:11:12.082 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T23:11:12.082 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T23:11:12.082 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T23:11:12.082 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T23:11:12.082 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T23:11:12.082 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T23:11:12.082 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T23:11:12.082 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T23:11:12.082 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T23:11:12.082 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T23:11:12.083 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T23:11:12.083 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T23:11:12.083 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T23:11:12.083 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T23:11:12.083 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T23:11:12.083 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T23:11:12.083 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T23:11:12.083 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T23:11:12.083 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T23:11:12.083 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T23:11:12.083 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T23:11:12.083 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T23:11:12.083 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T23:11:12.084 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T23:11:12.084 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T23:11:12.084 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T23:11:12.084 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T23:11:12.084 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T23:11:12.084 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T23:11:12.084 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T23:11:12.084 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T23:11:12.084 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T23:11:12.084 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T23:11:12.084 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T23:11:12.084 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T23:11:12.085 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T23:11:12.085 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T23:11:12.085 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T23:11:12.085 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T23:11:12.085 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T23:11:14.630 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T23:11:14.630 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-11T23:11:14.630 INFO:teuthology.orchestra.run.smithi090.stdout:Upgraded: 2024-09-11T23:11:14.630 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T23:11:14.630 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T23:11:14.630 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T23:11:14.630 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T23:11:14.631 INFO:teuthology.orchestra.run.smithi090.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.631 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.631 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T23:11:14.631 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T23:11:14.631 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-09-11T23:11:14.631 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T23:11:14.631 INFO:teuthology.orchestra.run.smithi090.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T23:11:14.631 INFO:teuthology.orchestra.run.smithi090.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T23:11:14.631 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T23:11:14.631 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.631 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.632 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.632 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.632 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:14.632 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.632 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.632 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.632 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:14.632 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:14.632 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:14.632 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:14.632 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:14.632 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.633 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.633 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:14.633 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.633 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.633 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.633 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:14.633 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:14.633 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.633 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.633 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T23:11:14.633 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T23:11:14.633 INFO:teuthology.orchestra.run.smithi090.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T23:11:14.634 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T23:11:14.634 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T23:11:14.634 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T23:11:14.634 INFO:teuthology.orchestra.run.smithi090.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T23:11:14.634 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T23:11:14.634 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T23:11:14.634 INFO:teuthology.orchestra.run.smithi090.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T23:11:14.634 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T23:11:14.634 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T23:11:14.634 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.634 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.634 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.635 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T23:11:14.635 INFO:teuthology.orchestra.run.smithi090.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T23:11:14.635 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T23:11:14.635 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T23:11:14.635 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T23:11:14.635 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.635 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.635 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T23:11:14.635 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.635 INFO:teuthology.orchestra.run.smithi090.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T23:11:14.635 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T23:11:14.635 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T23:11:14.635 INFO:teuthology.orchestra.run.smithi090.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T23:11:14.636 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T23:11:14.636 INFO:teuthology.orchestra.run.smithi090.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T23:11:14.636 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T23:11:14.636 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T23:11:14.636 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T23:11:14.636 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T23:11:14.636 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T23:11:14.636 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T23:11:14.636 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T23:11:14.636 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T23:11:14.636 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T23:11:14.636 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T23:11:14.637 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T23:11:14.637 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T23:11:14.637 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T23:11:14.637 INFO:teuthology.orchestra.run.smithi090.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T23:11:14.637 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T23:11:14.637 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T23:11:14.637 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T23:11:14.637 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.637 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.637 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.637 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T23:11:14.637 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T23:11:14.637 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T23:11:14.638 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T23:11:14.638 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T23:11:14.638 INFO:teuthology.orchestra.run.smithi090.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T23:11:14.638 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T23:11:14.638 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T23:11:14.638 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T23:11:14.638 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T23:11:14.638 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T23:11:14.638 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T23:11:14.638 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T23:11:14.638 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T23:11:14.639 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T23:11:14.639 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T23:11:14.639 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T23:11:14.639 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T23:11:14.639 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T23:11:14.639 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T23:11:14.639 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T23:11:14.639 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T23:11:14.639 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T23:11:14.640 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T23:11:14.640 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T23:11:14.640 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T23:11:14.640 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T23:11:14.640 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T23:11:14.640 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T23:11:14.640 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T23:11:14.640 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T23:11:14.640 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T23:11:14.640 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T23:11:14.640 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T23:11:14.640 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T23:11:14.640 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T23:11:14.641 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T23:11:14.641 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T23:11:14.641 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T23:11:14.641 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T23:11:14.641 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.641 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.641 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T23:11:14.641 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T23:11:14.641 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T23:11:14.641 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.641 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T23:11:14.641 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T23:11:14.642 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T23:11:14.642 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T23:11:14.642 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T23:11:14.642 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T23:11:14.642 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T23:11:14.642 INFO:teuthology.orchestra.run.smithi090.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T23:11:14.642 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T23:11:14.642 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T23:11:14.642 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T23:11:14.642 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T23:11:14.642 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T23:11:14.642 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T23:11:14.642 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T23:11:14.643 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.643 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.643 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T23:11:14.643 INFO:teuthology.orchestra.run.smithi090.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T23:11:14.643 INFO:teuthology.orchestra.run.smithi090.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T23:11:14.643 INFO:teuthology.orchestra.run.smithi090.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T23:11:14.643 INFO:teuthology.orchestra.run.smithi090.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T23:11:14.643 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T23:11:14.643 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T23:11:14.644 INFO:teuthology.orchestra.run.smithi090.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T23:11:14.644 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-11T23:11:14.644 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-09-11T23:11:15.135 DEBUG:teuthology.parallel:result is None 2024-09-11T23:11:15.138 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:11:15.300 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T23:11:15.328 INFO:teuthology.orchestra.run.smithi043.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T23:11:15.328 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T23:11:15.328 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T23:11:15.331 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:11:15.479 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T23:11:15.503 INFO:teuthology.orchestra.run.smithi090.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T23:11:15.503 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T23:11:15.504 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T23:11:15.505 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:11:15.659 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T23:11:15.686 INFO:teuthology.orchestra.run.smithi097.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T23:11:15.686 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T23:11:15.687 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T23:11:15.689 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T23:11:15.689 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T23:11:15.689 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T23:11:15.722 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-11T23:11:15.722 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T23:11:15.753 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-11T23:11:15.754 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T23:11:15.786 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T23:11:15.786 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T23:11:15.786 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T23:11:15.816 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T23:11:15.886 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-11T23:11:15.887 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T23:11:15.917 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T23:11:15.986 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-11T23:11:15.986 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T23:11:16.016 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T23:11:16.084 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T23:11:16.085 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T23:11:16.085 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T23:11:16.116 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T23:11:16.186 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-11T23:11:16.186 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T23:11:16.215 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T23:11:16.285 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-11T23:11:16.285 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T23:11:16.315 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T23:11:16.383 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T23:11:16.383 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T23:11:16.383 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T23:11:16.410 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T23:11:16.475 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-11T23:11:16.476 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T23:11:16.505 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T23:11:16.573 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-11T23:11:16.573 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T23:11:16.602 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T23:11:16.671 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-11T23:11:16.766 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request', 'SLOW_OPS', 'slow request'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-11T23:11:16.766 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:11:16.767 INFO:tasks.cephadm:Cluster fsid is 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:11:16.767 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-11T23:11:16.767 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.43', 'mon.b': '172.21.15.90', 'mon.c': '172.21.15.97'} 2024-09-11T23:11:16.767 INFO:tasks.cephadm:First mon is mon.a on smithi043 2024-09-11T23:11:16.767 INFO:tasks.cephadm:First mgr is x 2024-09-11T23:11:16.767 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-11T23:11:16.768 DEBUG:teuthology.orchestra.run.smithi043:> sudo hostname $(hostname -s) 2024-09-11T23:11:16.794 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname $(hostname -s) 2024-09-11T23:11:16.823 DEBUG:teuthology.orchestra.run.smithi097:> sudo hostname $(hostname -s) 2024-09-11T23:11:16.852 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-11T23:11:16.853 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:11:17.011 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-11T23:11:17.157 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:11:17.158 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-11T23:11:17.158 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-11T23:11:17.159 DEBUG:teuthology.orchestra.run.smithi043:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T23:11:17.560 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 23:11 /home/ubuntu/cephtest/cephadm 2024-09-11T23:11:17.561 DEBUG:teuthology.orchestra.run.smithi090:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T23:11:17.929 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 23:11 /home/ubuntu/cephtest/cephadm 2024-09-11T23:11:17.930 DEBUG:teuthology.orchestra.run.smithi097:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T23:11:18.291 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 23:11 /home/ubuntu/cephtest/cephadm 2024-09-11T23:11:18.292 DEBUG:teuthology.orchestra.run.smithi043:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T23:11:18.313 DEBUG:teuthology.orchestra.run.smithi090:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T23:11:18.334 DEBUG:teuthology.orchestra.run.smithi097:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T23:11:18.368 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-11T23:11:18.369 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T23:11:18.371 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T23:11:18.376 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T23:11:19.901 INFO:teuthology.orchestra.run.smithi097.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T23:11:19.913 INFO:teuthology.orchestra.run.smithi043.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T23:11:20.762 INFO:teuthology.orchestra.run.smithi090.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T23:11:50.896 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-11T23:11:50.896 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T23:11:50.896 INFO:teuthology.orchestra.run.smithi097.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T23:11:50.896 INFO:teuthology.orchestra.run.smithi097.stdout: "repo_digests": [ 2024-09-11T23:11:50.896 INFO:teuthology.orchestra.run.smithi097.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T23:11:50.896 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-11T23:11:50.897 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-11T23:11:52.864 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:11:52.865 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T23:11:52.865 INFO:teuthology.orchestra.run.smithi043.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T23:11:52.865 INFO:teuthology.orchestra.run.smithi043.stdout: "repo_digests": [ 2024-09-11T23:11:52.865 INFO:teuthology.orchestra.run.smithi043.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T23:11:52.865 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-11T23:11:52.865 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:11:54.220 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-11T23:11:54.220 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T23:11:54.220 INFO:teuthology.orchestra.run.smithi090.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T23:11:54.220 INFO:teuthology.orchestra.run.smithi090.stdout: "repo_digests": [ 2024-09-11T23:11:54.220 INFO:teuthology.orchestra.run.smithi090.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T23:11:54.221 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-09-11T23:11:54.221 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-11T23:11:54.246 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph 2024-09-11T23:11:54.279 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /etc/ceph 2024-09-11T23:11:54.308 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph 2024-09-11T23:11:54.341 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 777 /etc/ceph 2024-09-11T23:11:54.371 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 777 /etc/ceph 2024-09-11T23:11:54.395 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /etc/ceph 2024-09-11T23:11:54.423 INFO:tasks.cephadm:Writing seed config... 2024-09-11T23:11:54.425 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-11T23:11:54.425 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-11T23:11:54.425 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-11T23:11:54.425 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-11T23:11:54.425 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-11T23:11:54.425 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-11T23:11:54.425 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-11T23:11:54.425 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-11T23:11:54.425 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-11T23:11:54.425 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-11T23:11:54.425 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-11T23:11:54.426 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-11T23:11:54.426 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-11T23:11:54.426 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-11T23:11:54.426 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-11T23:11:54.426 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-11T23:11:54.426 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-11T23:11:54.426 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-11T23:11:54.426 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-11T23:11:54.426 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-11T23:11:54.426 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-11T23:11:54.426 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-11T23:11:54.426 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-11T23:11:54.426 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-11T23:11:54.426 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-11T23:11:54.427 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-11T23:11:54.427 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-11T23:11:54.427 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-11T23:11:54.427 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-11T23:11:54.427 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-11T23:11:54.427 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-11T23:11:54.427 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-11T23:11:54.427 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-11T23:11:54.427 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-11T23:11:54.427 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-11T23:11:54.427 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-11T23:11:54.427 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-11T23:11:54.427 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-11T23:11:54.427 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-11T23:11:54.428 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-11T23:11:54.428 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-11T23:11:54.428 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2024-09-11T23:11:54.428 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-11T23:11:54.428 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-11T23:11:54.428 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-11T23:11:54.428 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-11T23:11:54.428 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2024-09-11T23:11:54.428 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-11T23:11:54.428 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-11T23:11:54.428 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-11T23:11:54.428 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-11T23:11:54.429 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T23:11:54.429 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-11T23:11:54.448 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 25c0cd84-7093-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-11T23:11:54.448 DEBUG:teuthology.orchestra.run.smithi043:mon.a> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mon.a.service 2024-09-11T23:11:54.491 DEBUG:teuthology.orchestra.run.smithi043:mgr.x> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mgr.x.service 2024-09-11T23:11:54.535 INFO:tasks.cephadm:Bootstrapping... 2024-09-11T23:11:54.535 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.43 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:11:54.751 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-11T23:11:54.751 INFO:teuthology.orchestra.run.smithi043.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', '25c0cd84-7093-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.43', '--skip-admin-label'] 2024-09-11T23:11:54.751 INFO:teuthology.orchestra.run.smithi043.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-11T23:11:54.752 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying podman|docker is present... 2024-09-11T23:11:54.775 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 5.2.2 2024-09-11T23:11:54.775 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying lvm2 is present... 2024-09-11T23:11:54.775 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying time synchronization is in place... 2024-09-11T23:11:54.783 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T23:11:54.783 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T23:11:54.789 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T23:11:54.789 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-11T23:11:54.796 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout enabled 2024-09-11T23:11:54.803 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout active 2024-09-11T23:11:54.803 INFO:teuthology.orchestra.run.smithi043.stdout:Unit chronyd.service is enabled and running 2024-09-11T23:11:54.803 INFO:teuthology.orchestra.run.smithi043.stdout:Repeating the final host check... 2024-09-11T23:11:54.826 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 5.2.2 2024-09-11T23:11:54.826 INFO:teuthology.orchestra.run.smithi043.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-11T23:11:54.826 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl is present 2024-09-11T23:11:54.827 INFO:teuthology.orchestra.run.smithi043.stdout:lvcreate is present 2024-09-11T23:11:54.833 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T23:11:54.833 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T23:11:54.839 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T23:11:54.840 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-11T23:11:54.846 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout enabled 2024-09-11T23:11:54.853 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout active 2024-09-11T23:11:54.853 INFO:teuthology.orchestra.run.smithi043.stdout:Unit chronyd.service is enabled and running 2024-09-11T23:11:54.853 INFO:teuthology.orchestra.run.smithi043.stdout:Host looks OK 2024-09-11T23:11:54.853 INFO:teuthology.orchestra.run.smithi043.stdout:Cluster fsid: 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:11:54.853 INFO:teuthology.orchestra.run.smithi043.stdout:Acquiring lock 140146236568576 on /run/cephadm/25c0cd84-7093-11ef-bcea-c7b262605968.lock 2024-09-11T23:11:54.853 INFO:teuthology.orchestra.run.smithi043.stdout:Lock 140146236568576 acquired on /run/cephadm/25c0cd84-7093-11ef-bcea-c7b262605968.lock 2024-09-11T23:11:54.854 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying IP 172.21.15.43 port 3300 ... 2024-09-11T23:11:54.854 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying IP 172.21.15.43 port 6789 ... 2024-09-11T23:11:54.854 INFO:teuthology.orchestra.run.smithi043.stdout:Base mon IP(s) is [172.21.15.43:3300, 172.21.15.43:6789], mon addrv is [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-09-11T23:11:54.857 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.43 metric 100 2024-09-11T23:11:54.857 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.43 metric 100 2024-09-11T23:11:54.860 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-11T23:11:54.860 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-11T23:11:54.860 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1732sec hoplimit 64 pref medium 2024-09-11T23:11:54.863 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-11T23:11:54.863 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-11T23:11:54.863 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T23:11:54.863 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-11T23:11:54.863 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-11T23:11:54.863 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T23:11:54.864 INFO:teuthology.orchestra.run.smithi043.stdout:Mon IP `172.21.15.43` is in CIDR network `172.21.0.0/20` 2024-09-11T23:11:54.864 INFO:teuthology.orchestra.run.smithi043.stdout:Mon IP `172.21.15.43` is in CIDR network `172.21.0.0/20` 2024-09-11T23:11:54.864 INFO:teuthology.orchestra.run.smithi043.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-11T23:11:54.864 INFO:teuthology.orchestra.run.smithi043.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-11T23:11:54.865 INFO:teuthology.orchestra.run.smithi043.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T23:11:55.282 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-11T23:11:55.283 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T23:11:55.283 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Getting image source signatures 2024-09-11T23:11:55.283 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-11T23:11:55.283 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-11T23:11:55.283 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-11T23:11:55.283 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-11T23:11:56.194 INFO:teuthology.orchestra.run.smithi043.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-11T23:11:56.195 INFO:teuthology.orchestra.run.smithi043.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-11T23:11:56.195 INFO:teuthology.orchestra.run.smithi043.stdout:Extracting ceph user uid/gid from container image... 2024-09-11T23:11:57.159 INFO:teuthology.orchestra.run.smithi043.stdout:stat: stdout 167 167 2024-09-11T23:11:57.159 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial keys... 2024-09-11T23:11:58.214 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQC9I+JmeQ1vHhAA5QKHRiuUUGWPk+olkZ0FOA== 2024-09-11T23:11:59.128 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQC+I+JmhNyrHhAArfJ+e0yBDmDu4vXRx4BvVg== 2024-09-11T23:12:00.025 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQC/I+JmjXeBGRAATlegxmQ1IZjym/GglGu8yg== 2024-09-11T23:12:00.026 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial monmap... 2024-09-11T23:12:00.972 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T23:12:00.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-11T23:12:00.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:12:00.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T23:12:00.973 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool for a [v2:172.21.15.43:3300,v1:172.21.15.43:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T23:12:00.973 INFO:teuthology.orchestra.run.smithi043.stdout:setting min_mon_release = quincy 2024-09-11T23:12:00.973 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: set fsid to 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:12:00.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T23:12:00.974 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:12:00.974 INFO:teuthology.orchestra.run.smithi043.stdout:Creating mon... 2024-09-11T23:12:01.944 INFO:teuthology.orchestra.run.smithi043.stdout:create mon.a on 2024-09-11T23:12:02.161 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-11T23:12:02.328 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-11T23:12:02.587 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-25c0cd84-7093-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-25c0cd84-7093-11ef-bcea-c7b262605968.target. 2024-09-11T23:12:02.587 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-25c0cd84-7093-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-25c0cd84-7093-11ef-bcea-c7b262605968.target. 2024-09-11T23:12:02.851 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mon.a 2024-09-11T23:12:02.851 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to reset failed state of unit ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-11T23:12:03.034 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-25c0cd84-7093-11ef-bcea-c7b262605968.target.wants/ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-25c0cd84-7093-11ef-bcea-c7b262605968@.service. 2024-09-11T23:12:03.560 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T23:12:03.560 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-09-11T23:12:03.567 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T23:12:03.568 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-11T23:12:03.568 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-09-11T23:12:03.568 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T23:12:03.568 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mon to start... 2024-09-11T23:12:03.568 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mon... 2024-09-11T23:12:04.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:04 smithi043 ceph-mon[30583]: from='client.? 172.21.15.43:0/2987405227' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-11T23:12:04.612 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout cluster: 2024-09-11T23:12:04.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout id: 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:12:04.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-11T23:12:04.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T23:12:04.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout services: 2024-09-11T23:12:04.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.54639s) 2024-09-11T23:12:04.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-11T23:12:04.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-11T23:12:04.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T23:12:04.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout data: 2024-09-11T23:12:04.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-11T23:12:04.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-11T23:12:04.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-11T23:12:04.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout pgs: 2024-09-11T23:12:04.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T23:12:04.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.977+0000 7fa89d119640 1 Processor -- start 2024-09-11T23:12:04.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.977+0000 7fa89d119640 1 -- start start 2024-09-11T23:12:04.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.977+0000 7fa89d119640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa898107930 0x7fa898107d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:04.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.977+0000 7fa89d119640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa898108300 con 0x7fa898107930 2024-09-11T23:12:04.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.978+0000 7fa896d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa898107930 0x7fa898107d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:04.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.978+0000 7fa896d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa898107930 0x7fa898107d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39634/0 (socket says 172.21.15.43:39634) 2024-09-11T23:12:04.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.978+0000 7fa896d9d640 1 -- 172.21.15.43:0/2922418032 learned_addr learned my addr 172.21.15.43:0/2922418032 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:04.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.978+0000 7fa896d9d640 1 -- 172.21.15.43:0/2922418032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa898108a90 con 0x7fa898107930 2024-09-11T23:12:04.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.980+0000 7fa896d9d640 1 --2- 172.21.15.43:0/2922418032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa898107930 0x7fa898107d30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa888009820 tx=0x7fa88802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31a3229ad4187fe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:04.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.980+0000 7fa895d9b640 1 -- 172.21.15.43:0/2922418032 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa888037070 con 0x7fa898107930 2024-09-11T23:12:04.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.980+0000 7fa895d9b640 1 -- 172.21.15.43:0/2922418032 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa888032440 con 0x7fa898107930 2024-09-11T23:12:04.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.980+0000 7fa895d9b640 1 -- 172.21.15.43:0/2922418032 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa888032740 con 0x7fa898107930 2024-09-11T23:12:04.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.980+0000 7fa89d119640 1 -- 172.21.15.43:0/2922418032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa898107930 msgr2=0x7fa898107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:04.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.980+0000 7fa89d119640 1 --2- 172.21.15.43:0/2922418032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa898107930 0x7fa898107d30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa888009820 tx=0x7fa88802a270 comp rx=0 tx=0).stop 2024-09-11T23:12:04.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.980+0000 7fa89d119640 1 -- 172.21.15.43:0/2922418032 shutdown_connections 2024-09-11T23:12:04.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.980+0000 7fa89d119640 1 --2- 172.21.15.43:0/2922418032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa898107930 0x7fa898107d30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:04.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.980+0000 7fa89d119640 1 -- 172.21.15.43:0/2922418032 >> 172.21.15.43:0/2922418032 conn(0x7fa8981030e0 msgr2=0x7fa898105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:04.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.980+0000 7fa89d119640 1 -- 172.21.15.43:0/2922418032 shutdown_connections 2024-09-11T23:12:04.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.980+0000 7fa89d119640 1 -- 172.21.15.43:0/2922418032 wait complete. 2024-09-11T23:12:04.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.981+0000 7fa89d119640 1 Processor -- start 2024-09-11T23:12:04.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.981+0000 7fa89d119640 1 -- start start 2024-09-11T23:12:04.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.981+0000 7fa89d119640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa898107930 0x7fa89819b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:04.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.981+0000 7fa89d119640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa898109030 con 0x7fa898107930 2024-09-11T23:12:04.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.982+0000 7fa896d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa898107930 0x7fa89819b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:04.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.982+0000 7fa896d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa898107930 0x7fa89819b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39636/0 (socket says 172.21.15.43:39636) 2024-09-11T23:12:04.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.982+0000 7fa896d9d640 1 -- 172.21.15.43:0/2987405227 learned_addr learned my addr 172.21.15.43:0/2987405227 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:04.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.982+0000 7fa896d9d640 1 -- 172.21.15.43:0/2987405227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89819b570 con 0x7fa898107930 2024-09-11T23:12:04.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.982+0000 7fa896d9d640 1 --2- 172.21.15.43:0/2987405227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa898107930 0x7fa89819b030 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa8880090b0 tx=0x7fa8880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:04.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.982+0000 7fa88ffff640 1 -- 172.21.15.43:0/2987405227 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa888041070 con 0x7fa898107930 2024-09-11T23:12:04.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.982+0000 7fa88ffff640 1 -- 172.21.15.43:0/2987405227 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa88803c7a0 con 0x7fa898107930 2024-09-11T23:12:04.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.982+0000 7fa89d119640 1 -- 172.21.15.43:0/2987405227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa89819b800 con 0x7fa898107930 2024-09-11T23:12:04.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.982+0000 7fa88ffff640 1 -- 172.21.15.43:0/2987405227 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa888037040 con 0x7fa898107930 2024-09-11T23:12:04.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.982+0000 7fa89d119640 1 -- 172.21.15.43:0/2987405227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa89819e4f0 con 0x7fa898107930 2024-09-11T23:12:04.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.982+0000 7fa88ffff640 1 -- 172.21.15.43:0/2987405227 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa888044070 con 0x7fa898107930 2024-09-11T23:12:04.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.982+0000 7fa88ffff640 1 -- 172.21.15.43:0/2987405227 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa88803d3f0 con 0x7fa898107930 2024-09-11T23:12:04.622 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.983+0000 7fa89d119640 1 -- 172.21.15.43:0/2987405227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa898109680 con 0x7fa898107930 2024-09-11T23:12:04.623 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:03.986+0000 7fa88ffff640 1 -- 172.21.15.43:0/2987405227 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fa888046450 con 0x7fa898107930 2024-09-11T23:12:04.623 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:04.048+0000 7fa89d119640 1 -- 172.21.15.43:0/2987405227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fa89819ea40 con 0x7fa898107930 2024-09-11T23:12:04.623 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:04.048+0000 7fa88ffff640 1 -- 172.21.15.43:0/2987405227 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+316 (secure 0 0 0) 0x7fa888046630 con 0x7fa898107930 2024-09-11T23:12:04.623 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:04.049+0000 7fa89d119640 1 -- 172.21.15.43:0/2987405227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa898107930 msgr2=0x7fa89819b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:04.623 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:04.049+0000 7fa89d119640 1 --2- 172.21.15.43:0/2987405227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa898107930 0x7fa89819b030 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa8880090b0 tx=0x7fa8880046c0 comp rx=0 tx=0).stop 2024-09-11T23:12:04.623 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:04.049+0000 7fa89d119640 1 -- 172.21.15.43:0/2987405227 shutdown_connections 2024-09-11T23:12:04.623 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:04.049+0000 7fa89d119640 1 --2- 172.21.15.43:0/2987405227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa898107930 0x7fa89819b030 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:04.623 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:04.049+0000 7fa89d119640 1 -- 172.21.15.43:0/2987405227 >> 172.21.15.43:0/2987405227 conn(0x7fa8981030e0 msgr2=0x7fa89818f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:04.623 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:04.049+0000 7fa89d119640 1 -- 172.21.15.43:0/2987405227 shutdown_connections 2024-09-11T23:12:04.623 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:04.049+0000 7fa89d119640 1 -- 172.21.15.43:0/2987405227 wait complete. 2024-09-11T23:12:04.623 INFO:teuthology.orchestra.run.smithi043.stdout:mon is available 2024-09-11T23:12:04.623 INFO:teuthology.orchestra.run.smithi043.stdout:Assimilating anything we can from ceph.conf... 2024-09-11T23:12:05.675 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:05 smithi043 ceph-mon[30583]: from='client.? 172.21.15.43:0/3408090886' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T23:12:05.675 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:05 smithi043 ceph-mon[30583]: from='client.? 172.21.15.43:0/3408090886' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T23:12:05.678 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T23:12:05.678 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T23:12:05.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout fsid = 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:12:05.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-09-11T23:12:05.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T23:12:05.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T23:12:05.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T23:12:05.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T23:12:05.679 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T23:12:05.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T23:12:05.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T23:12:05.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T23:12:05.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T23:12:05.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-09-11T23:12:05.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-11T23:12:05.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-11T23:12:05.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-11T23:12:05.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T23:12:05.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T23:12:05.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.027+0000 7f3c55a3b640 1 Processor -- start 2024-09-11T23:12:05.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.027+0000 7f3c55a3b640 1 -- start start 2024-09-11T23:12:05.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.027+0000 7f3c55a3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c5007dd80 0x7f3c5007c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:05.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.027+0000 7f3c55a3b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5007c830 con 0x7f3c5007dd80 2024-09-11T23:12:05.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.027+0000 7f3c4effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c5007dd80 0x7f3c5007c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:05.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.027+0000 7f3c4effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c5007dd80 0x7f3c5007c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39640/0 (socket says 172.21.15.43:39640) 2024-09-11T23:12:05.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.027+0000 7f3c4effd640 1 -- 172.21.15.43:0/3924152370 learned_addr learned my addr 172.21.15.43:0/3924152370 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:05.694 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.028+0000 7f3c4effd640 1 -- 172.21.15.43:0/3924152370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5007c9b0 con 0x7f3c5007dd80 2024-09-11T23:12:05.694 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.028+0000 7f3c4effd640 1 --2- 172.21.15.43:0/3924152370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c5007dd80 0x7f3c5007c2f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009820 tx=0x7f3c4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b13e32500fcfbb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:05.694 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.029+0000 7f3c4dffb640 1 -- 172.21.15.43:0/3924152370 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c48037070 con 0x7f3c5007dd80 2024-09-11T23:12:05.694 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.029+0000 7f3c4dffb640 1 -- 172.21.15.43:0/3924152370 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c48032440 con 0x7f3c5007dd80 2024-09-11T23:12:05.694 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.029+0000 7f3c4dffb640 1 -- 172.21.15.43:0/3924152370 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c48032740 con 0x7f3c5007dd80 2024-09-11T23:12:05.694 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.029+0000 7f3c55a3b640 1 -- 172.21.15.43:0/3924152370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c5007dd80 msgr2=0x7f3c5007c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:05.695 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.029+0000 7f3c55a3b640 1 --2- 172.21.15.43:0/3924152370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c5007dd80 0x7f3c5007c2f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009820 tx=0x7f3c4802a270 comp rx=0 tx=0).stop 2024-09-11T23:12:05.695 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.029+0000 7f3c55a3b640 1 -- 172.21.15.43:0/3924152370 shutdown_connections 2024-09-11T23:12:05.695 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.029+0000 7f3c55a3b640 1 --2- 172.21.15.43:0/3924152370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c5007dd80 0x7f3c5007c2f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:05.695 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.029+0000 7f3c55a3b640 1 -- 172.21.15.43:0/3924152370 >> 172.21.15.43:0/3924152370 conn(0x7f3c501031c0 msgr2=0x7f3c501055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:05.695 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.029+0000 7f3c55a3b640 1 -- 172.21.15.43:0/3924152370 shutdown_connections 2024-09-11T23:12:05.695 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.029+0000 7f3c55a3b640 1 -- 172.21.15.43:0/3924152370 wait complete. 2024-09-11T23:12:05.696 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.030+0000 7f3c55a3b640 1 Processor -- start 2024-09-11T23:12:05.696 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.030+0000 7f3c55a3b640 1 -- start start 2024-09-11T23:12:05.696 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.030+0000 7f3c55a3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c5007dd80 0x7f3c5019d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:05.696 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.030+0000 7f3c55a3b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5007ce80 con 0x7f3c5007dd80 2024-09-11T23:12:05.696 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.030+0000 7f3c4effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c5007dd80 0x7f3c5019d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:05.697 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.030+0000 7f3c4effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c5007dd80 0x7f3c5019d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39656/0 (socket says 172.21.15.43:39656) 2024-09-11T23:12:05.697 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.030+0000 7f3c4effd640 1 -- 172.21.15.43:0/3408090886 learned_addr learned my addr 172.21.15.43:0/3408090886 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:05.697 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.031+0000 7f3c4effd640 1 -- 172.21.15.43:0/3408090886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5019d760 con 0x7f3c5007dd80 2024-09-11T23:12:05.698 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.031+0000 7f3c4effd640 1 --2- 172.21.15.43:0/3408090886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c5007dd80 0x7f3c5019d220 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3c480090b0 tx=0x7f3c480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:05.698 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.031+0000 7f3c54a39640 1 -- 172.21.15.43:0/3408090886 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c48041070 con 0x7f3c5007dd80 2024-09-11T23:12:05.698 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.031+0000 7f3c54a39640 1 -- 172.21.15.43:0/3408090886 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c4803c7a0 con 0x7f3c5007dd80 2024-09-11T23:12:05.698 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.031+0000 7f3c55a3b640 1 -- 172.21.15.43:0/3408090886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c5019d9f0 con 0x7f3c5007dd80 2024-09-11T23:12:05.698 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.031+0000 7f3c54a39640 1 -- 172.21.15.43:0/3408090886 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c48037040 con 0x7f3c5007dd80 2024-09-11T23:12:05.698 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.031+0000 7f3c55a3b640 1 -- 172.21.15.43:0/3408090886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c501a06e0 con 0x7f3c5007dd80 2024-09-11T23:12:05.699 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.031+0000 7f3c54a39640 1 -- 172.21.15.43:0/3408090886 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3c48044070 con 0x7f3c5007dd80 2024-09-11T23:12:05.699 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.031+0000 7f3c54a39640 1 -- 172.21.15.43:0/3408090886 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3c4803d3f0 con 0x7f3c5007dd80 2024-09-11T23:12:05.699 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.032+0000 7f3c55a3b640 1 -- 172.21.15.43:0/3408090886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c5007c370 con 0x7f3c5007dd80 2024-09-11T23:12:05.699 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.034+0000 7f3c54a39640 1 -- 172.21.15.43:0/3408090886 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f3c48046450 con 0x7f3c5007dd80 2024-09-11T23:12:05.699 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.095+0000 7f3c55a3b640 1 -- 172.21.15.43:0/3408090886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f3c501a1db0 con 0x7f3c5007dd80 2024-09-11T23:12:05.699 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.098+0000 7f3c54a39640 1 -- 172.21.15.43:0/3408090886 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+454 (secure 0 0 0) 0x7f3c48046630 con 0x7f3c5007dd80 2024-09-11T23:12:05.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.098+0000 7f3c54a39640 1 -- 172.21.15.43:0/3408090886 <== mon.0 v2:172.21.15.43:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3c48055070 con 0x7f3c5007dd80 2024-09-11T23:12:05.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.099+0000 7f3c55a3b640 1 -- 172.21.15.43:0/3408090886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c5007dd80 msgr2=0x7f3c5019d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:05.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.099+0000 7f3c55a3b640 1 --2- 172.21.15.43:0/3408090886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c5007dd80 0x7f3c5019d220 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3c480090b0 tx=0x7f3c480046c0 comp rx=0 tx=0).stop 2024-09-11T23:12:05.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.099+0000 7f3c55a3b640 1 -- 172.21.15.43:0/3408090886 shutdown_connections 2024-09-11T23:12:05.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.099+0000 7f3c55a3b640 1 --2- 172.21.15.43:0/3408090886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c5007dd80 0x7f3c5019d220 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:05.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.099+0000 7f3c55a3b640 1 -- 172.21.15.43:0/3408090886 >> 172.21.15.43:0/3408090886 conn(0x7f3c501031c0 msgr2=0x7f3c501048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:05.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.099+0000 7f3c55a3b640 1 -- 172.21.15.43:0/3408090886 shutdown_connections 2024-09-11T23:12:05.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:05.099+0000 7f3c55a3b640 1 -- 172.21.15.43:0/3408090886 wait complete. 2024-09-11T23:12:05.701 INFO:teuthology.orchestra.run.smithi043.stdout:Generating new minimal ceph.conf... 2024-09-11T23:12:06.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:06 smithi043 ceph-mon[30583]: from='client.? 172.21.15.43:0/2018354030' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:12:06.723 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.083+0000 7f36c3c05640 1 Processor -- start 2024-09-11T23:12:06.723 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.083+0000 7f36c3c05640 1 -- start start 2024-09-11T23:12:06.723 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.083+0000 7f36c3c05640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36bc106860 0x7f36bc106c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:06.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.083+0000 7f36c3c05640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc107230 con 0x7f36bc106860 2024-09-11T23:12:06.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.083+0000 7f36c19a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36bc106860 0x7f36bc106c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:06.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.084+0000 7f36c19a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36bc106860 0x7f36bc106c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39672/0 (socket says 172.21.15.43:39672) 2024-09-11T23:12:06.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.084+0000 7f36c19a1640 1 -- 172.21.15.43:0/3372980771 learned_addr learned my addr 172.21.15.43:0/3372980771 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:06.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.084+0000 7f36c19a1640 1 -- 172.21.15.43:0/3372980771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36bc107a10 con 0x7f36bc106860 2024-09-11T23:12:06.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.084+0000 7f36c19a1640 1 --2- 172.21.15.43:0/3372980771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36bc106860 0x7f36bc106c60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009820 tx=0x7f36b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0438059fe33ea02 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:06.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.085+0000 7f36c099f640 1 -- 172.21.15.43:0/3372980771 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36b8037070 con 0x7f36bc106860 2024-09-11T23:12:06.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.085+0000 7f36c099f640 1 -- 172.21.15.43:0/3372980771 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f36b8032900 con 0x7f36bc106860 2024-09-11T23:12:06.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.085+0000 7f36c099f640 1 -- 172.21.15.43:0/3372980771 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36b8030850 con 0x7f36bc106860 2024-09-11T23:12:06.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.085+0000 7f36c3c05640 1 -- 172.21.15.43:0/3372980771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36bc106860 msgr2=0x7f36bc106c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:06.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.085+0000 7f36c3c05640 1 --2- 172.21.15.43:0/3372980771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36bc106860 0x7f36bc106c60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009820 tx=0x7f36b802a270 comp rx=0 tx=0).stop 2024-09-11T23:12:06.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.085+0000 7f36c3c05640 1 -- 172.21.15.43:0/3372980771 shutdown_connections 2024-09-11T23:12:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.085+0000 7f36c3c05640 1 --2- 172.21.15.43:0/3372980771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36bc106860 0x7f36bc106c60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.085+0000 7f36c3c05640 1 -- 172.21.15.43:0/3372980771 >> 172.21.15.43:0/3372980771 conn(0x7f36bc102010 msgr2=0x7f36bc104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.086+0000 7f36c3c05640 1 -- 172.21.15.43:0/3372980771 shutdown_connections 2024-09-11T23:12:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.086+0000 7f36c3c05640 1 -- 172.21.15.43:0/3372980771 wait complete. 2024-09-11T23:12:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.086+0000 7f36c3c05640 1 Processor -- start 2024-09-11T23:12:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.086+0000 7f36c3c05640 1 -- start start 2024-09-11T23:12:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.086+0000 7f36c3c05640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36bc106860 0x7f36bc1919d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.086+0000 7f36c3c05640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc10adc0 con 0x7f36bc106860 2024-09-11T23:12:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.086+0000 7f36c19a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36bc106860 0x7f36bc1919d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.086+0000 7f36c19a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36bc106860 0x7f36bc1919d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39682/0 (socket says 172.21.15.43:39682) 2024-09-11T23:12:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.086+0000 7f36c19a1640 1 -- 172.21.15.43:0/2018354030 learned_addr learned my addr 172.21.15.43:0/2018354030 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.087+0000 7f36c19a1640 1 -- 172.21.15.43:0/2018354030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36bc191f10 con 0x7f36bc106860 2024-09-11T23:12:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.087+0000 7f36c19a1640 1 --2- 172.21.15.43:0/2018354030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36bc106860 0x7f36bc1919d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009940 tx=0x7f36b8032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.087+0000 7f36b2ffd640 1 -- 172.21.15.43:0/2018354030 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36b8042070 con 0x7f36bc106860 2024-09-11T23:12:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.087+0000 7f36b2ffd640 1 -- 172.21.15.43:0/2018354030 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f36b80308c0 con 0x7f36bc106860 2024-09-11T23:12:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.087+0000 7f36c3c05640 1 -- 172.21.15.43:0/2018354030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36bc190100 con 0x7f36bc106860 2024-09-11T23:12:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.087+0000 7f36b2ffd640 1 -- 172.21.15.43:0/2018354030 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36b8037040 con 0x7f36bc106860 2024-09-11T23:12:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.087+0000 7f36c3c05640 1 -- 172.21.15.43:0/2018354030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36bc190640 con 0x7f36bc106860 2024-09-11T23:12:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.088+0000 7f36b2ffd640 1 -- 172.21.15.43:0/2018354030 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f36b8045070 con 0x7f36bc106860 2024-09-11T23:12:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.088+0000 7f36b2ffd640 1 -- 172.21.15.43:0/2018354030 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f36b8047b70 con 0x7f36bc106860 2024-09-11T23:12:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.088+0000 7f36c3c05640 1 -- 172.21.15.43:0/2018354030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36bc106ce0 con 0x7f36bc106860 2024-09-11T23:12:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.091+0000 7f36b2ffd640 1 -- 172.21.15.43:0/2018354030 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f36b80327b0 con 0x7f36bc106860 2024-09-11T23:12:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.150+0000 7f36c3c05640 1 -- 172.21.15.43:0/2018354030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f36bc1952e0 con 0x7f36bc106860 2024-09-11T23:12:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.151+0000 7f36b2ffd640 1 -- 172.21.15.43:0/2018354030 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f36b803c8f0 con 0x7f36bc106860 2024-09-11T23:12:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.152+0000 7f36c3c05640 1 -- 172.21.15.43:0/2018354030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36bc106860 msgr2=0x7f36bc1919d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:06.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.152+0000 7f36c3c05640 1 --2- 172.21.15.43:0/2018354030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36bc106860 0x7f36bc1919d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009940 tx=0x7f36b8032ce0 comp rx=0 tx=0).stop 2024-09-11T23:12:06.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.152+0000 7f36c3c05640 1 -- 172.21.15.43:0/2018354030 shutdown_connections 2024-09-11T23:12:06.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.152+0000 7f36c3c05640 1 --2- 172.21.15.43:0/2018354030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36bc106860 0x7f36bc1919d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:06.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.152+0000 7f36c3c05640 1 -- 172.21.15.43:0/2018354030 >> 172.21.15.43:0/2018354030 conn(0x7f36bc102010 msgr2=0x7f36bc102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:06.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.152+0000 7f36c3c05640 1 -- 172.21.15.43:0/2018354030 shutdown_connections 2024-09-11T23:12:06.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:06.152+0000 7f36c3c05640 1 -- 172.21.15.43:0/2018354030 wait complete. 2024-09-11T23:12:06.727 INFO:teuthology.orchestra.run.smithi043.stdout:Restarting the monitor... 2024-09-11T23:12:07.002 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:06 smithi043 systemd[1]: Stopping Ceph mon.a for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:12:07.326 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:07 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-a[30579]: 2024-09-11T23:12:07.000+0000 7fa198483640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:12:07.326 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:07 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-a[30579]: 2024-09-11T23:12:07.000+0000 7fa198483640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-11T23:12:07.326 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:07 smithi043 podman[30825]: 2024-09-11 23:12:07.021974482 +0000 UTC m=+0.128507083 container stop 4c83127461f2c3d15aa209b8d96d9e00bd91939f7ab1820ba100c59aeb91e6ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T23:12:07.326 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:07 smithi043 podman[30825]: 2024-09-11 23:12:07.060087628 +0000 UTC m=+0.166620315 container died 4c83127461f2c3d15aa209b8d96d9e00bd91939f7ab1820ba100c59aeb91e6ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-b53b6d3) 2024-09-11T23:12:07.609 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:07 smithi043 podman[30825]: 2024-09-11 23:12:07.433991153 +0000 UTC m=+0.540523755 container cleanup 4c83127461f2c3d15aa209b8d96d9e00bd91939f7ab1820ba100c59aeb91e6ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-11T23:12:07.609 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:07 smithi043 bash[30825]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-a 2024-09-11T23:12:07.859 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:07 smithi043 podman[30837]: 2024-09-11 23:12:07.609935804 +0000 UTC m=+0.581987432 container remove 4c83127461f2c3d15aa209b8d96d9e00bd91939f7ab1820ba100c59aeb91e6ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-11T23:12:07.859 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:07 smithi043 systemd[1]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T23:12:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:07 smithi043 systemd[1]: Stopped Ceph mon.a for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:12:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:07 smithi043 systemd[1]: Starting Ceph mon.a for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:12:08.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 podman[30931]: 2024-09-11 23:12:07.985171982 +0000 UTC m=+0.016273478 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:12:08.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 podman[30931]: 2024-09-11 23:12:08.085795186 +0000 UTC m=+0.116896713 container create 176c0fba3df58a2066e3348fbb3d216f502a92f6be69c391569422176a7baee5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-11T23:12:08.350 INFO:teuthology.orchestra.run.smithi043.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-11T23:12:08.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 podman[30931]: 2024-09-11 23:12:08.271301495 +0000 UTC m=+0.302402986 container init 176c0fba3df58a2066e3348fbb3d216f502a92f6be69c391569422176a7baee5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-a, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-09-11T23:12:08.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 podman[30931]: 2024-09-11 23:12:08.274536688 +0000 UTC m=+0.305638183 container start 176c0fba3df58a2066e3348fbb3d216f502a92f6be69c391569422176a7baee5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-11T23:12:08.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T23:12:08.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-11T23:12:08.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: pidfile_write: ignore empty --pid-file 2024-09-11T23:12:08.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: load: jerasure load: lrc 2024-09-11T23:12:08.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: RocksDB version: 7.9.2 2024-09-11T23:12:08.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Git sha 0 2024-09-11T23:12:08.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-11T23:12:08.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: DB SUMMARY 2024-09-11T23:12:08.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: DB Session ID: MHXU7W3DOLBMWOIX31RT 2024-09-11T23:12:08.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: CURRENT file: CURRENT 2024-09-11T23:12:08.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T23:12:08.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-11T23:12:08.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-11T23:12:08.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106864 ; 2024-09-11T23:12:08.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.error_if_exists: 0 2024-09-11T23:12:08.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.create_if_missing: 0 2024-09-11T23:12:08.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.paranoid_checks: 1 2024-09-11T23:12:08.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-11T23:12:08.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T23:12:08.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-11T23:12:08.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.env: 0x55f875318940 2024-09-11T23:12:08.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.fs: PosixFileSystem 2024-09-11T23:12:08.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.info_log: 0x55f8765c56e0 2024-09-11T23:12:08.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T23:12:08.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.statistics: (nil) 2024-09-11T23:12:08.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.use_fsync: 0 2024-09-11T23:12:08.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_log_file_size: 0 2024-09-11T23:12:08.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T23:12:08.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T23:12:08.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T23:12:08.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T23:12:08.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.allow_fallocate: 1 2024-09-11T23:12:08.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T23:12:08.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T23:12:08.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.use_direct_reads: 0 2024-09-11T23:12:08.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T23:12:08.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T23:12:08.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.db_log_dir: 2024-09-11T23:12:08.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.wal_dir: 2024-09-11T23:12:08.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T23:12:08.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T23:12:08.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T23:12:08.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T23:12:08.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T23:12:08.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T23:12:08.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T23:12:08.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T23:12:08.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.write_buffer_manager: 0x55f8765d4dc0 2024-09-11T23:12:08.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T23:12:08.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T23:12:08.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T23:12:08.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T23:12:08.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T23:12:08.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T23:12:08.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T23:12:08.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T23:12:08.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.unordered_write: 0 2024-09-11T23:12:08.482 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T23:12:08.482 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T23:12:08.482 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T23:12:08.482 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T23:12:08.482 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.row_cache: None 2024-09-11T23:12:08.482 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.wal_filter: None 2024-09-11T23:12:08.482 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T23:12:08.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T23:12:08.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.two_write_queues: 0 2024-09-11T23:12:08.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T23:12:08.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.wal_compression: 0 2024-09-11T23:12:08.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.atomic_flush: 0 2024-09-11T23:12:08.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T23:12:08.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T23:12:08.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T23:12:08.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.log_readahead_size: 0 2024-09-11T23:12:08.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T23:12:08.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T23:12:08.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T23:12:08.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T23:12:08.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T23:12:08.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T23:12:08.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-11T23:12:08.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_background_jobs: 2 2024-09-11T23:12:08.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_background_compactions: -1 2024-09-11T23:12:08.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_subcompactions: 1 2024-09-11T23:12:08.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T23:12:08.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T23:12:08.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T23:12:08.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T23:12:08.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T23:12:08.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T23:12:08.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T23:12:08.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T23:12:08.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_open_files: -1 2024-09-11T23:12:08.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T23:12:08.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T23:12:08.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T23:12:08.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T23:12:08.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_background_flushes: -1 2024-09-11T23:12:08.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Compression algorithms supported: 2024-09-11T23:12:08.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: kZSTD supported: 0 2024-09-11T23:12:08.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: kXpressCompression supported: 0 2024-09-11T23:12:08.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: kBZip2Compression supported: 0 2024-09-11T23:12:08.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T23:12:08.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: kLZ4Compression supported: 1 2024-09-11T23:12:08.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: kZlibCompression supported: 1 2024-09-11T23:12:08.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T23:12:08.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: kSnappyCompression supported: 1 2024-09-11T23:12:08.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T23:12:08.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-11T23:12:08.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-11T23:12:08.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-11T23:12:08.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T23:12:08.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.merge_operator: 2024-09-11T23:12:08.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compaction_filter: None 2024-09-11T23:12:08.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T23:12:08.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T23:12:08.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T23:12:08.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T23:12:08.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f8765c5320) 2024-09-11T23:12:08.487 INFO:journalctl@ceph.mon.a.smithi043.stdout: cache_index_and_filter_blocks: 1 2024-09-11T23:12:08.487 INFO:journalctl@ceph.mon.a.smithi043.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T23:12:08.488 INFO:journalctl@ceph.mon.a.smithi043.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T23:12:08.488 INFO:journalctl@ceph.mon.a.smithi043.stdout: pin_top_level_index_and_filter: 1 2024-09-11T23:12:08.488 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_type: 0 2024-09-11T23:12:08.488 INFO:journalctl@ceph.mon.a.smithi043.stdout: data_block_index_type: 0 2024-09-11T23:12:08.488 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_shortening: 1 2024-09-11T23:12:08.488 INFO:journalctl@ceph.mon.a.smithi043.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T23:12:08.488 INFO:journalctl@ceph.mon.a.smithi043.stdout: checksum: 4 2024-09-11T23:12:08.488 INFO:journalctl@ceph.mon.a.smithi043.stdout: no_block_cache: 0 2024-09-11T23:12:08.488 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache: 0x55f8765b91f0 2024-09-11T23:12:08.488 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_name: BinnedLRUCache 2024-09-11T23:12:08.488 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_options: 2024-09-11T23:12:08.488 INFO:journalctl@ceph.mon.a.smithi043.stdout: capacity : 536870912 2024-09-11T23:12:08.489 INFO:journalctl@ceph.mon.a.smithi043.stdout: num_shard_bits : 4 2024-09-11T23:12:08.489 INFO:journalctl@ceph.mon.a.smithi043.stdout: strict_capacity_limit : 0 2024-09-11T23:12:08.489 INFO:journalctl@ceph.mon.a.smithi043.stdout: high_pri_pool_ratio: 0.000 2024-09-11T23:12:08.489 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_compressed: (nil) 2024-09-11T23:12:08.489 INFO:journalctl@ceph.mon.a.smithi043.stdout: persistent_cache: (nil) 2024-09-11T23:12:08.489 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_size: 4096 2024-09-11T23:12:08.489 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_size_deviation: 10 2024-09-11T23:12:08.489 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_restart_interval: 16 2024-09-11T23:12:08.489 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_block_restart_interval: 1 2024-09-11T23:12:08.489 INFO:journalctl@ceph.mon.a.smithi043.stdout: metadata_block_size: 4096 2024-09-11T23:12:08.489 INFO:journalctl@ceph.mon.a.smithi043.stdout: partition_filters: 0 2024-09-11T23:12:08.489 INFO:journalctl@ceph.mon.a.smithi043.stdout: use_delta_encoding: 1 2024-09-11T23:12:08.489 INFO:journalctl@ceph.mon.a.smithi043.stdout: filter_policy: bloomfilter 2024-09-11T23:12:08.489 INFO:journalctl@ceph.mon.a.smithi043.stdout: whole_key_filtering: 1 2024-09-11T23:12:08.490 INFO:journalctl@ceph.mon.a.smithi043.stdout: verify_compression: 0 2024-09-11T23:12:08.490 INFO:journalctl@ceph.mon.a.smithi043.stdout: read_amp_bytes_per_bit: 0 2024-09-11T23:12:08.490 INFO:journalctl@ceph.mon.a.smithi043.stdout: format_version: 5 2024-09-11T23:12:08.490 INFO:journalctl@ceph.mon.a.smithi043.stdout: enable_index_compression: 1 2024-09-11T23:12:08.490 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_align: 0 2024-09-11T23:12:08.490 INFO:journalctl@ceph.mon.a.smithi043.stdout: max_auto_readahead_size: 262144 2024-09-11T23:12:08.490 INFO:journalctl@ceph.mon.a.smithi043.stdout: prepopulate_block_cache: 0 2024-09-11T23:12:08.490 INFO:journalctl@ceph.mon.a.smithi043.stdout: initial_auto_readahead_size: 8192 2024-09-11T23:12:08.490 INFO:journalctl@ceph.mon.a.smithi043.stdout: num_file_reads_for_auto_readahead: 2 2024-09-11T23:12:08.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T23:12:08.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T23:12:08.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compression: NoCompression 2024-09-11T23:12:08.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T23:12:08.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T23:12:08.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T23:12:08.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.num_levels: 7 2024-09-11T23:12:08.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T23:12:08.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T23:12:08.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T23:12:08.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T23:12:08.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T23:12:08.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T23:12:08.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T23:12:08.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T23:12:08.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T23:12:08.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T23:12:08.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-11T23:12:08.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-11T23:12:08.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T23:12:08.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T23:12:08.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T23:12:08.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T23:12:08.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T23:12:08.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-11T23:12:08.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T23:12:08.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T23:12:08.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-11T23:12:08.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T23:12:08.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T23:12:08.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T23:12:08.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T23:12:08.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T23:12:08.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T23:12:08.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T23:12:08.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T23:12:08.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T23:12:08.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T23:12:08.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T23:12:08.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T23:12:08.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T23:12:08.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T23:12:08.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T23:12:08.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T23:12:08.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T23:12:08.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-11T23:12:08.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.arena_block_size: 1048576 2024-09-11T23:12:08.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T23:12:08.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T23:12:08.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T23:12:08.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T23:12:08.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T23:12:08.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T23:12:08.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T23:12:08.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T23:12:08.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T23:12:08.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T23:12:08.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T23:12:08.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T23:12:08.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T23:12:08.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T23:12:08.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.inplace_update_support: 0 2024-09-11T23:12:08.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T23:12:08.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T23:12:08.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T23:12:08.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T23:12:08.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.bloom_locality: 0 2024-09-11T23:12:08.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.max_successive_merges: 0 2024-09-11T23:12:08.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T23:12:08.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T23:12:08.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T23:12:08.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T23:12:08.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.ttl: 2592000 2024-09-11T23:12:08.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T23:12:08.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-11T23:12:08.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-11T23:12:08.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.enable_blob_files: false 2024-09-11T23:12:08.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.min_blob_size: 0 2024-09-11T23:12:08.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T23:12:08.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T23:12:08.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T23:12:08.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T23:12:08.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-11T23:12:08.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-11T23:12:08.496 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.blob_file_starting_level: 0 2024-09-11T23:12:08.497 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-11T23:12:08.497 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-11T23:12:08.497 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-11T23:12:08.497 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1700ad8b-98ad-4f71-b7a2-2d35f15a85b7 2024-09-11T23:12:08.497 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726096328301301, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-11T23:12:08.497 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-11T23:12:08.497 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726096328302414, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101748, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 334, "table_properties": {"data_size": 99635, "index_size": 305, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14681, "raw_average_key_size": 50, "raw_value_size": 91728, "raw_average_value_size": 313, "num_data_blocks": 12, "num_entries": 293, "num_filter_entries": 293, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726096328, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1700ad8b-98ad-4f71-b7a2-2d35f15a85b7", "db_session_id": "MHXU7W3DOLBMWOIX31RT", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-11T23:12:08.497 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726096328302513, "job": 1, "event": "recovery_finished"} 2024-09-11T23:12:08.497 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-11T23:12:08.497 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T23:12:08.497 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f8765e6e00 2024-09-11T23:12:08.497 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: DB pointer 0x55f8766e2000 2024-09-11T23:12:08.497 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-11T23:12:08.498 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-11T23:12:08.498 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** DB Stats ** 2024-09-11T23:12:08.498 INFO:journalctl@ceph.mon.a.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T23:12:08.498 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T23:12:08.498 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T23:12:08.498 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T23:12:08.498 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T23:12:08.498 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T23:12:08.498 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T23:12:08.498 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-11T23:12:08.498 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-11T23:12:08.498 INFO:journalctl@ceph.mon.a.smithi043.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T23:12:08.498 INFO:journalctl@ceph.mon.a.smithi043.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-11T23:12:08.498 INFO:journalctl@ceph.mon.a.smithi043.stdout: L0 2/0 101.22 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 137.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T23:12:08.499 INFO:journalctl@ceph.mon.a.smithi043.stdout: Sum 2/0 101.22 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 137.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T23:12:08.499 INFO:journalctl@ceph.mon.a.smithi043.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 137.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T23:12:08.499 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-11T23:12:08.499 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-11T23:12:08.499 INFO:journalctl@ceph.mon.a.smithi043.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T23:12:08.499 INFO:journalctl@ceph.mon.a.smithi043.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T23:12:08.499 INFO:journalctl@ceph.mon.a.smithi043.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 137.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T23:12:08.499 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-11T23:12:08.499 INFO:journalctl@ceph.mon.a.smithi043.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-11T23:12:08.499 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-11T23:12:08.499 INFO:journalctl@ceph.mon.a.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T23:12:08.499 INFO:journalctl@ceph.mon.a.smithi043.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T23:12:08.499 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T23:12:08.499 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T23:12:08.499 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T23:12:08.500 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T23:12:08.500 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative compaction: 0.00 GB write, 29.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T23:12:08.500 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval compaction: 0.00 GB write, 29.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T23:12:08.500 INFO:journalctl@ceph.mon.a.smithi043.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T23:12:08.500 INFO:journalctl@ceph.mon.a.smithi043.stdout: Block cache BinnedLRUCache@0x55f8765b91f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-11T23:12:08.500 INFO:journalctl@ceph.mon.a.smithi043.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-11T23:12:08.500 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-11T23:12:08.500 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T23:12:08.500 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: starting mon.a rank 0 at public addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] at bind addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:12:08.500 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: mon.a@-1(???) e1 preinit fsid 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:12:08.500 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: mon.a@-1(???).mds e1 new map 2024-09-11T23:12:08.500 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: mon.a@-1(???).mds e1 print_map 2024-09-11T23:12:08.501 INFO:journalctl@ceph.mon.a.smithi043.stdout: e1 2024-09-11T23:12:08.501 INFO:journalctl@ceph.mon.a.smithi043.stdout: btime 2024-09-11T23:12:03:504233+0000 2024-09-11T23:12:08.501 INFO:journalctl@ceph.mon.a.smithi043.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T23:12:08.501 INFO:journalctl@ceph.mon.a.smithi043.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T23:12:08.501 INFO:journalctl@ceph.mon.a.smithi043.stdout: legacy client fscid: -1 2024-09-11T23:12:08.501 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-11T23:12:08.501 INFO:journalctl@ceph.mon.a.smithi043.stdout: No filesystems configured 2024-09-11T23:12:08.501 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T23:12:08.501 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T23:12:08.501 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T23:12:08.501 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T23:12:08.501 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-11T23:12:08.501 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 bash[30931]: 176c0fba3df58a2066e3348fbb3d216f502a92f6be69c391569422176a7baee5 2024-09-11T23:12:08.501 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 systemd[1]: Started Ceph mon.a for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:12:08.502 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T23:12:08.502 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: monmap epoch 1 2024-09-11T23:12:08.502 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: fsid 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:12:08.502 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: last_changed 2024-09-11T23:12:00.389048+0000 2024-09-11T23:12:08.502 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: created 2024-09-11T23:12:00.389048+0000 2024-09-11T23:12:08.502 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: min_mon_release 19 (squid) 2024-09-11T23:12:08.502 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: election_strategy: 1 2024-09-11T23:12:08.502 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-11T23:12:08.502 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: fsmap 2024-09-11T23:12:08.502 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T23:12:08.502 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:08 smithi043 ceph-mon[30950]: mgrmap e1: no daemons active 2024-09-11T23:12:09.505 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.873+0000 7f1d219d3640 1 Processor -- start 2024-09-11T23:12:09.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.873+0000 7f1d219d3640 1 -- start start 2024-09-11T23:12:09.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.874+0000 7f1d219d3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d1c108360 0x7f1d1c10a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:09.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.874+0000 7f1d219d3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c07be60 con 0x7f1d1c108360 2024-09-11T23:12:09.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.874+0000 7f1d1affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d1c108360 0x7f1d1c10a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:09.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.874+0000 7f1d1affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d1c108360 0x7f1d1c10a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39686/0 (socket says 172.21.15.43:39686) 2024-09-11T23:12:09.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.874+0000 7f1d1affd640 1 -- 172.21.15.43:0/2002356728 learned_addr learned my addr 172.21.15.43:0/2002356728 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:09.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.874+0000 7f1d1affd640 1 -- 172.21.15.43:0/2002356728 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1c10ac90 con 0x7f1d1c108360 2024-09-11T23:12:09.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.876+0000 7f1d1affd640 1 --2- 172.21.15.43:0/2002356728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d1c108360 0x7f1d1c10a750 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1d14009e80 tx=0x7f1d1402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b664fe4467669e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:09.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.876+0000 7f1d19ffb640 1 -- 172.21.15.43:0/2002356728 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d14036070 con 0x7f1d1c108360 2024-09-11T23:12:09.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.876+0000 7f1d19ffb640 1 -- 172.21.15.43:0/2002356728 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1d140318c0 con 0x7f1d1c108360 2024-09-11T23:12:09.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.876+0000 7f1d19ffb640 1 -- 172.21.15.43:0/2002356728 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d14007ab0 con 0x7f1d1c108360 2024-09-11T23:12:09.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.877+0000 7f1d219d3640 1 -- 172.21.15.43:0/2002356728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d1c108360 msgr2=0x7f1d1c10a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:09.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.877+0000 7f1d219d3640 1 --2- 172.21.15.43:0/2002356728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d1c108360 0x7f1d1c10a750 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1d14009e80 tx=0x7f1d1402a7a0 comp rx=0 tx=0).stop 2024-09-11T23:12:09.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.877+0000 7f1d219d3640 1 -- 172.21.15.43:0/2002356728 shutdown_connections 2024-09-11T23:12:09.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.877+0000 7f1d219d3640 1 --2- 172.21.15.43:0/2002356728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d1c108360 0x7f1d1c10a750 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:09.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.877+0000 7f1d219d3640 1 -- 172.21.15.43:0/2002356728 >> 172.21.15.43:0/2002356728 conn(0x7f1d1c101f20 msgr2=0x7f1d1c104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:09.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.877+0000 7f1d219d3640 1 -- 172.21.15.43:0/2002356728 shutdown_connections 2024-09-11T23:12:09.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.877+0000 7f1d219d3640 1 -- 172.21.15.43:0/2002356728 wait complete. 2024-09-11T23:12:09.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.877+0000 7f1d219d3640 1 Processor -- start 2024-09-11T23:12:09.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.878+0000 7f1d219d3640 1 -- start start 2024-09-11T23:12:09.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.878+0000 7f1d219d3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d1c108360 0x7f1d1c107be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:09.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.878+0000 7f1d219d3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c10bcd0 con 0x7f1d1c108360 2024-09-11T23:12:09.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.878+0000 7f1d1affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d1c108360 0x7f1d1c107be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:09.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.878+0000 7f1d1affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d1c108360 0x7f1d1c107be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39696/0 (socket says 172.21.15.43:39696) 2024-09-11T23:12:09.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.878+0000 7f1d1affd640 1 -- 172.21.15.43:0/211788850 learned_addr learned my addr 172.21.15.43:0/211788850 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:09.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.878+0000 7f1d1affd640 1 -- 172.21.15.43:0/211788850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1c108120 con 0x7f1d1c108360 2024-09-11T23:12:09.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.878+0000 7f1d1affd640 1 --2- 172.21.15.43:0/211788850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d1c108360 0x7f1d1c107be0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1d1402acc0 tx=0x7f1d14031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:09.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.878+0000 7f1d209d1640 1 -- 172.21.15.43:0/211788850 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d14036050 con 0x7f1d1c108360 2024-09-11T23:12:09.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.879+0000 7f1d209d1640 1 -- 172.21.15.43:0/211788850 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1d14037040 con 0x7f1d1c108360 2024-09-11T23:12:09.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.879+0000 7f1d219d3640 1 -- 172.21.15.43:0/211788850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d1c106310 con 0x7f1d1c108360 2024-09-11T23:12:09.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.879+0000 7f1d209d1640 1 -- 172.21.15.43:0/211788850 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d1403d850 con 0x7f1d1c108360 2024-09-11T23:12:09.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.879+0000 7f1d219d3640 1 -- 172.21.15.43:0/211788850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d1c106850 con 0x7f1d1c108360 2024-09-11T23:12:09.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.879+0000 7f1d209d1640 1 -- 172.21.15.43:0/211788850 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1d14044020 con 0x7f1d1c108360 2024-09-11T23:12:09.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.879+0000 7f1d209d1640 1 -- 172.21.15.43:0/211788850 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1d14042020 con 0x7f1d1c108360 2024-09-11T23:12:09.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.880+0000 7f1d219d3640 1 -- 172.21.15.43:0/211788850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d1c1087c0 con 0x7f1d1c108360 2024-09-11T23:12:09.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.882+0000 7f1d209d1640 1 -- 172.21.15.43:0/211788850 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1d1404b020 con 0x7f1d1c108360 2024-09-11T23:12:09.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.941+0000 7f1d219d3640 1 -- 172.21.15.43:0/211788850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f1d1c106b50 con 0x7f1d1c108360 2024-09-11T23:12:09.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.943+0000 7f1d209d1640 1 -- 172.21.15.43:0/211788850 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f1d1403bb60 con 0x7f1d1c108360 2024-09-11T23:12:09.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.943+0000 7f1d219d3640 1 -- 172.21.15.43:0/211788850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d1c108360 msgr2=0x7f1d1c107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:09.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.943+0000 7f1d219d3640 1 --2- 172.21.15.43:0/211788850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d1c108360 0x7f1d1c107be0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1d1402acc0 tx=0x7f1d14031ca0 comp rx=0 tx=0).stop 2024-09-11T23:12:09.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.943+0000 7f1d219d3640 1 -- 172.21.15.43:0/211788850 shutdown_connections 2024-09-11T23:12:09.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.943+0000 7f1d219d3640 1 --2- 172.21.15.43:0/211788850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d1c108360 0x7f1d1c107be0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:09.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.943+0000 7f1d219d3640 1 -- 172.21.15.43:0/211788850 >> 172.21.15.43:0/211788850 conn(0x7f1d1c101f20 msgr2=0x7f1d1c102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:09.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.944+0000 7f1d219d3640 1 -- 172.21.15.43:0/211788850 shutdown_connections 2024-09-11T23:12:09.516 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:08.944+0000 7f1d219d3640 1 -- 172.21.15.43:0/211788850 wait complete. 2024-09-11T23:12:09.516 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-11T23:12:09.535 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:12:09.536 INFO:teuthology.orchestra.run.smithi043.stdout:Creating mgr... 2024-09-11T23:12:09.536 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-11T23:12:09.536 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-11T23:12:09.800 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mgr.x 2024-09-11T23:12:09.800 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to reset failed state of unit ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-11T23:12:09.974 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-25c0cd84-7093-11ef-bcea-c7b262605968.target.wants/ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-25c0cd84-7093-11ef-bcea-c7b262605968@.service. 2024-09-11T23:12:10.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:09 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/211788850' entity='client.admin' 2024-09-11T23:12:10.111 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:09 smithi043 systemd[1]: Starting Ceph mgr.x for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:12:10.498 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:10 smithi043 podman[31163]: 2024-09-11 23:12:10.204771222 +0000 UTC m=+0.112309777 container create 9dc9cd17074c4580028d6d23dac45d73c2f88368b73dea84a9d79b8ec200e462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:12:10.498 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:10 smithi043 podman[31163]: 2024-09-11 23:12:10.108374323 +0000 UTC m=+0.015912842 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T23:12:10.498 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:10 smithi043 podman[31163]: 2024-09-11 23:12:10.415232715 +0000 UTC m=+0.322771224 container init 9dc9cd17074c4580028d6d23dac45d73c2f88368b73dea84a9d79b8ec200e462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-11T23:12:10.498 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:10 smithi043 podman[31163]: 2024-09-11 23:12:10.417980979 +0000 UTC m=+0.325519489 container start 9dc9cd17074c4580028d6d23dac45d73c2f88368b73dea84a9d79b8ec200e462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:12:10.498 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:10 smithi043 bash[31163]: 9dc9cd17074c4580028d6d23dac45d73c2f88368b73dea84a9d79b8ec200e462 2024-09-11T23:12:10.514 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T23:12:10.514 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-09-11T23:12:10.523 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T23:12:10.523 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-11T23:12:10.523 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-09-11T23:12:10.523 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T23:12:10.534 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T23:12:10.534 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-09-11T23:12:10.543 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T23:12:10.543 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-11T23:12:10.543 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-09-11T23:12:10.543 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-11T23:12:10.543 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr to start... 2024-09-11T23:12:10.543 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr... 2024-09-11T23:12:10.775 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:10 smithi043 systemd[1]: Started Ceph mgr.x for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:12:10.775 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:10 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:10.549+0000 7fa3f85d3040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:12:10.775 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:10 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:10.616+0000 7fa3f85d3040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:12:10.775 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:10 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:10.773+0000 7fa3f85d3040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:12:11.026 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:10 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:10.848+0000 7fa3f85d3040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:12:11.026 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:11 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:11.004+0000 7fa3f85d3040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:12:11.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:11 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1125095972' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T23:12:11.361 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:11 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:11.161+0000 7fa3f85d3040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:12:11.737 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:11 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:11.497+0000 7fa3f85d3040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:12:11.738 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:11 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:11.560+0000 7fa3f85d3040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:12:11.741 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T23:12:11.751 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:12:11.751 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "25c0cd84-7093-11ef-bcea-c7b262605968", 2024-09-11T23:12:11.751 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T23:12:11.751 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T23:12:11.751 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T23:12:11.752 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T23:12:11.752 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:12:11.752 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T23:12:11.752 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T23:12:11.752 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-09-11T23:12:11.752 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:12:11.752 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T23:12:11.752 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T23:12:11.752 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:12:11.752 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-11T23:12:11.752 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T23:12:11.752 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:12:11.752 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T23:12:11.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T23:12:11.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:12:11.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T23:12:11.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:12:11.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T23:12:11.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T23:12:11.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T23:12:11.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T23:12:11.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T23:12:11.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T23:12:11.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:12:11.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T23:12:11.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T23:12:11.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T23:12:11.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T23:12:11.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T23:12:11.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T23:12:11.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T23:12:11.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T23:12:11.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T23:12:11.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:12:11.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T23:12:11.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:12:11.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T23:12:03:504233+0000", 2024-09-11T23:12:11.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T23:12:11.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T23:12:11.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:12:11.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T23:12:11.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T23:12:11.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T23:12:11.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T23:12:11.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T23:12:11.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T23:12:11.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T23:12:11.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:12:11.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:12:11.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:12:11.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T23:12:11.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:12:11.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T23:12:03.504925+0000", 2024-09-11T23:12:11.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:12:11.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:12:11.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T23:12:11.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:12:11.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.013+0000 7f5cf99e7640 1 Processor -- start 2024-09-11T23:12:11.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.013+0000 7f5cf99e7640 1 -- start start 2024-09-11T23:12:11.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.013+0000 7f5cf99e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cf4108400 0x7f5cf410a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:11.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.013+0000 7f5cf99e7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf407bdc0 con 0x7f5cf4108400 2024-09-11T23:12:11.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.014+0000 7f5cf2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cf4108400 0x7f5cf410a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:11.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.014+0000 7f5cf2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cf4108400 0x7f5cf410a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56084/0 (socket says 172.21.15.43:56084) 2024-09-11T23:12:11.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.014+0000 7f5cf2ffd640 1 -- 172.21.15.43:0/1855378378 learned_addr learned my addr 172.21.15.43:0/1855378378 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:11.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.014+0000 7f5cf2ffd640 1 -- 172.21.15.43:0/1855378378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf410ad30 con 0x7f5cf4108400 2024-09-11T23:12:11.757 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.015+0000 7f5cf2ffd640 1 --2- 172.21.15.43:0/1855378378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cf4108400 0x7f5cf410a7f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009820 tx=0x7f5ce402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329f7b20be1c31c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:11.757 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.015+0000 7f5cf1ffb640 1 -- 172.21.15.43:0/1855378378 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ce4037070 con 0x7f5cf4108400 2024-09-11T23:12:11.757 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.015+0000 7f5cf1ffb640 1 -- 172.21.15.43:0/1855378378 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5ce4032900 con 0x7f5cf4108400 2024-09-11T23:12:11.757 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.015+0000 7f5cf1ffb640 1 -- 172.21.15.43:0/1855378378 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ce4030850 con 0x7f5cf4108400 2024-09-11T23:12:11.757 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.016+0000 7f5cf99e7640 1 -- 172.21.15.43:0/1855378378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cf4108400 msgr2=0x7f5cf410a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:11.757 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.016+0000 7f5cf99e7640 1 --2- 172.21.15.43:0/1855378378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cf4108400 0x7f5cf410a7f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009820 tx=0x7f5ce402a270 comp rx=0 tx=0).stop 2024-09-11T23:12:11.757 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.016+0000 7f5cf99e7640 1 -- 172.21.15.43:0/1855378378 shutdown_connections 2024-09-11T23:12:11.757 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.016+0000 7f5cf99e7640 1 --2- 172.21.15.43:0/1855378378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cf4108400 0x7f5cf410a7f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:11.757 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.016+0000 7f5cf99e7640 1 -- 172.21.15.43:0/1855378378 >> 172.21.15.43:0/1855378378 conn(0x7f5cf4101f20 msgr2=0x7f5cf4104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:11.758 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.016+0000 7f5cf99e7640 1 -- 172.21.15.43:0/1855378378 shutdown_connections 2024-09-11T23:12:11.758 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.016+0000 7f5cf99e7640 1 -- 172.21.15.43:0/1855378378 wait complete. 2024-09-11T23:12:11.758 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.016+0000 7f5cf99e7640 1 Processor -- start 2024-09-11T23:12:11.758 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.016+0000 7f5cf99e7640 1 -- start start 2024-09-11T23:12:11.758 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.017+0000 7f5cf99e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cf4108400 0x7f5cf419b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:11.758 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.017+0000 7f5cf99e7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf410bcb0 con 0x7f5cf4108400 2024-09-11T23:12:11.758 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.017+0000 7f5cf2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cf4108400 0x7f5cf419b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:11.758 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.017+0000 7f5cf2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cf4108400 0x7f5cf419b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56100/0 (socket says 172.21.15.43:56100) 2024-09-11T23:12:11.758 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.017+0000 7f5cf2ffd640 1 -- 172.21.15.43:0/1125095972 learned_addr learned my addr 172.21.15.43:0/1125095972 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:11.758 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.017+0000 7f5cf2ffd640 1 -- 172.21.15.43:0/1125095972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf419bdf0 con 0x7f5cf4108400 2024-09-11T23:12:11.758 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.017+0000 7f5cf2ffd640 1 --2- 172.21.15.43:0/1125095972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cf4108400 0x7f5cf419b8b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40097f0 tx=0x7f5ce4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:11.758 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.017+0000 7f5cf89e5640 1 -- 172.21.15.43:0/1125095972 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ce4037040 con 0x7f5cf4108400 2024-09-11T23:12:11.759 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.017+0000 7f5cf89e5640 1 -- 172.21.15.43:0/1125095972 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5ce4040070 con 0x7f5cf4108400 2024-09-11T23:12:11.759 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.017+0000 7f5cf99e7640 1 -- 172.21.15.43:0/1125095972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf419c080 con 0x7f5cf4108400 2024-09-11T23:12:11.759 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.017+0000 7f5cf89e5640 1 -- 172.21.15.43:0/1125095972 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ce403a390 con 0x7f5cf4108400 2024-09-11T23:12:11.759 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.018+0000 7f5cf99e7640 1 -- 172.21.15.43:0/1125095972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf419ed70 con 0x7f5cf4108400 2024-09-11T23:12:11.759 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.018+0000 7f5cf89e5640 1 -- 172.21.15.43:0/1125095972 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5ce403a5e0 con 0x7f5cf4108400 2024-09-11T23:12:11.759 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.018+0000 7f5cf89e5640 1 -- 172.21.15.43:0/1125095972 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5ce4045950 con 0x7f5cf4108400 2024-09-11T23:12:11.760 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.019+0000 7f5cf99e7640 1 -- 172.21.15.43:0/1125095972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf4108860 con 0x7f5cf4108400 2024-09-11T23:12:11.760 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.021+0000 7f5cf89e5640 1 -- 172.21.15.43:0/1125095972 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f5ce4045460 con 0x7f5cf4108400 2024-09-11T23:12:11.760 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.084+0000 7f5cf99e7640 1 -- 172.21.15.43:0/1125095972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5cf419ef40 con 0x7f5cf4108400 2024-09-11T23:12:11.760 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.084+0000 7f5cf89e5640 1 -- 172.21.15.43:0/1125095972 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f5ce403b430 con 0x7f5cf4108400 2024-09-11T23:12:11.760 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.085+0000 7f5cf99e7640 1 -- 172.21.15.43:0/1125095972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cf4108400 msgr2=0x7f5cf419b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:11.760 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.085+0000 7f5cf99e7640 1 --2- 172.21.15.43:0/1125095972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cf4108400 0x7f5cf419b8b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40097f0 tx=0x7f5ce4030d80 comp rx=0 tx=0).stop 2024-09-11T23:12:11.760 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.085+0000 7f5cf99e7640 1 -- 172.21.15.43:0/1125095972 shutdown_connections 2024-09-11T23:12:11.760 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.085+0000 7f5cf99e7640 1 --2- 172.21.15.43:0/1125095972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cf4108400 0x7f5cf419b8b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:11.760 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.085+0000 7f5cf99e7640 1 -- 172.21.15.43:0/1125095972 >> 172.21.15.43:0/1125095972 conn(0x7f5cf4101f20 msgr2=0x7f5cf4102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:11.760 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.085+0000 7f5cf99e7640 1 -- 172.21.15.43:0/1125095972 shutdown_connections 2024-09-11T23:12:11.760 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:11.085+0000 7f5cf99e7640 1 -- 172.21.15.43:0/1125095972 wait complete. 2024-09-11T23:12:11.760 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (1/15)... 2024-09-11T23:12:12.110 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:11 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:11.756+0000 7fa3f85d3040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:12:12.963 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:12 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:12.677+0000 7fa3f85d3040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:12:12.964 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:12 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:12.747+0000 7fa3f85d3040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:12:13.793 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:13 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:13.501+0000 7fa3f85d3040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:12:13.794 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:13 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:13.641+0000 7fa3f85d3040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:12:14.048 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:13 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:13.792+0000 7fa3f85d3040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:12:14.048 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:13 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:13.856+0000 7fa3f85d3040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:12:14.048 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:14 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:14.016+0000 7fa3f85d3040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:12:14.327 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:14 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3980991744' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T23:12:14.327 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:14 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:14.088+0000 7fa3f85d3040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:12:14.327 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:14 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:14.154+0000 7fa3f85d3040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:12:14.327 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:14 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:14.325+0000 7fa3f85d3040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:12:14.610 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:14 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:14.394+0000 7fa3f85d3040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:12:14.610 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:14 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:14.464+0000 7fa3f85d3040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:12:14.610 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:14 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:14.550+0000 7fa3f85d3040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:12:14.969 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:14 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:14.613+0000 7fa3f85d3040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:12:14.969 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:14 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:14.675+0000 7fa3f85d3040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:12:14.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T23:12:14.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:12:14.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "25c0cd84-7093-11ef-bcea-c7b262605968", 2024-09-11T23:12:14.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T23:12:14.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T23:12:14.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T23:12:14.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T23:12:14.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:12:14.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T23:12:14.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T23:12:14.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-09-11T23:12:14.974 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:12:14.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T23:12:14.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T23:12:14.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:12:14.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-11T23:12:14.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T23:12:14.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:12:14.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T23:12:14.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T23:12:14.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:12:14.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T23:12:14.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:12:14.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T23:12:14.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T23:12:14.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T23:12:14.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T23:12:14.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T23:12:14.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T23:12:14.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:12:14.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T23:12:14.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T23:12:14.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T23:12:14.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T23:12:14.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T23:12:14.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T23:12:14.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T23:12:14.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T23:12:14.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T23:12:14.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:12:14.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T23:12:14.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:12:14.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T23:12:03:504233+0000", 2024-09-11T23:12:14.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T23:12:14.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T23:12:14.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:12:14.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T23:12:14.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T23:12:14.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T23:12:14.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T23:12:14.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T23:12:14.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T23:12:14.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T23:12:14.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:12:14.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:12:14.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:12:14.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T23:12:14.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:12:14.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T23:12:03.504925+0000", 2024-09-11T23:12:14.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:12:14.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:12:14.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T23:12:14.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:12:14.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.127+0000 7ff6e6d24640 1 Processor -- start 2024-09-11T23:12:14.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.127+0000 7ff6e6d24640 1 -- start start 2024-09-11T23:12:14.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.127+0000 7ff6e6d24640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e01067b0 0x7ff6e0106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:14.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.127+0000 7ff6e6d24640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e0107180 con 0x7ff6e01067b0 2024-09-11T23:12:14.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.127+0000 7ff6e4ac0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e01067b0 0x7ff6e0106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:14.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.127+0000 7ff6e4ac0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e01067b0 0x7ff6e0106bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56112/0 (socket says 172.21.15.43:56112) 2024-09-11T23:12:14.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.127+0000 7ff6e4ac0640 1 -- 172.21.15.43:0/3766592558 learned_addr learned my addr 172.21.15.43:0/3766592558 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:14.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.127+0000 7ff6e4ac0640 1 -- 172.21.15.43:0/3766592558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e01079b0 con 0x7ff6e01067b0 2024-09-11T23:12:14.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.128+0000 7ff6e4ac0640 1 --2- 172.21.15.43:0/3766592558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e01067b0 0x7ff6e0106bb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009820 tx=0x7ff6d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58da591d6ab93cc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:14.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.128+0000 7ff6df7fe640 1 -- 172.21.15.43:0/3766592558 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6d4037070 con 0x7ff6e01067b0 2024-09-11T23:12:14.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.128+0000 7ff6df7fe640 1 -- 172.21.15.43:0/3766592558 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff6d4032900 con 0x7ff6e01067b0 2024-09-11T23:12:14.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.128+0000 7ff6df7fe640 1 -- 172.21.15.43:0/3766592558 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6d4030850 con 0x7ff6e01067b0 2024-09-11T23:12:14.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.129+0000 7ff6e6d24640 1 -- 172.21.15.43:0/3766592558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e01067b0 msgr2=0x7ff6e0106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:14.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.129+0000 7ff6e6d24640 1 --2- 172.21.15.43:0/3766592558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e01067b0 0x7ff6e0106bb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009820 tx=0x7ff6d402a270 comp rx=0 tx=0).stop 2024-09-11T23:12:14.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.129+0000 7ff6e6d24640 1 -- 172.21.15.43:0/3766592558 shutdown_connections 2024-09-11T23:12:14.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.129+0000 7ff6e6d24640 1 --2- 172.21.15.43:0/3766592558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e01067b0 0x7ff6e0106bb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:14.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.129+0000 7ff6e6d24640 1 -- 172.21.15.43:0/3766592558 >> 172.21.15.43:0/3766592558 conn(0x7ff6e0101f20 msgr2=0x7ff6e0104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:14.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.129+0000 7ff6e6d24640 1 -- 172.21.15.43:0/3766592558 shutdown_connections 2024-09-11T23:12:14.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.129+0000 7ff6e6d24640 1 -- 172.21.15.43:0/3766592558 wait complete. 2024-09-11T23:12:14.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.130+0000 7ff6e6d24640 1 Processor -- start 2024-09-11T23:12:14.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.130+0000 7ff6e6d24640 1 -- start start 2024-09-11T23:12:14.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.130+0000 7ff6e6d24640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e01067b0 0x7ff6e019b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:14.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.130+0000 7ff6e6d24640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e010ad60 con 0x7ff6e01067b0 2024-09-11T23:12:14.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.130+0000 7ff6e4ac0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e01067b0 0x7ff6e019b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:14.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.130+0000 7ff6e4ac0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e01067b0 0x7ff6e019b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56128/0 (socket says 172.21.15.43:56128) 2024-09-11T23:12:14.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.130+0000 7ff6e4ac0640 1 -- 172.21.15.43:0/3980991744 learned_addr learned my addr 172.21.15.43:0/3980991744 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:14.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.130+0000 7ff6e4ac0640 1 -- 172.21.15.43:0/3980991744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e019be20 con 0x7ff6e01067b0 2024-09-11T23:12:14.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.131+0000 7ff6e4ac0640 1 --2- 172.21.15.43:0/3980991744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e01067b0 0x7ff6e019b8e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009940 tx=0x7ff6d4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:14.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.131+0000 7ff6ddffb640 1 -- 172.21.15.43:0/3980991744 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6d4042070 con 0x7ff6e01067b0 2024-09-11T23:12:14.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.131+0000 7ff6ddffb640 1 -- 172.21.15.43:0/3980991744 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff6d40308c0 con 0x7ff6e01067b0 2024-09-11T23:12:14.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.131+0000 7ff6e6d24640 1 -- 172.21.15.43:0/3980991744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e019c0b0 con 0x7ff6e01067b0 2024-09-11T23:12:14.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.131+0000 7ff6ddffb640 1 -- 172.21.15.43:0/3980991744 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6d4037040 con 0x7ff6e01067b0 2024-09-11T23:12:14.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.131+0000 7ff6e6d24640 1 -- 172.21.15.43:0/3980991744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e019eda0 con 0x7ff6e01067b0 2024-09-11T23:12:14.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.131+0000 7ff6ddffb640 1 -- 172.21.15.43:0/3980991744 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff6d4045070 con 0x7ff6e01067b0 2024-09-11T23:12:14.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.131+0000 7ff6ddffb640 1 -- 172.21.15.43:0/3980991744 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff6d4047bc0 con 0x7ff6e01067b0 2024-09-11T23:12:14.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.132+0000 7ff6e6d24640 1 -- 172.21.15.43:0/3980991744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6e010aee0 con 0x7ff6e01067b0 2024-09-11T23:12:14.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.134+0000 7ff6ddffb640 1 -- 172.21.15.43:0/3980991744 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ff6d40327b0 con 0x7ff6e01067b0 2024-09-11T23:12:14.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.197+0000 7ff6e6d24640 1 -- 172.21.15.43:0/3980991744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff6e0106c30 con 0x7ff6e01067b0 2024-09-11T23:12:14.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.197+0000 7ff6ddffb640 1 -- 172.21.15.43:0/3980991744 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7ff6e0106c30 con 0x7ff6e01067b0 2024-09-11T23:12:14.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.198+0000 7ff6e6d24640 1 -- 172.21.15.43:0/3980991744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e01067b0 msgr2=0x7ff6e019b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:14.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.198+0000 7ff6e6d24640 1 --2- 172.21.15.43:0/3980991744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e01067b0 0x7ff6e019b8e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009940 tx=0x7ff6d4032ce0 comp rx=0 tx=0).stop 2024-09-11T23:12:14.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.198+0000 7ff6e6d24640 1 -- 172.21.15.43:0/3980991744 shutdown_connections 2024-09-11T23:12:14.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.198+0000 7ff6e6d24640 1 --2- 172.21.15.43:0/3980991744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e01067b0 0x7ff6e019b8e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:14.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.198+0000 7ff6e6d24640 1 -- 172.21.15.43:0/3980991744 >> 172.21.15.43:0/3980991744 conn(0x7ff6e0101f20 msgr2=0x7ff6e0102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:14.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.198+0000 7ff6e6d24640 1 -- 172.21.15.43:0/3980991744 shutdown_connections 2024-09-11T23:12:14.991 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:14.198+0000 7ff6e6d24640 1 -- 172.21.15.43:0/3980991744 wait complete. 2024-09-11T23:12:14.991 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (2/15)... 2024-09-11T23:12:15.860 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:15 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:15.504+0000 7fa3f85d3040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:12:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:15 smithi043 ceph-mon[30950]: Activating manager daemon x 2024-09-11T23:12:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:15 smithi043 ceph-mon[30950]: mgrmap e2: x(active, starting, since 0.00300671s) 2024-09-11T23:12:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:15 smithi043 ceph-mon[30950]: from='mgr.14100 172.21.15.43:0/1165351562' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:12:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:15 smithi043 ceph-mon[30950]: from='mgr.14100 172.21.15.43:0/1165351562' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:12:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:15 smithi043 ceph-mon[30950]: from='mgr.14100 172.21.15.43:0/1165351562' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:12:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:15 smithi043 ceph-mon[30950]: from='mgr.14100 172.21.15.43:0/1165351562' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:12:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:15 smithi043 ceph-mon[30950]: from='mgr.14100 172.21.15.43:0/1165351562' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:12:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:15 smithi043 ceph-mon[30950]: Manager daemon x is now available 2024-09-11T23:12:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:15 smithi043 ceph-mon[30950]: from='mgr.14100 172.21.15.43:0/1165351562' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:12:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:15 smithi043 ceph-mon[30950]: from='mgr.14100 172.21.15.43:0/1165351562' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:12:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:15 smithi043 ceph-mon[30950]: from='mgr.14100 172.21.15.43:0/1165351562' entity='mgr.x' 2024-09-11T23:12:15.862 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:15 smithi043 ceph-mon[30950]: from='mgr.14100 172.21.15.43:0/1165351562' entity='mgr.x' 2024-09-11T23:12:15.862 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:15 smithi043 ceph-mon[30950]: from='mgr.14100 172.21.15.43:0/1165351562' entity='mgr.x' 2024-09-11T23:12:17.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:17 smithi043 ceph-mon[30950]: mgrmap e3: x(active, since 1.00728s) 2024-09-11T23:12:18.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:18 smithi043 ceph-mon[30950]: mgrmap e4: x(active, since 2s) 2024-09-11T23:12:20.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:20 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/465903789' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T23:12:26.612 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T23:12:26.612 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:12:26.612 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "25c0cd84-7093-11ef-bcea-c7b262605968", 2024-09-11T23:12:26.612 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T23:12:26.612 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T23:12:26.612 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T23:12:26.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T23:12:26.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:12:26.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T23:12:26.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T23:12:26.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-09-11T23:12:26.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:12:26.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T23:12:26.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T23:12:26.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:12:26.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-09-11T23:12:26.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T23:12:26.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:12:26.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T23:12:26.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T23:12:26.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:12:26.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T23:12:26.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:12:26.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T23:12:26.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T23:12:26.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T23:12:26.623 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T23:12:26.624 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T23:12:26.624 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T23:12:26.624 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:12:26.624 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T23:12:26.624 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T23:12:26.624 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T23:12:26.624 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T23:12:26.624 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T23:12:26.624 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T23:12:26.624 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T23:12:26.624 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T23:12:26.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T23:12:26.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:12:26.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T23:12:26.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:12:26.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T23:12:03:504233+0000", 2024-09-11T23:12:26.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T23:12:26.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T23:12:26.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:12:26.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T23:12:26.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T23:12:26.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T23:12:26.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T23:12:26.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T23:12:26.625 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T23:12:26.626 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T23:12:26.626 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-11T23:12:26.626 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:12:26.626 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:12:26.626 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T23:12:26.626 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T23:12:26.626 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T23:12:03.504925+0000", 2024-09-11T23:12:26.626 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T23:12:26.626 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-11T23:12:26.627 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T23:12:26.627 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:12:26.627 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.002+0000 7f78aec9e640 1 Processor -- start 2024-09-11T23:12:26.627 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.003+0000 7f78aec9e640 1 -- start start 2024-09-11T23:12:26.627 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.003+0000 7f78aec9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a8108380 0x7f78a810a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:26.627 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.003+0000 7f78aec9e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78a807be40 con 0x7f78a8108380 2024-09-11T23:12:26.627 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.003+0000 7f78aca3a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a8108380 0x7f78a810a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:26.627 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.003+0000 7f78aca3a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a8108380 0x7f78a810a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56206/0 (socket says 172.21.15.43:56206) 2024-09-11T23:12:26.627 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.003+0000 7f78aca3a640 1 -- 172.21.15.43:0/3100193609 learned_addr learned my addr 172.21.15.43:0/3100193609 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:26.627 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.003+0000 7f78aca3a640 1 -- 172.21.15.43:0/3100193609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a810acb0 con 0x7f78a8108380 2024-09-11T23:12:26.627 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.004+0000 7f78aca3a640 1 --2- 172.21.15.43:0/3100193609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a8108380 0x7f78a810a770 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7898009820 tx=0x7f789802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546eb32d8db170db server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:26.627 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.004+0000 7f78a77fe640 1 -- 172.21.15.43:0/3100193609 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7898037070 con 0x7f78a8108380 2024-09-11T23:12:26.627 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.004+0000 7f78a77fe640 1 -- 172.21.15.43:0/3100193609 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7898032900 con 0x7f78a8108380 2024-09-11T23:12:26.627 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.004+0000 7f78a77fe640 1 -- 172.21.15.43:0/3100193609 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7898030850 con 0x7f78a8108380 2024-09-11T23:12:26.628 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.005+0000 7f78aec9e640 1 -- 172.21.15.43:0/3100193609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a8108380 msgr2=0x7f78a810a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:26.628 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.005+0000 7f78aec9e640 1 --2- 172.21.15.43:0/3100193609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a8108380 0x7f78a810a770 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7898009820 tx=0x7f789802a270 comp rx=0 tx=0).stop 2024-09-11T23:12:26.628 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.005+0000 7f78aec9e640 1 -- 172.21.15.43:0/3100193609 shutdown_connections 2024-09-11T23:12:26.628 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.005+0000 7f78aec9e640 1 --2- 172.21.15.43:0/3100193609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a8108380 0x7f78a810a770 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:26.628 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.005+0000 7f78aec9e640 1 -- 172.21.15.43:0/3100193609 >> 172.21.15.43:0/3100193609 conn(0x7f78a8101f20 msgr2=0x7f78a8104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:26.628 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.005+0000 7f78aec9e640 1 -- 172.21.15.43:0/3100193609 shutdown_connections 2024-09-11T23:12:26.628 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.005+0000 7f78aec9e640 1 -- 172.21.15.43:0/3100193609 wait complete. 2024-09-11T23:12:26.628 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.006+0000 7f78aec9e640 1 Processor -- start 2024-09-11T23:12:26.628 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.006+0000 7f78aec9e640 1 -- start start 2024-09-11T23:12:26.628 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.006+0000 7f78aec9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a8108380 0x7f78a819b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:26.628 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.006+0000 7f78aec9e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78a810bcf0 con 0x7f78a8108380 2024-09-11T23:12:26.628 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.006+0000 7f78aca3a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a8108380 0x7f78a819b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:26.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.006+0000 7f78aca3a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a8108380 0x7f78a819b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56218/0 (socket says 172.21.15.43:56218) 2024-09-11T23:12:26.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.006+0000 7f78aca3a640 1 -- 172.21.15.43:0/465903789 learned_addr learned my addr 172.21.15.43:0/465903789 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:26.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.006+0000 7f78aca3a640 1 -- 172.21.15.43:0/465903789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a819bde0 con 0x7f78a8108380 2024-09-11T23:12:26.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.006+0000 7f78aca3a640 1 --2- 172.21.15.43:0/465903789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a8108380 0x7f78a819b8a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7898032ce0 tx=0x7f7898032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:26.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.007+0000 7f78a5ffb640 1 -- 172.21.15.43:0/465903789 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7898042070 con 0x7f78a8108380 2024-09-11T23:12:26.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.007+0000 7f78a5ffb640 1 -- 172.21.15.43:0/465903789 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f78980308c0 con 0x7f78a8108380 2024-09-11T23:12:26.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.007+0000 7f78aec9e640 1 -- 172.21.15.43:0/465903789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a819c070 con 0x7f78a8108380 2024-09-11T23:12:26.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.007+0000 7f78a5ffb640 1 -- 172.21.15.43:0/465903789 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7898037040 con 0x7f78a8108380 2024-09-11T23:12:26.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.007+0000 7f78aec9e640 1 -- 172.21.15.43:0/465903789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a819ed60 con 0x7f78a8108380 2024-09-11T23:12:26.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.008+0000 7f78a5ffb640 1 -- 172.21.15.43:0/465903789 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f7898045070 con 0x7f78a8108380 2024-09-11T23:12:26.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.008+0000 7f78a5ffb640 1 --2- 172.21.15.43:0/465903789 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f7890038f70 0x7f789003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:26.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.008+0000 7f78aec9e640 1 -- 172.21.15.43:0/465903789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a81087e0 con 0x7f78a8108380 2024-09-11T23:12:26.629 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.008+0000 7f78a5ffb640 1 -- 172.21.15.43:0/465903789 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f789806bfb0 con 0x7f78a8108380 2024-09-11T23:12:26.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.008+0000 7f78a7fff640 1 --2- 172.21.15.43:0/465903789 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f7890038f70 0x7f789003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:26.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.009+0000 7f78a7fff640 1 --2- 172.21.15.43:0/465903789 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f7890038f70 0x7f789003b430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f789c0098c0 tx=0x7f789c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:26.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.015+0000 7f78a5ffb640 1 -- 172.21.15.43:0/465903789 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f789806b9d0 con 0x7f78a8108380 2024-09-11T23:12:26.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.258+0000 7f78aec9e640 1 -- 172.21.15.43:0/465903789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f78a819ef30 con 0x7f78a8108380 2024-09-11T23:12:26.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.259+0000 7f78a5ffb640 1 -- 172.21.15.43:0/465903789 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f78a819ef30 con 0x7f78a8108380 2024-09-11T23:12:26.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.261+0000 7f78aec9e640 1 -- 172.21.15.43:0/465903789 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f7890038f70 msgr2=0x7f789003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:26.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.261+0000 7f78aec9e640 1 --2- 172.21.15.43:0/465903789 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f7890038f70 0x7f789003b430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f789c0098c0 tx=0x7f789c006db0 comp rx=0 tx=0).stop 2024-09-11T23:12:26.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.261+0000 7f78aec9e640 1 -- 172.21.15.43:0/465903789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a8108380 msgr2=0x7f78a819b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:26.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.261+0000 7f78aec9e640 1 --2- 172.21.15.43:0/465903789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a8108380 0x7f78a819b8a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7898032ce0 tx=0x7f7898032d10 comp rx=0 tx=0).stop 2024-09-11T23:12:26.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.261+0000 7f78aec9e640 1 -- 172.21.15.43:0/465903789 shutdown_connections 2024-09-11T23:12:26.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.261+0000 7f78aec9e640 1 --2- 172.21.15.43:0/465903789 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f7890038f70 0x7f789003b430 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:26.630 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.261+0000 7f78aec9e640 1 --2- 172.21.15.43:0/465903789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78a8108380 0x7f78a819b8a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:26.631 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.261+0000 7f78aec9e640 1 -- 172.21.15.43:0/465903789 >> 172.21.15.43:0/465903789 conn(0x7f78a8101f20 msgr2=0x7f78a8102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:26.631 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.261+0000 7f78aec9e640 1 -- 172.21.15.43:0/465903789 shutdown_connections 2024-09-11T23:12:26.631 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:20.261+0000 7f78aec9e640 1 -- 172.21.15.43:0/465903789 wait complete. 2024-09-11T23:12:26.631 INFO:teuthology.orchestra.run.smithi043.stdout:mgr is available 2024-09-11T23:12:29.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:29 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/485998400' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T23:12:29.928 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T23:12:29.928 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T23:12:29.928 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout fsid = 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:12:29.928 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-09-11T23:12:29.928 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T23:12:29.928 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T23:12:29.928 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T23:12:29.928 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T23:12:29.928 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T23:12:29.929 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T23:12:29.929 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T23:12:29.929 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-11T23:12:29.929 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T23:12:29.929 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-09-11T23:12:29.929 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-11T23:12:29.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-11T23:12:29.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-11T23:12:29.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T23:12:29.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T23:12:29.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.147+0000 7f23132ca640 1 Processor -- start 2024-09-11T23:12:29.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.147+0000 7f23132ca640 1 -- start start 2024-09-11T23:12:29.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.147+0000 7f23132ca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f230c108430 0x7f230c10a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:29.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.147+0000 7f23132ca640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f230c07bdf0 con 0x7f230c108430 2024-09-11T23:12:29.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.148+0000 7f2311066640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f230c108430 0x7f230c10a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:29.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.148+0000 7f2311066640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f230c108430 0x7f230c10a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43490/0 (socket says 172.21.15.43:43490) 2024-09-11T23:12:29.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.148+0000 7f2311066640 1 -- 172.21.15.43:0/686934154 learned_addr learned my addr 172.21.15.43:0/686934154 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:29.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.148+0000 7f2311066640 1 -- 172.21.15.43:0/686934154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f230c10ad60 con 0x7f230c108430 2024-09-11T23:12:29.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.149+0000 7f2311066640 1 --2- 172.21.15.43:0/686934154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f230c108430 0x7f230c10a820 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2308009820 tx=0x7f230802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d514d51dc1175fbb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:29.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.149+0000 7f2303fff640 1 -- 172.21.15.43:0/686934154 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2308037070 con 0x7f230c108430 2024-09-11T23:12:29.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.149+0000 7f2303fff640 1 -- 172.21.15.43:0/686934154 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2308032900 con 0x7f230c108430 2024-09-11T23:12:29.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.149+0000 7f2303fff640 1 -- 172.21.15.43:0/686934154 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2308030850 con 0x7f230c108430 2024-09-11T23:12:29.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.150+0000 7f23132ca640 1 -- 172.21.15.43:0/686934154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f230c108430 msgr2=0x7f230c10a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:29.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.150+0000 7f23132ca640 1 --2- 172.21.15.43:0/686934154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f230c108430 0x7f230c10a820 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2308009820 tx=0x7f230802a270 comp rx=0 tx=0).stop 2024-09-11T23:12:29.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.150+0000 7f23132ca640 1 -- 172.21.15.43:0/686934154 shutdown_connections 2024-09-11T23:12:29.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.150+0000 7f23132ca640 1 --2- 172.21.15.43:0/686934154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f230c108430 0x7f230c10a820 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:29.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.150+0000 7f23132ca640 1 -- 172.21.15.43:0/686934154 >> 172.21.15.43:0/686934154 conn(0x7f230c102010 msgr2=0x7f230c104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:29.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.150+0000 7f23132ca640 1 -- 172.21.15.43:0/686934154 shutdown_connections 2024-09-11T23:12:29.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.150+0000 7f23132ca640 1 -- 172.21.15.43:0/686934154 wait complete. 2024-09-11T23:12:29.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.151+0000 7f23132ca640 1 Processor -- start 2024-09-11T23:12:29.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.151+0000 7f23132ca640 1 -- start start 2024-09-11T23:12:29.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.151+0000 7f23132ca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f230c108430 0x7f230c19b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:29.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.151+0000 7f23132ca640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f230c10bcd0 con 0x7f230c108430 2024-09-11T23:12:29.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.151+0000 7f2311066640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f230c108430 0x7f230c19b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:29.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.151+0000 7f2311066640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f230c108430 0x7f230c19b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43504/0 (socket says 172.21.15.43:43504) 2024-09-11T23:12:29.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.151+0000 7f2311066640 1 -- 172.21.15.43:0/485998400 learned_addr learned my addr 172.21.15.43:0/485998400 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:29.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.151+0000 7f2311066640 1 -- 172.21.15.43:0/485998400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f230c19bdb0 con 0x7f230c108430 2024-09-11T23:12:29.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.152+0000 7f2311066640 1 --2- 172.21.15.43:0/485998400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f230c108430 0x7f230c19b870 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2308032ce0 tx=0x7f2308032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:29.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.152+0000 7f23027fc640 1 -- 172.21.15.43:0/485998400 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2308042070 con 0x7f230c108430 2024-09-11T23:12:29.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.152+0000 7f23027fc640 1 -- 172.21.15.43:0/485998400 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f23080308c0 con 0x7f230c108430 2024-09-11T23:12:29.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.152+0000 7f23132ca640 1 -- 172.21.15.43:0/485998400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f230c19c040 con 0x7f230c108430 2024-09-11T23:12:29.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.152+0000 7f23027fc640 1 -- 172.21.15.43:0/485998400 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2308037040 con 0x7f230c108430 2024-09-11T23:12:29.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.152+0000 7f23132ca640 1 -- 172.21.15.43:0/485998400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f230c19ed30 con 0x7f230c108430 2024-09-11T23:12:29.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.153+0000 7f23027fc640 1 -- 172.21.15.43:0/485998400 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f2308045070 con 0x7f230c108430 2024-09-11T23:12:29.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.153+0000 7f23132ca640 1 -- 172.21.15.43:0/485998400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f230c108890 con 0x7f230c108430 2024-09-11T23:12:29.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.153+0000 7f23027fc640 1 --2- 172.21.15.43:0/485998400 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f22f8038f70 0x7f22f803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:29.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.153+0000 7f23027fc640 1 -- 172.21.15.43:0/485998400 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f230806bfb0 con 0x7f230c108430 2024-09-11T23:12:29.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.153+0000 7f2310865640 1 --2- 172.21.15.43:0/485998400 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f22f8038f70 0x7f22f803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:29.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.154+0000 7f2310865640 1 --2- 172.21.15.43:0/485998400 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f22f8038f70 0x7f22f803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0098c0 tx=0x7f22fc006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:29.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.158+0000 7f23027fc640 1 -- 172.21.15.43:0/485998400 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f230806b9d0 con 0x7f230c108430 2024-09-11T23:12:29.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.333+0000 7f23132ca640 1 -- 172.21.15.43:0/485998400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f230c1a0530 con 0x7f230c108430 2024-09-11T23:12:29.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.334+0000 7f23027fc640 1 -- 172.21.15.43:0/485998400 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+454 (secure 0 0 0) 0x7f230803c740 con 0x7f230c108430 2024-09-11T23:12:29.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.336+0000 7f23132ca640 1 -- 172.21.15.43:0/485998400 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f22f8038f70 msgr2=0x7f22f803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:29.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.336+0000 7f23132ca640 1 --2- 172.21.15.43:0/485998400 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f22f8038f70 0x7f22f803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0098c0 tx=0x7f22fc006db0 comp rx=0 tx=0).stop 2024-09-11T23:12:29.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.336+0000 7f23132ca640 1 -- 172.21.15.43:0/485998400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f230c108430 msgr2=0x7f230c19b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:29.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.336+0000 7f23132ca640 1 --2- 172.21.15.43:0/485998400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f230c108430 0x7f230c19b870 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2308032ce0 tx=0x7f2308032d10 comp rx=0 tx=0).stop 2024-09-11T23:12:29.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.336+0000 7f23132ca640 1 -- 172.21.15.43:0/485998400 shutdown_connections 2024-09-11T23:12:29.935 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.336+0000 7f23132ca640 1 --2- 172.21.15.43:0/485998400 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f22f8038f70 0x7f22f803b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:29.935 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.336+0000 7f23132ca640 1 --2- 172.21.15.43:0/485998400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f230c108430 0x7f230c19b870 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:29.935 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.336+0000 7f23132ca640 1 -- 172.21.15.43:0/485998400 >> 172.21.15.43:0/485998400 conn(0x7f230c102010 msgr2=0x7f230c102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:29.935 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.337+0000 7f23132ca640 1 -- 172.21.15.43:0/485998400 shutdown_connections 2024-09-11T23:12:29.935 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:29.337+0000 7f23132ca640 1 -- 172.21.15.43:0/485998400 wait complete. 2024-09-11T23:12:29.935 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling cephadm module... 2024-09-11T23:12:30.824 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:30 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/362038478' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-11T23:12:30.825 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:30 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: ignoring --setuser ceph since I am not root 2024-09-11T23:12:30.825 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:30 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: ignoring --setgroup ceph since I am not root 2024-09-11T23:12:30.825 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:30 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:30.763+0000 7f535ec2b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:12:30.825 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:30 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:30.822+0000 7f535ec2b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:12:31.110 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:30 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:30.978+0000 7f535ec2b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:12:31.110 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:31 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:31.049+0000 7f535ec2b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:12:31.610 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:31 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:31.205+0000 7f535ec2b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:12:31.610 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:31 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:31.354+0000 7f535ec2b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:12:31.947 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:31 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/362038478' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-11T23:12:31.947 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:31 smithi043 ceph-mon[30950]: mgrmap e5: x(active, since 15s) 2024-09-11T23:12:31.947 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:31 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:31.686+0000 7f535ec2b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:12:31.947 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:31 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:31.750+0000 7f535ec2b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:12:31.947 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:31 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:31.946+0000 7f535ec2b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:12:32.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.329+0000 7f66a87e9640 1 Processor -- start 2024-09-11T23:12:32.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.329+0000 7f66a87e9640 1 -- start start 2024-09-11T23:12:32.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.329+0000 7f66a87e9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66a01083d0 0x7f66a010a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:32.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.329+0000 7f66a87e9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f66a007be50 con 0x7f66a01083d0 2024-09-11T23:12:32.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.330+0000 7f66a6585640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66a01083d0 0x7f66a010a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:32.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.330+0000 7f66a6585640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66a01083d0 0x7f66a010a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57242/0 (socket says 172.21.15.43:57242) 2024-09-11T23:12:32.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.330+0000 7f66a6585640 1 -- 172.21.15.43:0/1763004585 learned_addr learned my addr 172.21.15.43:0/1763004585 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:32.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.330+0000 7f66a6585640 1 -- 172.21.15.43:0/1763004585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a010ad00 con 0x7f66a01083d0 2024-09-11T23:12:32.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.331+0000 7f66a6585640 1 --2- 172.21.15.43:0/1763004585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66a01083d0 0x7f66a010a7c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6690009e80 tx=0x7f669002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6158855870aa60 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:32.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.331+0000 7f66a5583640 1 -- 172.21.15.43:0/1763004585 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6690036070 con 0x7f66a01083d0 2024-09-11T23:12:32.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.331+0000 7f66a5583640 1 -- 172.21.15.43:0/1763004585 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f66900318c0 con 0x7f66a01083d0 2024-09-11T23:12:32.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.331+0000 7f66a5583640 1 -- 172.21.15.43:0/1763004585 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6690007ab0 con 0x7f66a01083d0 2024-09-11T23:12:32.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.331+0000 7f66a87e9640 1 -- 172.21.15.43:0/1763004585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66a01083d0 msgr2=0x7f66a010a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:32.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.331+0000 7f66a87e9640 1 --2- 172.21.15.43:0/1763004585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66a01083d0 0x7f66a010a7c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6690009e80 tx=0x7f669002a7a0 comp rx=0 tx=0).stop 2024-09-11T23:12:32.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.331+0000 7f66a87e9640 1 -- 172.21.15.43:0/1763004585 shutdown_connections 2024-09-11T23:12:32.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.331+0000 7f66a87e9640 1 --2- 172.21.15.43:0/1763004585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66a01083d0 0x7f66a010a7c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:32.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.331+0000 7f66a87e9640 1 -- 172.21.15.43:0/1763004585 >> 172.21.15.43:0/1763004585 conn(0x7f66a0102010 msgr2=0x7f66a0104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:32.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.331+0000 7f66a87e9640 1 -- 172.21.15.43:0/1763004585 shutdown_connections 2024-09-11T23:12:32.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.332+0000 7f66a87e9640 1 -- 172.21.15.43:0/1763004585 wait complete. 2024-09-11T23:12:32.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.332+0000 7f66a87e9640 1 Processor -- start 2024-09-11T23:12:32.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.332+0000 7f66a87e9640 1 -- start start 2024-09-11T23:12:32.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.332+0000 7f66a87e9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66a01083d0 0x7f66a019b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:32.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.332+0000 7f66a87e9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f66a010bcd0 con 0x7f66a01083d0 2024-09-11T23:12:32.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.333+0000 7f66a6585640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66a01083d0 0x7f66a019b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:32.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.333+0000 7f66a6585640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66a01083d0 0x7f66a019b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57244/0 (socket says 172.21.15.43:57244) 2024-09-11T23:12:32.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.333+0000 7f66a6585640 1 -- 172.21.15.43:0/362038478 learned_addr learned my addr 172.21.15.43:0/362038478 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:32.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.333+0000 7f66a6585640 1 -- 172.21.15.43:0/362038478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a019bdb0 con 0x7f66a01083d0 2024-09-11T23:12:32.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.333+0000 7f66a6585640 1 --2- 172.21.15.43:0/362038478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66a01083d0 0x7f66a019b870 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f669002acc0 tx=0x7f6690031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:32.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.333+0000 7f669f7fe640 1 -- 172.21.15.43:0/362038478 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6690036050 con 0x7f66a01083d0 2024-09-11T23:12:32.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.333+0000 7f669f7fe640 1 -- 172.21.15.43:0/362038478 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6690037040 con 0x7f66a01083d0 2024-09-11T23:12:32.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.333+0000 7f66a87e9640 1 -- 172.21.15.43:0/362038478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a019c040 con 0x7f66a01083d0 2024-09-11T23:12:32.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.333+0000 7f669f7fe640 1 -- 172.21.15.43:0/362038478 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f669003d850 con 0x7f66a01083d0 2024-09-11T23:12:32.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.333+0000 7f66a87e9640 1 -- 172.21.15.43:0/362038478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a019ed30 con 0x7f66a01083d0 2024-09-11T23:12:32.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.334+0000 7f669f7fe640 1 -- 172.21.15.43:0/362038478 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f6690044020 con 0x7f66a01083d0 2024-09-11T23:12:32.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.334+0000 7f669f7fe640 1 --2- 172.21.15.43:0/362038478 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f6688038f20 0x7f668803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:32.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.334+0000 7f66a87e9640 1 -- 172.21.15.43:0/362038478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a0108830 con 0x7f66a01083d0 2024-09-11T23:12:32.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.334+0000 7f669f7fe640 1 -- 172.21.15.43:0/362038478 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6690042020 con 0x7f66a01083d0 2024-09-11T23:12:32.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.334+0000 7f66a5d84640 1 --2- 172.21.15.43:0/362038478 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f6688038f20 0x7f668803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:32.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.335+0000 7f66a5d84640 1 --2- 172.21.15.43:0/362038478 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f6688038f20 0x7f668803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f66940098c0 tx=0x7f6694006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:32.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.339+0000 7f669f7fe640 1 -- 172.21.15.43:0/362038478 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f669004b020 con 0x7f66a01083d0 2024-09-11T23:12:32.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.563+0000 7f66a87e9640 1 -- 172.21.15.43:0/362038478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f66a019f3b0 con 0x7f66a01083d0 2024-09-11T23:12:32.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.614+0000 7f669f7fe640 1 -- 172.21.15.43:0/362038478 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f669006c420 con 0x7f66a01083d0 2024-09-11T23:12:32.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.616+0000 7f669f7fe640 1 -- 172.21.15.43:0/362038478 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f6690037ce0 con 0x7f66a01083d0 2024-09-11T23:12:32.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.617+0000 7f66a87e9640 1 -- 172.21.15.43:0/362038478 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f6688038f20 msgr2=0x7f668803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:32.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.617+0000 7f66a87e9640 1 --2- 172.21.15.43:0/362038478 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f6688038f20 0x7f668803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f66940098c0 tx=0x7f6694006db0 comp rx=0 tx=0).stop 2024-09-11T23:12:32.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.617+0000 7f66a87e9640 1 -- 172.21.15.43:0/362038478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66a01083d0 msgr2=0x7f66a019b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:32.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.617+0000 7f66a87e9640 1 --2- 172.21.15.43:0/362038478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66a01083d0 0x7f66a019b870 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f669002acc0 tx=0x7f6690031ca0 comp rx=0 tx=0).stop 2024-09-11T23:12:32.036 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.617+0000 7f66a87e9640 1 -- 172.21.15.43:0/362038478 shutdown_connections 2024-09-11T23:12:32.036 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.617+0000 7f66a87e9640 1 --2- 172.21.15.43:0/362038478 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f6688038f20 0x7f668803b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:32.036 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.617+0000 7f66a87e9640 1 --2- 172.21.15.43:0/362038478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66a01083d0 0x7f66a019b870 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:32.036 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.617+0000 7f66a87e9640 1 -- 172.21.15.43:0/362038478 >> 172.21.15.43:0/362038478 conn(0x7f66a0102010 msgr2=0x7f66a0102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:32.036 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.617+0000 7f66a87e9640 1 -- 172.21.15.43:0/362038478 shutdown_connections 2024-09-11T23:12:32.036 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:30.617+0000 7f66a87e9640 1 -- 172.21.15.43:0/362038478 wait complete. 2024-09-11T23:12:32.977 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:32 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3855967282' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T23:12:32.977 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:32 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:32.904+0000 7f535ec2b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:12:33.263 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:32 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:32.976+0000 7f535ec2b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:12:33.271 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:12:33.271 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-11T23:12:33.271 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T23:12:33.272 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-11T23:12:33.272 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T23:12:33.272 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:12:33.272 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.457+0000 7fbcf3e30640 1 Processor -- start 2024-09-11T23:12:33.272 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.457+0000 7fbcf3e30640 1 -- start start 2024-09-11T23:12:33.272 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.457+0000 7fbcf3e30640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec108320 0x7fbcec10a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:33.272 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.457+0000 7fbcf3e30640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec07bea0 con 0x7fbcec108320 2024-09-11T23:12:33.272 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.457+0000 7fbcf1bcc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec108320 0x7fbcec10a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:33.272 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.457+0000 7fbcf1bcc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec108320 0x7fbcec10a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57278/0 (socket says 172.21.15.43:57278) 2024-09-11T23:12:33.272 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.457+0000 7fbcf1bcc640 1 -- 172.21.15.43:0/3307369437 learned_addr learned my addr 172.21.15.43:0/3307369437 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:33.272 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.458+0000 7fbcf1bcc640 1 -- 172.21.15.43:0/3307369437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcec10ac50 con 0x7fbcec108320 2024-09-11T23:12:33.272 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.458+0000 7fbcf1bcc640 1 --2- 172.21.15.43:0/3307369437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec108320 0x7fbcec10a710 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009820 tx=0x7fbce802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d655a845c9e579b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:33.272 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.459+0000 7fbcf0bca640 1 -- 172.21.15.43:0/3307369437 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbce8037070 con 0x7fbcec108320 2024-09-11T23:12:33.273 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.459+0000 7fbcf0bca640 1 -- 172.21.15.43:0/3307369437 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbce8032900 con 0x7fbcec108320 2024-09-11T23:12:33.273 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.459+0000 7fbcf0bca640 1 -- 172.21.15.43:0/3307369437 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbce8030850 con 0x7fbcec108320 2024-09-11T23:12:33.273 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.459+0000 7fbcf3e30640 1 -- 172.21.15.43:0/3307369437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec108320 msgr2=0x7fbcec10a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:33.273 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.459+0000 7fbcf3e30640 1 --2- 172.21.15.43:0/3307369437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec108320 0x7fbcec10a710 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009820 tx=0x7fbce802a270 comp rx=0 tx=0).stop 2024-09-11T23:12:33.273 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.459+0000 7fbcf3e30640 1 -- 172.21.15.43:0/3307369437 shutdown_connections 2024-09-11T23:12:33.274 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.459+0000 7fbcf3e30640 1 --2- 172.21.15.43:0/3307369437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec108320 0x7fbcec10a710 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:33.274 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.459+0000 7fbcf3e30640 1 -- 172.21.15.43:0/3307369437 >> 172.21.15.43:0/3307369437 conn(0x7fbcec101f20 msgr2=0x7fbcec104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:33.274 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.459+0000 7fbcf3e30640 1 -- 172.21.15.43:0/3307369437 shutdown_connections 2024-09-11T23:12:33.274 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.459+0000 7fbcf3e30640 1 -- 172.21.15.43:0/3307369437 wait complete. 2024-09-11T23:12:33.274 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.460+0000 7fbcf3e30640 1 Processor -- start 2024-09-11T23:12:33.274 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.460+0000 7fbcf3e30640 1 -- start start 2024-09-11T23:12:33.274 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.461+0000 7fbcf3e30640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec108320 0x7fbcec19b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:33.274 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.461+0000 7fbcf3e30640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec10bcf0 con 0x7fbcec108320 2024-09-11T23:12:33.274 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.461+0000 7fbcf1bcc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec108320 0x7fbcec19b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:33.274 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.461+0000 7fbcf1bcc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec108320 0x7fbcec19b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57288/0 (socket says 172.21.15.43:57288) 2024-09-11T23:12:33.274 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.461+0000 7fbcf1bcc640 1 -- 172.21.15.43:0/3855967282 learned_addr learned my addr 172.21.15.43:0/3855967282 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:33.274 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.461+0000 7fbcf1bcc640 1 -- 172.21.15.43:0/3855967282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcec19bde0 con 0x7fbcec108320 2024-09-11T23:12:33.275 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.461+0000 7fbcf1bcc640 1 --2- 172.21.15.43:0/3855967282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec108320 0x7fbcec19b8a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbce8032ce0 tx=0x7fbce8032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:33.275 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.461+0000 7fbce2ffd640 1 -- 172.21.15.43:0/3855967282 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbce8042070 con 0x7fbcec108320 2024-09-11T23:12:33.275 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.461+0000 7fbcf3e30640 1 -- 172.21.15.43:0/3855967282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcec19c070 con 0x7fbcec108320 2024-09-11T23:12:33.275 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.461+0000 7fbce2ffd640 1 -- 172.21.15.43:0/3855967282 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbce80308c0 con 0x7fbcec108320 2024-09-11T23:12:33.275 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.461+0000 7fbce2ffd640 1 -- 172.21.15.43:0/3855967282 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbce8037040 con 0x7fbcec108320 2024-09-11T23:12:33.275 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.461+0000 7fbcf3e30640 1 -- 172.21.15.43:0/3855967282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcec19ed60 con 0x7fbcec108320 2024-09-11T23:12:33.275 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.462+0000 7fbce2ffd640 1 -- 172.21.15.43:0/3855967282 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fbce8045070 con 0x7fbcec108320 2024-09-11T23:12:33.275 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.462+0000 7fbcf3e30640 1 -- 172.21.15.43:0/3855967282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcec108780 con 0x7fbcec108320 2024-09-11T23:12:33.275 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.463+0000 7fbce2ffd640 1 --2- 172.21.15.43:0/3855967282 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7fbcd8038f70 0x7fbcd803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:33.275 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.463+0000 7fbce2ffd640 1 -- 172.21.15.43:0/3855967282 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbce806bff0 con 0x7fbcec108320 2024-09-11T23:12:33.275 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.463+0000 7fbcf13cb640 1 -- 172.21.15.43:0/3855967282 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7fbcd8038f70 msgr2=0x7fbcd803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/4015351527 2024-09-11T23:12:33.275 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.463+0000 7fbcf13cb640 1 --2- 172.21.15.43:0/3855967282 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7fbcd8038f70 0x7fbcd803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:12:33.275 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.467+0000 7fbce2ffd640 1 -- 172.21.15.43:0/3855967282 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbce804b170 con 0x7fbcec108320 2024-09-11T23:12:33.276 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.664+0000 7fbcf13cb640 1 -- 172.21.15.43:0/3855967282 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7fbcd8038f70 msgr2=0x7fbcd803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/4015351527 2024-09-11T23:12:33.276 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.664+0000 7fbcf13cb640 1 --2- 172.21.15.43:0/3855967282 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7fbcd8038f70 0x7fbcd803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:12:33.276 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.690+0000 7fbcf3e30640 1 -- 172.21.15.43:0/3855967282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fbcec19f3e0 con 0x7fbcec108320 2024-09-11T23:12:33.276 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.691+0000 7fbce2ffd640 1 -- 172.21.15.43:0/3855967282 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fbce803cd50 con 0x7fbcec108320 2024-09-11T23:12:33.276 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.693+0000 7fbcf3e30640 1 -- 172.21.15.43:0/3855967282 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7fbcd8038f70 msgr2=0x7fbcd803b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:12:33.276 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.693+0000 7fbcf3e30640 1 --2- 172.21.15.43:0/3855967282 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7fbcd8038f70 0x7fbcd803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:33.276 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.693+0000 7fbcf3e30640 1 -- 172.21.15.43:0/3855967282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec108320 msgr2=0x7fbcec19b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:33.276 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.693+0000 7fbcf3e30640 1 --2- 172.21.15.43:0/3855967282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec108320 0x7fbcec19b8a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbce8032ce0 tx=0x7fbce8032d10 comp rx=0 tx=0).stop 2024-09-11T23:12:33.276 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.693+0000 7fbcf3e30640 1 -- 172.21.15.43:0/3855967282 shutdown_connections 2024-09-11T23:12:33.276 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.693+0000 7fbcf3e30640 1 --2- 172.21.15.43:0/3855967282 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7fbcd8038f70 0x7fbcd803b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:33.276 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.693+0000 7fbcf3e30640 1 --2- 172.21.15.43:0/3855967282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec108320 0x7fbcec19b8a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:33.276 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.693+0000 7fbcf3e30640 1 -- 172.21.15.43:0/3855967282 >> 172.21.15.43:0/3855967282 conn(0x7fbcec101f20 msgr2=0x7fbcec102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:33.276 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.694+0000 7fbcf3e30640 1 -- 172.21.15.43:0/3855967282 shutdown_connections 2024-09-11T23:12:33.277 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:32.694+0000 7fbcf3e30640 1 -- 172.21.15.43:0/3855967282 wait complete. 2024-09-11T23:12:33.277 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for the mgr to restart... 2024-09-11T23:12:33.277 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr epoch 5... 2024-09-11T23:12:33.769 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:33 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:33.769+0000 7f535ec2b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:12:34.041 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:33 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:33.891+0000 7f535ec2b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:12:34.335 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:34 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:34.041+0000 7f535ec2b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:12:34.335 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:34 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:34.106+0000 7f535ec2b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:12:34.336 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:34 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:34.263+0000 7f535ec2b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:12:34.336 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:34 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:34.335+0000 7f535ec2b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:12:34.610 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:34 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:34.398+0000 7f535ec2b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:12:34.610 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:34 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:34.566+0000 7f535ec2b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:12:34.917 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:34 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:34.634+0000 7f535ec2b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:12:34.917 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:34 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:34.704+0000 7f535ec2b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:12:34.918 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:34 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:34.792+0000 7f535ec2b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:12:34.918 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:34 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:34.854+0000 7f535ec2b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:12:34.918 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:34 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:34.917+0000 7f535ec2b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:12:36.110 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:35 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:35.754+0000 7f535ec2b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:12:36.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:35 smithi043 ceph-mon[30950]: Active manager daemon x restarted 2024-09-11T23:12:36.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:35 smithi043 ceph-mon[30950]: Activating manager daemon x 2024-09-11T23:12:36.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:35 smithi043 ceph-mon[30950]: osdmap e2: 0 total, 0 up, 0 in 2024-09-11T23:12:36.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:35 smithi043 ceph-mon[30950]: mgrmap e6: x(active, starting, since 0.0032562s) 2024-09-11T23:12:36.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:35 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:12:36.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:35 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:12:36.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:35 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:12:36.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:35 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:12:36.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:35 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:12:36.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:35 smithi043 ceph-mon[30950]: Manager daemon x is now available 2024-09-11T23:12:36.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:35 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' 2024-09-11T23:12:36.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:35 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' 2024-09-11T23:12:36.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:35 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:36.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:35 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:36.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:35 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:12:36.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:35 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:12:37.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:36 smithi043 ceph-mon[30950]: Found migration_current of "None". Setting to last migration. 2024-09-11T23:12:37.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:36 smithi043 ceph-mon[30950]: mgrmap e7: x(active, since 1.00549s) 2024-09-11T23:12:37.331 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:12:37.331 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-11T23:12:37.331 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T23:12:37.331 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:12:37.331 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.671+0000 7f6ef3c25640 1 Processor -- start 2024-09-11T23:12:37.331 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.672+0000 7f6ef3c25640 1 -- start start 2024-09-11T23:12:37.331 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.672+0000 7f6ef3c25640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eec106790 0x7f6eec106b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:37.331 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.672+0000 7f6ef3c25640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec107160 con 0x7f6eec106790 2024-09-11T23:12:37.331 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.672+0000 7f6ef19c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eec106790 0x7f6eec106b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:37.331 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.672+0000 7f6ef19c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eec106790 0x7f6eec106b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57296/0 (socket says 172.21.15.43:57296) 2024-09-11T23:12:37.338 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.672+0000 7f6ef19c1640 1 -- 172.21.15.43:0/2514619028 learned_addr learned my addr 172.21.15.43:0/2514619028 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:37.339 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.672+0000 7f6ef19c1640 1 -- 172.21.15.43:0/2514619028 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eec107990 con 0x7f6eec106790 2024-09-11T23:12:37.339 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.673+0000 7f6ef19c1640 1 --2- 172.21.15.43:0/2514619028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eec106790 0x7f6eec106b90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009820 tx=0x7f6ee802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6dc0babf5983312 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:37.339 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.673+0000 7f6ef09bf640 1 -- 172.21.15.43:0/2514619028 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ee8037070 con 0x7f6eec106790 2024-09-11T23:12:37.339 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.673+0000 7f6ef09bf640 1 -- 172.21.15.43:0/2514619028 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6ee8032900 con 0x7f6eec106790 2024-09-11T23:12:37.339 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.673+0000 7f6ef09bf640 1 -- 172.21.15.43:0/2514619028 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ee8030850 con 0x7f6eec106790 2024-09-11T23:12:37.339 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.674+0000 7f6ef3c25640 1 -- 172.21.15.43:0/2514619028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eec106790 msgr2=0x7f6eec106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:37.339 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.674+0000 7f6ef3c25640 1 --2- 172.21.15.43:0/2514619028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eec106790 0x7f6eec106b90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009820 tx=0x7f6ee802a270 comp rx=0 tx=0).stop 2024-09-11T23:12:37.339 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.674+0000 7f6ef3c25640 1 -- 172.21.15.43:0/2514619028 shutdown_connections 2024-09-11T23:12:37.339 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.674+0000 7f6ef3c25640 1 --2- 172.21.15.43:0/2514619028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eec106790 0x7f6eec106b90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:37.339 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.674+0000 7f6ef3c25640 1 -- 172.21.15.43:0/2514619028 >> 172.21.15.43:0/2514619028 conn(0x7f6eec101f20 msgr2=0x7f6eec104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:37.339 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.674+0000 7f6ef3c25640 1 -- 172.21.15.43:0/2514619028 shutdown_connections 2024-09-11T23:12:37.340 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.674+0000 7f6ef3c25640 1 -- 172.21.15.43:0/2514619028 wait complete. 2024-09-11T23:12:37.340 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.675+0000 7f6ef3c25640 1 Processor -- start 2024-09-11T23:12:37.340 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.675+0000 7f6ef3c25640 1 -- start start 2024-09-11T23:12:37.340 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.675+0000 7f6ef3c25640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eec106790 0x7f6eec19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:37.340 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.675+0000 7f6ef3c25640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec10ad40 con 0x7f6eec106790 2024-09-11T23:12:37.340 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.675+0000 7f6ef19c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eec106790 0x7f6eec19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:37.340 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.675+0000 7f6ef19c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eec106790 0x7f6eec19a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57304/0 (socket says 172.21.15.43:57304) 2024-09-11T23:12:37.340 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.676+0000 7f6ef19c1640 1 -- 172.21.15.43:0/1203587342 learned_addr learned my addr 172.21.15.43:0/1203587342 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:37.340 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.676+0000 7f6ef19c1640 1 -- 172.21.15.43:0/1203587342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eec19ad00 con 0x7f6eec106790 2024-09-11T23:12:37.340 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.676+0000 7f6ef19c1640 1 --2- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eec106790 0x7f6eec19a7c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80097f0 tx=0x7f6ee8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:37.340 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.676+0000 7f6edeffd640 1 -- 172.21.15.43:0/1203587342 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ee8037040 con 0x7f6eec106790 2024-09-11T23:12:37.340 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.676+0000 7f6edeffd640 1 -- 172.21.15.43:0/1203587342 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6ee8040070 con 0x7f6eec106790 2024-09-11T23:12:37.341 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.676+0000 7f6ef3c25640 1 -- 172.21.15.43:0/1203587342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eec19af90 con 0x7f6eec106790 2024-09-11T23:12:37.341 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.676+0000 7f6edeffd640 1 -- 172.21.15.43:0/1203587342 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ee803a390 con 0x7f6eec106790 2024-09-11T23:12:37.341 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.676+0000 7f6ef3c25640 1 -- 172.21.15.43:0/1203587342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eec19b390 con 0x7f6eec106790 2024-09-11T23:12:37.341 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.677+0000 7f6edeffd640 1 -- 172.21.15.43:0/1203587342 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f6ee803b430 con 0x7f6eec106790 2024-09-11T23:12:37.341 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.677+0000 7f6edeffd640 1 --2- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f6ed803b180 0x7f6ed803d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:37.341 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.677+0000 7f6edeffd640 1 -- 172.21.15.43:0/1203587342 --> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6ed803dd50 con 0x7f6ed803b180 2024-09-11T23:12:37.341 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.677+0000 7f6edeffd640 1 -- 172.21.15.43:0/1203587342 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6ee806c170 con 0x7f6eec106790 2024-09-11T23:12:37.341 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.677+0000 7f6ef11c0640 1 -- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f6ed803b180 msgr2=0x7f6ed803d640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/4015351527 2024-09-11T23:12:37.341 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.678+0000 7f6ef11c0640 1 --2- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f6ed803b180 0x7f6ed803d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:12:37.341 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.879+0000 7f6ef11c0640 1 -- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f6ed803b180 msgr2=0x7f6ed803d640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/4015351527 2024-09-11T23:12:37.341 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:33.879+0000 7f6ef11c0640 1 --2- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f6ed803b180 0x7f6ed803d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:12:37.341 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:34.280+0000 7f6ef11c0640 1 -- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f6ed803b180 msgr2=0x7f6ed803d640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/4015351527 2024-09-11T23:12:37.341 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:34.280+0000 7f6ef11c0640 1 --2- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f6ed803b180 0x7f6ed803d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-11T23:12:37.342 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:35.081+0000 7f6ef11c0640 1 -- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f6ed803b180 msgr2=0x7f6ed803d640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/4015351527 2024-09-11T23:12:37.342 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:35.081+0000 7f6ef11c0640 1 --2- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f6ed803b180 0x7f6ed803d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-11T23:12:37.342 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:35.757+0000 7f6edeffd640 1 -- 172.21.15.43:0/1203587342 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f6ee806af70 con 0x7f6eec106790 2024-09-11T23:12:37.342 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:35.758+0000 7f6edeffd640 1 -- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f6ed803b180 msgr2=0x7f6ed803d640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:12:37.342 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:35.758+0000 7f6edeffd640 1 --2- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f6ed803b180 0x7f6ed803d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:37.342 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:36.758+0000 7f6edeffd640 1 -- 172.21.15.43:0/1203587342 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f6ee804acf0 con 0x7f6eec106790 2024-09-11T23:12:37.342 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:36.759+0000 7f6edeffd640 1 --2- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f6ed803eab0 0x7f6ed8040ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:37.342 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:36.759+0000 7f6edeffd640 1 -- 172.21.15.43:0/1203587342 --> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6ee80655e0 con 0x7f6ed803eab0 2024-09-11T23:12:37.342 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:36.759+0000 7f6ef11c0640 1 --2- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f6ed803eab0 0x7f6ed8040ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:37.342 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:36.760+0000 7f6ef11c0640 1 --2- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f6ed803eab0 0x7f6ed8040ea0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0003980 tx=0x7f6ee0006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:37.342 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:36.760+0000 7f6edeffd640 1 -- 172.21.15.43:0/1203587342 <== mgr.14118 v2:172.21.15.43:6800/3892590401 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f6ee80655e0 con 0x7f6ed803eab0 2024-09-11T23:12:37.342 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:36.767+0000 7f6ef3c25640 1 -- 172.21.15.43:0/1203587342 --> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f6eec10aec0 con 0x7f6ed803eab0 2024-09-11T23:12:37.342 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:36.768+0000 7f6edeffd640 1 -- 172.21.15.43:0/1203587342 <== mgr.14118 v2:172.21.15.43:6800/3892590401 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f6eec10aec0 con 0x7f6ed803eab0 2024-09-11T23:12:37.343 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:36.768+0000 7f6ef3c25640 1 -- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f6ed803eab0 msgr2=0x7f6ed8040ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:37.343 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:36.768+0000 7f6ef3c25640 1 --2- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f6ed803eab0 0x7f6ed8040ea0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0003980 tx=0x7f6ee0006e10 comp rx=0 tx=0).stop 2024-09-11T23:12:37.343 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:36.768+0000 7f6ef3c25640 1 -- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eec106790 msgr2=0x7f6eec19a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:37.343 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:36.768+0000 7f6ef3c25640 1 --2- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eec106790 0x7f6eec19a7c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80097f0 tx=0x7f6ee8030d80 comp rx=0 tx=0).stop 2024-09-11T23:12:37.343 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:36.768+0000 7f6ef3c25640 1 -- 172.21.15.43:0/1203587342 shutdown_connections 2024-09-11T23:12:37.343 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:36.768+0000 7f6ef3c25640 1 --2- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f6ed803eab0 0x7f6ed8040ea0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:37.343 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:36.768+0000 7f6ef3c25640 1 --2- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:6800/4015351527,v1:172.21.15.43:6801/4015351527] conn(0x7f6ed803b180 0x7f6ed803d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:37.343 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:36.768+0000 7f6ef3c25640 1 --2- 172.21.15.43:0/1203587342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eec106790 0x7f6eec19a7c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:37.343 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:36.768+0000 7f6ef3c25640 1 -- 172.21.15.43:0/1203587342 >> 172.21.15.43:0/1203587342 conn(0x7f6eec101f20 msgr2=0x7f6eec102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:37.343 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:36.768+0000 7f6ef3c25640 1 -- 172.21.15.43:0/1203587342 shutdown_connections 2024-09-11T23:12:37.343 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:36.768+0000 7f6ef3c25640 1 -- 172.21.15.43:0/1203587342 wait complete. 2024-09-11T23:12:37.343 INFO:teuthology.orchestra.run.smithi043.stdout:mgr epoch 5 is available 2024-09-11T23:12:37.343 INFO:teuthology.orchestra.run.smithi043.stdout:Setting orchestrator backend to cephadm... 2024-09-11T23:12:38.232 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:37 smithi043 ceph-mon[30950]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T23:12:38.232 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:37 smithi043 ceph-mon[30950]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T23:12:38.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:37 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' 2024-09-11T23:12:38.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:37 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' 2024-09-11T23:12:38.524 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.717+0000 7fcb839b5640 1 Processor -- start 2024-09-11T23:12:38.524 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.718+0000 7fcb839b5640 1 -- start start 2024-09-11T23:12:38.524 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.718+0000 7fcb839b5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7c106190 0x7fcb7c106590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:38.524 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.718+0000 7fcb839b5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c07be20 con 0x7fcb7c106190 2024-09-11T23:12:38.524 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.718+0000 7fcb81751640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7c106190 0x7fcb7c106590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:38.524 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.718+0000 7fcb81751640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7c106190 0x7fcb7c106590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57376/0 (socket says 172.21.15.43:57376) 2024-09-11T23:12:38.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.718+0000 7fcb81751640 1 -- 172.21.15.43:0/2498019229 learned_addr learned my addr 172.21.15.43:0/2498019229 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:38.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.718+0000 7fcb81751640 1 -- 172.21.15.43:0/2498019229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7c106ad0 con 0x7fcb7c106190 2024-09-11T23:12:38.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.719+0000 7fcb81751640 1 --2- 172.21.15.43:0/2498019229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7c106190 0x7fcb7c106590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009820 tx=0x7fcb7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c8722f5662c851 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:38.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.719+0000 7fcb73fff640 1 -- 172.21.15.43:0/2498019229 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb78037070 con 0x7fcb7c106190 2024-09-11T23:12:38.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.719+0000 7fcb73fff640 1 -- 172.21.15.43:0/2498019229 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcb78032900 con 0x7fcb7c106190 2024-09-11T23:12:38.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.719+0000 7fcb73fff640 1 -- 172.21.15.43:0/2498019229 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb78030850 con 0x7fcb7c106190 2024-09-11T23:12:38.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.720+0000 7fcb839b5640 1 -- 172.21.15.43:0/2498019229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7c106190 msgr2=0x7fcb7c106590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:38.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.720+0000 7fcb839b5640 1 --2- 172.21.15.43:0/2498019229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7c106190 0x7fcb7c106590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009820 tx=0x7fcb7802a270 comp rx=0 tx=0).stop 2024-09-11T23:12:38.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.720+0000 7fcb839b5640 1 -- 172.21.15.43:0/2498019229 shutdown_connections 2024-09-11T23:12:38.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.720+0000 7fcb839b5640 1 --2- 172.21.15.43:0/2498019229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7c106190 0x7fcb7c106590 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:38.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.720+0000 7fcb839b5640 1 -- 172.21.15.43:0/2498019229 >> 172.21.15.43:0/2498019229 conn(0x7fcb7c101f20 msgr2=0x7fcb7c104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:38.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.720+0000 7fcb839b5640 1 -- 172.21.15.43:0/2498019229 shutdown_connections 2024-09-11T23:12:38.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.720+0000 7fcb839b5640 1 -- 172.21.15.43:0/2498019229 wait complete. 2024-09-11T23:12:38.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.721+0000 7fcb839b5640 1 Processor -- start 2024-09-11T23:12:38.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.721+0000 7fcb839b5640 1 -- start start 2024-09-11T23:12:38.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.721+0000 7fcb839b5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7c106190 0x7fcb7c19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:38.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.721+0000 7fcb839b5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c107bd0 con 0x7fcb7c106190 2024-09-11T23:12:38.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.721+0000 7fcb81751640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7c106190 0x7fcb7c19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:38.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.721+0000 7fcb81751640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7c106190 0x7fcb7c19da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57392/0 (socket says 172.21.15.43:57392) 2024-09-11T23:12:38.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.721+0000 7fcb81751640 1 -- 172.21.15.43:0/2337957367 learned_addr learned my addr 172.21.15.43:0/2337957367 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:38.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.721+0000 7fcb81751640 1 -- 172.21.15.43:0/2337957367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7c19dfb0 con 0x7fcb7c106190 2024-09-11T23:12:38.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.721+0000 7fcb81751640 1 --2- 172.21.15.43:0/2337957367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7c106190 0x7fcb7c19da70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcb78032ce0 tx=0x7fcb78032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:38.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.722+0000 7fcb727fc640 1 -- 172.21.15.43:0/2337957367 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb78042070 con 0x7fcb7c106190 2024-09-11T23:12:38.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.722+0000 7fcb727fc640 1 -- 172.21.15.43:0/2337957367 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcb780308c0 con 0x7fcb7c106190 2024-09-11T23:12:38.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.722+0000 7fcb839b5640 1 -- 172.21.15.43:0/2337957367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb7c19e240 con 0x7fcb7c106190 2024-09-11T23:12:38.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.722+0000 7fcb727fc640 1 -- 172.21.15.43:0/2337957367 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb78037040 con 0x7fcb7c106190 2024-09-11T23:12:38.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.722+0000 7fcb839b5640 1 -- 172.21.15.43:0/2337957367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb7c1a0f30 con 0x7fcb7c106190 2024-09-11T23:12:38.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.723+0000 7fcb727fc640 1 -- 172.21.15.43:0/2337957367 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fcb78045070 con 0x7fcb7c106190 2024-09-11T23:12:38.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.723+0000 7fcb839b5640 1 -- 172.21.15.43:0/2337957367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb7c106590 con 0x7fcb7c106190 2024-09-11T23:12:38.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.723+0000 7fcb727fc640 1 --2- 172.21.15.43:0/2337957367 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fcb68038e60 0x7fcb6803b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:38.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.723+0000 7fcb727fc640 1 -- 172.21.15.43:0/2337957367 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcb7806be80 con 0x7fcb7c106190 2024-09-11T23:12:38.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.723+0000 7fcb80f50640 1 --2- 172.21.15.43:0/2337957367 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fcb68038e60 0x7fcb6803b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:38.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.724+0000 7fcb80f50640 1 --2- 172.21.15.43:0/2337957367 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fcb68038e60 0x7fcb6803b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c0098c0 tx=0x7fcb6c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:38.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.728+0000 7fcb727fc640 1 -- 172.21.15.43:0/2337957367 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb78045520 con 0x7fcb7c106190 2024-09-11T23:12:38.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.921+0000 7fcb839b5640 1 -- 172.21.15.43:0/2337957367 --> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fcb7c02dab0 con 0x7fcb68038e60 2024-09-11T23:12:38.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.923+0000 7fcb727fc640 1 -- 172.21.15.43:0/2337957367 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fcb7803cd00 con 0x7fcb7c106190 2024-09-11T23:12:38.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.926+0000 7fcb727fc640 1 -- 172.21.15.43:0/2337957367 <== mgr.14118 v2:172.21.15.43:6800/3892590401 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcb7c02dab0 con 0x7fcb68038e60 2024-09-11T23:12:38.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.928+0000 7fcb839b5640 1 -- 172.21.15.43:0/2337957367 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fcb68038e60 msgr2=0x7fcb6803b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:38.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.928+0000 7fcb839b5640 1 --2- 172.21.15.43:0/2337957367 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fcb68038e60 0x7fcb6803b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c0098c0 tx=0x7fcb6c006db0 comp rx=0 tx=0).stop 2024-09-11T23:12:38.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.928+0000 7fcb839b5640 1 -- 172.21.15.43:0/2337957367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7c106190 msgr2=0x7fcb7c19da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:38.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.928+0000 7fcb839b5640 1 --2- 172.21.15.43:0/2337957367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7c106190 0x7fcb7c19da70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcb78032ce0 tx=0x7fcb78032d10 comp rx=0 tx=0).stop 2024-09-11T23:12:38.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.928+0000 7fcb839b5640 1 -- 172.21.15.43:0/2337957367 shutdown_connections 2024-09-11T23:12:38.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.928+0000 7fcb839b5640 1 --2- 172.21.15.43:0/2337957367 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fcb68038e60 0x7fcb6803b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:38.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.928+0000 7fcb839b5640 1 --2- 172.21.15.43:0/2337957367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7c106190 0x7fcb7c19da70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:38.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.928+0000 7fcb839b5640 1 -- 172.21.15.43:0/2337957367 >> 172.21.15.43:0/2337957367 conn(0x7fcb7c101f20 msgr2=0x7fcb7c1946f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:38.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.928+0000 7fcb839b5640 1 -- 172.21.15.43:0/2337957367 shutdown_connections 2024-09-11T23:12:38.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:37.928+0000 7fcb839b5640 1 -- 172.21.15.43:0/2337957367 wait complete. 2024-09-11T23:12:38.801 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:38.582+0000 7f52cb750640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:23:12:38] ENGINE Error in HTTPServer.serve 2024-09-11T23:12:38.802 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: Traceback (most recent call last): 2024-09-11T23:12:38.802 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:12:38.802 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: self._connections.run(self.expiration_interval) 2024-09-11T23:12:38.802 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:12:38.802 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: self._run(expiration_interval) 2024-09-11T23:12:38.802 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:12:38.802 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:12:38.802 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:12:38.802 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:12:38.802 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:12:38.802 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: s = self.context.wrap_socket( 2024-09-11T23:12:38.802 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:12:38.802 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: return self.sslsocket_class._create( 2024-09-11T23:12:38.803 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:12:38.803 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: self.do_handshake() 2024-09-11T23:12:38.803 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:12:38.803 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: self._sslobj.do_handshake() 2024-09-11T23:12:38.803 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:12:38.803 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:39.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-mon[30950]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:12:39.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-mon[30950]: mgrmap e8: x(active, since 2s) 2024-09-11T23:12:39.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' 2024-09-11T23:12:39.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:39.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-mon[30950]: [11/Sep/2024:23:12:38] ENGINE Bus STARTING 2024-09-11T23:12:39.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-mon[30950]: [11/Sep/2024:23:12:38] ENGINE Serving on https://172.21.15.43:7150 2024-09-11T23:12:39.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-mon[30950]: [11/Sep/2024:23:12:38] ENGINE Error in HTTPServer.serve 2024-09-11T23:12:39.111 INFO:journalctl@ceph.mon.a.smithi043.stdout: Traceback (most recent call last): 2024-09-11T23:12:39.111 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:12:39.111 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._connections.run(self.expiration_interval) 2024-09-11T23:12:39.111 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:12:39.111 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._run(expiration_interval) 2024-09-11T23:12:39.111 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:12:39.111 INFO:journalctl@ceph.mon.a.smithi043.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:12:39.111 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:12:39.112 INFO:journalctl@ceph.mon.a.smithi043.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:12:39.112 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:12:39.112 INFO:journalctl@ceph.mon.a.smithi043.stdout: s = self.context.wrap_socket( 2024-09-11T23:12:39.112 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:12:39.112 INFO:journalctl@ceph.mon.a.smithi043.stdout: return self.sslsocket_class._create( 2024-09-11T23:12:39.112 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:12:39.112 INFO:journalctl@ceph.mon.a.smithi043.stdout: self.do_handshake() 2024-09-11T23:12:39.112 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:12:39.112 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._sslobj.do_handshake() 2024-09-11T23:12:39.112 INFO:journalctl@ceph.mon.a.smithi043.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:12:39.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:38 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:39.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-11T23:12:39.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.967+0000 7fcc25a08640 1 Processor -- start 2024-09-11T23:12:39.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.967+0000 7fcc25a08640 1 -- start start 2024-09-11T23:12:39.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.968+0000 7fcc25a08640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc201061a0 0x7fcc201065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:39.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.968+0000 7fcc25a08640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2007be20 con 0x7fcc201061a0 2024-09-11T23:12:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.968+0000 7fcc1effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc201061a0 0x7fcc201065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.968+0000 7fcc1effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc201061a0 0x7fcc201065a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57398/0 (socket says 172.21.15.43:57398) 2024-09-11T23:12:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.968+0000 7fcc1effd640 1 -- 172.21.15.43:0/1729587083 learned_addr learned my addr 172.21.15.43:0/1729587083 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.968+0000 7fcc1effd640 1 -- 172.21.15.43:0/1729587083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc20106ae0 con 0x7fcc201061a0 2024-09-11T23:12:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.969+0000 7fcc1effd640 1 --2- 172.21.15.43:0/1729587083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc201061a0 0x7fcc201065a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009820 tx=0x7fcc1002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8382e1cffd8f589 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.969+0000 7fcc1dffb640 1 -- 172.21.15.43:0/1729587083 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc10037070 con 0x7fcc201061a0 2024-09-11T23:12:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.969+0000 7fcc1dffb640 1 -- 172.21.15.43:0/1729587083 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcc10032900 con 0x7fcc201061a0 2024-09-11T23:12:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.969+0000 7fcc1dffb640 1 -- 172.21.15.43:0/1729587083 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc10030850 con 0x7fcc201061a0 2024-09-11T23:12:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.970+0000 7fcc25a08640 1 -- 172.21.15.43:0/1729587083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc201061a0 msgr2=0x7fcc201065a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.970+0000 7fcc25a08640 1 --2- 172.21.15.43:0/1729587083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc201061a0 0x7fcc201065a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009820 tx=0x7fcc1002a270 comp rx=0 tx=0).stop 2024-09-11T23:12:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.970+0000 7fcc25a08640 1 -- 172.21.15.43:0/1729587083 shutdown_connections 2024-09-11T23:12:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.970+0000 7fcc25a08640 1 --2- 172.21.15.43:0/1729587083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc201061a0 0x7fcc201065a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.970+0000 7fcc25a08640 1 -- 172.21.15.43:0/1729587083 >> 172.21.15.43:0/1729587083 conn(0x7fcc20101f70 msgr2=0x7fcc20104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.970+0000 7fcc25a08640 1 -- 172.21.15.43:0/1729587083 shutdown_connections 2024-09-11T23:12:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.970+0000 7fcc25a08640 1 -- 172.21.15.43:0/1729587083 wait complete. 2024-09-11T23:12:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.970+0000 7fcc25a08640 1 Processor -- start 2024-09-11T23:12:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.971+0000 7fcc25a08640 1 -- start start 2024-09-11T23:12:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.971+0000 7fcc25a08640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc201061a0 0x7fcc2019da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.971+0000 7fcc25a08640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcc20107be0 con 0x7fcc201061a0 2024-09-11T23:12:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.971+0000 7fcc1effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc201061a0 0x7fcc2019da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.971+0000 7fcc1effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc201061a0 0x7fcc2019da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57410/0 (socket says 172.21.15.43:57410) 2024-09-11T23:12:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.971+0000 7fcc1effd640 1 -- 172.21.15.43:0/2773933598 learned_addr learned my addr 172.21.15.43:0/2773933598 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.971+0000 7fcc1effd640 1 -- 172.21.15.43:0/2773933598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2019dfa0 con 0x7fcc201061a0 2024-09-11T23:12:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.971+0000 7fcc1effd640 1 --2- 172.21.15.43:0/2773933598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc201061a0 0x7fcc2019da60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcc10032ce0 tx=0x7fcc10032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.971+0000 7fcc24a06640 1 -- 172.21.15.43:0/2773933598 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc10042070 con 0x7fcc201061a0 2024-09-11T23:12:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.971+0000 7fcc24a06640 1 -- 172.21.15.43:0/2773933598 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcc100308c0 con 0x7fcc201061a0 2024-09-11T23:12:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.971+0000 7fcc24a06640 1 -- 172.21.15.43:0/2773933598 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc10037040 con 0x7fcc201061a0 2024-09-11T23:12:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.972+0000 7fcc25a08640 1 -- 172.21.15.43:0/2773933598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2019e230 con 0x7fcc201061a0 2024-09-11T23:12:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.972+0000 7fcc25a08640 1 -- 172.21.15.43:0/2773933598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc201a0f20 con 0x7fcc201061a0 2024-09-11T23:12:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.972+0000 7fcc24a06640 1 -- 172.21.15.43:0/2773933598 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fcc10045070 con 0x7fcc201061a0 2024-09-11T23:12:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.973+0000 7fcc25a08640 1 -- 172.21.15.43:0/2773933598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc201065a0 con 0x7fcc201061a0 2024-09-11T23:12:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.973+0000 7fcc24a06640 1 --2- 172.21.15.43:0/2773933598 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fcc08038f70 0x7fcc0803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.973+0000 7fcc24a06640 1 -- 172.21.15.43:0/2773933598 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcc1006c180 con 0x7fcc201061a0 2024-09-11T23:12:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.973+0000 7fcc1e7fc640 1 --2- 172.21.15.43:0/2773933598 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fcc08038f70 0x7fcc0803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.973+0000 7fcc1e7fc640 1 --2- 172.21.15.43:0/2773933598 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fcc08038f70 0x7fcc0803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcc140098c0 tx=0x7fcc14006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:38.978+0000 7fcc24a06640 1 -- 172.21.15.43:0/2773933598 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc10045520 con 0x7fcc201061a0 2024-09-11T23:12:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:39.155+0000 7fcc25a08640 1 -- 172.21.15.43:0/2773933598 --> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fcc2002dab0 con 0x7fcc08038f70 2024-09-11T23:12:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:39.155+0000 7fcc24a06640 1 -- 172.21.15.43:0/2773933598 <== mgr.14118 v2:172.21.15.43:6800/3892590401 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fcc2002dab0 con 0x7fcc08038f70 2024-09-11T23:12:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:39.157+0000 7fcc25a08640 1 -- 172.21.15.43:0/2773933598 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fcc08038f70 msgr2=0x7fcc0803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:39.157+0000 7fcc25a08640 1 --2- 172.21.15.43:0/2773933598 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fcc08038f70 0x7fcc0803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcc140098c0 tx=0x7fcc14006db0 comp rx=0 tx=0).stop 2024-09-11T23:12:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:39.157+0000 7fcc25a08640 1 -- 172.21.15.43:0/2773933598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc201061a0 msgr2=0x7fcc2019da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:39.157+0000 7fcc25a08640 1 --2- 172.21.15.43:0/2773933598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc201061a0 0x7fcc2019da60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcc10032ce0 tx=0x7fcc10032d10 comp rx=0 tx=0).stop 2024-09-11T23:12:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:39.157+0000 7fcc25a08640 1 -- 172.21.15.43:0/2773933598 shutdown_connections 2024-09-11T23:12:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:39.157+0000 7fcc25a08640 1 --2- 172.21.15.43:0/2773933598 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fcc08038f70 0x7fcc0803b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:39.157+0000 7fcc25a08640 1 --2- 172.21.15.43:0/2773933598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc201061a0 0x7fcc2019da60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:39.157+0000 7fcc25a08640 1 -- 172.21.15.43:0/2773933598 >> 172.21.15.43:0/2773933598 conn(0x7fcc20101f70 msgr2=0x7fcc201946a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:39.157+0000 7fcc25a08640 1 -- 172.21.15.43:0/2773933598 shutdown_connections 2024-09-11T23:12:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:39.157+0000 7fcc25a08640 1 -- 172.21.15.43:0/2773933598 wait complete. 2024-09-11T23:12:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:Generating ssh key... 2024-09-11T23:12:39.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:39 smithi043 ceph-mon[30950]: [11/Sep/2024:23:12:38] ENGINE Serving on http://172.21.15.43:8765 2024-09-11T23:12:39.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:39 smithi043 ceph-mon[30950]: [11/Sep/2024:23:12:38] ENGINE Bus STARTED 2024-09-11T23:12:39.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:39 smithi043 ceph-mon[30950]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:12:40.860 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:40 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: Generating public/private rsa key pair. 2024-09-11T23:12:40.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:40 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: Your identification has been saved in /tmp/tmp0dedrsv_/key 2024-09-11T23:12:40.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:40 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: Your public key has been saved in /tmp/tmp0dedrsv_/key.pub 2024-09-11T23:12:40.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:40 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: The key fingerprint is: 2024-09-11T23:12:40.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:40 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: SHA256:n7PMtYowzWUiMcltkGgC+r6zr5IbF6yEkSOij0rt7oU ceph-25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:12:40.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:40 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: The key's randomart image is: 2024-09-11T23:12:40.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:40 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: +---[RSA 3072]----+ 2024-09-11T23:12:40.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:40 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: |.. ... | 2024-09-11T23:12:40.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:40 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: |... o..+ | 2024-09-11T23:12:40.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:40 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: |B o = o | 2024-09-11T23:12:40.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:40 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: |== + | 2024-09-11T23:12:40.862 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:40 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: |o.+ . S o | 2024-09-11T23:12:40.862 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:40 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: |.=... + = . | 2024-09-11T23:12:40.862 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:40 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: |+o+E .o o + . | 2024-09-11T23:12:40.862 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:40 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: |=oo.. o + + . | 2024-09-11T23:12:40.862 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:40 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: |oo*O. . =.. | 2024-09-11T23:12:40.862 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:40 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: +----[SHA256]-----+ 2024-09-11T23:12:41.170 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.139+0000 7f67aefab640 1 Processor -- start 2024-09-11T23:12:41.170 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.139+0000 7f67aefab640 1 -- start start 2024-09-11T23:12:41.170 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.139+0000 7f67aefab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67a807dd80 0x7f67a807c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:41.170 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.139+0000 7f67aefab640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f67a807c830 con 0x7f67a807dd80 2024-09-11T23:12:41.170 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.139+0000 7f67acd47640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67a807dd80 0x7f67a807c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:41.170 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.139+0000 7f67acd47640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67a807dd80 0x7f67a807c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57416/0 (socket says 172.21.15.43:57416) 2024-09-11T23:12:41.170 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.139+0000 7f67acd47640 1 -- 172.21.15.43:0/1890040331 learned_addr learned my addr 172.21.15.43:0/1890040331 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:41.170 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.139+0000 7f67acd47640 1 -- 172.21.15.43:0/1890040331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a807c9b0 con 0x7f67a807dd80 2024-09-11T23:12:41.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.140+0000 7f67acd47640 1 --2- 172.21.15.43:0/1890040331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67a807dd80 0x7f67a807c2f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f679c009820 tx=0x7f679c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc1597116ac71a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:41.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.140+0000 7f67a77fe640 1 -- 172.21.15.43:0/1890040331 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f679c037070 con 0x7f67a807dd80 2024-09-11T23:12:41.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.140+0000 7f67a77fe640 1 -- 172.21.15.43:0/1890040331 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f679c032900 con 0x7f67a807dd80 2024-09-11T23:12:41.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.140+0000 7f67a77fe640 1 -- 172.21.15.43:0/1890040331 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f679c030850 con 0x7f67a807dd80 2024-09-11T23:12:41.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.141+0000 7f67aefab640 1 -- 172.21.15.43:0/1890040331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67a807dd80 msgr2=0x7f67a807c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:41.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.141+0000 7f67aefab640 1 --2- 172.21.15.43:0/1890040331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67a807dd80 0x7f67a807c2f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f679c009820 tx=0x7f679c02a270 comp rx=0 tx=0).stop 2024-09-11T23:12:41.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.141+0000 7f67aefab640 1 -- 172.21.15.43:0/1890040331 shutdown_connections 2024-09-11T23:12:41.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.141+0000 7f67aefab640 1 --2- 172.21.15.43:0/1890040331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67a807dd80 0x7f67a807c2f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:41.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.141+0000 7f67aefab640 1 -- 172.21.15.43:0/1890040331 >> 172.21.15.43:0/1890040331 conn(0x7f67a81031c0 msgr2=0x7f67a81055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:41.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.141+0000 7f67aefab640 1 -- 172.21.15.43:0/1890040331 shutdown_connections 2024-09-11T23:12:41.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.141+0000 7f67aefab640 1 -- 172.21.15.43:0/1890040331 wait complete. 2024-09-11T23:12:41.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.141+0000 7f67aefab640 1 Processor -- start 2024-09-11T23:12:41.171 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.142+0000 7f67aefab640 1 -- start start 2024-09-11T23:12:41.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.142+0000 7f67aefab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67a807dd80 0x7f67a819db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:41.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.142+0000 7f67aefab640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f67a810de20 con 0x7f67a807dd80 2024-09-11T23:12:41.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.142+0000 7f67acd47640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67a807dd80 0x7f67a819db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:41.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.142+0000 7f67acd47640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67a807dd80 0x7f67a819db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57426/0 (socket says 172.21.15.43:57426) 2024-09-11T23:12:41.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.142+0000 7f67acd47640 1 -- 172.21.15.43:0/2051284335 learned_addr learned my addr 172.21.15.43:0/2051284335 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:41.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.142+0000 7f67acd47640 1 -- 172.21.15.43:0/2051284335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a819e040 con 0x7f67a807dd80 2024-09-11T23:12:41.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.142+0000 7f67acd47640 1 --2- 172.21.15.43:0/2051284335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67a807dd80 0x7f67a819db00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f679c031000 tx=0x7f679c0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:41.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.142+0000 7f67a5ffb640 1 -- 172.21.15.43:0/2051284335 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f679c042070 con 0x7f67a807dd80 2024-09-11T23:12:41.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.142+0000 7f67a5ffb640 1 -- 172.21.15.43:0/2051284335 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f679c032be0 con 0x7f67a807dd80 2024-09-11T23:12:41.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.142+0000 7f67aefab640 1 -- 172.21.15.43:0/2051284335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a819e2d0 con 0x7f67a807dd80 2024-09-11T23:12:41.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.143+0000 7f67a5ffb640 1 -- 172.21.15.43:0/2051284335 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f679c037040 con 0x7f67a807dd80 2024-09-11T23:12:41.172 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.143+0000 7f67aefab640 1 -- 172.21.15.43:0/2051284335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a81a0fc0 con 0x7f67a807dd80 2024-09-11T23:12:41.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.144+0000 7f67a5ffb640 1 -- 172.21.15.43:0/2051284335 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f679c045070 con 0x7f67a807dd80 2024-09-11T23:12:41.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.144+0000 7f67aefab640 1 -- 172.21.15.43:0/2051284335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67a807c370 con 0x7f67a807dd80 2024-09-11T23:12:41.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.144+0000 7f67a5ffb640 1 --2- 172.21.15.43:0/2051284335 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f6798038fc0 0x7f679803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:41.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.144+0000 7f67a5ffb640 1 -- 172.21.15.43:0/2051284335 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f679c06c090 con 0x7f67a807dd80 2024-09-11T23:12:41.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.144+0000 7f67a7fff640 1 --2- 172.21.15.43:0/2051284335 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f6798038fc0 0x7f679803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:41.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.144+0000 7f67a7fff640 1 --2- 172.21.15.43:0/2051284335 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f6798038fc0 0x7f679803b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f67940098c0 tx=0x7f6794006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:41.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.149+0000 7f67a5ffb640 1 -- 172.21.15.43:0/2051284335 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f679c03a780 con 0x7f67a807dd80 2024-09-11T23:12:41.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.326+0000 7f67aefab640 1 -- 172.21.15.43:0/2051284335 --> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f67a8104470 con 0x7f6798038fc0 2024-09-11T23:12:41.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.602+0000 7f67a5ffb640 1 -- 172.21.15.43:0/2051284335 <== mgr.14118 v2:172.21.15.43:6800/3892590401 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f67a8104470 con 0x7f6798038fc0 2024-09-11T23:12:41.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.605+0000 7f67aefab640 1 -- 172.21.15.43:0/2051284335 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f6798038fc0 msgr2=0x7f679803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:41.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.605+0000 7f67aefab640 1 --2- 172.21.15.43:0/2051284335 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f6798038fc0 0x7f679803b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f67940098c0 tx=0x7f6794006db0 comp rx=0 tx=0).stop 2024-09-11T23:12:41.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.605+0000 7f67aefab640 1 -- 172.21.15.43:0/2051284335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67a807dd80 msgr2=0x7f67a819db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:41.173 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.605+0000 7f67aefab640 1 --2- 172.21.15.43:0/2051284335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67a807dd80 0x7f67a819db00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f679c031000 tx=0x7f679c0326e0 comp rx=0 tx=0).stop 2024-09-11T23:12:41.174 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.605+0000 7f67aefab640 1 -- 172.21.15.43:0/2051284335 shutdown_connections 2024-09-11T23:12:41.174 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.605+0000 7f67aefab640 1 --2- 172.21.15.43:0/2051284335 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f6798038fc0 0x7f679803b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:41.174 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.605+0000 7f67aefab640 1 --2- 172.21.15.43:0/2051284335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67a807dd80 0x7f67a819db00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:41.174 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.605+0000 7f67aefab640 1 -- 172.21.15.43:0/2051284335 >> 172.21.15.43:0/2051284335 conn(0x7f67a81031c0 msgr2=0x7f67a8103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:41.174 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.605+0000 7f67aefab640 1 -- 172.21.15.43:0/2051284335 shutdown_connections 2024-09-11T23:12:41.174 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:40.605+0000 7f67aefab640 1 -- 172.21.15.43:0/2051284335 wait complete. 2024-09-11T23:12:41.783 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:41 smithi043 ceph-mon[30950]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:12:41.783 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:41 smithi043 ceph-mon[30950]: Generating ssh key... 2024-09-11T23:12:41.783 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:41 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' 2024-09-11T23:12:41.783 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:41 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' 2024-09-11T23:12:42.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCf4yzk7yyVuIOOSoconqtOAlG4SacTasG8/L3mcrdN59G6eiTu4208sTSTDNJfK34+kMiHwTK3E8GxzmaHxFd7JpauoqaMo1BlY7wTl0syU2C9sZmM880qIlk2mQGwwhOzUgBogizeS9cmeiagvYqW+gBgGdrh+lj1gq0s5HqM5lfgEO8uD/FAV8Sj23GfkH/al1eMMp2kOJ3lNfVI+Gr4Pvk3jhQacTUy40MLcpCoeF7FKgQsL1VlBQFx+kDZJbzCXpvzVFrfiWSp4K0PwMdWQzMUOitoOp8kog3AlcFiYC3CTYZYXM48o+5ggeKTTj1zgVwoN+H39bbut4SMBmnWSxWXvCgZ7Hd6RXq3sxR8+MAt0ueCAt5sEXCdOI1tdzpd/kioQFvsQ8coVNZXe5m/JadsOulWwFSfofKPPkmAjuyXwlBR+3HcNi8YzR4DaQbybqWuTTQc9iCe693lR8puuvLONr7QUG1hqqWgwYvQxvyPqktmrWKpQpRhaTjZPsk= ceph-25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:12:42.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.595+0000 7f3018a24640 1 Processor -- start 2024-09-11T23:12:42.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.595+0000 7f3018a24640 1 -- start start 2024-09-11T23:12:42.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.596+0000 7f3018a24640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3014107950 0x7f3014107d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:42.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.596+0000 7f3018a24640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3014108290 con 0x7f3014107950 2024-09-11T23:12:42.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.596+0000 7f301259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3014107950 0x7f3014107d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:42.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.596+0000 7f301259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3014107950 0x7f3014107d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43296/0 (socket says 172.21.15.43:43296) 2024-09-11T23:12:42.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.596+0000 7f301259c640 1 -- 172.21.15.43:0/36322471 learned_addr learned my addr 172.21.15.43:0/36322471 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:42.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.596+0000 7f301259c640 1 -- 172.21.15.43:0/36322471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3014108410 con 0x7f3014107950 2024-09-11T23:12:42.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.597+0000 7f301259c640 1 --2- 172.21.15.43:0/36322471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3014107950 0x7f3014107d50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3004009820 tx=0x7f300402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd138ef3a9f7f84 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:42.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.597+0000 7f301159a640 1 -- 172.21.15.43:0/36322471 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3004037070 con 0x7f3014107950 2024-09-11T23:12:42.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.597+0000 7f301159a640 1 -- 172.21.15.43:0/36322471 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3004032900 con 0x7f3014107950 2024-09-11T23:12:42.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.597+0000 7f301159a640 1 -- 172.21.15.43:0/36322471 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3004030850 con 0x7f3014107950 2024-09-11T23:12:42.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.598+0000 7f3018a24640 1 -- 172.21.15.43:0/36322471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3014107950 msgr2=0x7f3014107d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:42.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.598+0000 7f3018a24640 1 --2- 172.21.15.43:0/36322471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3014107950 0x7f3014107d50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3004009820 tx=0x7f300402a270 comp rx=0 tx=0).stop 2024-09-11T23:12:42.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.598+0000 7f3018a24640 1 -- 172.21.15.43:0/36322471 shutdown_connections 2024-09-11T23:12:42.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.598+0000 7f3018a24640 1 --2- 172.21.15.43:0/36322471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3014107950 0x7f3014107d50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:42.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.598+0000 7f3018a24640 1 -- 172.21.15.43:0/36322471 >> 172.21.15.43:0/36322471 conn(0x7f30141031c0 msgr2=0x7f30141055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:42.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.598+0000 7f3018a24640 1 -- 172.21.15.43:0/36322471 shutdown_connections 2024-09-11T23:12:42.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.598+0000 7f3018a24640 1 -- 172.21.15.43:0/36322471 wait complete. 2024-09-11T23:12:42.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.599+0000 7f3018a24640 1 Processor -- start 2024-09-11T23:12:42.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.599+0000 7f3018a24640 1 -- start start 2024-09-11T23:12:42.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.599+0000 7f3018a24640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3014107950 0x7f301407db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:42.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.599+0000 7f3018a24640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f301410bdc0 con 0x7f3014107950 2024-09-11T23:12:42.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.599+0000 7f301259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3014107950 0x7f301407db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:42.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.599+0000 7f301259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3014107950 0x7f301407db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43304/0 (socket says 172.21.15.43:43304) 2024-09-11T23:12:42.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.599+0000 7f301259c640 1 -- 172.21.15.43:0/1770386039 learned_addr learned my addr 172.21.15.43:0/1770386039 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:42.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.600+0000 7f301259c640 1 -- 172.21.15.43:0/1770386039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301407e0a0 con 0x7f3014107950 2024-09-11T23:12:42.405 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.600+0000 7f301259c640 1 --2- 172.21.15.43:0/1770386039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3014107950 0x7f301407db60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f30040097f0 tx=0x7f3004030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:42.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.601+0000 7f300b7fe640 1 -- 172.21.15.43:0/1770386039 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3004042070 con 0x7f3014107950 2024-09-11T23:12:42.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.601+0000 7f300b7fe640 1 -- 172.21.15.43:0/1770386039 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3004032ca0 con 0x7f3014107950 2024-09-11T23:12:42.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.601+0000 7f3018a24640 1 -- 172.21.15.43:0/1770386039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f301407c290 con 0x7f3014107950 2024-09-11T23:12:42.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.601+0000 7f300b7fe640 1 -- 172.21.15.43:0/1770386039 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3004037040 con 0x7f3014107950 2024-09-11T23:12:42.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.601+0000 7f3018a24640 1 -- 172.21.15.43:0/1770386039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f301407c7d0 con 0x7f3014107950 2024-09-11T23:12:42.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.602+0000 7f300b7fe640 1 -- 172.21.15.43:0/1770386039 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3004045070 con 0x7f3014107950 2024-09-11T23:12:42.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.602+0000 7f300b7fe640 1 --2- 172.21.15.43:0/1770386039 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f3000038fc0 0x7f300003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:42.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.602+0000 7f300b7fe640 1 -- 172.21.15.43:0/1770386039 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f300406c290 con 0x7f3014107950 2024-09-11T23:12:42.406 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.602+0000 7f3018a24640 1 -- 172.21.15.43:0/1770386039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3014107dd0 con 0x7f3014107950 2024-09-11T23:12:42.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.602+0000 7f3011d9b640 1 --2- 172.21.15.43:0/1770386039 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f3000038fc0 0x7f300003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:42.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.603+0000 7f3011d9b640 1 --2- 172.21.15.43:0/1770386039 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f3000038fc0 0x7f300003b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0098c0 tx=0x7f2ffc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:42.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.607+0000 7f300b7fe640 1 -- 172.21.15.43:0/1770386039 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f300403add0 con 0x7f3014107950 2024-09-11T23:12:42.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.782+0000 7f3018a24640 1 -- 172.21.15.43:0/1770386039 --> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f301407d0d0 con 0x7f3000038fc0 2024-09-11T23:12:42.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.783+0000 7f300b7fe640 1 -- 172.21.15.43:0/1770386039 <== mgr.14118 v2:172.21.15.43:6800/3892590401 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f301407d0d0 con 0x7f3000038fc0 2024-09-11T23:12:42.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.785+0000 7f3018a24640 1 -- 172.21.15.43:0/1770386039 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f3000038fc0 msgr2=0x7f300003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:42.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.785+0000 7f3018a24640 1 --2- 172.21.15.43:0/1770386039 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f3000038fc0 0x7f300003b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0098c0 tx=0x7f2ffc006db0 comp rx=0 tx=0).stop 2024-09-11T23:12:42.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.785+0000 7f3018a24640 1 -- 172.21.15.43:0/1770386039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3014107950 msgr2=0x7f301407db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:42.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.785+0000 7f3018a24640 1 --2- 172.21.15.43:0/1770386039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3014107950 0x7f301407db60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f30040097f0 tx=0x7f3004030ec0 comp rx=0 tx=0).stop 2024-09-11T23:12:42.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.785+0000 7f3018a24640 1 -- 172.21.15.43:0/1770386039 shutdown_connections 2024-09-11T23:12:42.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.785+0000 7f3018a24640 1 --2- 172.21.15.43:0/1770386039 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f3000038fc0 0x7f300003b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:42.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.785+0000 7f3018a24640 1 --2- 172.21.15.43:0/1770386039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3014107950 0x7f301407db60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:42.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.785+0000 7f3018a24640 1 -- 172.21.15.43:0/1770386039 >> 172.21.15.43:0/1770386039 conn(0x7f30141031c0 msgr2=0x7f3014103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:42.407 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.785+0000 7f3018a24640 1 -- 172.21.15.43:0/1770386039 shutdown_connections 2024-09-11T23:12:42.408 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:41.785+0000 7f3018a24640 1 -- 172.21.15.43:0/1770386039 wait complete. 2024-09-11T23:12:42.408 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-11T23:12:42.408 INFO:teuthology.orchestra.run.smithi043.stdout:Adding key to root@localhost authorized_keys... 2024-09-11T23:12:42.408 INFO:teuthology.orchestra.run.smithi043.stdout:Adding host smithi043... 2024-09-11T23:12:42.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:42 smithi043 ceph-mon[30950]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:12:43.777 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:43 smithi043 ceph-mon[30950]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi043", "addr": "172.21.15.43", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:12:45.010 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:44 smithi043 ceph-mon[30950]: Deploying cephadm binary to smithi043 2024-09-11T23:12:47.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Added host 'smithi043' with addr '172.21.15.43' 2024-09-11T23:12:47.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.878+0000 7fc4b9ce5640 1 Processor -- start 2024-09-11T23:12:47.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.878+0000 7fc4b9ce5640 1 -- start start 2024-09-11T23:12:47.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.878+0000 7fc4b9ce5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b4108410 0x7fc4b410a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:47.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.878+0000 7fc4b9ce5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b407be10 con 0x7fc4b4108410 2024-09-11T23:12:47.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.878+0000 7fc4b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b4108410 0x7fc4b410a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:47.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.878+0000 7fc4b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b4108410 0x7fc4b410a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43312/0 (socket says 172.21.15.43:43312) 2024-09-11T23:12:47.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.878+0000 7fc4b37fe640 1 -- 172.21.15.43:0/2953170176 learned_addr learned my addr 172.21.15.43:0/2953170176 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:47.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.879+0000 7fc4b37fe640 1 -- 172.21.15.43:0/2953170176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b410ad40 con 0x7fc4b4108410 2024-09-11T23:12:47.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.879+0000 7fc4b37fe640 1 --2- 172.21.15.43:0/2953170176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b4108410 0x7fc4b410a800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4009820 tx=0x7fc4a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405f465b5c8c2fee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:47.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.880+0000 7fc4b27fc640 1 -- 172.21.15.43:0/2953170176 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4a4037070 con 0x7fc4b4108410 2024-09-11T23:12:47.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.880+0000 7fc4b27fc640 1 -- 172.21.15.43:0/2953170176 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc4a4032900 con 0x7fc4b4108410 2024-09-11T23:12:47.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.880+0000 7fc4b27fc640 1 -- 172.21.15.43:0/2953170176 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4a4030850 con 0x7fc4b4108410 2024-09-11T23:12:47.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.880+0000 7fc4b9ce5640 1 -- 172.21.15.43:0/2953170176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b4108410 msgr2=0x7fc4b410a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:47.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.880+0000 7fc4b9ce5640 1 --2- 172.21.15.43:0/2953170176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b4108410 0x7fc4b410a800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4009820 tx=0x7fc4a402a270 comp rx=0 tx=0).stop 2024-09-11T23:12:47.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.880+0000 7fc4b9ce5640 1 -- 172.21.15.43:0/2953170176 shutdown_connections 2024-09-11T23:12:47.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.880+0000 7fc4b9ce5640 1 --2- 172.21.15.43:0/2953170176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b4108410 0x7fc4b410a800 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:47.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.880+0000 7fc4b9ce5640 1 -- 172.21.15.43:0/2953170176 >> 172.21.15.43:0/2953170176 conn(0x7fc4b4102010 msgr2=0x7fc4b4104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:47.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.880+0000 7fc4b9ce5640 1 -- 172.21.15.43:0/2953170176 shutdown_connections 2024-09-11T23:12:47.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.880+0000 7fc4b9ce5640 1 -- 172.21.15.43:0/2953170176 wait complete. 2024-09-11T23:12:47.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.881+0000 7fc4b9ce5640 1 Processor -- start 2024-09-11T23:12:47.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.881+0000 7fc4b9ce5640 1 -- start start 2024-09-11T23:12:47.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.881+0000 7fc4b9ce5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b4108410 0x7fc4b419b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:47.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.881+0000 7fc4b9ce5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b410bcb0 con 0x7fc4b4108410 2024-09-11T23:12:47.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.881+0000 7fc4b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b4108410 0x7fc4b419b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:47.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.881+0000 7fc4b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b4108410 0x7fc4b419b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43326/0 (socket says 172.21.15.43:43326) 2024-09-11T23:12:47.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.881+0000 7fc4b37fe640 1 -- 172.21.15.43:0/1454335735 learned_addr learned my addr 172.21.15.43:0/1454335735 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:47.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.882+0000 7fc4b37fe640 1 -- 172.21.15.43:0/1454335735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b419bda0 con 0x7fc4b4108410 2024-09-11T23:12:47.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.882+0000 7fc4b37fe640 1 --2- 172.21.15.43:0/1454335735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b4108410 0x7fc4b419b860 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40097f0 tx=0x7fc4a4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:47.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.882+0000 7fc4b0ff9640 1 -- 172.21.15.43:0/1454335735 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4a4037040 con 0x7fc4b4108410 2024-09-11T23:12:47.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.882+0000 7fc4b0ff9640 1 -- 172.21.15.43:0/1454335735 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc4a4040070 con 0x7fc4b4108410 2024-09-11T23:12:47.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.882+0000 7fc4b9ce5640 1 -- 172.21.15.43:0/1454335735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b419c030 con 0x7fc4b4108410 2024-09-11T23:12:47.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.882+0000 7fc4b0ff9640 1 -- 172.21.15.43:0/1454335735 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4a403a390 con 0x7fc4b4108410 2024-09-11T23:12:47.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.882+0000 7fc4b9ce5640 1 -- 172.21.15.43:0/1454335735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b419ed20 con 0x7fc4b4108410 2024-09-11T23:12:47.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.883+0000 7fc4b0ff9640 1 -- 172.21.15.43:0/1454335735 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc4a403a5e0 con 0x7fc4b4108410 2024-09-11T23:12:47.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.883+0000 7fc4b9ce5640 1 -- 172.21.15.43:0/1454335735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b4108870 con 0x7fc4b4108410 2024-09-11T23:12:47.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.883+0000 7fc4b0ff9640 1 --2- 172.21.15.43:0/1454335735 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fc49c038fc0 0x7fc49c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:47.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.883+0000 7fc4b0ff9640 1 -- 172.21.15.43:0/1454335735 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc4a406c480 con 0x7fc4b4108410 2024-09-11T23:12:47.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.883+0000 7fc4b2ffd640 1 --2- 172.21.15.43:0/1454335735 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fc49c038fc0 0x7fc49c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:47.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.884+0000 7fc4b2ffd640 1 --2- 172.21.15.43:0/1454335735 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fc49c038fc0 0x7fc49c03b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc4a80098c0 tx=0x7fc4a8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:47.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:42.888+0000 7fc4b0ff9640 1 -- 172.21.15.43:0/1454335735 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc4a4037d10 con 0x7fc4b4108410 2024-09-11T23:12:47.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:43.069+0000 7fc4b9ce5640 1 -- 172.21.15.43:0/1454335735 --> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi043", "addr": "172.21.15.43", "target": ["mon-mgr", ""]}) -- 0x7fc4b4103270 con 0x7fc49c038fc0 2024-09-11T23:12:47.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:46.945+0000 7fc4b0ff9640 1 -- 172.21.15.43:0/1454335735 <== mgr.14118 v2:172.21.15.43:6800/3892590401 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fc4b4103270 con 0x7fc49c038fc0 2024-09-11T23:12:47.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:46.948+0000 7fc4b9ce5640 1 -- 172.21.15.43:0/1454335735 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fc49c038fc0 msgr2=0x7fc49c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:47.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:46.948+0000 7fc4b9ce5640 1 --2- 172.21.15.43:0/1454335735 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fc49c038fc0 0x7fc49c03b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc4a80098c0 tx=0x7fc4a8006db0 comp rx=0 tx=0).stop 2024-09-11T23:12:47.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:46.948+0000 7fc4b9ce5640 1 -- 172.21.15.43:0/1454335735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b4108410 msgr2=0x7fc4b419b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:47.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:46.948+0000 7fc4b9ce5640 1 --2- 172.21.15.43:0/1454335735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b4108410 0x7fc4b419b860 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40097f0 tx=0x7fc4a4030d80 comp rx=0 tx=0).stop 2024-09-11T23:12:47.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:46.948+0000 7fc4b9ce5640 1 -- 172.21.15.43:0/1454335735 shutdown_connections 2024-09-11T23:12:47.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:46.948+0000 7fc4b9ce5640 1 --2- 172.21.15.43:0/1454335735 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fc49c038fc0 0x7fc49c03b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:47.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:46.948+0000 7fc4b9ce5640 1 --2- 172.21.15.43:0/1454335735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b4108410 0x7fc4b419b860 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:47.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:46.948+0000 7fc4b9ce5640 1 -- 172.21.15.43:0/1454335735 >> 172.21.15.43:0/1454335735 conn(0x7fc4b4102010 msgr2=0x7fc4b4102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:47.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:46.948+0000 7fc4b9ce5640 1 -- 172.21.15.43:0/1454335735 shutdown_connections 2024-09-11T23:12:47.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:46.948+0000 7fc4b9ce5640 1 -- 172.21.15.43:0/1454335735 wait complete. 2024-09-11T23:12:47.534 INFO:teuthology.orchestra.run.smithi043.stdout:Deploying unmanaged mon service... 2024-09-11T23:12:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:47 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' 2024-09-11T23:12:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:47 smithi043 ceph-mon[30950]: Added host smithi043 2024-09-11T23:12:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:47 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:12:48.903 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-11T23:12:48.903 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.983+0000 7fa1f8305640 1 Processor -- start 2024-09-11T23:12:48.903 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.983+0000 7fa1f8305640 1 -- start start 2024-09-11T23:12:48.903 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.983+0000 7fa1f8305640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1f0107930 0x7fa1f0107d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:48.903 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.983+0000 7fa1f8305640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f0108300 con 0x7fa1f0107930 2024-09-11T23:12:48.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.983+0000 7fa1f60a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1f0107930 0x7fa1f0107d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:48.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.983+0000 7fa1f60a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1f0107930 0x7fa1f0107d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43340/0 (socket says 172.21.15.43:43340) 2024-09-11T23:12:48.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.983+0000 7fa1f60a1640 1 -- 172.21.15.43:0/3748715097 learned_addr learned my addr 172.21.15.43:0/3748715097 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:48.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.984+0000 7fa1f60a1640 1 -- 172.21.15.43:0/3748715097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f0108a90 con 0x7fa1f0107930 2024-09-11T23:12:48.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.984+0000 7fa1f60a1640 1 --2- 172.21.15.43:0/3748715097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1f0107930 0x7fa1f0107d30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009820 tx=0x7fa1ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fdaab5a1ae7472d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:48.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.985+0000 7fa1f509f640 1 -- 172.21.15.43:0/3748715097 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ec037070 con 0x7fa1f0107930 2024-09-11T23:12:48.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.985+0000 7fa1f509f640 1 -- 172.21.15.43:0/3748715097 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa1ec032900 con 0x7fa1f0107930 2024-09-11T23:12:48.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.985+0000 7fa1f509f640 1 -- 172.21.15.43:0/3748715097 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ec030850 con 0x7fa1f0107930 2024-09-11T23:12:48.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.985+0000 7fa1f8305640 1 -- 172.21.15.43:0/3748715097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1f0107930 msgr2=0x7fa1f0107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:48.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.985+0000 7fa1f8305640 1 --2- 172.21.15.43:0/3748715097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1f0107930 0x7fa1f0107d30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009820 tx=0x7fa1ec02a270 comp rx=0 tx=0).stop 2024-09-11T23:12:48.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.985+0000 7fa1f8305640 1 -- 172.21.15.43:0/3748715097 shutdown_connections 2024-09-11T23:12:48.904 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.985+0000 7fa1f8305640 1 --2- 172.21.15.43:0/3748715097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1f0107930 0x7fa1f0107d30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:48.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.985+0000 7fa1f8305640 1 -- 172.21.15.43:0/3748715097 >> 172.21.15.43:0/3748715097 conn(0x7fa1f01030e0 msgr2=0x7fa1f0105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:48.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.985+0000 7fa1f8305640 1 -- 172.21.15.43:0/3748715097 shutdown_connections 2024-09-11T23:12:48.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.985+0000 7fa1f8305640 1 -- 172.21.15.43:0/3748715097 wait complete. 2024-09-11T23:12:48.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.986+0000 7fa1f8305640 1 Processor -- start 2024-09-11T23:12:48.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.986+0000 7fa1f8305640 1 -- start start 2024-09-11T23:12:48.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.986+0000 7fa1f8305640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1f0107930 0x7fa1f019b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:48.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.986+0000 7fa1f8305640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f010be40 con 0x7fa1f0107930 2024-09-11T23:12:48.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.986+0000 7fa1f60a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1f0107930 0x7fa1f019b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:48.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.986+0000 7fa1f60a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1f0107930 0x7fa1f019b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43342/0 (socket says 172.21.15.43:43342) 2024-09-11T23:12:48.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.986+0000 7fa1f60a1640 1 -- 172.21.15.43:0/2522991777 learned_addr learned my addr 172.21.15.43:0/2522991777 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:48.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.987+0000 7fa1f60a1640 1 -- 172.21.15.43:0/2522991777 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f019be20 con 0x7fa1f0107930 2024-09-11T23:12:48.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.987+0000 7fa1f60a1640 1 --2- 172.21.15.43:0/2522991777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1f0107930 0x7fa1f019b8e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009940 tx=0x7fa1ec0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:48.905 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.987+0000 7fa1e77fe640 1 -- 172.21.15.43:0/2522991777 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ec042070 con 0x7fa1f0107930 2024-09-11T23:12:48.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.987+0000 7fa1e77fe640 1 -- 172.21.15.43:0/2522991777 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa1ec032be0 con 0x7fa1f0107930 2024-09-11T23:12:48.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.987+0000 7fa1f8305640 1 -- 172.21.15.43:0/2522991777 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f019c0b0 con 0x7fa1f0107930 2024-09-11T23:12:48.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.987+0000 7fa1e77fe640 1 -- 172.21.15.43:0/2522991777 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ec037040 con 0x7fa1f0107930 2024-09-11T23:12:48.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.987+0000 7fa1f8305640 1 -- 172.21.15.43:0/2522991777 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f019eda0 con 0x7fa1f0107930 2024-09-11T23:12:48.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.988+0000 7fa1e77fe640 1 -- 172.21.15.43:0/2522991777 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa1ec045070 con 0x7fa1f0107930 2024-09-11T23:12:48.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.988+0000 7fa1f8305640 1 -- 172.21.15.43:0/2522991777 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f010bfc0 con 0x7fa1f0107930 2024-09-11T23:12:48.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.988+0000 7fa1e77fe640 1 --2- 172.21.15.43:0/2522991777 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fa1dc038f70 0x7fa1dc03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:48.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.988+0000 7fa1e77fe640 1 -- 172.21.15.43:0/2522991777 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa1ec06bef0 con 0x7fa1f0107930 2024-09-11T23:12:48.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.988+0000 7fa1f58a0640 1 --2- 172.21.15.43:0/2522991777 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fa1dc038f70 0x7fa1dc03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:48.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.989+0000 7fa1f58a0640 1 --2- 172.21.15.43:0/2522991777 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fa1dc038f70 0x7fa1dc03b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00098c0 tx=0x7fa1e0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:48.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:47.993+0000 7fa1e77fe640 1 -- 172.21.15.43:0/2522991777 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa1ec032d90 con 0x7fa1f0107930 2024-09-11T23:12:48.906 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:48.187+0000 7fa1f8305640 1 -- 172.21.15.43:0/2522991777 --> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fa1f0104430 con 0x7fa1dc038f70 2024-09-11T23:12:48.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:48.192+0000 7fa1e77fe640 1 -- 172.21.15.43:0/2522991777 <== mgr.14118 v2:172.21.15.43:6800/3892590401 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa1f0104430 con 0x7fa1dc038f70 2024-09-11T23:12:48.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:48.194+0000 7fa1f8305640 1 -- 172.21.15.43:0/2522991777 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fa1dc038f70 msgr2=0x7fa1dc03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:48.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:48.194+0000 7fa1f8305640 1 --2- 172.21.15.43:0/2522991777 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fa1dc038f70 0x7fa1dc03b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00098c0 tx=0x7fa1e0006db0 comp rx=0 tx=0).stop 2024-09-11T23:12:48.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:48.194+0000 7fa1f8305640 1 -- 172.21.15.43:0/2522991777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1f0107930 msgr2=0x7fa1f019b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:48.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:48.194+0000 7fa1f8305640 1 --2- 172.21.15.43:0/2522991777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1f0107930 0x7fa1f019b8e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009940 tx=0x7fa1ec0326e0 comp rx=0 tx=0).stop 2024-09-11T23:12:48.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:48.194+0000 7fa1f8305640 1 -- 172.21.15.43:0/2522991777 shutdown_connections 2024-09-11T23:12:48.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:48.194+0000 7fa1f8305640 1 --2- 172.21.15.43:0/2522991777 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fa1dc038f70 0x7fa1dc03b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:48.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:48.194+0000 7fa1f8305640 1 --2- 172.21.15.43:0/2522991777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1f0107930 0x7fa1f019b8e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:48.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:48.194+0000 7fa1f8305640 1 -- 172.21.15.43:0/2522991777 >> 172.21.15.43:0/2522991777 conn(0x7fa1f01030e0 msgr2=0x7fa1f0103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:48.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:48.194+0000 7fa1f8305640 1 -- 172.21.15.43:0/2522991777 shutdown_connections 2024-09-11T23:12:48.907 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:48.194+0000 7fa1f8305640 1 -- 172.21.15.43:0/2522991777 wait complete. 2024-09-11T23:12:48.907 INFO:teuthology.orchestra.run.smithi043.stdout:Deploying unmanaged mgr service... 2024-09-11T23:12:49.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:49 smithi043 ceph-mon[30950]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:12:49.476 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:49 smithi043 ceph-mon[30950]: Saving service mon spec with placement count:5 2024-09-11T23:12:49.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:49 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' 2024-09-11T23:12:50.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:50 smithi043 ceph-mon[30950]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:12:50.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:50 smithi043 ceph-mon[30950]: Saving service mgr spec with placement count:2 2024-09-11T23:12:50.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:50 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' 2024-09-11T23:12:50.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-11T23:12:50.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.329+0000 7fe18e16f640 1 Processor -- start 2024-09-11T23:12:50.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.329+0000 7fe18e16f640 1 -- start start 2024-09-11T23:12:50.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.330+0000 7fe18e16f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18807dd60 0x7fe18807c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:50.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.330+0000 7fe18e16f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe18807e160 con 0x7fe18807dd60 2024-09-11T23:12:50.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.330+0000 7fe1877fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18807dd60 0x7fe18807c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:50.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.330+0000 7fe1877fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18807dd60 0x7fe18807c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43358/0 (socket says 172.21.15.43:43358) 2024-09-11T23:12:50.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.330+0000 7fe1877fe640 1 -- 172.21.15.43:0/29300833 learned_addr learned my addr 172.21.15.43:0/29300833 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:50.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.330+0000 7fe1877fe640 1 -- 172.21.15.43:0/29300833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18807c810 con 0x7fe18807dd60 2024-09-11T23:12:50.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.331+0000 7fe1877fe640 1 --2- 172.21.15.43:0/29300833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18807dd60 0x7fe18807c2d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe178009820 tx=0x7fe17802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3ca67abcbbdfdb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:50.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.331+0000 7fe1867fc640 1 -- 172.21.15.43:0/29300833 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe178037070 con 0x7fe18807dd60 2024-09-11T23:12:50.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.331+0000 7fe1867fc640 1 -- 172.21.15.43:0/29300833 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe178032900 con 0x7fe18807dd60 2024-09-11T23:12:50.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.331+0000 7fe1867fc640 1 -- 172.21.15.43:0/29300833 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe178030850 con 0x7fe18807dd60 2024-09-11T23:12:50.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.332+0000 7fe18e16f640 1 -- 172.21.15.43:0/29300833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18807dd60 msgr2=0x7fe18807c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:50.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.332+0000 7fe18e16f640 1 --2- 172.21.15.43:0/29300833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18807dd60 0x7fe18807c2d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe178009820 tx=0x7fe17802a270 comp rx=0 tx=0).stop 2024-09-11T23:12:50.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.332+0000 7fe18e16f640 1 -- 172.21.15.43:0/29300833 shutdown_connections 2024-09-11T23:12:50.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.332+0000 7fe18e16f640 1 --2- 172.21.15.43:0/29300833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18807dd60 0x7fe18807c2d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:50.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.332+0000 7fe18e16f640 1 -- 172.21.15.43:0/29300833 >> 172.21.15.43:0/29300833 conn(0x7fe1881031c0 msgr2=0x7fe1881055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:50.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.332+0000 7fe18e16f640 1 -- 172.21.15.43:0/29300833 shutdown_connections 2024-09-11T23:12:50.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.332+0000 7fe18e16f640 1 -- 172.21.15.43:0/29300833 wait complete. 2024-09-11T23:12:50.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.332+0000 7fe18e16f640 1 Processor -- start 2024-09-11T23:12:50.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.333+0000 7fe18e16f640 1 -- start start 2024-09-11T23:12:50.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.333+0000 7fe18e16f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18807dd60 0x7fe18819daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:50.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.333+0000 7fe18e16f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe18810dde0 con 0x7fe18807dd60 2024-09-11T23:12:50.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.333+0000 7fe1877fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18807dd60 0x7fe18819daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:50.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.333+0000 7fe1877fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18807dd60 0x7fe18819daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43362/0 (socket says 172.21.15.43:43362) 2024-09-11T23:12:50.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.333+0000 7fe1877fe640 1 -- 172.21.15.43:0/1987090759 learned_addr learned my addr 172.21.15.43:0/1987090759 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:50.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.333+0000 7fe1877fe640 1 -- 172.21.15.43:0/1987090759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18819dfe0 con 0x7fe18807dd60 2024-09-11T23:12:50.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.333+0000 7fe1877fe640 1 --2- 172.21.15.43:0/1987090759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18807dd60 0x7fe18819daa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe1780097f0 tx=0x7fe178030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:50.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.334+0000 7fe184ff9640 1 -- 172.21.15.43:0/1987090759 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe178042070 con 0x7fe18807dd60 2024-09-11T23:12:50.933 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.334+0000 7fe184ff9640 1 -- 172.21.15.43:0/1987090759 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe178032ca0 con 0x7fe18807dd60 2024-09-11T23:12:50.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.334+0000 7fe18e16f640 1 -- 172.21.15.43:0/1987090759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18819e270 con 0x7fe18807dd60 2024-09-11T23:12:50.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.334+0000 7fe184ff9640 1 -- 172.21.15.43:0/1987090759 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe178037040 con 0x7fe18807dd60 2024-09-11T23:12:50.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.334+0000 7fe18e16f640 1 -- 172.21.15.43:0/1987090759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1881a0f60 con 0x7fe18807dd60 2024-09-11T23:12:50.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.335+0000 7fe184ff9640 1 -- 172.21.15.43:0/1987090759 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe178045070 con 0x7fe18807dd60 2024-09-11T23:12:50.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.335+0000 7fe184ff9640 1 --2- 172.21.15.43:0/1987090759 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fe170038fc0 0x7fe17003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:50.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.335+0000 7fe184ff9640 1 -- 172.21.15.43:0/1987090759 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe17806bfa0 con 0x7fe18807dd60 2024-09-11T23:12:50.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.335+0000 7fe18e16f640 1 -- 172.21.15.43:0/1987090759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe18807c350 con 0x7fe18807dd60 2024-09-11T23:12:50.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.335+0000 7fe186ffd640 1 --2- 172.21.15.43:0/1987090759 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fe170038fc0 0x7fe17003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:50.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.335+0000 7fe186ffd640 1 --2- 172.21.15.43:0/1987090759 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fe170038fc0 0x7fe17003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe17c0098c0 tx=0x7fe17c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:50.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.340+0000 7fe184ff9640 1 -- 172.21.15.43:0/1987090759 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe178045520 con 0x7fe18807dd60 2024-09-11T23:12:50.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.533+0000 7fe18e16f640 1 -- 172.21.15.43:0/1987090759 --> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe188104480 con 0x7fe170038fc0 2024-09-11T23:12:50.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.537+0000 7fe184ff9640 1 -- 172.21.15.43:0/1987090759 <== mgr.14118 v2:172.21.15.43:6800/3892590401 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe188104480 con 0x7fe170038fc0 2024-09-11T23:12:50.934 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.539+0000 7fe18e16f640 1 -- 172.21.15.43:0/1987090759 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fe170038fc0 msgr2=0x7fe17003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:50.935 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.539+0000 7fe18e16f640 1 --2- 172.21.15.43:0/1987090759 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fe170038fc0 0x7fe17003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe17c0098c0 tx=0x7fe17c006db0 comp rx=0 tx=0).stop 2024-09-11T23:12:50.935 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.539+0000 7fe18e16f640 1 -- 172.21.15.43:0/1987090759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18807dd60 msgr2=0x7fe18819daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:50.935 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.539+0000 7fe18e16f640 1 --2- 172.21.15.43:0/1987090759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18807dd60 0x7fe18819daa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe1780097f0 tx=0x7fe178030ec0 comp rx=0 tx=0).stop 2024-09-11T23:12:50.935 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.539+0000 7fe18e16f640 1 -- 172.21.15.43:0/1987090759 shutdown_connections 2024-09-11T23:12:50.935 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.539+0000 7fe18e16f640 1 --2- 172.21.15.43:0/1987090759 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fe170038fc0 0x7fe17003b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:50.935 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.539+0000 7fe18e16f640 1 --2- 172.21.15.43:0/1987090759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe18807dd60 0x7fe18819daa0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:50.935 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.539+0000 7fe18e16f640 1 -- 172.21.15.43:0/1987090759 >> 172.21.15.43:0/1987090759 conn(0x7fe1881031c0 msgr2=0x7fe188103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:50.935 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.539+0000 7fe18e16f640 1 -- 172.21.15.43:0/1987090759 shutdown_connections 2024-09-11T23:12:50.935 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:49.539+0000 7fe18e16f640 1 -- 172.21.15.43:0/1987090759 wait complete. 2024-09-11T23:12:51.800 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:51 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' 2024-09-11T23:12:51.800 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:51 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' 2024-09-11T23:12:52.843 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:52 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/4263224019' entity='client.admin' 2024-09-11T23:12:52.843 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:52 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' 2024-09-11T23:12:52.844 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.344+0000 7f2a9e06a640 1 Processor -- start 2024-09-11T23:12:52.852 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.344+0000 7f2a9e06a640 1 -- start start 2024-09-11T23:12:52.852 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.345+0000 7f2a9e06a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a98108380 0x7f2a9810a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:52.852 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.345+0000 7f2a9e06a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9807be40 con 0x7f2a98108380 2024-09-11T23:12:52.852 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.345+0000 7f2a977fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a98108380 0x7f2a9810a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:52.852 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.345+0000 7f2a977fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a98108380 0x7f2a9810a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47164/0 (socket says 172.21.15.43:47164) 2024-09-11T23:12:52.852 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.345+0000 7f2a977fe640 1 -- 172.21.15.43:0/2646604636 learned_addr learned my addr 172.21.15.43:0/2646604636 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:52.852 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.345+0000 7f2a977fe640 1 -- 172.21.15.43:0/2646604636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9810acb0 con 0x7f2a98108380 2024-09-11T23:12:52.852 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.346+0000 7f2a977fe640 1 --2- 172.21.15.43:0/2646604636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a98108380 0x7f2a9810a770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009820 tx=0x7f2a8802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5a747251bfc280 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:52.853 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.346+0000 7f2a967fc640 1 -- 172.21.15.43:0/2646604636 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a88037070 con 0x7f2a98108380 2024-09-11T23:12:52.853 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.346+0000 7f2a967fc640 1 -- 172.21.15.43:0/2646604636 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a88032920 con 0x7f2a98108380 2024-09-11T23:12:52.853 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.346+0000 7f2a967fc640 1 -- 172.21.15.43:0/2646604636 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a88030810 con 0x7f2a98108380 2024-09-11T23:12:52.853 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.347+0000 7f2a9e06a640 1 -- 172.21.15.43:0/2646604636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a98108380 msgr2=0x7f2a9810a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:52.853 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.347+0000 7f2a9e06a640 1 --2- 172.21.15.43:0/2646604636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a98108380 0x7f2a9810a770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009820 tx=0x7f2a8802a270 comp rx=0 tx=0).stop 2024-09-11T23:12:52.853 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.347+0000 7f2a9e06a640 1 -- 172.21.15.43:0/2646604636 shutdown_connections 2024-09-11T23:12:52.853 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.347+0000 7f2a9e06a640 1 --2- 172.21.15.43:0/2646604636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a98108380 0x7f2a9810a770 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:52.853 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.347+0000 7f2a9e06a640 1 -- 172.21.15.43:0/2646604636 >> 172.21.15.43:0/2646604636 conn(0x7f2a98101f20 msgr2=0x7f2a98104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:52.853 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.347+0000 7f2a9e06a640 1 -- 172.21.15.43:0/2646604636 shutdown_connections 2024-09-11T23:12:52.853 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.347+0000 7f2a9e06a640 1 -- 172.21.15.43:0/2646604636 wait complete. 2024-09-11T23:12:52.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.348+0000 7f2a9e06a640 1 Processor -- start 2024-09-11T23:12:52.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.348+0000 7f2a9e06a640 1 -- start start 2024-09-11T23:12:52.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.348+0000 7f2a9e06a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a98108380 0x7f2a98107c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:52.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.348+0000 7f2a9e06a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9810bd30 con 0x7f2a98108380 2024-09-11T23:12:52.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.348+0000 7f2a977fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a98108380 0x7f2a98107c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:52.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.348+0000 7f2a977fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a98108380 0x7f2a98107c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47176/0 (socket says 172.21.15.43:47176) 2024-09-11T23:12:52.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.348+0000 7f2a977fe640 1 -- 172.21.15.43:0/4263224019 learned_addr learned my addr 172.21.15.43:0/4263224019 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:52.854 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.348+0000 7f2a977fe640 1 -- 172.21.15.43:0/4263224019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a98108140 con 0x7f2a98108380 2024-09-11T23:12:52.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.349+0000 7f2a977fe640 1 --2- 172.21.15.43:0/4263224019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a98108380 0x7f2a98107c00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2a88030380 tx=0x7f2a88032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:52.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.349+0000 7f2a94ff9640 1 -- 172.21.15.43:0/4263224019 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a88037040 con 0x7f2a98108380 2024-09-11T23:12:52.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.349+0000 7f2a94ff9640 1 -- 172.21.15.43:0/4263224019 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a88040070 con 0x7f2a98108380 2024-09-11T23:12:52.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.349+0000 7f2a9e06a640 1 -- 172.21.15.43:0/4263224019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a98106330 con 0x7f2a98108380 2024-09-11T23:12:52.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.349+0000 7f2a94ff9640 1 -- 172.21.15.43:0/4263224019 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a8803a7b0 con 0x7f2a98108380 2024-09-11T23:12:52.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.349+0000 7f2a9e06a640 1 -- 172.21.15.43:0/4263224019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a98106870 con 0x7f2a98108380 2024-09-11T23:12:52.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.350+0000 7f2a94ff9640 1 -- 172.21.15.43:0/4263224019 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2a8803a9e0 con 0x7f2a98108380 2024-09-11T23:12:52.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.350+0000 7f2a94ff9640 1 --2- 172.21.15.43:0/4263224019 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f2a80038fc0 0x7f2a8003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:52.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.350+0000 7f2a94ff9640 1 -- 172.21.15.43:0/4263224019 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2a8806c5e0 con 0x7f2a98108380 2024-09-11T23:12:52.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.350+0000 7f2a96ffd640 1 --2- 172.21.15.43:0/4263224019 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f2a80038fc0 0x7f2a8003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:52.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.351+0000 7f2a9e06a640 1 -- 172.21.15.43:0/4263224019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a981087e0 con 0x7f2a98108380 2024-09-11T23:12:52.855 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.351+0000 7f2a96ffd640 1 --2- 172.21.15.43:0/4263224019 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f2a80038fc0 0x7f2a8003b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0098c0 tx=0x7f2a8c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:52.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.355+0000 7f2a94ff9640 1 -- 172.21.15.43:0/4263224019 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2a88044c70 con 0x7f2a98108380 2024-09-11T23:12:52.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.581+0000 7f2a9e06a640 1 -- 172.21.15.43:0/4263224019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f2a98106e20 con 0x7f2a98108380 2024-09-11T23:12:52.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.583+0000 7f2a94ff9640 1 -- 172.21.15.43:0/4263224019 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f2a88042df0 con 0x7f2a98108380 2024-09-11T23:12:52.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.587+0000 7f2a9e06a640 1 -- 172.21.15.43:0/4263224019 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f2a80038fc0 msgr2=0x7f2a8003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:52.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.587+0000 7f2a9e06a640 1 --2- 172.21.15.43:0/4263224019 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f2a80038fc0 0x7f2a8003b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0098c0 tx=0x7f2a8c006db0 comp rx=0 tx=0).stop 2024-09-11T23:12:52.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.587+0000 7f2a9e06a640 1 -- 172.21.15.43:0/4263224019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a98108380 msgr2=0x7f2a98107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:52.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.587+0000 7f2a9e06a640 1 --2- 172.21.15.43:0/4263224019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a98108380 0x7f2a98107c00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2a88030380 tx=0x7f2a88032670 comp rx=0 tx=0).stop 2024-09-11T23:12:52.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.587+0000 7f2a9e06a640 1 -- 172.21.15.43:0/4263224019 shutdown_connections 2024-09-11T23:12:52.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.587+0000 7f2a9e06a640 1 --2- 172.21.15.43:0/4263224019 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f2a80038fc0 0x7f2a8003b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:52.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.587+0000 7f2a9e06a640 1 --2- 172.21.15.43:0/4263224019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a98108380 0x7f2a98107c00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:52.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.587+0000 7f2a9e06a640 1 -- 172.21.15.43:0/4263224019 >> 172.21.15.43:0/4263224019 conn(0x7f2a98101f20 msgr2=0x7f2a98102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:52.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.587+0000 7f2a9e06a640 1 -- 172.21.15.43:0/4263224019 shutdown_connections 2024-09-11T23:12:52.856 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:51.587+0000 7f2a9e06a640 1 -- 172.21.15.43:0/4263224019 wait complete. 2024-09-11T23:12:54.072 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.271+0000 7fa426d97640 1 Processor -- start 2024-09-11T23:12:54.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.271+0000 7fa426d97640 1 -- start start 2024-09-11T23:12:54.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.271+0000 7fa426d97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420106240 0x7fa420106640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:54.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.271+0000 7fa426d97640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa42007bdd0 con 0x7fa420106240 2024-09-11T23:12:54.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.272+0000 7fa424b33640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420106240 0x7fa420106640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:54.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.272+0000 7fa424b33640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420106240 0x7fa420106640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47186/0 (socket says 172.21.15.43:47186) 2024-09-11T23:12:54.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.272+0000 7fa424b33640 1 -- 172.21.15.43:0/4011305921 learned_addr learned my addr 172.21.15.43:0/4011305921 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:54.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.272+0000 7fa424b33640 1 -- 172.21.15.43:0/4011305921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa420106b80 con 0x7fa420106240 2024-09-11T23:12:54.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.273+0000 7fa424b33640 1 --2- 172.21.15.43:0/4011305921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420106240 0x7fa420106640 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa414009820 tx=0x7fa41402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab454e5df938fb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:54.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.273+0000 7fa41f7fe640 1 -- 172.21.15.43:0/4011305921 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa414037070 con 0x7fa420106240 2024-09-11T23:12:54.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.273+0000 7fa41f7fe640 1 -- 172.21.15.43:0/4011305921 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa414032920 con 0x7fa420106240 2024-09-11T23:12:54.073 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.273+0000 7fa41f7fe640 1 -- 172.21.15.43:0/4011305921 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa414030810 con 0x7fa420106240 2024-09-11T23:12:54.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.273+0000 7fa426d97640 1 -- 172.21.15.43:0/4011305921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420106240 msgr2=0x7fa420106640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:54.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.273+0000 7fa426d97640 1 --2- 172.21.15.43:0/4011305921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420106240 0x7fa420106640 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa414009820 tx=0x7fa41402a270 comp rx=0 tx=0).stop 2024-09-11T23:12:54.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.273+0000 7fa426d97640 1 -- 172.21.15.43:0/4011305921 shutdown_connections 2024-09-11T23:12:54.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.273+0000 7fa426d97640 1 --2- 172.21.15.43:0/4011305921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420106240 0x7fa420106640 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:54.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.273+0000 7fa426d97640 1 -- 172.21.15.43:0/4011305921 >> 172.21.15.43:0/4011305921 conn(0x7fa420102010 msgr2=0x7fa420104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:54.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.273+0000 7fa426d97640 1 -- 172.21.15.43:0/4011305921 shutdown_connections 2024-09-11T23:12:54.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.273+0000 7fa426d97640 1 -- 172.21.15.43:0/4011305921 wait complete. 2024-09-11T23:12:54.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.274+0000 7fa426d97640 1 Processor -- start 2024-09-11T23:12:54.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.274+0000 7fa426d97640 1 -- start start 2024-09-11T23:12:54.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.274+0000 7fa426d97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420106240 0x7fa42019da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:54.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.274+0000 7fa426d97640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa420107ad0 con 0x7fa420106240 2024-09-11T23:12:54.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.274+0000 7fa424b33640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420106240 0x7fa42019da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:54.074 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.274+0000 7fa424b33640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420106240 0x7fa42019da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47188/0 (socket says 172.21.15.43:47188) 2024-09-11T23:12:54.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.274+0000 7fa424b33640 1 -- 172.21.15.43:0/722137967 learned_addr learned my addr 172.21.15.43:0/722137967 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:54.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.275+0000 7fa424b33640 1 -- 172.21.15.43:0/722137967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42019dfb0 con 0x7fa420106240 2024-09-11T23:12:54.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.275+0000 7fa424b33640 1 --2- 172.21.15.43:0/722137967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420106240 0x7fa42019da70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa4140097f0 tx=0x7fa4140327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:54.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.275+0000 7fa41dffb640 1 -- 172.21.15.43:0/722137967 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa414037040 con 0x7fa420106240 2024-09-11T23:12:54.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.275+0000 7fa426d97640 1 -- 172.21.15.43:0/722137967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42019e240 con 0x7fa420106240 2024-09-11T23:12:54.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.275+0000 7fa41dffb640 1 -- 172.21.15.43:0/722137967 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa414030cc0 con 0x7fa420106240 2024-09-11T23:12:54.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.275+0000 7fa41dffb640 1 -- 172.21.15.43:0/722137967 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa41403a6b0 con 0x7fa420106240 2024-09-11T23:12:54.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.275+0000 7fa426d97640 1 -- 172.21.15.43:0/722137967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4201a0f30 con 0x7fa420106240 2024-09-11T23:12:54.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.276+0000 7fa41dffb640 1 -- 172.21.15.43:0/722137967 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa41403a8a0 con 0x7fa420106240 2024-09-11T23:12:54.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.276+0000 7fa426d97640 1 -- 172.21.15.43:0/722137967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa420108180 con 0x7fa420106240 2024-09-11T23:12:54.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.277+0000 7fa41dffb640 1 --2- 172.21.15.43:0/722137967 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fa410038f70 0x7fa41003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:54.075 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.277+0000 7fa41dffb640 1 -- 172.21.15.43:0/722137967 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa41406c610 con 0x7fa420106240 2024-09-11T23:12:54.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.277+0000 7fa41ffff640 1 --2- 172.21.15.43:0/722137967 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fa410038f70 0x7fa41003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:54.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.277+0000 7fa41ffff640 1 --2- 172.21.15.43:0/722137967 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fa410038f70 0x7fa41003b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0098c0 tx=0x7fa40c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:54.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.281+0000 7fa41dffb640 1 -- 172.21.15.43:0/722137967 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa414044920 con 0x7fa420106240 2024-09-11T23:12:54.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.460+0000 7fa426d97640 1 -- 172.21.15.43:0/722137967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fa42002da10 con 0x7fa420106240 2024-09-11T23:12:54.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.463+0000 7fa41dffb640 1 -- 172.21.15.43:0/722137967 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fa414044400 con 0x7fa420106240 2024-09-11T23:12:54.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.466+0000 7fa426d97640 1 -- 172.21.15.43:0/722137967 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fa410038f70 msgr2=0x7fa41003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:54.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.466+0000 7fa426d97640 1 --2- 172.21.15.43:0/722137967 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fa410038f70 0x7fa41003b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0098c0 tx=0x7fa40c006db0 comp rx=0 tx=0).stop 2024-09-11T23:12:54.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.466+0000 7fa426d97640 1 -- 172.21.15.43:0/722137967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420106240 msgr2=0x7fa42019da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:54.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.466+0000 7fa426d97640 1 --2- 172.21.15.43:0/722137967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420106240 0x7fa42019da70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa4140097f0 tx=0x7fa4140327b0 comp rx=0 tx=0).stop 2024-09-11T23:12:54.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.466+0000 7fa426d97640 1 -- 172.21.15.43:0/722137967 shutdown_connections 2024-09-11T23:12:54.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.466+0000 7fa426d97640 1 --2- 172.21.15.43:0/722137967 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fa410038f70 0x7fa41003b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:54.076 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.466+0000 7fa426d97640 1 --2- 172.21.15.43:0/722137967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420106240 0x7fa42019da70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:54.077 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.466+0000 7fa426d97640 1 -- 172.21.15.43:0/722137967 >> 172.21.15.43:0/722137967 conn(0x7fa420102010 msgr2=0x7fa420194750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:54.077 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.466+0000 7fa426d97640 1 -- 172.21.15.43:0/722137967 shutdown_connections 2024-09-11T23:12:54.077 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:53.466+0000 7fa426d97640 1 -- 172.21.15.43:0/722137967 wait complete. 2024-09-11T23:12:54.077 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling the dashboard module... 2024-09-11T23:12:54.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:54 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/722137967' entity='client.admin' 2024-09-11T23:12:54.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:54 smithi043 ceph-mon[30950]: from='mgr.14118 172.21.15.43:0/4123011085' entity='mgr.x' 2024-09-11T23:12:55.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:55 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/993869391' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-11T23:12:55.610 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:55 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: ignoring --setuser ceph since I am not root 2024-09-11T23:12:55.611 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:55 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: ignoring --setgroup ceph since I am not root 2024-09-11T23:12:55.912 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:55 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:55.625+0000 7f5b9bbe8040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:12:55.913 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:55 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:55.685+0000 7f5b9bbe8040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:12:55.913 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:55 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:55.842+0000 7f5b9bbe8040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:12:55.913 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:55 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:55.912+0000 7f5b9bbe8040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:12:56.135 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.483+0000 7f6094bcb640 1 Processor -- start 2024-09-11T23:12:56.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.483+0000 7f6094bcb640 1 -- start start 2024-09-11T23:12:56.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.483+0000 7f6094bcb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6090108320 0x7f609010a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:56.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.484+0000 7f6094bcb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f609007bea0 con 0x7f6090108320 2024-09-11T23:12:56.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.484+0000 7f608e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6090108320 0x7f609010a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:56.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.484+0000 7f608e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6090108320 0x7f609010a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47194/0 (socket says 172.21.15.43:47194) 2024-09-11T23:12:56.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.484+0000 7f608e59c640 1 -- 172.21.15.43:0/2858239052 learned_addr learned my addr 172.21.15.43:0/2858239052 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:56.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.484+0000 7f608e59c640 1 -- 172.21.15.43:0/2858239052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609010ac50 con 0x7f6090108320 2024-09-11T23:12:56.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.485+0000 7f608e59c640 1 --2- 172.21.15.43:0/2858239052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6090108320 0x7f609010a710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6080009820 tx=0x7f608002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ecfb6be528f01e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:56.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.485+0000 7f608dd9b640 1 -- 172.21.15.43:0/2858239052 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6080037070 con 0x7f6090108320 2024-09-11T23:12:56.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.485+0000 7f608dd9b640 1 -- 172.21.15.43:0/2858239052 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6080032920 con 0x7f6090108320 2024-09-11T23:12:56.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.485+0000 7f608dd9b640 1 -- 172.21.15.43:0/2858239052 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6080030810 con 0x7f6090108320 2024-09-11T23:12:56.136 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.485+0000 7f6094bcb640 1 -- 172.21.15.43:0/2858239052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6090108320 msgr2=0x7f609010a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:56.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.485+0000 7f6094bcb640 1 --2- 172.21.15.43:0/2858239052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6090108320 0x7f609010a710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6080009820 tx=0x7f608002a270 comp rx=0 tx=0).stop 2024-09-11T23:12:56.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.486+0000 7f6094bcb640 1 -- 172.21.15.43:0/2858239052 shutdown_connections 2024-09-11T23:12:56.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.486+0000 7f6094bcb640 1 --2- 172.21.15.43:0/2858239052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6090108320 0x7f609010a710 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:56.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.486+0000 7f6094bcb640 1 -- 172.21.15.43:0/2858239052 >> 172.21.15.43:0/2858239052 conn(0x7f6090101f20 msgr2=0x7f6090104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:56.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.486+0000 7f6094bcb640 1 -- 172.21.15.43:0/2858239052 shutdown_connections 2024-09-11T23:12:56.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.486+0000 7f6094bcb640 1 -- 172.21.15.43:0/2858239052 wait complete. 2024-09-11T23:12:56.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.486+0000 7f6094bcb640 1 Processor -- start 2024-09-11T23:12:56.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.487+0000 7f6094bcb640 1 -- start start 2024-09-11T23:12:56.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.487+0000 7f6094bcb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6090108320 0x7f609019b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:56.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.487+0000 7f6094bcb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f609010bd30 con 0x7f6090108320 2024-09-11T23:12:56.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.487+0000 7f608e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6090108320 0x7f609019b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:56.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.487+0000 7f608e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6090108320 0x7f609019b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47198/0 (socket says 172.21.15.43:47198) 2024-09-11T23:12:56.137 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.487+0000 7f608e59c640 1 -- 172.21.15.43:0/993869391 learned_addr learned my addr 172.21.15.43:0/993869391 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:56.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.487+0000 7f608e59c640 1 -- 172.21.15.43:0/993869391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609019be00 con 0x7f6090108320 2024-09-11T23:12:56.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.487+0000 7f608e59c640 1 --2- 172.21.15.43:0/993869391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6090108320 0x7f609019b8c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6080030380 tx=0x7f6080032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:56.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.488+0000 7f6087fff640 1 -- 172.21.15.43:0/993869391 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6080037040 con 0x7f6090108320 2024-09-11T23:12:56.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.488+0000 7f6087fff640 1 -- 172.21.15.43:0/993869391 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6080040070 con 0x7f6090108320 2024-09-11T23:12:56.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.488+0000 7f6094bcb640 1 -- 172.21.15.43:0/993869391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f609019c090 con 0x7f6090108320 2024-09-11T23:12:56.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.488+0000 7f6087fff640 1 -- 172.21.15.43:0/993869391 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f608003a7b0 con 0x7f6090108320 2024-09-11T23:12:56.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.488+0000 7f6094bcb640 1 -- 172.21.15.43:0/993869391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f609019ed80 con 0x7f6090108320 2024-09-11T23:12:56.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.489+0000 7f6087fff640 1 -- 172.21.15.43:0/993869391 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f608003a9e0 con 0x7f6090108320 2024-09-11T23:12:56.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.489+0000 7f6094bcb640 1 -- 172.21.15.43:0/993869391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6090108780 con 0x7f6090108320 2024-09-11T23:12:56.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.489+0000 7f6087fff640 1 --2- 172.21.15.43:0/993869391 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f607c038fc0 0x7f607c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:56.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.489+0000 7f6087fff640 1 -- 172.21.15.43:0/993869391 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f608006c5f0 con 0x7f6090108320 2024-09-11T23:12:56.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.489+0000 7f6085d9b640 1 --2- 172.21.15.43:0/993869391 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f607c038fc0 0x7f607c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:56.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.489+0000 7f6085d9b640 1 --2- 172.21.15.43:0/993869391 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f607c038fc0 0x7f607c03b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f60780098c0 tx=0x7f6078006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:56.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.494+0000 7f6087fff640 1 -- 172.21.15.43:0/993869391 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6080044c70 con 0x7f6090108320 2024-09-11T23:12:56.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:54.723+0000 7f6094bcb640 1 -- 172.21.15.43:0/993869391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f609019f370 con 0x7f6090108320 2024-09-11T23:12:56.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:55.465+0000 7f6087fff640 1 -- 172.21.15.43:0/993869391 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f6080041270 con 0x7f6090108320 2024-09-11T23:12:56.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:55.467+0000 7f6087fff640 1 -- 172.21.15.43:0/993869391 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f6080045980 con 0x7f6090108320 2024-09-11T23:12:56.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:55.469+0000 7f6094bcb640 1 -- 172.21.15.43:0/993869391 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f607c038fc0 msgr2=0x7f607c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:56.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:55.469+0000 7f6094bcb640 1 --2- 172.21.15.43:0/993869391 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f607c038fc0 0x7f607c03b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f60780098c0 tx=0x7f6078006db0 comp rx=0 tx=0).stop 2024-09-11T23:12:56.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:55.469+0000 7f6094bcb640 1 -- 172.21.15.43:0/993869391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6090108320 msgr2=0x7f609019b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:56.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:55.469+0000 7f6094bcb640 1 --2- 172.21.15.43:0/993869391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6090108320 0x7f609019b8c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6080030380 tx=0x7f6080032670 comp rx=0 tx=0).stop 2024-09-11T23:12:56.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:55.469+0000 7f6094bcb640 1 -- 172.21.15.43:0/993869391 shutdown_connections 2024-09-11T23:12:56.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:55.469+0000 7f6094bcb640 1 --2- 172.21.15.43:0/993869391 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f607c038fc0 0x7f607c03b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:56.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:55.469+0000 7f6094bcb640 1 --2- 172.21.15.43:0/993869391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6090108320 0x7f609019b8c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:56.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:55.469+0000 7f6094bcb640 1 -- 172.21.15.43:0/993869391 >> 172.21.15.43:0/993869391 conn(0x7f6090101f20 msgr2=0x7f6090102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:56.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:55.469+0000 7f6094bcb640 1 -- 172.21.15.43:0/993869391 shutdown_connections 2024-09-11T23:12:56.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:55.469+0000 7f6094bcb640 1 -- 172.21.15.43:0/993869391 wait complete. 2024-09-11T23:12:56.216 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:56 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:56.066+0000 7f5b9bbe8040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:12:56.216 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:56 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:56.216+0000 7f5b9bbe8040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:12:56.466 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:56 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/993869391' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-11T23:12:56.466 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:56 smithi043 ceph-mon[30950]: mgrmap e9: x(active, since 19s) 2024-09-11T23:12:56.726 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:56 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:56.555+0000 7f5b9bbe8040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:12:56.726 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:56 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:56.619+0000 7f5b9bbe8040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:12:57.068 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:56 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:56.823+0000 7f5b9bbe8040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:12:57.369 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:12:57.369 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-11T23:12:57.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T23:12:57.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-11T23:12:57.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T23:12:57.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:12:57.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.519+0000 7f799644d640 1 Processor -- start 2024-09-11T23:12:57.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.519+0000 7f799644d640 1 -- start start 2024-09-11T23:12:57.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.519+0000 7f799644d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7990108320 0x7f799010a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:57.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.519+0000 7f799644d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f799007bea0 con 0x7f7990108320 2024-09-11T23:12:57.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.519+0000 7f798ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7990108320 0x7f799010a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:57.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.520+0000 7f798ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7990108320 0x7f799010a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47214/0 (socket says 172.21.15.43:47214) 2024-09-11T23:12:57.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.520+0000 7f798ffff640 1 -- 172.21.15.43:0/2564712914 learned_addr learned my addr 172.21.15.43:0/2564712914 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:57.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.520+0000 7f798ffff640 1 -- 172.21.15.43:0/2564712914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799010ac50 con 0x7f7990108320 2024-09-11T23:12:57.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.521+0000 7f798ffff640 1 --2- 172.21.15.43:0/2564712914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7990108320 0x7f799010a710 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7980009820 tx=0x7f798002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74042eca2d28a3a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:57.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.521+0000 7f798effd640 1 -- 172.21.15.43:0/2564712914 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7980037070 con 0x7f7990108320 2024-09-11T23:12:57.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.521+0000 7f798effd640 1 -- 172.21.15.43:0/2564712914 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7980032920 con 0x7f7990108320 2024-09-11T23:12:57.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.521+0000 7f798effd640 1 -- 172.21.15.43:0/2564712914 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7980030810 con 0x7f7990108320 2024-09-11T23:12:57.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.521+0000 7f799644d640 1 -- 172.21.15.43:0/2564712914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7990108320 msgr2=0x7f799010a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:57.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.521+0000 7f799644d640 1 --2- 172.21.15.43:0/2564712914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7990108320 0x7f799010a710 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7980009820 tx=0x7f798002a270 comp rx=0 tx=0).stop 2024-09-11T23:12:57.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.522+0000 7f799644d640 1 -- 172.21.15.43:0/2564712914 shutdown_connections 2024-09-11T23:12:57.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.522+0000 7f799644d640 1 --2- 172.21.15.43:0/2564712914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7990108320 0x7f799010a710 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:57.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.522+0000 7f799644d640 1 -- 172.21.15.43:0/2564712914 >> 172.21.15.43:0/2564712914 conn(0x7f7990101f20 msgr2=0x7f7990104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:57.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.522+0000 7f799644d640 1 -- 172.21.15.43:0/2564712914 shutdown_connections 2024-09-11T23:12:57.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.522+0000 7f799644d640 1 -- 172.21.15.43:0/2564712914 wait complete. 2024-09-11T23:12:57.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.522+0000 7f799644d640 1 Processor -- start 2024-09-11T23:12:57.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.522+0000 7f799644d640 1 -- start start 2024-09-11T23:12:57.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.523+0000 7f799644d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7990108320 0x7f7990107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:57.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.523+0000 7f799644d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f799010bd30 con 0x7f7990108320 2024-09-11T23:12:57.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.523+0000 7f798ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7990108320 0x7f7990107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:12:57.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.523+0000 7f798ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7990108320 0x7f7990107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47230/0 (socket says 172.21.15.43:47230) 2024-09-11T23:12:57.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.523+0000 7f798ffff640 1 -- 172.21.15.43:0/2977814029 learned_addr learned my addr 172.21.15.43:0/2977814029 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:12:57.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.523+0000 7f798ffff640 1 -- 172.21.15.43:0/2977814029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79901080e0 con 0x7f7990108320 2024-09-11T23:12:57.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.523+0000 7f798ffff640 1 --2- 172.21.15.43:0/2977814029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7990108320 0x7f7990107ba0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f79800097f0 tx=0x7f79800327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:12:57.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.524+0000 7f798d7fa640 1 -- 172.21.15.43:0/2977814029 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7980037040 con 0x7f7990108320 2024-09-11T23:12:57.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.524+0000 7f799644d640 1 -- 172.21.15.43:0/2977814029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79901062d0 con 0x7f7990108320 2024-09-11T23:12:57.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.524+0000 7f798d7fa640 1 -- 172.21.15.43:0/2977814029 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7980030cc0 con 0x7f7990108320 2024-09-11T23:12:57.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.524+0000 7f798d7fa640 1 -- 172.21.15.43:0/2977814029 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f798003a6b0 con 0x7f7990108320 2024-09-11T23:12:57.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.524+0000 7f799644d640 1 -- 172.21.15.43:0/2977814029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7990106810 con 0x7f7990108320 2024-09-11T23:12:57.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.525+0000 7f799644d640 1 -- 172.21.15.43:0/2977814029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7990108780 con 0x7f7990108320 2024-09-11T23:12:57.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.525+0000 7f798d7fa640 1 -- 172.21.15.43:0/2977814029 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f798003a850 con 0x7f7990108320 2024-09-11T23:12:57.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.525+0000 7f798d7fa640 1 --2- 172.21.15.43:0/2977814029 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f7978039020 0x7f797803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:12:57.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.525+0000 7f798d7fa640 1 -- 172.21.15.43:0/2977814029 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f798006c510 con 0x7f7990108320 2024-09-11T23:12:57.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.525+0000 7f798f7fe640 1 -- 172.21.15.43:0/2977814029 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f7978039020 msgr2=0x7f797803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3892590401 2024-09-11T23:12:57.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.525+0000 7f798f7fe640 1 --2- 172.21.15.43:0/2977814029 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f7978039020 0x7f797803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:12:57.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.534+0000 7f798d7fa640 1 -- 172.21.15.43:0/2977814029 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7980044920 con 0x7f7990108320 2024-09-11T23:12:57.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.726+0000 7f798f7fe640 1 -- 172.21.15.43:0/2977814029 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f7978039020 msgr2=0x7f797803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3892590401 2024-09-11T23:12:57.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.726+0000 7f798f7fe640 1 --2- 172.21.15.43:0/2977814029 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f7978039020 0x7f797803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:12:57.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.761+0000 7f799644d640 1 -- 172.21.15.43:0/2977814029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f7990106a90 con 0x7f7990108320 2024-09-11T23:12:57.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.762+0000 7f798d7fa640 1 -- 172.21.15.43:0/2977814029 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f7980044400 con 0x7f7990108320 2024-09-11T23:12:57.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.764+0000 7f799644d640 1 -- 172.21.15.43:0/2977814029 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f7978039020 msgr2=0x7f797803b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:12:57.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.764+0000 7f799644d640 1 --2- 172.21.15.43:0/2977814029 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f7978039020 0x7f797803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:57.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.764+0000 7f799644d640 1 -- 172.21.15.43:0/2977814029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7990108320 msgr2=0x7f7990107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:12:57.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.764+0000 7f799644d640 1 --2- 172.21.15.43:0/2977814029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7990108320 0x7f7990107ba0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f79800097f0 tx=0x7f79800327b0 comp rx=0 tx=0).stop 2024-09-11T23:12:57.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.764+0000 7f799644d640 1 -- 172.21.15.43:0/2977814029 shutdown_connections 2024-09-11T23:12:57.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.764+0000 7f799644d640 1 --2- 172.21.15.43:0/2977814029 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7f7978039020 0x7f797803b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:57.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.764+0000 7f799644d640 1 --2- 172.21.15.43:0/2977814029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7990108320 0x7f7990107ba0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:12:57.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.764+0000 7f799644d640 1 -- 172.21.15.43:0/2977814029 >> 172.21.15.43:0/2977814029 conn(0x7f7990101f20 msgr2=0x7f7990102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:12:57.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.764+0000 7f799644d640 1 -- 172.21.15.43:0/2977814029 shutdown_connections 2024-09-11T23:12:57.375 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:56.764+0000 7f799644d640 1 -- 172.21.15.43:0/2977814029 wait complete. 2024-09-11T23:12:57.375 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for the mgr to restart... 2024-09-11T23:12:57.376 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr epoch 9... 2024-09-11T23:12:57.688 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:12:57 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2977814029' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T23:12:57.975 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:57 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:57.763+0000 7f5b9bbe8040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:12:57.975 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:57 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:57.835+0000 7f5b9bbe8040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:12:58.703 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:58 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:58.582+0000 7f5b9bbe8040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:12:58.703 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:58 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:58.703+0000 7f5b9bbe8040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:12:59.073 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:58 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:58.852+0000 7f5b9bbe8040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:12:59.073 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:58 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:58.916+0000 7f5b9bbe8040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:12:59.073 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:59 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:59.073+0000 7f5b9bbe8040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:12:59.360 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:59 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:59.144+0000 7f5b9bbe8040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:12:59.360 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:59 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:59.207+0000 7f5b9bbe8040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:12:59.659 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:59 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:59.372+0000 7f5b9bbe8040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:12:59.659 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:59 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:59.441+0000 7f5b9bbe8040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:12:59.659 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:59 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:59.510+0000 7f5b9bbe8040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:12:59.659 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:59 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:59.597+0000 7f5b9bbe8040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:12:59.660 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:59 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:59.659+0000 7f5b9bbe8040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:13:00.110 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:12:59 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:12:59.720+0000 7f5b9bbe8040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:13:00.798 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:00 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:13:00.546+0000 7f5b9bbe8040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:13:00.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:00 smithi043 ceph-mon[30950]: Active manager daemon x restarted 2024-09-11T23:13:00.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:00 smithi043 ceph-mon[30950]: Activating manager daemon x 2024-09-11T23:13:00.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:00 smithi043 ceph-mon[30950]: osdmap e3: 0 total, 0 up, 0 in 2024-09-11T23:13:00.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:00 smithi043 ceph-mon[30950]: mgrmap e10: x(active, starting, since 0.00324139s) 2024-09-11T23:13:00.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:00 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:13:00.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:00 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:13:00.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:00 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:13:00.799 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:00 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:13:00.799 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:00 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:13:00.799 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:00 smithi043 ceph-mon[30950]: Manager daemon x is now available 2024-09-11T23:13:00.799 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:00 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:00.799 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:00 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:13:00.799 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:00 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:13:02.473 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-11T23:13:02.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-11T23:13:02.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T23:13:02.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-11T23:13:02.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.768+0000 7fed1f448640 1 Processor -- start 2024-09-11T23:13:02.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.768+0000 7fed1f448640 1 -- start start 2024-09-11T23:13:02.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.768+0000 7fed1f448640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed18108320 0x7fed1810a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:02.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.768+0000 7fed1f448640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fed1807be80 con 0x7fed18108320 2024-09-11T23:13:02.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.768+0000 7fed1d1e4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed18108320 0x7fed1810a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:02.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.768+0000 7fed1d1e4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed18108320 0x7fed1810a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47238/0 (socket says 172.21.15.43:47238) 2024-09-11T23:13:02.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.768+0000 7fed1d1e4640 1 -- 172.21.15.43:0/339228412 learned_addr learned my addr 172.21.15.43:0/339228412 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:13:02.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.769+0000 7fed1d1e4640 1 -- 172.21.15.43:0/339228412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1810ac50 con 0x7fed18108320 2024-09-11T23:13:02.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.769+0000 7fed1d1e4640 1 --2- 172.21.15.43:0/339228412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed18108320 0x7fed1810a710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fed14009820 tx=0x7fed1402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0f902dd34a0564 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:02.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.770+0000 7fed0ffff640 1 -- 172.21.15.43:0/339228412 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed14037070 con 0x7fed18108320 2024-09-11T23:13:02.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.770+0000 7fed0ffff640 1 -- 172.21.15.43:0/339228412 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed14032920 con 0x7fed18108320 2024-09-11T23:13:02.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.770+0000 7fed0ffff640 1 -- 172.21.15.43:0/339228412 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed14030810 con 0x7fed18108320 2024-09-11T23:13:02.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.771+0000 7fed1f448640 1 -- 172.21.15.43:0/339228412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed18108320 msgr2=0x7fed1810a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:02.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.771+0000 7fed1f448640 1 --2- 172.21.15.43:0/339228412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed18108320 0x7fed1810a710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fed14009820 tx=0x7fed1402a270 comp rx=0 tx=0).stop 2024-09-11T23:13:02.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.771+0000 7fed1f448640 1 -- 172.21.15.43:0/339228412 shutdown_connections 2024-09-11T23:13:02.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.771+0000 7fed1f448640 1 --2- 172.21.15.43:0/339228412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed18108320 0x7fed1810a710 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:02.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.771+0000 7fed1f448640 1 -- 172.21.15.43:0/339228412 >> 172.21.15.43:0/339228412 conn(0x7fed18101f20 msgr2=0x7fed18104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:02.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.771+0000 7fed1f448640 1 -- 172.21.15.43:0/339228412 shutdown_connections 2024-09-11T23:13:02.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.771+0000 7fed1f448640 1 -- 172.21.15.43:0/339228412 wait complete. 2024-09-11T23:13:02.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.771+0000 7fed1f448640 1 Processor -- start 2024-09-11T23:13:02.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.772+0000 7fed1f448640 1 -- start start 2024-09-11T23:13:02.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.772+0000 7fed1f448640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed18108320 0x7fed18199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:02.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.772+0000 7fed1f448640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fed18106f50 con 0x7fed18108320 2024-09-11T23:13:02.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.772+0000 7fed1d1e4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed18108320 0x7fed18199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:02.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.772+0000 7fed1d1e4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed18108320 0x7fed18199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47250/0 (socket says 172.21.15.43:47250) 2024-09-11T23:13:02.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.772+0000 7fed1d1e4640 1 -- 172.21.15.43:0/1554993528 learned_addr learned my addr 172.21.15.43:0/1554993528 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:13:02.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.772+0000 7fed1d1e4640 1 -- 172.21.15.43:0/1554993528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed18199b70 con 0x7fed18108320 2024-09-11T23:13:02.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.772+0000 7fed1d1e4640 1 --2- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed18108320 0x7fed18199630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fed14030380 tx=0x7fed14032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:02.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.772+0000 7fed0e7fc640 1 -- 172.21.15.43:0/1554993528 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed14037040 con 0x7fed18108320 2024-09-11T23:13:02.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.772+0000 7fed1f448640 1 -- 172.21.15.43:0/1554993528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed18199e00 con 0x7fed18108320 2024-09-11T23:13:02.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.772+0000 7fed0e7fc640 1 -- 172.21.15.43:0/1554993528 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed14040070 con 0x7fed18108320 2024-09-11T23:13:02.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.772+0000 7fed0e7fc640 1 -- 172.21.15.43:0/1554993528 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed1403a7b0 con 0x7fed18108320 2024-09-11T23:13:02.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.773+0000 7fed1f448640 1 -- 172.21.15.43:0/1554993528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed1819caf0 con 0x7fed18108320 2024-09-11T23:13:02.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.773+0000 7fed0e7fc640 1 -- 172.21.15.43:0/1554993528 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fed1403a9e0 con 0x7fed18108320 2024-09-11T23:13:02.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.774+0000 7fed0e7fc640 1 --2- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fed0403d470 0x7fed0403f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:02.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.774+0000 7fed0e7fc640 1 -- 172.21.15.43:0/1554993528 --> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fed04040040 con 0x7fed0403d470 2024-09-11T23:13:02.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.774+0000 7fed0e7fc640 1 -- 172.21.15.43:0/1554993528 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fed1406c8b0 con 0x7fed18108320 2024-09-11T23:13:02.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.774+0000 7fed1c9e3640 1 -- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fed0403d470 msgr2=0x7fed0403f930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3892590401 2024-09-11T23:13:02.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.774+0000 7fed1c9e3640 1 --2- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fed0403d470 0x7fed0403f930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:13:02.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.975+0000 7fed1c9e3640 1 -- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fed0403d470 msgr2=0x7fed0403f930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3892590401 2024-09-11T23:13:02.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:57.975+0000 7fed1c9e3640 1 --2- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fed0403d470 0x7fed0403f930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:13:02.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:58.376+0000 7fed1c9e3640 1 -- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fed0403d470 msgr2=0x7fed0403f930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3892590401 2024-09-11T23:13:02.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:58.376+0000 7fed1c9e3640 1 --2- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fed0403d470 0x7fed0403f930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-11T23:13:02.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:59.177+0000 7fed1c9e3640 1 -- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fed0403d470 msgr2=0x7fed0403f930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/3892590401 2024-09-11T23:13:02.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:12:59.177+0000 7fed1c9e3640 1 --2- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fed0403d470 0x7fed0403f930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-11T23:13:02.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:00.549+0000 7fed0e7fc640 1 -- 172.21.15.43:0/1554993528 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7fed1406b360 con 0x7fed18108320 2024-09-11T23:13:02.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:00.549+0000 7fed0e7fc640 1 -- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fed0403d470 msgr2=0x7fed0403f930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:13:02.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:00.549+0000 7fed0e7fc640 1 --2- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fed0403d470 0x7fed0403f930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:02.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:01.551+0000 7fed0e7fc640 1 -- 172.21.15.43:0/1554993528 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fed14045980 con 0x7fed18108320 2024-09-11T23:13:02.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:01.551+0000 7fed0e7fc640 1 --2- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fed04040f30 0x7fed04043320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:02.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:01.551+0000 7fed0e7fc640 1 -- 172.21.15.43:0/1554993528 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fed1402b9f0 con 0x7fed04040f30 2024-09-11T23:13:02.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:01.552+0000 7fed1c9e3640 1 --2- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fed04040f30 0x7fed04043320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:02.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:01.552+0000 7fed1c9e3640 1 --2- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fed04040f30 0x7fed04043320 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fed08003980 tx=0x7fed080092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:02.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:01.553+0000 7fed0e7fc640 1 -- 172.21.15.43:0/1554993528 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fed1402b9f0 con 0x7fed04040f30 2024-09-11T23:13:02.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:01.565+0000 7fed1f448640 1 -- 172.21.15.43:0/1554993528 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fed18107620 con 0x7fed04040f30 2024-09-11T23:13:02.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:01.565+0000 7fed0e7fc640 1 -- 172.21.15.43:0/1554993528 <== mgr.14150 v2:172.21.15.43:6800/2370495020 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fed18107620 con 0x7fed04040f30 2024-09-11T23:13:02.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:01.566+0000 7fed1f448640 1 -- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fed04040f30 msgr2=0x7fed04043320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:02.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:01.566+0000 7fed1f448640 1 --2- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fed04040f30 0x7fed04043320 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fed08003980 tx=0x7fed080092b0 comp rx=0 tx=0).stop 2024-09-11T23:13:02.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:01.566+0000 7fed1f448640 1 -- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed18108320 msgr2=0x7fed18199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:02.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:01.566+0000 7fed1f448640 1 --2- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed18108320 0x7fed18199630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fed14030380 tx=0x7fed14032670 comp rx=0 tx=0).stop 2024-09-11T23:13:02.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:01.566+0000 7fed1f448640 1 -- 172.21.15.43:0/1554993528 shutdown_connections 2024-09-11T23:13:02.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:01.566+0000 7fed1f448640 1 --2- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fed04040f30 0x7fed04043320 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:02.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:01.566+0000 7fed1f448640 1 --2- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:6800/3892590401,v1:172.21.15.43:6801/3892590401] conn(0x7fed0403d470 0x7fed0403f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:02.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:01.566+0000 7fed1f448640 1 --2- 172.21.15.43:0/1554993528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed18108320 0x7fed18199630 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:02.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:01.566+0000 7fed1f448640 1 -- 172.21.15.43:0/1554993528 >> 172.21.15.43:0/1554993528 conn(0x7fed18101f20 msgr2=0x7fed18102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:02.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:01.566+0000 7fed1f448640 1 -- 172.21.15.43:0/1554993528 shutdown_connections 2024-09-11T23:13:02.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:01.566+0000 7fed1f448640 1 -- 172.21.15.43:0/1554993528 wait complete. 2024-09-11T23:13:02.479 INFO:teuthology.orchestra.run.smithi043.stdout:mgr epoch 9 is available 2024-09-11T23:13:02.479 INFO:teuthology.orchestra.run.smithi043.stdout:Generating a dashboard self-signed certificate... 2024-09-11T23:13:02.551 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:13:02.411+0000 7f5b06efc640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:23:13:02] ENGINE Error in HTTPServer.serve 2024-09-11T23:13:02.551 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: Traceback (most recent call last): 2024-09-11T23:13:02.551 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:13:02.552 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: self._connections.run(self.expiration_interval) 2024-09-11T23:13:02.552 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:13:02.552 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: self._run(expiration_interval) 2024-09-11T23:13:02.552 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:13:02.552 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:13:02.552 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:13:02.552 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:13:02.552 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:13:02.552 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: s = self.context.wrap_socket( 2024-09-11T23:13:02.552 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:13:02.552 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: return self.sslsocket_class._create( 2024-09-11T23:13:02.552 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:13:02.552 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: self.do_handshake() 2024-09-11T23:13:02.553 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:13:02.553 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: self._sslobj.do_handshake() 2024-09-11T23:13:02.553 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:13:02.553 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:13:02.831 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-mon[30950]: mgrmap e11: x(active, since 1.00585s) 2024-09-11T23:13:02.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:02.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-mon[30950]: [11/Sep/2024:23:13:02] ENGINE Bus STARTING 2024-09-11T23:13:02.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-mon[30950]: [11/Sep/2024:23:13:02] ENGINE Serving on https://172.21.15.43:7150 2024-09-11T23:13:02.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-mon[30950]: [11/Sep/2024:23:13:02] ENGINE Error in HTTPServer.serve 2024-09-11T23:13:02.832 INFO:journalctl@ceph.mon.a.smithi043.stdout: Traceback (most recent call last): 2024-09-11T23:13:02.832 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:13:02.832 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._connections.run(self.expiration_interval) 2024-09-11T23:13:02.832 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:13:02.832 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._run(expiration_interval) 2024-09-11T23:13:02.832 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:13:02.832 INFO:journalctl@ceph.mon.a.smithi043.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:13:02.833 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:13:02.833 INFO:journalctl@ceph.mon.a.smithi043.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:13:02.833 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:13:02.833 INFO:journalctl@ceph.mon.a.smithi043.stdout: s = self.context.wrap_socket( 2024-09-11T23:13:02.833 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:13:02.833 INFO:journalctl@ceph.mon.a.smithi043.stdout: return self.sslsocket_class._create( 2024-09-11T23:13:02.833 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:13:02.833 INFO:journalctl@ceph.mon.a.smithi043.stdout: self.do_handshake() 2024-09-11T23:13:02.833 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:13:02.833 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._sslobj.do_handshake() 2024-09-11T23:13:02.833 INFO:journalctl@ceph.mon.a.smithi043.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:13:02.833 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-mon[30950]: [11/Sep/2024:23:13:02] ENGINE Serving on http://172.21.15.43:8765 2024-09-11T23:13:02.833 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:02 smithi043 ceph-mon[30950]: [11/Sep/2024:23:13:02] ENGINE Bus STARTED 2024-09-11T23:13:03.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-11T23:13:03.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.913+0000 7f3ca979e640 1 Processor -- start 2024-09-11T23:13:03.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.913+0000 7f3ca979e640 1 -- start start 2024-09-11T23:13:03.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.913+0000 7f3ca979e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca41067b0 0x7f3ca4106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:03.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.913+0000 7f3ca979e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca4107180 con 0x7f3ca41067b0 2024-09-11T23:13:03.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.913+0000 7f3ca2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca41067b0 0x7f3ca4106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:03.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.913+0000 7f3ca2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca41067b0 0x7f3ca4106bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40908/0 (socket says 172.21.15.43:40908) 2024-09-11T23:13:03.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.913+0000 7f3ca2ffd640 1 -- 172.21.15.43:0/3991054353 learned_addr learned my addr 172.21.15.43:0/3991054353 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:13:03.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.914+0000 7f3ca2ffd640 1 -- 172.21.15.43:0/3991054353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca41079b0 con 0x7f3ca41067b0 2024-09-11T23:13:03.809 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.914+0000 7f3ca2ffd640 1 --2- 172.21.15.43:0/3991054353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca41067b0 0x7f3ca4106bb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009e80 tx=0x7f3c9402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1ae280b7aff758 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:03.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.914+0000 7f3ca1ffb640 1 -- 172.21.15.43:0/3991054353 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c94036070 con 0x7f3ca41067b0 2024-09-11T23:13:03.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.914+0000 7f3ca1ffb640 1 -- 172.21.15.43:0/3991054353 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c940318e0 con 0x7f3ca41067b0 2024-09-11T23:13:03.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.914+0000 7f3ca1ffb640 1 -- 172.21.15.43:0/3991054353 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c94007a00 con 0x7f3ca41067b0 2024-09-11T23:13:03.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.915+0000 7f3ca979e640 1 -- 172.21.15.43:0/3991054353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca41067b0 msgr2=0x7f3ca4106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:03.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.915+0000 7f3ca979e640 1 --2- 172.21.15.43:0/3991054353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca41067b0 0x7f3ca4106bb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009e80 tx=0x7f3c9402a7a0 comp rx=0 tx=0).stop 2024-09-11T23:13:03.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.915+0000 7f3ca979e640 1 -- 172.21.15.43:0/3991054353 shutdown_connections 2024-09-11T23:13:03.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.915+0000 7f3ca979e640 1 --2- 172.21.15.43:0/3991054353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca41067b0 0x7f3ca4106bb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:03.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.915+0000 7f3ca979e640 1 -- 172.21.15.43:0/3991054353 >> 172.21.15.43:0/3991054353 conn(0x7f3ca4101f20 msgr2=0x7f3ca4104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:03.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.915+0000 7f3ca979e640 1 -- 172.21.15.43:0/3991054353 shutdown_connections 2024-09-11T23:13:03.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.915+0000 7f3ca979e640 1 -- 172.21.15.43:0/3991054353 wait complete. 2024-09-11T23:13:03.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.916+0000 7f3ca979e640 1 Processor -- start 2024-09-11T23:13:03.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.916+0000 7f3ca979e640 1 -- start start 2024-09-11T23:13:03.810 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.916+0000 7f3ca979e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca41067b0 0x7f3ca419b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:03.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.916+0000 7f3ca979e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca410ad60 con 0x7f3ca41067b0 2024-09-11T23:13:03.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.916+0000 7f3ca2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca41067b0 0x7f3ca419b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:03.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.916+0000 7f3ca2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca41067b0 0x7f3ca419b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40922/0 (socket says 172.21.15.43:40922) 2024-09-11T23:13:03.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.916+0000 7f3ca2ffd640 1 -- 172.21.15.43:0/320367654 learned_addr learned my addr 172.21.15.43:0/320367654 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:13:03.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.916+0000 7f3ca2ffd640 1 -- 172.21.15.43:0/320367654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca419be20 con 0x7f3ca41067b0 2024-09-11T23:13:03.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.917+0000 7f3ca2ffd640 1 --2- 172.21.15.43:0/320367654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca41067b0 0x7f3ca419b8e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3c9402acc0 tx=0x7f3c94031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:03.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.917+0000 7f3c8bfff640 1 -- 172.21.15.43:0/320367654 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c94036050 con 0x7f3ca41067b0 2024-09-11T23:13:03.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.917+0000 7f3c8bfff640 1 -- 172.21.15.43:0/320367654 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c94037040 con 0x7f3ca41067b0 2024-09-11T23:13:03.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.917+0000 7f3ca979e640 1 -- 172.21.15.43:0/320367654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca419c0b0 con 0x7f3ca41067b0 2024-09-11T23:13:03.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.917+0000 7f3c8bfff640 1 -- 172.21.15.43:0/320367654 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c9403b890 con 0x7f3ca41067b0 2024-09-11T23:13:03.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.917+0000 7f3ca979e640 1 -- 172.21.15.43:0/320367654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca419eda0 con 0x7f3ca41067b0 2024-09-11T23:13:03.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.918+0000 7f3c8bfff640 1 -- 172.21.15.43:0/320367654 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f3c94044020 con 0x7f3ca41067b0 2024-09-11T23:13:03.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.918+0000 7f3c8bfff640 1 --2- 172.21.15.43:0/320367654 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3c8c039020 0x7f3c8c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:03.811 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.918+0000 7f3c8bfff640 1 -- 172.21.15.43:0/320367654 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3c94042020 con 0x7f3ca41067b0 2024-09-11T23:13:03.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.918+0000 7f3ca27fc640 1 --2- 172.21.15.43:0/320367654 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3c8c039020 0x7f3c8c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:03.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.918+0000 7f3ca979e640 1 -- 172.21.15.43:0/320367654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca410aee0 con 0x7f3ca41067b0 2024-09-11T23:13:03.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.918+0000 7f3ca27fc640 1 --2- 172.21.15.43:0/320367654 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3c8c039020 0x7f3c8c03b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3c980098c0 tx=0x7f3c98006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:03.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:02.923+0000 7f3c8bfff640 1 -- 172.21.15.43:0/320367654 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3c9404b020 con 0x7f3ca41067b0 2024-09-11T23:13:03.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:03.108+0000 7f3ca979e640 1 -- 172.21.15.43:0/320367654 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f3ca41031b0 con 0x7f3c8c039020 2024-09-11T23:13:03.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:03.221+0000 7f3c8bfff640 1 -- 172.21.15.43:0/320367654 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f3ca41031b0 con 0x7f3c8c039020 2024-09-11T23:13:03.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:03.223+0000 7f3ca979e640 1 -- 172.21.15.43:0/320367654 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3c8c039020 msgr2=0x7f3c8c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:03.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:03.223+0000 7f3ca979e640 1 --2- 172.21.15.43:0/320367654 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3c8c039020 0x7f3c8c03b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3c980098c0 tx=0x7f3c98006db0 comp rx=0 tx=0).stop 2024-09-11T23:13:03.812 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:03.223+0000 7f3ca979e640 1 -- 172.21.15.43:0/320367654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca41067b0 msgr2=0x7f3ca419b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:03.813 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:03.223+0000 7f3ca979e640 1 --2- 172.21.15.43:0/320367654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca41067b0 0x7f3ca419b8e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3c9402acc0 tx=0x7f3c94031cc0 comp rx=0 tx=0).stop 2024-09-11T23:13:03.813 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:03.224+0000 7f3ca979e640 1 -- 172.21.15.43:0/320367654 shutdown_connections 2024-09-11T23:13:03.813 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:03.224+0000 7f3ca979e640 1 --2- 172.21.15.43:0/320367654 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3c8c039020 0x7f3c8c03b4e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:03.813 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:03.224+0000 7f3ca979e640 1 --2- 172.21.15.43:0/320367654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca41067b0 0x7f3ca419b8e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:03.813 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:03.224+0000 7f3ca979e640 1 -- 172.21.15.43:0/320367654 >> 172.21.15.43:0/320367654 conn(0x7f3ca4101f20 msgr2=0x7f3ca4102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:03.813 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:03.224+0000 7f3ca979e640 1 -- 172.21.15.43:0/320367654 shutdown_connections 2024-09-11T23:13:03.813 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:03.224+0000 7f3ca979e640 1 -- 172.21.15.43:0/320367654 wait complete. 2024-09-11T23:13:03.813 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial admin user... 2024-09-11T23:13:04.196 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:03 smithi043 ceph-mon[30950]: mgrmap e12: x(active, since 2s) 2024-09-11T23:13:04.196 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:04.196 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:03 smithi043 ceph-mon[30950]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:13:04.196 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:04.196 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:05.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:04 smithi043 ceph-mon[30950]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:13:05.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:04 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:05.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$fuaBgI1DCzmhHnJuaSBfi.ssJ4.9rIZksLZmvJAdfU20rHJ8zoJI.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726096384, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-11T23:13:05.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.202+0000 7f1c82831640 1 Processor -- start 2024-09-11T23:13:05.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.202+0000 7f1c82831640 1 -- start start 2024-09-11T23:13:05.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.202+0000 7f1c82831640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c7c108410 0x7f1c7c10a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:05.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.202+0000 7f1c82831640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c07be10 con 0x7f1c7c108410 2024-09-11T23:13:05.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.202+0000 7f1c7bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c7c108410 0x7f1c7c10a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:05.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.202+0000 7f1c7bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c7c108410 0x7f1c7c10a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40938/0 (socket says 172.21.15.43:40938) 2024-09-11T23:13:05.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.202+0000 7f1c7bfff640 1 -- 172.21.15.43:0/2541099944 learned_addr learned my addr 172.21.15.43:0/2541099944 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:13:05.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.203+0000 7f1c7bfff640 1 -- 172.21.15.43:0/2541099944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7c10ad40 con 0x7f1c7c108410 2024-09-11T23:13:05.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.203+0000 7f1c7bfff640 1 --2- 172.21.15.43:0/2541099944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c7c108410 0x7f1c7c10a800 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009820 tx=0x7f1c6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2d610a07c780a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:05.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.203+0000 7f1c7affd640 1 -- 172.21.15.43:0/2541099944 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c6c037070 con 0x7f1c7c108410 2024-09-11T23:13:05.653 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.204+0000 7f1c7affd640 1 -- 172.21.15.43:0/2541099944 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c6c032920 con 0x7f1c7c108410 2024-09-11T23:13:05.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.204+0000 7f1c7affd640 1 -- 172.21.15.43:0/2541099944 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c6c030810 con 0x7f1c7c108410 2024-09-11T23:13:05.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.204+0000 7f1c82831640 1 -- 172.21.15.43:0/2541099944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c7c108410 msgr2=0x7f1c7c10a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:05.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.204+0000 7f1c82831640 1 --2- 172.21.15.43:0/2541099944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c7c108410 0x7f1c7c10a800 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009820 tx=0x7f1c6c02a270 comp rx=0 tx=0).stop 2024-09-11T23:13:05.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.204+0000 7f1c82831640 1 -- 172.21.15.43:0/2541099944 shutdown_connections 2024-09-11T23:13:05.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.204+0000 7f1c82831640 1 --2- 172.21.15.43:0/2541099944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c7c108410 0x7f1c7c10a800 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:05.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.204+0000 7f1c82831640 1 -- 172.21.15.43:0/2541099944 >> 172.21.15.43:0/2541099944 conn(0x7f1c7c102010 msgr2=0x7f1c7c104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:05.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.204+0000 7f1c82831640 1 -- 172.21.15.43:0/2541099944 shutdown_connections 2024-09-11T23:13:05.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.204+0000 7f1c82831640 1 -- 172.21.15.43:0/2541099944 wait complete. 2024-09-11T23:13:05.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.205+0000 7f1c82831640 1 Processor -- start 2024-09-11T23:13:05.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.205+0000 7f1c82831640 1 -- start start 2024-09-11T23:13:05.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.205+0000 7f1c82831640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c7c108410 0x7f1c7c107c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:05.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.205+0000 7f1c82831640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c10bcf0 con 0x7f1c7c108410 2024-09-11T23:13:05.654 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.205+0000 7f1c7bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c7c108410 0x7f1c7c107c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:05.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.205+0000 7f1c7bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c7c108410 0x7f1c7c107c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40942/0 (socket says 172.21.15.43:40942) 2024-09-11T23:13:05.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.205+0000 7f1c7bfff640 1 -- 172.21.15.43:0/1297600561 learned_addr learned my addr 172.21.15.43:0/1297600561 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:13:05.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.206+0000 7f1c7bfff640 1 -- 172.21.15.43:0/1297600561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7c1081d0 con 0x7f1c7c108410 2024-09-11T23:13:05.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.206+0000 7f1c7bfff640 1 --2- 172.21.15.43:0/1297600561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c7c108410 0x7f1c7c107c90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c030380 tx=0x7f1c6c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:05.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.206+0000 7f1c797fa640 1 -- 172.21.15.43:0/1297600561 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c6c037040 con 0x7f1c7c108410 2024-09-11T23:13:05.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.206+0000 7f1c82831640 1 -- 172.21.15.43:0/1297600561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c7c1063c0 con 0x7f1c7c108410 2024-09-11T23:13:05.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.206+0000 7f1c797fa640 1 -- 172.21.15.43:0/1297600561 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c6c040070 con 0x7f1c7c108410 2024-09-11T23:13:05.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.206+0000 7f1c797fa640 1 -- 172.21.15.43:0/1297600561 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c6c03a7b0 con 0x7f1c7c108410 2024-09-11T23:13:05.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.206+0000 7f1c82831640 1 -- 172.21.15.43:0/1297600561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c7c106900 con 0x7f1c7c108410 2024-09-11T23:13:05.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.207+0000 7f1c797fa640 1 -- 172.21.15.43:0/1297600561 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f1c6c03aa30 con 0x7f1c7c108410 2024-09-11T23:13:05.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.207+0000 7f1c82831640 1 -- 172.21.15.43:0/1297600561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c7c108870 con 0x7f1c7c108410 2024-09-11T23:13:05.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.207+0000 7f1c797fa640 1 --2- 172.21.15.43:0/1297600561 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f1c64038fc0 0x7f1c6403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:05.655 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.207+0000 7f1c797fa640 1 -- 172.21.15.43:0/1297600561 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1c6c06c9f0 con 0x7f1c7c108410 2024-09-11T23:13:05.656 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.207+0000 7f1c7b7fe640 1 --2- 172.21.15.43:0/1297600561 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f1c64038fc0 0x7f1c6403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:05.656 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.208+0000 7f1c7b7fe640 1 --2- 172.21.15.43:0/1297600561 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f1c64038fc0 0x7f1c6403b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1c700098c0 tx=0x7f1c70006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:05.656 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.212+0000 7f1c797fa640 1 -- 172.21.15.43:0/1297600561 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c6c044c70 con 0x7f1c7c108410 2024-09-11T23:13:05.656 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.403+0000 7f1c82831640 1 -- 172.21.15.43:0/1297600561 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f1c7c1070a0 con 0x7f1c64038fc0 2024-09-11T23:13:05.656 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.652+0000 7f1c797fa640 1 -- 172.21.15.43:0/1297600561 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f1c7c1070a0 con 0x7f1c64038fc0 2024-09-11T23:13:05.656 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.654+0000 7f1c82831640 1 -- 172.21.15.43:0/1297600561 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f1c64038fc0 msgr2=0x7f1c6403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:05.656 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.654+0000 7f1c82831640 1 --2- 172.21.15.43:0/1297600561 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f1c64038fc0 0x7f1c6403b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1c700098c0 tx=0x7f1c70006db0 comp rx=0 tx=0).stop 2024-09-11T23:13:05.656 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.654+0000 7f1c82831640 1 -- 172.21.15.43:0/1297600561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c7c108410 msgr2=0x7f1c7c107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:05.656 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.654+0000 7f1c82831640 1 --2- 172.21.15.43:0/1297600561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c7c108410 0x7f1c7c107c90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c030380 tx=0x7f1c6c032670 comp rx=0 tx=0).stop 2024-09-11T23:13:05.656 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.654+0000 7f1c82831640 1 -- 172.21.15.43:0/1297600561 shutdown_connections 2024-09-11T23:13:05.656 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.654+0000 7f1c82831640 1 --2- 172.21.15.43:0/1297600561 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f1c64038fc0 0x7f1c6403b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:05.656 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.654+0000 7f1c82831640 1 --2- 172.21.15.43:0/1297600561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c7c108410 0x7f1c7c107c90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:05.656 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.654+0000 7f1c82831640 1 -- 172.21.15.43:0/1297600561 >> 172.21.15.43:0/1297600561 conn(0x7f1c7c102010 msgr2=0x7f1c7c102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:05.656 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.654+0000 7f1c82831640 1 -- 172.21.15.43:0/1297600561 shutdown_connections 2024-09-11T23:13:05.657 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:04.654+0000 7f1c82831640 1 -- 172.21.15.43:0/1297600561 wait complete. 2024-09-11T23:13:05.657 INFO:teuthology.orchestra.run.smithi043.stdout:Fetching dashboard port number... 2024-09-11T23:13:07.278 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 8443 2024-09-11T23:13:07.278 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.488+0000 7f8b3c330640 1 Processor -- start 2024-09-11T23:13:07.278 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.488+0000 7f8b3c330640 1 -- start start 2024-09-11T23:13:07.278 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.488+0000 7f8b3c330640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b341067b0 0x7f8b34106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:07.278 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.488+0000 7f8b3c330640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8b34107180 con 0x7f8b341067b0 2024-09-11T23:13:07.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.489+0000 7f8b3a0cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b341067b0 0x7f8b34106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:07.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.489+0000 7f8b3a0cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b341067b0 0x7f8b34106bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40958/0 (socket says 172.21.15.43:40958) 2024-09-11T23:13:07.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.489+0000 7f8b3a0cc640 1 -- 172.21.15.43:0/3839092394 learned_addr learned my addr 172.21.15.43:0/3839092394 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:13:07.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.489+0000 7f8b3a0cc640 1 -- 172.21.15.43:0/3839092394 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b341079b0 con 0x7f8b341067b0 2024-09-11T23:13:07.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.490+0000 7f8b3a0cc640 1 --2- 172.21.15.43:0/3839092394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b341067b0 0x7f8b34106bb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009e80 tx=0x7f8b2402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62903460c87e32bd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:07.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.490+0000 7f8b390ca640 1 -- 172.21.15.43:0/3839092394 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b24036070 con 0x7f8b341067b0 2024-09-11T23:13:07.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.490+0000 7f8b390ca640 1 -- 172.21.15.43:0/3839092394 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b240318e0 con 0x7f8b341067b0 2024-09-11T23:13:07.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.490+0000 7f8b390ca640 1 -- 172.21.15.43:0/3839092394 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b24007a00 con 0x7f8b341067b0 2024-09-11T23:13:07.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.490+0000 7f8b3c330640 1 -- 172.21.15.43:0/3839092394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b341067b0 msgr2=0x7f8b34106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:07.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.490+0000 7f8b3c330640 1 --2- 172.21.15.43:0/3839092394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b341067b0 0x7f8b34106bb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009e80 tx=0x7f8b2402a7a0 comp rx=0 tx=0).stop 2024-09-11T23:13:07.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.491+0000 7f8b3c330640 1 -- 172.21.15.43:0/3839092394 shutdown_connections 2024-09-11T23:13:07.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.491+0000 7f8b3c330640 1 --2- 172.21.15.43:0/3839092394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b341067b0 0x7f8b34106bb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:07.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.491+0000 7f8b3c330640 1 -- 172.21.15.43:0/3839092394 >> 172.21.15.43:0/3839092394 conn(0x7f8b34101f20 msgr2=0x7f8b34104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:07.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.491+0000 7f8b3c330640 1 -- 172.21.15.43:0/3839092394 shutdown_connections 2024-09-11T23:13:07.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.491+0000 7f8b3c330640 1 -- 172.21.15.43:0/3839092394 wait complete. 2024-09-11T23:13:07.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.491+0000 7f8b3c330640 1 Processor -- start 2024-09-11T23:13:07.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.491+0000 7f8b3c330640 1 -- start start 2024-09-11T23:13:07.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.491+0000 7f8b3c330640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b341067b0 0x7f8b3419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:07.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.491+0000 7f8b3c330640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3410ad60 con 0x7f8b341067b0 2024-09-11T23:13:07.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.492+0000 7f8b3a0cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b341067b0 0x7f8b3419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:07.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.492+0000 7f8b3a0cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b341067b0 0x7f8b3419a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40960/0 (socket says 172.21.15.43:40960) 2024-09-11T23:13:07.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.492+0000 7f8b3a0cc640 1 -- 172.21.15.43:0/3851857257 learned_addr learned my addr 172.21.15.43:0/3851857257 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:13:07.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.492+0000 7f8b3a0cc640 1 -- 172.21.15.43:0/3851857257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3419ad20 con 0x7f8b341067b0 2024-09-11T23:13:07.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.492+0000 7f8b3a0cc640 1 --2- 172.21.15.43:0/3851857257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b341067b0 0x7f8b3419a7e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8b2402acc0 tx=0x7f8b24031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:07.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.492+0000 7f8b337fe640 1 -- 172.21.15.43:0/3851857257 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b24036050 con 0x7f8b341067b0 2024-09-11T23:13:07.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.492+0000 7f8b337fe640 1 -- 172.21.15.43:0/3851857257 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b24037040 con 0x7f8b341067b0 2024-09-11T23:13:07.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.492+0000 7f8b3c330640 1 -- 172.21.15.43:0/3851857257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b3419afb0 con 0x7f8b341067b0 2024-09-11T23:13:07.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.492+0000 7f8b337fe640 1 -- 172.21.15.43:0/3851857257 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b2403b890 con 0x7f8b341067b0 2024-09-11T23:13:07.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.492+0000 7f8b3c330640 1 -- 172.21.15.43:0/3851857257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b3419dca0 con 0x7f8b341067b0 2024-09-11T23:13:07.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.493+0000 7f8b337fe640 1 -- 172.21.15.43:0/3851857257 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f8b24044020 con 0x7f8b341067b0 2024-09-11T23:13:07.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.493+0000 7f8b3c330640 1 -- 172.21.15.43:0/3851857257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b3410aee0 con 0x7f8b341067b0 2024-09-11T23:13:07.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.493+0000 7f8b337fe640 1 --2- 172.21.15.43:0/3851857257 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f8b1c039020 0x7f8b1c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:07.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.493+0000 7f8b337fe640 1 -- 172.21.15.43:0/3851857257 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8b24042020 con 0x7f8b341067b0 2024-09-11T23:13:07.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.494+0000 7f8b398cb640 1 --2- 172.21.15.43:0/3851857257 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f8b1c039020 0x7f8b1c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:07.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.494+0000 7f8b398cb640 1 --2- 172.21.15.43:0/3851857257 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f8b1c039020 0x7f8b1c03b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8b280098c0 tx=0x7f8b28006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:07.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.498+0000 7f8b337fe640 1 -- 172.21.15.43:0/3851857257 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8b2404b020 con 0x7f8b341067b0 2024-09-11T23:13:07.282 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.602+0000 7f8b337fe640 1 -- 172.21.15.43:0/3851857257 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8b24037ce0 con 0x7f8b341067b0 2024-09-11T23:13:07.282 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.674+0000 7f8b3c330640 1 -- 172.21.15.43:0/3851857257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f8b3419de70 con 0x7f8b341067b0 2024-09-11T23:13:07.282 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.674+0000 7f8b337fe640 1 -- 172.21.15.43:0/3851857257 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f8b240432c0 con 0x7f8b341067b0 2024-09-11T23:13:07.282 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.676+0000 7f8b3c330640 1 -- 172.21.15.43:0/3851857257 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f8b1c039020 msgr2=0x7f8b1c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:07.282 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.676+0000 7f8b3c330640 1 --2- 172.21.15.43:0/3851857257 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f8b1c039020 0x7f8b1c03b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8b280098c0 tx=0x7f8b28006db0 comp rx=0 tx=0).stop 2024-09-11T23:13:07.282 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.676+0000 7f8b3c330640 1 -- 172.21.15.43:0/3851857257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b341067b0 msgr2=0x7f8b3419a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:07.282 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.676+0000 7f8b3c330640 1 --2- 172.21.15.43:0/3851857257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b341067b0 0x7f8b3419a7e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8b2402acc0 tx=0x7f8b24031cc0 comp rx=0 tx=0).stop 2024-09-11T23:13:07.282 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.676+0000 7f8b3c330640 1 -- 172.21.15.43:0/3851857257 shutdown_connections 2024-09-11T23:13:07.282 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.676+0000 7f8b3c330640 1 --2- 172.21.15.43:0/3851857257 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f8b1c039020 0x7f8b1c03b4e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:07.282 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.676+0000 7f8b3c330640 1 --2- 172.21.15.43:0/3851857257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b341067b0 0x7f8b3419a7e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:07.282 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.676+0000 7f8b3c330640 1 -- 172.21.15.43:0/3851857257 >> 172.21.15.43:0/3851857257 conn(0x7f8b34101f20 msgr2=0x7f8b34102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:07.282 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.676+0000 7f8b3c330640 1 -- 172.21.15.43:0/3851857257 shutdown_connections 2024-09-11T23:13:07.282 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:06.676+0000 7f8b3c330640 1 -- 172.21.15.43:0/3851857257 wait complete. 2024-09-11T23:13:07.287 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T23:13:07.288 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-09-11T23:13:07.296 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T23:13:07.296 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-11T23:13:07.296 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-09-11T23:13:07.296 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-11T23:13:07.298 INFO:teuthology.orchestra.run.smithi043.stdout:Ceph Dashboard is now available at: 2024-09-11T23:13:07.298 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:13:07.298 INFO:teuthology.orchestra.run.smithi043.stdout: URL: https://smithi043.front.sepia.ceph.com:8443/ 2024-09-11T23:13:07.298 INFO:teuthology.orchestra.run.smithi043.stdout: User: admin 2024-09-11T23:13:07.298 INFO:teuthology.orchestra.run.smithi043.stdout: Password: kwznpqhzdr 2024-09-11T23:13:07.298 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:13:07.299 INFO:teuthology.orchestra.run.smithi043.stdout:Saving cluster configuration to /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config directory 2024-09-11T23:13:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:07 smithi043 ceph-mon[30950]: mgrmap e13: x(active, since 6s) 2024-09-11T23:13:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:07 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3851857257' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-11T23:13:08.567 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.702+0000 7f107af46640 1 Processor -- start 2024-09-11T23:13:08.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.703+0000 7f107af46640 1 -- start start 2024-09-11T23:13:08.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.703+0000 7f107af46640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10741067b0 0x7f1074106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:08.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.703+0000 7f107af46640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1074107180 con 0x7f10741067b0 2024-09-11T23:13:08.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.703+0000 7f1078ce2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10741067b0 0x7f1074106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:08.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.703+0000 7f1078ce2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10741067b0 0x7f1074106bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40964/0 (socket says 172.21.15.43:40964) 2024-09-11T23:13:08.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.703+0000 7f1078ce2640 1 -- 172.21.15.43:0/481238195 learned_addr learned my addr 172.21.15.43:0/481238195 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:13:08.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.703+0000 7f1078ce2640 1 -- 172.21.15.43:0/481238195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10741079b0 con 0x7f10741067b0 2024-09-11T23:13:08.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.704+0000 7f1078ce2640 1 --2- 172.21.15.43:0/481238195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10741067b0 0x7f1074106bb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1064009820 tx=0x7f106402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8155dfb8558420 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:08.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.704+0000 7f10737fe640 1 -- 172.21.15.43:0/481238195 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1064037070 con 0x7f10741067b0 2024-09-11T23:13:08.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.704+0000 7f10737fe640 1 -- 172.21.15.43:0/481238195 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1064032920 con 0x7f10741067b0 2024-09-11T23:13:08.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.704+0000 7f10737fe640 1 -- 172.21.15.43:0/481238195 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1064030810 con 0x7f10741067b0 2024-09-11T23:13:08.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.705+0000 7f107af46640 1 -- 172.21.15.43:0/481238195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10741067b0 msgr2=0x7f1074106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:08.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.705+0000 7f107af46640 1 --2- 172.21.15.43:0/481238195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10741067b0 0x7f1074106bb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1064009820 tx=0x7f106402a270 comp rx=0 tx=0).stop 2024-09-11T23:13:08.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.705+0000 7f107af46640 1 -- 172.21.15.43:0/481238195 shutdown_connections 2024-09-11T23:13:08.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.705+0000 7f107af46640 1 --2- 172.21.15.43:0/481238195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10741067b0 0x7f1074106bb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:08.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.705+0000 7f107af46640 1 -- 172.21.15.43:0/481238195 >> 172.21.15.43:0/481238195 conn(0x7f1074101f20 msgr2=0x7f1074104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:08.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.705+0000 7f107af46640 1 -- 172.21.15.43:0/481238195 shutdown_connections 2024-09-11T23:13:08.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.705+0000 7f107af46640 1 -- 172.21.15.43:0/481238195 wait complete. 2024-09-11T23:13:08.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.706+0000 7f107af46640 1 Processor -- start 2024-09-11T23:13:08.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.706+0000 7f107af46640 1 -- start start 2024-09-11T23:13:08.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.706+0000 7f107af46640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10741067b0 0x7f10741919e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:08.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.706+0000 7f107af46640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f107410ad60 con 0x7f10741067b0 2024-09-11T23:13:08.569 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.706+0000 7f1078ce2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10741067b0 0x7f10741919e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:08.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.706+0000 7f1078ce2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10741067b0 0x7f10741919e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40972/0 (socket says 172.21.15.43:40972) 2024-09-11T23:13:08.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.706+0000 7f1078ce2640 1 -- 172.21.15.43:0/1256456891 learned_addr learned my addr 172.21.15.43:0/1256456891 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:13:08.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.706+0000 7f1078ce2640 1 -- 172.21.15.43:0/1256456891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1074191f20 con 0x7f10741067b0 2024-09-11T23:13:08.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.706+0000 7f1078ce2640 1 --2- 172.21.15.43:0/1256456891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10741067b0 0x7f10741919e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f10640097f0 tx=0x7f10640327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:08.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.707+0000 7f1071ffb640 1 -- 172.21.15.43:0/1256456891 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1064037040 con 0x7f10741067b0 2024-09-11T23:13:08.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.707+0000 7f1071ffb640 1 -- 172.21.15.43:0/1256456891 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1064030cc0 con 0x7f10741067b0 2024-09-11T23:13:08.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.707+0000 7f107af46640 1 -- 172.21.15.43:0/1256456891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1074190110 con 0x7f10741067b0 2024-09-11T23:13:08.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.707+0000 7f1071ffb640 1 -- 172.21.15.43:0/1256456891 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f106403a640 con 0x7f10741067b0 2024-09-11T23:13:08.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.707+0000 7f107af46640 1 -- 172.21.15.43:0/1256456891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1074190650 con 0x7f10741067b0 2024-09-11T23:13:08.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.708+0000 7f1071ffb640 1 -- 172.21.15.43:0/1256456891 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f106403b3f0 con 0x7f10741067b0 2024-09-11T23:13:08.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.708+0000 7f107af46640 1 -- 172.21.15.43:0/1256456891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f107410aee0 con 0x7f10741067b0 2024-09-11T23:13:08.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.708+0000 7f1071ffb640 1 --2- 172.21.15.43:0/1256456891 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f105c0390c0 0x7f105c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:08.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.708+0000 7f1071ffb640 1 -- 172.21.15.43:0/1256456891 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f106406c6a0 con 0x7f10741067b0 2024-09-11T23:13:08.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.708+0000 7f1073fff640 1 --2- 172.21.15.43:0/1256456891 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f105c0390c0 0x7f105c03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:08.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.709+0000 7f1073fff640 1 --2- 172.21.15.43:0/1256456891 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f105c0390c0 0x7f105c03b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f10680098c0 tx=0x7f1068006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:08.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.713+0000 7f1071ffb640 1 -- 172.21.15.43:0/1256456891 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1064041710 con 0x7f10741067b0 2024-09-11T23:13:08.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.984+0000 7f107af46640 1 -- 172.21.15.43:0/1256456891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f1074106c30 con 0x7f10741067b0 2024-09-11T23:13:08.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.986+0000 7f1071ffb640 1 -- 172.21.15.43:0/1256456891 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f10640328c0 con 0x7f10741067b0 2024-09-11T23:13:08.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-11T23:13:08.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.988+0000 7f107af46640 1 -- 172.21.15.43:0/1256456891 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f105c0390c0 msgr2=0x7f105c03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:08.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.988+0000 7f107af46640 1 --2- 172.21.15.43:0/1256456891 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f105c0390c0 0x7f105c03b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f10680098c0 tx=0x7f1068006db0 comp rx=0 tx=0).stop 2024-09-11T23:13:08.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.988+0000 7f107af46640 1 -- 172.21.15.43:0/1256456891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10741067b0 msgr2=0x7f10741919e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:08.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.988+0000 7f107af46640 1 --2- 172.21.15.43:0/1256456891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10741067b0 0x7f10741919e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f10640097f0 tx=0x7f10640327b0 comp rx=0 tx=0).stop 2024-09-11T23:13:08.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.989+0000 7f107af46640 1 -- 172.21.15.43:0/1256456891 shutdown_connections 2024-09-11T23:13:08.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.989+0000 7f107af46640 1 --2- 172.21.15.43:0/1256456891 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f105c0390c0 0x7f105c03b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:08.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.989+0000 7f107af46640 1 --2- 172.21.15.43:0/1256456891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10741067b0 0x7f10741919e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:08.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.989+0000 7f107af46640 1 -- 172.21.15.43:0/1256456891 >> 172.21.15.43:0/1256456891 conn(0x7f1074101f20 msgr2=0x7f1074102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:08.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.989+0000 7f107af46640 1 -- 172.21.15.43:0/1256456891 shutdown_connections 2024-09-11T23:13:08.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-09-11T23:13:07.989+0000 7f107af46640 1 -- 172.21.15.43:0/1256456891 wait complete. 2024-09-11T23:13:08.572 INFO:teuthology.orchestra.run.smithi043.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-11T23:13:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:13:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:13:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:13:08.572 INFO:teuthology.orchestra.run.smithi043.stdout:Or, if you are only running a single cluster on this host: 2024-09-11T23:13:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:13:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-11T23:13:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:13:08.572 INFO:teuthology.orchestra.run.smithi043.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-11T23:13:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:13:08.572 INFO:teuthology.orchestra.run.smithi043.stdout: ceph telemetry on 2024-09-11T23:13:08.573 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:13:08.573 INFO:teuthology.orchestra.run.smithi043.stdout:For more information see: 2024-09-11T23:13:08.573 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:13:08.573 INFO:teuthology.orchestra.run.smithi043.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-11T23:13:08.573 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:13:08.573 INFO:teuthology.orchestra.run.smithi043.stdout:Bootstrap complete. 2024-09-11T23:13:08.614 INFO:tasks.cephadm:Fetching config... 2024-09-11T23:13:08.615 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T23:13:08.615 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-11T23:13:08.633 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-11T23:13:08.633 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T23:13:08.633 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-11T23:13:08.689 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-11T23:13:08.689 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T23:13:08.690 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-11T23:13:08.757 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-11T23:13:08.757 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T23:13:08.757 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-11T23:13:08.813 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-11T23:13:08.814 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCf4yzk7yyVuIOOSoconqtOAlG4SacTasG8/L3mcrdN59G6eiTu4208sTSTDNJfK34+kMiHwTK3E8GxzmaHxFd7JpauoqaMo1BlY7wTl0syU2C9sZmM880qIlk2mQGwwhOzUgBogizeS9cmeiagvYqW+gBgGdrh+lj1gq0s5HqM5lfgEO8uD/FAV8Sj23GfkH/al1eMMp2kOJ3lNfVI+Gr4Pvk3jhQacTUy40MLcpCoeF7FKgQsL1VlBQFx+kDZJbzCXpvzVFrfiWSp4K0PwMdWQzMUOitoOp8kog3AlcFiYC3CTYZYXM48o+5ggeKTTj1zgVwoN+H39bbut4SMBmnWSxWXvCgZ7Hd6RXq3sxR8+MAt0ueCAt5sEXCdOI1tdzpd/kioQFvsQ8coVNZXe5m/JadsOulWwFSfofKPPkmAjuyXwlBR+3HcNi8YzR4DaQbybqWuTTQc9iCe693lR8puuvLONr7QUG1hqqWgwYvQxvyPqktmrWKpQpRhaTjZPsk= ceph-25c0cd84-7093-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T23:13:08.889 INFO:teuthology.orchestra.run.smithi043.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCf4yzk7yyVuIOOSoconqtOAlG4SacTasG8/L3mcrdN59G6eiTu4208sTSTDNJfK34+kMiHwTK3E8GxzmaHxFd7JpauoqaMo1BlY7wTl0syU2C9sZmM880qIlk2mQGwwhOzUgBogizeS9cmeiagvYqW+gBgGdrh+lj1gq0s5HqM5lfgEO8uD/FAV8Sj23GfkH/al1eMMp2kOJ3lNfVI+Gr4Pvk3jhQacTUy40MLcpCoeF7FKgQsL1VlBQFx+kDZJbzCXpvzVFrfiWSp4K0PwMdWQzMUOitoOp8kog3AlcFiYC3CTYZYXM48o+5ggeKTTj1zgVwoN+H39bbut4SMBmnWSxWXvCgZ7Hd6RXq3sxR8+MAt0ueCAt5sEXCdOI1tdzpd/kioQFvsQ8coVNZXe5m/JadsOulWwFSfofKPPkmAjuyXwlBR+3HcNi8YzR4DaQbybqWuTTQc9iCe693lR8puuvLONr7QUG1hqqWgwYvQxvyPqktmrWKpQpRhaTjZPsk= ceph-25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:13:08.901 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCf4yzk7yyVuIOOSoconqtOAlG4SacTasG8/L3mcrdN59G6eiTu4208sTSTDNJfK34+kMiHwTK3E8GxzmaHxFd7JpauoqaMo1BlY7wTl0syU2C9sZmM880qIlk2mQGwwhOzUgBogizeS9cmeiagvYqW+gBgGdrh+lj1gq0s5HqM5lfgEO8uD/FAV8Sj23GfkH/al1eMMp2kOJ3lNfVI+Gr4Pvk3jhQacTUy40MLcpCoeF7FKgQsL1VlBQFx+kDZJbzCXpvzVFrfiWSp4K0PwMdWQzMUOitoOp8kog3AlcFiYC3CTYZYXM48o+5ggeKTTj1zgVwoN+H39bbut4SMBmnWSxWXvCgZ7Hd6RXq3sxR8+MAt0ueCAt5sEXCdOI1tdzpd/kioQFvsQ8coVNZXe5m/JadsOulWwFSfofKPPkmAjuyXwlBR+3HcNi8YzR4DaQbybqWuTTQc9iCe693lR8puuvLONr7QUG1hqqWgwYvQxvyPqktmrWKpQpRhaTjZPsk= ceph-25c0cd84-7093-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T23:13:08.942 INFO:teuthology.orchestra.run.smithi090.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCf4yzk7yyVuIOOSoconqtOAlG4SacTasG8/L3mcrdN59G6eiTu4208sTSTDNJfK34+kMiHwTK3E8GxzmaHxFd7JpauoqaMo1BlY7wTl0syU2C9sZmM880qIlk2mQGwwhOzUgBogizeS9cmeiagvYqW+gBgGdrh+lj1gq0s5HqM5lfgEO8uD/FAV8Sj23GfkH/al1eMMp2kOJ3lNfVI+Gr4Pvk3jhQacTUy40MLcpCoeF7FKgQsL1VlBQFx+kDZJbzCXpvzVFrfiWSp4K0PwMdWQzMUOitoOp8kog3AlcFiYC3CTYZYXM48o+5ggeKTTj1zgVwoN+H39bbut4SMBmnWSxWXvCgZ7Hd6RXq3sxR8+MAt0ueCAt5sEXCdOI1tdzpd/kioQFvsQ8coVNZXe5m/JadsOulWwFSfofKPPkmAjuyXwlBR+3HcNi8YzR4DaQbybqWuTTQc9iCe693lR8puuvLONr7QUG1hqqWgwYvQxvyPqktmrWKpQpRhaTjZPsk= ceph-25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:13:08.955 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCf4yzk7yyVuIOOSoconqtOAlG4SacTasG8/L3mcrdN59G6eiTu4208sTSTDNJfK34+kMiHwTK3E8GxzmaHxFd7JpauoqaMo1BlY7wTl0syU2C9sZmM880qIlk2mQGwwhOzUgBogizeS9cmeiagvYqW+gBgGdrh+lj1gq0s5HqM5lfgEO8uD/FAV8Sj23GfkH/al1eMMp2kOJ3lNfVI+Gr4Pvk3jhQacTUy40MLcpCoeF7FKgQsL1VlBQFx+kDZJbzCXpvzVFrfiWSp4K0PwMdWQzMUOitoOp8kog3AlcFiYC3CTYZYXM48o+5ggeKTTj1zgVwoN+H39bbut4SMBmnWSxWXvCgZ7Hd6RXq3sxR8+MAt0ueCAt5sEXCdOI1tdzpd/kioQFvsQ8coVNZXe5m/JadsOulWwFSfofKPPkmAjuyXwlBR+3HcNi8YzR4DaQbybqWuTTQc9iCe693lR8puuvLONr7QUG1hqqWgwYvQxvyPqktmrWKpQpRhaTjZPsk= ceph-25c0cd84-7093-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T23:13:08.996 INFO:teuthology.orchestra.run.smithi097.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCf4yzk7yyVuIOOSoconqtOAlG4SacTasG8/L3mcrdN59G6eiTu4208sTSTDNJfK34+kMiHwTK3E8GxzmaHxFd7JpauoqaMo1BlY7wTl0syU2C9sZmM880qIlk2mQGwwhOzUgBogizeS9cmeiagvYqW+gBgGdrh+lj1gq0s5HqM5lfgEO8uD/FAV8Sj23GfkH/al1eMMp2kOJ3lNfVI+Gr4Pvk3jhQacTUy40MLcpCoeF7FKgQsL1VlBQFx+kDZJbzCXpvzVFrfiWSp4K0PwMdWQzMUOitoOp8kog3AlcFiYC3CTYZYXM48o+5ggeKTTj1zgVwoN+H39bbut4SMBmnWSxWXvCgZ7Hd6RXq3sxR8+MAt0ueCAt5sEXCdOI1tdzpd/kioQFvsQ8coVNZXe5m/JadsOulWwFSfofKPPkmAjuyXwlBR+3HcNi8YzR4DaQbybqWuTTQc9iCe693lR8puuvLONr7QUG1hqqWgwYvQxvyPqktmrWKpQpRhaTjZPsk= ceph-25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:13:09.009 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-11T23:13:09.163 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:08 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1256456891' entity='client.admin' 2024-09-11T23:13:09.263 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:13:09.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.776+0000 7f423bfad640 1 -- 172.21.15.43:0/2111955830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42341004e0 msgr2=0x7f42341008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:09.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.776+0000 7f423bfad640 1 --2- 172.21.15.43:0/2111955830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42341004e0 0x7f42341008e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4230009880 tx=0x7f423002a5d0 comp rx=0 tx=0).stop 2024-09-11T23:13:09.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.776+0000 7f423bfad640 1 -- 172.21.15.43:0/2111955830 shutdown_connections 2024-09-11T23:13:09.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.776+0000 7f423bfad640 1 --2- 172.21.15.43:0/2111955830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42341004e0 0x7f42341008e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:09.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.776+0000 7f423bfad640 1 -- 172.21.15.43:0/2111955830 >> 172.21.15.43:0/2111955830 conn(0x7f42340fbc90 msgr2=0x7f42340fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:09.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.776+0000 7f423bfad640 1 -- 172.21.15.43:0/2111955830 shutdown_connections 2024-09-11T23:13:09.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.777+0000 7f423bfad640 1 -- 172.21.15.43:0/2111955830 wait complete. 2024-09-11T23:13:09.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.777+0000 7f423bfad640 1 Processor -- start 2024-09-11T23:13:09.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.777+0000 7f423bfad640 1 -- start start 2024-09-11T23:13:09.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.777+0000 7f423bfad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42340fdac0 0x7f42340fc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:09.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.777+0000 7f423bfad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f42341065f0 con 0x7f42340fdac0 2024-09-11T23:13:09.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.777+0000 7f4239d49640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42340fdac0 0x7f42340fc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:09.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.778+0000 7f4239d49640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42340fdac0 0x7f42340fc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40982/0 (socket says 172.21.15.43:40982) 2024-09-11T23:13:09.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.778+0000 7f4239d49640 1 -- 172.21.15.43:0/1836521338 learned_addr learned my addr 172.21.15.43:0/1836521338 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:13:09.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.778+0000 7f4239d49640 1 -- 172.21.15.43:0/1836521338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42340fdee0 con 0x7f42340fdac0 2024-09-11T23:13:09.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.778+0000 7f4239d49640 1 --2- 172.21.15.43:0/1836521338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42340fdac0 0x7f42340fc230 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4230004f60 tx=0x7f42300048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:09.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.778+0000 7f422affd640 1 -- 172.21.15.43:0/1836521338 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f423003f070 con 0x7f42340fdac0 2024-09-11T23:13:09.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.778+0000 7f423bfad640 1 -- 172.21.15.43:0/1836521338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42340fc770 con 0x7f42340fdac0 2024-09-11T23:13:09.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.778+0000 7f422affd640 1 -- 172.21.15.43:0/1836521338 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4230004c70 con 0x7f42340fdac0 2024-09-11T23:13:09.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.778+0000 7f422affd640 1 -- 172.21.15.43:0/1836521338 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4230035030 con 0x7f42340fdac0 2024-09-11T23:13:09.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.779+0000 7f423bfad640 1 -- 172.21.15.43:0/1836521338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42340fcc50 con 0x7f42340fdac0 2024-09-11T23:13:09.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.780+0000 7f423bfad640 1 -- 172.21.15.43:0/1836521338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42340fe0b0 con 0x7f42340fdac0 2024-09-11T23:13:09.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.781+0000 7f422affd640 1 -- 172.21.15.43:0/1836521338 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4230042070 con 0x7f42340fdac0 2024-09-11T23:13:09.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.781+0000 7f422affd640 1 --2- 172.21.15.43:0/1836521338 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f42200390c0 0x7f422003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:09.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.781+0000 7f422affd640 1 -- 172.21.15.43:0/1836521338 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f423006cca0 con 0x7f42340fdac0 2024-09-11T23:13:09.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.781+0000 7f4239548640 1 --2- 172.21.15.43:0/1836521338 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f42200390c0 0x7f422003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:09.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.781+0000 7f4239548640 1 --2- 172.21.15.43:0/1836521338 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f42200390c0 0x7f422003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f42240098c0 tx=0x7f4224006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:09.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.785+0000 7f422affd640 1 -- 172.21.15.43:0/1836521338 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4230042360 con 0x7f42340fdac0 2024-09-11T23:13:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.965+0000 7f423bfad640 1 -- 172.21.15.43:0/1836521338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f42341008e0 con 0x7f42340fdac0 2024-09-11T23:13:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.967+0000 7f422affd640 1 -- 172.21.15.43:0/1836521338 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f423002f300 con 0x7f42340fdac0 2024-09-11T23:13:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.970+0000 7f423bfad640 1 -- 172.21.15.43:0/1836521338 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f42200390c0 msgr2=0x7f422003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.970+0000 7f423bfad640 1 --2- 172.21.15.43:0/1836521338 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f42200390c0 0x7f422003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f42240098c0 tx=0x7f4224006db0 comp rx=0 tx=0).stop 2024-09-11T23:13:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.970+0000 7f423bfad640 1 -- 172.21.15.43:0/1836521338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42340fdac0 msgr2=0x7f42340fc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.970+0000 7f423bfad640 1 --2- 172.21.15.43:0/1836521338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42340fdac0 0x7f42340fc230 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4230004f60 tx=0x7f42300048f0 comp rx=0 tx=0).stop 2024-09-11T23:13:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.970+0000 7f423bfad640 1 -- 172.21.15.43:0/1836521338 shutdown_connections 2024-09-11T23:13:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.970+0000 7f423bfad640 1 --2- 172.21.15.43:0/1836521338 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f42200390c0 0x7f422003b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.970+0000 7f423bfad640 1 --2- 172.21.15.43:0/1836521338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42340fdac0 0x7f42340fc230 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.970+0000 7f423bfad640 1 -- 172.21.15.43:0/1836521338 >> 172.21.15.43:0/1836521338 conn(0x7f42340fbc90 msgr2=0x7f4234069eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.970+0000 7f423bfad640 1 -- 172.21.15.43:0/1836521338 shutdown_connections 2024-09-11T23:13:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:09.970+0000 7f423bfad640 1 -- 172.21.15.43:0/1836521338 wait complete. 2024-09-11T23:13:10.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:10 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1836521338' entity='client.admin' 2024-09-11T23:13:11.023 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-11T23:13:11.023 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-11T23:13:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:13:12.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.044+0000 7f39f5747640 1 -- 172.21.15.43:0/784739508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39f01015e0 msgr2=0x7f39f01019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:12.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.044+0000 7f39f5747640 1 --2- 172.21.15.43:0/784739508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39f01015e0 0x7f39f01019e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f39e0009880 tx=0x7f39e002a5d0 comp rx=0 tx=0).stop 2024-09-11T23:13:12.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.044+0000 7f39f5747640 1 -- 172.21.15.43:0/784739508 shutdown_connections 2024-09-11T23:13:12.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.044+0000 7f39f5747640 1 --2- 172.21.15.43:0/784739508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39f01015e0 0x7f39f01019e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:12.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.044+0000 7f39f5747640 1 -- 172.21.15.43:0/784739508 >> 172.21.15.43:0/784739508 conn(0x7f39f00fcd70 msgr2=0x7f39f00ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:12.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.044+0000 7f39f5747640 1 -- 172.21.15.43:0/784739508 shutdown_connections 2024-09-11T23:13:12.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.044+0000 7f39f5747640 1 -- 172.21.15.43:0/784739508 wait complete. 2024-09-11T23:13:12.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.045+0000 7f39f5747640 1 Processor -- start 2024-09-11T23:13:12.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.045+0000 7f39f5747640 1 -- start start 2024-09-11T23:13:12.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.045+0000 7f39f5747640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39f01015e0 0x7f39f0198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:12.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.045+0000 7f39f5747640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f39f01076f0 con 0x7f39f01015e0 2024-09-11T23:13:12.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.045+0000 7f39eeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39f01015e0 0x7f39f0198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:12.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.045+0000 7f39eeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39f01015e0 0x7f39f0198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37300/0 (socket says 172.21.15.43:37300) 2024-09-11T23:13:12.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.045+0000 7f39eeffd640 1 -- 172.21.15.43:0/1802535040 learned_addr learned my addr 172.21.15.43:0/1802535040 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:13:12.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.046+0000 7f39eeffd640 1 -- 172.21.15.43:0/1802535040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f01994a0 con 0x7f39f01015e0 2024-09-11T23:13:12.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.046+0000 7f39eeffd640 1 --2- 172.21.15.43:0/1802535040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39f01015e0 0x7f39f0198f60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f39e0004f60 tx=0x7f39e00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:12.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.046+0000 7f39d7fff640 1 -- 172.21.15.43:0/1802535040 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39e003f070 con 0x7f39f01015e0 2024-09-11T23:13:12.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.046+0000 7f39d7fff640 1 -- 172.21.15.43:0/1802535040 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e0004d40 con 0x7f39f01015e0 2024-09-11T23:13:12.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.046+0000 7f39f5747640 1 -- 172.21.15.43:0/1802535040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f0199730 con 0x7f39f01015e0 2024-09-11T23:13:12.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.047+0000 7f39d7fff640 1 -- 172.21.15.43:0/1802535040 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39e0035030 con 0x7f39f01015e0 2024-09-11T23:13:12.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.047+0000 7f39f5747640 1 -- 172.21.15.43:0/1802535040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f019c420 con 0x7f39f01015e0 2024-09-11T23:13:12.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.047+0000 7f39d7fff640 1 -- 172.21.15.43:0/1802535040 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f39e0042070 con 0x7f39f01015e0 2024-09-11T23:13:12.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.047+0000 7f39d7fff640 1 --2- 172.21.15.43:0/1802535040 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f39d8039060 0x7f39d803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:12.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.047+0000 7f39d7fff640 1 -- 172.21.15.43:0/1802535040 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f39e0073060 con 0x7f39f01015e0 2024-09-11T23:13:12.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.047+0000 7f39ee7fc640 1 --2- 172.21.15.43:0/1802535040 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f39d8039060 0x7f39d803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:12.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.048+0000 7f39ee7fc640 1 --2- 172.21.15.43:0/1802535040 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f39d8039060 0x7f39d803b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f39e40098c0 tx=0x7f39e4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:12.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.048+0000 7f39f5747640 1 -- 172.21.15.43:0/1802535040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f0107e70 con 0x7f39f01015e0 2024-09-11T23:13:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.053+0000 7f39d7fff640 1 -- 172.21.15.43:0/1802535040 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39e0043ce0 con 0x7f39f01015e0 2024-09-11T23:13:12.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.237+0000 7f39f5747640 1 -- 172.21.15.43:0/1802535040 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f39f0062630 con 0x7f39d8039060 2024-09-11T23:13:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.241+0000 7f39d7fff640 1 -- 172.21.15.43:0/1802535040 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f39f0062630 con 0x7f39d8039060 2024-09-11T23:13:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.243+0000 7f39f5747640 1 -- 172.21.15.43:0/1802535040 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f39d8039060 msgr2=0x7f39d803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.243+0000 7f39f5747640 1 --2- 172.21.15.43:0/1802535040 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f39d8039060 0x7f39d803b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f39e40098c0 tx=0x7f39e4006db0 comp rx=0 tx=0).stop 2024-09-11T23:13:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.243+0000 7f39f5747640 1 -- 172.21.15.43:0/1802535040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39f01015e0 msgr2=0x7f39f0198f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.243+0000 7f39f5747640 1 --2- 172.21.15.43:0/1802535040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39f01015e0 0x7f39f0198f60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f39e0004f60 tx=0x7f39e00048f0 comp rx=0 tx=0).stop 2024-09-11T23:13:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.243+0000 7f39f5747640 1 -- 172.21.15.43:0/1802535040 shutdown_connections 2024-09-11T23:13:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.243+0000 7f39f5747640 1 --2- 172.21.15.43:0/1802535040 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f39d8039060 0x7f39d803b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.243+0000 7f39f5747640 1 --2- 172.21.15.43:0/1802535040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39f01015e0 0x7f39f0198f60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.243+0000 7f39f5747640 1 -- 172.21.15.43:0/1802535040 >> 172.21.15.43:0/1802535040 conn(0x7f39f00fcd70 msgr2=0x7f39f00ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.243+0000 7f39f5747640 1 -- 172.21.15.43:0/1802535040 shutdown_connections 2024-09-11T23:13:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:12.243+0000 7f39f5747640 1 -- 172.21.15.43:0/1802535040 wait complete. 2024-09-11T23:13:12.831 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi090 2024-09-11T23:13:12.831 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-11T23:13:12.831 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/etc/ceph/ceph.conf 2024-09-11T23:13:12.850 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-11T23:13:12.850 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:13:12.909 INFO:tasks.cephadm:Adding host smithi090 to orchestrator... 2024-09-11T23:13:12.909 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph orch host add smithi090 2024-09-11T23:13:13.271 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:13:13.447 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:13 smithi043 ceph-mon[30950]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:13:13.447 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:13 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:13.447 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:13 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:13.447 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:13 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:13.448 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:13 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi043", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:13.448 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:13 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:13.448 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:13 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:13.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.812+0000 7f5bf8d63640 1 -- 172.21.15.43:0/3379795818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bf4100510 msgr2=0x7f5bf4100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:13.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.812+0000 7f5bf8d63640 1 --2- 172.21.15.43:0/3379795818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bf4100510 0x7f5bf4100910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5be4009880 tx=0x7f5be402a5d0 comp rx=0 tx=0).stop 2024-09-11T23:13:13.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.813+0000 7f5bf8d63640 1 -- 172.21.15.43:0/3379795818 shutdown_connections 2024-09-11T23:13:13.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.813+0000 7f5bf8d63640 1 --2- 172.21.15.43:0/3379795818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bf4100510 0x7f5bf4100910 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:13.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.813+0000 7f5bf8d63640 1 -- 172.21.15.43:0/3379795818 >> 172.21.15.43:0/3379795818 conn(0x7f5bf40fbca0 msgr2=0x7f5bf40fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:13.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.813+0000 7f5bf8d63640 1 -- 172.21.15.43:0/3379795818 shutdown_connections 2024-09-11T23:13:13.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.813+0000 7f5bf8d63640 1 -- 172.21.15.43:0/3379795818 wait complete. 2024-09-11T23:13:13.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.813+0000 7f5bf8d63640 1 Processor -- start 2024-09-11T23:13:13.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.814+0000 7f5bf8d63640 1 -- start start 2024-09-11T23:13:13.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.814+0000 7f5bf8d63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bf4100510 0x7f5bf406a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:13.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.814+0000 7f5bf8d63640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf4106620 con 0x7f5bf4100510 2024-09-11T23:13:13.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.814+0000 7f5bf259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bf4100510 0x7f5bf406a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:13.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.814+0000 7f5bf259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bf4100510 0x7f5bf406a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37328/0 (socket says 172.21.15.43:37328) 2024-09-11T23:13:13.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.814+0000 7f5bf259c640 1 -- 172.21.15.43:0/3225746435 learned_addr learned my addr 172.21.15.43:0/3225746435 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:13:13.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.814+0000 7f5bf259c640 1 -- 172.21.15.43:0/3225746435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf4199a80 con 0x7f5bf4100510 2024-09-11T23:13:13.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.814+0000 7f5bf259c640 1 --2- 172.21.15.43:0/3225746435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bf4100510 0x7f5bf406a990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5be4004f60 tx=0x7f5be40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:13.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.814+0000 7f5beb7fe640 1 -- 172.21.15.43:0/3225746435 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5be403f070 con 0x7f5bf4100510 2024-09-11T23:13:13.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.815+0000 7f5bf8d63640 1 -- 172.21.15.43:0/3225746435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf4199c50 con 0x7f5bf4100510 2024-09-11T23:13:13.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.815+0000 7f5bf8d63640 1 -- 172.21.15.43:0/3225746435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf4199ff0 con 0x7f5bf4100510 2024-09-11T23:13:13.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.815+0000 7f5beb7fe640 1 -- 172.21.15.43:0/3225746435 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5be4004cc0 con 0x7f5bf4100510 2024-09-11T23:13:13.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.815+0000 7f5beb7fe640 1 -- 172.21.15.43:0/3225746435 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5be4035030 con 0x7f5bf4100510 2024-09-11T23:13:13.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.816+0000 7f5beb7fe640 1 -- 172.21.15.43:0/3225746435 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5be4042070 con 0x7f5bf4100510 2024-09-11T23:13:13.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.816+0000 7f5bf8d63640 1 -- 172.21.15.43:0/3225746435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd40052c0 con 0x7f5bf4100510 2024-09-11T23:13:13.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.816+0000 7f5beb7fe640 1 --2- 172.21.15.43:0/3225746435 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f5be00390c0 0x7f5be003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:13.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.816+0000 7f5beb7fe640 1 -- 172.21.15.43:0/3225746435 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5be406dd60 con 0x7f5bf4100510 2024-09-11T23:13:13.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.816+0000 7f5bebfff640 1 --2- 172.21.15.43:0/3225746435 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f5be00390c0 0x7f5be003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:13.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.816+0000 7f5bebfff640 1 --2- 172.21.15.43:0/3225746435 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f5be00390c0 0x7f5be003b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0098c0 tx=0x7f5bdc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:13.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:13.821+0000 7f5beb7fe640 1 -- 172.21.15.43:0/3225746435 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5be40433b0 con 0x7f5bf4100510 2024-09-11T23:13:14.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:14.003+0000 7f5bf8d63640 1 -- 172.21.15.43:0/3225746435 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi090", "target": ["mon-mgr", ""]}) -- 0x7f5bd4000bf0 con 0x7f5be00390c0 2024-09-11T23:13:14.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:14 smithi043 ceph-mon[30950]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-11T23:13:14.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:14 smithi043 ceph-mon[30950]: Updating smithi043:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:13:14.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:14 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:14.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:14 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:14.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:14 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:14.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:14 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:14.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:14 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:14.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:14 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:14.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:14 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:15.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:15 smithi043 ceph-mon[30950]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:13:15.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:15 smithi043 ceph-mon[30950]: Updating smithi043:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:13:15.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:15 smithi043 ceph-mon[30950]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi090", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:13:16.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:16 smithi043 ceph-mon[30950]: Deploying cephadm binary to smithi090 2024-09-11T23:13:17.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:17.894+0000 7f5beb7fe640 1 -- 172.21.15.43:0/3225746435 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f5bd4000bf0 con 0x7f5be00390c0 2024-09-11T23:13:17.895 INFO:teuthology.orchestra.run.smithi043.stdout:Added host 'smithi090' with addr '172.21.15.90' 2024-09-11T23:13:17.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:17.897+0000 7f5bf8d63640 1 -- 172.21.15.43:0/3225746435 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f5be00390c0 msgr2=0x7f5be003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:17.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:17.897+0000 7f5bf8d63640 1 --2- 172.21.15.43:0/3225746435 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f5be00390c0 0x7f5be003b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0098c0 tx=0x7f5bdc006db0 comp rx=0 tx=0).stop 2024-09-11T23:13:17.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:17.897+0000 7f5bf8d63640 1 -- 172.21.15.43:0/3225746435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bf4100510 msgr2=0x7f5bf406a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:17.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:17.897+0000 7f5bf8d63640 1 --2- 172.21.15.43:0/3225746435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bf4100510 0x7f5bf406a990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5be4004f60 tx=0x7f5be40048f0 comp rx=0 tx=0).stop 2024-09-11T23:13:17.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:17.897+0000 7f5bf8d63640 1 -- 172.21.15.43:0/3225746435 shutdown_connections 2024-09-11T23:13:17.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:17.897+0000 7f5bf8d63640 1 --2- 172.21.15.43:0/3225746435 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f5be00390c0 0x7f5be003b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:17.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:17.897+0000 7f5bf8d63640 1 --2- 172.21.15.43:0/3225746435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bf4100510 0x7f5bf406a990 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:17.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:17.897+0000 7f5bf8d63640 1 -- 172.21.15.43:0/3225746435 >> 172.21.15.43:0/3225746435 conn(0x7f5bf40fbca0 msgr2=0x7f5bf40fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:17.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:17.897+0000 7f5bf8d63640 1 -- 172.21.15.43:0/3225746435 shutdown_connections 2024-09-11T23:13:17.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:17.897+0000 7f5bf8d63640 1 -- 172.21.15.43:0/3225746435 wait complete. 2024-09-11T23:13:18.535 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T23:13:18.786 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:13:19.208 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:18 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:19.208 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:18 smithi043 ceph-mon[30950]: Added host smithi090 2024-09-11T23:13:19.208 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:18 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:19.208 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:18 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:19.208 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:18 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:19.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.217+0000 7fa3ce492640 1 -- 172.21.15.43:0/1287131502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3c8106540 msgr2=0x7fa3c8106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:19.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.217+0000 7fa3ce492640 1 --2- 172.21.15.43:0/1287131502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3c8106540 0x7fa3c8106920 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009810 tx=0x7fa3b802a540 comp rx=0 tx=0).stop 2024-09-11T23:13:19.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.217+0000 7fa3ce492640 1 -- 172.21.15.43:0/1287131502 shutdown_connections 2024-09-11T23:13:19.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.217+0000 7fa3ce492640 1 --2- 172.21.15.43:0/1287131502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3c8106540 0x7fa3c8106920 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:19.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.217+0000 7fa3ce492640 1 -- 172.21.15.43:0/1287131502 >> 172.21.15.43:0/1287131502 conn(0x7fa3c8077310 msgr2=0x7fa3c80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.217+0000 7fa3ce492640 1 -- 172.21.15.43:0/1287131502 shutdown_connections 2024-09-11T23:13:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.217+0000 7fa3ce492640 1 -- 172.21.15.43:0/1287131502 wait complete. 2024-09-11T23:13:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.217+0000 7fa3ce492640 1 Processor -- start 2024-09-11T23:13:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.217+0000 7fa3ce492640 1 -- start start 2024-09-11T23:13:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.218+0000 7fa3ce492640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3c8106540 0x7fa3c80fd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.218+0000 7fa3ce492640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c810ab70 con 0x7fa3c8106540 2024-09-11T23:13:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.218+0000 7fa3c7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3c8106540 0x7fa3c80fd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.218+0000 7fa3c7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3c8106540 0x7fa3c80fd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37346/0 (socket says 172.21.15.43:37346) 2024-09-11T23:13:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.218+0000 7fa3c7fff640 1 -- 172.21.15.43:0/1498111544 learned_addr learned my addr 172.21.15.43:0/1498111544 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:13:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.218+0000 7fa3c7fff640 1 -- 172.21.15.43:0/1498111544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c80ff270 con 0x7fa3c8106540 2024-09-11T23:13:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.218+0000 7fa3c7fff640 1 --2- 172.21.15.43:0/1498111544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3c8106540 0x7fa3c80fd380 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80045e0 tx=0x7fa3b8004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.219+0000 7fa3c57fa640 1 -- 172.21.15.43:0/1498111544 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3b8030e70 con 0x7fa3c8106540 2024-09-11T23:13:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.219+0000 7fa3c57fa640 1 -- 172.21.15.43:0/1498111544 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3b80318e0 con 0x7fa3c8106540 2024-09-11T23:13:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.219+0000 7fa3ce492640 1 -- 172.21.15.43:0/1498111544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c80fd8c0 con 0x7fa3c8106540 2024-09-11T23:13:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.219+0000 7fa3c57fa640 1 -- 172.21.15.43:0/1498111544 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3b80049c0 con 0x7fa3c8106540 2024-09-11T23:13:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.219+0000 7fa3ce492640 1 -- 172.21.15.43:0/1498111544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c80fdd80 con 0x7fa3c8106540 2024-09-11T23:13:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.220+0000 7fa3c57fa640 1 -- 172.21.15.43:0/1498111544 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa3b8039040 con 0x7fa3c8106540 2024-09-11T23:13:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.220+0000 7fa3ce492640 1 -- 172.21.15.43:0/1498111544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c8101bb0 con 0x7fa3c8106540 2024-09-11T23:13:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.220+0000 7fa3c57fa640 1 --2- 172.21.15.43:0/1498111544 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa3b00390c0 0x7fa3b003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.220+0000 7fa3c57fa640 1 -- 172.21.15.43:0/1498111544 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa3b806d300 con 0x7fa3c8106540 2024-09-11T23:13:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.220+0000 7fa3c77fe640 1 --2- 172.21.15.43:0/1498111544 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa3b00390c0 0x7fa3b003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.220+0000 7fa3c77fe640 1 --2- 172.21.15.43:0/1498111544 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa3b00390c0 0x7fa3b003b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0098c0 tx=0x7fa3bc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:19.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.225+0000 7fa3c57fa640 1 -- 172.21.15.43:0/1498111544 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa3b8040030 con 0x7fa3c8106540 2024-09-11T23:13:19.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.403+0000 7fa3ce492640 1 -- 172.21.15.43:0/1498111544 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa3c80fe920 con 0x7fa3b00390c0 2024-09-11T23:13:19.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.405+0000 7fa3c57fa640 1 -- 172.21.15.43:0/1498111544 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fa3c80fe920 con 0x7fa3b00390c0 2024-09-11T23:13:19.405 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:13:19.405 INFO:teuthology.orchestra.run.smithi043.stdout:[{"addr": "172.21.15.43", "hostname": "smithi043", "labels": [], "status": ""}, {"addr": "172.21.15.90", "hostname": "smithi090", "labels": [], "status": ""}] 2024-09-11T23:13:19.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.407+0000 7fa3ce492640 1 -- 172.21.15.43:0/1498111544 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa3b00390c0 msgr2=0x7fa3b003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:19.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.407+0000 7fa3ce492640 1 --2- 172.21.15.43:0/1498111544 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa3b00390c0 0x7fa3b003b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0098c0 tx=0x7fa3bc006db0 comp rx=0 tx=0).stop 2024-09-11T23:13:19.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.407+0000 7fa3ce492640 1 -- 172.21.15.43:0/1498111544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3c8106540 msgr2=0x7fa3c80fd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:19.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.407+0000 7fa3ce492640 1 --2- 172.21.15.43:0/1498111544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3c8106540 0x7fa3c80fd380 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80045e0 tx=0x7fa3b8004610 comp rx=0 tx=0).stop 2024-09-11T23:13:19.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.407+0000 7fa3ce492640 1 -- 172.21.15.43:0/1498111544 shutdown_connections 2024-09-11T23:13:19.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.407+0000 7fa3ce492640 1 --2- 172.21.15.43:0/1498111544 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa3b00390c0 0x7fa3b003b580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:19.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.407+0000 7fa3ce492640 1 --2- 172.21.15.43:0/1498111544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3c8106540 0x7fa3c80fd380 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:19.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.407+0000 7fa3ce492640 1 -- 172.21.15.43:0/1498111544 >> 172.21.15.43:0/1498111544 conn(0x7fa3c8077310 msgr2=0x7fa3c80696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:19.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.407+0000 7fa3ce492640 1 -- 172.21.15.43:0/1498111544 shutdown_connections 2024-09-11T23:13:19.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:19.407+0000 7fa3ce492640 1 -- 172.21.15.43:0/1498111544 wait complete. 2024-09-11T23:13:19.973 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi097 2024-09-11T23:13:19.973 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-11T23:13:19.973 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/etc/ceph/ceph.conf 2024-09-11T23:13:19.993 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-11T23:13:19.993 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:13:20.052 INFO:tasks.cephadm:Adding host smithi097 to orchestrator... 2024-09-11T23:13:20.052 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph orch host add smithi097 2024-09-11T23:13:20.301 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:13:20.526 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:20 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:20.526 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:20 smithi043 ceph-mon[30950]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:13:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.730+0000 7fb5bbe84640 1 -- 172.21.15.43:0/699506776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b4107660 msgr2=0x7fb5b4107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.730+0000 7fb5bbe84640 1 --2- 172.21.15.43:0/699506776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b4107660 0x7fb5b4107a40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0009880 tx=0x7fb5b002a5d0 comp rx=0 tx=0).stop 2024-09-11T23:13:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.730+0000 7fb5bbe84640 1 -- 172.21.15.43:0/699506776 shutdown_connections 2024-09-11T23:13:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.730+0000 7fb5bbe84640 1 --2- 172.21.15.43:0/699506776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b4107660 0x7fb5b4107a40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.730+0000 7fb5bbe84640 1 -- 172.21.15.43:0/699506776 >> 172.21.15.43:0/699506776 conn(0x7fb5b40fd290 msgr2=0x7fb5b40ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.730+0000 7fb5bbe84640 1 -- 172.21.15.43:0/699506776 shutdown_connections 2024-09-11T23:13:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.730+0000 7fb5bbe84640 1 -- 172.21.15.43:0/699506776 wait complete. 2024-09-11T23:13:20.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.731+0000 7fb5bbe84640 1 Processor -- start 2024-09-11T23:13:20.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.731+0000 7fb5bbe84640 1 -- start start 2024-09-11T23:13:20.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.731+0000 7fb5bbe84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b4107660 0x7fb5b41970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:20.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.731+0000 7fb5bbe84640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b410bc90 con 0x7fb5b4107660 2024-09-11T23:13:20.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.731+0000 7fb5b9c20640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b4107660 0x7fb5b41970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:20.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.731+0000 7fb5b9c20640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b4107660 0x7fb5b41970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58044/0 (socket says 172.21.15.43:58044) 2024-09-11T23:13:20.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.731+0000 7fb5b9c20640 1 -- 172.21.15.43:0/1123153510 learned_addr learned my addr 172.21.15.43:0/1123153510 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:13:20.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.732+0000 7fb5b9c20640 1 -- 172.21.15.43:0/1123153510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b4197600 con 0x7fb5b4107660 2024-09-11T23:13:20.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.732+0000 7fb5b9c20640 1 --2- 172.21.15.43:0/1123153510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b4107660 0x7fb5b41970c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0009230 tx=0x7fb5b0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:20.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.732+0000 7fb5aaffd640 1 -- 172.21.15.43:0/1123153510 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5b003f070 con 0x7fb5b4107660 2024-09-11T23:13:20.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.732+0000 7fb5aaffd640 1 -- 172.21.15.43:0/1123153510 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5b003bc80 con 0x7fb5b4107660 2024-09-11T23:13:20.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.732+0000 7fb5bbe84640 1 -- 172.21.15.43:0/1123153510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b4197890 con 0x7fb5b4107660 2024-09-11T23:13:20.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.732+0000 7fb5aaffd640 1 -- 172.21.15.43:0/1123153510 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5b0035030 con 0x7fb5b4107660 2024-09-11T23:13:20.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.732+0000 7fb5bbe84640 1 -- 172.21.15.43:0/1123153510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b419b990 con 0x7fb5b4107660 2024-09-11T23:13:20.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.733+0000 7fb5aaffd640 1 -- 172.21.15.43:0/1123153510 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb5b0042070 con 0x7fb5b4107660 2024-09-11T23:13:20.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.733+0000 7fb5aaffd640 1 --2- 172.21.15.43:0/1123153510 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb5a0039060 0x7fb5a003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:20.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.733+0000 7fb5bbe84640 1 -- 172.21.15.43:0/1123153510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5b4102cd0 con 0x7fb5b4107660 2024-09-11T23:13:20.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.733+0000 7fb5aaffd640 1 -- 172.21.15.43:0/1123153510 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb5b006cd00 con 0x7fb5b4107660 2024-09-11T23:13:20.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.733+0000 7fb5b941f640 1 --2- 172.21.15.43:0/1123153510 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb5a0039060 0x7fb5a003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:20.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.734+0000 7fb5b941f640 1 --2- 172.21.15.43:0/1123153510 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb5a0039060 0x7fb5a003b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb5a40098c0 tx=0x7fb5a4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:20.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.738+0000 7fb5aaffd640 1 -- 172.21.15.43:0/1123153510 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5b00314c0 con 0x7fb5b4107660 2024-09-11T23:13:20.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:20.916+0000 7fb5bbe84640 1 -- 172.21.15.43:0/1123153510 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi097", "target": ["mon-mgr", ""]}) -- 0x7fb5b4107b20 con 0x7fb5a0039060 2024-09-11T23:13:21.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:21 smithi043 ceph-mon[30950]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:21.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:21 smithi043 ceph-mon[30950]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi097", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:13:23.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:23.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:23.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:23 smithi043 ceph-mon[30950]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-11T23:13:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:23 smithi043 ceph-mon[30950]: Deploying cephadm binary to smithi097 2024-09-11T23:13:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:23 smithi043 ceph-mon[30950]: Updating smithi090:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:13:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:23 smithi043 ceph-mon[30950]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:13:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:24.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:24 smithi043 ceph-mon[30950]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:24.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:24 smithi043 ceph-mon[30950]: Updating smithi090:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:13:24.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:24.789+0000 7fb5aaffd640 1 -- 172.21.15.43:0/1123153510 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fb5b4107b20 con 0x7fb5a0039060 2024-09-11T23:13:24.790 INFO:teuthology.orchestra.run.smithi043.stdout:Added host 'smithi097' with addr '172.21.15.97' 2024-09-11T23:13:24.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:24.791+0000 7fb5bbe84640 1 -- 172.21.15.43:0/1123153510 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb5a0039060 msgr2=0x7fb5a003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:24.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:24.791+0000 7fb5bbe84640 1 --2- 172.21.15.43:0/1123153510 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb5a0039060 0x7fb5a003b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb5a40098c0 tx=0x7fb5a4006db0 comp rx=0 tx=0).stop 2024-09-11T23:13:24.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:24.791+0000 7fb5bbe84640 1 -- 172.21.15.43:0/1123153510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b4107660 msgr2=0x7fb5b41970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:24.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:24.791+0000 7fb5bbe84640 1 --2- 172.21.15.43:0/1123153510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b4107660 0x7fb5b41970c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0009230 tx=0x7fb5b0003d40 comp rx=0 tx=0).stop 2024-09-11T23:13:24.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:24.791+0000 7fb5bbe84640 1 -- 172.21.15.43:0/1123153510 shutdown_connections 2024-09-11T23:13:24.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:24.791+0000 7fb5bbe84640 1 --2- 172.21.15.43:0/1123153510 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb5a0039060 0x7fb5a003b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:24.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:24.791+0000 7fb5bbe84640 1 --2- 172.21.15.43:0/1123153510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b4107660 0x7fb5b41970c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:24.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:24.792+0000 7fb5bbe84640 1 -- 172.21.15.43:0/1123153510 >> 172.21.15.43:0/1123153510 conn(0x7fb5b40fd290 msgr2=0x7fb5b40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:24.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:24.792+0000 7fb5bbe84640 1 -- 172.21.15.43:0/1123153510 shutdown_connections 2024-09-11T23:13:24.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:24.792+0000 7fb5bbe84640 1 -- 172.21.15.43:0/1123153510 wait complete. 2024-09-11T23:13:25.484 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T23:13:25.741 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:13:26.040 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:25 smithi043 ceph-mon[30950]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:26.040 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:25 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:26.040 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:25 smithi043 ceph-mon[30950]: Added host smithi097 2024-09-11T23:13:26.040 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:25 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:26.040 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:25 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:26.040 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:25 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:26.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.237+0000 7f3f5f73c640 1 -- 172.21.15.43:0/1588418330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f581002e0 msgr2=0x7f3f581006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:26.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.237+0000 7f3f5f73c640 1 --2- 172.21.15.43:0/1588418330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f581002e0 0x7f3f581006c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009c20 tx=0x7f3f4802aa90 comp rx=0 tx=0).stop 2024-09-11T23:13:26.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.237+0000 7f3f5f73c640 1 -- 172.21.15.43:0/1588418330 shutdown_connections 2024-09-11T23:13:26.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.237+0000 7f3f5f73c640 1 --2- 172.21.15.43:0/1588418330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f581002e0 0x7f3f581006c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:26.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.237+0000 7f3f5f73c640 1 -- 172.21.15.43:0/1588418330 >> 172.21.15.43:0/1588418330 conn(0x7f3f580fc1b0 msgr2=0x7f3f580fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:26.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.237+0000 7f3f5f73c640 1 -- 172.21.15.43:0/1588418330 shutdown_connections 2024-09-11T23:13:26.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.237+0000 7f3f5f73c640 1 -- 172.21.15.43:0/1588418330 wait complete. 2024-09-11T23:13:26.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.238+0000 7f3f5f73c640 1 Processor -- start 2024-09-11T23:13:26.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.238+0000 7f3f5f73c640 1 -- start start 2024-09-11T23:13:26.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.238+0000 7f3f5f73c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f581002e0 0x7f3f581993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.238+0000 7f3f5f73c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5810dea0 con 0x7f3f581002e0 2024-09-11T23:13:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.238+0000 7f3f5d4d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f581002e0 0x7f3f581993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.238+0000 7f3f5d4d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f581002e0 0x7f3f581993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58062/0 (socket says 172.21.15.43:58062) 2024-09-11T23:13:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.238+0000 7f3f5d4d8640 1 -- 172.21.15.43:0/996632973 learned_addr learned my addr 172.21.15.43:0/996632973 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:13:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.238+0000 7f3f5d4d8640 1 -- 172.21.15.43:0/996632973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f58199910 con 0x7f3f581002e0 2024-09-11T23:13:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.239+0000 7f3f5d4d8640 1 --2- 172.21.15.43:0/996632973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f581002e0 0x7f3f581993d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3f48005e30 tx=0x7f3f4802fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.239+0000 7f3f567fc640 1 -- 172.21.15.43:0/996632973 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f48047070 con 0x7f3f581002e0 2024-09-11T23:13:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.239+0000 7f3f567fc640 1 -- 172.21.15.43:0/996632973 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f48003df0 con 0x7f3f581002e0 2024-09-11T23:13:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.239+0000 7f3f5f73c640 1 -- 172.21.15.43:0/996632973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f58199ba0 con 0x7f3f581002e0 2024-09-11T23:13:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.239+0000 7f3f567fc640 1 -- 172.21.15.43:0/996632973 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f48033050 con 0x7f3f581002e0 2024-09-11T23:13:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.239+0000 7f3f5f73c640 1 -- 172.21.15.43:0/996632973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f5819a000 con 0x7f3f581002e0 2024-09-11T23:13:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.240+0000 7f3f567fc640 1 -- 172.21.15.43:0/996632973 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3f4804a070 con 0x7f3f581002e0 2024-09-11T23:13:26.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.240+0000 7f3f5f73c640 1 -- 172.21.15.43:0/996632973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f5806a7b0 con 0x7f3f581002e0 2024-09-11T23:13:26.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.240+0000 7f3f567fc640 1 --2- 172.21.15.43:0/996632973 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3f400390c0 0x7f3f4003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:26.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.240+0000 7f3f567fc640 1 -- 172.21.15.43:0/996632973 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3f4803e030 con 0x7f3f581002e0 2024-09-11T23:13:26.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.240+0000 7f3f5ccd7640 1 --2- 172.21.15.43:0/996632973 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3f400390c0 0x7f3f4003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:26.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.240+0000 7f3f5ccd7640 1 --2- 172.21.15.43:0/996632973 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3f400390c0 0x7f3f4003b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0098c0 tx=0x7f3f4c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:26.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.245+0000 7f3f567fc640 1 -- 172.21.15.43:0/996632973 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3f4804a360 con 0x7f3f581002e0 2024-09-11T23:13:26.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.425+0000 7f3f5f73c640 1 -- 172.21.15.43:0/996632973 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3f5819a730 con 0x7f3f400390c0 2024-09-11T23:13:26.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.426+0000 7f3f567fc640 1 -- 172.21.15.43:0/996632973 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7f3f5819a730 con 0x7f3f400390c0 2024-09-11T23:13:26.426 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:13:26.427 INFO:teuthology.orchestra.run.smithi043.stdout:[{"addr": "172.21.15.43", "hostname": "smithi043", "labels": [], "status": ""}, {"addr": "172.21.15.90", "hostname": "smithi090", "labels": [], "status": ""}, {"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}] 2024-09-11T23:13:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.428+0000 7f3f5f73c640 1 -- 172.21.15.43:0/996632973 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3f400390c0 msgr2=0x7f3f4003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.428+0000 7f3f5f73c640 1 --2- 172.21.15.43:0/996632973 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3f400390c0 0x7f3f4003b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0098c0 tx=0x7f3f4c006db0 comp rx=0 tx=0).stop 2024-09-11T23:13:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.428+0000 7f3f5f73c640 1 -- 172.21.15.43:0/996632973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f581002e0 msgr2=0x7f3f581993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.428+0000 7f3f5f73c640 1 --2- 172.21.15.43:0/996632973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f581002e0 0x7f3f581993d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3f48005e30 tx=0x7f3f4802fc10 comp rx=0 tx=0).stop 2024-09-11T23:13:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.428+0000 7f3f5f73c640 1 -- 172.21.15.43:0/996632973 shutdown_connections 2024-09-11T23:13:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.428+0000 7f3f5f73c640 1 --2- 172.21.15.43:0/996632973 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3f400390c0 0x7f3f4003b580 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.428+0000 7f3f5f73c640 1 --2- 172.21.15.43:0/996632973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f581002e0 0x7f3f581993d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.429+0000 7f3f5f73c640 1 -- 172.21.15.43:0/996632973 >> 172.21.15.43:0/996632973 conn(0x7f3f580fc1b0 msgr2=0x7f3f580fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.429+0000 7f3f5f73c640 1 -- 172.21.15.43:0/996632973 shutdown_connections 2024-09-11T23:13:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:26.429+0000 7f3f5f73c640 1 -- 172.21.15.43:0/996632973 wait complete. 2024-09-11T23:13:27.024 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-11T23:13:27.024 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-11T23:13:27.275 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:13:27.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:27 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:27.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:27 smithi043 ceph-mon[30950]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:13:27.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.718+0000 7f3983efe640 1 -- 172.21.15.43:0/1310703388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f397c107660 msgr2=0x7f397c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:27.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.718+0000 7f3983efe640 1 --2- 172.21.15.43:0/1310703388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f397c107660 0x7f397c107a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3978009880 tx=0x7f397802a5d0 comp rx=0 tx=0).stop 2024-09-11T23:13:27.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.718+0000 7f3983efe640 1 -- 172.21.15.43:0/1310703388 shutdown_connections 2024-09-11T23:13:27.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.718+0000 7f3983efe640 1 --2- 172.21.15.43:0/1310703388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f397c107660 0x7f397c107a40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:27.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.718+0000 7f3983efe640 1 -- 172.21.15.43:0/1310703388 >> 172.21.15.43:0/1310703388 conn(0x7f397c0fd290 msgr2=0x7f397c0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:27.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.719+0000 7f3983efe640 1 -- 172.21.15.43:0/1310703388 shutdown_connections 2024-09-11T23:13:27.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.719+0000 7f3983efe640 1 -- 172.21.15.43:0/1310703388 wait complete. 2024-09-11T23:13:27.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.719+0000 7f3983efe640 1 Processor -- start 2024-09-11T23:13:27.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.719+0000 7f3983efe640 1 -- start start 2024-09-11T23:13:27.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.719+0000 7f3983efe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f397c107660 0x7f397c1970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:27.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.719+0000 7f3983efe640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f397c10bc90 con 0x7f397c107660 2024-09-11T23:13:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.720+0000 7f3981c9a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f397c107660 0x7f397c1970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.720+0000 7f3981c9a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f397c107660 0x7f397c1970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58078/0 (socket says 172.21.15.43:58078) 2024-09-11T23:13:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.720+0000 7f3981c9a640 1 -- 172.21.15.43:0/1921841381 learned_addr learned my addr 172.21.15.43:0/1921841381 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:13:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.720+0000 7f3981c9a640 1 -- 172.21.15.43:0/1921841381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f397c197600 con 0x7f397c107660 2024-09-11T23:13:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.720+0000 7f3981c9a640 1 --2- 172.21.15.43:0/1921841381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f397c107660 0x7f397c1970c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3978009230 tx=0x7f3978003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.720+0000 7f3972ffd640 1 -- 172.21.15.43:0/1921841381 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f397803f070 con 0x7f397c107660 2024-09-11T23:13:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.720+0000 7f3972ffd640 1 -- 172.21.15.43:0/1921841381 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f397803bc80 con 0x7f397c107660 2024-09-11T23:13:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.720+0000 7f3983efe640 1 -- 172.21.15.43:0/1921841381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f397c197890 con 0x7f397c107660 2024-09-11T23:13:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.720+0000 7f3972ffd640 1 -- 172.21.15.43:0/1921841381 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3978035030 con 0x7f397c107660 2024-09-11T23:13:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.720+0000 7f3983efe640 1 -- 172.21.15.43:0/1921841381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f397c19b990 con 0x7f397c107660 2024-09-11T23:13:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.721+0000 7f3972ffd640 1 -- 172.21.15.43:0/1921841381 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3978042070 con 0x7f397c107660 2024-09-11T23:13:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.721+0000 7f3983efe640 1 -- 172.21.15.43:0/1921841381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f397c102cd0 con 0x7f397c107660 2024-09-11T23:13:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.721+0000 7f3972ffd640 1 --2- 172.21.15.43:0/1921841381 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3968039060 0x7f396803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.721+0000 7f3972ffd640 1 -- 172.21.15.43:0/1921841381 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f397806c9d0 con 0x7f397c107660 2024-09-11T23:13:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.722+0000 7f3981499640 1 --2- 172.21.15.43:0/1921841381 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3968039060 0x7f396803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.722+0000 7f3981499640 1 --2- 172.21.15.43:0/1921841381 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3968039060 0x7f396803b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f396c0098c0 tx=0x7f396c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:27.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.726+0000 7f3972ffd640 1 -- 172.21.15.43:0/1921841381 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3978031e20 con 0x7f397c107660 2024-09-11T23:13:27.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:27.906+0000 7f3983efe640 1 -- 172.21.15.43:0/1921841381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f397c198420 con 0x7f397c107660 2024-09-11T23:13:28.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:28.133+0000 7f3972ffd640 1 -- 172.21.15.43:0/1921841381 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f39780372d0 con 0x7f397c107660 2024-09-11T23:13:28.133 INFO:teuthology.orchestra.run.smithi043.stderr:adjusted tunables profile to default 2024-09-11T23:13:28.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:28.135+0000 7f3983efe640 1 -- 172.21.15.43:0/1921841381 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3968039060 msgr2=0x7f396803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:28.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:28.135+0000 7f3983efe640 1 --2- 172.21.15.43:0/1921841381 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3968039060 0x7f396803b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f396c0098c0 tx=0x7f396c006db0 comp rx=0 tx=0).stop 2024-09-11T23:13:28.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:28.135+0000 7f3983efe640 1 -- 172.21.15.43:0/1921841381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f397c107660 msgr2=0x7f397c1970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:28.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:28.135+0000 7f3983efe640 1 --2- 172.21.15.43:0/1921841381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f397c107660 0x7f397c1970c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3978009230 tx=0x7f3978003d40 comp rx=0 tx=0).stop 2024-09-11T23:13:28.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:28.136+0000 7f3983efe640 1 -- 172.21.15.43:0/1921841381 shutdown_connections 2024-09-11T23:13:28.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:28.136+0000 7f3983efe640 1 --2- 172.21.15.43:0/1921841381 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3968039060 0x7f396803b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:28.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:28.136+0000 7f3983efe640 1 --2- 172.21.15.43:0/1921841381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f397c107660 0x7f397c1970c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:28.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:28.136+0000 7f3983efe640 1 -- 172.21.15.43:0/1921841381 >> 172.21.15.43:0/1921841381 conn(0x7f397c0fd290 msgr2=0x7f397c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:28.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:28.136+0000 7f3983efe640 1 -- 172.21.15.43:0/1921841381 shutdown_connections 2024-09-11T23:13:28.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:28.136+0000 7f3983efe640 1 -- 172.21.15.43:0/1921841381 wait complete. 2024-09-11T23:13:28.438 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:28 smithi043 ceph-mon[30950]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:28.438 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:28 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1921841381' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T23:13:28.781 INFO:tasks.cephadm:Adding mon.a on smithi043 2024-09-11T23:13:28.781 INFO:tasks.cephadm:Adding mon.b on smithi090 2024-09-11T23:13:28.781 INFO:tasks.cephadm:Adding mon.c on smithi097 2024-09-11T23:13:28.782 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi043:172.21.15.43=a;smithi090:172.21.15.90=b;smithi097:172.21.15.97=c' 2024-09-11T23:13:29.022 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-11T23:13:29.074 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-11T23:13:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.582+0000 7f0bd766c640 1 -- 172.21.15.97:0/2613235557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bd00ffec0 msgr2=0x7f0bd01002c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.582+0000 7f0bd766c640 1 --2- 172.21.15.97:0/2613235557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bd00ffec0 0x7f0bd01002c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009880 tx=0x7f0bcc02a5d0 comp rx=0 tx=0).stop 2024-09-11T23:13:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.582+0000 7f0bd766c640 1 -- 172.21.15.97:0/2613235557 shutdown_connections 2024-09-11T23:13:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.582+0000 7f0bd766c640 1 --2- 172.21.15.97:0/2613235557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bd00ffec0 0x7f0bd01002c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.582+0000 7f0bd766c640 1 -- 172.21.15.97:0/2613235557 >> 172.21.15.97:0/2613235557 conn(0x7f0bd00fbcd0 msgr2=0x7f0bd00fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.582+0000 7f0bd766c640 1 -- 172.21.15.97:0/2613235557 shutdown_connections 2024-09-11T23:13:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.582+0000 7f0bd766c640 1 -- 172.21.15.97:0/2613235557 wait complete. 2024-09-11T23:13:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.583+0000 7f0bd766c640 1 Processor -- start 2024-09-11T23:13:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.583+0000 7f0bd766c640 1 -- start start 2024-09-11T23:13:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.583+0000 7f0bd766c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bd00ffec0 0x7f0bd0198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.583+0000 7f0bd766c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd01098d0 con 0x7f0bd00ffec0 2024-09-11T23:13:29.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.583+0000 7f0bd5408640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bd00ffec0 0x7f0bd0198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:29.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.583+0000 7f0bd5408640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bd00ffec0 0x7f0bd0198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.97:50124/0 (socket says 172.21.15.97:50124) 2024-09-11T23:13:29.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.583+0000 7f0bd5408640 1 -- 172.21.15.97:0/3140585103 learned_addr learned my addr 172.21.15.97:0/3140585103 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-11T23:13:29.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.584+0000 7f0bd5408640 1 -- 172.21.15.97:0/3140585103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd0199480 con 0x7f0bd00ffec0 2024-09-11T23:13:29.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.584+0000 7f0bd5408640 1 --2- 172.21.15.97:0/3140585103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bd00ffec0 0x7f0bd0198f40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc004f60 tx=0x7f0bcc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:29.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.584+0000 7f0bc67fc640 1 -- 172.21.15.97:0/3140585103 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bcc03f070 con 0x7f0bd00ffec0 2024-09-11T23:13:29.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.584+0000 7f0bc67fc640 1 -- 172.21.15.97:0/3140585103 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bcc003ad0 con 0x7f0bd00ffec0 2024-09-11T23:13:29.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.584+0000 7f0bd766c640 1 -- 172.21.15.97:0/3140585103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd0199710 con 0x7f0bd00ffec0 2024-09-11T23:13:29.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.584+0000 7f0bc67fc640 1 -- 172.21.15.97:0/3140585103 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bcc035030 con 0x7f0bd00ffec0 2024-09-11T23:13:29.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.584+0000 7f0bd766c640 1 -- 172.21.15.97:0/3140585103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd019c400 con 0x7f0bd00ffec0 2024-09-11T23:13:29.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.585+0000 7f0bc67fc640 1 -- 172.21.15.97:0/3140585103 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0bcc042070 con 0x7f0bd00ffec0 2024-09-11T23:13:29.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.585+0000 7f0bd766c640 1 -- 172.21.15.97:0/3140585103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bd010a050 con 0x7f0bd00ffec0 2024-09-11T23:13:29.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.585+0000 7f0bc67fc640 1 --2- 172.21.15.97:0/3140585103 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0bbc039060 0x7f0bbc03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:29.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.586+0000 7f0bc67fc640 1 -- 172.21.15.97:0/3140585103 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0bcc06ce70 con 0x7f0bd00ffec0 2024-09-11T23:13:29.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.586+0000 7f0bd4c07640 1 --2- 172.21.15.97:0/3140585103 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0bbc039060 0x7f0bbc03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:29.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.586+0000 7f0bd4c07640 1 --2- 172.21.15.97:0/3140585103 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0bbc039060 0x7f0bbc03b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00098c0 tx=0x7f0bc0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:29.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.591+0000 7f0bc67fc640 1 -- 172.21.15.97:0/3140585103 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0bcc04c350 con 0x7f0bd00ffec0 2024-09-11T23:13:29.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:29 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1921841381' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T23:13:29.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:29 smithi043 ceph-mon[30950]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:13:29.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:29 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:29.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:29 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:29.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:29 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:29.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:29 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:29.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:29 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:29.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:29 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:29.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:29 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:29.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.779+0000 7f0bd766c640 1 -- 172.21.15.97:0/3140585103 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi090:172.21.15.90=b;smithi097:172.21.15.97=c", "target": ["mon-mgr", ""]}) -- 0x7f0bd002d8e0 con 0x7f0bbc039060 2024-09-11T23:13:29.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.785+0000 7f0bc67fc640 1 -- 172.21.15.97:0/3140585103 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0bd002d8e0 con 0x7f0bbc039060 2024-09-11T23:13:29.785 INFO:teuthology.orchestra.run.smithi097.stdout:Scheduled mon update... 2024-09-11T23:13:29.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.787+0000 7f0bd766c640 1 -- 172.21.15.97:0/3140585103 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0bbc039060 msgr2=0x7f0bbc03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:29.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.787+0000 7f0bd766c640 1 --2- 172.21.15.97:0/3140585103 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0bbc039060 0x7f0bbc03b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00098c0 tx=0x7f0bc0006db0 comp rx=0 tx=0).stop 2024-09-11T23:13:29.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.787+0000 7f0bd766c640 1 -- 172.21.15.97:0/3140585103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bd00ffec0 msgr2=0x7f0bd0198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:29.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.787+0000 7f0bd766c640 1 --2- 172.21.15.97:0/3140585103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bd00ffec0 0x7f0bd0198f40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc004f60 tx=0x7f0bcc0048f0 comp rx=0 tx=0).stop 2024-09-11T23:13:29.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.787+0000 7f0bd766c640 1 -- 172.21.15.97:0/3140585103 shutdown_connections 2024-09-11T23:13:29.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.787+0000 7f0bd766c640 1 --2- 172.21.15.97:0/3140585103 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0bbc039060 0x7f0bbc03b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:29.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.787+0000 7f0bd766c640 1 --2- 172.21.15.97:0/3140585103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bd00ffec0 0x7f0bd0198f40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:29.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.787+0000 7f0bd766c640 1 -- 172.21.15.97:0/3140585103 >> 172.21.15.97:0/3140585103 conn(0x7f0bd00fbcd0 msgr2=0x7f0bd00fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:29.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.787+0000 7f0bd766c640 1 -- 172.21.15.97:0/3140585103 shutdown_connections 2024-09-11T23:13:29.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:29.787+0000 7f0bd766c640 1 -- 172.21.15.97:0/3140585103 wait complete. 2024-09-11T23:13:30.448 DEBUG:teuthology.orchestra.run.smithi090:mon.b> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mon.b.service 2024-09-11T23:13:30.452 DEBUG:teuthology.orchestra.run.smithi097:mon.c> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mon.c.service 2024-09-11T23:13:30.454 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T23:13:30.454 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T23:13:30.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:30 smithi043 ceph-mon[30950]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:30.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:30 smithi043 ceph-mon[30950]: Updating smithi097:/etc/ceph/ceph.conf 2024-09-11T23:13:30.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:30 smithi043 ceph-mon[30950]: Updating smithi097:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:13:30.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:30 smithi043 ceph-mon[30950]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:13:30.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:30 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:30.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:30 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:30.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:30 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:30.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:30 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:30.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:30 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:13:30.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:30 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:31.416 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:13:31.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:31 smithi043 ceph-mon[30950]: Updating smithi097:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:13:31.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:31 smithi043 ceph-mon[30950]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi090:172.21.15.90=b;smithi097:172.21.15.97=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:13:31.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:31 smithi043 ceph-mon[30950]: Saving service mon spec with placement smithi043:172.21.15.43=a;smithi090:172.21.15.90=b;smithi097:172.21.15.97=c;count:3 2024-09-11T23:13:31.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:31 smithi043 ceph-mon[30950]: Deploying daemon mon.c on smithi097 2024-09-11T23:13:32.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.048+0000 7f02572c8640 1 -- 172.21.15.97:0/4096011790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0250106560 msgr2=0x7f0250106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:32.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.048+0000 7f02572c8640 1 --2- 172.21.15.97:0/4096011790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0250106560 0x7f0250106940 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f024c009880 tx=0x7f024c02a5d0 comp rx=0 tx=0).stop 2024-09-11T23:13:32.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.048+0000 7f02572c8640 1 -- 172.21.15.97:0/4096011790 shutdown_connections 2024-09-11T23:13:32.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.048+0000 7f02572c8640 1 --2- 172.21.15.97:0/4096011790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0250106560 0x7f0250106940 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:32.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.048+0000 7f02572c8640 1 -- 172.21.15.97:0/4096011790 >> 172.21.15.97:0/4096011790 conn(0x7f02500fc1b0 msgr2=0x7f02500fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:32.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.048+0000 7f02572c8640 1 -- 172.21.15.97:0/4096011790 shutdown_connections 2024-09-11T23:13:32.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.048+0000 7f02572c8640 1 -- 172.21.15.97:0/4096011790 wait complete. 2024-09-11T23:13:32.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.049+0000 7f02572c8640 1 Processor -- start 2024-09-11T23:13:32.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.049+0000 7f02572c8640 1 -- start start 2024-09-11T23:13:32.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.049+0000 7f02572c8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0250106560 0x7f0250197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:32.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.049+0000 7f02572c8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f025010ab90 con 0x7f0250106560 2024-09-11T23:13:32.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.049+0000 7f0255064640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0250106560 0x7f0250197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:32.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.049+0000 7f0255064640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0250106560 0x7f0250197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.97:50146/0 (socket says 172.21.15.97:50146) 2024-09-11T23:13:32.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.049+0000 7f0255064640 1 -- 172.21.15.97:0/66118479 learned_addr learned my addr 172.21.15.97:0/66118479 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-11T23:13:32.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.050+0000 7f0255064640 1 -- 172.21.15.97:0/66118479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0250197660 con 0x7f0250106560 2024-09-11T23:13:32.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.050+0000 7f0255064640 1 --2- 172.21.15.97:0/66118479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0250106560 0x7f0250197120 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f024c009230 tx=0x7f024c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:32.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.050+0000 7f02467fc640 1 -- 172.21.15.97:0/66118479 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f024c03f070 con 0x7f0250106560 2024-09-11T23:13:32.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.050+0000 7f02572c8640 1 -- 172.21.15.97:0/66118479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02501978f0 con 0x7f0250106560 2024-09-11T23:13:32.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.050+0000 7f02467fc640 1 -- 172.21.15.97:0/66118479 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f024c03bc80 con 0x7f0250106560 2024-09-11T23:13:32.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.050+0000 7f02467fc640 1 -- 172.21.15.97:0/66118479 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f024c035030 con 0x7f0250106560 2024-09-11T23:13:32.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.050+0000 7f02572c8640 1 -- 172.21.15.97:0/66118479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f025019b9f0 con 0x7f0250106560 2024-09-11T23:13:32.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.051+0000 7f02467fc640 1 -- 172.21.15.97:0/66118479 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f024c042070 con 0x7f0250106560 2024-09-11T23:13:32.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.051+0000 7f02467fc640 1 --2- 172.21.15.97:0/66118479 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f023c039060 0x7f023c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:32.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.051+0000 7f02467fc640 1 -- 172.21.15.97:0/66118479 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f024c06ca10 con 0x7f0250106560 2024-09-11T23:13:32.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.051+0000 7f02572c8640 1 -- 172.21.15.97:0/66118479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0250101bd0 con 0x7f0250106560 2024-09-11T23:13:32.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.052+0000 7f0254863640 1 --2- 172.21.15.97:0/66118479 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f023c039060 0x7f023c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:32.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.052+0000 7f0254863640 1 --2- 172.21.15.97:0/66118479 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f023c039060 0x7f023c03b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f02400098c0 tx=0x7f0240006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:32.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.057+0000 7f02467fc640 1 -- 172.21.15.97:0/66118479 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f024c0314c0 con 0x7f0250106560 2024-09-11T23:13:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.300+0000 7f02572c8640 1 -- 172.21.15.97:0/66118479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f02501069c0 con 0x7f0250106560 2024-09-11T23:13:32.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.301+0000 7f02467fc640 1 -- 172.21.15.97:0/66118479 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f024c0046c0 con 0x7f0250106560 2024-09-11T23:13:32.301 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:13:32.301 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"fsid":"25c0cd84-7093-11ef-bcea-c7b262605968","modified":"2024-09-11T23:12:00.389048Z","created":"2024-09-11T23:12:00.389048Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T23:13:32.302 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 1 2024-09-11T23:13:32.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.303+0000 7f02572c8640 1 -- 172.21.15.97:0/66118479 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f023c039060 msgr2=0x7f023c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:32.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.303+0000 7f02572c8640 1 --2- 172.21.15.97:0/66118479 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f023c039060 0x7f023c03b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f02400098c0 tx=0x7f0240006db0 comp rx=0 tx=0).stop 2024-09-11T23:13:32.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.303+0000 7f02572c8640 1 -- 172.21.15.97:0/66118479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0250106560 msgr2=0x7f0250197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:32.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.303+0000 7f02572c8640 1 --2- 172.21.15.97:0/66118479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0250106560 0x7f0250197120 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f024c009230 tx=0x7f024c003d40 comp rx=0 tx=0).stop 2024-09-11T23:13:32.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.303+0000 7f02572c8640 1 -- 172.21.15.97:0/66118479 shutdown_connections 2024-09-11T23:13:32.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.303+0000 7f02572c8640 1 --2- 172.21.15.97:0/66118479 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f023c039060 0x7f023c03b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:32.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.303+0000 7f02572c8640 1 --2- 172.21.15.97:0/66118479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0250106560 0x7f0250197120 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:32.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.303+0000 7f02572c8640 1 -- 172.21.15.97:0/66118479 >> 172.21.15.97:0/66118479 conn(0x7f02500fc1b0 msgr2=0x7f0250069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:32.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.303+0000 7f02572c8640 1 -- 172.21.15.97:0/66118479 shutdown_connections 2024-09-11T23:13:32.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:32.303+0000 7f02572c8640 1 -- 172.21.15.97:0/66118479 wait complete. 2024-09-11T23:13:32.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:32 smithi043 ceph-mon[30950]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:33.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:33 smithi043 ceph-mon[30950]: from='client.? 172.21.15.97:0/66118479' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:13:33.994 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T23:13:33.994 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T23:13:34.245 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:13:34.514 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:34 smithi097 systemd[1]: Starting Ceph mon.c for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:13:34.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:34 smithi043 ceph-mon[30950]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:34.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.704+0000 7fc542bed640 1 -- 172.21.15.97:0/3713930063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c106790 msgr2=0x7fc53c106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:34.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.704+0000 7fc542bed640 1 --2- 172.21.15.97:0/3713930063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c106790 0x7fc53c106b70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009880 tx=0x7fc52c02a5d0 comp rx=0 tx=0).stop 2024-09-11T23:13:34.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.704+0000 7fc542bed640 1 -- 172.21.15.97:0/3713930063 shutdown_connections 2024-09-11T23:13:34.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.704+0000 7fc542bed640 1 --2- 172.21.15.97:0/3713930063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c106790 0x7fc53c106b70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:34.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.704+0000 7fc542bed640 1 -- 172.21.15.97:0/3713930063 >> 172.21.15.97:0/3713930063 conn(0x7fc53c0fb580 msgr2=0x7fc53c0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:34.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.704+0000 7fc542bed640 1 -- 172.21.15.97:0/3713930063 shutdown_connections 2024-09-11T23:13:34.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.704+0000 7fc542bed640 1 -- 172.21.15.97:0/3713930063 wait complete. 2024-09-11T23:13:34.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.705+0000 7fc542bed640 1 Processor -- start 2024-09-11T23:13:34.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.705+0000 7fc542bed640 1 -- start start 2024-09-11T23:13:34.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.705+0000 7fc542bed640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c106790 0x7fc53c1011c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:34.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.705+0000 7fc542bed640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c10bd50 con 0x7fc53c106790 2024-09-11T23:13:34.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.705+0000 7fc540989640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c106790 0x7fc53c1011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:34.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.705+0000 7fc540989640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c106790 0x7fc53c1011c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.97:50158/0 (socket says 172.21.15.97:50158) 2024-09-11T23:13:34.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.705+0000 7fc540989640 1 -- 172.21.15.97:0/707173068 learned_addr learned my addr 172.21.15.97:0/707173068 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-11T23:13:34.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.706+0000 7fc540989640 1 -- 172.21.15.97:0/707173068 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53c1030b0 con 0x7fc53c106790 2024-09-11T23:13:34.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.706+0000 7fc540989640 1 --2- 172.21.15.97:0/707173068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c106790 0x7fc53c1011c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009230 tx=0x7fc52c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:34.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.706+0000 7fc539ffb640 1 -- 172.21.15.97:0/707173068 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc52c03f070 con 0x7fc53c106790 2024-09-11T23:13:34.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.706+0000 7fc539ffb640 1 -- 172.21.15.97:0/707173068 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc52c03bc80 con 0x7fc53c106790 2024-09-11T23:13:34.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.706+0000 7fc542bed640 1 -- 172.21.15.97:0/707173068 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53c101700 con 0x7fc53c106790 2024-09-11T23:13:34.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.706+0000 7fc539ffb640 1 -- 172.21.15.97:0/707173068 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc52c035030 con 0x7fc53c106790 2024-09-11T23:13:34.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.706+0000 7fc542bed640 1 -- 172.21.15.97:0/707173068 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc53c101bc0 con 0x7fc53c106790 2024-09-11T23:13:34.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.707+0000 7fc542bed640 1 -- 172.21.15.97:0/707173068 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc53c100b90 con 0x7fc53c106790 2024-09-11T23:13:34.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.707+0000 7fc539ffb640 1 -- 172.21.15.97:0/707173068 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc52c042070 con 0x7fc53c106790 2024-09-11T23:13:34.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.707+0000 7fc539ffb640 1 --2- 172.21.15.97:0/707173068 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fc524039060 0x7fc52403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:34.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.707+0000 7fc539ffb640 1 -- 172.21.15.97:0/707173068 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc52c06cda0 con 0x7fc53c106790 2024-09-11T23:13:34.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.708+0000 7fc53bfff640 1 --2- 172.21.15.97:0/707173068 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fc524039060 0x7fc52403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:34.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.708+0000 7fc53bfff640 1 --2- 172.21.15.97:0/707173068 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fc524039060 0x7fc52403b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc5300098c0 tx=0x7fc530006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:34.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.712+0000 7fc539ffb640 1 -- 172.21.15.97:0/707173068 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc52c031e20 con 0x7fc53c106790 2024-09-11T23:13:34.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.955+0000 7fc542bed640 1 -- 172.21.15.97:0/707173068 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc53c1026f0 con 0x7fc53c106790 2024-09-11T23:13:34.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.955+0000 7fc539ffb640 1 -- 172.21.15.97:0/707173068 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fc52c0372d0 con 0x7fc53c106790 2024-09-11T23:13:34.956 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:13:34.956 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"fsid":"25c0cd84-7093-11ef-bcea-c7b262605968","modified":"2024-09-11T23:12:00.389048Z","created":"2024-09-11T23:12:00.389048Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T23:13:34.956 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 1 2024-09-11T23:13:34.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.957+0000 7fc542bed640 1 -- 172.21.15.97:0/707173068 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fc524039060 msgr2=0x7fc52403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:34.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.957+0000 7fc542bed640 1 --2- 172.21.15.97:0/707173068 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fc524039060 0x7fc52403b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc5300098c0 tx=0x7fc530006db0 comp rx=0 tx=0).stop 2024-09-11T23:13:34.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.957+0000 7fc542bed640 1 -- 172.21.15.97:0/707173068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c106790 msgr2=0x7fc53c1011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:34.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.957+0000 7fc542bed640 1 --2- 172.21.15.97:0/707173068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c106790 0x7fc53c1011c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009230 tx=0x7fc52c003d40 comp rx=0 tx=0).stop 2024-09-11T23:13:34.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.957+0000 7fc542bed640 1 -- 172.21.15.97:0/707173068 shutdown_connections 2024-09-11T23:13:34.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.957+0000 7fc542bed640 1 --2- 172.21.15.97:0/707173068 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fc524039060 0x7fc52403b520 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:34.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.957+0000 7fc542bed640 1 --2- 172.21.15.97:0/707173068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c106790 0x7fc53c1011c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:34.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.958+0000 7fc542bed640 1 -- 172.21.15.97:0/707173068 >> 172.21.15.97:0/707173068 conn(0x7fc53c0fb580 msgr2=0x7fc53c103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:34.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.958+0000 7fc542bed640 1 -- 172.21.15.97:0/707173068 shutdown_connections 2024-09-11T23:13:34.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:34.958+0000 7fc542bed640 1 -- 172.21.15.97:0/707173068 wait complete. 2024-09-11T23:13:35.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:34 smithi097 podman[35528]: 2024-09-11 23:13:34.76328336 +0000 UTC m=+0.019362828 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:13:35.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:34 smithi097 podman[35528]: 2024-09-11 23:13:34.874767145 +0000 UTC m=+0.130846591 container create fba50890eb516b71025d6dc58dcaeba7aa072005f64b9571c16610ab3f9d03ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T23:13:35.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:35 smithi043 ceph-mon[30950]: from='client.? 172.21.15.97:0/707173068' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:13:36.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 podman[35528]: 2024-09-11 23:13:35.696703548 +0000 UTC m=+0.952782993 container init fba50890eb516b71025d6dc58dcaeba7aa072005f64b9571c16610ab3f9d03ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-11T23:13:36.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 podman[35528]: 2024-09-11 23:13:35.700589762 +0000 UTC m=+0.956669209 container start fba50890eb516b71025d6dc58dcaeba7aa072005f64b9571c16610ab3f9d03ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-11T23:13:36.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T23:13:36.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-11T23:13:36.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: pidfile_write: ignore empty --pid-file 2024-09-11T23:13:36.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: load: jerasure load: lrc 2024-09-11T23:13:36.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: RocksDB version: 7.9.2 2024-09-11T23:13:36.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Git sha 0 2024-09-11T23:13:36.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-11T23:13:36.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: DB SUMMARY 2024-09-11T23:13:36.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: DB Session ID: Y11KOOG60W8ZQGDG4XJG 2024-09-11T23:13:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: CURRENT file: CURRENT 2024-09-11T23:13:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T23:13:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-11T23:13:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-11T23:13:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-11T23:13:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.error_if_exists: 0 2024-09-11T23:13:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.create_if_missing: 0 2024-09-11T23:13:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.paranoid_checks: 1 2024-09-11T23:13:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-11T23:13:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T23:13:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-11T23:13:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.env: 0x5560c14cd940 2024-09-11T23:13:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.fs: PosixFileSystem 2024-09-11T23:13:36.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.info_log: 0x5560c304f6e0 2024-09-11T23:13:36.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T23:13:36.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.statistics: (nil) 2024-09-11T23:13:36.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.use_fsync: 0 2024-09-11T23:13:36.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_log_file_size: 0 2024-09-11T23:13:36.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T23:13:36.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T23:13:36.055 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T23:13:36.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T23:13:36.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.allow_fallocate: 1 2024-09-11T23:13:36.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T23:13:36.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T23:13:36.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.use_direct_reads: 0 2024-09-11T23:13:36.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T23:13:36.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T23:13:36.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.db_log_dir: 2024-09-11T23:13:36.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.wal_dir: 2024-09-11T23:13:36.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T23:13:36.056 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T23:13:36.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T23:13:36.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T23:13:36.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T23:13:36.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T23:13:36.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T23:13:36.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T23:13:36.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.write_buffer_manager: 0x5560c305edc0 2024-09-11T23:13:36.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T23:13:36.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T23:13:36.057 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T23:13:36.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T23:13:36.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T23:13:36.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T23:13:36.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T23:13:36.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T23:13:36.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.unordered_write: 0 2024-09-11T23:13:36.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T23:13:36.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T23:13:36.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T23:13:36.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T23:13:36.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.row_cache: None 2024-09-11T23:13:36.058 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.wal_filter: None 2024-09-11T23:13:36.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T23:13:36.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T23:13:36.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.two_write_queues: 0 2024-09-11T23:13:36.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T23:13:36.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.wal_compression: 0 2024-09-11T23:13:36.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.atomic_flush: 0 2024-09-11T23:13:36.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T23:13:36.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T23:13:36.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T23:13:36.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.log_readahead_size: 0 2024-09-11T23:13:36.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T23:13:36.059 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T23:13:36.060 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T23:13:36.060 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T23:13:36.060 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T23:13:36.060 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T23:13:36.060 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-11T23:13:36.060 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_background_jobs: 2 2024-09-11T23:13:36.060 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_background_compactions: -1 2024-09-11T23:13:36.060 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_subcompactions: 1 2024-09-11T23:13:36.060 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T23:13:36.060 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T23:13:36.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T23:13:36.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T23:13:36.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T23:13:36.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T23:13:36.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T23:13:36.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T23:13:36.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_open_files: -1 2024-09-11T23:13:36.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T23:13:36.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T23:13:36.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T23:13:36.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T23:13:36.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_background_flushes: -1 2024-09-11T23:13:36.061 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Compression algorithms supported: 2024-09-11T23:13:36.062 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: kZSTD supported: 0 2024-09-11T23:13:36.062 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: kXpressCompression supported: 0 2024-09-11T23:13:36.062 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: kBZip2Compression supported: 0 2024-09-11T23:13:36.062 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T23:13:36.062 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: kLZ4Compression supported: 1 2024-09-11T23:13:36.062 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: kZlibCompression supported: 1 2024-09-11T23:13:36.062 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T23:13:36.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: kSnappyCompression supported: 1 2024-09-11T23:13:36.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T23:13:36.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-11T23:13:36.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-11T23:13:36.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-11T23:13:36.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T23:13:36.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.merge_operator: 2024-09-11T23:13:36.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compaction_filter: None 2024-09-11T23:13:36.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T23:13:36.063 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T23:13:36.064 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T23:13:36.064 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T23:13:36.064 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5560c304f320) 2024-09-11T23:13:36.064 INFO:journalctl@ceph.mon.c.smithi097.stdout: cache_index_and_filter_blocks: 1 2024-09-11T23:13:36.064 INFO:journalctl@ceph.mon.c.smithi097.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T23:13:36.064 INFO:journalctl@ceph.mon.c.smithi097.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T23:13:36.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: pin_top_level_index_and_filter: 1 2024-09-11T23:13:36.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: index_type: 0 2024-09-11T23:13:36.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: data_block_index_type: 0 2024-09-11T23:13:36.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: index_shortening: 1 2024-09-11T23:13:36.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T23:13:36.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: checksum: 4 2024-09-11T23:13:36.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: no_block_cache: 0 2024-09-11T23:13:36.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_cache: 0x5560c30431f0 2024-09-11T23:13:36.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_cache_name: BinnedLRUCache 2024-09-11T23:13:36.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_cache_options: 2024-09-11T23:13:36.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: capacity : 536870912 2024-09-11T23:13:36.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: num_shard_bits : 4 2024-09-11T23:13:36.065 INFO:journalctl@ceph.mon.c.smithi097.stdout: strict_capacity_limit : 0 2024-09-11T23:13:36.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: high_pri_pool_ratio: 0.000 2024-09-11T23:13:36.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_cache_compressed: (nil) 2024-09-11T23:13:36.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: persistent_cache: (nil) 2024-09-11T23:13:36.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_size: 4096 2024-09-11T23:13:36.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_size_deviation: 10 2024-09-11T23:13:36.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_restart_interval: 16 2024-09-11T23:13:36.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: index_block_restart_interval: 1 2024-09-11T23:13:36.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: metadata_block_size: 4096 2024-09-11T23:13:36.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: partition_filters: 0 2024-09-11T23:13:36.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: use_delta_encoding: 1 2024-09-11T23:13:36.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: filter_policy: bloomfilter 2024-09-11T23:13:36.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: whole_key_filtering: 1 2024-09-11T23:13:36.066 INFO:journalctl@ceph.mon.c.smithi097.stdout: verify_compression: 0 2024-09-11T23:13:36.067 INFO:journalctl@ceph.mon.c.smithi097.stdout: read_amp_bytes_per_bit: 0 2024-09-11T23:13:36.067 INFO:journalctl@ceph.mon.c.smithi097.stdout: format_version: 5 2024-09-11T23:13:36.067 INFO:journalctl@ceph.mon.c.smithi097.stdout: enable_index_compression: 1 2024-09-11T23:13:36.067 INFO:journalctl@ceph.mon.c.smithi097.stdout: block_align: 0 2024-09-11T23:13:36.067 INFO:journalctl@ceph.mon.c.smithi097.stdout: max_auto_readahead_size: 262144 2024-09-11T23:13:36.067 INFO:journalctl@ceph.mon.c.smithi097.stdout: prepopulate_block_cache: 0 2024-09-11T23:13:36.067 INFO:journalctl@ceph.mon.c.smithi097.stdout: initial_auto_readahead_size: 8192 2024-09-11T23:13:36.067 INFO:journalctl@ceph.mon.c.smithi097.stdout: num_file_reads_for_auto_readahead: 2 2024-09-11T23:13:36.067 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T23:13:36.067 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T23:13:36.067 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compression: NoCompression 2024-09-11T23:13:36.067 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T23:13:36.067 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T23:13:36.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T23:13:36.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.num_levels: 7 2024-09-11T23:13:36.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T23:13:36.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T23:13:36.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T23:13:36.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T23:13:36.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T23:13:36.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T23:13:36.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T23:13:36.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T23:13:36.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T23:13:36.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T23:13:36.068 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-11T23:13:36.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-11T23:13:36.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T23:13:36.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T23:13:36.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T23:13:36.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T23:13:36.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T23:13:36.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-11T23:13:36.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T23:13:36.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T23:13:36.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-11T23:13:36.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T23:13:36.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T23:13:36.069 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T23:13:36.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T23:13:36.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T23:13:36.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T23:13:36.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T23:13:36.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T23:13:36.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T23:13:36.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T23:13:36.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T23:13:36.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T23:13:36.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T23:13:36.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T23:13:36.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T23:13:36.070 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T23:13:36.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T23:13:36.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-11T23:13:36.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.arena_block_size: 1048576 2024-09-11T23:13:36.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T23:13:36.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T23:13:36.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T23:13:36.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T23:13:36.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T23:13:36.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T23:13:36.071 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T23:13:36.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T23:13:36.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T23:13:36.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T23:13:36.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T23:13:36.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T23:13:36.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T23:13:36.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T23:13:36.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.inplace_update_support: 0 2024-09-11T23:13:36.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T23:13:36.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T23:13:36.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T23:13:36.072 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T23:13:36.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.bloom_locality: 0 2024-09-11T23:13:36.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.max_successive_merges: 0 2024-09-11T23:13:36.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T23:13:36.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T23:13:36.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T23:13:36.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T23:13:36.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.ttl: 2592000 2024-09-11T23:13:36.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T23:13:36.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-11T23:13:36.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-11T23:13:36.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.enable_blob_files: false 2024-09-11T23:13:36.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.min_blob_size: 0 2024-09-11T23:13:36.073 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T23:13:36.074 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T23:13:36.074 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T23:13:36.074 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T23:13:36.074 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-11T23:13:36.074 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-11T23:13:36.074 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.blob_file_starting_level: 0 2024-09-11T23:13:36.074 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-11T23:13:36.074 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T23:13:36.074 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-11T23:13:36.074 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d825e307-f20f-4e47-87aa-66ee27598cbe 2024-09-11T23:13:36.074 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726096415729433, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-11T23:13:36.074 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-11T23:13:36.074 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726096415729877, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726096415, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d825e307-f20f-4e47-87aa-66ee27598cbe", "db_session_id": "Y11KOOG60W8ZQGDG4XJG", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-11T23:13:36.075 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726096415729987, "job": 1, "event": "recovery_finished"} 2024-09-11T23:13:36.075 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-11T23:13:36.075 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T23:13:36.075 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5560c3070e00 2024-09-11T23:13:36.075 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: DB pointer 0x5560c316c000 2024-09-11T23:13:36.075 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-11T23:13:36.075 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-11T23:13:36.075 INFO:journalctl@ceph.mon.c.smithi097.stdout: ** DB Stats ** 2024-09-11T23:13:36.075 INFO:journalctl@ceph.mon.c.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T23:13:36.075 INFO:journalctl@ceph.mon.c.smithi097.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T23:13:36.075 INFO:journalctl@ceph.mon.c.smithi097.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T23:13:36.075 INFO:journalctl@ceph.mon.c.smithi097.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T23:13:36.076 INFO:journalctl@ceph.mon.c.smithi097.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T23:13:36.076 INFO:journalctl@ceph.mon.c.smithi097.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T23:13:36.076 INFO:journalctl@ceph.mon.c.smithi097.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T23:13:36.076 INFO:journalctl@ceph.mon.c.smithi097.stdout: 2024-09-11T23:13:36.076 INFO:journalctl@ceph.mon.c.smithi097.stdout: ** Compaction Stats [default] ** 2024-09-11T23:13:36.076 INFO:journalctl@ceph.mon.c.smithi097.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T23:13:36.076 INFO:journalctl@ceph.mon.c.smithi097.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-11T23:13:36.076 INFO:journalctl@ceph.mon.c.smithi097.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-11T23:13:36.076 INFO:journalctl@ceph.mon.c.smithi097.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-11T23:13:36.076 INFO:journalctl@ceph.mon.c.smithi097.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-11T23:13:36.076 INFO:journalctl@ceph.mon.c.smithi097.stdout: 2024-09-11T23:13:36.076 INFO:journalctl@ceph.mon.c.smithi097.stdout: ** Compaction Stats [default] ** 2024-09-11T23:13:36.076 INFO:journalctl@ceph.mon.c.smithi097.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T23:13:36.077 INFO:journalctl@ceph.mon.c.smithi097.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T23:13:36.077 INFO:journalctl@ceph.mon.c.smithi097.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-11T23:13:36.077 INFO:journalctl@ceph.mon.c.smithi097.stdout: 2024-09-11T23:13:36.077 INFO:journalctl@ceph.mon.c.smithi097.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-11T23:13:36.077 INFO:journalctl@ceph.mon.c.smithi097.stdout: 2024-09-11T23:13:36.077 INFO:journalctl@ceph.mon.c.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T23:13:36.077 INFO:journalctl@ceph.mon.c.smithi097.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T23:13:36.077 INFO:journalctl@ceph.mon.c.smithi097.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T23:13:36.077 INFO:journalctl@ceph.mon.c.smithi097.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T23:13:36.077 INFO:journalctl@ceph.mon.c.smithi097.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T23:13:36.077 INFO:journalctl@ceph.mon.c.smithi097.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T23:13:36.077 INFO:journalctl@ceph.mon.c.smithi097.stdout: Cumulative compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T23:13:36.078 INFO:journalctl@ceph.mon.c.smithi097.stdout: Interval compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T23:13:36.078 INFO:journalctl@ceph.mon.c.smithi097.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T23:13:36.078 INFO:journalctl@ceph.mon.c.smithi097.stdout: Block cache BinnedLRUCache@0x5560c30431f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-11T23:13:36.078 INFO:journalctl@ceph.mon.c.smithi097.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-11T23:13:36.078 INFO:journalctl@ceph.mon.c.smithi097.stdout: 2024-09-11T23:13:36.078 INFO:journalctl@ceph.mon.c.smithi097.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T23:13:36.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-11T23:13:36.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: using public_addr v2:172.21.15.97:0/0 -> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] 2024-09-11T23:13:36.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: starting mon.c rank -1 at public addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] at bind addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:13:36.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: mon.c@-1(???) e0 preinit fsid 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:13:36.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: mon.c@-1(synchronizing).mds e1 new map 2024-09-11T23:13:36.078 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-11T23:13:36.078 INFO:journalctl@ceph.mon.c.smithi097.stdout: e1 2024-09-11T23:13:36.078 INFO:journalctl@ceph.mon.c.smithi097.stdout: btime 2024-09-11T23:12:03:504233+0000 2024-09-11T23:13:36.079 INFO:journalctl@ceph.mon.c.smithi097.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T23:13:36.079 INFO:journalctl@ceph.mon.c.smithi097.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T23:13:36.079 INFO:journalctl@ceph.mon.c.smithi097.stdout: legacy client fscid: -1 2024-09-11T23:13:36.079 INFO:journalctl@ceph.mon.c.smithi097.stdout: 2024-09-11T23:13:36.079 INFO:journalctl@ceph.mon.c.smithi097.stdout: No filesystems configured 2024-09-11T23:13:36.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-11T23:13:36.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-11T23:13:36.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-11T23:13:36.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-11T23:13:36.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-11T23:13:36.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-11T23:13:36.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T23:13:36.079 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T23:13:36.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T23:13:36.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T23:13:36.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:36.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:36.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: Added host smithi097 2024-09-11T23:13:36.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:36.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:36.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:36.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:36.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:13:36.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:36.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1921841381' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T23:13:36.080 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1921841381' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T23:13:36.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:13:36.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:36.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:36.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:36.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:36.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:13:36.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:36.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:36.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:36.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: Updating smithi097:/etc/ceph/ceph.conf 2024-09-11T23:13:36.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: Updating smithi097:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:13:36.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:13:36.081 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:36.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:36.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:36.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:36.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:13:36.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:36.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: Updating smithi097:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:13:36.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi090:172.21.15.90=b;smithi097:172.21.15.97=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:13:36.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: Saving service mon spec with placement smithi043:172.21.15.43=a;smithi090:172.21.15.90=b;smithi097:172.21.15.97=c;count:3 2024-09-11T23:13:36.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: Deploying daemon mon.c on smithi097 2024-09-11T23:13:36.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:36.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='client.? 172.21.15.97:0/66118479' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:13:36.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:36.082 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: from='client.? 172.21.15.97:0/707173068' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:13:36.083 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 ceph-mon[35562]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-11T23:13:36.083 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 bash[35528]: fba50890eb516b71025d6dc58dcaeba7aa072005f64b9571c16610ab3f9d03ef 2024-09-11T23:13:36.083 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:35 smithi097 systemd[1]: Started Ceph mon.c for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:13:36.599 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T23:13:36.599 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T23:13:36.853 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:13:40.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.864+0000 7fd31af70640 1 -- 172.21.15.97:0/2156812705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd300003530 msgr2=0x7fd3000059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:40.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.864+0000 7fd31af70640 1 --2- 172.21.15.97:0/2156812705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd300003530 0x7fd3000059c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd314102af0 tx=0x7fd30402bdd0 comp rx=0 tx=0).stop 2024-09-11T23:13:40.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.865+0000 7fd31af70640 1 -- 172.21.15.97:0/2156812705 shutdown_connections 2024-09-11T23:13:40.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.865+0000 7fd31af70640 1 --2- 172.21.15.97:0/2156812705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd300003530 0x7fd3000059c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:40.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.865+0000 7fd31af70640 1 -- 172.21.15.97:0/2156812705 >> 172.21.15.97:0/2156812705 conn(0x7fd3140fd320 msgr2=0x7fd3140ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:40.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.865+0000 7fd31af70640 1 -- 172.21.15.97:0/2156812705 shutdown_connections 2024-09-11T23:13:40.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.865+0000 7fd31af70640 1 -- 172.21.15.97:0/2156812705 wait complete. 2024-09-11T23:13:40.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.866+0000 7fd31af70640 1 Processor -- start 2024-09-11T23:13:40.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.866+0000 7fd31af70640 1 -- start start 2024-09-11T23:13:40.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.866+0000 7fd31af70640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd300003530 0x7fd314197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:40.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.866+0000 7fd31af70640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd314197760 0x7fd31419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:40.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.866+0000 7fd31af70640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd314109780 con 0x7fd300003530 2024-09-11T23:13:40.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.866+0000 7fd31af70640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd314109600 con 0x7fd314197760 2024-09-11T23:13:40.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.866+0000 7fd313fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd314197760 0x7fd31419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:40.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.866+0000 7fd313fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd314197760 0x7fd31419baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47990/0 (socket says 172.21.15.97:47990) 2024-09-11T23:13:40.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.866+0000 7fd313fff640 1 -- 172.21.15.97:0/3359110262 learned_addr learned my addr 172.21.15.97:0/3359110262 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-11T23:13:40.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.866+0000 7fd318d0c640 1 --2- 172.21.15.97:0/3359110262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd300003530 0x7fd314197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:40.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.867+0000 7fd313fff640 1 -- 172.21.15.97:0/3359110262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd314197760 msgr2=0x7fd31419baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-11T23:13:40.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.867+0000 7fd313fff640 1 -- 172.21.15.97:0/3359110262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd314197760 msgr2=0x7fd31419baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T23:13:40.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.867+0000 7fd313fff640 1 --2- 172.21.15.97:0/3359110262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd314197760 0x7fd31419baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T23:13:40.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.867+0000 7fd313fff640 1 --2- 172.21.15.97:0/3359110262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd314197760 0x7fd31419baf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:13:40.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.867+0000 7fd318d0c640 1 -- 172.21.15.97:0/3359110262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd314197760 msgr2=0x7fd31419baf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:13:40.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.867+0000 7fd318d0c640 1 --2- 172.21.15.97:0/3359110262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd314197760 0x7fd31419baf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:40.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.867+0000 7fd318d0c640 1 -- 172.21.15.97:0/3359110262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd314197d80 con 0x7fd300003530 2024-09-11T23:13:40.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.867+0000 7fd318d0c640 1 --2- 172.21.15.97:0/3359110262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd300003530 0x7fd314197220 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd314102af0 tx=0x7fd30402ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:40.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.867+0000 7fd311ffb640 1 -- 172.21.15.97:0/3359110262 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd304032ac0 con 0x7fd300003530 2024-09-11T23:13:40.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.867+0000 7fd311ffb640 1 -- 172.21.15.97:0/3359110262 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd304032c60 con 0x7fd300003530 2024-09-11T23:13:40.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.867+0000 7fd31af70640 1 -- 172.21.15.97:0/3359110262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31419c030 con 0x7fd300003530 2024-09-11T23:13:40.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.867+0000 7fd311ffb640 1 -- 172.21.15.97:0/3359110262 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd304005020 con 0x7fd300003530 2024-09-11T23:13:40.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.868+0000 7fd31af70640 1 -- 172.21.15.97:0/3359110262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31419c570 con 0x7fd300003530 2024-09-11T23:13:40.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.869+0000 7fd311ffb640 1 -- 172.21.15.97:0/3359110262 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd304043020 con 0x7fd300003530 2024-09-11T23:13:40.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.869+0000 7fd31af70640 1 -- 172.21.15.97:0/3359110262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd314102da0 con 0x7fd300003530 2024-09-11T23:13:40.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.869+0000 7fd311ffb640 1 --2- 172.21.15.97:0/3359110262 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd2f4038e00 0x7fd2f403b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:40.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.869+0000 7fd311ffb640 1 -- 172.21.15.97:0/3359110262 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd304041020 con 0x7fd300003530 2024-09-11T23:13:40.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.869+0000 7fd313fff640 1 --2- 172.21.15.97:0/3359110262 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd2f4038e00 0x7fd2f403b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:40.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.870+0000 7fd313fff640 1 --2- 172.21.15.97:0/3359110262 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd2f4038e00 0x7fd2f403b2c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009100 tx=0x7fd2fc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:40.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:40.874+0000 7fd311ffb640 1 -- 172.21.15.97:0/3359110262 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd304033e80 con 0x7fd300003530 2024-09-11T23:13:41.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:41.108+0000 7fd31af70640 1 -- 172.21.15.97:0/3359110262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd31419c8e0 con 0x7fd300003530 2024-09-11T23:13:41.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:41.109+0000 7fd311ffb640 1 -- 172.21.15.97:0/3359110262 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7fd3040346d0 con 0x7fd300003530 2024-09-11T23:13:41.110 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:13:41.110 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:13:41.110 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: mon.a calling monitor election 2024-09-11T23:13:41.110 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:41.111 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:13:41.111 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:13:41.111 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: mon.c calling monitor election 2024-09-11T23:13:41.111 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:41.111 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:13:41.111 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:13:41.111 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:13:41.111 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T23:13:41.112 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: monmap epoch 2 2024-09-11T23:13:41.112 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: fsid 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:13:41.112 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: last_changed 2024-09-11T23:13:35.756958+0000 2024-09-11T23:13:41.112 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: created 2024-09-11T23:12:00.389048+0000 2024-09-11T23:13:41.112 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: min_mon_release 19 (squid) 2024-09-11T23:13:41.112 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: election_strategy: 1 2024-09-11T23:13:41.112 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-11T23:13:41.112 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: 1: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.c 2024-09-11T23:13:41.113 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: fsmap 2024-09-11T23:13:41.113 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:13:41.113 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: mgrmap e13: x(active, since 40s) 2024-09-11T23:13:41.113 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: overall HEALTH_OK 2024-09-11T23:13:41.113 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:41.113 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:41.113 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:41.113 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:13:41.113 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:40 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:41.114 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:13:41.114 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":2,"fsid":"25c0cd84-7093-11ef-bcea-c7b262605968","modified":"2024-09-11T23:13:35.756958Z","created":"2024-09-11T23:12:00.389048Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T23:13:41.115 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 2 2024-09-11T23:13:41.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:41.112+0000 7fd31af70640 1 -- 172.21.15.97:0/3359110262 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd2f4038e00 msgr2=0x7fd2f403b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:41.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:41.112+0000 7fd31af70640 1 --2- 172.21.15.97:0/3359110262 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd2f4038e00 0x7fd2f403b2c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009100 tx=0x7fd2fc008040 comp rx=0 tx=0).stop 2024-09-11T23:13:41.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:41.112+0000 7fd31af70640 1 -- 172.21.15.97:0/3359110262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd300003530 msgr2=0x7fd314197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:41.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:41.112+0000 7fd31af70640 1 --2- 172.21.15.97:0/3359110262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd300003530 0x7fd314197220 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd314102af0 tx=0x7fd30402ca90 comp rx=0 tx=0).stop 2024-09-11T23:13:41.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:41.112+0000 7fd31af70640 1 -- 172.21.15.97:0/3359110262 shutdown_connections 2024-09-11T23:13:41.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:41.112+0000 7fd31af70640 1 --2- 172.21.15.97:0/3359110262 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd2f4038e00 0x7fd2f403b2c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:41.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:41.112+0000 7fd31af70640 1 --2- 172.21.15.97:0/3359110262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd314197760 0x7fd31419baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:41.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:41.112+0000 7fd31af70640 1 --2- 172.21.15.97:0/3359110262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd300003530 0x7fd314197220 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:41.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:41.112+0000 7fd31af70640 1 -- 172.21.15.97:0/3359110262 >> 172.21.15.97:0/3359110262 conn(0x7fd3140fd320 msgr2=0x7fd3140ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:41.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:41.112+0000 7fd31af70640 1 -- 172.21.15.97:0/3359110262 shutdown_connections 2024-09-11T23:13:41.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:41.112+0000 7fd31af70640 1 -- 172.21.15.97:0/3359110262 wait complete. 2024-09-11T23:13:41.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:13:41.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:13:41.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: mon.a calling monitor election 2024-09-11T23:13:41.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:41.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:13:41.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:13:41.118 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: mon.c calling monitor election 2024-09-11T23:13:41.118 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:41.118 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:13:41.118 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:13:41.118 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:13:41.118 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T23:13:41.118 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: monmap epoch 2 2024-09-11T23:13:41.118 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: fsid 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:13:41.118 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: last_changed 2024-09-11T23:13:35.756958+0000 2024-09-11T23:13:41.118 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: created 2024-09-11T23:12:00.389048+0000 2024-09-11T23:13:41.118 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: min_mon_release 19 (squid) 2024-09-11T23:13:41.118 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: election_strategy: 1 2024-09-11T23:13:41.119 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-11T23:13:41.119 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: 1: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.c 2024-09-11T23:13:41.119 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: fsmap 2024-09-11T23:13:41.119 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:13:41.119 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: mgrmap e13: x(active, since 40s) 2024-09-11T23:13:41.119 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: overall HEALTH_OK 2024-09-11T23:13:41.119 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:41.119 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:41.119 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:41.119 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:13:41.119 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:40 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:42.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:41 smithi097 ceph-mon[35562]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:42.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:41 smithi097 ceph-mon[35562]: Deploying daemon mon.b on smithi090 2024-09-11T23:13:42.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:41 smithi097 ceph-mon[35562]: from='client.? 172.21.15.97:0/3359110262' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:13:42.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:13:42.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:41 smithi043 ceph-mon[30950]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:42.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:41 smithi043 ceph-mon[30950]: Deploying daemon mon.b on smithi090 2024-09-11T23:13:42.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:41 smithi043 ceph-mon[30950]: from='client.? 172.21.15.97:0/3359110262' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:13:42.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:13:42.807 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T23:13:42.807 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T23:13:43.058 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:13:43.110 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:13:42 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:13:42.755+0000 7f5b3a8f1640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-11T23:13:43.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.451+0000 7fddff5da640 1 -- 172.21.15.97:0/1864677159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf8106630 msgr2=0x7fddf8106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:43.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.451+0000 7fddff5da640 1 --2- 172.21.15.97:0/1864677159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf8106630 0x7fddf8106a10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009880 tx=0x7fddf402a5d0 comp rx=0 tx=0).stop 2024-09-11T23:13:43.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.451+0000 7fddff5da640 1 -- 172.21.15.97:0/1864677159 shutdown_connections 2024-09-11T23:13:43.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.451+0000 7fddff5da640 1 --2- 172.21.15.97:0/1864677159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf8106630 0x7fddf8106a10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:43.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.451+0000 7fddff5da640 1 -- 172.21.15.97:0/1864677159 >> 172.21.15.97:0/1864677159 conn(0x7fddf80fc220 msgr2=0x7fddf80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:43.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.451+0000 7fddff5da640 1 -- 172.21.15.97:0/1864677159 shutdown_connections 2024-09-11T23:13:43.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.451+0000 7fddff5da640 1 -- 172.21.15.97:0/1864677159 wait complete. 2024-09-11T23:13:43.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.452+0000 7fddff5da640 1 Processor -- start 2024-09-11T23:13:43.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.452+0000 7fddff5da640 1 -- start start 2024-09-11T23:13:43.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.452+0000 7fddff5da640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf8106630 0x7fddf8197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:43.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.452+0000 7fddff5da640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf8197960 0x7fddf819bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:43.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.452+0000 7fddff5da640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fddf810ac60 con 0x7fddf8106630 2024-09-11T23:13:43.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.452+0000 7fddff5da640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fddf810aae0 con 0x7fddf8197960 2024-09-11T23:13:43.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.452+0000 7fddfcb75640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf8197960 0x7fddf819bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:43.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.452+0000 7fddfcb75640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf8197960 0x7fddf819bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47998/0 (socket says 172.21.15.97:47998) 2024-09-11T23:13:43.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.452+0000 7fddfcb75640 1 -- 172.21.15.97:0/2109435116 learned_addr learned my addr 172.21.15.97:0/2109435116 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-11T23:13:43.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.452+0000 7fddfcb75640 1 -- 172.21.15.97:0/2109435116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf8197960 msgr2=0x7fddf819bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-11T23:13:43.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.452+0000 7fddfcb75640 1 -- 172.21.15.97:0/2109435116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf8197960 msgr2=0x7fddf819bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T23:13:43.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.452+0000 7fddfcb75640 1 --2- 172.21.15.97:0/2109435116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf8197960 0x7fddf819bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T23:13:43.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.452+0000 7fddfcb75640 1 --2- 172.21.15.97:0/2109435116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf8197960 0x7fddf819bcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:13:43.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.453+0000 7fddfd376640 1 --2- 172.21.15.97:0/2109435116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf8106630 0x7fddf8197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:43.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.453+0000 7fddfd376640 1 -- 172.21.15.97:0/2109435116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf8197960 msgr2=0x7fddf819bcf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:13:43.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.453+0000 7fddfd376640 1 --2- 172.21.15.97:0/2109435116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf8197960 0x7fddf819bcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:43.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.453+0000 7fddfd376640 1 -- 172.21.15.97:0/2109435116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf8197f80 con 0x7fddf8106630 2024-09-11T23:13:43.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.453+0000 7fddfd376640 1 --2- 172.21.15.97:0/2109435116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf8106630 0x7fddf8197420 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fddf4006010 tx=0x7fddf4032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:43.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.453+0000 7fddee7fc640 1 -- 172.21.15.97:0/2109435116 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fddf4035070 con 0x7fddf8106630 2024-09-11T23:13:43.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.453+0000 7fddee7fc640 1 -- 172.21.15.97:0/2109435116 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddf4032e40 con 0x7fddf8106630 2024-09-11T23:13:43.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.453+0000 7fddff5da640 1 -- 172.21.15.97:0/2109435116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf819c230 con 0x7fddf8106630 2024-09-11T23:13:43.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.453+0000 7fddee7fc640 1 -- 172.21.15.97:0/2109435116 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fddf403bb50 con 0x7fddf8106630 2024-09-11T23:13:43.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.453+0000 7fddff5da640 1 -- 172.21.15.97:0/2109435116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf819c710 con 0x7fddf8106630 2024-09-11T23:13:43.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.454+0000 7fddee7fc640 1 -- 172.21.15.97:0/2109435116 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fddf4042030 con 0x7fddf8106630 2024-09-11T23:13:43.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.454+0000 7fddff5da640 1 -- 172.21.15.97:0/2109435116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf8101ca0 con 0x7fddf8106630 2024-09-11T23:13:43.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.454+0000 7fddee7fc640 1 --2- 172.21.15.97:0/2109435116 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fdde4038e00 0x7fdde403b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:43.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.454+0000 7fddee7fc640 1 -- 172.21.15.97:0/2109435116 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fddf4040070 con 0x7fddf8106630 2024-09-11T23:13:43.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.455+0000 7fddfcb75640 1 --2- 172.21.15.97:0/2109435116 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fdde4038e00 0x7fdde403b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:43.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.455+0000 7fddfcb75640 1 --2- 172.21.15.97:0/2109435116 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fdde4038e00 0x7fdde403b2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdde8009100 tx=0x7fdde8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:43.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.460+0000 7fddee7fc640 1 -- 172.21.15.97:0/2109435116 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fddf403b460 con 0x7fddf8106630 2024-09-11T23:13:43.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.687+0000 7fddff5da640 1 -- 172.21.15.97:0/2109435116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fddf819c9c0 con 0x7fddf8106630 2024-09-11T23:13:43.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.687+0000 7fddee7fc640 1 -- 172.21.15.97:0/2109435116 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7fddf4041d80 con 0x7fddf8106630 2024-09-11T23:13:43.688 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:13:43.688 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":2,"fsid":"25c0cd84-7093-11ef-bcea-c7b262605968","modified":"2024-09-11T23:13:35.756958Z","created":"2024-09-11T23:12:00.389048Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T23:13:43.688 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 2 2024-09-11T23:13:43.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.689+0000 7fddff5da640 1 -- 172.21.15.97:0/2109435116 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fdde4038e00 msgr2=0x7fdde403b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:43.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.689+0000 7fddff5da640 1 --2- 172.21.15.97:0/2109435116 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fdde4038e00 0x7fdde403b2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdde8009100 tx=0x7fdde8008040 comp rx=0 tx=0).stop 2024-09-11T23:13:43.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.689+0000 7fddff5da640 1 -- 172.21.15.97:0/2109435116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf8106630 msgr2=0x7fddf8197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:43.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.689+0000 7fddff5da640 1 --2- 172.21.15.97:0/2109435116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf8106630 0x7fddf8197420 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fddf4006010 tx=0x7fddf4032c50 comp rx=0 tx=0).stop 2024-09-11T23:13:43.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.689+0000 7fddff5da640 1 -- 172.21.15.97:0/2109435116 shutdown_connections 2024-09-11T23:13:43.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.689+0000 7fddff5da640 1 --2- 172.21.15.97:0/2109435116 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fdde4038e00 0x7fdde403b2c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:43.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.689+0000 7fddff5da640 1 --2- 172.21.15.97:0/2109435116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf8197960 0x7fddf819bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:43.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.689+0000 7fddff5da640 1 --2- 172.21.15.97:0/2109435116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf8106630 0x7fddf8197420 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:43.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.689+0000 7fddff5da640 1 -- 172.21.15.97:0/2109435116 >> 172.21.15.97:0/2109435116 conn(0x7fddf80fc220 msgr2=0x7fddf81900f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:43.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.689+0000 7fddff5da640 1 -- 172.21.15.97:0/2109435116 shutdown_connections 2024-09-11T23:13:43.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:43.689+0000 7fddff5da640 1 -- 172.21.15.97:0/2109435116 wait complete. 2024-09-11T23:13:44.041 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:43 smithi097 ceph-mon[35562]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:44.041 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:43 smithi097 ceph-mon[35562]: from='client.? 172.21.15.97:0/2109435116' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:13:44.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:43 smithi043 ceph-mon[30950]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:44.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:43 smithi043 ceph-mon[30950]: from='client.? 172.21.15.97:0/2109435116' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:13:44.636 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:44 smithi090 ceph-mon[35130]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-11T23:13:44.636 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:44 smithi090 bash[35111]: 172cb0b311f0fa00e6cf1d9b273c04a111c4cc04939f268b0e9bfe563bfa647d 2024-09-11T23:13:45.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:44 smithi090 systemd[1]: Started Ceph mon.b for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:13:45.350 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T23:13:45.350 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T23:13:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:13:49.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.628+0000 7fca9fd0d640 1 -- 172.21.15.97:0/3136424372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca84003530 msgr2=0x7fca840059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:49.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.628+0000 7fca9fd0d640 1 --2- 172.21.15.97:0/3136424372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca84003530 0x7fca840059c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fca98102af0 tx=0x7fca9402c3f0 comp rx=0 tx=0).stop 2024-09-11T23:13:49.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.628+0000 7fca9fd0d640 1 -- 172.21.15.97:0/3136424372 shutdown_connections 2024-09-11T23:13:49.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.628+0000 7fca9fd0d640 1 --2- 172.21.15.97:0/3136424372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca84003530 0x7fca840059c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:49.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.628+0000 7fca9fd0d640 1 -- 172.21.15.97:0/3136424372 >> 172.21.15.97:0/3136424372 conn(0x7fca980fd320 msgr2=0x7fca980ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:49.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.628+0000 7fca9fd0d640 1 -- 172.21.15.97:0/3136424372 shutdown_connections 2024-09-11T23:13:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.629+0000 7fca9fd0d640 1 -- 172.21.15.97:0/3136424372 wait complete. 2024-09-11T23:13:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.629+0000 7fca9fd0d640 1 Processor -- start 2024-09-11T23:13:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.629+0000 7fca9fd0d640 1 -- start start 2024-09-11T23:13:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9fd0d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca84003530 0x7fca98197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9fd0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca98197840 0x7fca9819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9fd0d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca9819c110 0x7fca9819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9fd0d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fca98109780 con 0x7fca98197840 2024-09-11T23:13:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9fd0d640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fca98109600 con 0x7fca9819c110 2024-09-11T23:13:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9fd0d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fca98109900 con 0x7fca84003530 2024-09-11T23:13:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9daa9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca84003530 0x7fca98197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9daa9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca84003530 0x7fca98197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33956/0 (socket says 172.21.15.97:33956) 2024-09-11T23:13:49.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9daa9640 1 -- 172.21.15.97:0/1838627185 learned_addr learned my addr 172.21.15.97:0/1838627185 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-11T23:13:49.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9d2a8640 1 --2- 172.21.15.97:0/1838627185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca98197840 0x7fca9819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:49.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9daa9640 1 -- 172.21.15.97:0/1838627185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca84003530 msgr2=0x7fca98197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-11T23:13:49.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9daa9640 1 -- 172.21.15.97:0/1838627185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca84003530 msgr2=0x7fca98197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T23:13:49.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9daa9640 1 --2- 172.21.15.97:0/1838627185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca84003530 0x7fca98197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T23:13:49.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9daa9640 1 --2- 172.21.15.97:0/1838627185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca84003530 0x7fca98197300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:13:49.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9e2aa640 1 --2- 172.21.15.97:0/1838627185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca9819c110 0x7fca9819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:49.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9e2aa640 1 -- 172.21.15.97:0/1838627185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca9819c110 msgr2=0x7fca9819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-11T23:13:49.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9e2aa640 1 -- 172.21.15.97:0/1838627185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca9819c110 msgr2=0x7fca9819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T23:13:49.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9e2aa640 1 --2- 172.21.15.97:0/1838627185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca9819c110 0x7fca9819c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T23:13:49.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9e2aa640 1 --2- 172.21.15.97:0/1838627185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca9819c110 0x7fca9819c5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:13:49.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9d2a8640 1 -- 172.21.15.97:0/1838627185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca84003530 msgr2=0x7fca98197300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:13:49.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9d2a8640 1 --2- 172.21.15.97:0/1838627185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca84003530 0x7fca98197300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:49.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9d2a8640 1 -- 172.21.15.97:0/1838627185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca9819c110 msgr2=0x7fca9819c5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:13:49.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9d2a8640 1 --2- 172.21.15.97:0/1838627185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca9819c110 0x7fca9819c5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:49.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.630+0000 7fca9d2a8640 1 -- 172.21.15.97:0/1838627185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca981a3e80 con 0x7fca98197840 2024-09-11T23:13:49.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.631+0000 7fca9d2a8640 1 --2- 172.21.15.97:0/1838627185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca98197840 0x7fca9819bbd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fca8800b450 tx=0x7fca8800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:49.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.631+0000 7fca8effd640 1 -- 172.21.15.97:0/1838627185 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88004230 con 0x7fca98197840 2024-09-11T23:13:49.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.631+0000 7fca8effd640 1 -- 172.21.15.97:0/1838627185 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca880043d0 con 0x7fca98197840 2024-09-11T23:13:49.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.631+0000 7fca9fd0d640 1 -- 172.21.15.97:0/1838627185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca981a40b0 con 0x7fca98197840 2024-09-11T23:13:49.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.631+0000 7fca8effd640 1 -- 172.21.15.97:0/1838627185 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88003cb0 con 0x7fca98197840 2024-09-11T23:13:49.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.631+0000 7fca9fd0d640 1 -- 172.21.15.97:0/1838627185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca981a45c0 con 0x7fca98197840 2024-09-11T23:13:49.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.632+0000 7fca8effd640 1 -- 172.21.15.97:0/1838627185 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fca8800f050 con 0x7fca98197840 2024-09-11T23:13:49.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.632+0000 7fca9fd0d640 1 -- 172.21.15.97:0/1838627185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca9810a8b0 con 0x7fca98197840 2024-09-11T23:13:49.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.633+0000 7fca8effd640 1 --2- 172.21.15.97:0/1838627185 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fca7c0394f0 0x7fca7c03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:49.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.633+0000 7fca8effd640 1 -- 172.21.15.97:0/1838627185 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fca8804be70 con 0x7fca98197840 2024-09-11T23:13:49.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.633+0000 7fca9daa9640 1 --2- 172.21.15.97:0/1838627185 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fca7c0394f0 0x7fca7c03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:49.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.633+0000 7fca9daa9640 1 --2- 172.21.15.97:0/1838627185 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fca7c0394f0 0x7fca7c03b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fca94005fd0 tx=0x7fca94005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:49.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.643+0000 7fca8effd640 1 -- 172.21.15.97:0/1838627185 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca8800d040 con 0x7fca98197840 2024-09-11T23:13:49.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.885+0000 7fca9fd0d640 1 -- 172.21.15.97:0/1838627185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fca9802d7c0 con 0x7fca98197840 2024-09-11T23:13:49.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:49.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:13:49.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: mon.a calling monitor election 2024-09-11T23:13:49.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: mon.c calling monitor election 2024-09-11T23:13:49.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:49.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:13:49.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:49.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:49.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:49.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:49.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:49.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:49.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:49.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T23:13:49.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: monmap epoch 3 2024-09-11T23:13:49.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: fsid 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:13:49.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: last_changed 2024-09-11T23:13:44.616446+0000 2024-09-11T23:13:49.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: created 2024-09-11T23:12:00.389048+0000 2024-09-11T23:13:49.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: min_mon_release 19 (squid) 2024-09-11T23:13:49.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: election_strategy: 1 2024-09-11T23:13:49.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-11T23:13:49.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: 1: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.c 2024-09-11T23:13:49.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: 2: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.b 2024-09-11T23:13:49.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: fsmap 2024-09-11T23:13:49.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:13:49.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: mgrmap e13: x(active, since 49s) 2024-09-11T23:13:49.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: overall HEALTH_OK 2024-09-11T23:13:49.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:49.888 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:49.889 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:49.889 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:49.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.886+0000 7fca8effd640 1 -- 172.21.15.97:0/1838627185 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1275 (secure 0 0 0) 0x7fca88003e50 con 0x7fca98197840 2024-09-11T23:13:49.889 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:13:49.890 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":3,"fsid":"25c0cd84-7093-11ef-bcea-c7b262605968","modified":"2024-09-11T23:13:44.616446Z","created":"2024-09-11T23:12:00.389048Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T23:13:49.890 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 3 2024-09-11T23:13:49.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.889+0000 7fca9fd0d640 1 -- 172.21.15.97:0/1838627185 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fca7c0394f0 msgr2=0x7fca7c03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:49.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.889+0000 7fca9fd0d640 1 --2- 172.21.15.97:0/1838627185 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fca7c0394f0 0x7fca7c03b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fca94005fd0 tx=0x7fca94005f60 comp rx=0 tx=0).stop 2024-09-11T23:13:49.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.889+0000 7fca9fd0d640 1 -- 172.21.15.97:0/1838627185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca98197840 msgr2=0x7fca9819bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:49.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.889+0000 7fca9fd0d640 1 --2- 172.21.15.97:0/1838627185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca98197840 0x7fca9819bbd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fca8800b450 tx=0x7fca8800b910 comp rx=0 tx=0).stop 2024-09-11T23:13:49.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.889+0000 7fca9fd0d640 1 -- 172.21.15.97:0/1838627185 shutdown_connections 2024-09-11T23:13:49.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.889+0000 7fca9fd0d640 1 --2- 172.21.15.97:0/1838627185 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fca7c0394f0 0x7fca7c03b9b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:49.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.889+0000 7fca9fd0d640 1 --2- 172.21.15.97:0/1838627185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca9819c110 0x7fca9819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:49.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.889+0000 7fca9fd0d640 1 --2- 172.21.15.97:0/1838627185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca98197840 0x7fca9819bbd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:49.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.889+0000 7fca9fd0d640 1 --2- 172.21.15.97:0/1838627185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca84003530 0x7fca98197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:49.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.889+0000 7fca9fd0d640 1 -- 172.21.15.97:0/1838627185 >> 172.21.15.97:0/1838627185 conn(0x7fca980fd320 msgr2=0x7fca980ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:49.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.889+0000 7fca9fd0d640 1 -- 172.21.15.97:0/1838627185 shutdown_connections 2024-09-11T23:13:49.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:49.889+0000 7fca9fd0d640 1 -- 172.21.15.97:0/1838627185 wait complete. 2024-09-11T23:13:50.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:50.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:13:50.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: mon.a calling monitor election 2024-09-11T23:13:50.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: mon.c calling monitor election 2024-09-11T23:13:50.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:50.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:13:50.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:50.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:50.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:50.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:50.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:50.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:50.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:50.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T23:13:50.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: monmap epoch 3 2024-09-11T23:13:50.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: fsid 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:13:50.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: last_changed 2024-09-11T23:13:44.616446+0000 2024-09-11T23:13:50.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: created 2024-09-11T23:12:00.389048+0000 2024-09-11T23:13:50.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: min_mon_release 19 (squid) 2024-09-11T23:13:50.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: election_strategy: 1 2024-09-11T23:13:50.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-11T23:13:50.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: 1: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.c 2024-09-11T23:13:50.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: 2: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.b 2024-09-11T23:13:50.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: fsmap 2024-09-11T23:13:50.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:13:50.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: mgrmap e13: x(active, since 49s) 2024-09-11T23:13:50.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: overall HEALTH_OK 2024-09-11T23:13:50.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:50.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:50.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:50.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:50.509 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-11T23:13:50.509 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-11T23:13:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:13:50.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:50 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:50.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:50 smithi097 ceph-mon[35562]: from='client.? 172.21.15.97:0/1838627185' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:13:50.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:50 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:50.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:50 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:50.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:50 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:50.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:50 smithi043 ceph-mon[30950]: from='client.? 172.21.15.97:0/1838627185' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:13:50.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:50 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:50.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:50 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:51.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.223+0000 7f17f671b640 1 -- 172.21.15.43:0/85104189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17f0106540 msgr2=0x7f17f0106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:51.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.223+0000 7f17f671b640 1 --2- 172.21.15.43:0/85104189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17f0106540 0x7f17f0106920 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009810 tx=0x7f17e002a540 comp rx=0 tx=0).stop 2024-09-11T23:13:51.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.223+0000 7f17f671b640 1 -- 172.21.15.43:0/85104189 shutdown_connections 2024-09-11T23:13:51.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.223+0000 7f17f671b640 1 --2- 172.21.15.43:0/85104189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17f0106540 0x7f17f0106920 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:51.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.223+0000 7f17f671b640 1 -- 172.21.15.43:0/85104189 >> 172.21.15.43:0/85104189 conn(0x7f17f0077310 msgr2=0x7f17f00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:51.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.223+0000 7f17f671b640 1 -- 172.21.15.43:0/85104189 shutdown_connections 2024-09-11T23:13:51.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.223+0000 7f17f671b640 1 -- 172.21.15.43:0/85104189 wait complete. 2024-09-11T23:13:51.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.223+0000 7f17f671b640 1 Processor -- start 2024-09-11T23:13:51.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.224+0000 7f17f671b640 1 -- start start 2024-09-11T23:13:51.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.224+0000 7f17f671b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17f0106540 0x7f17f0197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:51.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.224+0000 7f17f671b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17f01978c0 0x7f17f019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:51.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.224+0000 7f17f671b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17f019c190 0x7f17f019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:51.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.224+0000 7f17f671b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f17f010ab20 con 0x7f17f019c190 2024-09-11T23:13:51.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.224+0000 7f17f671b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f17f010a9a0 con 0x7f17f01978c0 2024-09-11T23:13:51.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.224+0000 7f17f671b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f17f010aca0 con 0x7f17f0106540 2024-09-11T23:13:51.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.224+0000 7f17f4cb8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17f019c190 0x7f17f019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:51.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.224+0000 7f17f4cb8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17f019c190 0x7f17f019c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43182/0 (socket says 172.21.15.43:43182) 2024-09-11T23:13:51.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.224+0000 7f17f4cb8640 1 -- 172.21.15.43:0/2897050166 learned_addr learned my addr 172.21.15.43:0/2897050166 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:13:51.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.224+0000 7f17effff640 1 --2- 172.21.15.43:0/2897050166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17f0106540 0x7f17f0197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:51.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.224+0000 7f17e7fff640 1 --2- 172.21.15.43:0/2897050166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17f01978c0 0x7f17f019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:51.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.224+0000 7f17f4cb8640 1 -- 172.21.15.43:0/2897050166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17f0106540 msgr2=0x7f17f0197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:51.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.224+0000 7f17f4cb8640 1 --2- 172.21.15.43:0/2897050166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17f0106540 0x7f17f0197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:51.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.224+0000 7f17f4cb8640 1 -- 172.21.15.43:0/2897050166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17f01978c0 msgr2=0x7f17f019bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:51.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.224+0000 7f17f4cb8640 1 --2- 172.21.15.43:0/2897050166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17f01978c0 0x7f17f019bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:51.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.224+0000 7f17f4cb8640 1 -- 172.21.15.43:0/2897050166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f01a3f00 con 0x7f17f019c190 2024-09-11T23:13:51.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.224+0000 7f17effff640 1 --2- 172.21.15.43:0/2897050166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17f0106540 0x7f17f0197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:13:51.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.224+0000 7f17e7fff640 1 --2- 172.21.15.43:0/2897050166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17f01978c0 0x7f17f019bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:13:51.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.225+0000 7f17f4cb8640 1 --2- 172.21.15.43:0/2897050166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17f019c190 0x7f17f019c640 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f17e800ba50 tx=0x7f17e800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:51.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.225+0000 7f17edffb640 1 -- 172.21.15.43:0/2897050166 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8019070 con 0x7f17f019c190 2024-09-11T23:13:51.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.225+0000 7f17edffb640 1 -- 172.21.15.43:0/2897050166 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e800cbe0 con 0x7f17f019c190 2024-09-11T23:13:51.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.225+0000 7f17f671b640 1 -- 172.21.15.43:0/2897050166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f01a4130 con 0x7f17f019c190 2024-09-11T23:13:51.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.225+0000 7f17edffb640 1 -- 172.21.15.43:0/2897050166 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e80076a0 con 0x7f17f019c190 2024-09-11T23:13:51.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.225+0000 7f17f671b640 1 -- 172.21.15.43:0/2897050166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f01a4640 con 0x7f17f019c190 2024-09-11T23:13:51.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.226+0000 7f17edffb640 1 -- 172.21.15.43:0/2897050166 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f17e801c070 con 0x7f17f019c190 2024-09-11T23:13:51.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.226+0000 7f17edffb640 1 --2- 172.21.15.43:0/2897050166 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f17dc0394f0 0x7f17dc03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:51.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.226+0000 7f17f671b640 1 -- 172.21.15.43:0/2897050166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f0101bb0 con 0x7f17f019c190 2024-09-11T23:13:51.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.226+0000 7f17edffb640 1 -- 172.21.15.43:0/2897050166 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f17e804b9e0 con 0x7f17f019c190 2024-09-11T23:13:51.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.226+0000 7f17effff640 1 --2- 172.21.15.43:0/2897050166 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f17dc0394f0 0x7f17dc03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:51.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.226+0000 7f17effff640 1 --2- 172.21.15.43:0/2897050166 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f17dc0394f0 0x7f17dc03b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f17e00058e0 tx=0x7f17e0005870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:51.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.231+0000 7f17edffb640 1 -- 172.21.15.43:0/2897050166 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f17e8012e50 con 0x7f17f019c190 2024-09-11T23:13:51.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.408+0000 7f17f671b640 1 -- 172.21.15.43:0/2897050166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f17f0106920 con 0x7f17f019c190 2024-09-11T23:13:51.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.410+0000 7f17edffb640 1 -- 172.21.15.43:0/2897050166 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7f17f0106920 con 0x7f17f019c190 2024-09-11T23:13:51.411 INFO:teuthology.orchestra.run.smithi043.stdout:# minimal ceph.conf for 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:13:51.411 INFO:teuthology.orchestra.run.smithi043.stdout:[global] 2024-09-11T23:13:51.411 INFO:teuthology.orchestra.run.smithi043.stdout: fsid = 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:13:51.411 INFO:teuthology.orchestra.run.smithi043.stdout: mon_host = [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] 2024-09-11T23:13:51.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.412+0000 7f17f671b640 1 -- 172.21.15.43:0/2897050166 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f17dc0394f0 msgr2=0x7f17dc03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:51.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.412+0000 7f17f671b640 1 --2- 172.21.15.43:0/2897050166 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f17dc0394f0 0x7f17dc03b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f17e00058e0 tx=0x7f17e0005870 comp rx=0 tx=0).stop 2024-09-11T23:13:51.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.412+0000 7f17f671b640 1 -- 172.21.15.43:0/2897050166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17f019c190 msgr2=0x7f17f019c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:51.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.412+0000 7f17f671b640 1 --2- 172.21.15.43:0/2897050166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17f019c190 0x7f17f019c640 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f17e800ba50 tx=0x7f17e800bf10 comp rx=0 tx=0).stop 2024-09-11T23:13:51.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.412+0000 7f17f671b640 1 -- 172.21.15.43:0/2897050166 shutdown_connections 2024-09-11T23:13:51.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.412+0000 7f17f671b640 1 --2- 172.21.15.43:0/2897050166 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f17dc0394f0 0x7f17dc03b9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:51.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.412+0000 7f17f671b640 1 --2- 172.21.15.43:0/2897050166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17f019c190 0x7f17f019c640 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:51.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.412+0000 7f17f671b640 1 --2- 172.21.15.43:0/2897050166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17f01978c0 0x7f17f019bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:51.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.412+0000 7f17f671b640 1 --2- 172.21.15.43:0/2897050166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17f0106540 0x7f17f0197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:51.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.412+0000 7f17f671b640 1 -- 172.21.15.43:0/2897050166 >> 172.21.15.43:0/2897050166 conn(0x7f17f0077310 msgr2=0x7f17f018ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:51.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.412+0000 7f17f671b640 1 -- 172.21.15.43:0/2897050166 shutdown_connections 2024-09-11T23:13:51.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:51.412+0000 7f17f671b640 1 -- 172.21.15.43:0/2897050166 wait complete. 2024-09-11T23:13:52.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:52.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:13:52.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: mon.a calling monitor election 2024-09-11T23:13:52.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: mon.c calling monitor election 2024-09-11T23:13:52.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:52.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:13:52.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:52.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:52.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:52.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:52.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:52.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:52.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:52.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T23:13:52.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: monmap epoch 3 2024-09-11T23:13:52.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: fsid 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:13:52.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: last_changed 2024-09-11T23:13:44.616446+0000 2024-09-11T23:13:52.022 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: created 2024-09-11T23:12:00.389048+0000 2024-09-11T23:13:52.022 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: min_mon_release 19 (squid) 2024-09-11T23:13:52.022 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: election_strategy: 1 2024-09-11T23:13:52.022 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-11T23:13:52.022 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: 1: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.c 2024-09-11T23:13:52.022 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: 2: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.b 2024-09-11T23:13:52.022 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: fsmap 2024-09-11T23:13:52.022 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:13:52.022 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: mgrmap e13: x(active, since 49s) 2024-09-11T23:13:52.022 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: overall HEALTH_OK 2024-09-11T23:13:52.022 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.022 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.022 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.022 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.023 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:52.023 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: from='client.? 172.21.15.97:0/1838627185' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T23:13:52.023 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.023 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:51 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:52.167 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-11T23:13:52.167 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T23:13:52.167 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T23:13:52.196 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T23:13:52.196 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:13:52.265 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-11T23:13:52.265 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T23:13:52.292 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-11T23:13:52.292 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:13:52.357 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-11T23:13:52.358 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T23:13:52.385 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-11T23:13:52.385 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:13:52.391 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: mon.b calling monitor election 2024-09-11T23:13:52.391 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: mon.b calling monitor election 2024-09-11T23:13:52.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: mon.c calling monitor election 2024-09-11T23:13:52.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: mon.a calling monitor election 2024-09-11T23:13:52.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T23:13:52.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: monmap epoch 3 2024-09-11T23:13:52.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: fsid 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:13:52.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: last_changed 2024-09-11T23:13:44.616446+0000 2024-09-11T23:13:52.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: created 2024-09-11T23:12:00.389048+0000 2024-09-11T23:13:52.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: min_mon_release 19 (squid) 2024-09-11T23:13:52.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: election_strategy: 1 2024-09-11T23:13:52.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-11T23:13:52.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: 1: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.c 2024-09-11T23:13:52.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: 2: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.b 2024-09-11T23:13:52.392 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: fsmap 2024-09-11T23:13:52.393 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:13:52.393 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: mgrmap e13: x(active, since 51s) 2024-09-11T23:13:52.393 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: overall HEALTH_OK 2024-09-11T23:13:52.393 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.393 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.393 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.393 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.393 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.393 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.393 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.393 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.393 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.393 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.393 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:13:52.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:13:52.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:52 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:52.450 INFO:tasks.cephadm:Adding mgr.x on smithi043 2024-09-11T23:13:52.450 INFO:tasks.cephadm:Adding mgr.y on smithi090 2024-09-11T23:13:52.451 INFO:tasks.cephadm:Adding mgr.z on smithi097 2024-09-11T23:13:52.451 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi043=x;smithi090=y;smithi097=z' 2024-09-11T23:13:52.723 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:13:52.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: mon.b calling monitor election 2024-09-11T23:13:52.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: mon.b calling monitor election 2024-09-11T23:13:52.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: mon.c calling monitor election 2024-09-11T23:13:52.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: mon.a calling monitor election 2024-09-11T23:13:52.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T23:13:52.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: monmap epoch 3 2024-09-11T23:13:52.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: fsid 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:13:52.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: last_changed 2024-09-11T23:13:44.616446+0000 2024-09-11T23:13:52.772 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: created 2024-09-11T23:12:00.389048+0000 2024-09-11T23:13:52.772 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: min_mon_release 19 (squid) 2024-09-11T23:13:52.772 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: election_strategy: 1 2024-09-11T23:13:52.772 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-11T23:13:52.772 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: 1: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.c 2024-09-11T23:13:52.772 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: 2: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.b 2024-09-11T23:13:52.772 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: fsmap 2024-09-11T23:13:52.772 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:13:52.772 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: mgrmap e13: x(active, since 51s) 2024-09-11T23:13:52.772 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: overall HEALTH_OK 2024-09-11T23:13:52.772 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.772 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.772 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:13:52.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:13:52.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:52 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:52.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: mon.b calling monitor election 2024-09-11T23:13:52.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: mon.b calling monitor election 2024-09-11T23:13:52.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: mon.c calling monitor election 2024-09-11T23:13:52.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: mon.a calling monitor election 2024-09-11T23:13:52.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T23:13:52.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: monmap epoch 3 2024-09-11T23:13:52.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: fsid 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:13:52.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: last_changed 2024-09-11T23:13:44.616446+0000 2024-09-11T23:13:52.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: created 2024-09-11T23:12:00.389048+0000 2024-09-11T23:13:52.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: min_mon_release 19 (squid) 2024-09-11T23:13:52.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: election_strategy: 1 2024-09-11T23:13:52.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-11T23:13:52.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: 1: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.c 2024-09-11T23:13:52.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: 2: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.b 2024-09-11T23:13:52.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: fsmap 2024-09-11T23:13:52.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T23:13:52.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: mgrmap e13: x(active, since 51s) 2024-09-11T23:13:52.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: overall HEALTH_OK 2024-09-11T23:13:52.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.803 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:52.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:13:52.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:13:52.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:52 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:53.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.145+0000 7f6fcf6be640 1 -- 172.21.15.97:0/4109155936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fc81005f0 msgr2=0x7f6fc81009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:53.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.145+0000 7f6fcf6be640 1 --2- 172.21.15.97:0/4109155936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fc81005f0 0x7f6fc81009f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009880 tx=0x7f6fc402a5d0 comp rx=0 tx=0).stop 2024-09-11T23:13:53.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.145+0000 7f6fcf6be640 1 -- 172.21.15.97:0/4109155936 shutdown_connections 2024-09-11T23:13:53.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.145+0000 7f6fcf6be640 1 --2- 172.21.15.97:0/4109155936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fc81005f0 0x7f6fc81009f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:53.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.145+0000 7f6fcf6be640 1 -- 172.21.15.97:0/4109155936 >> 172.21.15.97:0/4109155936 conn(0x7f6fc80fbda0 msgr2=0x7f6fc80fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:53.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.145+0000 7f6fcf6be640 1 -- 172.21.15.97:0/4109155936 shutdown_connections 2024-09-11T23:13:53.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.145+0000 7f6fcf6be640 1 -- 172.21.15.97:0/4109155936 wait complete. 2024-09-11T23:13:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.146+0000 7f6fcf6be640 1 Processor -- start 2024-09-11T23:13:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.146+0000 7f6fcf6be640 1 -- start start 2024-09-11T23:13:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.146+0000 7f6fcf6be640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fc81005f0 0x7f6fc80fdc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.146+0000 7f6fcf6be640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fc80fc340 0x7f6fc80fc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.146+0000 7f6fcf6be640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc80fcd00 0x7f6fc81a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.146+0000 7f6fcf6be640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc8106700 con 0x7f6fc80fc340 2024-09-11T23:13:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.146+0000 7f6fcf6be640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc8106580 con 0x7f6fc81005f0 2024-09-11T23:13:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.146+0000 7f6fcf6be640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc8106880 con 0x7f6fc80fcd00 2024-09-11T23:13:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.146+0000 7f6fcdc5b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc80fcd00 0x7f6fc81a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.146+0000 7f6fcdc5b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc80fcd00 0x7f6fc81a1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33966/0 (socket says 172.21.15.97:33966) 2024-09-11T23:13:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.146+0000 7f6fcdc5b640 1 -- 172.21.15.97:0/4039906757 learned_addr learned my addr 172.21.15.97:0/4039906757 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-11T23:13:53.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.146+0000 7f6fcd45a640 1 --2- 172.21.15.97:0/4039906757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fc81005f0 0x7f6fc80fdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:53.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.146+0000 7f6fccc59640 1 --2- 172.21.15.97:0/4039906757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fc80fc340 0x7f6fc80fc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:53.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.146+0000 7f6fcdc5b640 1 -- 172.21.15.97:0/4039906757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fc81005f0 msgr2=0x7f6fc80fdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:53.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.146+0000 7f6fcdc5b640 1 --2- 172.21.15.97:0/4039906757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fc81005f0 0x7f6fc80fdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:53.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.146+0000 7f6fcdc5b640 1 -- 172.21.15.97:0/4039906757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fc80fc340 msgr2=0x7f6fc80fc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:53.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.147+0000 7f6fcdc5b640 1 --2- 172.21.15.97:0/4039906757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fc80fc340 0x7f6fc80fc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:53.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.147+0000 7f6fcdc5b640 1 -- 172.21.15.97:0/4039906757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc81a15c0 con 0x7f6fc80fcd00 2024-09-11T23:13:53.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.147+0000 7f6fcdc5b640 1 --2- 172.21.15.97:0/4039906757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc80fcd00 0x7f6fc81a1080 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000ba50 tx=0x7f6fc000bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:53.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.147+0000 7f6fbe7fc640 1 -- 172.21.15.97:0/4039906757 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0019070 con 0x7f6fc80fcd00 2024-09-11T23:13:53.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.147+0000 7f6fbe7fc640 1 -- 172.21.15.97:0/4039906757 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fc000cbe0 con 0x7f6fc80fcd00 2024-09-11T23:13:53.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.147+0000 7f6fcf6be640 1 -- 172.21.15.97:0/4039906757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc81a1850 con 0x7f6fc80fcd00 2024-09-11T23:13:53.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.147+0000 7f6fbe7fc640 1 -- 172.21.15.97:0/4039906757 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc00076a0 con 0x7f6fc80fcd00 2024-09-11T23:13:53.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.147+0000 7f6fcf6be640 1 -- 172.21.15.97:0/4039906757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc81a1de0 con 0x7f6fc80fcd00 2024-09-11T23:13:53.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.148+0000 7f6fbe7fc640 1 -- 172.21.15.97:0/4039906757 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6fc001c070 con 0x7f6fc80fcd00 2024-09-11T23:13:53.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.148+0000 7f6fcf6be640 1 -- 172.21.15.97:0/4039906757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fc8100a70 con 0x7f6fc80fcd00 2024-09-11T23:13:53.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.148+0000 7f6fbe7fc640 1 --2- 172.21.15.97:0/4039906757 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f6fb40394f0 0x7f6fb403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:53.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.148+0000 7f6fbe7fc640 1 -- 172.21.15.97:0/4039906757 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6fc004bc70 con 0x7f6fc80fcd00 2024-09-11T23:13:53.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.148+0000 7f6fcd45a640 1 --2- 172.21.15.97:0/4039906757 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f6fb40394f0 0x7f6fb403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:53.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.149+0000 7f6fcd45a640 1 --2- 172.21.15.97:0/4039906757 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f6fb40394f0 0x7f6fb403b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4005ab0 tx=0x7f6fc40315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:53.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.153+0000 7f6fbe7fc640 1 -- 172.21.15.97:0/4039906757 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6fc00112a0 con 0x7f6fc80fcd00 2024-09-11T23:13:53.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.331+0000 7f6fcf6be640 1 -- 172.21.15.97:0/4039906757 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi043=x;smithi090=y;smithi097=z", "target": ["mon-mgr", ""]}) -- 0x7f6fc80626c0 con 0x7f6fb40394f0 2024-09-11T23:13:53.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.337+0000 7f6fbe7fc640 1 -- 172.21.15.97:0/4039906757 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6fc80626c0 con 0x7f6fb40394f0 2024-09-11T23:13:53.337 INFO:teuthology.orchestra.run.smithi097.stdout:Scheduled mgr update... 2024-09-11T23:13:53.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.339+0000 7f6fcf6be640 1 -- 172.21.15.97:0/4039906757 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f6fb40394f0 msgr2=0x7f6fb403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:53.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.339+0000 7f6fcf6be640 1 --2- 172.21.15.97:0/4039906757 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f6fb40394f0 0x7f6fb403b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4005ab0 tx=0x7f6fc40315d0 comp rx=0 tx=0).stop 2024-09-11T23:13:53.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.339+0000 7f6fcf6be640 1 -- 172.21.15.97:0/4039906757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc80fcd00 msgr2=0x7f6fc81a1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:53.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.339+0000 7f6fcf6be640 1 --2- 172.21.15.97:0/4039906757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc80fcd00 0x7f6fc81a1080 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000ba50 tx=0x7f6fc000bf10 comp rx=0 tx=0).stop 2024-09-11T23:13:53.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.339+0000 7f6fcf6be640 1 -- 172.21.15.97:0/4039906757 shutdown_connections 2024-09-11T23:13:53.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.339+0000 7f6fcf6be640 1 --2- 172.21.15.97:0/4039906757 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f6fb40394f0 0x7f6fb403b9b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:53.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.339+0000 7f6fcf6be640 1 --2- 172.21.15.97:0/4039906757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc80fcd00 0x7f6fc81a1080 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:53.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.339+0000 7f6fcf6be640 1 --2- 172.21.15.97:0/4039906757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fc80fc340 0x7f6fc80fc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:53.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.339+0000 7f6fcf6be640 1 --2- 172.21.15.97:0/4039906757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fc81005f0 0x7f6fc80fdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:53.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.339+0000 7f6fcf6be640 1 -- 172.21.15.97:0/4039906757 >> 172.21.15.97:0/4039906757 conn(0x7f6fc80fbda0 msgr2=0x7f6fc806a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:53.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.339+0000 7f6fcf6be640 1 -- 172.21.15.97:0/4039906757 shutdown_connections 2024-09-11T23:13:53.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:13:53.339+0000 7f6fcf6be640 1 -- 172.21.15.97:0/4039906757 wait complete. 2024-09-11T23:13:53.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:53 smithi097 ceph-mon[35562]: Updating smithi097:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:13:53.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:53 smithi097 ceph-mon[35562]: Updating smithi090:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:13:53.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:53 smithi097 ceph-mon[35562]: Updating smithi043:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:13:53.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:53 smithi097 ceph-mon[35562]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T23:13:53.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:53 smithi097 ceph-mon[35562]: Reconfiguring daemon mon.a on smithi043 2024-09-11T23:13:53.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:53 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:53.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:53 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:53.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:53 smithi043 ceph-mon[30950]: Updating smithi097:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:13:53.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:53 smithi043 ceph-mon[30950]: Updating smithi090:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:13:53.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:53 smithi043 ceph-mon[30950]: Updating smithi043:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:13:53.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:53 smithi043 ceph-mon[30950]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T23:13:53.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:53 smithi043 ceph-mon[30950]: Reconfiguring daemon mon.a on smithi043 2024-09-11T23:13:53.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:53 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:53.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:53 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:53.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:53 smithi090 ceph-mon[35130]: Updating smithi097:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:13:53.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:53 smithi090 ceph-mon[35130]: Updating smithi090:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:13:53.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:53 smithi090 ceph-mon[35130]: Updating smithi043:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:13:53.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:53 smithi090 ceph-mon[35130]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T23:13:53.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:53 smithi090 ceph-mon[35130]: Reconfiguring daemon mon.a on smithi043 2024-09-11T23:13:53.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:53 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:13:53.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:53 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:53.867 DEBUG:teuthology.orchestra.run.smithi090:mgr.y> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mgr.y.service 2024-09-11T23:13:53.870 DEBUG:teuthology.orchestra.run.smithi097:mgr.z> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mgr.z.service 2024-09-11T23:13:53.872 INFO:tasks.cephadm:Deploying OSDs... 2024-09-11T23:13:53.873 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T23:13:53.873 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T23:13:53.889 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T23:13:53.890 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 2024-09-11T23:13:53.949 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T23:13:53.949 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:13:53.949 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-11T23:13:53.949 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:13:53.949 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:13:53.949 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-11 23:13:11.899278416 +0000 2024-09-11T23:13:53.949 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-11 23:08:56.289342953 +0000 2024-09-11T23:13:53.949 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-11 23:08:56.289342953 +0000 2024-09-11T23:13:53.949 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-11 23:08:56.289342953 +0000 2024-09-11T23:13:53.949 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T23:13:54.016 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-11T23:13:54.017 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-11T23:13:54.017 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000104394 s, 4.9 MB/s 2024-09-11T23:13:54.018 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T23:13:54.075 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 2024-09-11T23:13:54.132 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T23:13:54.132 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:13:54.132 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-11T23:13:54.132 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:13:54.132 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:13:54.132 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-11 23:13:11.927278121 +0000 2024-09-11T23:13:54.133 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-11 23:08:56.607339534 +0000 2024-09-11T23:13:54.133 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-11 23:08:56.607339534 +0000 2024-09-11T23:13:54.133 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-11 23:08:56.607339534 +0000 2024-09-11T23:13:54.133 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T23:13:54.195 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-11T23:13:54.195 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-11T23:13:54.196 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000107301 s, 4.8 MB/s 2024-09-11T23:13:54.197 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T23:13:54.253 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 2024-09-11T23:13:54.309 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T23:13:54.310 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:13:54.310 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-11T23:13:54.310 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:13:54.310 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:13:54.310 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-11 23:13:11.955277826 +0000 2024-09-11T23:13:54.310 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-11 23:08:56.922336146 +0000 2024-09-11T23:13:54.310 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-11 23:08:56.922336146 +0000 2024-09-11T23:13:54.310 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-11 23:08:56.922336146 +0000 2024-09-11T23:13:54.310 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T23:13:54.374 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-11T23:13:54.375 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-11T23:13:54.375 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000106948 s, 4.8 MB/s 2024-09-11T23:13:54.376 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T23:13:54.432 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 2024-09-11T23:13:54.489 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T23:13:54.489 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:13:54.489 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-11T23:13:54.489 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:13:54.489 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:13:54.490 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-11 23:13:11.984277521 +0000 2024-09-11T23:13:54.490 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-11 23:08:57.248332641 +0000 2024-09-11T23:13:54.490 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-11 23:08:57.248332641 +0000 2024-09-11T23:13:54.490 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-11 23:08:57.248332641 +0000 2024-09-11T23:13:54.490 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T23:13:54.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:54 smithi090 ceph-mon[35130]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:54.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:54 smithi090 ceph-mon[35130]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi043=x;smithi090=y;smithi097=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:13:54.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:54 smithi090 ceph-mon[35130]: Saving service mgr spec with placement smithi043=x;smithi090=y;smithi097=z;count:3 2024-09-11T23:13:54.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:54 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:54.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:54 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:54.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:54 smithi090 ceph-mon[35130]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T23:13:54.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:54 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:13:54.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:54 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:13:54.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:54 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:54.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:54 smithi090 ceph-mon[35130]: Reconfiguring daemon mgr.x on smithi043 2024-09-11T23:13:54.554 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-11T23:13:54.554 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-11T23:13:54.554 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000109482 s, 4.7 MB/s 2024-09-11T23:13:54.555 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T23:13:54.613 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-11T23:13:54.613 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T23:13:54.629 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T23:13:54.629 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_1 2024-09-11T23:13:54.684 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T23:13:54.684 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:13:54.684 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-11T23:13:54.684 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:13:54.684 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:13:54.684 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-09-11 23:13:44.086612233 +0000 2024-09-11T23:13:54.684 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-09-11 23:09:08.628437511 +0000 2024-09-11T23:13:54.684 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-09-11 23:09:08.628437511 +0000 2024-09-11T23:13:54.685 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-09-11 23:09:08.628437511 +0000 2024-09-11T23:13:54.685 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T23:13:54.726 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:54 smithi043 ceph-mon[30950]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:54.726 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:54 smithi043 ceph-mon[30950]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi043=x;smithi090=y;smithi097=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:13:54.727 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:54 smithi043 ceph-mon[30950]: Saving service mgr spec with placement smithi043=x;smithi090=y;smithi097=z;count:3 2024-09-11T23:13:54.727 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:54 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:54.727 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:54 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:54.727 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:54 smithi043 ceph-mon[30950]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T23:13:54.727 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:54 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:13:54.727 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:54 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:13:54.727 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:54 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:54.727 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:54 smithi043 ceph-mon[30950]: Reconfiguring daemon mgr.x on smithi043 2024-09-11T23:13:54.749 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-09-11T23:13:54.750 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-09-11T23:13:54.750 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000269848 s, 1.9 MB/s 2024-09-11T23:13:54.750 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T23:13:54.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:54 smithi097 ceph-mon[35562]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:54.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:54 smithi097 ceph-mon[35562]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi043=x;smithi090=y;smithi097=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:13:54.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:54 smithi097 ceph-mon[35562]: Saving service mgr spec with placement smithi043=x;smithi090=y;smithi097=z;count:3 2024-09-11T23:13:54.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:54 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:54.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:54 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:54.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:54 smithi097 ceph-mon[35562]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T23:13:54.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:54 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:13:54.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:54 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:13:54.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:54 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:54.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:54 smithi097 ceph-mon[35562]: Reconfiguring daemon mgr.x on smithi043 2024-09-11T23:13:54.806 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_2 2024-09-11T23:13:54.860 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T23:13:54.861 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:13:54.861 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-09-11T23:13:54.861 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:13:54.861 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:13:54.861 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-09-11 23:13:44.087612218 +0000 2024-09-11T23:13:54.861 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-09-11 23:09:08.952432525 +0000 2024-09-11T23:13:54.861 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-09-11 23:09:08.952432525 +0000 2024-09-11T23:13:54.861 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-09-11 23:09:08.952432525 +0000 2024-09-11T23:13:54.861 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T23:13:54.924 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-09-11T23:13:54.925 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-09-11T23:13:54.925 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000202909 s, 2.5 MB/s 2024-09-11T23:13:54.926 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T23:13:54.983 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_3 2024-09-11T23:13:55.038 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T23:13:55.038 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:13:55.038 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-11T23:13:55.039 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:13:55.039 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:13:55.039 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-09-11 23:13:44.087612218 +0000 2024-09-11T23:13:55.039 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-09-11 23:09:09.279427494 +0000 2024-09-11T23:13:55.039 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-09-11 23:09:09.279427494 +0000 2024-09-11T23:13:55.039 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-09-11 23:09:09.279427494 +0000 2024-09-11T23:13:55.039 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T23:13:55.110 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-09-11T23:13:55.110 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-09-11T23:13:55.110 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000192235 s, 2.7 MB/s 2024-09-11T23:13:55.112 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T23:13:55.180 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_4 2024-09-11T23:13:55.239 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T23:13:55.239 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:13:55.239 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-11T23:13:55.239 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:13:55.239 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:13:55.239 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-09-11 23:13:44.087612218 +0000 2024-09-11T23:13:55.239 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-09-11 23:09:09.620422247 +0000 2024-09-11T23:13:55.239 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-09-11 23:09:09.620422247 +0000 2024-09-11T23:13:55.239 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-09-11 23:09:09.620422247 +0000 2024-09-11T23:13:55.240 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T23:13:55.307 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-09-11T23:13:55.307 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-09-11T23:13:55.308 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000192613 s, 2.7 MB/s 2024-09-11T23:13:55.309 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T23:13:55.367 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-11T23:13:55.367 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T23:13:55.388 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T23:13:55.389 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2024-09-11T23:13:55.449 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T23:13:55.450 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:13:55.450 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-11T23:13:55.450 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:13:55.450 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:13:55.450 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-11 23:13:34.323600387 +0000 2024-09-11T23:13:55.450 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-11 23:09:08.833945934 +0000 2024-09-11T23:13:55.450 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-11 23:09:08.833945934 +0000 2024-09-11T23:13:55.450 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-09-11 23:09:08.833945934 +0000 2024-09-11T23:13:55.450 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T23:13:55.518 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-11T23:13:55.518 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-11T23:13:55.518 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000258233 s, 2.0 MB/s 2024-09-11T23:13:55.519 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T23:13:55.578 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2024-09-11T23:13:55.636 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T23:13:55.636 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:13:55.636 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-11T23:13:55.636 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:13:55.637 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:13:55.637 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-11 23:13:34.323600387 +0000 2024-09-11T23:13:55.637 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-11 23:09:09.160940755 +0000 2024-09-11T23:13:55.637 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-11 23:09:09.160940755 +0000 2024-09-11T23:13:55.637 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-09-11 23:09:09.160940755 +0000 2024-09-11T23:13:55.637 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T23:13:55.702 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-11T23:13:55.703 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-11T23:13:55.703 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000230851 s, 2.2 MB/s 2024-09-11T23:13:55.704 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T23:13:55.762 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2024-09-11T23:13:55.819 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T23:13:55.819 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:13:55.820 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-11T23:13:55.820 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:13:55.820 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:13:55.820 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-11 23:13:34.324600371 +0000 2024-09-11T23:13:55.820 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-11 23:09:09.489935545 +0000 2024-09-11T23:13:55.820 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-11 23:09:09.489935545 +0000 2024-09-11T23:13:55.820 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-09-11 23:09:09.489935545 +0000 2024-09-11T23:13:55.820 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T23:13:55.885 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-11T23:13:55.886 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-11T23:13:55.886 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000207152 s, 2.5 MB/s 2024-09-11T23:13:55.887 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T23:13:55.944 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2024-09-11T23:13:56.000 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T23:13:56.000 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T23:13:56.000 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-11T23:13:56.000 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:13:56.000 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T23:13:56.001 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-11 23:13:34.324600371 +0000 2024-09-11T23:13:56.001 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-11 23:09:09.808930493 +0000 2024-09-11T23:13:56.001 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-11 23:09:09.808930493 +0000 2024-09-11T23:13:56.001 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-09-11 23:09:09.808930493 +0000 2024-09-11T23:13:56.001 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T23:13:56.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:55 smithi097 ceph-mon[35562]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:56.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:55 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:56.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:55 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:56.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:55 smithi097 ceph-mon[35562]: Reconfiguring mon.b (monmap changed)... 2024-09-11T23:13:56.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:55 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:13:56.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:55 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:13:56.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:55 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:56.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:55 smithi097 ceph-mon[35562]: Reconfiguring daemon mon.b on smithi090 2024-09-11T23:13:56.064 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-11T23:13:56.064 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-11T23:13:56.064 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000208525 s, 2.5 MB/s 2024-09-11T23:13:56.065 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T23:13:56.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:55 smithi043 ceph-mon[30950]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:56.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:55 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:56.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:55 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:56.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:55 smithi043 ceph-mon[30950]: Reconfiguring mon.b (monmap changed)... 2024-09-11T23:13:56.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:55 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:13:56.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:55 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:13:56.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:55 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:56.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:55 smithi043 ceph-mon[30950]: Reconfiguring daemon mon.b on smithi090 2024-09-11T23:13:56.123 INFO:tasks.cephadm:Deploying osd.0 on smithi043 with /dev/vg_nvme/lv_4... 2024-09-11T23:13:56.123 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T23:13:56.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:55 smithi090 ceph-mon[35130]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:56.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:55 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:56.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:55 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:56.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:55 smithi090 ceph-mon[35130]: Reconfiguring mon.b (monmap changed)... 2024-09-11T23:13:56.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:55 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:13:56.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:55 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:13:56.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:55 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:56.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:55 smithi090 ceph-mon[35130]: Reconfiguring daemon mon.b on smithi090 2024-09-11T23:13:56.385 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:13:57.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:57 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:57.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:57 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:57.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:57 smithi043 ceph-mon[30950]: Reconfiguring mon.c (monmap changed)... 2024-09-11T23:13:57.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:57 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:13:57.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:57 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:13:57.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:57 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:57.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:57 smithi043 ceph-mon[30950]: Reconfiguring daemon mon.c on smithi097 2024-09-11T23:13:57.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:57 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:57.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:57 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:57.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:57 smithi097 ceph-mon[35562]: Reconfiguring mon.c (monmap changed)... 2024-09-11T23:13:57.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:57 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:13:57.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:57 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:13:57.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:57 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:57.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:57 smithi097 ceph-mon[35562]: Reconfiguring daemon mon.c on smithi097 2024-09-11T23:13:57.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:57 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:57.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:57 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:57.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:57 smithi090 ceph-mon[35130]: Reconfiguring mon.c (monmap changed)... 2024-09-11T23:13:57.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:57 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T23:13:57.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:57 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T23:13:57.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:57 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:57.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:57 smithi090 ceph-mon[35130]: Reconfiguring daemon mon.c on smithi097 2024-09-11T23:13:58.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:58 smithi043 ceph-mon[30950]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:58.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:58 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:58.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:58 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:58.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:58 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:58.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:58 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:58.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:58 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:58.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:58 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:58.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:58 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:13:58.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:58 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:13:58.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:58 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:13:58.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:58 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:58.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:13:58 smithi043 ceph-mon[30950]: Deploying daemon mgr.z on smithi097 2024-09-11T23:13:58.642 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:58 smithi097 ceph-mon[35562]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:58.642 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:58 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:58.642 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:58 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:58.642 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:58 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:58.642 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:58 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:58.642 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:58 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:58.642 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:58 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:58.642 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:58 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:13:58.642 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:58 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:13:58.643 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:58 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:13:58.643 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:58 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:58.643 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:13:58 smithi097 ceph-mon[35562]: Deploying daemon mgr.z on smithi097 2024-09-11T23:13:58.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:58 smithi090 ceph-mon[35130]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:13:58.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:58 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:58.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:58 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:58.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:58 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:13:58.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:58 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:58.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:58 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:13:58.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:58 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:13:58.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:58 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:13:58.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:58 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:13:58.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:58 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:13:58.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:58 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:13:58.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:13:58 smithi090 ceph-mon[35130]: Deploying daemon mgr.z on smithi097 2024-09-11T23:13:58.836 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:13:58.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_4 2024-09-11T23:13:59.120 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:13:59.551 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:13:59 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:13:59.514+0000 7f9822ba6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:13:59.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.566+0000 7f60b319e640 1 -- 172.21.15.43:0/2908936640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ac100950 msgr2=0x7f60ac100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:59.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.566+0000 7f60b319e640 1 --2- 172.21.15.43:0/2908936640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ac100950 0x7f60ac100d50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009900 tx=0x7f60a802a410 comp rx=0 tx=0).stop 2024-09-11T23:13:59.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.566+0000 7f60b319e640 1 -- 172.21.15.43:0/2908936640 shutdown_connections 2024-09-11T23:13:59.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.566+0000 7f60b319e640 1 --2- 172.21.15.43:0/2908936640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60ac102510 0x7f60ac108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:59.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.566+0000 7f60b319e640 1 --2- 172.21.15.43:0/2908936640 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60ac101b50 0x7f60ac101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:59.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.566+0000 7f60b319e640 1 --2- 172.21.15.43:0/2908936640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ac100950 0x7f60ac100d50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:59.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.566+0000 7f60b319e640 1 -- 172.21.15.43:0/2908936640 >> 172.21.15.43:0/2908936640 conn(0x7f60ac0fc0e0 msgr2=0x7f60ac0fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:13:59.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.566+0000 7f60b319e640 1 -- 172.21.15.43:0/2908936640 shutdown_connections 2024-09-11T23:13:59.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.566+0000 7f60b319e640 1 -- 172.21.15.43:0/2908936640 wait complete. 2024-09-11T23:13:59.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.567+0000 7f60b319e640 1 Processor -- start 2024-09-11T23:13:59.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.567+0000 7f60b319e640 1 -- start start 2024-09-11T23:13:59.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.567+0000 7f60b319e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ac100950 0x7f60ac197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:59.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.567+0000 7f60b319e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60ac101b50 0x7f60ac198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:59.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.567+0000 7f60b319e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60ac102510 0x7f60ac19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:59.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.567+0000 7f60b319e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac10c5e0 con 0x7f60ac100950 2024-09-11T23:13:59.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.567+0000 7f60b319e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac10c460 con 0x7f60ac102510 2024-09-11T23:13:59.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.567+0000 7f60b319e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac10c760 con 0x7f60ac101b50 2024-09-11T23:13:59.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.567+0000 7f60b219c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ac100950 0x7f60ac197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:59.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.567+0000 7f60b219c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ac100950 0x7f60ac197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43208/0 (socket says 172.21.15.43:43208) 2024-09-11T23:13:59.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.567+0000 7f60b219c640 1 -- 172.21.15.43:0/1194181639 learned_addr learned my addr 172.21.15.43:0/1194181639 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:13:59.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.567+0000 7f60b199b640 1 --2- 172.21.15.43:0/1194181639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60ac101b50 0x7f60ac198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:59.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.567+0000 7f60b299d640 1 --2- 172.21.15.43:0/1194181639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60ac102510 0x7f60ac19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:59.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.568+0000 7f60b219c640 1 -- 172.21.15.43:0/1194181639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60ac101b50 msgr2=0x7f60ac198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:59.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.568+0000 7f60b219c640 1 --2- 172.21.15.43:0/1194181639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60ac101b50 0x7f60ac198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:59.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.568+0000 7f60b219c640 1 -- 172.21.15.43:0/1194181639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60ac102510 msgr2=0x7f60ac19f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:13:59.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.568+0000 7f60b219c640 1 --2- 172.21.15.43:0/1194181639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60ac102510 0x7f60ac19f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:13:59.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.568+0000 7f60b219c640 1 -- 172.21.15.43:0/1194181639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ac19f950 con 0x7f60ac100950 2024-09-11T23:13:59.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.568+0000 7f60b199b640 1 --2- 172.21.15.43:0/1194181639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60ac101b50 0x7f60ac198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:13:59.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.568+0000 7f60b219c640 1 --2- 172.21.15.43:0/1194181639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ac100950 0x7f60ac197d30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f60a8031610 tx=0x7f60a8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:59.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.568+0000 7f60a37fe640 1 -- 172.21.15.43:0/1194181639 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a803f030 con 0x7f60ac100950 2024-09-11T23:13:59.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.568+0000 7f60a37fe640 1 -- 172.21.15.43:0/1194181639 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60a8003ad0 con 0x7f60ac100950 2024-09-11T23:13:59.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.568+0000 7f60a37fe640 1 -- 172.21.15.43:0/1194181639 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8035070 con 0x7f60ac100950 2024-09-11T23:13:59.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.568+0000 7f60b319e640 1 -- 172.21.15.43:0/1194181639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ac19fbe0 con 0x7f60ac100950 2024-09-11T23:13:59.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.568+0000 7f60b319e640 1 -- 172.21.15.43:0/1194181639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ac1a0070 con 0x7f60ac100950 2024-09-11T23:13:59.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.569+0000 7f60a37fe640 1 -- 172.21.15.43:0/1194181639 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f60a8042030 con 0x7f60ac100950 2024-09-11T23:13:59.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.569+0000 7f60a37fe640 1 --2- 172.21.15.43:0/1194181639 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f6098039290 0x7f609803b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:13:59.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.569+0000 7f60b319e640 1 -- 172.21.15.43:0/1194181639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60ac1a02a0 con 0x7f60ac100950 2024-09-11T23:13:59.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.569+0000 7f60a37fe640 1 -- 172.21.15.43:0/1194181639 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f60a806c930 con 0x7f60ac100950 2024-09-11T23:13:59.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.569+0000 7f60b199b640 1 --2- 172.21.15.43:0/1194181639 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f6098039290 0x7f609803b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:13:59.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.570+0000 7f60b199b640 1 --2- 172.21.15.43:0/1194181639 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f6098039290 0x7f609803b750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f609c005d00 tx=0x7f609c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:13:59.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.574+0000 7f60a37fe640 1 -- 172.21.15.43:0/1194181639 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f60a8043d80 con 0x7f60ac100950 2024-09-11T23:13:59.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:13:59.760+0000 7f60b319e640 1 -- 172.21.15.43:0/1194181639 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f60ac1a0470 con 0x7f6098039290 2024-09-11T23:13:59.929 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:13:59 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:13:59.574+0000 7f9822ba6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:13:59.929 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:13:59 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:13:59.720+0000 7f9822ba6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:13:59.929 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:13:59 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:13:59.785+0000 7f9822ba6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:13:59.929 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:13:59 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:13:59.929+0000 7f9822ba6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:14:00.301 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:14:00 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:14:00.066+0000 7f9822ba6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:14:00.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:00 smithi043 ceph-mon[30950]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:00.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:00 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:00.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:00 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:00.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:00 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:00.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:00 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:14:00.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:00 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:14:00.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:00 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:14:00.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:00 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:00.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:00 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:14:00.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:00 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:14:00.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:00 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:00.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:00 smithi090 ceph-mon[35130]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:00.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:00 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:00.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:00 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:00.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:00 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:00.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:00 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:14:00.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:00 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:14:00.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:00 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:14:00.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:00 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:00.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:00 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:14:00.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:00 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:14:00.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:00 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:00.621 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:14:00 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:14:00.378+0000 7f9822ba6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:14:00.621 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:14:00 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:14:00.439+0000 7f9822ba6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:14:00.622 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:00 smithi097 ceph-mon[35562]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:00.622 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:00 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:00.622 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:00 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:00.622 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:00 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:00.622 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:00 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T23:14:00.622 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:00 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T23:14:00.622 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:00 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T23:14:00.622 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:00 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:00.622 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:00 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:14:00.622 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:00 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:14:00.622 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:00 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:01.051 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:14:00 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:14:00.621+0000 7f9822ba6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:14:01.513 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:01 smithi090 ceph-mon[35130]: Deploying daemon mgr.y on smithi090 2024-09-11T23:14:01.513 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:01 smithi090 ceph-mon[35130]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:14:01.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:01 smithi043 ceph-mon[30950]: Deploying daemon mgr.y on smithi090 2024-09-11T23:14:01.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:01 smithi043 ceph-mon[30950]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:14:01.770 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:01 smithi090 ceph-mgr[36498]: -- 172.21.15.90:0/631018569 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e2d4e750e0 con 0x55e2d4e53400 2024-09-11T23:14:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:01 smithi097 ceph-mon[35562]: Deploying daemon mgr.y on smithi090 2024-09-11T23:14:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:01 smithi097 ceph-mon[35562]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:14:01.801 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:14:01 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:14:01.498+0000 7f9822ba6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:14:01.801 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:14:01 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:14:01.564+0000 7f9822ba6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:14:02.120 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:01 smithi090 bash[36478]: 98d6ff6839de7980bb97cbbfd861cf53d533c04f9547fe84a7923c0529f855f7 2024-09-11T23:14:02.120 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:01 smithi090 systemd[1]: Started Ceph mgr.y for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:14:02.121 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:01 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:01.835+0000 7f479c665040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:14:02.121 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:01 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:01.895+0000 7f479c665040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:14:02.121 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:02 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:02.051+0000 7f479c665040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:14:02.121 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:02 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:02.120+0000 7f479c665040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:14:02.432 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:02 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:02.278+0000 7f479c665040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:14:02.432 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:02 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:02.432+0000 7f479c665040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:14:02.433 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:02 smithi090 ceph-mon[35130]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:02.433 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:02 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:02.433 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:02 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:02.433 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:02 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:02.433 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:02 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:02.433 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:02 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:02.433 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:02 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/356059360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24076596-3909-4c87-85f1-cbf168070dbb"}]: dispatch 2024-09-11T23:14:02.433 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:02 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/356059360' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24076596-3909-4c87-85f1-cbf168070dbb"}]': finished 2024-09-11T23:14:02.433 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:02 smithi090 ceph-mon[35130]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T23:14:02.433 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:02 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:14:02.461 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:02 smithi043 ceph-mon[30950]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:02.461 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:02.461 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:02.461 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:02.461 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:02.461 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:02.461 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:02 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/356059360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24076596-3909-4c87-85f1-cbf168070dbb"}]: dispatch 2024-09-11T23:14:02.461 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:02 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/356059360' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24076596-3909-4c87-85f1-cbf168070dbb"}]': finished 2024-09-11T23:14:02.461 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:02 smithi043 ceph-mon[30950]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T23:14:02.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:14:02.574 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:02 smithi097 ceph-mon[35562]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:02.574 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:02 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:02.574 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:02 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:02.574 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:02 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:02.574 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:02 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:02.575 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:02 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:02.575 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:02 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/356059360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24076596-3909-4c87-85f1-cbf168070dbb"}]: dispatch 2024-09-11T23:14:02.575 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:02 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/356059360' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24076596-3909-4c87-85f1-cbf168070dbb"}]': finished 2024-09-11T23:14:02.575 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:02 smithi097 ceph-mon[35562]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T23:14:02.575 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:02 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:14:02.575 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:14:02 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:14:02.301+0000 7f9822ba6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:14:02.575 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:14:02 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:14:02.431+0000 7f9822ba6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:14:02.575 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:14:02 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:14:02.573+0000 7f9822ba6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:14:02.873 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:14:02 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:14:02.639+0000 7f9822ba6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:14:02.874 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:14:02 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:14:02.805+0000 7f9822ba6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:14:02.874 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:14:02 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:14:02.873+0000 7f9822ba6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:14:03.013 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:02 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:02.758+0000 7f479c665040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:14:03.013 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:02 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:02.820+0000 7f479c665040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:14:03.169 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:14:02 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:14:02.933+0000 7f9822ba6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:14:03.169 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:14:03 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:14:03.102+0000 7f9822ba6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:14:03.169 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:14:03 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:14:03.168+0000 7f9822ba6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:14:03.269 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:03 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:03.012+0000 7f479c665040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:14:03.439 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:14:03 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:14:03.235+0000 7f9822ba6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:14:03.439 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:14:03 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:14:03.317+0000 7f9822ba6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:14:03.439 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:14:03 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:14:03.379+0000 7f9822ba6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:14:03.439 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:14:03 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:14:03.438+0000 7f9822ba6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:14:03.440 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:03 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/661741732' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:14:03.440 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:03 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:03.440 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:03 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:03.440 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:03 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:03.440 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:03 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:03.440 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:03 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:03.440 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:03 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:03.440 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:03 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:03.440 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:03 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:03.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:03 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/661741732' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:14:03.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:03 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:03.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:03 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:03.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:03 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:03.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:03 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:03.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:03 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:03.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:03 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:03.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:03 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:03.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:03 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:03.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:03 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/661741732' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:14:03.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:03.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:03.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:03.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:03.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:03.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:03.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:03.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:04.270 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:03 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:03.905+0000 7f479c665040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:14:04.270 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:03 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:03.972+0000 7f479c665040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:14:04.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:14:04.539+0000 7f60a37fe640 1 -- 172.21.15.43:0/1194181639 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f60a8003c80 con 0x7f60ac100950 2024-09-11T23:14:04.551 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:14:04 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:14:04.232+0000 7f9822ba6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:14:04.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:04 smithi097 ceph-mon[35562]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:04.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:04 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:04.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:04 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:04.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:04 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:04.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:04 smithi097 ceph-mon[35562]: Standby manager daemon z started 2024-09-11T23:14:04.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:04 smithi097 ceph-mon[35562]: from='mgr.? 172.21.15.97:0/2593724109' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:14:04.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:04 smithi097 ceph-mon[35562]: from='mgr.? 172.21.15.97:0/2593724109' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:14:04.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:04 smithi097 ceph-mon[35562]: from='mgr.? 172.21.15.97:0/2593724109' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:14:04.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:04 smithi097 ceph-mon[35562]: from='mgr.? 172.21.15.97:0/2593724109' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:14:04.688 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:04 smithi090 ceph-mon[35130]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:04.688 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:04 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:04.689 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:04 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:04.689 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:04 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:04.689 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:04 smithi090 ceph-mon[35130]: Standby manager daemon z started 2024-09-11T23:14:04.689 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:04 smithi090 ceph-mon[35130]: from='mgr.? 172.21.15.97:0/2593724109' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:14:04.689 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:04 smithi090 ceph-mon[35130]: from='mgr.? 172.21.15.97:0/2593724109' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:14:04.689 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:04 smithi090 ceph-mon[35130]: from='mgr.? 172.21.15.97:0/2593724109' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:14:04.689 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:04 smithi090 ceph-mon[35130]: from='mgr.? 172.21.15.97:0/2593724109' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:14:04.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:04 smithi043 ceph-mon[30950]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:04.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:04 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:04.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:04 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:04.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:04 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:04.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:04 smithi043 ceph-mon[30950]: Standby manager daemon z started 2024-09-11T23:14:04.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:04 smithi043 ceph-mon[30950]: from='mgr.? 172.21.15.97:0/2593724109' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:14:04.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:04 smithi043 ceph-mon[30950]: from='mgr.? 172.21.15.97:0/2593724109' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:14:04.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:04 smithi043 ceph-mon[30950]: from='mgr.? 172.21.15.97:0/2593724109' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:14:04.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:04 smithi043 ceph-mon[30950]: from='mgr.? 172.21.15.97:0/2593724109' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:14:04.943 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:04 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:04.688+0000 7f479c665040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:14:04.943 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:04 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:04.802+0000 7f479c665040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:14:04.943 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:04 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:04.943+0000 7f479c665040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:14:05.215 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:05 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:05.003+0000 7f479c665040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:14:05.215 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:05 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:05.149+0000 7f479c665040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:14:05.216 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:05 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:05.215+0000 7f479c665040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:14:05.501 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:05 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:05.275+0000 7f479c665040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:14:05.501 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:05 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:05.435+0000 7f479c665040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:14:05.501 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:05 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:05.500+0000 7f479c665040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:14:05.768 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:05 smithi090 ceph-mon[35130]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-11T23:14:05.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:05 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:14:05.769 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:05 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:05.568+0000 7f479c665040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:14:05.769 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:05 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:05.651+0000 7f479c665040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:14:05.769 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:05 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:05.709+0000 7f479c665040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:14:05.769 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:05 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:05.768+0000 7f479c665040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:14:05.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:05 smithi097 ceph-mon[35562]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-11T23:14:05.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:05 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:14:05.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:05 smithi043 ceph-mon[30950]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-11T23:14:05.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:05 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:14:06.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:06 smithi097 ceph-mon[35562]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:06.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:06 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:06.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:06 smithi043 ceph-mon[30950]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:06.833 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:06 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:07.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:06 smithi090 ceph-mon[35130]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:07.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:06 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:07.020 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:14:06 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:14:06.554+0000 7f479c665040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:14:07.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:14:07.553+0000 7f60a37fe640 1 -- 172.21.15.43:0/1194181639 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f60a8049c40 con 0x7f60ac100950 2024-09-11T23:14:07.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:07 smithi097 ceph-mon[35562]: Standby manager daemon y started 2024-09-11T23:14:07.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:07 smithi097 ceph-mon[35562]: from='mgr.? 172.21.15.90:0/3642939314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:14:07.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:07 smithi097 ceph-mon[35562]: from='mgr.? 172.21.15.90:0/3642939314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:14:07.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:07 smithi097 ceph-mon[35562]: from='mgr.? 172.21.15.90:0/3642939314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:14:07.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:07 smithi097 ceph-mon[35562]: from='mgr.? 172.21.15.90:0/3642939314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:14:07.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:07 smithi097 ceph-mon[35562]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:07 smithi043 ceph-mon[30950]: Standby manager daemon y started 2024-09-11T23:14:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:07 smithi043 ceph-mon[30950]: from='mgr.? 172.21.15.90:0/3642939314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:14:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:07 smithi043 ceph-mon[30950]: from='mgr.? 172.21.15.90:0/3642939314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:14:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:07 smithi043 ceph-mon[30950]: from='mgr.? 172.21.15.90:0/3642939314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:14:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:07 smithi043 ceph-mon[30950]: from='mgr.? 172.21.15.90:0/3642939314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:14:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:07 smithi043 ceph-mon[30950]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:08.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:07 smithi090 ceph-mon[35130]: Standby manager daemon y started 2024-09-11T23:14:08.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:07 smithi090 ceph-mon[35130]: from='mgr.? 172.21.15.90:0/3642939314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:14:08.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:07 smithi090 ceph-mon[35130]: from='mgr.? 172.21.15.90:0/3642939314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:14:08.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:07 smithi090 ceph-mon[35130]: from='mgr.? 172.21.15.90:0/3642939314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:14:08.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:07 smithi090 ceph-mon[35130]: from='mgr.? 172.21.15.90:0/3642939314' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:14:08.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:07 smithi090 ceph-mon[35130]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:08.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:08 smithi097 ceph-mon[35562]: mgrmap e15: x(active, since 67s), standbys: z, y 2024-09-11T23:14:08.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:08 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:14:08.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:08 smithi043 ceph-mon[30950]: mgrmap e15: x(active, since 67s), standbys: z, y 2024-09-11T23:14:08.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:08 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:14:09.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:08 smithi090 ceph-mon[35130]: mgrmap e15: x(active, since 67s), standbys: z, y 2024-09-11T23:14:09.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:08 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:14:09.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:09 smithi043 ceph-mon[30950]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:10.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:09 smithi090 ceph-mon[35130]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:10.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:09 smithi097 ceph-mon[35562]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:10.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:10 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T23:14:10.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:10 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:10.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:10 smithi043 ceph-mon[30950]: Deploying daemon osd.0 on smithi043 2024-09-11T23:14:11.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:10 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T23:14:11.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:10 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:11.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:10 smithi090 ceph-mon[35130]: Deploying daemon osd.0 on smithi043 2024-09-11T23:14:11.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:10 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T23:14:11.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:10 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:11.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:10 smithi097 ceph-mon[35562]: Deploying daemon osd.0 on smithi043 2024-09-11T23:14:11.825 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:11 smithi043 ceph-mon[30950]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:11.921 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:11 smithi090 ceph-mon[35130]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:11.953 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:11 smithi097 ceph-mon[35562]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:13.720 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:13 smithi043 ceph-mon[30950]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:14.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:13 smithi090 ceph-mon[35130]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:14.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:13 smithi097 ceph-mon[35562]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:15.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:15 smithi043 ceph-mon[30950]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:16.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:15 smithi090 ceph-mon[35130]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:16.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:15 smithi097 ceph-mon[35562]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:16.997 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:16 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:16.997 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:16 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:16.997 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:16 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:17.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:16 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:17.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:16 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:17.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:16 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:17.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:16 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:17.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:16 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:17.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:16 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:18.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:17 smithi090 ceph-mon[35130]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:18.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:17 smithi043 ceph-mon[30950]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:18.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:17 smithi097 ceph-mon[35562]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:20 smithi090 ceph-mon[35130]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:20 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:20 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:20 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:20 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:20 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:20.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:20 smithi043 ceph-mon[30950]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:20.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:20 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:20.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:20 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:20.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:20 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:20.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:20 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:20.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:20 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:20.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:20 smithi097 ceph-mon[35562]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:20.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:20 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:20.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:20 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:20.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:20 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:20.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:20 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:20.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:20 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:20.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:14:20.907+0000 7f60a37fe640 1 -- 172.21.15.43:0/1194181639 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f60ac1a0470 con 0x7f6098039290 2024-09-11T23:14:20.908 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 0 on host 'smithi043' 2024-09-11T23:14:20.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:14:20.910+0000 7f60b319e640 1 -- 172.21.15.43:0/1194181639 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f6098039290 msgr2=0x7f609803b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:14:20.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:14:20.910+0000 7f60b319e640 1 --2- 172.21.15.43:0/1194181639 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f6098039290 0x7f609803b750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f609c005d00 tx=0x7f609c00b410 comp rx=0 tx=0).stop 2024-09-11T23:14:20.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:14:20.910+0000 7f60b319e640 1 -- 172.21.15.43:0/1194181639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ac100950 msgr2=0x7f60ac197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:14:20.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:14:20.910+0000 7f60b319e640 1 --2- 172.21.15.43:0/1194181639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ac100950 0x7f60ac197d30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f60a8031610 tx=0x7f60a8004270 comp rx=0 tx=0).stop 2024-09-11T23:14:20.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:14:20.910+0000 7f60b319e640 1 -- 172.21.15.43:0/1194181639 shutdown_connections 2024-09-11T23:14:20.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:14:20.910+0000 7f60b319e640 1 --2- 172.21.15.43:0/1194181639 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f6098039290 0x7f609803b750 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:20.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:14:20.910+0000 7f60b319e640 1 --2- 172.21.15.43:0/1194181639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60ac102510 0x7f60ac19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:20.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:14:20.910+0000 7f60b319e640 1 --2- 172.21.15.43:0/1194181639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60ac101b50 0x7f60ac198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:20.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:14:20.910+0000 7f60b319e640 1 --2- 172.21.15.43:0/1194181639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60ac100950 0x7f60ac197d30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:20.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:14:20.910+0000 7f60b319e640 1 -- 172.21.15.43:0/1194181639 >> 172.21.15.43:0/1194181639 conn(0x7f60ac0fc0e0 msgr2=0x7f60ac069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:14:20.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:14:20.911+0000 7f60b319e640 1 -- 172.21.15.43:0/1194181639 shutdown_connections 2024-09-11T23:14:20.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:14:20.911+0000 7f60b319e640 1 -- 172.21.15.43:0/1194181639 wait complete. 2024-09-11T23:14:21.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:21 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:21.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:21 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:21.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:21 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:21.548 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:21 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:21.548 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:21 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:21.548 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:21 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:21.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:21 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:21.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:21 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:21.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:21 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:21.581 DEBUG:teuthology.orchestra.run.smithi043:osd.0> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.0.service 2024-09-11T23:14:21.583 INFO:tasks.cephadm:Deploying osd.1 on smithi090 with /dev/vg_nvme/lv_4... 2024-09-11T23:14:21.584 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T23:14:21.821 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:14:22.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:22 smithi090 ceph-mon[35130]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:22.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:22 smithi097 ceph-mon[35562]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:22.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:22 smithi043 ceph-mon[30950]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:22.611 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:14:22 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-0[44650]: 2024-09-11T23:14:22.365+0000 7f7c35422640 -1 osd.0 0 log_to_monitors true 2024-09-11T23:14:23.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:23 smithi090 ceph-mon[35130]: from='osd.0 [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T23:14:23.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:23 smithi043 ceph-mon[30950]: from='osd.0 [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T23:14:23.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:23 smithi097 ceph-mon[35562]: from='osd.0 [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T23:14:24.078 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-11T23:14:24.101 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_4 2024-09-11T23:14:24.347 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:14:24.471 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:24 smithi043 ceph-mon[30950]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:24.471 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:24 smithi043 ceph-mon[30950]: from='osd.0 [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T23:14:24.471 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:24 smithi043 ceph-mon[30950]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T23:14:24.471 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:24 smithi043 ceph-mon[30950]: from='osd.0 [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T23:14:24.472 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:24 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:14:24.472 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:14:24 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-0[44650]: 2024-09-11T23:14:24.277+0000 7f7c313a5640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T23:14:24.472 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:14:24 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-0[44650]: 2024-09-11T23:14:24.279+0000 7f7c2d1cd640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:14:24.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:24 smithi097 ceph-mon[35562]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:24.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:24 smithi097 ceph-mon[35562]: from='osd.0 [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T23:14:24.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:24 smithi097 ceph-mon[35562]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T23:14:24.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:24 smithi097 ceph-mon[35562]: from='osd.0 [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T23:14:24.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:24 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:14:24.596 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:24 smithi090 ceph-mon[35130]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:24.596 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:24 smithi090 ceph-mon[35130]: from='osd.0 [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T23:14:24.596 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:24 smithi090 ceph-mon[35130]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T23:14:24.596 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:24 smithi090 ceph-mon[35130]: from='osd.0 [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T23:14:24.596 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:24 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:14:24.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.812+0000 7fcb78422640 1 -- 172.21.15.90:0/2289654320 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb70101cd0 msgr2=0x7fcb70102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:14:24.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.812+0000 7fcb78422640 1 --2- 172.21.15.90:0/2289654320 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb70101cd0 0x7fcb70102150 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009900 tx=0x7fcb6002a410 comp rx=0 tx=0).stop 2024-09-11T23:14:24.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.812+0000 7fcb78422640 1 -- 172.21.15.90:0/2289654320 shutdown_connections 2024-09-11T23:14:24.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.812+0000 7fcb78422640 1 --2- 172.21.15.90:0/2289654320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb70102690 0x7fcb70108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:24.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.812+0000 7fcb78422640 1 --2- 172.21.15.90:0/2289654320 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb70101cd0 0x7fcb70102150 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:24.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.812+0000 7fcb78422640 1 --2- 172.21.15.90:0/2289654320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb70100ad0 0x7fcb70100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:24.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.812+0000 7fcb78422640 1 -- 172.21.15.90:0/2289654320 >> 172.21.15.90:0/2289654320 conn(0x7fcb700fc280 msgr2=0x7fcb700fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:14:24.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.812+0000 7fcb78422640 1 -- 172.21.15.90:0/2289654320 shutdown_connections 2024-09-11T23:14:24.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.812+0000 7fcb78422640 1 -- 172.21.15.90:0/2289654320 wait complete. 2024-09-11T23:14:24.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.813+0000 7fcb78422640 1 Processor -- start 2024-09-11T23:14:24.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.813+0000 7fcb78422640 1 -- start start 2024-09-11T23:14:24.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.813+0000 7fcb78422640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb70100ad0 0x7fcb70104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:14:24.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.813+0000 7fcb78422640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb70101cd0 0x7fcb701030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:14:24.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.813+0000 7fcb78422640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb70102690 0x7fcb701035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:14:24.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.813+0000 7fcb78422640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7010c750 con 0x7fcb70102690 2024-09-11T23:14:24.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.813+0000 7fcb78422640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7010c5d0 con 0x7fcb70101cd0 2024-09-11T23:14:24.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.813+0000 7fcb78422640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7010c8d0 con 0x7fcb70100ad0 2024-09-11T23:14:24.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.813+0000 7fcb759bd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb70101cd0 0x7fcb701030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:14:24.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.813+0000 7fcb759bd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb70101cd0 0x7fcb701030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53212/0 (socket says 172.21.15.90:53212) 2024-09-11T23:14:24.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.813+0000 7fcb759bd640 1 -- 172.21.15.90:0/335927494 learned_addr learned my addr 172.21.15.90:0/335927494 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-11T23:14:24.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.813+0000 7fcb769bf640 1 --2- 172.21.15.90:0/335927494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb70102690 0x7fcb701035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:14:24.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.814+0000 7fcb761be640 1 --2- 172.21.15.90:0/335927494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb70100ad0 0x7fcb70104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:14:24.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.814+0000 7fcb759bd640 1 -- 172.21.15.90:0/335927494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb70100ad0 msgr2=0x7fcb70104a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:14:24.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.814+0000 7fcb759bd640 1 --2- 172.21.15.90:0/335927494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb70100ad0 0x7fcb70104a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:24.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.814+0000 7fcb759bd640 1 -- 172.21.15.90:0/335927494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb70102690 msgr2=0x7fcb701035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:14:24.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.814+0000 7fcb759bd640 1 --2- 172.21.15.90:0/335927494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb70102690 0x7fcb701035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:24.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.814+0000 7fcb759bd640 1 -- 172.21.15.90:0/335927494 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb70103eb0 con 0x7fcb70101cd0 2024-09-11T23:14:24.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.814+0000 7fcb759bd640 1 --2- 172.21.15.90:0/335927494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb70101cd0 0x7fcb701030b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009230 tx=0x7fcb60004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:14:24.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.814+0000 7fcb677fe640 1 -- 172.21.15.90:0/335927494 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6003f030 con 0x7fcb70101cd0 2024-09-11T23:14:24.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.814+0000 7fcb78422640 1 -- 172.21.15.90:0/335927494 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb701a1ff0 con 0x7fcb70101cd0 2024-09-11T23:14:24.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.814+0000 7fcb677fe640 1 -- 172.21.15.90:0/335927494 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb60003ce0 con 0x7fcb70101cd0 2024-09-11T23:14:24.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.814+0000 7fcb677fe640 1 -- 172.21.15.90:0/335927494 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60035070 con 0x7fcb70101cd0 2024-09-11T23:14:24.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.814+0000 7fcb78422640 1 -- 172.21.15.90:0/335927494 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb701a23e0 con 0x7fcb70101cd0 2024-09-11T23:14:24.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.815+0000 7fcb78422640 1 -- 172.21.15.90:0/335927494 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb701a2610 con 0x7fcb70101cd0 2024-09-11T23:14:24.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.816+0000 7fcb677fe640 1 -- 172.21.15.90:0/335927494 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fcb60042030 con 0x7fcb70101cd0 2024-09-11T23:14:24.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.817+0000 7fcb677fe640 1 --2- 172.21.15.90:0/335927494 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fcb5c0a2af0 0x7fcb5c0a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:14:24.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.817+0000 7fcb677fe640 1 -- 172.21.15.90:0/335927494 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7fcb600edf90 con 0x7fcb70101cd0 2024-09-11T23:14:24.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.817+0000 7fcb761be640 1 --2- 172.21.15.90:0/335927494 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fcb5c0a2af0 0x7fcb5c0a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:14:24.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.818+0000 7fcb761be640 1 --2- 172.21.15.90:0/335927494 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fcb5c0a2af0 0x7fcb5c0a4fb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c0058c0 tx=0x7fcb6c00a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:14:24.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.820+0000 7fcb677fe640 1 -- 172.21.15.90:0/335927494 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb600bca80 con 0x7fcb70101cd0 2024-09-11T23:14:25.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:24.999+0000 7fcb78422640 1 -- 172.21.15.90:0/335927494 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fcb7006a6b0 con 0x7fcb5c0a2af0 2024-09-11T23:14:25.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:25 smithi097 ceph-mon[35562]: from='osd.0 [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T23:14:25.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:25 smithi097 ceph-mon[35562]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T23:14:25.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:25 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:14:25.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:25 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:14:25.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:25 smithi097 ceph-mon[35562]: Detected new or changed devices on smithi043 2024-09-11T23:14:25.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:25 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:25.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:25 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:25.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:25 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:25.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:25 smithi097 ceph-mon[35562]: Adjusting osd_memory_target on smithi043 to 17092M 2024-09-11T23:14:25.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:25 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:25.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:25 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:25.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:25 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:25.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:25 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:25.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:25 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:14:25.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:25 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:14:25.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:25 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:25.600 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:25 smithi090 ceph-mon[35130]: from='osd.0 [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T23:14:25.600 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:25 smithi090 ceph-mon[35130]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T23:14:25.600 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:25 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:14:25.600 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:25 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:14:25.601 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:25 smithi090 ceph-mon[35130]: Detected new or changed devices on smithi043 2024-09-11T23:14:25.601 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:25 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:25.601 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:25 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:25.601 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:25 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:25.601 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:25 smithi090 ceph-mon[35130]: Adjusting osd_memory_target on smithi043 to 17092M 2024-09-11T23:14:25.601 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:25 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:25.601 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:25 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:25.601 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:25 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:25.601 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:25 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:25.601 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:25 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:14:25.601 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:25 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:14:25.601 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:25 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:25.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:25 smithi043 ceph-mon[30950]: from='osd.0 [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T23:14:25.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:25 smithi043 ceph-mon[30950]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T23:14:25.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:25 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:14:25.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:25 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:14:25.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:25 smithi043 ceph-mon[30950]: Detected new or changed devices on smithi043 2024-09-11T23:14:25.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:25 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:25.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:25 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:25.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:25 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:25.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:25 smithi043 ceph-mon[30950]: Adjusting osd_memory_target on smithi043 to 17092M 2024-09-11T23:14:25.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:25 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:25.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:25 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:25.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:25 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:25.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:25 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:25.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:25 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:14:25.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:25 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:14:25.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:25 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:26.304 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:26 smithi090 ceph-mon[35130]: purged_snaps scrub starts 2024-09-11T23:14:26.304 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:26 smithi090 ceph-mon[35130]: purged_snaps scrub ok 2024-09-11T23:14:26.304 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:26 smithi090 ceph-mon[35130]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:26.304 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:26 smithi090 ceph-mon[35130]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:14:26.304 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:26 smithi090 ceph-mon[35130]: osd.0 [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997] boot 2024-09-11T23:14:26.304 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:26 smithi090 ceph-mon[35130]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T23:14:26.304 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:26 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:14:26.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:26 smithi097 ceph-mon[35562]: purged_snaps scrub starts 2024-09-11T23:14:26.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:26 smithi097 ceph-mon[35562]: purged_snaps scrub ok 2024-09-11T23:14:26.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:26 smithi097 ceph-mon[35562]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:26.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:26 smithi097 ceph-mon[35562]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:14:26.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:26 smithi097 ceph-mon[35562]: osd.0 [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997] boot 2024-09-11T23:14:26.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:26 smithi097 ceph-mon[35562]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T23:14:26.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:26 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:14:26.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:26 smithi043 ceph-mon[30950]: purged_snaps scrub starts 2024-09-11T23:14:26.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:26 smithi043 ceph-mon[30950]: purged_snaps scrub ok 2024-09-11T23:14:26.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:26 smithi043 ceph-mon[30950]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T23:14:26.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:26 smithi043 ceph-mon[30950]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:14:26.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:26 smithi043 ceph-mon[30950]: osd.0 [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997] boot 2024-09-11T23:14:26.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:26 smithi043 ceph-mon[30950]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T23:14:26.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:26 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:14:27.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:27 smithi097 ceph-mon[35562]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T23:14:27.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:27 smithi097 ceph-mon[35562]: from='client.? 172.21.15.90:0/1809413342' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0472a57-243c-489d-a247-972ee10a71f8"}]: dispatch 2024-09-11T23:14:27.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:27 smithi097 ceph-mon[35562]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0472a57-243c-489d-a247-972ee10a71f8"}]: dispatch 2024-09-11T23:14:27.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:27 smithi097 ceph-mon[35562]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0472a57-243c-489d-a247-972ee10a71f8"}]': finished 2024-09-11T23:14:27.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:27 smithi097 ceph-mon[35562]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T23:14:27.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:27 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:14:27.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:27 smithi043 ceph-mon[30950]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T23:14:27.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:27 smithi043 ceph-mon[30950]: from='client.? 172.21.15.90:0/1809413342' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0472a57-243c-489d-a247-972ee10a71f8"}]: dispatch 2024-09-11T23:14:27.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:27 smithi043 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0472a57-243c-489d-a247-972ee10a71f8"}]: dispatch 2024-09-11T23:14:27.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:27 smithi043 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0472a57-243c-489d-a247-972ee10a71f8"}]': finished 2024-09-11T23:14:27.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:27 smithi043 ceph-mon[30950]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T23:14:27.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:27 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:14:27.751 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:27 smithi090 ceph-mon[35130]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T23:14:27.751 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:27 smithi090 ceph-mon[35130]: from='client.? 172.21.15.90:0/1809413342' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0472a57-243c-489d-a247-972ee10a71f8"}]: dispatch 2024-09-11T23:14:27.751 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:27 smithi090 ceph-mon[35130]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0472a57-243c-489d-a247-972ee10a71f8"}]: dispatch 2024-09-11T23:14:27.751 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:27 smithi090 ceph-mon[35130]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0472a57-243c-489d-a247-972ee10a71f8"}]': finished 2024-09-11T23:14:27.751 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:27 smithi090 ceph-mon[35130]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T23:14:27.752 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:27 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:14:28.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:28 smithi097 ceph-mon[35562]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:28.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:28 smithi097 ceph-mon[35562]: from='client.? 172.21.15.90:0/539858171' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:14:28.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:28 smithi043 ceph-mon[30950]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:28.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:28 smithi043 ceph-mon[30950]: from='client.? 172.21.15.90:0/539858171' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:14:28.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:28 smithi090 ceph-mon[35130]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:28.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:28 smithi090 ceph-mon[35130]: from='client.? 172.21.15.90:0/539858171' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:14:30.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:30 smithi097 ceph-mon[35562]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:30.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:30 smithi043 ceph-mon[30950]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:30.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:30 smithi090 ceph-mon[35130]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:32.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:32 smithi097 ceph-mon[35562]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:32.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:32 smithi043 ceph-mon[30950]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:32.690 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:32 smithi090 ceph-mon[35130]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:34.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:34 smithi097 ceph-mon[35562]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:34.564 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:34 smithi090 ceph-mon[35130]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:34.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:34 smithi043 ceph-mon[30950]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:36.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:36 smithi090 ceph-mon[35130]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:36.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:36 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T23:14:36.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:36 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:36.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:36 smithi090 ceph-mon[35130]: Deploying daemon osd.1 on smithi090 2024-09-11T23:14:36.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:36 smithi043 ceph-mon[30950]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:36.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:36 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T23:14:36.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:36 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:36.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:36 smithi043 ceph-mon[30950]: Deploying daemon osd.1 on smithi090 2024-09-11T23:14:36.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:36 smithi097 ceph-mon[35562]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:36.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:36 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T23:14:36.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:36 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:36.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:36 smithi097 ceph-mon[35562]: Deploying daemon osd.1 on smithi090 2024-09-11T23:14:38.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:38 smithi043 ceph-mon[30950]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:38.721 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:38 smithi090 ceph-mon[35130]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:38.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:38 smithi097 ceph-mon[35562]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:40.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:40 smithi043 ceph-mon[30950]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:40.657 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:40 smithi090 ceph-mon[35130]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:40.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:40 smithi097 ceph-mon[35562]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:42.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:42 smithi090 ceph-mon[35130]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:42.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:42 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:42.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:42 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:42.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:42 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:42.603 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:42 smithi097 ceph-mon[35562]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:42.604 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:42 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:42.604 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:42 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:42.604 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:42 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:42.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:42 smithi043 ceph-mon[30950]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:42.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:42 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:42.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:42 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:42.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:42 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:44.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:44 smithi090 ceph-mon[35130]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:44.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:44 smithi043 ceph-mon[30950]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:44.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:44 smithi097 ceph-mon[35562]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:45.714 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:45 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:45.714 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:45 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:45.714 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:45 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:45.714 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:45 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:45.715 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:45 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:45.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:45.756+0000 7fcb677fe640 1 -- 172.21.15.90:0/335927494 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcb7006a6b0 con 0x7fcb5c0a2af0 2024-09-11T23:14:45.756 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 1 on host 'smithi090' 2024-09-11T23:14:45.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:45.758+0000 7fcb78422640 1 -- 172.21.15.90:0/335927494 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fcb5c0a2af0 msgr2=0x7fcb5c0a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:14:45.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:45.758+0000 7fcb78422640 1 --2- 172.21.15.90:0/335927494 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fcb5c0a2af0 0x7fcb5c0a4fb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c0058c0 tx=0x7fcb6c00a4c0 comp rx=0 tx=0).stop 2024-09-11T23:14:45.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:45.758+0000 7fcb78422640 1 -- 172.21.15.90:0/335927494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb70101cd0 msgr2=0x7fcb701030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:14:45.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:45.758+0000 7fcb78422640 1 --2- 172.21.15.90:0/335927494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb70101cd0 0x7fcb701030b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009230 tx=0x7fcb60004430 comp rx=0 tx=0).stop 2024-09-11T23:14:45.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:45.758+0000 7fcb78422640 1 -- 172.21.15.90:0/335927494 shutdown_connections 2024-09-11T23:14:45.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:45.758+0000 7fcb78422640 1 --2- 172.21.15.90:0/335927494 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fcb5c0a2af0 0x7fcb5c0a4fb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:45.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:45.758+0000 7fcb78422640 1 --2- 172.21.15.90:0/335927494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb70102690 0x7fcb701035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:45.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:45.758+0000 7fcb78422640 1 --2- 172.21.15.90:0/335927494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb70101cd0 0x7fcb701030b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:45.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:45.758+0000 7fcb78422640 1 --2- 172.21.15.90:0/335927494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb70100ad0 0x7fcb70104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:45.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:45.759+0000 7fcb78422640 1 -- 172.21.15.90:0/335927494 >> 172.21.15.90:0/335927494 conn(0x7fcb700fc280 msgr2=0x7fcb70069fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:14:45.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:45.759+0000 7fcb78422640 1 -- 172.21.15.90:0/335927494 shutdown_connections 2024-09-11T23:14:45.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:14:45.759+0000 7fcb78422640 1 -- 172.21.15.90:0/335927494 wait complete. 2024-09-11T23:14:45.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:45 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:45.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:45 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:45.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:45 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:45.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:45 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:45.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:45 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:45.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:45 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:45.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:45 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:45.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:45 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:45.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:45 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:45.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:45 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:46.389 DEBUG:teuthology.orchestra.run.smithi090:osd.1> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.1.service 2024-09-11T23:14:46.392 INFO:tasks.cephadm:Deploying osd.2 on smithi097 with /dev/vg_nvme/lv_4... 2024-09-11T23:14:46.392 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T23:14:46.642 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:14:46.690 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:46 smithi090 ceph-mon[35130]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:46.690 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:46 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:46.690 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:46 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:46.690 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:46 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:46.763 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:46 smithi097 ceph-mon[35562]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:46.764 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:46 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:46.764 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:46 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:46.764 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:46 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:46.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:46 smithi043 ceph-mon[30950]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:46.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:46 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:14:46.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:46 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:46.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:46 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:48.020 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:14:47 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-1[39555]: 2024-09-11T23:14:47.862+0000 7fbe39a3b640 -1 osd.1 0 log_to_monitors true 2024-09-11T23:14:48.729 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:48 smithi090 ceph-mon[35130]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:48.729 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:48 smithi090 ceph-mon[35130]: from='osd.1 [v2:172.21.15.90:6800/1825149246,v1:172.21.15.90:6801/1825149246]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:14:48.729 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:48 smithi090 ceph-mon[35130]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:14:48.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:48 smithi097 ceph-mon[35562]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:48.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:48 smithi097 ceph-mon[35562]: from='osd.1 [v2:172.21.15.90:6800/1825149246,v1:172.21.15.90:6801/1825149246]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:14:48.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:48 smithi097 ceph-mon[35562]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:14:48.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:48 smithi043 ceph-mon[30950]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:48.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:48 smithi043 ceph-mon[30950]: from='osd.1 [v2:172.21.15.90:6800/1825149246,v1:172.21.15.90:6801/1825149246]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:14:48.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:48 smithi043 ceph-mon[30950]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T23:14:49.182 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:14:49.210 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_4 2024-09-11T23:14:49.456 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:14:49.731 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:49 smithi097 ceph-mon[35562]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T23:14:49.731 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:49 smithi097 ceph-mon[35562]: from='osd.1 [v2:172.21.15.90:6800/1825149246,v1:172.21.15.90:6801/1825149246]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:14:49.731 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:49 smithi097 ceph-mon[35562]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T23:14:49.731 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:14:49.731 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:49 smithi097 ceph-mon[35562]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:14:49.731 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:49.732 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:49.732 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:49.732 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:49.732 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:49.732 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:49.732 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:49.732 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:49 smithi097 ceph-mon[35562]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-11T23:14:49.732 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:49 smithi097 ceph-mon[35562]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T23:14:49.732 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:14:49.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:49 smithi090 ceph-mon[35130]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T23:14:49.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:49 smithi090 ceph-mon[35130]: from='osd.1 [v2:172.21.15.90:6800/1825149246,v1:172.21.15.90:6801/1825149246]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:14:49.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:49 smithi090 ceph-mon[35130]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T23:14:49.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:14:49.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:49 smithi090 ceph-mon[35130]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:14:49.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:49.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:49.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:49.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:49.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:49.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:49.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:49.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:49 smithi090 ceph-mon[35130]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-11T23:14:49.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:49 smithi090 ceph-mon[35130]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T23:14:49.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:14:49.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:49 smithi043 ceph-mon[30950]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T23:14:49.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:49 smithi043 ceph-mon[30950]: from='osd.1 [v2:172.21.15.90:6800/1825149246,v1:172.21.15.90:6801/1825149246]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:14:49.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:49 smithi043 ceph-mon[30950]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T23:14:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:14:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:49 smithi043 ceph-mon[30950]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:14:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:14:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:14:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:14:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:49 smithi043 ceph-mon[30950]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-11T23:14:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:49 smithi043 ceph-mon[30950]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T23:14:49.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:14:49.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.959+0000 7fa4e447b640 1 -- 172.21.15.97:0/143461024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4dc102df0 msgr2=0x7fa4dc103270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:14:49.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.959+0000 7fa4e447b640 1 --2- 172.21.15.97:0/143461024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4dc102df0 0x7fa4dc103270 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009900 tx=0x7fa4cc02a410 comp rx=0 tx=0).stop 2024-09-11T23:14:49.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.959+0000 7fa4e447b640 1 -- 172.21.15.97:0/143461024 shutdown_connections 2024-09-11T23:14:49.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.959+0000 7fa4e447b640 1 --2- 172.21.15.97:0/143461024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4dc1037b0 0x7fa4dc10a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:49.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.959+0000 7fa4e447b640 1 --2- 172.21.15.97:0/143461024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4dc102df0 0x7fa4dc103270 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:49.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.959+0000 7fa4e447b640 1 --2- 172.21.15.97:0/143461024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4dc101bf0 0x7fa4dc101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:49.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.959+0000 7fa4e447b640 1 -- 172.21.15.97:0/143461024 >> 172.21.15.97:0/143461024 conn(0x7fa4dc0fd3c0 msgr2=0x7fa4dc0ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:14:49.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.959+0000 7fa4e447b640 1 -- 172.21.15.97:0/143461024 shutdown_connections 2024-09-11T23:14:49.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.959+0000 7fa4e447b640 1 -- 172.21.15.97:0/143461024 wait complete. 2024-09-11T23:14:49.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.959+0000 7fa4e447b640 1 Processor -- start 2024-09-11T23:14:49.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4e447b640 1 -- start start 2024-09-11T23:14:49.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4e447b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4dc101bf0 0x7fa4dc197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:14:49.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4e447b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4dc102df0 0x7fa4dc198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:14:49.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4e447b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4dc1037b0 0x7fa4dc19f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:14:49.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4e447b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc10d850 con 0x7fa4dc102df0 2024-09-11T23:14:49.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4e447b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc10d6d0 con 0x7fa4dc101bf0 2024-09-11T23:14:49.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4e447b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc10d9d0 con 0x7fa4dc1037b0 2024-09-11T23:14:49.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4e2a18640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4dc1037b0 0x7fa4dc19f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:14:49.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4e2a18640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4dc1037b0 0x7fa4dc19f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52086/0 (socket says 172.21.15.97:52086) 2024-09-11T23:14:49.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4e2a18640 1 -- 172.21.15.97:0/3234849273 learned_addr learned my addr 172.21.15.97:0/3234849273 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-11T23:14:49.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4e2217640 1 --2- 172.21.15.97:0/3234849273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4dc101bf0 0x7fa4dc197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:14:49.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4e1a16640 1 --2- 172.21.15.97:0/3234849273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4dc102df0 0x7fa4dc198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:14:49.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4e2a18640 1 -- 172.21.15.97:0/3234849273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4dc101bf0 msgr2=0x7fa4dc197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:14:49.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4e2a18640 1 --2- 172.21.15.97:0/3234849273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4dc101bf0 0x7fa4dc197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:49.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4e2a18640 1 -- 172.21.15.97:0/3234849273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4dc102df0 msgr2=0x7fa4dc198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:14:49.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4e2a18640 1 --2- 172.21.15.97:0/3234849273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4dc102df0 0x7fa4dc198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:14:49.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4e2a18640 1 -- 172.21.15.97:0/3234849273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4dc19fb00 con 0x7fa4dc1037b0 2024-09-11T23:14:49.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4e1a16640 1 --2- 172.21.15.97:0/3234849273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4dc102df0 0x7fa4dc198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:14:49.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4e2a18640 1 --2- 172.21.15.97:0/3234849273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4dc1037b0 0x7fa4dc19f490 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400efd0 tx=0x7fa4d400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:14:49.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4d37fe640 1 -- 172.21.15.97:0/3234849273 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4003b70 con 0x7fa4dc1037b0 2024-09-11T23:14:49.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4d37fe640 1 -- 172.21.15.97:0/3234849273 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d4003d10 con 0x7fa4dc1037b0 2024-09-11T23:14:49.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.960+0000 7fa4d37fe640 1 -- 172.21.15.97:0/3234849273 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d40109f0 con 0x7fa4dc1037b0 2024-09-11T23:14:49.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.961+0000 7fa4e447b640 1 -- 172.21.15.97:0/3234849273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4dc19fdf0 con 0x7fa4dc1037b0 2024-09-11T23:14:49.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.961+0000 7fa4e447b640 1 -- 172.21.15.97:0/3234849273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4dc1a02a0 con 0x7fa4dc1037b0 2024-09-11T23:14:49.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.962+0000 7fa4e447b640 1 -- 172.21.15.97:0/3234849273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4dc108fc0 con 0x7fa4dc1037b0 2024-09-11T23:14:49.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.963+0000 7fa4d37fe640 1 -- 172.21.15.97:0/3234849273 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fa4d4004000 con 0x7fa4dc1037b0 2024-09-11T23:14:49.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.963+0000 7fa4d37fe640 1 --2- 172.21.15.97:0/3234849273 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa4c80a2ba0 0x7fa4c80a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:14:49.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.963+0000 7fa4d37fe640 1 -- 172.21.15.97:0/3234849273 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7fa4d40cda90 con 0x7fa4dc1037b0 2024-09-11T23:14:49.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.964+0000 7fa4e2217640 1 --2- 172.21.15.97:0/3234849273 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa4c80a2ba0 0x7fa4c80a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:14:49.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.964+0000 7fa4e2217640 1 --2- 172.21.15.97:0/3234849273 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa4c80a2ba0 0x7fa4c80a5060 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc102c50 tx=0x7fa4d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:14:49.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:49.968+0000 7fa4d37fe640 1 -- 172.21.15.97:0/3234849273 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa4d409c360 con 0x7fa4dc1037b0 2024-09-11T23:14:50.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:14:50.147+0000 7fa4e447b640 1 -- 172.21.15.97:0/3234849273 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa4dc062630 con 0x7fa4c80a2ba0 2024-09-11T23:14:50.270 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:14:49 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-1[39555]: 2024-09-11T23:14:49.869+0000 7fbe361d2640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T23:14:50.270 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:14:49 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-1[39555]: 2024-09-11T23:14:49.872+0000 7fbe2d7de640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:14:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:50 smithi090 ceph-mon[35130]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:50 smithi090 ceph-mon[35130]: Detected new or changed devices on smithi090 2024-09-11T23:14:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:50 smithi090 ceph-mon[35130]: Adjusting osd_memory_target on smithi090 to 17088M 2024-09-11T23:14:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:50 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:14:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:50 smithi090 ceph-mon[35130]: from='osd.1 ' entity='osd.1' 2024-09-11T23:14:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:50 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:14:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:50 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:14:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:50 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:50 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:14:50.777 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:50 smithi097 ceph-mon[35562]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:50.777 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:50 smithi097 ceph-mon[35562]: Detected new or changed devices on smithi090 2024-09-11T23:14:50.777 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:50 smithi097 ceph-mon[35562]: Adjusting osd_memory_target on smithi090 to 17088M 2024-09-11T23:14:50.777 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:50 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:14:50.777 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:50 smithi097 ceph-mon[35562]: from='osd.1 ' entity='osd.1' 2024-09-11T23:14:50.777 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:50 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:14:50.777 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:50 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:14:50.777 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:50 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:50.778 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:50 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:14:50.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:50 smithi043 ceph-mon[30950]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:50.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:50 smithi043 ceph-mon[30950]: Detected new or changed devices on smithi090 2024-09-11T23:14:50.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:50 smithi043 ceph-mon[30950]: Adjusting osd_memory_target on smithi090 to 17088M 2024-09-11T23:14:50.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:50 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:14:50.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:50 smithi043 ceph-mon[30950]: from='osd.1 ' entity='osd.1' 2024-09-11T23:14:50.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:50 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:14:50.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:50 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:14:50.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:50 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:14:50.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:50 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:14:51.762 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:51 smithi043 ceph-mon[30950]: purged_snaps scrub starts 2024-09-11T23:14:51.762 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:51 smithi043 ceph-mon[30950]: purged_snaps scrub ok 2024-09-11T23:14:51.762 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:51 smithi043 ceph-mon[30950]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:14:51.762 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:51 smithi043 ceph-mon[30950]: osd.1 [v2:172.21.15.90:6800/1825149246,v1:172.21.15.90:6801/1825149246] boot 2024-09-11T23:14:51.762 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:51 smithi043 ceph-mon[30950]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T23:14:51.762 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:51 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:14:51.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:51 smithi090 ceph-mon[35130]: purged_snaps scrub starts 2024-09-11T23:14:51.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:51 smithi090 ceph-mon[35130]: purged_snaps scrub ok 2024-09-11T23:14:51.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:51 smithi090 ceph-mon[35130]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:14:51.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:51 smithi090 ceph-mon[35130]: osd.1 [v2:172.21.15.90:6800/1825149246,v1:172.21.15.90:6801/1825149246] boot 2024-09-11T23:14:51.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:51 smithi090 ceph-mon[35130]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T23:14:51.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:51 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:14:51.790 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:51 smithi097 ceph-mon[35562]: purged_snaps scrub starts 2024-09-11T23:14:51.791 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:51 smithi097 ceph-mon[35562]: purged_snaps scrub ok 2024-09-11T23:14:51.791 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:51 smithi097 ceph-mon[35562]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:14:51.791 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:51 smithi097 ceph-mon[35562]: osd.1 [v2:172.21.15.90:6800/1825149246,v1:172.21.15.90:6801/1825149246] boot 2024-09-11T23:14:51.791 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:51 smithi097 ceph-mon[35562]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T23:14:51.791 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:51 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:14:52.734 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:52 smithi097 ceph-mon[35562]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:52.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:52 smithi090 ceph-mon[35130]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:52.777 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:52 smithi043 ceph-mon[30950]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T23:14:53.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:53 smithi090 ceph-mon[35130]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T23:14:53.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:53 smithi090 ceph-mon[35130]: from='client.? 172.21.15.97:0/2664364028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40366536-0fcf-45a0-b8e3-49178401dc34"}]: dispatch 2024-09-11T23:14:53.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:53 smithi090 ceph-mon[35130]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40366536-0fcf-45a0-b8e3-49178401dc34"}]: dispatch 2024-09-11T23:14:53.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:53 smithi090 ceph-mon[35130]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40366536-0fcf-45a0-b8e3-49178401dc34"}]': finished 2024-09-11T23:14:53.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:53 smithi090 ceph-mon[35130]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T23:14:53.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:53 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:14:53.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:53 smithi090 ceph-mon[35130]: from='client.? 172.21.15.97:0/3870682375' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:14:53.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:53 smithi097 ceph-mon[35562]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T23:14:53.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:53 smithi097 ceph-mon[35562]: from='client.? 172.21.15.97:0/2664364028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40366536-0fcf-45a0-b8e3-49178401dc34"}]: dispatch 2024-09-11T23:14:53.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:53 smithi097 ceph-mon[35562]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40366536-0fcf-45a0-b8e3-49178401dc34"}]: dispatch 2024-09-11T23:14:53.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:53 smithi097 ceph-mon[35562]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40366536-0fcf-45a0-b8e3-49178401dc34"}]': finished 2024-09-11T23:14:53.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:53 smithi097 ceph-mon[35562]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T23:14:53.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:53 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:14:53.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:53 smithi097 ceph-mon[35562]: from='client.? 172.21.15.97:0/3870682375' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:14:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:53 smithi043 ceph-mon[30950]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T23:14:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:53 smithi043 ceph-mon[30950]: from='client.? 172.21.15.97:0/2664364028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40366536-0fcf-45a0-b8e3-49178401dc34"}]: dispatch 2024-09-11T23:14:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:53 smithi043 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40366536-0fcf-45a0-b8e3-49178401dc34"}]: dispatch 2024-09-11T23:14:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:53 smithi043 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40366536-0fcf-45a0-b8e3-49178401dc34"}]': finished 2024-09-11T23:14:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:53 smithi043 ceph-mon[30950]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T23:14:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:53 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:14:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:53 smithi043 ceph-mon[30950]: from='client.? 172.21.15.97:0/3870682375' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:14:54.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:54 smithi090 ceph-mon[35130]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:14:54.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:54 smithi097 ceph-mon[35562]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:14:54.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:54 smithi043 ceph-mon[30950]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:14:56.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:56 smithi090 ceph-mon[35130]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:14:56.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:56 smithi097 ceph-mon[35562]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:14:56.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:56 smithi043 ceph-mon[30950]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:14:58.749 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:14:58 smithi097 ceph-mon[35562]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:14:58.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:14:58 smithi090 ceph-mon[35130]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:14:58.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:14:58 smithi043 ceph-mon[30950]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:00.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:00 smithi097 ceph-mon[35562]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:00.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:00 smithi090 ceph-mon[35130]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:00.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:00 smithi043 ceph-mon[30950]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:01.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:01 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T23:15:01.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:01 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:01 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T23:15:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:01 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:01.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:01 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T23:15:01.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:01 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:02.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:02 smithi090 ceph-mon[35130]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:02.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:02 smithi090 ceph-mon[35130]: Deploying daemon osd.2 on smithi097 2024-09-11T23:15:02.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:02 smithi097 ceph-mon[35562]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:02.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:02 smithi097 ceph-mon[35562]: Deploying daemon osd.2 on smithi097 2024-09-11T23:15:02.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:02 smithi043 ceph-mon[30950]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:02.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:02 smithi043 ceph-mon[30950]: Deploying daemon osd.2 on smithi097 2024-09-11T23:15:04.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:04 smithi097 ceph-mon[35562]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:04.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:04 smithi043 ceph-mon[30950]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:05.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:04 smithi090 ceph-mon[35130]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:06.755 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:06 smithi097 ceph-mon[35562]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:06.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:06 smithi043 ceph-mon[30950]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:07.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:06 smithi090 ceph-mon[35130]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:07.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:07.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:07.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:08.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:08.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:08.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:08.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:08 smithi097 ceph-mon[35562]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:08.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:08 smithi043 ceph-mon[30950]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:09.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:08 smithi090 ceph-mon[35130]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:10.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:10 smithi090 ceph-mon[35130]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:10.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:10 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:10.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:10 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:10.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:10 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:10.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:10 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:10.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:10 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:10.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:10 smithi097 ceph-mon[35562]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:10.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:10 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:10.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:10 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:10.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:10 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:10.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:10 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:10.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:10 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:10.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:10 smithi043 ceph-mon[30950]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:10.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:10 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:10.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:10 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:10.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:10 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:10.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:10 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:10.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:10 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:11.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:15:11.178+0000 7fa4d37fe640 1 -- 172.21.15.97:0/3234849273 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa4dc062630 con 0x7fa4c80a2ba0 2024-09-11T23:15:11.179 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 2 on host 'smithi097' 2024-09-11T23:15:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:15:11.180+0000 7fa4e447b640 1 -- 172.21.15.97:0/3234849273 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa4c80a2ba0 msgr2=0x7fa4c80a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:15:11.180+0000 7fa4e447b640 1 --2- 172.21.15.97:0/3234849273 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa4c80a2ba0 0x7fa4c80a5060 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc102c50 tx=0x7fa4d8005f90 comp rx=0 tx=0).stop 2024-09-11T23:15:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:15:11.180+0000 7fa4e447b640 1 -- 172.21.15.97:0/3234849273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4dc1037b0 msgr2=0x7fa4dc19f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:15:11.180+0000 7fa4e447b640 1 --2- 172.21.15.97:0/3234849273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4dc1037b0 0x7fa4dc19f490 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400efd0 tx=0x7fa4d400c6a0 comp rx=0 tx=0).stop 2024-09-11T23:15:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:15:11.181+0000 7fa4e447b640 1 -- 172.21.15.97:0/3234849273 shutdown_connections 2024-09-11T23:15:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:15:11.181+0000 7fa4e447b640 1 --2- 172.21.15.97:0/3234849273 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa4c80a2ba0 0x7fa4c80a5060 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:15:11.181+0000 7fa4e447b640 1 --2- 172.21.15.97:0/3234849273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4dc1037b0 0x7fa4dc19f490 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:11.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:15:11.181+0000 7fa4e447b640 1 --2- 172.21.15.97:0/3234849273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4dc102df0 0x7fa4dc198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:11.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:15:11.181+0000 7fa4e447b640 1 --2- 172.21.15.97:0/3234849273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4dc101bf0 0x7fa4dc197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:11.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:15:11.181+0000 7fa4e447b640 1 -- 172.21.15.97:0/3234849273 >> 172.21.15.97:0/3234849273 conn(0x7fa4dc0fd3c0 msgr2=0x7fa4dc106190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:11.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:15:11.181+0000 7fa4e447b640 1 -- 172.21.15.97:0/3234849273 shutdown_connections 2024-09-11T23:15:11.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:15:11.181+0000 7fa4e447b640 1 -- 172.21.15.97:0/3234849273 wait complete. 2024-09-11T23:15:11.869 DEBUG:teuthology.orchestra.run.smithi097:osd.2> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.2.service 2024-09-11T23:15:11.872 INFO:tasks.cephadm:Deploying osd.3 on smithi043 with /dev/vg_nvme/lv_3... 2024-09-11T23:15:11.872 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T23:15:12.137 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:15:12.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:12 smithi043 ceph-mon[30950]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:12.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:12 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:12.259 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:12 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:12.259 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:12 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:12.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:12 smithi090 ceph-mon[35130]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:12.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:12 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:12.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:12 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:12.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:12 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:12.526 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:12 smithi097 ceph-mon[35562]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:12.526 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:12 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:12.526 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:12 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:12.526 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:12 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:12.801 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:15:12 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-2[40538]: 2024-09-11T23:15:12.525+0000 7fbc9abf0640 -1 osd.2 0 log_to_monitors true 2024-09-11T23:15:13.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:13 smithi043 ceph-mon[30950]: from='osd.2 [v2:172.21.15.97:6800/1456846443,v1:172.21.15.97:6801/1456846443]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:15:13.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:13 smithi043 ceph-mon[30950]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:15:13.278 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:13 smithi097 ceph-mon[35562]: from='osd.2 [v2:172.21.15.97:6800/1456846443,v1:172.21.15.97:6801/1456846443]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:15:13.278 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:13 smithi097 ceph-mon[35562]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:15:13.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:13 smithi090 ceph-mon[35130]: from='osd.2 [v2:172.21.15.97:6800/1456846443,v1:172.21.15.97:6801/1456846443]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:15:13.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:13 smithi090 ceph-mon[35130]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T23:15:14.301 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:15:14 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-2[40538]: 2024-09-11T23:15:14.134+0000 7fbc96b73640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T23:15:14.301 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:15:14 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-2[40538]: 2024-09-11T23:15:14.138+0000 7fbc9299b640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:15:14.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:14 smithi097 ceph-mon[35562]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:14.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:14 smithi097 ceph-mon[35562]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T23:15:14.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:14 smithi097 ceph-mon[35562]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T23:15:14.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:14 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:15:14.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:14 smithi097 ceph-mon[35562]: from='osd.2 [v2:172.21.15.97:6800/1456846443,v1:172.21.15.97:6801/1456846443]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:15:14.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:14 smithi097 ceph-mon[35562]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:15:14.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:14 smithi043 ceph-mon[30950]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:14.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:14 smithi043 ceph-mon[30950]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T23:15:14.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:14 smithi043 ceph-mon[30950]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T23:15:14.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:14 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:15:14.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:14 smithi043 ceph-mon[30950]: from='osd.2 [v2:172.21.15.97:6800/1456846443,v1:172.21.15.97:6801/1456846443]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:15:14.463 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:14 smithi043 ceph-mon[30950]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:15:14.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:14 smithi090 ceph-mon[35130]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:14.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:14 smithi090 ceph-mon[35130]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T23:15:14.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:14 smithi090 ceph-mon[35130]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T23:15:14.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:14 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:15:14.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:14 smithi090 ceph-mon[35130]: from='osd.2 [v2:172.21.15.97:6800/1456846443,v1:172.21.15.97:6801/1456846443]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:15:14.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:14 smithi090 ceph-mon[35130]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:15:14.727 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:15:14.755 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_3 2024-09-11T23:15:15.030 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:15:15.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:15 smithi043 ceph-mon[30950]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-11T23:15:15.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:15 smithi043 ceph-mon[30950]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T23:15:15.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:15 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:15:15.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:15 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:15:15.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:15 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:15.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:15 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:15.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:15 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:15.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:15 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:15.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:15 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:15.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:15 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:15.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:15 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:15.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:15 smithi090 ceph-mon[35130]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-11T23:15:15.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:15 smithi090 ceph-mon[35130]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T23:15:15.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:15 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:15:15.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:15 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:15:15.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:15 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:15.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:15 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:15.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:15 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:15.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:15 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:15.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:15 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:15.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:15 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:15.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:15 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:15.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.488+0000 7f9ee6b3b640 1 -- 172.21.15.43:0/2230205183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0107960 msgr2=0x7f9ee0109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:15.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.488+0000 7f9ee6b3b640 1 --2- 172.21.15.43:0/2230205183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0107960 0x7f9ee0109d50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009ca0 tx=0x7f9ed802a8c0 comp rx=0 tx=0).stop 2024-09-11T23:15:15.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.488+0000 7f9ee6b3b640 1 -- 172.21.15.43:0/2230205183 shutdown_connections 2024-09-11T23:15:15.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.488+0000 7f9ee6b3b640 1 --2- 172.21.15.43:0/2230205183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0107960 0x7f9ee0109d50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:15.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.488+0000 7f9ee6b3b640 1 --2- 172.21.15.43:0/2230205183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ee0105030 0x7f9ee0107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:15.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.488+0000 7f9ee6b3b640 1 --2- 172.21.15.43:0/2230205183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ee0102700 0x7f9ee0104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:15.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.488+0000 7f9ee6b3b640 1 -- 172.21.15.43:0/2230205183 >> 172.21.15.43:0/2230205183 conn(0x7f9ee00fc280 msgr2=0x7f9ee00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:15.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.488+0000 7f9ee6b3b640 1 -- 172.21.15.43:0/2230205183 shutdown_connections 2024-09-11T23:15:15.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.488+0000 7f9ee6b3b640 1 -- 172.21.15.43:0/2230205183 wait complete. 2024-09-11T23:15:15.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.489+0000 7f9ee6b3b640 1 Processor -- start 2024-09-11T23:15:15.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.489+0000 7f9ee6b3b640 1 -- start start 2024-09-11T23:15:15.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.489+0000 7f9ee6b3b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ee0102700 0x7f9ee019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:15.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.489+0000 7f9ee6b3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0105030 0x7f9ee019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:15.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.489+0000 7f9ee6b3b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ee0107960 0x7f9ee01a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:15.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.489+0000 7f9ee6b3b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee010d860 con 0x7f9ee0105030 2024-09-11T23:15:15.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.489+0000 7f9ee6b3b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee010d6e0 con 0x7f9ee0102700 2024-09-11T23:15:15.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.489+0000 7f9ee6b3b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee010d9e0 con 0x7f9ee0107960 2024-09-11T23:15:15.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.489+0000 7f9edffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0105030 0x7f9ee019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:15.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.489+0000 7f9edffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0105030 0x7f9ee019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43394/0 (socket says 172.21.15.43:43394) 2024-09-11T23:15:15.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.489+0000 7f9edffff640 1 -- 172.21.15.43:0/2852107631 learned_addr learned my addr 172.21.15.43:0/2852107631 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:15:15.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.489+0000 7f9ee50d8640 1 --2- 172.21.15.43:0/2852107631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ee0107960 0x7f9ee01a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:15.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.489+0000 7f9ee48d7640 1 --2- 172.21.15.43:0/2852107631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ee0102700 0x7f9ee019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:15.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.489+0000 7f9edffff640 1 -- 172.21.15.43:0/2852107631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ee0107960 msgr2=0x7f9ee01a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:15.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.489+0000 7f9edffff640 1 --2- 172.21.15.43:0/2852107631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ee0107960 0x7f9ee01a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:15.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.489+0000 7f9edffff640 1 -- 172.21.15.43:0/2852107631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ee0102700 msgr2=0x7f9ee019a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:15.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.489+0000 7f9edffff640 1 --2- 172.21.15.43:0/2852107631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ee0102700 0x7f9ee019a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:15.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.489+0000 7f9edffff640 1 -- 172.21.15.43:0/2852107631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee01a1ce0 con 0x7f9ee0105030 2024-09-11T23:15:15.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.490+0000 7f9edffff640 1 --2- 172.21.15.43:0/2852107631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0105030 0x7f9ee019a5f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400e9c0 tx=0x7f9ed400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:15.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.490+0000 7f9eddffb640 1 -- 172.21.15.43:0/2852107631 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed400cd80 con 0x7f9ee0105030 2024-09-11T23:15:15.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.490+0000 7f9eddffb640 1 -- 172.21.15.43:0/2852107631 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed4005500 con 0x7f9ee0105030 2024-09-11T23:15:15.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.490+0000 7f9ee6b3b640 1 -- 172.21.15.43:0/2852107631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee01a1fd0 con 0x7f9ee0105030 2024-09-11T23:15:15.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.490+0000 7f9eddffb640 1 -- 172.21.15.43:0/2852107631 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4010690 con 0x7f9ee0105030 2024-09-11T23:15:15.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.490+0000 7f9ee6b3b640 1 -- 172.21.15.43:0/2852107631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee01a2480 con 0x7f9ee0105030 2024-09-11T23:15:15.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.491+0000 7f9ee6b3b640 1 -- 172.21.15.43:0/2852107631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ee0109450 con 0x7f9ee0105030 2024-09-11T23:15:15.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.492+0000 7f9eddffb640 1 -- 172.21.15.43:0/2852107631 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f9ed40046c0 con 0x7f9ee0105030 2024-09-11T23:15:15.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.493+0000 7f9eddffb640 1 --2- 172.21.15.43:0/2852107631 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9ec80a2b20 0x7f9ec80a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:15.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.493+0000 7f9ee48d7640 1 --2- 172.21.15.43:0/2852107631 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9ec80a2b20 0x7f9ec80a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:15.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.493+0000 7f9eddffb640 1 -- 172.21.15.43:0/2852107631 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7f9ed40cddb0 con 0x7f9ee0105030 2024-09-11T23:15:15.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.493+0000 7f9ee48d7640 1 --2- 172.21.15.43:0/2852107631 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9ec80a2b20 0x7f9ec80a4fe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009b60 tx=0x7f9ed0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:15.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.497+0000 7f9eddffb640 1 -- 172.21.15.43:0/2852107631 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ed409c340 con 0x7f9ee0105030 2024-09-11T23:15:15.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:15 smithi097 ceph-mon[35562]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-11T23:15:15.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:15 smithi097 ceph-mon[35562]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T23:15:15.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:15 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:15:15.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:15 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:15:15.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:15 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:15.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:15 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:15.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:15 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:15.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:15 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:15.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:15 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:15.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:15 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:15.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:15 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:15.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:15.683+0000 7f9ee6b3b640 1 -- 172.21.15.43:0/2852107631 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9ee0062630 con 0x7f9ec80a2b20 2024-09-11T23:15:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:16 smithi043 ceph-mon[30950]: purged_snaps scrub starts 2024-09-11T23:15:16.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:16 smithi043 ceph-mon[30950]: purged_snaps scrub ok 2024-09-11T23:15:16.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:16 smithi043 ceph-mon[30950]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:16.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:16 smithi043 ceph-mon[30950]: Detected new or changed devices on smithi097 2024-09-11T23:15:16.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:16 smithi043 ceph-mon[30950]: Adjusting osd_memory_target on smithi097 to 17088M 2024-09-11T23:15:16.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:16 smithi043 ceph-mon[30950]: osd.2 [v2:172.21.15.97:6800/1456846443,v1:172.21.15.97:6801/1456846443] boot 2024-09-11T23:15:16.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:16 smithi043 ceph-mon[30950]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T23:15:16.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:16 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:15:16.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:16 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:15:16.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:16 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:15:16.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:16 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:16.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:16 smithi090 ceph-mon[35130]: purged_snaps scrub starts 2024-09-11T23:15:16.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:16 smithi090 ceph-mon[35130]: purged_snaps scrub ok 2024-09-11T23:15:16.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:16 smithi090 ceph-mon[35130]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:16.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:16 smithi090 ceph-mon[35130]: Detected new or changed devices on smithi097 2024-09-11T23:15:16.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:16 smithi090 ceph-mon[35130]: Adjusting osd_memory_target on smithi097 to 17088M 2024-09-11T23:15:16.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:16 smithi090 ceph-mon[35130]: osd.2 [v2:172.21.15.97:6800/1456846443,v1:172.21.15.97:6801/1456846443] boot 2024-09-11T23:15:16.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:16 smithi090 ceph-mon[35130]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T23:15:16.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:16 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:15:16.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:16 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:15:16.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:16 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:15:16.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:16 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:16.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:16 smithi097 ceph-mon[35562]: purged_snaps scrub starts 2024-09-11T23:15:16.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:16 smithi097 ceph-mon[35562]: purged_snaps scrub ok 2024-09-11T23:15:16.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:16 smithi097 ceph-mon[35562]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T23:15:16.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:16 smithi097 ceph-mon[35562]: Detected new or changed devices on smithi097 2024-09-11T23:15:16.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:16 smithi097 ceph-mon[35562]: Adjusting osd_memory_target on smithi097 to 17088M 2024-09-11T23:15:16.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:16 smithi097 ceph-mon[35562]: osd.2 [v2:172.21.15.97:6800/1456846443,v1:172.21.15.97:6801/1456846443] boot 2024-09-11T23:15:16.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:16 smithi097 ceph-mon[35562]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T23:15:16.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:16 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:15:16.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:16 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:15:16.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:16 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:15:16.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:16 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:17.434 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:17 smithi043 ceph-mon[30950]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:15:17.434 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:17 smithi043 ceph-mon[30950]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T23:15:17.434 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:17 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:15:17.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:17 smithi090 ceph-mon[35130]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:15:17.647 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:17 smithi090 ceph-mon[35130]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T23:15:17.647 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:17 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:15:17.648 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:17 smithi097 ceph-mon[35562]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:15:17.648 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:17 smithi097 ceph-mon[35562]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T23:15:17.648 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:17 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:15:18.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:18 smithi043 ceph-mon[30950]: pgmap v77: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:15:18.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:18 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T23:15:18.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:18 smithi043 ceph-mon[30950]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T23:15:18.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:18 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:15:18.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:18 smithi090 ceph-mon[35130]: pgmap v77: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:15:18.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:18 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T23:15:18.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:18 smithi090 ceph-mon[35130]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T23:15:18.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:18 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:15:18.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:18 smithi097 ceph-mon[35562]: pgmap v77: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:15:18.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:18 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T23:15:18.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:18 smithi097 ceph-mon[35562]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T23:15:18.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:18 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:15:18.724 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:15:18 smithi043 sudo[49393]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:15:18.724 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:15:18 smithi043 sudo[49393]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:15:18.724 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:15:18 smithi043 sudo[49393]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:15:18.724 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:15:18 smithi043 sudo[49393]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:15:19.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:19.178+0000 7f9eddffb640 1 -- 172.21.15.43:0/2852107631 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9ed40982b0 con 0x7f9ee0105030 2024-09-11T23:15:19.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:19 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:15:19.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:19 smithi090 ceph-mon[35130]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T23:15:19.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:19 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2140831391' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3062133e-000b-4421-9d53-ac434b9ed9b8"}]: dispatch 2024-09-11T23:15:19.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:19 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2140831391' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3062133e-000b-4421-9d53-ac434b9ed9b8"}]': finished 2024-09-11T23:15:19.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:19 smithi090 ceph-mon[35130]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T23:15:19.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:19 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:15:19.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:19 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2904976759' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:15:19.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:19 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:15:19.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:19 smithi097 ceph-mon[35562]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T23:15:19.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:19 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2140831391' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3062133e-000b-4421-9d53-ac434b9ed9b8"}]: dispatch 2024-09-11T23:15:19.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:19 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2140831391' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3062133e-000b-4421-9d53-ac434b9ed9b8"}]': finished 2024-09-11T23:15:19.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:19 smithi097 ceph-mon[35562]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T23:15:19.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:19 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:15:19.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:19 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2904976759' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:15:19.610 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:15:19 smithi043 sudo[49442]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:15:19.610 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:15:19 smithi043 sudo[49442]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:15:19.610 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:15:19 smithi043 sudo[49442]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:15:19.610 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:15:19 smithi043 sudo[49442]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:15:19.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:19 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:15:19.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:19 smithi043 ceph-mon[30950]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T23:15:19.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:19 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2140831391' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3062133e-000b-4421-9d53-ac434b9ed9b8"}]: dispatch 2024-09-11T23:15:19.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:19 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2140831391' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3062133e-000b-4421-9d53-ac434b9ed9b8"}]': finished 2024-09-11T23:15:19.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:19 smithi043 ceph-mon[30950]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T23:15:19.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:19 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:15:19.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:19 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2904976759' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:15:19.686 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:15:19 smithi090 sudo[43542]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:15:19.686 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:15:19 smithi090 sudo[43542]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:15:19.686 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:15:19 smithi090 sudo[43542]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:15:19.687 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:15:19 smithi090 sudo[43542]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:15:20.020 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:15:19 smithi090 sudo[43545]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:15:20.020 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:15:19 smithi090 sudo[43545]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:15:20.020 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:15:19 smithi090 sudo[43545]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:15:20.020 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:15:19 smithi090 sudo[43545]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:15:20.210 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:15:19 smithi097 sudo[44166]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:15:20.210 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:15:19 smithi097 sudo[44166]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:15:20.210 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:15:19 smithi097 sudo[44166]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:15:20.211 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:15:19 smithi097 sudo[44166]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:15:20.211 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:20 smithi097 ceph-mon[35562]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:15:20.211 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:20 smithi097 ceph-mon[35562]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T23:15:20.211 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:20 smithi097 ceph-mon[35562]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-11T23:15:20.211 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:20 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:15:20.211 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:20 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:20.211 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:20 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:20 smithi090 ceph-mon[35130]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:15:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:20 smithi090 ceph-mon[35130]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T23:15:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:20 smithi090 ceph-mon[35130]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-11T23:15:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:20 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:15:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:20 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:20 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:20.551 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:15:20 smithi097 sudo[44169]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:15:20.551 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:15:20 smithi097 sudo[44169]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:15:20.551 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:15:20 smithi097 sudo[44169]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:15:20.551 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:15:20 smithi097 sudo[44169]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:15:20.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:20 smithi043 ceph-mon[30950]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:15:20.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:20 smithi043 ceph-mon[30950]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T23:15:20.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:20 smithi043 ceph-mon[30950]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-11T23:15:20.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:20 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:15:20.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:20 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:20.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:20 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:21.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:20 smithi043 sudo[49817]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:15:21.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:20 smithi043 sudo[49817]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:15:21.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:20 smithi043 sudo[49817]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:15:21.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:20 smithi043 sudo[49817]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:15:21.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:21 smithi043 sudo[50006]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:15:21.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:21 smithi043 sudo[50006]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:15:21.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:21 smithi043 sudo[50006]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:15:21.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:21 smithi043 sudo[50006]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:15:21.642 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:21 smithi097 sudo[44172]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:15:21.642 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:21 smithi097 sudo[44172]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:15:21.642 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:21 smithi097 sudo[44172]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:15:21.642 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:21 smithi097 sudo[44172]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:15:22.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:21 smithi097 sudo[44175]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:15:22.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:21 smithi097 sudo[44175]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:15:22.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:21 smithi097 sudo[44175]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:15:22.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:21 smithi097 sudo[44175]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:15:22.139 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:21 smithi090 sudo[43548]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T23:15:22.139 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:21 smithi090 sudo[43548]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:15:22.139 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:21 smithi090 sudo[43548]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:15:22.140 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:21 smithi090 sudo[43548]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:15:22.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:22 smithi090 sudo[43551]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T23:15:22.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:22 smithi090 sudo[43551]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T23:15:22.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:22 smithi090 sudo[43551]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T23:15:22.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:22 smithi090 sudo[43551]: pam_unix(sudo:session): session closed for user root 2024-09-11T23:15:22.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:22 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:22.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:22 smithi090 ceph-mon[35130]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:15:22.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:22 smithi090 ceph-mon[35130]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:15:22.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:22 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:15:22.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:22 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:15:22.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:22 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:15:22.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:22 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:15:22.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:22 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:15:22.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:22 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:15:22.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:22 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:22.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:22 smithi097 ceph-mon[35562]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:15:22.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:22 smithi097 ceph-mon[35562]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:15:22.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:22 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:15:22.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:22 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:15:22.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:22 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:15:22.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:22 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:15:22.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:22 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:15:22.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:22 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:15:22.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:22 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:22.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:22 smithi043 ceph-mon[30950]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:15:22.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:22 smithi043 ceph-mon[30950]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:15:22.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:22 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:15:22.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:22 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:15:22.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:22 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:15:22.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:22 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:15:22.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:22 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:15:22.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:22 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:15:23.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:23 smithi043 ceph-mon[30950]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:15:23.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:23 smithi043 ceph-mon[30950]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:15:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:23 smithi043 ceph-mon[30950]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:15:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:23 smithi043 ceph-mon[30950]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:15:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:15:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:15:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:15:23.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:23 smithi043 ceph-mon[30950]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:15:23.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:23 smithi090 ceph-mon[35130]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:15:23.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:23 smithi090 ceph-mon[35130]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:15:23.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:23 smithi090 ceph-mon[35130]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:15:23.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:23 smithi090 ceph-mon[35130]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:15:23.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:23 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:15:23.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:23 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:15:23.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:23 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:15:23.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:23 smithi090 ceph-mon[35130]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:15:23.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:23 smithi097 ceph-mon[35562]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:15:23.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:23 smithi097 ceph-mon[35562]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:15:23.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:23 smithi097 ceph-mon[35562]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:15:23.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:23 smithi097 ceph-mon[35562]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T23:15:23.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:23 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:15:23.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:23 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:15:23.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:23 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:15:23.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:23 smithi097 ceph-mon[35562]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T23:15:24.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:24 smithi097 ceph-mon[35562]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:15:24.472 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:24 smithi043 ceph-mon[30950]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:15:24.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:24 smithi090 ceph-mon[35130]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:15:26.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:26 smithi090 ceph-mon[35130]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:15:26.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:26 smithi097 ceph-mon[35562]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:15:26.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:26 smithi043 ceph-mon[30950]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:15:27.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:27 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T23:15:27.353 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:27 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:27.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:27 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T23:15:27.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:27 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:27.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:27 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T23:15:27.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:27 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:28.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:28 smithi043 ceph-mon[30950]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-11T23:15:28.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:28 smithi043 ceph-mon[30950]: Deploying daemon osd.3 on smithi043 2024-09-11T23:15:28.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:28 smithi090 ceph-mon[35130]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-11T23:15:28.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:28 smithi090 ceph-mon[35130]: Deploying daemon osd.3 on smithi043 2024-09-11T23:15:28.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:28 smithi097 ceph-mon[35562]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-11T23:15:28.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:28 smithi097 ceph-mon[35562]: Deploying daemon osd.3 on smithi043 2024-09-11T23:15:30.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:30 smithi043 ceph-mon[30950]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-11T23:15:30.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:30 smithi090 ceph-mon[35130]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-11T23:15:30.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:30 smithi097 ceph-mon[35562]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-11T23:15:32.355 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:32 smithi043 ceph-mon[30950]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-11T23:15:32.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:32 smithi090 ceph-mon[35130]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-11T23:15:32.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:32 smithi097 ceph-mon[35562]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-11T23:15:33.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:33 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:33.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:33 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:33.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:33 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:33.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:33 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:33.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:33 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:33.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:33 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:33.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:33 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:33.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:33 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:33.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:33 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:34.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:34 smithi043 ceph-mon[30950]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-11T23:15:34.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:34 smithi090 ceph-mon[35130]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-11T23:15:34.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:34 smithi097 ceph-mon[35562]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-11T23:15:36.359 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:36 smithi043 ceph-mon[30950]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T23:15:36.359 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:36 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:36.359 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:36 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:36.359 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:36 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:36.359 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:36 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:36.359 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:36 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:36.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:36 smithi090 ceph-mon[35130]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T23:15:36.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:36 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:36.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:36 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:36.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:36 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:36.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:36 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:36.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:36 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:36.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:36 smithi097 ceph-mon[35562]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T23:15:36.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:36 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:36.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:36 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:36.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:36 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:36.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:36 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:36.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:36 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:36.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:36.697+0000 7f9eddffb640 1 -- 172.21.15.43:0/2852107631 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9ee0062630 con 0x7f9ec80a2b20 2024-09-11T23:15:36.698 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 3 on host 'smithi043' 2024-09-11T23:15:36.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:36.700+0000 7f9ee6b3b640 1 -- 172.21.15.43:0/2852107631 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9ec80a2b20 msgr2=0x7f9ec80a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:36.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:36.700+0000 7f9ee6b3b640 1 --2- 172.21.15.43:0/2852107631 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9ec80a2b20 0x7f9ec80a4fe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009b60 tx=0x7f9ed0005f90 comp rx=0 tx=0).stop 2024-09-11T23:15:36.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:36.700+0000 7f9ee6b3b640 1 -- 172.21.15.43:0/2852107631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0105030 msgr2=0x7f9ee019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:36.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:36.700+0000 7f9ee6b3b640 1 --2- 172.21.15.43:0/2852107631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0105030 0x7f9ee019a5f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400e9c0 tx=0x7f9ed400ee80 comp rx=0 tx=0).stop 2024-09-11T23:15:36.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:36.700+0000 7f9ee6b3b640 1 -- 172.21.15.43:0/2852107631 shutdown_connections 2024-09-11T23:15:36.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:36.700+0000 7f9ee6b3b640 1 --2- 172.21.15.43:0/2852107631 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9ec80a2b20 0x7f9ec80a4fe0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:36.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:36.700+0000 7f9ee6b3b640 1 --2- 172.21.15.43:0/2852107631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ee0107960 0x7f9ee01a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:36.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:36.700+0000 7f9ee6b3b640 1 --2- 172.21.15.43:0/2852107631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0105030 0x7f9ee019a5f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:36.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:36.700+0000 7f9ee6b3b640 1 --2- 172.21.15.43:0/2852107631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ee0102700 0x7f9ee019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:36.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:36.700+0000 7f9ee6b3b640 1 -- 172.21.15.43:0/2852107631 >> 172.21.15.43:0/2852107631 conn(0x7f9ee00fc280 msgr2=0x7f9ee0101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:36.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:36.700+0000 7f9ee6b3b640 1 -- 172.21.15.43:0/2852107631 shutdown_connections 2024-09-11T23:15:36.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:15:36.700+0000 7f9ee6b3b640 1 -- 172.21.15.43:0/2852107631 wait complete. 2024-09-11T23:15:37.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:37 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:37.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:37 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:37.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:37 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:37.342 DEBUG:teuthology.orchestra.run.smithi043:osd.3> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.3.service 2024-09-11T23:15:37.344 INFO:tasks.cephadm:Deploying osd.4 on smithi090 with /dev/vg_nvme/lv_3... 2024-09-11T23:15:37.344 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T23:15:37.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:37 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:37.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:37 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:37.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:37 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:37.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:37 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:37.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:37 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:37.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:37 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:37.600 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:15:38.421 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:38 smithi043 ceph-mon[30950]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T23:15:38.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:38 smithi090 ceph-mon[35130]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T23:15:38.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:38 smithi097 ceph-mon[35562]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T23:15:39.266 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:39 smithi090 ceph-mon[35130]: from='osd.3 [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T23:15:39.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:39 smithi043 ceph-mon[30950]: from='osd.3 [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T23:15:39.487 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 23:15:39 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-3[51410]: 2024-09-11T23:15:39.114+0000 7f8e92a9b640 -1 osd.3 0 log_to_monitors true 2024-09-11T23:15:39.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:39 smithi097 ceph-mon[35562]: from='osd.3 [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T23:15:40.151 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-11T23:15:40.177 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_3 2024-09-11T23:15:40.434 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:15:40.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:40 smithi090 ceph-mon[35130]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:15:40.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:40 smithi090 ceph-mon[35130]: from='osd.3 [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T23:15:40.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:40 smithi090 ceph-mon[35130]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T23:15:40.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:40 smithi090 ceph-mon[35130]: from='osd.3 [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T23:15:40.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:40 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:15:40.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:40 smithi097 ceph-mon[35562]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:15:40.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:40 smithi097 ceph-mon[35562]: from='osd.3 [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T23:15:40.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:40 smithi097 ceph-mon[35562]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T23:15:40.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:40 smithi097 ceph-mon[35562]: from='osd.3 [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T23:15:40.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:40 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:15:40.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:40 smithi043 ceph-mon[30950]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:15:40.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:40 smithi043 ceph-mon[30950]: from='osd.3 [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T23:15:40.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:40 smithi043 ceph-mon[30950]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T23:15:40.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:40 smithi043 ceph-mon[30950]: from='osd.3 [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T23:15:40.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:40 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:15:40.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.926+0000 7f0c08f44640 1 -- 172.21.15.90:0/1009413662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c041037c0 msgr2=0x7f0c0410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:40.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.926+0000 7f0c08f44640 1 --2- 172.21.15.90:0/1009413662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c041037c0 0x7f0c0410a050 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009ca0 tx=0x7f0bfc02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:15:40.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.926+0000 7f0c08f44640 1 -- 172.21.15.90:0/1009413662 shutdown_connections 2024-09-11T23:15:40.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.926+0000 7f0c08f44640 1 --2- 172.21.15.90:0/1009413662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c041037c0 0x7f0c0410a050 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:40.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.926+0000 7f0c08f44640 1 --2- 172.21.15.90:0/1009413662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c04102e00 0x7f0c04103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:40.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.926+0000 7f0c08f44640 1 --2- 172.21.15.90:0/1009413662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c04101c00 0x7f0c04102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:40.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.926+0000 7f0c08f44640 1 -- 172.21.15.90:0/1009413662 >> 172.21.15.90:0/1009413662 conn(0x7f0c040fd390 msgr2=0x7f0c040ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:15:40.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.926+0000 7f0c08f44640 1 -- 172.21.15.90:0/1009413662 shutdown_connections 2024-09-11T23:15:40.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.926+0000 7f0c08f44640 1 -- 172.21.15.90:0/1009413662 wait complete. 2024-09-11T23:15:40.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.927+0000 7f0c08f44640 1 Processor -- start 2024-09-11T23:15:40.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.927+0000 7f0c08f44640 1 -- start start 2024-09-11T23:15:40.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.927+0000 7f0c08f44640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c04101c00 0x7f0c04197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:40.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.927+0000 7f0c08f44640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c04102e00 0x7f0c041983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:40.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.927+0000 7f0c08f44640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c041037c0 0x7f0c0419f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:40.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.927+0000 7f0c08f44640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0410d880 con 0x7f0c04102e00 2024-09-11T23:15:40.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.927+0000 7f0c08f44640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0410d700 con 0x7f0c04101c00 2024-09-11T23:15:40.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.927+0000 7f0c08f44640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0410da00 con 0x7f0c041037c0 2024-09-11T23:15:40.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.927+0000 7f0c0259c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c04101c00 0x7f0c04197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:40.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.927+0000 7f0c0259c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c04101c00 0x7f0c04197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38296/0 (socket says 172.21.15.90:38296) 2024-09-11T23:15:40.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.927+0000 7f0c0259c640 1 -- 172.21.15.90:0/2081982260 learned_addr learned my addr 172.21.15.90:0/2081982260 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-11T23:15:40.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.928+0000 7f0c02d9d640 1 --2- 172.21.15.90:0/2081982260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c041037c0 0x7f0c0419f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:40.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.928+0000 7f0c01d9b640 1 --2- 172.21.15.90:0/2081982260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c04102e00 0x7f0c041983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:40.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.928+0000 7f0c0259c640 1 -- 172.21.15.90:0/2081982260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c041037c0 msgr2=0x7f0c0419f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:40.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.928+0000 7f0c0259c640 1 --2- 172.21.15.90:0/2081982260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c041037c0 0x7f0c0419f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:40.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.928+0000 7f0c0259c640 1 -- 172.21.15.90:0/2081982260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c04102e00 msgr2=0x7f0c041983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:15:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.928+0000 7f0c0259c640 1 --2- 172.21.15.90:0/2081982260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c04102e00 0x7f0c041983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:15:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.928+0000 7f0c0259c640 1 -- 172.21.15.90:0/2081982260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0419fad0 con 0x7f0c04101c00 2024-09-11T23:15:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.928+0000 7f0c0259c640 1 --2- 172.21.15.90:0/2081982260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c04101c00 0x7f0c04197ea0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800d950 tx=0x7f0bf800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.928+0000 7f0bf37fe640 1 -- 172.21.15.90:0/2081982260 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8017070 con 0x7f0c04101c00 2024-09-11T23:15:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.928+0000 7f0c08f44640 1 -- 172.21.15.90:0/2081982260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c0419fdc0 con 0x7f0c04101c00 2024-09-11T23:15:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.928+0000 7f0bf37fe640 1 -- 172.21.15.90:0/2081982260 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bf8009c30 con 0x7f0c04101c00 2024-09-11T23:15:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.928+0000 7f0bf37fe640 1 -- 172.21.15.90:0/2081982260 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf800f600 con 0x7f0c04101c00 2024-09-11T23:15:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.928+0000 7f0c08f44640 1 -- 172.21.15.90:0/2081982260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c041a0220 con 0x7f0c04101c00 2024-09-11T23:15:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.929+0000 7f0c08f44640 1 -- 172.21.15.90:0/2081982260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c04109000 con 0x7f0c04101c00 2024-09-11T23:15:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.930+0000 7f0bf37fe640 1 -- 172.21.15.90:0/2081982260 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0bf801a070 con 0x7f0c04101c00 2024-09-11T23:15:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.931+0000 7f0bf37fe640 1 --2- 172.21.15.90:0/2081982260 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0bec0a2bb0 0x7f0bec0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:15:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.931+0000 7f0bf37fe640 1 -- 172.21.15.90:0/2081982260 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3389+0+0 (secure 0 0 0) 0x7f0bf80cd810 con 0x7f0c04101c00 2024-09-11T23:15:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.931+0000 7f0c01d9b640 1 --2- 172.21.15.90:0/2081982260 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0bec0a2bb0 0x7f0bec0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:15:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.931+0000 7f0c01d9b640 1 --2- 172.21.15.90:0/2081982260 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0bec0a2bb0 0x7f0bec0a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0c041993c0 tx=0x7f0bf4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:15:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:40.936+0000 7f0bf37fe640 1 -- 172.21.15.90:0/2081982260 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0bf8009de0 con 0x7f0c04101c00 2024-09-11T23:15:41.110 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 23:15:40 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-3[51410]: 2024-09-11T23:15:40.617+0000 7f8e8ea1e640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T23:15:41.111 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 23:15:40 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-3[51410]: 2024-09-11T23:15:40.624+0000 7f8e8683e640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:15:41.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:15:41.109+0000 7f0c08f44640 1 -- 172.21.15.90:0/2081982260 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0c04062630 con 0x7f0bec0a2bb0 2024-09-11T23:15:41.505 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:41 smithi090 ceph-mon[35130]: Detected new or changed devices on smithi043 2024-09-11T23:15:41.505 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:41.505 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:41 smithi090 ceph-mon[35130]: from='osd.3 [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T23:15:41.505 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:41 smithi090 ceph-mon[35130]: osdmap e25: 4 total, 3 up, 4 in 2024-09-11T23:15:41.506 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:15:41.506 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:41.506 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:15:41.506 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:41.506 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:41.506 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:41 smithi090 ceph-mon[35130]: Adjusting osd_memory_target on smithi043 to 8546M 2024-09-11T23:15:41.506 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:41.506 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:41.506 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:41.506 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:41.506 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:41 smithi090 ceph-mon[35130]: from='osd.3 [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804]' entity='osd.3' 2024-09-11T23:15:41.506 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:15:41.506 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:15:41.507 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:41.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:41 smithi097 ceph-mon[35562]: Detected new or changed devices on smithi043 2024-09-11T23:15:41.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:41.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:41 smithi097 ceph-mon[35562]: from='osd.3 [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T23:15:41.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:41 smithi097 ceph-mon[35562]: osdmap e25: 4 total, 3 up, 4 in 2024-09-11T23:15:41.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:15:41.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:41.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:15:41.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:41.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:41.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:41 smithi097 ceph-mon[35562]: Adjusting osd_memory_target on smithi043 to 8546M 2024-09-11T23:15:41.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:41.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:41.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:41.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:41.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:41 smithi097 ceph-mon[35562]: from='osd.3 [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804]' entity='osd.3' 2024-09-11T23:15:41.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:15:41.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:15:41.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:41.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:41 smithi043 ceph-mon[30950]: Detected new or changed devices on smithi043 2024-09-11T23:15:41.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:41.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:41 smithi043 ceph-mon[30950]: from='osd.3 [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T23:15:41.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:41 smithi043 ceph-mon[30950]: osdmap e25: 4 total, 3 up, 4 in 2024-09-11T23:15:41.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:15:41.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:41.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:15:41.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:41.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:15:41.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:41 smithi043 ceph-mon[30950]: Adjusting osd_memory_target on smithi043 to 8546M 2024-09-11T23:15:41.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:41.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:41.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:15:41.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:41.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:41 smithi043 ceph-mon[30950]: from='osd.3 [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804]' entity='osd.3' 2024-09-11T23:15:41.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:15:41.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:15:41.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:42.453 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:42 smithi097 ceph-mon[35562]: purged_snaps scrub starts 2024-09-11T23:15:42.453 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:42 smithi097 ceph-mon[35562]: purged_snaps scrub ok 2024-09-11T23:15:42.453 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:42 smithi097 ceph-mon[35562]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:15:42.453 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:42 smithi097 ceph-mon[35562]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:15:42.454 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:42 smithi097 ceph-mon[35562]: osd.3 [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804] boot 2024-09-11T23:15:42.454 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:42 smithi097 ceph-mon[35562]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T23:15:42.454 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:42 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:15:42.487 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:42 smithi090 ceph-mon[35130]: purged_snaps scrub starts 2024-09-11T23:15:42.487 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:42 smithi090 ceph-mon[35130]: purged_snaps scrub ok 2024-09-11T23:15:42.487 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:42 smithi090 ceph-mon[35130]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:15:42.487 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:42 smithi090 ceph-mon[35130]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:15:42.487 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:42 smithi090 ceph-mon[35130]: osd.3 [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804] boot 2024-09-11T23:15:42.487 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:42 smithi090 ceph-mon[35130]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T23:15:42.487 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:42 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:15:42.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:42 smithi043 ceph-mon[30950]: purged_snaps scrub starts 2024-09-11T23:15:42.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:42 smithi043 ceph-mon[30950]: purged_snaps scrub ok 2024-09-11T23:15:42.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:42 smithi043 ceph-mon[30950]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-11T23:15:42.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:42 smithi043 ceph-mon[30950]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:15:42.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:42 smithi043 ceph-mon[30950]: osd.3 [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804] boot 2024-09-11T23:15:42.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:42 smithi043 ceph-mon[30950]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T23:15:42.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:42 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:15:43.473 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:43 smithi097 ceph-mon[35562]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T23:15:43.474 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:43 smithi097 ceph-mon[35562]: osdmap e28: 4 total, 4 up, 4 in 2024-09-11T23:15:43.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:43 smithi090 ceph-mon[35130]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T23:15:43.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:43 smithi090 ceph-mon[35130]: osdmap e28: 4 total, 4 up, 4 in 2024-09-11T23:15:43.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:43 smithi043 ceph-mon[30950]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T23:15:43.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:43 smithi043 ceph-mon[30950]: osdmap e28: 4 total, 4 up, 4 in 2024-09-11T23:15:44.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:44 smithi043 ceph-mon[30950]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:15:44.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:44 smithi043 ceph-mon[30950]: osdmap e29: 4 total, 4 up, 4 in 2024-09-11T23:15:44.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:44 smithi043 ceph-mon[30950]: from='client.? 172.21.15.90:0/2674436034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e73fa4b-f22c-4319-828d-87b91b9f19f7"}]: dispatch 2024-09-11T23:15:44.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:44 smithi043 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e73fa4b-f22c-4319-828d-87b91b9f19f7"}]: dispatch 2024-09-11T23:15:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:44 smithi043 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2e73fa4b-f22c-4319-828d-87b91b9f19f7"}]': finished 2024-09-11T23:15:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:44 smithi043 ceph-mon[30950]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T23:15:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:44 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:15:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:44 smithi043 ceph-mon[30950]: from='client.? 172.21.15.90:0/4218832702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:15:44.683 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:44 smithi097 ceph-mon[35562]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:15:44.683 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:44 smithi097 ceph-mon[35562]: osdmap e29: 4 total, 4 up, 4 in 2024-09-11T23:15:44.683 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:44 smithi097 ceph-mon[35562]: from='client.? 172.21.15.90:0/2674436034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e73fa4b-f22c-4319-828d-87b91b9f19f7"}]: dispatch 2024-09-11T23:15:44.684 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:44 smithi097 ceph-mon[35562]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e73fa4b-f22c-4319-828d-87b91b9f19f7"}]: dispatch 2024-09-11T23:15:44.684 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:44 smithi097 ceph-mon[35562]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2e73fa4b-f22c-4319-828d-87b91b9f19f7"}]': finished 2024-09-11T23:15:44.684 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:44 smithi097 ceph-mon[35562]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T23:15:44.684 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:44 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:15:44.684 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:44 smithi097 ceph-mon[35562]: from='client.? 172.21.15.90:0/4218832702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:15:44.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:44 smithi090 ceph-mon[35130]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:15:44.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:44 smithi090 ceph-mon[35130]: osdmap e29: 4 total, 4 up, 4 in 2024-09-11T23:15:44.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:44 smithi090 ceph-mon[35130]: from='client.? 172.21.15.90:0/2674436034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e73fa4b-f22c-4319-828d-87b91b9f19f7"}]: dispatch 2024-09-11T23:15:44.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:44 smithi090 ceph-mon[35130]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e73fa4b-f22c-4319-828d-87b91b9f19f7"}]: dispatch 2024-09-11T23:15:44.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:44 smithi090 ceph-mon[35130]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2e73fa4b-f22c-4319-828d-87b91b9f19f7"}]': finished 2024-09-11T23:15:44.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:44 smithi090 ceph-mon[35130]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T23:15:44.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:44 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:15:44.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:44 smithi090 ceph-mon[35130]: from='client.? 172.21.15.90:0/4218832702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:15:46.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:46 smithi043 ceph-mon[30950]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:15:46.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:46 smithi090 ceph-mon[35130]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:15:46.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:46 smithi097 ceph-mon[35562]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:15:48.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:48 smithi043 ceph-mon[30950]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-09-11T23:15:48.638 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:48 smithi090 ceph-mon[35130]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-09-11T23:15:48.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:48 smithi097 ceph-mon[35562]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-09-11T23:15:50.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:50 smithi043 ceph-mon[30950]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T23:15:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:50 smithi090 ceph-mon[35130]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T23:15:50.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:50 smithi097 ceph-mon[35562]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T23:15:52.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:52 smithi090 ceph-mon[35130]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-11T23:15:52.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:52 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T23:15:52.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:52 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:52.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:52 smithi043 ceph-mon[30950]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-11T23:15:52.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:52 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T23:15:52.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:52 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:52.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:52 smithi097 ceph-mon[35562]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-11T23:15:52.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:52 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T23:15:52.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:52 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:15:53.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:53 smithi043 ceph-mon[30950]: Deploying daemon osd.4 on smithi090 2024-09-11T23:15:53.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:53 smithi090 ceph-mon[35130]: Deploying daemon osd.4 on smithi090 2024-09-11T23:15:53.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:53 smithi097 ceph-mon[35562]: Deploying daemon osd.4 on smithi090 2024-09-11T23:15:54.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:54 smithi097 ceph-mon[35562]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T23:15:54.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:54 smithi043 ceph-mon[30950]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T23:15:54.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:54 smithi090 ceph-mon[35130]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T23:15:56.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:56 smithi090 ceph-mon[35130]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-11T23:15:56.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:56 smithi097 ceph-mon[35562]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-11T23:15:56.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:56 smithi043 ceph-mon[30950]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-11T23:15:58.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:58 smithi090 ceph-mon[35130]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:15:58.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:58 smithi097 ceph-mon[35562]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:15:58.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:58 smithi043 ceph-mon[30950]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T23:15:59.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:59 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:59.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:59 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:59.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:15:59 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:59.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:59 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:59.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:59 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:59.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:15:59 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:59.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:59 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:15:59.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:59 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:15:59.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:15:59 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:00.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:00 smithi090 ceph-mon[35130]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:16:00.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:00 smithi097 ceph-mon[35562]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:16:00.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:00 smithi043 ceph-mon[30950]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:16:02.339 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:02 smithi090 ceph-mon[35130]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:16:02.339 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:02 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:02.339 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:02 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:02.339 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:02 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:02.339 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:02 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:02.339 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:02 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:02.380 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 4 on host 'smithi090' 2024-09-11T23:16:02.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:02.380+0000 7f0bf37fe640 1 -- 172.21.15.90:0/2081982260 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0c04062630 con 0x7f0bec0a2bb0 2024-09-11T23:16:02.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:02.382+0000 7f0c08f44640 1 -- 172.21.15.90:0/2081982260 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0bec0a2bb0 msgr2=0x7f0bec0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:02.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:02.382+0000 7f0c08f44640 1 --2- 172.21.15.90:0/2081982260 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0bec0a2bb0 0x7f0bec0a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0c041993c0 tx=0x7f0bf4005f90 comp rx=0 tx=0).stop 2024-09-11T23:16:02.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:02.382+0000 7f0c08f44640 1 -- 172.21.15.90:0/2081982260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c04101c00 msgr2=0x7f0c04197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:02.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:02.382+0000 7f0c08f44640 1 --2- 172.21.15.90:0/2081982260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c04101c00 0x7f0c04197ea0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800d950 tx=0x7f0bf800de10 comp rx=0 tx=0).stop 2024-09-11T23:16:02.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:02.382+0000 7f0c08f44640 1 -- 172.21.15.90:0/2081982260 shutdown_connections 2024-09-11T23:16:02.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:02.382+0000 7f0c08f44640 1 --2- 172.21.15.90:0/2081982260 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0bec0a2bb0 0x7f0bec0a5070 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:02.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:02.382+0000 7f0c08f44640 1 --2- 172.21.15.90:0/2081982260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c041037c0 0x7f0c0419f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:02.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:02.382+0000 7f0c08f44640 1 --2- 172.21.15.90:0/2081982260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c04102e00 0x7f0c041983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:02.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:02.382+0000 7f0c08f44640 1 --2- 172.21.15.90:0/2081982260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c04101c00 0x7f0c04197ea0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:02.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:02.382+0000 7f0c08f44640 1 -- 172.21.15.90:0/2081982260 >> 172.21.15.90:0/2081982260 conn(0x7f0c040fd390 msgr2=0x7f0c041061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:02.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:02.382+0000 7f0c08f44640 1 -- 172.21.15.90:0/2081982260 shutdown_connections 2024-09-11T23:16:02.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:02.382+0000 7f0c08f44640 1 -- 172.21.15.90:0/2081982260 wait complete. 2024-09-11T23:16:02.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:02 smithi097 ceph-mon[35562]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:16:02.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:02 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:02.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:02 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:02.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:02 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:02.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:02 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:02.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:02 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:02.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:02 smithi043 ceph-mon[30950]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:16:02.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:02.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:02.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:02.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:02.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:03.072 DEBUG:teuthology.orchestra.run.smithi090:osd.4> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.4.service 2024-09-11T23:16:03.075 INFO:tasks.cephadm:Deploying osd.5 on smithi097 with /dev/vg_nvme/lv_3... 2024-09-11T23:16:03.075 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T23:16:03.331 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:16:03.354 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:03 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:16:03.354 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:03 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:03.354 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:03 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:03.452 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:03 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:16:03.452 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:03 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:03.452 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:03 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:03.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:16:03.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:03.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:04.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:04 smithi090 ceph-mon[35130]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:16:04.520 INFO:journalctl@ceph.osd.4.smithi090.stdout:Sep 11 23:16:04 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-4[46309]: 2024-09-11T23:16:04.476+0000 7f6da5891640 -1 osd.4 0 log_to_monitors true 2024-09-11T23:16:04.697 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:04 smithi097 ceph-mon[35562]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:16:04.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:04 smithi043 ceph-mon[30950]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:16:05.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:05 smithi090 ceph-mon[35130]: from='osd.4 [v2:172.21.15.90:6808/181614173,v1:172.21.15.90:6809/181614173]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:16:05.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:05 smithi090 ceph-mon[35130]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:16:05.692 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:05 smithi097 ceph-mon[35562]: from='osd.4 [v2:172.21.15.90:6808/181614173,v1:172.21.15.90:6809/181614173]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:16:05.692 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:05 smithi097 ceph-mon[35562]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:16:05.839 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:16:05.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:05 smithi043 ceph-mon[30950]: from='osd.4 [v2:172.21.15.90:6808/181614173,v1:172.21.15.90:6809/181614173]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:16:05.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:05 smithi043 ceph-mon[30950]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T23:16:05.865 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_3 2024-09-11T23:16:06.117 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:16:06.506 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:06 smithi097 ceph-mon[35562]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:16:06.506 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:06 smithi097 ceph-mon[35562]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T23:16:06.506 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:06 smithi097 ceph-mon[35562]: osdmap e31: 5 total, 4 up, 5 in 2024-09-11T23:16:06.507 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:06 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:16:06.507 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:06 smithi097 ceph-mon[35562]: from='osd.4 [v2:172.21.15.90:6808/181614173,v1:172.21.15.90:6809/181614173]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:16:06.507 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:06 smithi097 ceph-mon[35562]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:16:06.507 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:06 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:06.507 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:06 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:06.507 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:06 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:16:06.507 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:06 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:16:06.507 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:06 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:06.507 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:06 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:06.507 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:06 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:06.507 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:06 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:06.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.589+0000 7f29cc9f4640 1 -- 172.21.15.97:0/3263653627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c8104fd0 msgr2=0x7f29c81073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:06.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.589+0000 7f29cc9f4640 1 --2- 172.21.15.97:0/3263653627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c8104fd0 0x7f29c81073c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f29b8009900 tx=0x7f29b802a410 comp rx=0 tx=0).stop 2024-09-11T23:16:06.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.589+0000 7f29cc9f4640 1 -- 172.21.15.97:0/3263653627 shutdown_connections 2024-09-11T23:16:06.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.589+0000 7f29cc9f4640 1 --2- 172.21.15.97:0/3263653627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c8107900 0x7f29c8109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:06.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.589+0000 7f29cc9f4640 1 --2- 172.21.15.97:0/3263653627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c8104fd0 0x7f29c81073c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:06.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.589+0000 7f29cc9f4640 1 --2- 172.21.15.97:0/3263653627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29c81026a0 0x7f29c8104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:06.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.589+0000 7f29cc9f4640 1 -- 172.21.15.97:0/3263653627 >> 172.21.15.97:0/3263653627 conn(0x7f29c80fc2a0 msgr2=0x7f29c80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:06.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.589+0000 7f29cc9f4640 1 -- 172.21.15.97:0/3263653627 shutdown_connections 2024-09-11T23:16:06.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.589+0000 7f29cc9f4640 1 -- 172.21.15.97:0/3263653627 wait complete. 2024-09-11T23:16:06.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.590+0000 7f29cc9f4640 1 Processor -- start 2024-09-11T23:16:06.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.590+0000 7f29cc9f4640 1 -- start start 2024-09-11T23:16:06.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.590+0000 7f29cc9f4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c81026a0 0x7f29c8197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:06.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.590+0000 7f29cc9f4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29c8104fd0 0x7f29c8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:06.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.590+0000 7f29cc9f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c8107900 0x7f29c819f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:06.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.590+0000 7f29cc9f4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f29c810d8a0 con 0x7f29c8107900 2024-09-11T23:16:06.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.590+0000 7f29cc9f4640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f29c810d720 con 0x7f29c8104fd0 2024-09-11T23:16:06.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.590+0000 7f29cc9f4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f29c810da20 con 0x7f29c81026a0 2024-09-11T23:16:06.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.590+0000 7f29c659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c81026a0 0x7f29c8197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:06.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.590+0000 7f29c659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c81026a0 0x7f29c8197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45830/0 (socket says 172.21.15.97:45830) 2024-09-11T23:16:06.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.590+0000 7f29c659c640 1 -- 172.21.15.97:0/2067755593 learned_addr learned my addr 172.21.15.97:0/2067755593 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-11T23:16:06.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.590+0000 7f29c5d9b640 1 --2- 172.21.15.97:0/2067755593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29c8104fd0 0x7f29c8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:06.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.590+0000 7f29c6d9d640 1 --2- 172.21.15.97:0/2067755593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c8107900 0x7f29c819f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:06.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.591+0000 7f29c659c640 1 -- 172.21.15.97:0/2067755593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29c8104fd0 msgr2=0x7f29c8198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:06.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.591+0000 7f29c659c640 1 --2- 172.21.15.97:0/2067755593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29c8104fd0 0x7f29c8198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:06.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.591+0000 7f29c659c640 1 -- 172.21.15.97:0/2067755593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c8107900 msgr2=0x7f29c819f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:06.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.591+0000 7f29c659c640 1 --2- 172.21.15.97:0/2067755593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c8107900 0x7f29c819f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:06.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.591+0000 7f29c659c640 1 -- 172.21.15.97:0/2067755593 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c819fb80 con 0x7f29c81026a0 2024-09-11T23:16:06.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.591+0000 7f29c6d9d640 1 --2- 172.21.15.97:0/2067755593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c8107900 0x7f29c819f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:16:06.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.591+0000 7f29c659c640 1 --2- 172.21.15.97:0/2067755593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c81026a0 0x7f29c8197f50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f29b000e9f0 tx=0x7f29b000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:06.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.591+0000 7f29bf7fe640 1 -- 172.21.15.97:0/2067755593 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b000cd80 con 0x7f29c81026a0 2024-09-11T23:16:06.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.591+0000 7f29cc9f4640 1 -- 172.21.15.97:0/2067755593 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29c819fe70 con 0x7f29c81026a0 2024-09-11T23:16:06.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.591+0000 7f29bf7fe640 1 -- 172.21.15.97:0/2067755593 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29b0005500 con 0x7f29c81026a0 2024-09-11T23:16:06.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.591+0000 7f29bf7fe640 1 -- 172.21.15.97:0/2067755593 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b0010690 con 0x7f29c81026a0 2024-09-11T23:16:06.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.591+0000 7f29cc9f4640 1 -- 172.21.15.97:0/2067755593 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29c81a0270 con 0x7f29c81026a0 2024-09-11T23:16:06.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.592+0000 7f29cc9f4640 1 -- 172.21.15.97:0/2067755593 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29c80fdd90 con 0x7f29c81026a0 2024-09-11T23:16:06.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.593+0000 7f29bf7fe640 1 -- 172.21.15.97:0/2067755593 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f29b00046c0 con 0x7f29c81026a0 2024-09-11T23:16:06.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.594+0000 7f29bf7fe640 1 --2- 172.21.15.97:0/2067755593 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f29b40a2bb0 0x7f29b40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:06.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.594+0000 7f29bf7fe640 1 -- 172.21.15.97:0/2067755593 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 3821+0+0 (secure 0 0 0) 0x7f29b00ce3b0 con 0x7f29c81026a0 2024-09-11T23:16:06.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.594+0000 7f29c5d9b640 1 --2- 172.21.15.97:0/2067755593 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f29b40a2bb0 0x7f29b40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:06.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.595+0000 7f29c5d9b640 1 --2- 172.21.15.97:0/2067755593 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f29b40a2bb0 0x7f29b40a5070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f29b8005ab0 tx=0x7f29b8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:06.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.598+0000 7f29bf7fe640 1 -- 172.21.15.97:0/2067755593 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f29b009c400 con 0x7f29c81026a0 2024-09-11T23:16:06.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:06 smithi090 ceph-mon[35130]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:16:06.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:06 smithi090 ceph-mon[35130]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T23:16:06.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:06 smithi090 ceph-mon[35130]: osdmap e31: 5 total, 4 up, 5 in 2024-09-11T23:16:06.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:06 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:16:06.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:06 smithi090 ceph-mon[35130]: from='osd.4 [v2:172.21.15.90:6808/181614173,v1:172.21.15.90:6809/181614173]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:16:06.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:06 smithi090 ceph-mon[35130]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:16:06.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:06 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:06.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:06 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:06.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:06 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:16:06.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:06 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:16:06.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:06 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:06.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:06 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:06.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:06 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:06.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:06 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:06.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:06.774+0000 7f29cc9f4640 1 -- 172.21.15.97:0/2067755593 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f29c8062630 con 0x7f29b40a2bb0 2024-09-11T23:16:06.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:06 smithi043 ceph-mon[30950]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:16:06.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:06 smithi043 ceph-mon[30950]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T23:16:06.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:06 smithi043 ceph-mon[30950]: osdmap e31: 5 total, 4 up, 5 in 2024-09-11T23:16:06.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:06 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:16:06.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:06 smithi043 ceph-mon[30950]: from='osd.4 [v2:172.21.15.90:6808/181614173,v1:172.21.15.90:6809/181614173]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:16:06.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:06 smithi043 ceph-mon[30950]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:16:06.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:06 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:06.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:06 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:06.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:06 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:16:06.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:06 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:16:06.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:06 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:06.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:06 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:06.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:06 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:06.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:06 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:07.270 INFO:journalctl@ceph.osd.4.smithi090.stdout:Sep 11 23:16:06 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-4[46309]: 2024-09-11T23:16:06.770+0000 7f6da1814640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T23:16:07.270 INFO:journalctl@ceph.osd.4.smithi090.stdout:Sep 11 23:16:06 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-4[46309]: 2024-09-11T23:16:06.776+0000 7f6d99634640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:16:07.500 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:07 smithi097 ceph-mon[35562]: Detected new or changed devices on smithi090 2024-09-11T23:16:07.501 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:07 smithi097 ceph-mon[35562]: Adjusting osd_memory_target on smithi090 to 8544M 2024-09-11T23:16:07.501 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:07 smithi097 ceph-mon[35562]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-11T23:16:07.501 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:07 smithi097 ceph-mon[35562]: osdmap e32: 5 total, 4 up, 5 in 2024-09-11T23:16:07.501 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:16:07.501 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:16:07.501 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:07 smithi097 ceph-mon[35562]: from='osd.4 ' entity='osd.4' 2024-09-11T23:16:07.501 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:16:07.501 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:16:07.501 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:07.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:07 smithi090 ceph-mon[35130]: Detected new or changed devices on smithi090 2024-09-11T23:16:07.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:07 smithi090 ceph-mon[35130]: Adjusting osd_memory_target on smithi090 to 8544M 2024-09-11T23:16:07.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:07 smithi090 ceph-mon[35130]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-11T23:16:07.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:07 smithi090 ceph-mon[35130]: osdmap e32: 5 total, 4 up, 5 in 2024-09-11T23:16:07.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:16:07.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:16:07.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:07 smithi090 ceph-mon[35130]: from='osd.4 ' entity='osd.4' 2024-09-11T23:16:07.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:16:07.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:16:07.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:07 smithi043 ceph-mon[30950]: Detected new or changed devices on smithi090 2024-09-11T23:16:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:07 smithi043 ceph-mon[30950]: Adjusting osd_memory_target on smithi090 to 8544M 2024-09-11T23:16:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:07 smithi043 ceph-mon[30950]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-11T23:16:07.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:07 smithi043 ceph-mon[30950]: osdmap e32: 5 total, 4 up, 5 in 2024-09-11T23:16:07.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:16:07.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:16:07.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:07 smithi043 ceph-mon[30950]: from='osd.4 ' entity='osd.4' 2024-09-11T23:16:07.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:16:07.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:16:07.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:08.465 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:08 smithi097 ceph-mon[35562]: purged_snaps scrub starts 2024-09-11T23:16:08.465 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:08 smithi097 ceph-mon[35562]: purged_snaps scrub ok 2024-09-11T23:16:08.465 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:08 smithi097 ceph-mon[35562]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:16:08.465 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:08 smithi097 ceph-mon[35562]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:08.466 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:08 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:16:08.466 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:08 smithi097 ceph-mon[35562]: osd.4 [v2:172.21.15.90:6808/181614173,v1:172.21.15.90:6809/181614173] boot 2024-09-11T23:16:08.466 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:08 smithi097 ceph-mon[35562]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T23:16:08.466 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:08 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:16:08.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:08 smithi090 ceph-mon[35130]: purged_snaps scrub starts 2024-09-11T23:16:08.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:08 smithi090 ceph-mon[35130]: purged_snaps scrub ok 2024-09-11T23:16:08.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:08 smithi090 ceph-mon[35130]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:16:08.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:08 smithi090 ceph-mon[35130]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:08.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:08 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:16:08.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:08 smithi090 ceph-mon[35130]: osd.4 [v2:172.21.15.90:6808/181614173,v1:172.21.15.90:6809/181614173] boot 2024-09-11T23:16:08.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:08 smithi090 ceph-mon[35130]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T23:16:08.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:08 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:16:08.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:08 smithi043 ceph-mon[30950]: purged_snaps scrub starts 2024-09-11T23:16:08.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:08 smithi043 ceph-mon[30950]: purged_snaps scrub ok 2024-09-11T23:16:08.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:08 smithi043 ceph-mon[30950]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T23:16:08.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:08 smithi043 ceph-mon[30950]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:08.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:08 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:16:08.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:08 smithi043 ceph-mon[30950]: osd.4 [v2:172.21.15.90:6808/181614173,v1:172.21.15.90:6809/181614173] boot 2024-09-11T23:16:08.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:08 smithi043 ceph-mon[30950]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T23:16:08.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:08 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:16:09.745 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:09 smithi097 ceph-mon[35562]: from='client.? 172.21.15.97:0/102482880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e1ad4f9-e724-440f-b515-3ec1ad97d6e4"}]: dispatch 2024-09-11T23:16:09.745 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:09 smithi097 ceph-mon[35562]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e1ad4f9-e724-440f-b515-3ec1ad97d6e4"}]: dispatch 2024-09-11T23:16:09.745 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:09 smithi097 ceph-mon[35562]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e1ad4f9-e724-440f-b515-3ec1ad97d6e4"}]': finished 2024-09-11T23:16:09.745 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:09 smithi097 ceph-mon[35562]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T23:16:09.746 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:09 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:16:09.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:09 smithi090 ceph-mon[35130]: from='client.? 172.21.15.97:0/102482880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e1ad4f9-e724-440f-b515-3ec1ad97d6e4"}]: dispatch 2024-09-11T23:16:09.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:09 smithi090 ceph-mon[35130]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e1ad4f9-e724-440f-b515-3ec1ad97d6e4"}]: dispatch 2024-09-11T23:16:09.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:09 smithi090 ceph-mon[35130]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e1ad4f9-e724-440f-b515-3ec1ad97d6e4"}]': finished 2024-09-11T23:16:09.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:09 smithi090 ceph-mon[35130]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T23:16:09.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:09 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:16:09.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:09 smithi043 ceph-mon[30950]: from='client.? 172.21.15.97:0/102482880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e1ad4f9-e724-440f-b515-3ec1ad97d6e4"}]: dispatch 2024-09-11T23:16:09.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:09 smithi043 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e1ad4f9-e724-440f-b515-3ec1ad97d6e4"}]: dispatch 2024-09-11T23:16:09.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:09 smithi043 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e1ad4f9-e724-440f-b515-3ec1ad97d6e4"}]': finished 2024-09-11T23:16:09.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:09 smithi043 ceph-mon[30950]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T23:16:09.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:09 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:16:10.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:10 smithi090 ceph-mon[35130]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:10.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:10 smithi090 ceph-mon[35130]: from='client.? 172.21.15.97:0/1694055436' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:16:10.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:10 smithi097 ceph-mon[35562]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:10.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:10 smithi097 ceph-mon[35562]: from='client.? 172.21.15.97:0/1694055436' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:16:10.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:10 smithi043 ceph-mon[30950]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:10.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:10 smithi043 ceph-mon[30950]: from='client.? 172.21.15.97:0/1694055436' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:16:12.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:12 smithi090 ceph-mon[35130]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:12.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:12 smithi097 ceph-mon[35562]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:12.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:12 smithi043 ceph-mon[30950]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:14.737 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:14 smithi097 ceph-mon[35562]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:14.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:14 smithi090 ceph-mon[35130]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:14.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:14 smithi043 ceph-mon[30950]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:16.478 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:16 smithi097 ceph-mon[35562]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:16.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:16 smithi090 ceph-mon[35130]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:16.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:16 smithi043 ceph-mon[30950]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:18.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:18 smithi097 ceph-mon[35562]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:18.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:18 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T23:16:18.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:18 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:18.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:18 smithi090 ceph-mon[35130]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:18.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:18 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T23:16:18.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:18 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:18.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:18 smithi043 ceph-mon[30950]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:18.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:18 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T23:16:18.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:18 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:19.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:19 smithi097 ceph-mon[35562]: Deploying daemon osd.5 on smithi097 2024-09-11T23:16:19.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:19 smithi090 ceph-mon[35130]: Deploying daemon osd.5 on smithi097 2024-09-11T23:16:19.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:19 smithi043 ceph-mon[30950]: Deploying daemon osd.5 on smithi097 2024-09-11T23:16:20.706 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:20 smithi097 ceph-mon[35562]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:20.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:20 smithi090 ceph-mon[35130]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:20.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:20 smithi043 ceph-mon[30950]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:22.689 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:22 smithi097 ceph-mon[35562]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:22.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:22 smithi090 ceph-mon[35130]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:22.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:22 smithi043 ceph-mon[30950]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:24.691 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:24 smithi097 ceph-mon[35562]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:24.692 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:24 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:16:24.692 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:24 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:24.692 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:24 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:24.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:24 smithi090 ceph-mon[35130]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:24.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:24 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:16:24.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:24 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:24.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:24 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:24.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:24 smithi043 ceph-mon[30950]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:24.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:24 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:16:24.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:24 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:24.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:24 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:26.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:26 smithi090 ceph-mon[35130]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:26.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:26 smithi097 ceph-mon[35562]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:26.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:26 smithi043 ceph-mon[30950]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:28.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:27 smithi090 ceph-mon[35130]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:28.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:27 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:28.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:27 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:28.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:27 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:28.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:27 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:28.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:27 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:28.271 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:27 smithi097 ceph-mon[35562]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:28.271 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:27 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:28.271 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:27 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:28.271 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:27 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:28.271 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:27 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:28.271 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:27 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:28.311 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 5 on host 'smithi097' 2024-09-11T23:16:28.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:28.311+0000 7f29bf7fe640 1 -- 172.21.15.97:0/2067755593 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f29c8062630 con 0x7f29b40a2bb0 2024-09-11T23:16:28.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:28.312+0000 7f29cc9f4640 1 -- 172.21.15.97:0/2067755593 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f29b40a2bb0 msgr2=0x7f29b40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:28.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:28.312+0000 7f29cc9f4640 1 --2- 172.21.15.97:0/2067755593 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f29b40a2bb0 0x7f29b40a5070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f29b8005ab0 tx=0x7f29b8000bc0 comp rx=0 tx=0).stop 2024-09-11T23:16:28.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:28.312+0000 7f29cc9f4640 1 -- 172.21.15.97:0/2067755593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c81026a0 msgr2=0x7f29c8197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:28.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:28.312+0000 7f29cc9f4640 1 --2- 172.21.15.97:0/2067755593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c81026a0 0x7f29c8197f50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f29b000e9f0 tx=0x7f29b000edf0 comp rx=0 tx=0).stop 2024-09-11T23:16:28.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:28.312+0000 7f29cc9f4640 1 -- 172.21.15.97:0/2067755593 shutdown_connections 2024-09-11T23:16:28.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:28.312+0000 7f29cc9f4640 1 --2- 172.21.15.97:0/2067755593 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f29b40a2bb0 0x7f29b40a5070 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:28.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:28.312+0000 7f29cc9f4640 1 --2- 172.21.15.97:0/2067755593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c8107900 0x7f29c819f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:28.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:28.312+0000 7f29cc9f4640 1 --2- 172.21.15.97:0/2067755593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29c8104fd0 0x7f29c8198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:28.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:28.312+0000 7f29cc9f4640 1 --2- 172.21.15.97:0/2067755593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c81026a0 0x7f29c8197f50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:28.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:28.312+0000 7f29cc9f4640 1 -- 172.21.15.97:0/2067755593 >> 172.21.15.97:0/2067755593 conn(0x7f29c80fc2a0 msgr2=0x7f29c8103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:28.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:28.312+0000 7f29cc9f4640 1 -- 172.21.15.97:0/2067755593 shutdown_connections 2024-09-11T23:16:28.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:16:28.312+0000 7f29cc9f4640 1 -- 172.21.15.97:0/2067755593 wait complete. 2024-09-11T23:16:28.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:27 smithi043 ceph-mon[30950]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:28.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:27 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:28.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:27 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:28.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:27 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:28.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:27 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:28.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:27 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:28.951 DEBUG:teuthology.orchestra.run.smithi097:osd.5> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.5.service 2024-09-11T23:16:28.953 INFO:tasks.cephadm:Deploying osd.6 on smithi043 with /dev/vg_nvme/lv_2... 2024-09-11T23:16:28.954 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T23:16:29.183 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:28 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:16:29.183 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:28 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:29.183 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:28 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:29.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:28 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:16:29.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:28 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:29.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:28 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:29.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:28 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:16:29.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:28 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:29.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:28 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:30.443 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:30 smithi097 ceph-mon[35562]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:30.443 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:30 smithi097 ceph-mon[35562]: from='osd.5 [v2:172.21.15.97:6808/1540039543,v1:172.21.15.97:6809/1540039543]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:16:30.444 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:30 smithi097 ceph-mon[35562]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:16:30.444 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 11 23:16:30 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-5[46989]: 2024-09-11T23:16:30.284+0000 7fc92470d640 -1 osd.5 0 log_to_monitors true 2024-09-11T23:16:30.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:30 smithi043 ceph-mon[30950]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:30.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:30 smithi043 ceph-mon[30950]: from='osd.5 [v2:172.21.15.97:6808/1540039543,v1:172.21.15.97:6809/1540039543]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:16:30.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:30 smithi043 ceph-mon[30950]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:16:30.772 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:30 smithi090 ceph-mon[35130]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:30.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:30 smithi090 ceph-mon[35130]: from='osd.5 [v2:172.21.15.97:6808/1540039543,v1:172.21.15.97:6809/1540039543]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:16:30.773 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:30 smithi090 ceph-mon[35130]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T23:16:31.505 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:31 smithi097 ceph-mon[35562]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T23:16:31.505 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:31 smithi097 ceph-mon[35562]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T23:16:31.506 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:31 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:16:31.506 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:31 smithi097 ceph-mon[35562]: from='osd.5 [v2:172.21.15.97:6808/1540039543,v1:172.21.15.97:6809/1540039543]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:16:31.506 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:31 smithi097 ceph-mon[35562]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:16:31.506 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 11 23:16:31 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-5[46989]: 2024-09-11T23:16:31.323+0000 7fc920690640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T23:16:31.507 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 11 23:16:31 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-5[46989]: 2024-09-11T23:16:31.328+0000 7fc91c4b8640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:16:31.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:31 smithi043 ceph-mon[30950]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T23:16:31.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:31 smithi043 ceph-mon[30950]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T23:16:31.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:31 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:16:31.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:31 smithi043 ceph-mon[30950]: from='osd.5 [v2:172.21.15.97:6808/1540039543,v1:172.21.15.97:6809/1540039543]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:16:31.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:31 smithi043 ceph-mon[30950]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:16:31.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:31 smithi090 ceph-mon[35130]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T23:16:31.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:31 smithi090 ceph-mon[35130]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T23:16:31.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:31 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:16:31.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:31 smithi090 ceph-mon[35130]: from='osd.5 [v2:172.21.15.97:6808/1540039543,v1:172.21.15.97:6809/1540039543]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:16:31.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:31 smithi090 ceph-mon[35130]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:16:32.019 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:16:32.045 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_2 2024-09-11T23:16:32.300 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:16:32.578 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:32 smithi043 ceph-mon[30950]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:32.579 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:32 smithi043 ceph-mon[30950]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-11T23:16:32.579 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:32 smithi043 ceph-mon[30950]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T23:16:32.579 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:32 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:16:32.579 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:32 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:16:32.579 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:32 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:32.579 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:32 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:32.579 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:32 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:16:32.579 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:32 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:16:32.579 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:32 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:32.579 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:32 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:32.579 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:32 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:32.580 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:32 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:32.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:32 smithi090 ceph-mon[35130]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:32.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:32 smithi090 ceph-mon[35130]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-11T23:16:32.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:32 smithi090 ceph-mon[35130]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T23:16:32.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:32 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:16:32.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:32 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:16:32.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:32 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:32.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:32 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:32.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:32 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:16:32.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:32 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:16:32.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:32 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:32.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:32 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:32.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:32 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:32.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:32 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:32.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:32 smithi097 ceph-mon[35562]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:32.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:32 smithi097 ceph-mon[35562]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-11T23:16:32.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:32 smithi097 ceph-mon[35562]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T23:16:32.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:32 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:16:32.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:32 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:16:32.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:32 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:32.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:32 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:32.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:32 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:16:32.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:32 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:16:32.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:32 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:32.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:32 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:32.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:32 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:32.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:32 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:32.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.807+0000 7f559b80e640 1 -- 172.21.15.43:0/2393139015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5594101730 msgr2=0x7f5594101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:32.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.807+0000 7f559b80e640 1 --2- 172.21.15.43:0/2393139015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5594101730 0x7f5594101be0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f558c009ca0 tx=0x7f558c02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:16:32.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.807+0000 7f559b80e640 1 -- 172.21.15.43:0/2393139015 shutdown_connections 2024-09-11T23:16:32.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.807+0000 7f559b80e640 1 --2- 172.21.15.43:0/2393139015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5594101730 0x7f5594101be0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:32.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.807+0000 7f559b80e640 1 --2- 172.21.15.43:0/2393139015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5594100d90 0x7f55941011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:32.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.807+0000 7f559b80e640 1 --2- 172.21.15.43:0/2393139015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5594100450 0x7f5594100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:32.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.807+0000 7f559b80e640 1 -- 172.21.15.43:0/2393139015 >> 172.21.15.43:0/2393139015 conn(0x7f55940fc280 msgr2=0x7f55940fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:32.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.807+0000 7f559b80e640 1 -- 172.21.15.43:0/2393139015 shutdown_connections 2024-09-11T23:16:32.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.807+0000 7f559b80e640 1 -- 172.21.15.43:0/2393139015 wait complete. 2024-09-11T23:16:32.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.807+0000 7f559b80e640 1 Processor -- start 2024-09-11T23:16:32.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.807+0000 7f559b80e640 1 -- start start 2024-09-11T23:16:32.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f559b80e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5594100450 0x7f559419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:32.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f559b80e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5594100d90 0x7f559419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:32.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f559b80e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5594101730 0x7f55941a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:32.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f559b80e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f559410fa40 con 0x7f5594100450 2024-09-11T23:16:32.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f559b80e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f559410f8c0 con 0x7f5594101730 2024-09-11T23:16:32.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f559b80e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f559410fbc0 con 0x7f5594100d90 2024-09-11T23:16:32.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f55995aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5594100450 0x7f559419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:32.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f55995aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5594100450 0x7f559419a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50030/0 (socket says 172.21.15.43:50030) 2024-09-11T23:16:32.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f55995aa640 1 -- 172.21.15.43:0/471938503 learned_addr learned my addr 172.21.15.43:0/471938503 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:16:32.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f5598da9640 1 --2- 172.21.15.43:0/471938503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5594100d90 0x7f559419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:32.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f5599dab640 1 --2- 172.21.15.43:0/471938503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5594101730 0x7f55941a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:32.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f55995aa640 1 -- 172.21.15.43:0/471938503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5594100d90 msgr2=0x7f559419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:32.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f55995aa640 1 --2- 172.21.15.43:0/471938503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5594100d90 0x7f559419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:32.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f55995aa640 1 -- 172.21.15.43:0/471938503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5594101730 msgr2=0x7f55941a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:32.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f55995aa640 1 --2- 172.21.15.43:0/471938503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5594101730 0x7f55941a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:32.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f55995aa640 1 -- 172.21.15.43:0/471938503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55941a1c40 con 0x7f5594100450 2024-09-11T23:16:32.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f5599dab640 1 --2- 172.21.15.43:0/471938503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5594101730 0x7f55941a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:16:32.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f55995aa640 1 --2- 172.21.15.43:0/471938503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5594100450 0x7f559419a010 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f559000d770 tx=0x7f559000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:32.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f558a7fc640 1 -- 172.21.15.43:0/471938503 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590017070 con 0x7f5594100450 2024-09-11T23:16:32.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f558a7fc640 1 -- 172.21.15.43:0/471938503 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5590009820 con 0x7f5594100450 2024-09-11T23:16:32.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f559b80e640 1 -- 172.21.15.43:0/471938503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55941a1f30 con 0x7f5594100450 2024-09-11T23:16:32.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f558a7fc640 1 -- 172.21.15.43:0/471938503 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590004cd0 con 0x7f5594100450 2024-09-11T23:16:32.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.808+0000 7f559b80e640 1 -- 172.21.15.43:0/471938503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55941a2380 con 0x7f5594100450 2024-09-11T23:16:32.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.809+0000 7f559b80e640 1 -- 172.21.15.43:0/471938503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f559410b1a0 con 0x7f5594100450 2024-09-11T23:16:32.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.810+0000 7f558a7fc640 1 -- 172.21.15.43:0/471938503 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f55900099d0 con 0x7f5594100450 2024-09-11T23:16:32.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.811+0000 7f558a7fc640 1 --2- 172.21.15.43:0/471938503 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f55800a2c30 0x7f55800a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:32.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.811+0000 7f5598da9640 1 --2- 172.21.15.43:0/471938503 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f55800a2c30 0x7f55800a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:32.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.811+0000 7f558a7fc640 1 -- 172.21.15.43:0/471938503 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4566+0+0 (secure 0 0 0) 0x7f55900cdb90 con 0x7f5594100450 2024-09-11T23:16:32.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.812+0000 7f5598da9640 1 --2- 172.21.15.43:0/471938503 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f55800a2c30 0x7f55800a50f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f559419b530 tx=0x7f558400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:32.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:32.816+0000 7f558a7fc640 1 -- 172.21.15.43:0/471938503 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f559009b930 con 0x7f5594100450 2024-09-11T23:16:33.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:33.002+0000 7f559b80e640 1 -- 172.21.15.43:0/471938503 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5594062630 con 0x7f55800a2c30 2024-09-11T23:16:33.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:33 smithi043 ceph-mon[30950]: purged_snaps scrub starts 2024-09-11T23:16:33.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:33 smithi043 ceph-mon[30950]: purged_snaps scrub ok 2024-09-11T23:16:33.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:33 smithi043 ceph-mon[30950]: Detected new or changed devices on smithi097 2024-09-11T23:16:33.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:33 smithi043 ceph-mon[30950]: Adjusting osd_memory_target on smithi097 to 8544M 2024-09-11T23:16:33.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:33 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:16:33.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:33 smithi043 ceph-mon[30950]: osd.5 [v2:172.21.15.97:6808/1540039543,v1:172.21.15.97:6809/1540039543] boot 2024-09-11T23:16:33.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:33 smithi043 ceph-mon[30950]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T23:16:33.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:33 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:16:33.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:33 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:16:33.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:33 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:16:33.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:33 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:33.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:33 smithi090 ceph-mon[35130]: purged_snaps scrub starts 2024-09-11T23:16:33.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:33 smithi090 ceph-mon[35130]: purged_snaps scrub ok 2024-09-11T23:16:33.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:33 smithi090 ceph-mon[35130]: Detected new or changed devices on smithi097 2024-09-11T23:16:33.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:33 smithi090 ceph-mon[35130]: Adjusting osd_memory_target on smithi097 to 8544M 2024-09-11T23:16:33.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:33 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:16:33.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:33 smithi090 ceph-mon[35130]: osd.5 [v2:172.21.15.97:6808/1540039543,v1:172.21.15.97:6809/1540039543] boot 2024-09-11T23:16:33.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:33 smithi090 ceph-mon[35130]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T23:16:33.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:33 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:16:33.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:33 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:16:33.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:33 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:16:33.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:33 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:33.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:33 smithi097 ceph-mon[35562]: purged_snaps scrub starts 2024-09-11T23:16:33.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:33 smithi097 ceph-mon[35562]: purged_snaps scrub ok 2024-09-11T23:16:33.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:33 smithi097 ceph-mon[35562]: Detected new or changed devices on smithi097 2024-09-11T23:16:33.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:33 smithi097 ceph-mon[35562]: Adjusting osd_memory_target on smithi097 to 8544M 2024-09-11T23:16:33.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:33 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:16:33.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:33 smithi097 ceph-mon[35562]: osd.5 [v2:172.21.15.97:6808/1540039543,v1:172.21.15.97:6809/1540039543] boot 2024-09-11T23:16:33.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:33 smithi097 ceph-mon[35562]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T23:16:33.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:33 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:16:33.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:33 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:16:33.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:33 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:16:33.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:33 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:34.436 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:34 smithi043 ceph-mon[30950]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:34.436 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:34 smithi043 ceph-mon[30950]: from='client.14379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:34.436 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:34 smithi043 ceph-mon[30950]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T23:16:34.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:34 smithi090 ceph-mon[35130]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:34.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:34 smithi090 ceph-mon[35130]: from='client.14379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:34.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:34 smithi090 ceph-mon[35130]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T23:16:34.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:34 smithi097 ceph-mon[35562]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-11T23:16:34.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:34 smithi097 ceph-mon[35562]: from='client.14379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:34.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:34 smithi097 ceph-mon[35562]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T23:16:35.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:35 smithi043 ceph-mon[30950]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T23:16:35.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:35 smithi090 ceph-mon[35130]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T23:16:35.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:35 smithi097 ceph-mon[35562]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T23:16:36.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:36 smithi043 ceph-mon[30950]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:16:36.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:36 smithi043 ceph-mon[30950]: osdmap e40: 6 total, 6 up, 6 in 2024-09-11T23:16:36.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:36 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1239823392' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d34df8df-f40d-4159-8b29-5401b1b0e431"}]: dispatch 2024-09-11T23:16:36.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:36 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1239823392' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d34df8df-f40d-4159-8b29-5401b1b0e431"}]': finished 2024-09-11T23:16:36.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:36 smithi043 ceph-mon[30950]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T23:16:36.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:36 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:16:36.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:36 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/56742925' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:16:36.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:36 smithi090 ceph-mon[35130]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:16:36.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:36 smithi090 ceph-mon[35130]: osdmap e40: 6 total, 6 up, 6 in 2024-09-11T23:16:36.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:36 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1239823392' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d34df8df-f40d-4159-8b29-5401b1b0e431"}]: dispatch 2024-09-11T23:16:36.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:36 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1239823392' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d34df8df-f40d-4159-8b29-5401b1b0e431"}]': finished 2024-09-11T23:16:36.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:36 smithi090 ceph-mon[35130]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T23:16:36.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:36 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:16:36.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:36 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/56742925' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:16:36.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:36 smithi097 ceph-mon[35562]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:16:36.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:36 smithi097 ceph-mon[35562]: osdmap e40: 6 total, 6 up, 6 in 2024-09-11T23:16:36.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:36 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1239823392' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d34df8df-f40d-4159-8b29-5401b1b0e431"}]: dispatch 2024-09-11T23:16:36.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:36 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1239823392' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d34df8df-f40d-4159-8b29-5401b1b0e431"}]': finished 2024-09-11T23:16:36.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:36 smithi097 ceph-mon[35562]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T23:16:36.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:36 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:16:36.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:36 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/56742925' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:16:38.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:38 smithi043 ceph-mon[30950]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T23:16:38.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:38 smithi090 ceph-mon[35130]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T23:16:38.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:38 smithi097 ceph-mon[35562]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T23:16:40.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:40 smithi043 ceph-mon[30950]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-11T23:16:40.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:40 smithi090 ceph-mon[35130]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-11T23:16:40.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:40 smithi097 ceph-mon[35562]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-11T23:16:42.603 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:42 smithi097 ceph-mon[35562]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-11T23:16:42.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:42 smithi043 ceph-mon[30950]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-11T23:16:42.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:42 smithi090 ceph-mon[35130]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-11T23:16:44.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:44 smithi043 ceph-mon[30950]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:16:44.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:44 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T23:16:44.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:44 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:44.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:44 smithi090 ceph-mon[35130]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:16:44.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:44 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T23:16:44.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:44 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:44.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:44 smithi097 ceph-mon[35562]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T23:16:44.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:44 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T23:16:44.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:44 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:45.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:45 smithi043 ceph-mon[30950]: Deploying daemon osd.6 on smithi043 2024-09-11T23:16:45.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:45 smithi090 ceph-mon[35130]: Deploying daemon osd.6 on smithi043 2024-09-11T23:16:45.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:45 smithi097 ceph-mon[35562]: Deploying daemon osd.6 on smithi043 2024-09-11T23:16:46.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:46 smithi043 ceph-mon[30950]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T23:16:46.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:46 smithi090 ceph-mon[35130]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T23:16:46.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:46 smithi097 ceph-mon[35562]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T23:16:48.513 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:48 smithi043 ceph-mon[30950]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-11T23:16:48.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:48 smithi090 ceph-mon[35130]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-11T23:16:48.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:48 smithi097 ceph-mon[35562]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-11T23:16:50.518 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:50 smithi043 ceph-mon[30950]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:16:50.518 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:50 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:16:50.518 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:50 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:50.518 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:50 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:50 smithi090 ceph-mon[35130]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:16:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:50 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:16:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:50 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:50 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:50.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:50 smithi097 ceph-mon[35562]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:16:50.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:50 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:16:50.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:50 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:50.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:50 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:52.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:52 smithi090 ceph-mon[35130]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:16:52.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:52 smithi097 ceph-mon[35562]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:16:52.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:52 smithi043 ceph-mon[30950]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:16:53.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:53.798+0000 7f558a7fc640 1 -- 172.21.15.43:0/471938503 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5594062630 con 0x7f55800a2c30 2024-09-11T23:16:53.800 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 6 on host 'smithi043' 2024-09-11T23:16:53.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:53.801+0000 7f559b80e640 1 -- 172.21.15.43:0/471938503 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f55800a2c30 msgr2=0x7f55800a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:53.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:53.801+0000 7f559b80e640 1 --2- 172.21.15.43:0/471938503 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f55800a2c30 0x7f55800a50f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f559419b530 tx=0x7f558400a2d0 comp rx=0 tx=0).stop 2024-09-11T23:16:53.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:53.801+0000 7f559b80e640 1 -- 172.21.15.43:0/471938503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5594100450 msgr2=0x7f559419a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:53.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:53.801+0000 7f559b80e640 1 --2- 172.21.15.43:0/471938503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5594100450 0x7f559419a010 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f559000d770 tx=0x7f559000db70 comp rx=0 tx=0).stop 2024-09-11T23:16:53.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:53.801+0000 7f559b80e640 1 -- 172.21.15.43:0/471938503 shutdown_connections 2024-09-11T23:16:53.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:53.801+0000 7f559b80e640 1 --2- 172.21.15.43:0/471938503 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f55800a2c30 0x7f55800a50f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:53.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:53.801+0000 7f559b80e640 1 --2- 172.21.15.43:0/471938503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5594101730 0x7f55941a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:53.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:53.801+0000 7f559b80e640 1 --2- 172.21.15.43:0/471938503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5594100d90 0x7f559419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:53.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:53.801+0000 7f559b80e640 1 --2- 172.21.15.43:0/471938503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5594100450 0x7f559419a010 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:53.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:53.801+0000 7f559b80e640 1 -- 172.21.15.43:0/471938503 >> 172.21.15.43:0/471938503 conn(0x7f55940fc280 msgr2=0x7f55940fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:53.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:53.801+0000 7f559b80e640 1 -- 172.21.15.43:0/471938503 shutdown_connections 2024-09-11T23:16:53.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:16:53.801+0000 7f559b80e640 1 -- 172.21.15.43:0/471938503 wait complete. 2024-09-11T23:16:54.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:53 smithi090 ceph-mon[35130]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:16:54.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:53 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:54.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:53 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:54.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:53 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:54.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:53 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:54.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:53 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:54.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:53 smithi097 ceph-mon[35562]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:16:54.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:53 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:54.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:53 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:54.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:53 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:54.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:53 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:54.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:53 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:54.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:53 smithi043 ceph-mon[30950]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:16:54.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:53 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:54.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:53 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:54.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:53 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:54.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:53 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:54.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:53 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:54.525 DEBUG:teuthology.orchestra.run.smithi043:osd.6> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.6.service 2024-09-11T23:16:54.527 INFO:tasks.cephadm:Deploying osd.7 on smithi090 with /dev/vg_nvme/lv_2... 2024-09-11T23:16:54.527 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T23:16:54.784 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:16:54.897 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:54 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:16:54.897 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:54 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:54.897 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:54 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:55.025 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:54 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:16:55.025 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:54 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:55.025 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:54 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:55.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:54 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:16:55.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:54 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:55.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:54 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:55.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:55 smithi043 ceph-mon[30950]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:16:55.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:55 smithi043 ceph-mon[30950]: from='osd.6 [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T23:16:55.957 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 11 23:16:55 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-11T23:16:55.623+0000 7fb3eff17640 -1 osd.6 0 log_to_monitors true 2024-09-11T23:16:56.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:55 smithi090 ceph-mon[35130]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:16:56.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:55 smithi090 ceph-mon[35130]: from='osd.6 [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T23:16:56.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:55 smithi097 ceph-mon[35562]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:16:56.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:55 smithi097 ceph-mon[35562]: from='osd.6 [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T23:16:57.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:56 smithi090 ceph-mon[35130]: from='osd.6 [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T23:16:57.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:56 smithi090 ceph-mon[35130]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T23:16:57.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:56 smithi090 ceph-mon[35130]: from='osd.6 [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T23:16:57.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:56 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:16:57.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:56 smithi043 ceph-mon[30950]: from='osd.6 [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T23:16:57.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:56 smithi043 ceph-mon[30950]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T23:16:57.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:56 smithi043 ceph-mon[30950]: from='osd.6 [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T23:16:57.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:56 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:16:57.111 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 11 23:16:56 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-11T23:16:56.814+0000 7fb3ebe9a640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T23:16:57.111 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 11 23:16:56 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-11T23:16:56.825+0000 7fb3e7cc2640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:16:57.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:56 smithi097 ceph-mon[35562]: from='osd.6 [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T23:16:57.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:56 smithi097 ceph-mon[35562]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T23:16:57.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:56 smithi097 ceph-mon[35562]: from='osd.6 [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T23:16:57.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:56 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:16:57.561 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-11T23:16:57.587 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_2 2024-09-11T23:16:57.852 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:16:58.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:57 smithi043 ceph-mon[30950]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:16:58.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:57 smithi043 ceph-mon[30950]: from='osd.6 [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T23:16:58.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:57 smithi043 ceph-mon[30950]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T23:16:58.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:57 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:16:58.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:57 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:16:58.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:57 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:58.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:57 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:58.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:57 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:16:58.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:57 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:16:58.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:57 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:16:58.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:57 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:58.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:57 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:58.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:57 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:58.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:57 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:58.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:57 smithi090 ceph-mon[35130]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:16:58.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:57 smithi090 ceph-mon[35130]: from='osd.6 [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T23:16:58.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:57 smithi090 ceph-mon[35130]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T23:16:58.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:57 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:16:58.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:57 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:16:58.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:57 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:58.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:57 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:58.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:57 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:16:58.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:57 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:16:58.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:57 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:16:58.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:57 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:58.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:57 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:58.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:57 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:58.125 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:57 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:58.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:57 smithi097 ceph-mon[35562]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T23:16:58.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:57 smithi097 ceph-mon[35562]: from='osd.6 [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T23:16:58.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:57 smithi097 ceph-mon[35562]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T23:16:58.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:57 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:16:58.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:57 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:16:58.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:57 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:58.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:57 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:58.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:57 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:16:58.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:57 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:16:58.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:57 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:16:58.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:57 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:58.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:57 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:58.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:57 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:16:58.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:57 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:16:58.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.341+0000 7f0c179d4640 1 -- 172.21.15.90:0/1980798027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c10100ad0 msgr2=0x7f0c10100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:58.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.341+0000 7f0c179d4640 1 --2- 172.21.15.90:0/1980798027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c10100ad0 0x7f0c10100ed0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c009900 tx=0x7f0c0c02a410 comp rx=0 tx=0).stop 2024-09-11T23:16:58.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.341+0000 7f0c179d4640 1 -- 172.21.15.90:0/1980798027 shutdown_connections 2024-09-11T23:16:58.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.341+0000 7f0c179d4640 1 --2- 172.21.15.90:0/1980798027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c10102690 0x7f0c10108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:58.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.341+0000 7f0c179d4640 1 --2- 172.21.15.90:0/1980798027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c10101cd0 0x7f0c10102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:58.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.341+0000 7f0c179d4640 1 --2- 172.21.15.90:0/1980798027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c10100ad0 0x7f0c10100ed0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:58.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.341+0000 7f0c179d4640 1 -- 172.21.15.90:0/1980798027 >> 172.21.15.90:0/1980798027 conn(0x7f0c100fc280 msgr2=0x7f0c100fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:16:58.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.341+0000 7f0c179d4640 1 -- 172.21.15.90:0/1980798027 shutdown_connections 2024-09-11T23:16:58.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.341+0000 7f0c179d4640 1 -- 172.21.15.90:0/1980798027 wait complete. 2024-09-11T23:16:58.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.342+0000 7f0c179d4640 1 Processor -- start 2024-09-11T23:16:58.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.342+0000 7f0c179d4640 1 -- start start 2024-09-11T23:16:58.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.342+0000 7f0c179d4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c10100ad0 0x7f0c10196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:58.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.342+0000 7f0c179d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c10101cd0 0x7f0c10197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:58.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.342+0000 7f0c179d4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c10102690 0x7f0c1019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:58.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.342+0000 7f0c179d4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1010c730 con 0x7f0c10101cd0 2024-09-11T23:16:58.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.342+0000 7f0c179d4640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1010c5b0 con 0x7f0c10102690 2024-09-11T23:16:58.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.342+0000 7f0c179d4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1010c8b0 con 0x7f0c10100ad0 2024-09-11T23:16:58.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.343+0000 7f0c15f71640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c10102690 0x7f0c1019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:58.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.343+0000 7f0c15f71640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c10102690 0x7f0c1019e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33142/0 (socket says 172.21.15.90:33142) 2024-09-11T23:16:58.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.343+0000 7f0c15f71640 1 -- 172.21.15.90:0/3900145160 learned_addr learned my addr 172.21.15.90:0/3900145160 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-11T23:16:58.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.343+0000 7f0c14f6f640 1 --2- 172.21.15.90:0/3900145160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c10101cd0 0x7f0c10197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:58.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.343+0000 7f0c15770640 1 --2- 172.21.15.90:0/3900145160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c10100ad0 0x7f0c10196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:58.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.343+0000 7f0c15f71640 1 -- 172.21.15.90:0/3900145160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c10100ad0 msgr2=0x7f0c10196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:58.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.343+0000 7f0c15f71640 1 --2- 172.21.15.90:0/3900145160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c10100ad0 0x7f0c10196df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:58.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.343+0000 7f0c15f71640 1 -- 172.21.15.90:0/3900145160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c10101cd0 msgr2=0x7f0c10197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:16:58.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.343+0000 7f0c15f71640 1 --2- 172.21.15.90:0/3900145160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c10101cd0 0x7f0c10197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:16:58.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.343+0000 7f0c15f71640 1 -- 172.21.15.90:0/3900145160 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1019ea20 con 0x7f0c10102690 2024-09-11T23:16:58.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.343+0000 7f0c15f71640 1 --2- 172.21.15.90:0/3900145160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c10102690 0x7f0c1019e3b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800eaa0 tx=0x7f0c0800ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:58.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.343+0000 7f0c067fc640 1 -- 172.21.15.90:0/3900145160 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0800cdb0 con 0x7f0c10102690 2024-09-11T23:16:58.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.343+0000 7f0c179d4640 1 -- 172.21.15.90:0/3900145160 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c1019ed10 con 0x7f0c10102690 2024-09-11T23:16:58.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.343+0000 7f0c067fc640 1 -- 172.21.15.90:0/3900145160 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c08005020 con 0x7f0c10102690 2024-09-11T23:16:58.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.343+0000 7f0c067fc640 1 -- 172.21.15.90:0/3900145160 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c080116b0 con 0x7f0c10102690 2024-09-11T23:16:58.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.343+0000 7f0c179d4640 1 -- 172.21.15.90:0/3900145160 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c1019f1f0 con 0x7f0c10102690 2024-09-11T23:16:58.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.344+0000 7f0c179d4640 1 -- 172.21.15.90:0/3900145160 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c10107f20 con 0x7f0c10102690 2024-09-11T23:16:58.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.346+0000 7f0c067fc640 1 -- 172.21.15.90:0/3900145160 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0c08005560 con 0x7f0c10102690 2024-09-11T23:16:58.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.347+0000 7f0c067fc640 1 --2- 172.21.15.90:0/3900145160 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0bfc0a2bb0 0x7f0bfc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:16:58.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.347+0000 7f0c067fc640 1 -- 172.21.15.90:0/3900145160 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7f0c080ce8e0 con 0x7f0c10102690 2024-09-11T23:16:58.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.347+0000 7f0c15770640 1 --2- 172.21.15.90:0/3900145160 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0bfc0a2bb0 0x7f0bfc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:16:58.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.348+0000 7f0c15770640 1 --2- 172.21.15.90:0/3900145160 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0bfc0a2bb0 0x7f0bfc0a5070 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c031610 tx=0x7f0c0c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:16:58.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.350+0000 7f0c067fc640 1 -- 172.21.15.90:0/3900145160 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0c08016030 con 0x7f0c10102690 2024-09-11T23:16:58.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:16:58.534+0000 7f0c179d4640 1 -- 172.21.15.90:0/3900145160 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0c10062630 con 0x7f0bfc0a2bb0 2024-09-11T23:16:59.102 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:58 smithi090 ceph-mon[35130]: purged_snaps scrub starts 2024-09-11T23:16:59.102 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:58 smithi090 ceph-mon[35130]: purged_snaps scrub ok 2024-09-11T23:16:59.102 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:58 smithi090 ceph-mon[35130]: Detected new or changed devices on smithi043 2024-09-11T23:16:59.102 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:58 smithi090 ceph-mon[35130]: Adjusting osd_memory_target on smithi043 to 5697M 2024-09-11T23:16:59.102 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:58 smithi090 ceph-mon[35130]: osd.6 [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637] boot 2024-09-11T23:16:59.102 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:58 smithi090 ceph-mon[35130]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T23:16:59.103 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:58 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:16:59.103 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:58 smithi090 ceph-mon[35130]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:59.103 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:58 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:16:59.103 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:58 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:16:59.103 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:58 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:59.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:58 smithi043 ceph-mon[30950]: purged_snaps scrub starts 2024-09-11T23:16:59.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:58 smithi043 ceph-mon[30950]: purged_snaps scrub ok 2024-09-11T23:16:59.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:58 smithi043 ceph-mon[30950]: Detected new or changed devices on smithi043 2024-09-11T23:16:59.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:58 smithi043 ceph-mon[30950]: Adjusting osd_memory_target on smithi043 to 5697M 2024-09-11T23:16:59.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:58 smithi043 ceph-mon[30950]: osd.6 [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637] boot 2024-09-11T23:16:59.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:58 smithi043 ceph-mon[30950]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T23:16:59.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:58 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:16:59.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:58 smithi043 ceph-mon[30950]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:59.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:58 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:16:59.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:58 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:16:59.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:58 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:59.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:58 smithi097 ceph-mon[35562]: purged_snaps scrub starts 2024-09-11T23:16:59.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:58 smithi097 ceph-mon[35562]: purged_snaps scrub ok 2024-09-11T23:16:59.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:58 smithi097 ceph-mon[35562]: Detected new or changed devices on smithi043 2024-09-11T23:16:59.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:58 smithi097 ceph-mon[35562]: Adjusting osd_memory_target on smithi043 to 5697M 2024-09-11T23:16:59.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:58 smithi097 ceph-mon[35562]: osd.6 [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637] boot 2024-09-11T23:16:59.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:58 smithi097 ceph-mon[35562]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T23:16:59.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:58 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:16:59.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:58 smithi097 ceph-mon[35562]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:16:59.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:58 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:16:59.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:58 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:16:59.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:58 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:16:59.999 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:59 smithi090 ceph-mon[35130]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:00.000 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:16:59 smithi090 ceph-mon[35130]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T23:17:00.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:59 smithi043 ceph-mon[30950]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:00.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:16:59 smithi043 ceph-mon[30950]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T23:17:00.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:59 smithi097 ceph-mon[35562]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:00.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:16:59 smithi097 ceph-mon[35562]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T23:17:02.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:01 smithi043 ceph-mon[30950]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:02.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:01 smithi043 ceph-mon[30950]: from='client.? 172.21.15.90:0/2373897424' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2e01078-528b-49fd-b000-9f584f784b1d"}]: dispatch 2024-09-11T23:17:02.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:01 smithi043 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2e01078-528b-49fd-b000-9f584f784b1d"}]: dispatch 2024-09-11T23:17:02.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:01 smithi043 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b2e01078-528b-49fd-b000-9f584f784b1d"}]': finished 2024-09-11T23:17:02.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:01 smithi043 ceph-mon[30950]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T23:17:02.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:01 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:17:02.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:01 smithi043 ceph-mon[30950]: from='client.? 172.21.15.90:0/351852405' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:17:02.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:01 smithi090 ceph-mon[35130]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:02.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:01 smithi090 ceph-mon[35130]: from='client.? 172.21.15.90:0/2373897424' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2e01078-528b-49fd-b000-9f584f784b1d"}]: dispatch 2024-09-11T23:17:02.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:01 smithi090 ceph-mon[35130]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2e01078-528b-49fd-b000-9f584f784b1d"}]: dispatch 2024-09-11T23:17:02.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:01 smithi090 ceph-mon[35130]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b2e01078-528b-49fd-b000-9f584f784b1d"}]': finished 2024-09-11T23:17:02.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:01 smithi090 ceph-mon[35130]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T23:17:02.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:01 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:17:02.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:01 smithi090 ceph-mon[35130]: from='client.? 172.21.15.90:0/351852405' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:17:02.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:01 smithi097 ceph-mon[35562]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:02.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:01 smithi097 ceph-mon[35562]: from='client.? 172.21.15.90:0/2373897424' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2e01078-528b-49fd-b000-9f584f784b1d"}]: dispatch 2024-09-11T23:17:02.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:01 smithi097 ceph-mon[35562]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2e01078-528b-49fd-b000-9f584f784b1d"}]: dispatch 2024-09-11T23:17:02.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:01 smithi097 ceph-mon[35562]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b2e01078-528b-49fd-b000-9f584f784b1d"}]': finished 2024-09-11T23:17:02.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:01 smithi097 ceph-mon[35562]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T23:17:02.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:01 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:17:02.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:01 smithi097 ceph-mon[35562]: from='client.? 172.21.15.90:0/351852405' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:17:04.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:03 smithi043 ceph-mon[30950]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:04.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:03 smithi090 ceph-mon[35130]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:04.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:03 smithi097 ceph-mon[35562]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:06.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:05 smithi043 ceph-mon[30950]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:06.124 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:05 smithi090 ceph-mon[35130]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:06.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:05 smithi097 ceph-mon[35562]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:08.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:07 smithi043 ceph-mon[30950]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:08.239 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:07 smithi090 ceph-mon[35130]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:08.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:07 smithi097 ceph-mon[35562]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:10.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:09 smithi043 ceph-mon[30950]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:10.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:09 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T23:17:10.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:09 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:10.212 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:09 smithi090 ceph-mon[35130]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:10.212 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:09 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T23:17:10.212 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:09 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:10.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:09 smithi097 ceph-mon[35562]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:10.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:09 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T23:17:10.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:09 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:11.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:10 smithi043 ceph-mon[30950]: Deploying daemon osd.7 on smithi090 2024-09-11T23:17:11.118 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:10 smithi090 ceph-mon[35130]: Deploying daemon osd.7 on smithi090 2024-09-11T23:17:11.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:10 smithi097 ceph-mon[35562]: Deploying daemon osd.7 on smithi090 2024-09-11T23:17:12.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:11 smithi090 ceph-mon[35130]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:12.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:11 smithi043 ceph-mon[30950]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:12.202 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:11 smithi097 ceph-mon[35562]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:14.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:13 smithi090 ceph-mon[35130]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:14.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:13 smithi043 ceph-mon[30950]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:14.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:13 smithi097 ceph-mon[35562]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:16.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:15 smithi090 ceph-mon[35130]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:16.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:15 smithi043 ceph-mon[30950]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:16.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:15 smithi097 ceph-mon[35562]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:17.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:16 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:17:17.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:16 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:17.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:16 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:17.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:16 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:17:17.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:16 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:17.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:16 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:17.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:16 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:17:17.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:16 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:17.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:16 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:18.096 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:17 smithi090 ceph-mon[35130]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:18.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:17 smithi097 ceph-mon[35562]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:18.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:17 smithi043 ceph-mon[30950]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:19.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:17:19.480+0000 7f0c067fc640 1 -- 172.21.15.90:0/3900145160 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0c10062630 con 0x7f0bfc0a2bb0 2024-09-11T23:17:19.482 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 7 on host 'smithi090' 2024-09-11T23:17:19.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:17:19.483+0000 7f0c179d4640 1 -- 172.21.15.90:0/3900145160 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0bfc0a2bb0 msgr2=0x7f0bfc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:19.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:17:19.483+0000 7f0c179d4640 1 --2- 172.21.15.90:0/3900145160 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0bfc0a2bb0 0x7f0bfc0a5070 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c031610 tx=0x7f0c0c000bc0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:17:19.483+0000 7f0c179d4640 1 -- 172.21.15.90:0/3900145160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c10102690 msgr2=0x7f0c1019e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:19.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:17:19.483+0000 7f0c179d4640 1 --2- 172.21.15.90:0/3900145160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c10102690 0x7f0c1019e3b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800eaa0 tx=0x7f0c0800ef60 comp rx=0 tx=0).stop 2024-09-11T23:17:19.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:17:19.483+0000 7f0c179d4640 1 -- 172.21.15.90:0/3900145160 shutdown_connections 2024-09-11T23:17:19.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:17:19.483+0000 7f0c179d4640 1 --2- 172.21.15.90:0/3900145160 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0bfc0a2bb0 0x7f0bfc0a5070 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:17:19.483+0000 7f0c179d4640 1 --2- 172.21.15.90:0/3900145160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c10102690 0x7f0c1019e3b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:17:19.483+0000 7f0c179d4640 1 --2- 172.21.15.90:0/3900145160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c10101cd0 0x7f0c10197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:17:19.483+0000 7f0c179d4640 1 --2- 172.21.15.90:0/3900145160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c10100ad0 0x7f0c10196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:19.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:17:19.483+0000 7f0c179d4640 1 -- 172.21.15.90:0/3900145160 >> 172.21.15.90:0/3900145160 conn(0x7f0c100fc280 msgr2=0x7f0c101050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:19.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:17:19.483+0000 7f0c179d4640 1 -- 172.21.15.90:0/3900145160 shutdown_connections 2024-09-11T23:17:19.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:17:19.483+0000 7f0c179d4640 1 -- 172.21.15.90:0/3900145160 wait complete. 2024-09-11T23:17:20.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:19 smithi043 ceph-mon[30950]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:20.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:19 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:20.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:19 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:20.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:19 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:20.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:19 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:17:20.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:19 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:20.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:19 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:17:20.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:19 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:20.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:19 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:20.189 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:19 smithi090 ceph-mon[35130]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:20.189 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:19 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:20.189 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:19 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:20.189 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:19 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:20.189 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:19 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:17:20.190 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:19 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:20.190 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:19 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:17:20.190 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:19 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:20.190 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:19 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:20.215 DEBUG:teuthology.orchestra.run.smithi090:osd.7> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T23:17:20.218 INFO:tasks.cephadm:Deploying osd.8 on smithi097 with /dev/vg_nvme/lv_2... 2024-09-11T23:17:20.218 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T23:17:20.245 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:19 smithi097 ceph-mon[35562]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:20.246 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:19 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:20.246 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:19 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:20.246 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:19 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:20.246 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:19 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:17:20.246 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:19 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:20.246 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:19 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:17:20.246 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:19 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:20.246 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:19 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:20.471 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:17:21.770 INFO:journalctl@ceph.osd.7.smithi090.stdout:Sep 11 23:17:21 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-7[52823]: 2024-09-11T23:17:21.690+0000 7f2e6f602640 -1 osd.7 0 log_to_monitors true 2024-09-11T23:17:21.883 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:21 smithi097 ceph-mon[35562]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:21.883 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:21 smithi097 ceph-mon[35562]: from='osd.7 [v2:172.21.15.90:6816/236874962,v1:172.21.15.90:6817/236874962]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:17:21.883 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:21 smithi097 ceph-mon[35562]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:17:22.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:21 smithi043 ceph-mon[30950]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:22.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:21 smithi043 ceph-mon[30950]: from='osd.7 [v2:172.21.15.90:6816/236874962,v1:172.21.15.90:6817/236874962]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:17:22.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:21 smithi043 ceph-mon[30950]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:17:22.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:21 smithi090 ceph-mon[35130]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:22.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:21 smithi090 ceph-mon[35130]: from='osd.7 [v2:172.21.15.90:6816/236874962,v1:172.21.15.90:6817/236874962]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:17:22.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:21 smithi090 ceph-mon[35130]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T23:17:23.161 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:22 smithi097 ceph-mon[35562]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T23:17:23.161 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:22 smithi097 ceph-mon[35562]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T23:17:23.161 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:22 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:17:23.161 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:22 smithi097 ceph-mon[35562]: from='osd.7 [v2:172.21.15.90:6816/236874962,v1:172.21.15.90:6817/236874962]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:17:23.161 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:22 smithi097 ceph-mon[35562]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:17:23.163 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:17:23.189 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_2 2024-09-11T23:17:23.219 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:22 smithi090 ceph-mon[35130]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T23:17:23.219 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:22 smithi090 ceph-mon[35130]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T23:17:23.219 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:22 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:17:23.219 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:22 smithi090 ceph-mon[35130]: from='osd.7 [v2:172.21.15.90:6816/236874962,v1:172.21.15.90:6817/236874962]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:17:23.219 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:22 smithi090 ceph-mon[35130]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:17:23.308 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:22 smithi043 ceph-mon[30950]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T23:17:23.309 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:22 smithi043 ceph-mon[30950]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T23:17:23.309 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:22 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:17:23.309 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:22 smithi043 ceph-mon[30950]: from='osd.7 [v2:172.21.15.90:6816/236874962,v1:172.21.15.90:6817/236874962]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:17:23.309 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:22 smithi043 ceph-mon[30950]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:17:23.489 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:17:23.520 INFO:journalctl@ceph.osd.7.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-7[52823]: 2024-09-11T23:17:23.266+0000 7f2e6b585640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T23:17:23.520 INFO:journalctl@ceph.osd.7.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-7[52823]: 2024-09-11T23:17:23.277+0000 7f2e633a5640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:17:24.003 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:23 smithi097 ceph-mon[35562]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:24.004 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:23 smithi097 ceph-mon[35562]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-11T23:17:24.004 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:23 smithi097 ceph-mon[35562]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T23:17:24.004 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:23 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:17:24.004 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:23 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:17:24.004 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:23 smithi097 ceph-mon[35562]: Detected new or changed devices on smithi090 2024-09-11T23:17:24.004 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:23 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:24.004 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:23 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:24.004 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:23 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:17:24.004 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:23 smithi097 ceph-mon[35562]: from='osd.7 ' entity='osd.7' 2024-09-11T23:17:24.004 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:23 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:17:24.005 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:23 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:17:24.005 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:23 smithi097 ceph-mon[35562]: Adjusting osd_memory_target on smithi090 to 5696M 2024-09-11T23:17:24.005 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:23 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:24.005 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:23 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:24.005 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:23 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:17:24.005 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:23 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:24.005 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:23 smithi097 ceph-mon[35562]: osd.7 [v2:172.21.15.90:6816/236874962,v1:172.21.15.90:6817/236874962] boot 2024-09-11T23:17:24.005 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:23 smithi097 ceph-mon[35562]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T23:17:24.005 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:23 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:17:24.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.998+0000 7ff87f1e7640 1 -- 172.21.15.97:0/2283554190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff878101d10 msgr2=0x7ff878102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:24.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.998+0000 7ff87f1e7640 1 --2- 172.21.15.97:0/2283554190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff878101d10 0x7ff878102190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff864009900 tx=0x7ff86402a410 comp rx=0 tx=0).stop 2024-09-11T23:17:24.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.998+0000 7ff87f1e7640 1 -- 172.21.15.97:0/2283554190 shutdown_connections 2024-09-11T23:17:24.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.998+0000 7ff87f1e7640 1 --2- 172.21.15.97:0/2283554190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8781026d0 0x7ff878108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:24.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.998+0000 7ff87f1e7640 1 --2- 172.21.15.97:0/2283554190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff878101d10 0x7ff878102190 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:24.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.998+0000 7ff87f1e7640 1 --2- 172.21.15.97:0/2283554190 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff878100b10 0x7ff878100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:24.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.998+0000 7ff87f1e7640 1 -- 172.21.15.97:0/2283554190 >> 172.21.15.97:0/2283554190 conn(0x7ff8780fc2a0 msgr2=0x7ff8780fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:24.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.998+0000 7ff87f1e7640 1 -- 172.21.15.97:0/2283554190 shutdown_connections 2024-09-11T23:17:24.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.998+0000 7ff87f1e7640 1 -- 172.21.15.97:0/2283554190 wait complete. 2024-09-11T23:17:24.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.999+0000 7ff87f1e7640 1 Processor -- start 2024-09-11T23:17:24.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.999+0000 7ff87f1e7640 1 -- start start 2024-09-11T23:17:24.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.999+0000 7ff87f1e7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff878100b10 0x7ff878104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:24.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.999+0000 7ff87f1e7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff878101d10 0x7ff8781030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:24.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.999+0000 7ff87f1e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8781026d0 0x7ff878103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:24.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.999+0000 7ff87f1e7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff87810c790 con 0x7ff8781026d0 2024-09-11T23:17:24.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.999+0000 7ff87f1e7640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff87810c610 con 0x7ff878101d10 2024-09-11T23:17:24.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.999+0000 7ff87f1e7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff87810c910 con 0x7ff878100b10 2024-09-11T23:17:24.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.999+0000 7ff87cf83640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff878100b10 0x7ff878104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:24.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.999+0000 7ff87cf83640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff878100b10 0x7ff878104aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35038/0 (socket says 172.21.15.97:35038) 2024-09-11T23:17:24.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.999+0000 7ff87cf83640 1 -- 172.21.15.97:0/1946250718 learned_addr learned my addr 172.21.15.97:0/1946250718 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-11T23:17:24.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.999+0000 7ff87d784640 1 --2- 172.21.15.97:0/1946250718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8781026d0 0x7ff878103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:24.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.999+0000 7ff877fff640 1 --2- 172.21.15.97:0/1946250718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff878101d10 0x7ff8781030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:24.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.999+0000 7ff87cf83640 1 -- 172.21.15.97:0/1946250718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff878101d10 msgr2=0x7ff8781030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:24.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.999+0000 7ff87cf83640 1 --2- 172.21.15.97:0/1946250718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff878101d10 0x7ff8781030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:24.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.999+0000 7ff87cf83640 1 -- 172.21.15.97:0/1946250718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8781026d0 msgr2=0x7ff878103630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:24.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.999+0000 7ff87cf83640 1 --2- 172.21.15.97:0/1946250718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8781026d0 0x7ff878103630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:24.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:23.999+0000 7ff87cf83640 1 -- 172.21.15.97:0/1946250718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff878103ef0 con 0x7ff878100b10 2024-09-11T23:17:24.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:24.000+0000 7ff87cf83640 1 --2- 172.21.15.97:0/1946250718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff878100b10 0x7ff878104aa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00d950 tx=0x7ff86c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:24.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:24.000+0000 7ff875ffb640 1 -- 172.21.15.97:0/1946250718 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c017070 con 0x7ff878100b10 2024-09-11T23:17:24.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:24.000+0000 7ff87f1e7640 1 -- 172.21.15.97:0/1946250718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8781a20e0 con 0x7ff878100b10 2024-09-11T23:17:24.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:24.000+0000 7ff875ffb640 1 -- 172.21.15.97:0/1946250718 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff86c009c30 con 0x7ff878100b10 2024-09-11T23:17:24.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:24.000+0000 7ff875ffb640 1 -- 172.21.15.97:0/1946250718 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c00f600 con 0x7ff878100b10 2024-09-11T23:17:24.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:24.000+0000 7ff87f1e7640 1 -- 172.21.15.97:0/1946250718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8781a25a0 con 0x7ff878100b10 2024-09-11T23:17:24.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:24.001+0000 7ff87f1e7640 1 -- 172.21.15.97:0/1946250718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8781a27d0 con 0x7ff878100b10 2024-09-11T23:17:24.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:24.002+0000 7ff875ffb640 1 -- 172.21.15.97:0/1946250718 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff86c01a070 con 0x7ff878100b10 2024-09-11T23:17:24.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:24.002+0000 7ff875ffb640 1 --2- 172.21.15.97:0/1946250718 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff8680a2b80 0x7ff8680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:24.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:24.003+0000 7ff875ffb640 1 -- 172.21.15.97:0/1946250718 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5430+0+0 (secure 0 0 0) 0x7ff86c0cdef0 con 0x7ff878100b10 2024-09-11T23:17:24.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:24.003+0000 7ff877fff640 1 --2- 172.21.15.97:0/1946250718 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff8680a2b80 0x7ff8680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:24.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:24.003+0000 7ff877fff640 1 --2- 172.21.15.97:0/1946250718 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff8680a2b80 0x7ff8680a5040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff8640314e0 tx=0x7ff864000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:24.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:24.007+0000 7ff875ffb640 1 -- 172.21.15.97:0/1946250718 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff86c009de0 con 0x7ff878100b10 2024-09-11T23:17:24.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:24.183+0000 7ff87f1e7640 1 -- 172.21.15.97:0/1946250718 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff87806af90 con 0x7ff8680a2b80 2024-09-11T23:17:24.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-mon[35130]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:24.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-mon[35130]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-11T23:17:24.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-mon[35130]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T23:17:24.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:17:24.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:17:24.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-mon[35130]: Detected new or changed devices on smithi090 2024-09-11T23:17:24.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:24.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:24.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:17:24.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-mon[35130]: from='osd.7 ' entity='osd.7' 2024-09-11T23:17:24.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:17:24.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:17:24.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-mon[35130]: Adjusting osd_memory_target on smithi090 to 5696M 2024-09-11T23:17:24.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:24.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:24.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:17:24.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:24.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-mon[35130]: osd.7 [v2:172.21.15.90:6816/236874962,v1:172.21.15.90:6817/236874962] boot 2024-09-11T23:17:24.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-mon[35130]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T23:17:24.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:23 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:17:24.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:23 smithi043 ceph-mon[30950]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T23:17:24.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:23 smithi043 ceph-mon[30950]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-11T23:17:24.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:23 smithi043 ceph-mon[30950]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T23:17:24.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:17:24.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:17:24.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:23 smithi043 ceph-mon[30950]: Detected new or changed devices on smithi090 2024-09-11T23:17:24.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:24.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:24.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:17:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:23 smithi043 ceph-mon[30950]: from='osd.7 ' entity='osd.7' 2024-09-11T23:17:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:17:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:17:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:23 smithi043 ceph-mon[30950]: Adjusting osd_memory_target on smithi090 to 5696M 2024-09-11T23:17:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:17:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:23 smithi043 ceph-mon[30950]: osd.7 [v2:172.21.15.90:6816/236874962,v1:172.21.15.90:6817/236874962] boot 2024-09-11T23:17:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:23 smithi043 ceph-mon[30950]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T23:17:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:17:25.255 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:24 smithi097 ceph-mon[35562]: purged_snaps scrub starts 2024-09-11T23:17:25.255 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:24 smithi097 ceph-mon[35562]: purged_snaps scrub ok 2024-09-11T23:17:25.255 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:24 smithi097 ceph-mon[35562]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:17:25.255 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:24 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:17:25.255 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:24 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:17:25.255 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:24 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:25.255 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:24 smithi097 ceph-mon[35562]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T23:17:25.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:24 smithi090 ceph-mon[35130]: purged_snaps scrub starts 2024-09-11T23:17:25.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:24 smithi090 ceph-mon[35130]: purged_snaps scrub ok 2024-09-11T23:17:25.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:24 smithi090 ceph-mon[35130]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:17:25.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:24 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:17:25.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:24 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:17:25.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:24 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:25.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:24 smithi090 ceph-mon[35130]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T23:17:25.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:24 smithi043 ceph-mon[30950]: purged_snaps scrub starts 2024-09-11T23:17:25.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:24 smithi043 ceph-mon[30950]: purged_snaps scrub ok 2024-09-11T23:17:25.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:24 smithi043 ceph-mon[30950]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:17:25.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:24 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:17:25.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:24 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:17:25.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:24 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:25.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:24 smithi043 ceph-mon[30950]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T23:17:26.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:26 smithi097 ceph-mon[35562]: pgmap v172: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:26.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:26 smithi097 ceph-mon[35562]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T23:17:26.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:26 smithi090 ceph-mon[35130]: pgmap v172: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:26.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:26 smithi090 ceph-mon[35130]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T23:17:26.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:26 smithi043 ceph-mon[30950]: pgmap v172: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:26.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:26 smithi043 ceph-mon[30950]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T23:17:27.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:27 smithi090 ceph-mon[35130]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T23:17:27.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:27 smithi090 ceph-mon[35130]: from='client.? 172.21.15.97:0/2484886931' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abef8f27-bf9b-4d63-bfbe-f967fc3d8b17"}]: dispatch 2024-09-11T23:17:27.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:27 smithi090 ceph-mon[35130]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abef8f27-bf9b-4d63-bfbe-f967fc3d8b17"}]: dispatch 2024-09-11T23:17:27.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:27 smithi090 ceph-mon[35130]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "abef8f27-bf9b-4d63-bfbe-f967fc3d8b17"}]': finished 2024-09-11T23:17:27.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:27 smithi090 ceph-mon[35130]: osdmap e53: 9 total, 8 up, 9 in 2024-09-11T23:17:27.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:27 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:17:27.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:27 smithi090 ceph-mon[35130]: from='client.? 172.21.15.97:0/1862087096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:17:27.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:27 smithi097 ceph-mon[35562]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T23:17:27.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:27 smithi097 ceph-mon[35562]: from='client.? 172.21.15.97:0/2484886931' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abef8f27-bf9b-4d63-bfbe-f967fc3d8b17"}]: dispatch 2024-09-11T23:17:27.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:27 smithi097 ceph-mon[35562]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abef8f27-bf9b-4d63-bfbe-f967fc3d8b17"}]: dispatch 2024-09-11T23:17:27.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:27 smithi097 ceph-mon[35562]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "abef8f27-bf9b-4d63-bfbe-f967fc3d8b17"}]': finished 2024-09-11T23:17:27.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:27 smithi097 ceph-mon[35562]: osdmap e53: 9 total, 8 up, 9 in 2024-09-11T23:17:27.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:27 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:17:27.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:27 smithi097 ceph-mon[35562]: from='client.? 172.21.15.97:0/1862087096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:17:27.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:27 smithi043 ceph-mon[30950]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T23:17:27.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:27 smithi043 ceph-mon[30950]: from='client.? 172.21.15.97:0/2484886931' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abef8f27-bf9b-4d63-bfbe-f967fc3d8b17"}]: dispatch 2024-09-11T23:17:27.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:27 smithi043 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abef8f27-bf9b-4d63-bfbe-f967fc3d8b17"}]: dispatch 2024-09-11T23:17:27.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:27 smithi043 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "abef8f27-bf9b-4d63-bfbe-f967fc3d8b17"}]': finished 2024-09-11T23:17:27.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:27 smithi043 ceph-mon[30950]: osdmap e53: 9 total, 8 up, 9 in 2024-09-11T23:17:27.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:27 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:17:27.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:27 smithi043 ceph-mon[30950]: from='client.? 172.21.15.97:0/1862087096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:17:28.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:28 smithi043 ceph-mon[30950]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:29.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:28 smithi090 ceph-mon[35130]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:29.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:28 smithi097 ceph-mon[35562]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:30.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:30 smithi043 ceph-mon[30950]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:31.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:30 smithi090 ceph-mon[35130]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:31.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:30 smithi097 ceph-mon[35562]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:32.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:32 smithi043 ceph-mon[30950]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:32.867 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:32 smithi097 ceph-mon[35562]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:33.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:32 smithi090 ceph-mon[35130]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:34.818 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:34 smithi097 ceph-mon[35562]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:34.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:34 smithi043 ceph-mon[30950]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:35.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:34 smithi090 ceph-mon[35130]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:35.735 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:35 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T23:17:35.735 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:35 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:35.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:35 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T23:17:35.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:35 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:36.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:35 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T23:17:36.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:35 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:36.779 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:36 smithi097 ceph-mon[35562]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:36.779 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:36 smithi097 ceph-mon[35562]: Deploying daemon osd.8 on smithi097 2024-09-11T23:17:36.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:36 smithi043 ceph-mon[30950]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:36.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:36 smithi043 ceph-mon[30950]: Deploying daemon osd.8 on smithi097 2024-09-11T23:17:37.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:36 smithi090 ceph-mon[35130]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:37.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:36 smithi090 ceph-mon[35130]: Deploying daemon osd.8 on smithi097 2024-09-11T23:17:38.700 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:38 smithi097 ceph-mon[35562]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:38.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:38 smithi043 ceph-mon[30950]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:39.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:38 smithi090 ceph-mon[35130]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:40.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:40 smithi097 ceph-mon[35562]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:40.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:40 smithi043 ceph-mon[30950]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:41.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:40 smithi090 ceph-mon[35130]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:41.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:17:41.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:41.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:41.870 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:17:41.870 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:41.870 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:42.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:17:42.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:42.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:42.850 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:42 smithi097 ceph-mon[35562]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:42.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:42 smithi043 ceph-mon[30950]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:42.921 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:42 smithi090 ceph-mon[35130]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:43.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:43 smithi043 ceph-mon[30950]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:43.933 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:43 smithi090 ceph-mon[35130]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:43.993 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:43 smithi097 ceph-mon[35562]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:45.009 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 8 on host 'smithi097' 2024-09-11T23:17:45.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:45.008+0000 7ff875ffb640 1 -- 172.21.15.97:0/1946250718 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff87806af90 con 0x7ff8680a2b80 2024-09-11T23:17:45.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:45.011+0000 7ff87f1e7640 1 -- 172.21.15.97:0/1946250718 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff8680a2b80 msgr2=0x7ff8680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:45.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:45.011+0000 7ff87f1e7640 1 --2- 172.21.15.97:0/1946250718 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff8680a2b80 0x7ff8680a5040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff8640314e0 tx=0x7ff864000bc0 comp rx=0 tx=0).stop 2024-09-11T23:17:45.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:45.011+0000 7ff87f1e7640 1 -- 172.21.15.97:0/1946250718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff878100b10 msgr2=0x7ff878104aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:45.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:45.011+0000 7ff87f1e7640 1 --2- 172.21.15.97:0/1946250718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff878100b10 0x7ff878104aa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00d950 tx=0x7ff86c00de10 comp rx=0 tx=0).stop 2024-09-11T23:17:45.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:45.011+0000 7ff87f1e7640 1 -- 172.21.15.97:0/1946250718 shutdown_connections 2024-09-11T23:17:45.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:45.011+0000 7ff87f1e7640 1 --2- 172.21.15.97:0/1946250718 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff8680a2b80 0x7ff8680a5040 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:45.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:45.011+0000 7ff87f1e7640 1 --2- 172.21.15.97:0/1946250718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8781026d0 0x7ff878103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:45.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:45.011+0000 7ff87f1e7640 1 --2- 172.21.15.97:0/1946250718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff878101d10 0x7ff8781030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:45.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:45.011+0000 7ff87f1e7640 1 --2- 172.21.15.97:0/1946250718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff878100b10 0x7ff878104aa0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:45.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:45.011+0000 7ff87f1e7640 1 -- 172.21.15.97:0/1946250718 >> 172.21.15.97:0/1946250718 conn(0x7ff8780fc2a0 msgr2=0x7ff87806a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:45.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:45.011+0000 7ff87f1e7640 1 -- 172.21.15.97:0/1946250718 shutdown_connections 2024-09-11T23:17:45.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:17:45.011+0000 7ff87f1e7640 1 -- 172.21.15.97:0/1946250718 wait complete. 2024-09-11T23:17:45.244 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:45 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:45.244 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:45 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:45.244 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:45 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:45.244 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:45 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:17:45.244 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:45 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:45.244 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:45 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:17:45.244 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:45 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:45.244 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:45 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:45.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:45 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:45.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:45 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:45.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:45 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:45.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:45 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:17:45.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:45 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:45.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:45 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:17:45.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:45 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:45.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:45 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:45.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:45 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:45.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:45 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:45.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:45 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:45.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:45 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:17:45.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:45 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:45.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:45 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:17:45.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:45 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:45.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:45 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:45.742 DEBUG:teuthology.orchestra.run.smithi097:osd.8> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.8.service 2024-09-11T23:17:45.744 INFO:tasks.cephadm:Deploying osd.9 on smithi043 with /dev/vg_nvme/lv_1... 2024-09-11T23:17:45.744 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T23:17:46.012 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:46.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:46 smithi097 ceph-mon[35562]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:46.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:46 smithi043 ceph-mon[30950]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:46.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:46 smithi090 ceph-mon[35130]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:46.997 INFO:journalctl@ceph.osd.8.smithi097.stdout:Sep 11 23:17:46 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-8[53913]: 2024-09-11T23:17:46.916+0000 7f3b7d488640 -1 osd.8 0 log_to_monitors true 2024-09-11T23:17:47.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:47 smithi097 ceph-mon[35562]: from='osd.8 [v2:172.21.15.97:6816/1707327301,v1:172.21.15.97:6817/1707327301]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:17:47.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:47 smithi097 ceph-mon[35562]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:17:47.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:47 smithi043 ceph-mon[30950]: from='osd.8 [v2:172.21.15.97:6816/1707327301,v1:172.21.15.97:6817/1707327301]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:17:47.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:47 smithi043 ceph-mon[30950]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:17:47.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:47 smithi090 ceph-mon[35130]: from='osd.8 [v2:172.21.15.97:6816/1707327301,v1:172.21.15.97:6817/1707327301]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:17:47.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:47 smithi090 ceph-mon[35130]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T23:17:48.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:48 smithi043 ceph-mon[30950]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:48.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:48 smithi043 ceph-mon[30950]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T23:17:48.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:48 smithi043 ceph-mon[30950]: osdmap e54: 9 total, 8 up, 9 in 2024-09-11T23:17:48.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:48 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:17:48.082 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:48 smithi043 ceph-mon[30950]: from='osd.8 [v2:172.21.15.97:6816/1707327301,v1:172.21.15.97:6817/1707327301]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:17:48.082 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:48 smithi043 ceph-mon[30950]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:17:48.421 INFO:journalctl@ceph.osd.8.smithi097.stdout:Sep 11 23:17:48 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-8[53913]: 2024-09-11T23:17:48.063+0000 7f3b7940b640 -1 osd.8 0 waiting for initial osdmap 2024-09-11T23:17:48.421 INFO:journalctl@ceph.osd.8.smithi097.stdout:Sep 11 23:17:48 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-8[53913]: 2024-09-11T23:17:48.072+0000 7f3b75233640 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:17:48.422 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:48 smithi097 ceph-mon[35562]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:48.422 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:48 smithi097 ceph-mon[35562]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T23:17:48.422 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:48 smithi097 ceph-mon[35562]: osdmap e54: 9 total, 8 up, 9 in 2024-09-11T23:17:48.422 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:48 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:17:48.422 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:48 smithi097 ceph-mon[35562]: from='osd.8 [v2:172.21.15.97:6816/1707327301,v1:172.21.15.97:6817/1707327301]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:17:48.423 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:48 smithi097 ceph-mon[35562]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:17:48.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:48 smithi090 ceph-mon[35130]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:48.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:48 smithi090 ceph-mon[35130]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T23:17:48.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:48 smithi090 ceph-mon[35130]: osdmap e54: 9 total, 8 up, 9 in 2024-09-11T23:17:48.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:48 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:17:48.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:48 smithi090 ceph-mon[35130]: from='osd.8 [v2:172.21.15.97:6816/1707327301,v1:172.21.15.97:6817/1707327301]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:17:48.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:48 smithi090 ceph-mon[35130]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:17:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:17:49.182 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_1 2024-09-11T23:17:49.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:49 smithi043 ceph-mon[30950]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-11T23:17:49.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:49 smithi043 ceph-mon[30950]: osdmap e55: 9 total, 8 up, 9 in 2024-09-11T23:17:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:17:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:17:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:49 smithi043 ceph-mon[30950]: osd.8 [v2:172.21.15.97:6816/1707327301,v1:172.21.15.97:6817/1707327301] boot 2024-09-11T23:17:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:49 smithi043 ceph-mon[30950]: osdmap e56: 9 total, 9 up, 9 in 2024-09-11T23:17:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:17:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:17:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:17:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:17:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:17:49.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:17:49.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:49 smithi090 ceph-mon[35130]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-11T23:17:49.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:49 smithi090 ceph-mon[35130]: osdmap e55: 9 total, 8 up, 9 in 2024-09-11T23:17:49.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:17:49.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:17:49.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:49 smithi090 ceph-mon[35130]: osd.8 [v2:172.21.15.97:6816/1707327301,v1:172.21.15.97:6817/1707327301] boot 2024-09-11T23:17:49.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:49 smithi090 ceph-mon[35130]: osdmap e56: 9 total, 9 up, 9 in 2024-09-11T23:17:49.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:17:49.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:49.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:49.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:17:49.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:17:49.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:17:49.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:49.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:49.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:17:49.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:49.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:49 smithi097 ceph-mon[35562]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-11T23:17:49.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:49 smithi097 ceph-mon[35562]: osdmap e55: 9 total, 8 up, 9 in 2024-09-11T23:17:49.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:17:49.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:17:49.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:49 smithi097 ceph-mon[35562]: osd.8 [v2:172.21.15.97:6816/1707327301,v1:172.21.15.97:6817/1707327301] boot 2024-09-11T23:17:49.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:49 smithi097 ceph-mon[35562]: osdmap e56: 9 total, 9 up, 9 in 2024-09-11T23:17:49.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:17:49.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:49.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:49.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:17:49.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:17:49.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:17:49.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:49.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:49.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:17:49.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:17:50.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.058+0000 7ff3c0c3b640 1 -- 172.21.15.43:0/1983854218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3bc101d10 msgr2=0x7ff3bc102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:50.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.058+0000 7ff3c0c3b640 1 --2- 172.21.15.43:0/1983854218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3bc101d10 0x7ff3bc102190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009900 tx=0x7ff3b002a3f0 comp rx=0 tx=0).stop 2024-09-11T23:17:50.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.058+0000 7ff3c0c3b640 1 -- 172.21.15.43:0/1983854218 shutdown_connections 2024-09-11T23:17:50.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.058+0000 7ff3c0c3b640 1 --2- 172.21.15.43:0/1983854218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3bc1026d0 0x7ff3bc108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:50.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.058+0000 7ff3c0c3b640 1 --2- 172.21.15.43:0/1983854218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3bc101d10 0x7ff3bc102190 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:50.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.058+0000 7ff3c0c3b640 1 --2- 172.21.15.43:0/1983854218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3bc100b10 0x7ff3bc100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:50.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.058+0000 7ff3c0c3b640 1 -- 172.21.15.43:0/1983854218 >> 172.21.15.43:0/1983854218 conn(0x7ff3bc0fc2a0 msgr2=0x7ff3bc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:17:50.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.058+0000 7ff3c0c3b640 1 -- 172.21.15.43:0/1983854218 shutdown_connections 2024-09-11T23:17:50.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.058+0000 7ff3c0c3b640 1 -- 172.21.15.43:0/1983854218 wait complete. 2024-09-11T23:17:50.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.059+0000 7ff3c0c3b640 1 Processor -- start 2024-09-11T23:17:50.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.059+0000 7ff3c0c3b640 1 -- start start 2024-09-11T23:17:50.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.059+0000 7ff3c0c3b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3bc100b10 0x7ff3bc197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:50.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.059+0000 7ff3c0c3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3bc101d10 0x7ff3bc198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.059+0000 7ff3c0c3b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3bc1026d0 0x7ff3bc19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.059+0000 7ff3c0c3b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc10c770 con 0x7ff3bc101d10 2024-09-11T23:17:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.059+0000 7ff3c0c3b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc10c5f0 con 0x7ff3bc100b10 2024-09-11T23:17:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.059+0000 7ff3c0c3b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc10c8f0 con 0x7ff3bc1026d0 2024-09-11T23:17:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.059+0000 7ff3b9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3bc101d10 0x7ff3bc198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.059+0000 7ff3b9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3bc101d10 0x7ff3bc198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35660/0 (socket says 172.21.15.43:35660) 2024-09-11T23:17:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.059+0000 7ff3b9d9b640 1 -- 172.21.15.43:0/147435590 learned_addr learned my addr 172.21.15.43:0/147435590 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:17:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.060+0000 7ff3ba59c640 1 --2- 172.21.15.43:0/147435590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3bc100b10 0x7ff3bc197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.060+0000 7ff3bad9d640 1 --2- 172.21.15.43:0/147435590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3bc1026d0 0x7ff3bc19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.060+0000 7ff3b9d9b640 1 -- 172.21.15.43:0/147435590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3bc1026d0 msgr2=0x7ff3bc19f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.060+0000 7ff3b9d9b640 1 --2- 172.21.15.43:0/147435590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3bc1026d0 0x7ff3bc19f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.060+0000 7ff3b9d9b640 1 -- 172.21.15.43:0/147435590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3bc100b10 msgr2=0x7ff3bc197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:17:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.060+0000 7ff3b9d9b640 1 --2- 172.21.15.43:0/147435590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3bc100b10 0x7ff3bc197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:17:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.060+0000 7ff3b9d9b640 1 -- 172.21.15.43:0/147435590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3bc19fa40 con 0x7ff3bc101d10 2024-09-11T23:17:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.060+0000 7ff3b9d9b640 1 --2- 172.21.15.43:0/147435590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3bc101d10 0x7ff3bc198350 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009230 tx=0x7ff3b0004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.060+0000 7ff3ab7fe640 1 -- 172.21.15.43:0/147435590 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b003f030 con 0x7ff3bc101d10 2024-09-11T23:17:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.060+0000 7ff3ab7fe640 1 -- 172.21.15.43:0/147435590 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b0003ce0 con 0x7ff3bc101d10 2024-09-11T23:17:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.060+0000 7ff3c0c3b640 1 -- 172.21.15.43:0/147435590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3bc19fcd0 con 0x7ff3bc101d10 2024-09-11T23:17:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.060+0000 7ff3ab7fe640 1 -- 172.21.15.43:0/147435590 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0035070 con 0x7ff3bc101d10 2024-09-11T23:17:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.060+0000 7ff3c0c3b640 1 -- 172.21.15.43:0/147435590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3bc1a01b0 con 0x7ff3bc101d10 2024-09-11T23:17:50.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.061+0000 7ff3c0c3b640 1 -- 172.21.15.43:0/147435590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3bc1a0380 con 0x7ff3bc101d10 2024-09-11T23:17:50.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.062+0000 7ff3ab7fe640 1 -- 172.21.15.43:0/147435590 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff3b0042030 con 0x7ff3bc101d10 2024-09-11T23:17:50.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.063+0000 7ff3ab7fe640 1 --2- 172.21.15.43:0/147435590 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff3a40a2bb0 0x7ff3a40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:17:50.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.063+0000 7ff3ba59c640 1 --2- 172.21.15.43:0/147435590 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff3a40a2bb0 0x7ff3a40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:17:50.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.063+0000 7ff3ab7fe640 1 -- 172.21.15.43:0/147435590 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5858+0+0 (secure 0 0 0) 0x7ff3b00eede0 con 0x7ff3bc101d10 2024-09-11T23:17:50.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.064+0000 7ff3ba59c640 1 --2- 172.21.15.43:0/147435590 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff3a40a2bb0 0x7ff3a40a5070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc101b70 tx=0x7ff3ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:17:50.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.067+0000 7ff3ab7fe640 1 -- 172.21.15.43:0/147435590 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff3b00bc6d0 con 0x7ff3bc101d10 2024-09-11T23:17:50.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:17:50.251+0000 7ff3c0c3b640 1 -- 172.21.15.43:0/147435590 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff3bc0fe490 con 0x7ff3a40a2bb0 2024-09-11T23:17:50.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:50 smithi043 ceph-mon[30950]: purged_snaps scrub starts 2024-09-11T23:17:50.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:50 smithi043 ceph-mon[30950]: purged_snaps scrub ok 2024-09-11T23:17:50.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:50 smithi043 ceph-mon[30950]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:50.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:50 smithi043 ceph-mon[30950]: Detected new or changed devices on smithi097 2024-09-11T23:17:50.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:50 smithi043 ceph-mon[30950]: Adjusting osd_memory_target on smithi097 to 5696M 2024-09-11T23:17:50.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:50 smithi043 ceph-mon[30950]: osdmap e57: 9 total, 9 up, 9 in 2024-09-11T23:17:50.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:50 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:17:50.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:50 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:17:50.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:50 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:50 smithi090 ceph-mon[35130]: purged_snaps scrub starts 2024-09-11T23:17:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:50 smithi090 ceph-mon[35130]: purged_snaps scrub ok 2024-09-11T23:17:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:50 smithi090 ceph-mon[35130]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:50 smithi090 ceph-mon[35130]: Detected new or changed devices on smithi097 2024-09-11T23:17:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:50 smithi090 ceph-mon[35130]: Adjusting osd_memory_target on smithi097 to 5696M 2024-09-11T23:17:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:50 smithi090 ceph-mon[35130]: osdmap e57: 9 total, 9 up, 9 in 2024-09-11T23:17:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:50 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:17:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:50 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:17:50.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:50 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:50.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:50 smithi097 ceph-mon[35562]: purged_snaps scrub starts 2024-09-11T23:17:50.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:50 smithi097 ceph-mon[35562]: purged_snaps scrub ok 2024-09-11T23:17:50.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:50 smithi097 ceph-mon[35562]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T23:17:50.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:50 smithi097 ceph-mon[35562]: Detected new or changed devices on smithi097 2024-09-11T23:17:50.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:50 smithi097 ceph-mon[35562]: Adjusting osd_memory_target on smithi097 to 5696M 2024-09-11T23:17:50.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:50 smithi097 ceph-mon[35562]: osdmap e57: 9 total, 9 up, 9 in 2024-09-11T23:17:50.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:50 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:17:50.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:50 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:17:50.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:50 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:17:51.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:51 smithi090 ceph-mon[35130]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:17:51.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:51 smithi090 ceph-mon[35130]: osdmap e58: 9 total, 9 up, 9 in 2024-09-11T23:17:51.797 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:51 smithi043 ceph-mon[30950]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:17:51.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:51 smithi043 ceph-mon[30950]: osdmap e58: 9 total, 9 up, 9 in 2024-09-11T23:17:51.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:51 smithi097 ceph-mon[35562]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:17:51.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:51 smithi097 ceph-mon[35562]: osdmap e58: 9 total, 9 up, 9 in 2024-09-11T23:17:52.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:52 smithi043 ceph-mon[30950]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:17:52.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:52 smithi043 ceph-mon[30950]: osdmap e59: 9 total, 9 up, 9 in 2024-09-11T23:17:52.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:52 smithi090 ceph-mon[35130]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:17:52.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:52 smithi090 ceph-mon[35130]: osdmap e59: 9 total, 9 up, 9 in 2024-09-11T23:17:52.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:52 smithi097 ceph-mon[35562]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:17:52.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:52 smithi097 ceph-mon[35562]: osdmap e59: 9 total, 9 up, 9 in 2024-09-11T23:17:53.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:53 smithi090 ceph-mon[35130]: osdmap e60: 9 total, 9 up, 9 in 2024-09-11T23:17:53.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:53 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/173440308' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "190ab8b2-a835-47e6-9b39-16a58331c58a"}]: dispatch 2024-09-11T23:17:53.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:53 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/173440308' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "190ab8b2-a835-47e6-9b39-16a58331c58a"}]': finished 2024-09-11T23:17:53.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:53 smithi090 ceph-mon[35130]: osdmap e61: 10 total, 9 up, 10 in 2024-09-11T23:17:53.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:53 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:17:53.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:53 smithi097 ceph-mon[35562]: osdmap e60: 9 total, 9 up, 9 in 2024-09-11T23:17:53.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:53 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/173440308' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "190ab8b2-a835-47e6-9b39-16a58331c58a"}]: dispatch 2024-09-11T23:17:53.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:53 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/173440308' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "190ab8b2-a835-47e6-9b39-16a58331c58a"}]': finished 2024-09-11T23:17:53.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:53 smithi097 ceph-mon[35562]: osdmap e61: 10 total, 9 up, 10 in 2024-09-11T23:17:53.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:53 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:17:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:53 smithi043 ceph-mon[30950]: osdmap e60: 9 total, 9 up, 9 in 2024-09-11T23:17:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:53 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/173440308' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "190ab8b2-a835-47e6-9b39-16a58331c58a"}]: dispatch 2024-09-11T23:17:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:53 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/173440308' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "190ab8b2-a835-47e6-9b39-16a58331c58a"}]': finished 2024-09-11T23:17:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:53 smithi043 ceph-mon[30950]: osdmap e61: 10 total, 9 up, 10 in 2024-09-11T23:17:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:53 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:17:54.651 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:54 smithi090 ceph-mon[35130]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:17:54.651 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:54 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/585030557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:17:54.722 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:54 smithi097 ceph-mon[35562]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:17:54.722 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:54 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/585030557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:17:54.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:54 smithi043 ceph-mon[30950]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:17:54.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:54 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/585030557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:17:56.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:56 smithi090 ceph-mon[35130]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-11T23:17:56.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:56 smithi097 ceph-mon[35562]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-11T23:17:56.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:56 smithi043 ceph-mon[30950]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-11T23:17:58.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:17:58 smithi043 ceph-mon[30950]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T23:17:58.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:17:58 smithi090 ceph-mon[35130]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T23:17:58.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:17:58 smithi097 ceph-mon[35562]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T23:18:00.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:00 smithi043 ceph-mon[30950]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-11T23:18:00.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:00 smithi090 ceph-mon[35130]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-11T23:18:00.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:00 smithi097 ceph-mon[35562]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-11T23:18:01.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:01 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T23:18:01.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:01 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:01.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:01 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T23:18:01.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:01 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:01 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T23:18:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:01 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:02.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:02 smithi043 ceph-mon[30950]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-11T23:18:02.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:02 smithi043 ceph-mon[30950]: Deploying daemon osd.9 on smithi043 2024-09-11T23:18:02.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:02 smithi090 ceph-mon[35130]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-11T23:18:02.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:02 smithi090 ceph-mon[35130]: Deploying daemon osd.9 on smithi043 2024-09-11T23:18:02.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:02 smithi097 ceph-mon[35562]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-11T23:18:02.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:02 smithi097 ceph-mon[35562]: Deploying daemon osd.9 on smithi043 2024-09-11T23:18:04.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:04 smithi043 ceph-mon[30950]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T23:18:04.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:04 smithi090 ceph-mon[35130]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T23:18:04.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:04 smithi097 ceph-mon[35562]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T23:18:06.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:06 smithi043 ceph-mon[30950]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-11T23:18:06.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:06 smithi090 ceph-mon[35130]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-11T23:18:06.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:06 smithi097 ceph-mon[35562]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-11T23:18:07.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:07.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:07.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:07.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:07.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:07.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:07.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:07.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:07.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:08.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:08 smithi090 ceph-mon[35130]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:18:08.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:08 smithi097 ceph-mon[35562]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:18:08.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:08 smithi043 ceph-mon[30950]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:18:10.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:10 smithi090 ceph-mon[35130]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:18:10.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:10 smithi097 ceph-mon[35562]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:18:10.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:10 smithi043 ceph-mon[30950]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:18:11.316 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 9 on host 'smithi043' 2024-09-11T23:18:11.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:18:11.315+0000 7ff3ab7fe640 1 -- 172.21.15.43:0/147435590 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff3bc0fe490 con 0x7ff3a40a2bb0 2024-09-11T23:18:11.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:18:11.318+0000 7ff3c0c3b640 1 -- 172.21.15.43:0/147435590 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff3a40a2bb0 msgr2=0x7ff3a40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:11.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:18:11.318+0000 7ff3c0c3b640 1 --2- 172.21.15.43:0/147435590 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff3a40a2bb0 0x7ff3a40a5070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc101b70 tx=0x7ff3ac005f90 comp rx=0 tx=0).stop 2024-09-11T23:18:11.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:18:11.318+0000 7ff3c0c3b640 1 -- 172.21.15.43:0/147435590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3bc101d10 msgr2=0x7ff3bc198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:11.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:18:11.318+0000 7ff3c0c3b640 1 --2- 172.21.15.43:0/147435590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3bc101d10 0x7ff3bc198350 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009230 tx=0x7ff3b0004430 comp rx=0 tx=0).stop 2024-09-11T23:18:11.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:18:11.318+0000 7ff3c0c3b640 1 -- 172.21.15.43:0/147435590 shutdown_connections 2024-09-11T23:18:11.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:18:11.318+0000 7ff3c0c3b640 1 --2- 172.21.15.43:0/147435590 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff3a40a2bb0 0x7ff3a40a5070 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:18:11.318+0000 7ff3c0c3b640 1 --2- 172.21.15.43:0/147435590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3bc1026d0 0x7ff3bc19f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:18:11.318+0000 7ff3c0c3b640 1 --2- 172.21.15.43:0/147435590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3bc101d10 0x7ff3bc198350 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:18:11.318+0000 7ff3c0c3b640 1 --2- 172.21.15.43:0/147435590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3bc100b10 0x7ff3bc197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:11.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:18:11.318+0000 7ff3c0c3b640 1 -- 172.21.15.43:0/147435590 >> 172.21.15.43:0/147435590 conn(0x7ff3bc0fc2a0 msgr2=0x7ff3bc0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:11.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:18:11.318+0000 7ff3c0c3b640 1 -- 172.21.15.43:0/147435590 shutdown_connections 2024-09-11T23:18:11.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:18:11.318+0000 7ff3c0c3b640 1 -- 172.21.15.43:0/147435590 wait complete. 2024-09-11T23:18:11.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:11 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:11.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:11 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:11.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:11 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:11.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:11 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:11.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:11 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:11.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:11 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:11.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:11 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:11.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:11 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:11.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:11 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:11.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:11 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:11.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:11 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:11.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:11 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:11.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:11 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:11.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:11 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:11.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:11 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:11.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:11 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:11.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:11 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:11.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:11 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:11.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:11 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:11.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:11 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:11.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:11 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:11.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:11 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:11.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:11 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:11.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:11 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:12.025 DEBUG:teuthology.orchestra.run.smithi043:osd.9> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.9.service 2024-09-11T23:18:12.027 INFO:tasks.cephadm:Deploying osd.10 on smithi090 with /dev/vg_nvme/lv_1... 2024-09-11T23:18:12.027 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T23:18:12.282 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:18:12.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:12 smithi043 ceph-mon[30950]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:18:12.653 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:12 smithi090 ceph-mon[35130]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:18:12.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:12 smithi097 ceph-mon[35562]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:18:13.518 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:18:13 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9[65131]: 2024-09-11T23:18:13.389+0000 7f05fb868640 -1 osd.9 0 log_to_monitors true 2024-09-11T23:18:13.722 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:13 smithi090 ceph-mon[35130]: from='osd.9 [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T23:18:13.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:13 smithi097 ceph-mon[35562]: from='osd.9 [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T23:18:13.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:13 smithi043 ceph-mon[30950]: from='osd.9 [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T23:18:14.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:14 smithi090 ceph-mon[35130]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:18:14.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:14 smithi090 ceph-mon[35130]: from='osd.9 [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T23:18:14.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:14 smithi090 ceph-mon[35130]: osdmap e62: 10 total, 9 up, 10 in 2024-09-11T23:18:14.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:14 smithi090 ceph-mon[35130]: from='osd.9 [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T23:18:14.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:14 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:18:14.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:14 smithi097 ceph-mon[35562]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:18:14.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:14 smithi097 ceph-mon[35562]: from='osd.9 [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T23:18:14.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:14 smithi097 ceph-mon[35562]: osdmap e62: 10 total, 9 up, 10 in 2024-09-11T23:18:14.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:14 smithi097 ceph-mon[35562]: from='osd.9 [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T23:18:14.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:14 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:18:14.819 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:14 smithi043 ceph-mon[30950]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:18:14.820 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:14 smithi043 ceph-mon[30950]: from='osd.9 [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T23:18:14.820 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:14 smithi043 ceph-mon[30950]: osdmap e62: 10 total, 9 up, 10 in 2024-09-11T23:18:14.820 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:14 smithi043 ceph-mon[30950]: from='osd.9 [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-11T23:18:14.820 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:14 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:18:15.109 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-11T23:18:15.110 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:18:14 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9[65131]: 2024-09-11T23:18:14.947+0000 7f05f77eb640 -1 osd.9 0 waiting for initial osdmap 2024-09-11T23:18:15.110 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:18:14 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9[65131]: 2024-09-11T23:18:14.960+0000 7f05ef60b640 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:18:15.134 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_1 2024-09-11T23:18:15.387 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:18:15.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:15 smithi090 ceph-mon[35130]: from='osd.9 [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T23:18:15.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:15 smithi090 ceph-mon[35130]: osdmap e63: 10 total, 9 up, 10 in 2024-09-11T23:18:15.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:15 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:18:15.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:15 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:18:15.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:15 smithi090 ceph-mon[35130]: from='osd.9 [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011]' entity='osd.9' 2024-09-11T23:18:15.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:15 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:15.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:15 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:15.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:15 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:15.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:15 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:15.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:15 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:15.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:15 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:15.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:15 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:15.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:15 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:15.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:15 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:15.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:15 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:15.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:15 smithi097 ceph-mon[35562]: from='osd.9 [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T23:18:15.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:15 smithi097 ceph-mon[35562]: osdmap e63: 10 total, 9 up, 10 in 2024-09-11T23:18:15.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:15 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:18:15.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:15 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:18:15.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:15 smithi097 ceph-mon[35562]: from='osd.9 [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011]' entity='osd.9' 2024-09-11T23:18:15.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:15 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:15.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:15 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:15.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:15 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:15.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:15 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:15.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:15 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:15.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:15 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:15.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:15 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:15.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:15 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:15.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:15 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:15.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:15 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:15.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:15 smithi043 ceph-mon[30950]: from='osd.9 [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-11T23:18:15.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:15 smithi043 ceph-mon[30950]: osdmap e63: 10 total, 9 up, 10 in 2024-09-11T23:18:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:15 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:18:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:15 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:18:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:15 smithi043 ceph-mon[30950]: from='osd.9 [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011]' entity='osd.9' 2024-09-11T23:18:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:15 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:15 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:15 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:15 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:15 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:15 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:15 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:15 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:15 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:15.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:15 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:15.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.899+0000 7f4c75b01640 1 -- 172.21.15.90:0/2908442187 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c701037c0 msgr2=0x7f4c7010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:15.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.899+0000 7f4c75b01640 1 --2- 172.21.15.90:0/2908442187 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c701037c0 0x7f4c7010a050 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4c68009ca0 tx=0x7f4c6802a8c0 comp rx=0 tx=0).stop 2024-09-11T23:18:15.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.899+0000 7f4c75b01640 1 -- 172.21.15.90:0/2908442187 shutdown_connections 2024-09-11T23:18:15.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.899+0000 7f4c75b01640 1 --2- 172.21.15.90:0/2908442187 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c701037c0 0x7f4c7010a050 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.899+0000 7f4c75b01640 1 --2- 172.21.15.90:0/2908442187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c70102e00 0x7f4c70103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.899+0000 7f4c75b01640 1 --2- 172.21.15.90:0/2908442187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c70101c00 0x7f4c70102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.899+0000 7f4c75b01640 1 -- 172.21.15.90:0/2908442187 >> 172.21.15.90:0/2908442187 conn(0x7f4c700fd390 msgr2=0x7f4c700ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.899+0000 7f4c75b01640 1 -- 172.21.15.90:0/2908442187 shutdown_connections 2024-09-11T23:18:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.899+0000 7f4c75b01640 1 -- 172.21.15.90:0/2908442187 wait complete. 2024-09-11T23:18:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.900+0000 7f4c75b01640 1 Processor -- start 2024-09-11T23:18:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.900+0000 7f4c75b01640 1 -- start start 2024-09-11T23:18:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.900+0000 7f4c75b01640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c70101c00 0x7f4c70198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.900+0000 7f4c75b01640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c70102e00 0x7f4c70198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.900+0000 7f4c75b01640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c701037c0 0x7f4c7019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.900+0000 7f4c75b01640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7010d980 con 0x7f4c70102e00 2024-09-11T23:18:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.900+0000 7f4c75b01640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7010d800 con 0x7f4c701037c0 2024-09-11T23:18:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.900+0000 7f4c75b01640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7010db00 con 0x7f4c70101c00 2024-09-11T23:18:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.900+0000 7f4c6ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c701037c0 0x7f4c7019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.901+0000 7f4c6ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c701037c0 0x7f4c7019f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39644/0 (socket says 172.21.15.90:39644) 2024-09-11T23:18:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.901+0000 7f4c6ffff640 1 -- 172.21.15.90:0/1083458824 learned_addr learned my addr 172.21.15.90:0/1083458824 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-11T23:18:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.901+0000 7f4c6f7fe640 1 --2- 172.21.15.90:0/1083458824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c70101c00 0x7f4c70198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.901+0000 7f4c6ffff640 1 -- 172.21.15.90:0/1083458824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c70101c00 msgr2=0x7f4c70198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.901+0000 7f4c6ffff640 1 --2- 172.21.15.90:0/1083458824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c70101c00 0x7f4c70198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.901+0000 7f4c6ffff640 1 -- 172.21.15.90:0/1083458824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c70102e00 msgr2=0x7f4c70198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.901+0000 7f4c6effd640 1 --2- 172.21.15.90:0/1083458824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c70102e00 0x7f4c70198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.901+0000 7f4c6ffff640 1 --2- 172.21.15.90:0/1083458824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c70102e00 0x7f4c70198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.901+0000 7f4c6ffff640 1 -- 172.21.15.90:0/1083458824 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7019fc70 con 0x7f4c701037c0 2024-09-11T23:18:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.901+0000 7f4c6effd640 1 --2- 172.21.15.90:0/1083458824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c70102e00 0x7f4c70198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:18:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.901+0000 7f4c6ffff640 1 --2- 172.21.15.90:0/1083458824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c701037c0 0x7f4c7019f600 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4c680098d0 tx=0x7f4c6802f840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.901+0000 7f4c6cff9640 1 -- 172.21.15.90:0/1083458824 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6802fc30 con 0x7f4c701037c0 2024-09-11T23:18:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.901+0000 7f4c6cff9640 1 -- 172.21.15.90:0/1083458824 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c68003e50 con 0x7f4c701037c0 2024-09-11T23:18:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.901+0000 7f4c75b01640 1 -- 172.21.15.90:0/1083458824 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c7019ff00 con 0x7f4c701037c0 2024-09-11T23:18:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.901+0000 7f4c6cff9640 1 -- 172.21.15.90:0/1083458824 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c680305c0 con 0x7f4c701037c0 2024-09-11T23:18:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.901+0000 7f4c75b01640 1 -- 172.21.15.90:0/1083458824 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c701a0480 con 0x7f4c701037c0 2024-09-11T23:18:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.902+0000 7f4c75b01640 1 -- 172.21.15.90:0/1083458824 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c70109020 con 0x7f4c701037c0 2024-09-11T23:18:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.903+0000 7f4c6cff9640 1 -- 172.21.15.90:0/1083458824 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4c68004b40 con 0x7f4c701037c0 2024-09-11T23:18:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.904+0000 7f4c6cff9640 1 --2- 172.21.15.90:0/1083458824 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f4c580a2c30 0x7f4c580a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.904+0000 7f4c6cff9640 1 -- 172.21.15.90:0/1083458824 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5981+0+0 (secure 0 0 0) 0x7f4c680efce0 con 0x7f4c701037c0 2024-09-11T23:18:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.904+0000 7f4c6f7fe640 1 --2- 172.21.15.90:0/1083458824 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f4c580a2c30 0x7f4c580a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.905+0000 7f4c6f7fe640 1 --2- 172.21.15.90:0/1083458824 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f4c580a2c30 0x7f4c580a50f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4c70102c60 tx=0x7f4c60008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:15.908+0000 7f4c6cff9640 1 -- 172.21.15.90:0/1083458824 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4c680433d0 con 0x7f4c701037c0 2024-09-11T23:18:16.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:16.085+0000 7f4c75b01640 1 -- 172.21.15.90:0/1083458824 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4c70062630 con 0x7f4c580a2c30 2024-09-11T23:18:16.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:16 smithi090 ceph-mon[35130]: purged_snaps scrub starts 2024-09-11T23:18:16.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:16 smithi090 ceph-mon[35130]: purged_snaps scrub ok 2024-09-11T23:18:16.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:16 smithi090 ceph-mon[35130]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:18:16.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:16 smithi090 ceph-mon[35130]: Detected new or changed devices on smithi043 2024-09-11T23:18:16.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:16 smithi090 ceph-mon[35130]: Adjusting osd_memory_target on smithi043 to 4273M 2024-09-11T23:18:16.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:16 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:18:16.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:16 smithi090 ceph-mon[35130]: osd.9 [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011] boot 2024-09-11T23:18:16.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:16 smithi090 ceph-mon[35130]: osdmap e64: 10 total, 10 up, 10 in 2024-09-11T23:18:16.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:16 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:18:16.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:16 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:18:16.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:16 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:18:16.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:16 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:16.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:16 smithi097 ceph-mon[35562]: purged_snaps scrub starts 2024-09-11T23:18:16.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:16 smithi097 ceph-mon[35562]: purged_snaps scrub ok 2024-09-11T23:18:16.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:16 smithi097 ceph-mon[35562]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:18:16.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:16 smithi097 ceph-mon[35562]: Detected new or changed devices on smithi043 2024-09-11T23:18:16.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:16 smithi097 ceph-mon[35562]: Adjusting osd_memory_target on smithi043 to 4273M 2024-09-11T23:18:16.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:16 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:18:16.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:16 smithi097 ceph-mon[35562]: osd.9 [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011] boot 2024-09-11T23:18:16.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:16 smithi097 ceph-mon[35562]: osdmap e64: 10 total, 10 up, 10 in 2024-09-11T23:18:16.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:16 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:18:16.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:16 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:18:16.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:16 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:18:16.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:16 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:16.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:16 smithi043 ceph-mon[30950]: purged_snaps scrub starts 2024-09-11T23:18:16.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:16 smithi043 ceph-mon[30950]: purged_snaps scrub ok 2024-09-11T23:18:16.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:16 smithi043 ceph-mon[30950]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T23:18:16.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:16 smithi043 ceph-mon[30950]: Detected new or changed devices on smithi043 2024-09-11T23:18:16.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:16 smithi043 ceph-mon[30950]: Adjusting osd_memory_target on smithi043 to 4273M 2024-09-11T23:18:16.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:16 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:18:16.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:16 smithi043 ceph-mon[30950]: osd.9 [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011] boot 2024-09-11T23:18:16.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:16 smithi043 ceph-mon[30950]: osdmap e64: 10 total, 10 up, 10 in 2024-09-11T23:18:16.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:16 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:18:16.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:16 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:18:16.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:16 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:18:16.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:16 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:17.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:17 smithi090 ceph-mon[35130]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:18:17.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:17 smithi090 ceph-mon[35130]: osdmap e65: 10 total, 10 up, 10 in 2024-09-11T23:18:17.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:17 smithi097 ceph-mon[35562]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:18:17.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:17 smithi097 ceph-mon[35562]: osdmap e65: 10 total, 10 up, 10 in 2024-09-11T23:18:17.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:17 smithi043 ceph-mon[30950]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:18:17.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:17 smithi043 ceph-mon[30950]: osdmap e65: 10 total, 10 up, 10 in 2024-09-11T23:18:18.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:18 smithi097 ceph-mon[35562]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:18.811 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:18 smithi090 ceph-mon[35130]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:18.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:18 smithi043 ceph-mon[30950]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:19.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:19 smithi097 ceph-mon[35562]: from='client.? 172.21.15.90:0/2664681008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b32ab5fe-466f-4c8e-8696-cd9f820a3263"}]: dispatch 2024-09-11T23:18:19.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:19 smithi097 ceph-mon[35562]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b32ab5fe-466f-4c8e-8696-cd9f820a3263"}]: dispatch 2024-09-11T23:18:19.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:19 smithi097 ceph-mon[35562]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b32ab5fe-466f-4c8e-8696-cd9f820a3263"}]': finished 2024-09-11T23:18:19.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:19 smithi097 ceph-mon[35562]: osdmap e66: 11 total, 10 up, 11 in 2024-09-11T23:18:19.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:19 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:18:19.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:19 smithi097 ceph-mon[35562]: from='client.? 172.21.15.90:0/2032830189' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:18:19.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:19 smithi043 ceph-mon[30950]: from='client.? 172.21.15.90:0/2664681008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b32ab5fe-466f-4c8e-8696-cd9f820a3263"}]: dispatch 2024-09-11T23:18:19.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:19 smithi043 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b32ab5fe-466f-4c8e-8696-cd9f820a3263"}]: dispatch 2024-09-11T23:18:19.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:19 smithi043 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b32ab5fe-466f-4c8e-8696-cd9f820a3263"}]': finished 2024-09-11T23:18:19.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:19 smithi043 ceph-mon[30950]: osdmap e66: 11 total, 10 up, 11 in 2024-09-11T23:18:19.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:19 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:18:19.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:19 smithi043 ceph-mon[30950]: from='client.? 172.21.15.90:0/2032830189' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:18:20.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:19 smithi090 ceph-mon[35130]: from='client.? 172.21.15.90:0/2664681008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b32ab5fe-466f-4c8e-8696-cd9f820a3263"}]: dispatch 2024-09-11T23:18:20.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:19 smithi090 ceph-mon[35130]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b32ab5fe-466f-4c8e-8696-cd9f820a3263"}]: dispatch 2024-09-11T23:18:20.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:19 smithi090 ceph-mon[35130]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b32ab5fe-466f-4c8e-8696-cd9f820a3263"}]': finished 2024-09-11T23:18:20.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:19 smithi090 ceph-mon[35130]: osdmap e66: 11 total, 10 up, 11 in 2024-09-11T23:18:20.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:19 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:18:20.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:19 smithi090 ceph-mon[35130]: from='client.? 172.21.15.90:0/2032830189' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:18:20.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:20 smithi043 ceph-mon[30950]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:21.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:20 smithi090 ceph-mon[35130]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:21.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:20 smithi097 ceph-mon[35562]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:22.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:22 smithi043 ceph-mon[30950]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:23.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:22 smithi090 ceph-mon[35130]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:23.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:22 smithi097 ceph-mon[35562]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:24.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:24 smithi043 ceph-mon[30950]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:24.962 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:24 smithi090 ceph-mon[35130]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:25.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:24 smithi097 ceph-mon[35562]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:26.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:26 smithi043 ceph-mon[30950]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:27.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:26 smithi090 ceph-mon[35130]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:27.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:26 smithi097 ceph-mon[35562]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:27.851 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:27 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T23:18:27.851 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:27 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:27.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:27 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T23:18:27.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:27 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:28.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:27 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T23:18:28.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:27 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:28.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:28 smithi043 ceph-mon[30950]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:28.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:28 smithi043 ceph-mon[30950]: Deploying daemon osd.10 on smithi090 2024-09-11T23:18:28.880 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:28 smithi090 ceph-mon[35130]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:28.880 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:28 smithi090 ceph-mon[35130]: Deploying daemon osd.10 on smithi090 2024-09-11T23:18:29.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:28 smithi097 ceph-mon[35562]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:29.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:28 smithi097 ceph-mon[35562]: Deploying daemon osd.10 on smithi090 2024-09-11T23:18:30.693 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:30 smithi090 ceph-mon[35130]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:30.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:30 smithi043 ceph-mon[30950]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:31.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:30 smithi097 ceph-mon[35562]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:32.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:32 smithi043 ceph-mon[30950]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:32.898 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:32 smithi090 ceph-mon[35130]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:33.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:32 smithi097 ceph-mon[35562]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:33.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:33 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:33.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:33 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:33.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:33 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:33.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:33 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:33.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:33 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:33.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:33 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:34.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:33 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:34.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:33 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:34.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:33 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:34.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:34 smithi043 ceph-mon[30950]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:35.012 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:34 smithi090 ceph-mon[35130]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:35.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:34 smithi097 ceph-mon[35562]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:36.714 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:36 smithi090 ceph-mon[35130]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:36.714 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:36 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:36.714 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:36 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:36.714 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:36 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:36.714 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:36 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:36.714 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:36 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:36.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:36 smithi043 ceph-mon[30950]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:36.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:36 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:36.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:36 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:36.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:36 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:36.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:36 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:36.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:36 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:37.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:36 smithi097 ceph-mon[35562]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:37.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:36 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:37.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:36 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:37.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:36 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:37.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:36 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:37.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:36 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:37.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:37.074+0000 7f4c6cff9640 1 -- 172.21.15.90:0/1083458824 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4c70062630 con 0x7f4c580a2c30 2024-09-11T23:18:37.074 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 10 on host 'smithi090' 2024-09-11T23:18:37.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:37.077+0000 7f4c75b01640 1 -- 172.21.15.90:0/1083458824 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f4c580a2c30 msgr2=0x7f4c580a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:37.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:37.077+0000 7f4c75b01640 1 --2- 172.21.15.90:0/1083458824 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f4c580a2c30 0x7f4c580a50f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4c70102c60 tx=0x7f4c60008040 comp rx=0 tx=0).stop 2024-09-11T23:18:37.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:37.077+0000 7f4c75b01640 1 -- 172.21.15.90:0/1083458824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c701037c0 msgr2=0x7f4c7019f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:37.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:37.077+0000 7f4c75b01640 1 --2- 172.21.15.90:0/1083458824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c701037c0 0x7f4c7019f600 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4c680098d0 tx=0x7f4c6802f840 comp rx=0 tx=0).stop 2024-09-11T23:18:37.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:37.077+0000 7f4c75b01640 1 -- 172.21.15.90:0/1083458824 shutdown_connections 2024-09-11T23:18:37.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:37.077+0000 7f4c75b01640 1 --2- 172.21.15.90:0/1083458824 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f4c580a2c30 0x7f4c580a50f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:37.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:37.077+0000 7f4c75b01640 1 --2- 172.21.15.90:0/1083458824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c701037c0 0x7f4c7019f600 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:37.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:37.077+0000 7f4c75b01640 1 --2- 172.21.15.90:0/1083458824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c70102e00 0x7f4c70198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:37.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:37.077+0000 7f4c75b01640 1 --2- 172.21.15.90:0/1083458824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c70101c00 0x7f4c70198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:37.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:37.077+0000 7f4c75b01640 1 -- 172.21.15.90:0/1083458824 >> 172.21.15.90:0/1083458824 conn(0x7f4c700fd390 msgr2=0x7f4c701061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:37.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:37.077+0000 7f4c75b01640 1 -- 172.21.15.90:0/1083458824 shutdown_connections 2024-09-11T23:18:37.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-11T23:18:37.077+0000 7f4c75b01640 1 -- 172.21.15.90:0/1083458824 wait complete. 2024-09-11T23:18:37.739 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:37 smithi090 ceph-mon[35130]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:37.739 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:37 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:37.739 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:37 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:37.739 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:37 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:37.765 DEBUG:teuthology.orchestra.run.smithi090:osd.10> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.10.service 2024-09-11T23:18:37.769 INFO:tasks.cephadm:Deploying osd.11 on smithi097 with /dev/vg_nvme/lv_1... 2024-09-11T23:18:37.769 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T23:18:37.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:37 smithi043 ceph-mon[30950]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:37.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:37 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:37.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:37 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:37.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:37 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:38.022 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:18:38.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:37 smithi097 ceph-mon[35562]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:38.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:37 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:18:38.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:37 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:38.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:37 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:39.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:39 smithi090 ceph-mon[35130]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:39.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:39 smithi090 ceph-mon[35130]: from='osd.10 [v2:172.21.15.90:6824/59487041,v1:172.21.15.90:6825/59487041]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:18:39.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:39 smithi090 ceph-mon[35130]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:18:39.770 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:18:39 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:18:39.568+0000 7fc85d5ee640 -1 osd.10 0 log_to_monitors true 2024-09-11T23:18:39.872 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:39 smithi097 ceph-mon[35562]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:39.872 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:39 smithi097 ceph-mon[35562]: from='osd.10 [v2:172.21.15.90:6824/59487041,v1:172.21.15.90:6825/59487041]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:18:39.872 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:39 smithi097 ceph-mon[35562]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:18:40.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:39 smithi043 ceph-mon[30950]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:40.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:39 smithi043 ceph-mon[30950]: from='osd.10 [v2:172.21.15.90:6824/59487041,v1:172.21.15.90:6825/59487041]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:18:40.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:39 smithi043 ceph-mon[30950]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T23:18:40.689 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:40 smithi090 ceph-mon[35130]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T23:18:40.689 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:40 smithi090 ceph-mon[35130]: osdmap e67: 11 total, 10 up, 11 in 2024-09-11T23:18:40.689 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:40 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:18:40.689 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:40 smithi090 ceph-mon[35130]: from='osd.10 [v2:172.21.15.90:6824/59487041,v1:172.21.15.90:6825/59487041]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:18:40.690 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:40 smithi090 ceph-mon[35130]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:18:40.690 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:18:40 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:18:40.678+0000 7fc859571640 -1 osd.10 0 waiting for initial osdmap 2024-09-11T23:18:40.690 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:18:40 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:18:40.689+0000 7fc855399640 -1 osd.10 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:18:40.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:40 smithi097 ceph-mon[35562]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T23:18:40.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:40 smithi097 ceph-mon[35562]: osdmap e67: 11 total, 10 up, 11 in 2024-09-11T23:18:40.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:40 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:18:40.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:40 smithi097 ceph-mon[35562]: from='osd.10 [v2:172.21.15.90:6824/59487041,v1:172.21.15.90:6825/59487041]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:18:40.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:40 smithi097 ceph-mon[35562]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:18:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-11T23:18:40.940 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_1 2024-09-11T23:18:41.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:40 smithi043 ceph-mon[30950]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T23:18:41.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:40 smithi043 ceph-mon[30950]: osdmap e67: 11 total, 10 up, 11 in 2024-09-11T23:18:41.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:40 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:18:41.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:40 smithi043 ceph-mon[30950]: from='osd.10 [v2:172.21.15.90:6824/59487041,v1:172.21.15.90:6825/59487041]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:18:41.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:40 smithi043 ceph-mon[30950]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-11T23:18:41.189 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:18:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.699+0000 7f18192a1640 1 -- 172.21.15.97:0/2966207888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1814101cd0 msgr2=0x7f1814102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.699+0000 7f18192a1640 1 --2- 172.21.15.97:0/2966207888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1814101cd0 0x7f1814102150 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009900 tx=0x7f17fc02a410 comp rx=0 tx=0).stop 2024-09-11T23:18:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.699+0000 7f18192a1640 1 -- 172.21.15.97:0/2966207888 shutdown_connections 2024-09-11T23:18:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.699+0000 7f18192a1640 1 --2- 172.21.15.97:0/2966207888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1814102690 0x7f1814108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:41.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.700+0000 7f18192a1640 1 --2- 172.21.15.97:0/2966207888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1814101cd0 0x7f1814102150 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.700+0000 7f18192a1640 1 --2- 172.21.15.97:0/2966207888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1814100ad0 0x7f1814100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.700+0000 7f18192a1640 1 -- 172.21.15.97:0/2966207888 >> 172.21.15.97:0/2966207888 conn(0x7f18140fc280 msgr2=0x7f18140fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:18:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.700+0000 7f18192a1640 1 -- 172.21.15.97:0/2966207888 shutdown_connections 2024-09-11T23:18:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.700+0000 7f18192a1640 1 -- 172.21.15.97:0/2966207888 wait complete. 2024-09-11T23:18:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.700+0000 7f18192a1640 1 Processor -- start 2024-09-11T23:18:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.700+0000 7f18192a1640 1 -- start start 2024-09-11T23:18:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.701+0000 7f18192a1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1814100ad0 0x7f1814197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.701+0000 7f18192a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1814101cd0 0x7f1814198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.701+0000 7f18192a1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1814102690 0x7f181419f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:41.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.701+0000 7f18192a1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f181410d8e0 con 0x7f1814101cd0 2024-09-11T23:18:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.701+0000 7f18192a1640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f181410d760 con 0x7f1814100ad0 2024-09-11T23:18:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.701+0000 7f18192a1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f181410da60 con 0x7f1814102690 2024-09-11T23:18:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.701+0000 7f18137fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1814102690 0x7f181419f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.701+0000 7f18137fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1814102690 0x7f181419f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36096/0 (socket says 172.21.15.97:36096) 2024-09-11T23:18:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.701+0000 7f18137fe640 1 -- 172.21.15.97:0/1236111350 learned_addr learned my addr 172.21.15.97:0/1236111350 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-11T23:18:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.701+0000 7f180a7fc640 1 --2- 172.21.15.97:0/1236111350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1814101cd0 0x7f1814198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.701+0000 7f1812ffd640 1 --2- 172.21.15.97:0/1236111350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1814100ad0 0x7f1814197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.701+0000 7f18137fe640 1 -- 172.21.15.97:0/1236111350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1814100ad0 msgr2=0x7f1814197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:41.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.701+0000 7f18137fe640 1 --2- 172.21.15.97:0/1236111350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1814100ad0 0x7f1814197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:41.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.701+0000 7f18137fe640 1 -- 172.21.15.97:0/1236111350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1814101cd0 msgr2=0x7f1814198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:18:41.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.701+0000 7f18137fe640 1 --2- 172.21.15.97:0/1236111350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1814101cd0 0x7f1814198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:18:41.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.701+0000 7f18137fe640 1 -- 172.21.15.97:0/1236111350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181419fb60 con 0x7f1814102690 2024-09-11T23:18:41.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.701+0000 7f18137fe640 1 --2- 172.21.15.97:0/1236111350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1814102690 0x7f181419f4f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f180c00b970 tx=0x7f180c00be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:41.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.701+0000 7f1810ff9640 1 -- 172.21.15.97:0/1236111350 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c019070 con 0x7f1814102690 2024-09-11T23:18:41.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.701+0000 7f1810ff9640 1 -- 172.21.15.97:0/1236111350 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f180c00cb40 con 0x7f1814102690 2024-09-11T23:18:41.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.702+0000 7f18192a1640 1 -- 172.21.15.97:0/1236111350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181419fe50 con 0x7f1814102690 2024-09-11T23:18:41.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.702+0000 7f1810ff9640 1 -- 172.21.15.97:0/1236111350 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c007650 con 0x7f1814102690 2024-09-11T23:18:41.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.702+0000 7f18192a1640 1 -- 172.21.15.97:0/1236111350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18141a0380 con 0x7f1814102690 2024-09-11T23:18:41.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.703+0000 7f18192a1640 1 -- 172.21.15.97:0/1236111350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1814107ea0 con 0x7f1814102690 2024-09-11T23:18:41.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.704+0000 7f1810ff9640 1 -- 172.21.15.97:0/1236111350 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f180c01c070 con 0x7f1814102690 2024-09-11T23:18:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.704+0000 7f1810ff9640 1 --2- 172.21.15.97:0/1236111350 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f18000a2c30 0x7f18000a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:18:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.705+0000 7f1810ff9640 1 -- 172.21.15.97:0/1236111350 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6727+0+0 (secure 0 0 0) 0x7f180c0ce3b0 con 0x7f1814102690 2024-09-11T23:18:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.705+0000 7f1812ffd640 1 --2- 172.21.15.97:0/1236111350 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f18000a2c30 0x7f18000a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:18:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.705+0000 7f1812ffd640 1 --2- 172.21.15.97:0/1236111350 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f18000a2c30 0x7f18000a50f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f18040058c0 tx=0x7f180400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:18:41.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.708+0000 7f1810ff9640 1 -- 172.21.15.97:0/1236111350 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f180c09b2f0 con 0x7f1814102690 2024-09-11T23:18:41.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:18:41.885+0000 7f18192a1640 1 -- 172.21.15.97:0/1236111350 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1814062630 con 0x7f18000a2c30 2024-09-11T23:18:41.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:41 smithi097 ceph-mon[35562]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:41.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:41 smithi097 ceph-mon[35562]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-11T23:18:41.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:41 smithi097 ceph-mon[35562]: osdmap e68: 11 total, 10 up, 11 in 2024-09-11T23:18:41.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:18:41.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:18:41.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:41 smithi097 ceph-mon[35562]: Detected new or changed devices on smithi090 2024-09-11T23:18:41.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:41.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:41.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:41.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:41.886 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:41.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:41.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:41 smithi097 ceph-mon[35562]: Adjusting osd_memory_target on smithi090 to 4272M 2024-09-11T23:18:41.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:41.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:41.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:41.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:41.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:41 smithi097 ceph-mon[35562]: osd.10 [v2:172.21.15.90:6824/59487041,v1:172.21.15.90:6825/59487041] boot 2024-09-11T23:18:41.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:41 smithi097 ceph-mon[35562]: osdmap e69: 11 total, 11 up, 11 in 2024-09-11T23:18:41.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:18:42.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:41 smithi090 ceph-mon[35130]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:42.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:41 smithi090 ceph-mon[35130]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-11T23:18:42.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:41 smithi090 ceph-mon[35130]: osdmap e68: 11 total, 10 up, 11 in 2024-09-11T23:18:42.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:18:42.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:18:42.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:41 smithi090 ceph-mon[35130]: Detected new or changed devices on smithi090 2024-09-11T23:18:42.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:42.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:42.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:42.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:42.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:42.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:42.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:41 smithi090 ceph-mon[35130]: Adjusting osd_memory_target on smithi090 to 4272M 2024-09-11T23:18:42.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:42.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:42.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:42.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:42.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:41 smithi090 ceph-mon[35130]: osd.10 [v2:172.21.15.90:6824/59487041,v1:172.21.15.90:6825/59487041] boot 2024-09-11T23:18:42.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:41 smithi090 ceph-mon[35130]: osdmap e69: 11 total, 11 up, 11 in 2024-09-11T23:18:42.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:18:42.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:41 smithi043 ceph-mon[30950]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-11T23:18:42.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:41 smithi043 ceph-mon[30950]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-11T23:18:42.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:41 smithi043 ceph-mon[30950]: osdmap e68: 11 total, 10 up, 11 in 2024-09-11T23:18:42.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:18:42.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:18:42.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:41 smithi043 ceph-mon[30950]: Detected new or changed devices on smithi090 2024-09-11T23:18:42.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:42.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:42.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:42.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:42.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:42.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:18:42.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:41 smithi043 ceph-mon[30950]: Adjusting osd_memory_target on smithi090 to 4272M 2024-09-11T23:18:42.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:42.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:42.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:18:42.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:18:42.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:41 smithi043 ceph-mon[30950]: osd.10 [v2:172.21.15.90:6824/59487041,v1:172.21.15.90:6825/59487041] boot 2024-09-11T23:18:42.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:41 smithi043 ceph-mon[30950]: osdmap e69: 11 total, 11 up, 11 in 2024-09-11T23:18:42.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:18:43.012 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:42 smithi097 ceph-mon[35562]: purged_snaps scrub starts 2024-09-11T23:18:43.013 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:42 smithi097 ceph-mon[35562]: purged_snaps scrub ok 2024-09-11T23:18:43.013 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:42 smithi097 ceph-mon[35562]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:18:43.013 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:42 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:18:43.013 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:42 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:18:43.013 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:42 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:43.013 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:42 smithi097 ceph-mon[35562]: osdmap e70: 11 total, 11 up, 11 in 2024-09-11T23:18:43.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:42 smithi090 ceph-mon[35130]: purged_snaps scrub starts 2024-09-11T23:18:43.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:42 smithi090 ceph-mon[35130]: purged_snaps scrub ok 2024-09-11T23:18:43.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:42 smithi090 ceph-mon[35130]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:18:43.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:42 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:18:43.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:42 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:18:43.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:42 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:43.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:42 smithi090 ceph-mon[35130]: osdmap e70: 11 total, 11 up, 11 in 2024-09-11T23:18:43.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:42 smithi043 ceph-mon[30950]: purged_snaps scrub starts 2024-09-11T23:18:43.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:42 smithi043 ceph-mon[30950]: purged_snaps scrub ok 2024-09-11T23:18:43.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:42 smithi043 ceph-mon[30950]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:18:43.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:42 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T23:18:43.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:42 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T23:18:43.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:42 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:43.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:42 smithi043 ceph-mon[30950]: osdmap e70: 11 total, 11 up, 11 in 2024-09-11T23:18:43.878 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:43 smithi097 ceph-mon[35562]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:18:44.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:43 smithi090 ceph-mon[35130]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:18:44.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:43 smithi043 ceph-mon[30950]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:18:44.953 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:44 smithi097 ceph-mon[35562]: osdmap e71: 11 total, 11 up, 11 in 2024-09-11T23:18:44.953 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:44 smithi097 ceph-mon[35562]: from='client.? 172.21.15.97:0/122730682' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0faa589-21c8-4a9f-8a73-813aafe6d33c"}]: dispatch 2024-09-11T23:18:44.953 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:44 smithi097 ceph-mon[35562]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0faa589-21c8-4a9f-8a73-813aafe6d33c"}]: dispatch 2024-09-11T23:18:44.953 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:44 smithi097 ceph-mon[35562]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0faa589-21c8-4a9f-8a73-813aafe6d33c"}]': finished 2024-09-11T23:18:44.953 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:44 smithi097 ceph-mon[35562]: osdmap e72: 12 total, 11 up, 12 in 2024-09-11T23:18:44.953 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:44 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:18:45.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:44 smithi090 ceph-mon[35130]: osdmap e71: 11 total, 11 up, 11 in 2024-09-11T23:18:45.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:44 smithi090 ceph-mon[35130]: from='client.? 172.21.15.97:0/122730682' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0faa589-21c8-4a9f-8a73-813aafe6d33c"}]: dispatch 2024-09-11T23:18:45.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:44 smithi090 ceph-mon[35130]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0faa589-21c8-4a9f-8a73-813aafe6d33c"}]: dispatch 2024-09-11T23:18:45.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:44 smithi090 ceph-mon[35130]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0faa589-21c8-4a9f-8a73-813aafe6d33c"}]': finished 2024-09-11T23:18:45.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:44 smithi090 ceph-mon[35130]: osdmap e72: 12 total, 11 up, 12 in 2024-09-11T23:18:45.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:44 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:18:45.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:44 smithi043 ceph-mon[30950]: osdmap e71: 11 total, 11 up, 11 in 2024-09-11T23:18:45.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:44 smithi043 ceph-mon[30950]: from='client.? 172.21.15.97:0/122730682' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0faa589-21c8-4a9f-8a73-813aafe6d33c"}]: dispatch 2024-09-11T23:18:45.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:44 smithi043 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0faa589-21c8-4a9f-8a73-813aafe6d33c"}]: dispatch 2024-09-11T23:18:45.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:44 smithi043 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0faa589-21c8-4a9f-8a73-813aafe6d33c"}]': finished 2024-09-11T23:18:45.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:44 smithi043 ceph-mon[30950]: osdmap e72: 12 total, 11 up, 12 in 2024-09-11T23:18:45.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:44 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:18:46.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:45 smithi090 ceph-mon[35130]: pgmap v234: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-09-11T23:18:46.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:45 smithi090 ceph-mon[35130]: from='client.? 172.21.15.97:0/125401455' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:18:46.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:45 smithi090 ceph-mon[35130]: osdmap e73: 12 total, 11 up, 12 in 2024-09-11T23:18:46.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:45 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:18:46.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:45 smithi097 ceph-mon[35562]: pgmap v234: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-09-11T23:18:46.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:45 smithi097 ceph-mon[35562]: from='client.? 172.21.15.97:0/125401455' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:18:46.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:45 smithi097 ceph-mon[35562]: osdmap e73: 12 total, 11 up, 12 in 2024-09-11T23:18:46.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:45 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:18:46.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:45 smithi043 ceph-mon[30950]: pgmap v234: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-09-11T23:18:46.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:45 smithi043 ceph-mon[30950]: from='client.? 172.21.15.97:0/125401455' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T23:18:46.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:45 smithi043 ceph-mon[30950]: osdmap e73: 12 total, 11 up, 12 in 2024-09-11T23:18:46.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:45 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:18:48.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:47 smithi090 ceph-mon[35130]: pgmap v236: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T23:18:48.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:47 smithi097 ceph-mon[35562]: pgmap v236: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T23:18:48.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:47 smithi043 ceph-mon[30950]: pgmap v236: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T23:18:49.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:49 smithi097 ceph-mon[35562]: pgmap v237: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-11T23:18:50.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:49 smithi090 ceph-mon[35130]: pgmap v237: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-11T23:18:50.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:49 smithi043 ceph-mon[30950]: pgmap v237: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-11T23:18:51.979 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:51 smithi097 ceph-mon[35562]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-11T23:18:52.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:51 smithi090 ceph-mon[35130]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-11T23:18:52.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:51 smithi043 ceph-mon[30950]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-11T23:18:53.995 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:53 smithi097 ceph-mon[35562]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:18:53.995 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:53 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T23:18:53.996 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:53 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:53.996 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:53 smithi097 ceph-mon[35562]: Deploying daemon osd.11 on smithi097 2024-09-11T23:18:54.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:53 smithi090 ceph-mon[35130]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:18:54.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:53 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T23:18:54.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:53 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:54.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:53 smithi090 ceph-mon[35130]: Deploying daemon osd.11 on smithi097 2024-09-11T23:18:54.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:53 smithi043 ceph-mon[30950]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:18:54.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:53 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T23:18:54.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:53 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:18:54.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:53 smithi043 ceph-mon[30950]: Deploying daemon osd.11 on smithi097 2024-09-11T23:18:56.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:55 smithi090 ceph-mon[35130]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:18:56.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:55 smithi097 ceph-mon[35562]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:18:56.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:55 smithi043 ceph-mon[30950]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:18:58.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:57 smithi090 ceph-mon[35130]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:18:58.029 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:57 smithi097 ceph-mon[35562]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:18:58.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:57 smithi043 ceph-mon[30950]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:19:00.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:18:59 smithi090 ceph-mon[35130]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:19:00.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:18:59 smithi097 ceph-mon[35562]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:19:00.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:18:59 smithi043 ceph-mon[30950]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:19:01.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:00 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:19:01.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:00 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:01.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:00 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:01.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:00 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:19:01.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:00 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:01.053 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:00 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:01.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:00 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:19:01.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:00 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:01.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:00 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:02.219 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:01 smithi097 ceph-mon[35562]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:19:02.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:01 smithi090 ceph-mon[35130]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:19:02.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:01 smithi043 ceph-mon[30950]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:19:03.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:03.586+0000 7f1810ff9640 1 -- 172.21.15.97:0/1236111350 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1814062630 con 0x7f18000a2c30 2024-09-11T23:19:03.587 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 11 on host 'smithi097' 2024-09-11T23:19:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:03.588+0000 7f18192a1640 1 -- 172.21.15.97:0/1236111350 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f18000a2c30 msgr2=0x7f18000a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:03.588+0000 7f18192a1640 1 --2- 172.21.15.97:0/1236111350 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f18000a2c30 0x7f18000a50f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f18040058c0 tx=0x7f180400a4c0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:03.588+0000 7f18192a1640 1 -- 172.21.15.97:0/1236111350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1814102690 msgr2=0x7f181419f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:03.588+0000 7f18192a1640 1 --2- 172.21.15.97:0/1236111350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1814102690 0x7f181419f4f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f180c00b970 tx=0x7f180c00be30 comp rx=0 tx=0).stop 2024-09-11T23:19:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:03.589+0000 7f18192a1640 1 -- 172.21.15.97:0/1236111350 shutdown_connections 2024-09-11T23:19:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:03.589+0000 7f18192a1640 1 --2- 172.21.15.97:0/1236111350 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f18000a2c30 0x7f18000a50f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:03.589+0000 7f18192a1640 1 --2- 172.21.15.97:0/1236111350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1814102690 0x7f181419f4f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:03.589+0000 7f18192a1640 1 --2- 172.21.15.97:0/1236111350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1814101cd0 0x7f1814198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:03.589+0000 7f18192a1640 1 --2- 172.21.15.97:0/1236111350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1814100ad0 0x7f1814197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:03.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:03.589+0000 7f18192a1640 1 -- 172.21.15.97:0/1236111350 >> 172.21.15.97:0/1236111350 conn(0x7f18140fc280 msgr2=0x7f1814105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:03.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:03.589+0000 7f18192a1640 1 -- 172.21.15.97:0/1236111350 shutdown_connections 2024-09-11T23:19:03.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:03.589+0000 7f18192a1640 1 -- 172.21.15.97:0/1236111350 wait complete. 2024-09-11T23:19:03.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:03 smithi097 ceph-mon[35562]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:19:03.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:03 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:03.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:03 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:03.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:03 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:03.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:03 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:19:03.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:03 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:03.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:03 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:19:03.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:03 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:03.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:03 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:04.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:03 smithi043 ceph-mon[30950]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:19:04.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:04.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:04.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:04.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:19:04.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:04.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:19:04.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:04.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:03 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:04.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:03 smithi090 ceph-mon[35130]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:19:04.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:03 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:04.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:03 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:04.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:03 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:04.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:03 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:19:04.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:03 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:04.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:03 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:19:04.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:03 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:04.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:03 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:04.438 DEBUG:teuthology.orchestra.run.smithi097:osd.11> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.11.service 2024-09-11T23:19:04.440 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-11T23:19:04.441 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T23:19:04.708 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:05.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.180+0000 7fc88c135640 1 -- 172.21.15.43:0/3298017405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc884101c00 msgr2=0x7fc884102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:05.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.180+0000 7fc88c135640 1 --2- 172.21.15.43:0/3298017405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc884101c00 0x7fc884102000 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc880009900 tx=0x7fc88002a410 comp rx=0 tx=0).stop 2024-09-11T23:19:05.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.180+0000 7fc88c135640 1 -- 172.21.15.43:0/3298017405 shutdown_connections 2024-09-11T23:19:05.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.180+0000 7fc88c135640 1 --2- 172.21.15.43:0/3298017405 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8841037c0 0x7fc88410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:05.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.180+0000 7fc88c135640 1 --2- 172.21.15.43:0/3298017405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc884102e00 0x7fc884103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:05.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.180+0000 7fc88c135640 1 --2- 172.21.15.43:0/3298017405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc884101c00 0x7fc884102000 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:05.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.180+0000 7fc88c135640 1 -- 172.21.15.43:0/3298017405 >> 172.21.15.43:0/3298017405 conn(0x7fc8840fd390 msgr2=0x7fc8840ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:05.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.180+0000 7fc88c135640 1 -- 172.21.15.43:0/3298017405 shutdown_connections 2024-09-11T23:19:05.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.180+0000 7fc88c135640 1 -- 172.21.15.43:0/3298017405 wait complete. 2024-09-11T23:19:05.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.181+0000 7fc88c135640 1 Processor -- start 2024-09-11T23:19:05.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.181+0000 7fc88c135640 1 -- start start 2024-09-11T23:19:05.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.181+0000 7fc88c135640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc884101c00 0x7fc884197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:05.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.181+0000 7fc88c135640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc884102e00 0x7fc884198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:05.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.181+0000 7fc88c135640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8841037c0 0x7fc88419f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:05.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.181+0000 7fc88c135640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc88410d880 con 0x7fc884102e00 2024-09-11T23:19:05.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.181+0000 7fc88c135640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc88410d700 con 0x7fc8841037c0 2024-09-11T23:19:05.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.181+0000 7fc88c135640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc88410da00 con 0x7fc884101c00 2024-09-11T23:19:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.181+0000 7fc8896d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc884102e00 0x7fc884198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.181+0000 7fc8896d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc884102e00 0x7fc884198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38986/0 (socket says 172.21.15.43:38986) 2024-09-11T23:19:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.181+0000 7fc8896d0640 1 -- 172.21.15.43:0/759777938 learned_addr learned my addr 172.21.15.43:0/759777938 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:19:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.182+0000 7fc889ed1640 1 --2- 172.21.15.43:0/759777938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc884101c00 0x7fc884197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.182+0000 7fc88a6d2640 1 --2- 172.21.15.43:0/759777938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8841037c0 0x7fc88419f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.182+0000 7fc8896d0640 1 -- 172.21.15.43:0/759777938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc884101c00 msgr2=0x7fc884197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.182+0000 7fc8896d0640 1 --2- 172.21.15.43:0/759777938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc884101c00 0x7fc884197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.182+0000 7fc8896d0640 1 -- 172.21.15.43:0/759777938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8841037c0 msgr2=0x7fc88419f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.182+0000 7fc8896d0640 1 --2- 172.21.15.43:0/759777938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8841037c0 0x7fc88419f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.182+0000 7fc8896d0640 1 -- 172.21.15.43:0/759777938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88419fb40 con 0x7fc884102e00 2024-09-11T23:19:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.182+0000 7fc889ed1640 1 --2- 172.21.15.43:0/759777938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc884101c00 0x7fc884197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:19:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.182+0000 7fc8896d0640 1 --2- 172.21.15.43:0/759777938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc884102e00 0x7fc884198450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc87800e9f0 tx=0x7fc87800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.182+0000 7fc876ffd640 1 -- 172.21.15.43:0/759777938 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87800cd80 con 0x7fc884102e00 2024-09-11T23:19:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.182+0000 7fc876ffd640 1 -- 172.21.15.43:0/759777938 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc878005500 con 0x7fc884102e00 2024-09-11T23:19:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.182+0000 7fc88c135640 1 -- 172.21.15.43:0/759777938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc88419fe30 con 0x7fc884102e00 2024-09-11T23:19:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.182+0000 7fc876ffd640 1 -- 172.21.15.43:0/759777938 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878010690 con 0x7fc884102e00 2024-09-11T23:19:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.182+0000 7fc88c135640 1 -- 172.21.15.43:0/759777938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8841a0290 con 0x7fc884102e00 2024-09-11T23:19:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.183+0000 7fc88c135640 1 -- 172.21.15.43:0/759777938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc884108ff0 con 0x7fc884102e00 2024-09-11T23:19:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.185+0000 7fc876ffd640 1 -- 172.21.15.43:0/759777938 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc8780046c0 con 0x7fc884102e00 2024-09-11T23:19:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.185+0000 7fc876ffd640 1 --2- 172.21.15.43:0/759777938 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fc8700a2bb0 0x7fc8700a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.185+0000 7fc889ed1640 1 --2- 172.21.15.43:0/759777938 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fc8700a2bb0 0x7fc8700a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.185+0000 7fc876ffd640 1 -- 172.21.15.43:0/759777938 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6809+0+0 (secure 0 0 0) 0x7fc8780cf140 con 0x7fc884102e00 2024-09-11T23:19:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.186+0000 7fc889ed1640 1 --2- 172.21.15.43:0/759777938 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fc8700a2bb0 0x7fc8700a5070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc880006010 tx=0x7fc880032cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:05.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.189+0000 7fc876ffd640 1 -- 172.21.15.43:0/759777938 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc87809c710 con 0x7fc884102e00 2024-09-11T23:19:05.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.366+0000 7fc88c135640 1 -- 172.21.15.43:0/759777938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc88402da10 con 0x7fc884102e00 2024-09-11T23:19:05.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.366+0000 7fc876ffd640 1 -- 172.21.15.43:0/759777938 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7fc87809c010 con 0x7fc884102e00 2024-09-11T23:19:05.367 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:19:05.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.368+0000 7fc88c135640 1 -- 172.21.15.43:0/759777938 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fc8700a2bb0 msgr2=0x7fc8700a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:05.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.368+0000 7fc88c135640 1 --2- 172.21.15.43:0/759777938 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fc8700a2bb0 0x7fc8700a5070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc880006010 tx=0x7fc880032cf0 comp rx=0 tx=0).stop 2024-09-11T23:19:05.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.368+0000 7fc88c135640 1 -- 172.21.15.43:0/759777938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc884102e00 msgr2=0x7fc884198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:05.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.368+0000 7fc88c135640 1 --2- 172.21.15.43:0/759777938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc884102e00 0x7fc884198450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc87800e9f0 tx=0x7fc87800edf0 comp rx=0 tx=0).stop 2024-09-11T23:19:05.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.368+0000 7fc88c135640 1 -- 172.21.15.43:0/759777938 shutdown_connections 2024-09-11T23:19:05.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.368+0000 7fc88c135640 1 --2- 172.21.15.43:0/759777938 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fc8700a2bb0 0x7fc8700a5070 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:05.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.368+0000 7fc88c135640 1 --2- 172.21.15.43:0/759777938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8841037c0 0x7fc88419f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:05.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.368+0000 7fc88c135640 1 --2- 172.21.15.43:0/759777938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc884102e00 0x7fc884198450 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:05.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.368+0000 7fc88c135640 1 --2- 172.21.15.43:0/759777938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc884101c00 0x7fc884197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:05.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.368+0000 7fc88c135640 1 -- 172.21.15.43:0/759777938 >> 172.21.15.43:0/759777938 conn(0x7fc8840fd390 msgr2=0x7fc8841061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:05.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.369+0000 7fc88c135640 1 -- 172.21.15.43:0/759777938 shutdown_connections 2024-09-11T23:19:05.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:05.369+0000 7fc88c135640 1 -- 172.21.15.43:0/759777938 wait complete. 2024-09-11T23:19:05.792 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:19:05 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:19:05.478+0000 7f8ab386c640 -1 osd.11 0 log_to_monitors true 2024-09-11T23:19:06.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:05 smithi097 ceph-mon[35562]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:19:06.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:05 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/759777938' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:19:06.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:05 smithi097 ceph-mon[35562]: from='osd.11 [v2:172.21.15.97:6824/194699315,v1:172.21.15.97:6825/194699315]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:19:06.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:05 smithi097 ceph-mon[35562]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:19:06.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:05 smithi043 ceph-mon[30950]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:19:06.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:05 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/759777938' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:19:06.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:05 smithi043 ceph-mon[30950]: from='osd.11 [v2:172.21.15.97:6824/194699315,v1:172.21.15.97:6825/194699315]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:19:06.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:05 smithi043 ceph-mon[30950]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:19:06.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:05 smithi090 ceph-mon[35130]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:19:06.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:05 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/759777938' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:19:06.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:05 smithi090 ceph-mon[35130]: from='osd.11 [v2:172.21.15.97:6824/194699315,v1:172.21.15.97:6825/194699315]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:19:06.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:05 smithi090 ceph-mon[35130]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T23:19:06.308 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1726096721,"num_in_osds":12,"osd_in_since":1726096724,"num_remapped_pgs":0} 2024-09-11T23:19:07.051 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:19:06 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:19:06.802+0000 7f8aaf7ef640 -1 osd.11 0 waiting for initial osdmap 2024-09-11T23:19:07.051 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:19:06 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:19:06.813+0000 7f8aab617640 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T23:19:07.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:06 smithi097 ceph-mon[35562]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T23:19:07.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:06 smithi097 ceph-mon[35562]: osdmap e74: 12 total, 11 up, 12 in 2024-09-11T23:19:07.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:06 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:19:07.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:06 smithi097 ceph-mon[35562]: from='osd.11 [v2:172.21.15.97:6824/194699315,v1:172.21.15.97:6825/194699315]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:19:07.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:06 smithi097 ceph-mon[35562]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:19:07.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:06 smithi043 ceph-mon[30950]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T23:19:07.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:06 smithi043 ceph-mon[30950]: osdmap e74: 12 total, 11 up, 12 in 2024-09-11T23:19:07.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:06 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:19:07.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:06 smithi043 ceph-mon[30950]: from='osd.11 [v2:172.21.15.97:6824/194699315,v1:172.21.15.97:6825/194699315]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:19:07.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:06 smithi043 ceph-mon[30950]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:19:07.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:06 smithi090 ceph-mon[35130]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T23:19:07.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:06 smithi090 ceph-mon[35130]: osdmap e74: 12 total, 11 up, 12 in 2024-09-11T23:19:07.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:06 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:19:07.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:06 smithi090 ceph-mon[35130]: from='osd.11 [v2:172.21.15.97:6824/194699315,v1:172.21.15.97:6825/194699315]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:19:07.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:06 smithi090 ceph-mon[35130]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-11T23:19:07.309 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T23:19:07.575 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:07.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:07 smithi043 ceph-mon[30950]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:19:07.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:07 smithi043 ceph-mon[30950]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-11T23:19:07.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:07 smithi043 ceph-mon[30950]: osdmap e75: 12 total, 11 up, 12 in 2024-09-11T23:19:07.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:19:07.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:19:07.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:07 smithi043 ceph-mon[30950]: Detected new or changed devices on smithi097 2024-09-11T23:19:07.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:07.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:07.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:19:07.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:19:07.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:19:07.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:19:07.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:07 smithi043 ceph-mon[30950]: Adjusting osd_memory_target on smithi097 to 4272M 2024-09-11T23:19:07.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:07.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:07.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:19:07.981 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:08.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.188+0000 7f3eef397640 1 -- 172.21.15.43:0/2571357593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee8101c00 msgr2=0x7f3ee8102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:08.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.188+0000 7f3eef397640 1 --2- 172.21.15.43:0/2571357593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee8101c00 0x7f3ee8102000 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009900 tx=0x7f3ee402a410 comp rx=0 tx=0).stop 2024-09-11T23:19:08.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.188+0000 7f3eef397640 1 -- 172.21.15.43:0/2571357593 shutdown_connections 2024-09-11T23:19:08.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.188+0000 7f3eef397640 1 --2- 172.21.15.43:0/2571357593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ee81037c0 0x7f3ee810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:08.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.188+0000 7f3eef397640 1 --2- 172.21.15.43:0/2571357593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee8102e00 0x7f3ee8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:08.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.188+0000 7f3eef397640 1 --2- 172.21.15.43:0/2571357593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee8101c00 0x7f3ee8102000 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:08.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.188+0000 7f3eef397640 1 -- 172.21.15.43:0/2571357593 >> 172.21.15.43:0/2571357593 conn(0x7f3ee80fd390 msgr2=0x7f3ee80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.189+0000 7f3eef397640 1 -- 172.21.15.43:0/2571357593 shutdown_connections 2024-09-11T23:19:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.189+0000 7f3eef397640 1 -- 172.21.15.43:0/2571357593 wait complete. 2024-09-11T23:19:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.189+0000 7f3eef397640 1 Processor -- start 2024-09-11T23:19:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.189+0000 7f3eef397640 1 -- start start 2024-09-11T23:19:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.190+0000 7f3eef397640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee8101c00 0x7f3ee8197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.190+0000 7f3eef397640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ee8102e00 0x7f3ee8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.190+0000 7f3eef397640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee81037c0 0x7f3ee819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.190+0000 7f3eef397640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee810d880 con 0x7f3ee8101c00 2024-09-11T23:19:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.190+0000 7f3eef397640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee810d700 con 0x7f3ee8102e00 2024-09-11T23:19:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.190+0000 7f3eef397640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee810da00 con 0x7f3ee81037c0 2024-09-11T23:19:08.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.190+0000 7f3eed133640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee8101c00 0x7f3ee8197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.190+0000 7f3eed133640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee8101c00 0x7f3ee8197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39014/0 (socket says 172.21.15.43:39014) 2024-09-11T23:19:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.190+0000 7f3eed133640 1 -- 172.21.15.43:0/4254931536 learned_addr learned my addr 172.21.15.43:0/4254931536 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:19:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.190+0000 7f3eec932640 1 --2- 172.21.15.43:0/4254931536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ee8102e00 0x7f3ee8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.190+0000 7f3eed934640 1 --2- 172.21.15.43:0/4254931536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee81037c0 0x7f3ee819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.190+0000 7f3eed133640 1 -- 172.21.15.43:0/4254931536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee81037c0 msgr2=0x7f3ee819f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.190+0000 7f3eed133640 1 --2- 172.21.15.43:0/4254931536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee81037c0 0x7f3ee819f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.190+0000 7f3eed133640 1 -- 172.21.15.43:0/4254931536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ee8102e00 msgr2=0x7f3ee8198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.190+0000 7f3eed133640 1 --2- 172.21.15.43:0/4254931536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ee8102e00 0x7f3ee8198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.190+0000 7f3eed133640 1 -- 172.21.15.43:0/4254931536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee819fb40 con 0x7f3ee8101c00 2024-09-11T23:19:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.190+0000 7f3eed934640 1 --2- 172.21.15.43:0/4254931536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee81037c0 0x7f3ee819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:19:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.190+0000 7f3eed133640 1 --2- 172.21.15.43:0/4254931536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee8101c00 0x7f3ee8197f10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4031610 tx=0x7f3ee40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.191+0000 7f3ede7fc640 1 -- 172.21.15.43:0/4254931536 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee403f030 con 0x7f3ee8101c00 2024-09-11T23:19:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.191+0000 7f3ede7fc640 1 -- 172.21.15.43:0/4254931536 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ee4003ce0 con 0x7f3ee8101c00 2024-09-11T23:19:08.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.191+0000 7f3eef397640 1 -- 172.21.15.43:0/4254931536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee819fdd0 con 0x7f3ee8101c00 2024-09-11T23:19:08.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.191+0000 7f3ede7fc640 1 -- 172.21.15.43:0/4254931536 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4035070 con 0x7f3ee8101c00 2024-09-11T23:19:08.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.191+0000 7f3eef397640 1 -- 172.21.15.43:0/4254931536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee81a01f0 con 0x7f3ee8101c00 2024-09-11T23:19:08.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.192+0000 7f3eef397640 1 -- 172.21.15.43:0/4254931536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee8108ff0 con 0x7f3ee8101c00 2024-09-11T23:19:08.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.193+0000 7f3ede7fc640 1 -- 172.21.15.43:0/4254931536 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3ee4042030 con 0x7f3ee8101c00 2024-09-11T23:19:08.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.194+0000 7f3ede7fc640 1 --2- 172.21.15.43:0/4254931536 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3ed40a2bb0 0x7f3ed40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:08.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.194+0000 7f3eec932640 1 --2- 172.21.15.43:0/4254931536 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3ed40a2bb0 0x7f3ed40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:08.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.194+0000 7f3ede7fc640 1 -- 172.21.15.43:0/4254931536 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f3ee40ef4f0 con 0x7f3ee8101c00 2024-09-11T23:19:08.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.194+0000 7f3eec932640 1 --2- 172.21.15.43:0/4254931536 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3ed40a2bb0 0x7f3ed40a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8199430 tx=0x7f3ed800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:08.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.197+0000 7f3ede7fc640 1 -- 172.21.15.43:0/4254931536 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ee40bc850 con 0x7f3ee8101c00 2024-09-11T23:19:08.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:07 smithi090 ceph-mon[35130]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:19:08.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:07 smithi090 ceph-mon[35130]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-11T23:19:08.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:07 smithi090 ceph-mon[35130]: osdmap e75: 12 total, 11 up, 12 in 2024-09-11T23:19:08.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:19:08.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:19:08.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:07 smithi090 ceph-mon[35130]: Detected new or changed devices on smithi097 2024-09-11T23:19:08.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:08.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:08.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:19:08.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:19:08.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:19:08.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:19:08.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:07 smithi090 ceph-mon[35130]: Adjusting osd_memory_target on smithi097 to 4272M 2024-09-11T23:19:08.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:08.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:08.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:19:08.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:08.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:07 smithi097 ceph-mon[35562]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:19:08.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:07 smithi097 ceph-mon[35562]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-11T23:19:08.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:07 smithi097 ceph-mon[35562]: osdmap e75: 12 total, 11 up, 12 in 2024-09-11T23:19:08.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:19:08.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:19:08.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:07 smithi097 ceph-mon[35562]: Detected new or changed devices on smithi097 2024-09-11T23:19:08.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:08.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:08.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:19:08.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:19:08.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:19:08.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:19:08.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:07 smithi097 ceph-mon[35562]: Adjusting osd_memory_target on smithi097 to 4272M 2024-09-11T23:19:08.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:08.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:08.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:19:08.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:08.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.373+0000 7f3eef397640 1 -- 172.21.15.43:0/4254931536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3ee8062e90 con 0x7f3ee8101c00 2024-09-11T23:19:08.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.373+0000 7f3ede7fc640 1 -- 172.21.15.43:0/4254931536 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7f3ee40bc150 con 0x7f3ee8101c00 2024-09-11T23:19:08.373 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:19:08.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.375+0000 7f3eef397640 1 -- 172.21.15.43:0/4254931536 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3ed40a2bb0 msgr2=0x7f3ed40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:08.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.375+0000 7f3eef397640 1 --2- 172.21.15.43:0/4254931536 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3ed40a2bb0 0x7f3ed40a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8199430 tx=0x7f3ed800a250 comp rx=0 tx=0).stop 2024-09-11T23:19:08.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.375+0000 7f3eef397640 1 -- 172.21.15.43:0/4254931536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee8101c00 msgr2=0x7f3ee8197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:08.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.375+0000 7f3eef397640 1 --2- 172.21.15.43:0/4254931536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee8101c00 0x7f3ee8197f10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4031610 tx=0x7f3ee40043b0 comp rx=0 tx=0).stop 2024-09-11T23:19:08.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.375+0000 7f3eef397640 1 -- 172.21.15.43:0/4254931536 shutdown_connections 2024-09-11T23:19:08.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.375+0000 7f3eef397640 1 --2- 172.21.15.43:0/4254931536 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3ed40a2bb0 0x7f3ed40a5070 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:08.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.375+0000 7f3eef397640 1 --2- 172.21.15.43:0/4254931536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee81037c0 0x7f3ee819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:08.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.375+0000 7f3eef397640 1 --2- 172.21.15.43:0/4254931536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ee8102e00 0x7f3ee8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:08.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.375+0000 7f3eef397640 1 --2- 172.21.15.43:0/4254931536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee8101c00 0x7f3ee8197f10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:08.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.375+0000 7f3eef397640 1 -- 172.21.15.43:0/4254931536 >> 172.21.15.43:0/4254931536 conn(0x7f3ee80fd390 msgr2=0x7f3ee81061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:08.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.375+0000 7f3eef397640 1 -- 172.21.15.43:0/4254931536 shutdown_connections 2024-09-11T23:19:08.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:08.375+0000 7f3eef397640 1 -- 172.21.15.43:0/4254931536 wait complete. 2024-09-11T23:19:09.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:08 smithi043 ceph-mon[30950]: purged_snaps scrub starts 2024-09-11T23:19:09.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:08 smithi043 ceph-mon[30950]: purged_snaps scrub ok 2024-09-11T23:19:09.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:08 smithi043 ceph-mon[30950]: osd.11 [v2:172.21.15.97:6824/194699315,v1:172.21.15.97:6825/194699315] boot 2024-09-11T23:19:09.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:08 smithi043 ceph-mon[30950]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T23:19:09.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:08 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:19:09.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:08 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/4254931536' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:19:09.239 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1726096747,"num_in_osds":12,"osd_in_since":1726096724,"num_remapped_pgs":0} 2024-09-11T23:19:09.239 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:19:09.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:08 smithi090 ceph-mon[35130]: purged_snaps scrub starts 2024-09-11T23:19:09.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:08 smithi090 ceph-mon[35130]: purged_snaps scrub ok 2024-09-11T23:19:09.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:08 smithi090 ceph-mon[35130]: osd.11 [v2:172.21.15.97:6824/194699315,v1:172.21.15.97:6825/194699315] boot 2024-09-11T23:19:09.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:08 smithi090 ceph-mon[35130]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T23:19:09.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:08 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:19:09.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:08 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/4254931536' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:19:09.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:08 smithi097 ceph-mon[35562]: purged_snaps scrub starts 2024-09-11T23:19:09.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:08 smithi097 ceph-mon[35562]: purged_snaps scrub ok 2024-09-11T23:19:09.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:08 smithi097 ceph-mon[35562]: osd.11 [v2:172.21.15.97:6824/194699315,v1:172.21.15.97:6825/194699315] boot 2024-09-11T23:19:09.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:08 smithi097 ceph-mon[35562]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T23:19:09.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:08 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:19:09.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:08 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/4254931536' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T23:19:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:10.074 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:09 smithi043 ceph-mon[30950]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:19:10.074 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:09 smithi043 ceph-mon[30950]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T23:19:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.156+0000 7f7801d65640 1 -- 172.21.15.43:0/1855048644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77fc100a80 msgr2=0x7f77fc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.156+0000 7f7801d65640 1 --2- 172.21.15.43:0/1855048644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77fc100a80 0x7f77fc100ee0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009930 tx=0x7f77f002a420 comp rx=0 tx=0).stop 2024-09-11T23:19:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.156+0000 7f7801d65640 1 -- 172.21.15.43:0/1855048644 shutdown_connections 2024-09-11T23:19:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.156+0000 7f7801d65640 1 --2- 172.21.15.43:0/1855048644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77fc101420 0x7f77fc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.156+0000 7f7801d65640 1 --2- 172.21.15.43:0/1855048644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77fc100a80 0x7f77fc100ee0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.156+0000 7f7801d65640 1 --2- 172.21.15.43:0/1855048644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77fc106a80 0x7f77fc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.156+0000 7f7801d65640 1 -- 172.21.15.43:0/1855048644 >> 172.21.15.43:0/1855048644 conn(0x7f77fc0fc7a0 msgr2=0x7f77fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:10.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.156+0000 7f7801d65640 1 -- 172.21.15.43:0/1855048644 shutdown_connections 2024-09-11T23:19:10.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.156+0000 7f7801d65640 1 -- 172.21.15.43:0/1855048644 wait complete. 2024-09-11T23:19:10.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.157+0000 7f7801d65640 1 Processor -- start 2024-09-11T23:19:10.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.157+0000 7f7801d65640 1 -- start start 2024-09-11T23:19:10.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.157+0000 7f7801d65640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77fc100a80 0x7f77fc199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:10.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.157+0000 7f7801d65640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77fc101420 0x7f77fc199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:10.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.157+0000 7f7801d65640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77fc106a80 0x7f77fc1a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:10.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.157+0000 7f7801d65640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc110ec0 con 0x7f77fc101420 2024-09-11T23:19:10.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.157+0000 7f7801d65640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc110d40 con 0x7f77fc106a80 2024-09-11T23:19:10.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.157+0000 7f7801d65640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc111040 con 0x7f77fc100a80 2024-09-11T23:19:10.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.157+0000 7f77faffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77fc101420 0x7f77fc199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:10.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.157+0000 7f77faffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77fc101420 0x7f77fc199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39042/0 (socket says 172.21.15.43:39042) 2024-09-11T23:19:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.158+0000 7f77faffd640 1 -- 172.21.15.43:0/2713423142 learned_addr learned my addr 172.21.15.43:0/2713423142 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:19:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.158+0000 7f77fb7fe640 1 --2- 172.21.15.43:0/2713423142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77fc100a80 0x7f77fc199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.158+0000 7f77fbfff640 1 --2- 172.21.15.43:0/2713423142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77fc106a80 0x7f77fc1a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.158+0000 7f77faffd640 1 -- 172.21.15.43:0/2713423142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77fc100a80 msgr2=0x7f77fc199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.158+0000 7f77faffd640 1 --2- 172.21.15.43:0/2713423142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77fc100a80 0x7f77fc199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.158+0000 7f77faffd640 1 -- 172.21.15.43:0/2713423142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77fc106a80 msgr2=0x7f77fc1a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.158+0000 7f77faffd640 1 --2- 172.21.15.43:0/2713423142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77fc106a80 0x7f77fc1a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.158+0000 7f77faffd640 1 -- 172.21.15.43:0/2713423142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77fc1a4910 con 0x7f77fc101420 2024-09-11T23:19:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.158+0000 7f77faffd640 1 --2- 172.21.15.43:0/2713423142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77fc101420 0x7f77fc199890 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009230 tx=0x7f77f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.158+0000 7f77f8ff9640 1 -- 172.21.15.43:0/2713423142 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f003f030 con 0x7f77fc101420 2024-09-11T23:19:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.158+0000 7f77f8ff9640 1 -- 172.21.15.43:0/2713423142 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77f0003ad0 con 0x7f77fc101420 2024-09-11T23:19:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.158+0000 7f77f8ff9640 1 -- 172.21.15.43:0/2713423142 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0035070 con 0x7f77fc101420 2024-09-11T23:19:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.158+0000 7f7801d65640 1 -- 172.21.15.43:0/2713423142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77fc1a4ae0 con 0x7f77fc101420 2024-09-11T23:19:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.158+0000 7f7801d65640 1 -- 172.21.15.43:0/2713423142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77fc1a4ee0 con 0x7f77fc101420 2024-09-11T23:19:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.159+0000 7f7801d65640 1 -- 172.21.15.43:0/2713423142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77fc100ee0 con 0x7f77fc101420 2024-09-11T23:19:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.161+0000 7f77f8ff9640 1 -- 172.21.15.43:0/2713423142 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f77f0042030 con 0x7f77fc101420 2024-09-11T23:19:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.161+0000 7f77f8ff9640 1 --2- 172.21.15.43:0/2713423142 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f77e40a2b80 0x7f77e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.161+0000 7f77fb7fe640 1 --2- 172.21.15.43:0/2713423142 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f77e40a2b80 0x7f77e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.161+0000 7f77f8ff9640 1 -- 172.21.15.43:0/2713423142 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f77f00ef4d0 con 0x7f77fc101420 2024-09-11T23:19:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.162+0000 7f77fb7fe640 1 --2- 172.21.15.43:0/2713423142 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f77e40a2b80 0x7f77e40a5040 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f77ec005fd0 tx=0x7f77ec00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.165+0000 7f77f8ff9640 1 -- 172.21.15.43:0/2713423142 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77f00bc8d0 con 0x7f77fc101420 2024-09-11T23:19:10.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:09 smithi090 ceph-mon[35130]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:19:10.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:09 smithi090 ceph-mon[35130]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T23:19:10.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:09 smithi097 ceph-mon[35562]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-11T23:19:10.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:09 smithi097 ceph-mon[35562]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T23:19:10.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.340+0000 7f7801d65640 1 -- 172.21.15.43:0/2713423142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77fc06ad80 con 0x7f77fc101420 2024-09-11T23:19:10.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.341+0000 7f77f8ff9640 1 -- 172.21.15.43:0/2713423142 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+18334 (secure 0 0 0) 0x7f77f00dabb0 con 0x7f77fc101420 2024-09-11T23:19:10.342 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:19:10.342 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":77,"fsid":"25c0cd84-7093-11ef-bcea-c7b262605968","created":"2024-09-11T23:12:03.504406+0000","modified":"2024-09-11T23:19:08.804606+0000","last_up_change":"2024-09-11T23:19:07.797873+0000","last_in_change":"2024-09-11T23:18:44.595176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:15:17.058713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24076596-3909-4c87-85f1-cbf168070dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6803","nonce":43839997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6805","nonce":43839997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6809","nonce":43839997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6807","nonce":43839997}]},"public_addr":"172.21.15.43:6803/43839997","cluster_addr":"172.21.15.43:6805/43839997","heartbeat_back_addr":"172.21.15.43:6809/43839997","heartbeat_front_addr":"172.21.15.43:6807/43839997","state":["exists","up"]},{"osd":1,"uuid":"a0472a57-243c-489d-a247-972ee10a71f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6801","nonce":1825149246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6803","nonce":1825149246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6807","nonce":1825149246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6805","nonce":1825149246}]},"public_addr":"172.21.15.90:6801/1825149246","cluster_addr":"172.21.15.90:6803/1825149246","heartbeat_back_addr":"172.21.15.90:6807/1825149246","heartbeat_front_addr":"172.21.15.90:6805/1825149246","state":["exists","up"]},{"osd":2,"uuid":"40366536-0fcf-45a0-b8e3-49178401dc34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6801","nonce":1456846443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6803","nonce":1456846443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6807","nonce":1456846443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6805","nonce":1456846443}]},"public_addr":"172.21.15.97:6801/1456846443","cluster_addr":"172.21.15.97:6803/1456846443","heartbeat_back_addr":"172.21.15.97:6807/1456846443","heartbeat_front_addr":"172.21.15.97:6805/1456846443","state":["exists","up"]},{"osd":3,"uuid":"3062133e-000b-4421-9d53-ac434b9ed9b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6811","nonce":1962837804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6813","nonce":1962837804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6817","nonce":1962837804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6815","nonce":1962837804}]},"public_addr":"172.21.15.43:6811/1962837804","cluster_addr":"172.21.15.43:6813/1962837804","heartbeat_back_addr":"172.21.15.43:6817/1962837804","heartbeat_front_addr":"172.21.15.43:6815/1962837804","state":["exists","up"]},{"osd":4,"uuid":"2e73fa4b-f22c-4319-828d-87b91b9f19f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6809","nonce":181614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6811","nonce":181614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6815","nonce":181614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6813","nonce":181614173}]},"public_addr":"172.21.15.90:6809/181614173","cluster_addr":"172.21.15.90:6811/181614173","heartbeat_back_addr":"172.21.15.90:6815/181614173","heartbeat_front_addr":"172.21.15.90:6813/181614173","state":["exists","up"]},{"osd":5,"uuid":"1e1ad4f9-e724-440f-b515-3ec1ad97d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6809","nonce":1540039543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6811","nonce":1540039543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6815","nonce":1540039543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6813","nonce":1540039543}]},"public_addr":"172.21.15.97:6809/1540039543","cluster_addr":"172.21.15.97:6811/1540039543","heartbeat_back_addr":"172.21.15.97:6815/1540039543","heartbeat_front_addr":"172.21.15.97:6813/1540039543","state":["exists","up"]},{"osd":6,"uuid":"d34df8df-f40d-4159-8b29-5401b1b0e431","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6819","nonce":3785677637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6821","nonce":3785677637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6825","nonce":3785677637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6823","nonce":3785677637}]},"public_addr":"172.21.15.43:6819/3785677637","cluster_addr":"172.21.15.43:6821/3785677637","heartbeat_back_addr":"172.21.15.43:6825/3785677637","heartbeat_front_addr":"172.21.15.43:6823/3785677637","state":["exists","up"]},{"osd":7,"uuid":"b2e01078-528b-49fd-b000-9f584f784b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6817","nonce":236874962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6819","nonce":236874962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6823","nonce":236874962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6821","nonce":236874962}]},"public_addr":"172.21.15.90:6817/236874962","cluster_addr":"172.21.15.90:6819/236874962","heartbeat_back_addr":"172.21.15.90:6823/236874962","heartbeat_front_addr":"172.21.15.90:6821/236874962","state":["exists","up"]},{"osd":8,"uuid":"abef8f27-bf9b-4d63-bfbe-f967fc3d8b17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6817","nonce":1707327301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6819","nonce":1707327301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6823","nonce":1707327301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6821","nonce":1707327301}]},"public_addr":"172.21.15.97:6817/1707327301","cluster_addr":"172.21.15.97:6819/1707327301","heartbeat_back_addr":"172.21.15.97:6823/1707327301","heartbeat_front_addr":"172.21.15.97:6821/1707327301","state":["exists","up"]},{"osd":9,"uuid":"190ab8b2-a835-47e6-9b39-16a58331c58a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6827","nonce":3170306011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6829","nonce":3170306011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6833","nonce":3170306011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6831","nonce":3170306011}]},"public_addr":"172.21.15.43:6827/3170306011","cluster_addr":"172.21.15.43:6829/3170306011","heartbeat_back_addr":"172.21.15.43:6833/3170306011","heartbeat_front_addr":"172.21.15.43:6831/3170306011","state":["exists","up"]},{"osd":10,"uuid":"b32ab5fe-466f-4c8e-8696-cd9f820a3263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6825","nonce":59487041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6827","nonce":59487041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6831","nonce":59487041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6829","nonce":59487041}]},"public_addr":"172.21.15.90:6825/59487041","cluster_addr":"172.21.15.90:6827/59487041","heartbeat_back_addr":"172.21.15.90:6831/59487041","heartbeat_front_addr":"172.21.15.90:6829/59487041","state":["exists","up"]},{"osd":11,"uuid":"d0faa589-21c8-4a9f-8a73-813aafe6d33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6825","nonce":194699315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6827","nonce":194699315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6831","nonce":194699315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6829","nonce":194699315}]},"public_addr":"172.21.15.97:6825/194699315","cluster_addr":"172.21.15.97:6827/194699315","heartbeat_back_addr":"172.21.15.97:6831/194699315","heartbeat_front_addr":"172.21.15.97:6829/194699315","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:23.381247+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:48.843190+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:13.538979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:40.136983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:05.513790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:31.247090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:56.635221+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:22.667327+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:47.910422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:14.353914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:40.584503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:19:06.476766+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/76575371":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/4006124758":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/120495444":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6800/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:6800/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/206044413":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1512201905":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1375379108":"2024-09-12T23:12:35.756598+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:19:10.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.344+0000 7f7801d65640 1 -- 172.21.15.43:0/2713423142 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f77e40a2b80 msgr2=0x7f77e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:10.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.344+0000 7f7801d65640 1 --2- 172.21.15.43:0/2713423142 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f77e40a2b80 0x7f77e40a5040 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f77ec005fd0 tx=0x7f77ec00b390 comp rx=0 tx=0).stop 2024-09-11T23:19:10.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.344+0000 7f7801d65640 1 -- 172.21.15.43:0/2713423142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77fc101420 msgr2=0x7f77fc199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:10.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.344+0000 7f7801d65640 1 --2- 172.21.15.43:0/2713423142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77fc101420 0x7f77fc199890 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009230 tx=0x7f77f0004270 comp rx=0 tx=0).stop 2024-09-11T23:19:10.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.344+0000 7f7801d65640 1 -- 172.21.15.43:0/2713423142 shutdown_connections 2024-09-11T23:19:10.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.344+0000 7f7801d65640 1 --2- 172.21.15.43:0/2713423142 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f77e40a2b80 0x7f77e40a5040 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:10.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.344+0000 7f7801d65640 1 --2- 172.21.15.43:0/2713423142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77fc106a80 0x7f77fc1a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:10.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.344+0000 7f7801d65640 1 --2- 172.21.15.43:0/2713423142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77fc101420 0x7f77fc199890 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:10.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.344+0000 7f7801d65640 1 --2- 172.21.15.43:0/2713423142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77fc100a80 0x7f77fc199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:10.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.344+0000 7f7801d65640 1 -- 172.21.15.43:0/2713423142 >> 172.21.15.43:0/2713423142 conn(0x7f77fc0fc7a0 msgr2=0x7f77fc0fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:10.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.344+0000 7f7801d65640 1 -- 172.21.15.43:0/2713423142 shutdown_connections 2024-09-11T23:19:10.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:10.344+0000 7f7801d65640 1 -- 172.21.15.43:0/2713423142 wait complete. 2024-09-11T23:19:11.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:10 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2713423142' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:19:11.159 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T23:15:17.058713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T23:19:11.159 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-11T23:19:11.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:10 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2713423142' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:19:11.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:10 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2713423142' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:19:11.417 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:12.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.010+0000 7f47a6651640 1 -- 172.21.15.43:0/535000669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a0106a60 msgr2=0x7f47a0106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:12.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.010+0000 7f47a6651640 1 --2- 172.21.15.43:0/535000669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a0106a60 0x7f47a0106e40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4790009930 tx=0x7f479002a420 comp rx=0 tx=0).stop 2024-09-11T23:19:12.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.010+0000 7f47a6651640 1 -- 172.21.15.43:0/535000669 shutdown_connections 2024-09-11T23:19:12.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.010+0000 7f47a6651640 1 --2- 172.21.15.43:0/535000669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47a0101400 0x7f47a010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:12.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.010+0000 7f47a6651640 1 --2- 172.21.15.43:0/535000669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a0100a60 0x7f47a0100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:12.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.010+0000 7f47a6651640 1 --2- 172.21.15.43:0/535000669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a0106a60 0x7f47a0106e40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:12.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.010+0000 7f47a6651640 1 -- 172.21.15.43:0/535000669 >> 172.21.15.43:0/535000669 conn(0x7f47a00fc780 msgr2=0x7f47a00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:12.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.010+0000 7f47a6651640 1 -- 172.21.15.43:0/535000669 shutdown_connections 2024-09-11T23:19:12.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.010+0000 7f47a6651640 1 -- 172.21.15.43:0/535000669 wait complete. 2024-09-11T23:19:12.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.011+0000 7f47a6651640 1 Processor -- start 2024-09-11T23:19:12.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.011+0000 7f47a6651640 1 -- start start 2024-09-11T23:19:12.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.011+0000 7f47a6651640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47a0100a60 0x7f47a0197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:12.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.011+0000 7f47a6651640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a0101400 0x7f47a0197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:12.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.011+0000 7f47a6651640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a0106a60 0x7f47a0197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:12.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.011+0000 7f47a6651640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f47a0110e60 con 0x7f47a0106a60 2024-09-11T23:19:12.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.011+0000 7f47a6651640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f47a0110ce0 con 0x7f47a0100a60 2024-09-11T23:19:12.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.011+0000 7f47a6651640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f47a0110fe0 con 0x7f47a0101400 2024-09-11T23:19:12.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.011+0000 7f47a4bee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a0106a60 0x7f47a0197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:12.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.011+0000 7f47a4bee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a0106a60 0x7f47a0197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58574/0 (socket says 172.21.15.43:58574) 2024-09-11T23:19:12.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.011+0000 7f47a4bee640 1 -- 172.21.15.43:0/110453968 learned_addr learned my addr 172.21.15.43:0/110453968 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:19:12.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.011+0000 7f479ffff640 1 --2- 172.21.15.43:0/110453968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47a0100a60 0x7f47a0197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:12.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.011+0000 7f4797fff640 1 --2- 172.21.15.43:0/110453968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a0101400 0x7f47a0197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:12.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.012+0000 7f47a4bee640 1 -- 172.21.15.43:0/110453968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a0101400 msgr2=0x7f47a0197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:12.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.012+0000 7f47a4bee640 1 --2- 172.21.15.43:0/110453968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a0101400 0x7f47a0197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:12.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.012+0000 7f47a4bee640 1 -- 172.21.15.43:0/110453968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47a0100a60 msgr2=0x7f47a0197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:12.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.012+0000 7f47a4bee640 1 --2- 172.21.15.43:0/110453968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47a0100a60 0x7f47a0197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:12.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.012+0000 7f47a4bee640 1 -- 172.21.15.43:0/110453968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a01a6990 con 0x7f47a0106a60 2024-09-11T23:19:12.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.012+0000 7f47a4bee640 1 --2- 172.21.15.43:0/110453968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a0106a60 0x7f47a0197ba0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f479800eaa0 tx=0x7f479800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:12.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.012+0000 7f479dffb640 1 -- 172.21.15.43:0/110453968 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479800cdb0 con 0x7f47a0106a60 2024-09-11T23:19:12.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.012+0000 7f479dffb640 1 -- 172.21.15.43:0/110453968 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4798005020 con 0x7f47a0106a60 2024-09-11T23:19:12.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.012+0000 7f479dffb640 1 -- 172.21.15.43:0/110453968 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47980116b0 con 0x7f47a0106a60 2024-09-11T23:19:12.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.012+0000 7f47a6651640 1 -- 172.21.15.43:0/110453968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a01a6c80 con 0x7f47a0106a60 2024-09-11T23:19:12.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.012+0000 7f47a6651640 1 -- 172.21.15.43:0/110453968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a01a7170 con 0x7f47a0106a60 2024-09-11T23:19:12.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.013+0000 7f47a6651640 1 -- 172.21.15.43:0/110453968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a0100ec0 con 0x7f47a0106a60 2024-09-11T23:19:12.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.014+0000 7f479dffb640 1 -- 172.21.15.43:0/110453968 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4798005560 con 0x7f47a0106a60 2024-09-11T23:19:12.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.015+0000 7f479dffb640 1 --2- 172.21.15.43:0/110453968 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f478c0a2b80 0x7f478c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:12.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.015+0000 7f479ffff640 1 --2- 172.21.15.43:0/110453968 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f478c0a2b80 0x7f478c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:12.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.015+0000 7f479dffb640 1 -- 172.21.15.43:0/110453968 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f47980ceda0 con 0x7f47a0106a60 2024-09-11T23:19:12.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.016+0000 7f479ffff640 1 --2- 172.21.15.43:0/110453968 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f478c0a2b80 0x7f478c0a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4790009900 tx=0x7f47900058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:12.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.019+0000 7f479dffb640 1 -- 172.21.15.43:0/110453968 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4798016030 con 0x7f47a0106a60 2024-09-11T23:19:12.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:11 smithi043 ceph-mon[30950]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:12.171 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:11 smithi090 ceph-mon[35130]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:12.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.194+0000 7f47a6651640 1 -- 172.21.15.43:0/110453968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f47a002d7c0 con 0x7f47a0106a60 2024-09-11T23:19:12.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.195+0000 7f479dffb640 1 -- 172.21.15.43:0/110453968 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f479809bab0 con 0x7f47a0106a60 2024-09-11T23:19:12.195 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-09-11T23:19:12.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.197+0000 7f47a6651640 1 -- 172.21.15.43:0/110453968 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f478c0a2b80 msgr2=0x7f478c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:12.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.197+0000 7f47a6651640 1 --2- 172.21.15.43:0/110453968 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f478c0a2b80 0x7f478c0a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4790009900 tx=0x7f47900058d0 comp rx=0 tx=0).stop 2024-09-11T23:19:12.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.197+0000 7f47a6651640 1 -- 172.21.15.43:0/110453968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a0106a60 msgr2=0x7f47a0197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:12.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.197+0000 7f47a6651640 1 --2- 172.21.15.43:0/110453968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a0106a60 0x7f47a0197ba0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f479800eaa0 tx=0x7f479800ef60 comp rx=0 tx=0).stop 2024-09-11T23:19:12.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.197+0000 7f47a6651640 1 -- 172.21.15.43:0/110453968 shutdown_connections 2024-09-11T23:19:12.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.197+0000 7f47a6651640 1 --2- 172.21.15.43:0/110453968 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f478c0a2b80 0x7f478c0a5040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:12.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.197+0000 7f47a6651640 1 --2- 172.21.15.43:0/110453968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a0106a60 0x7f47a0197ba0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:12.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.197+0000 7f47a6651640 1 --2- 172.21.15.43:0/110453968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a0101400 0x7f47a0197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:12.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.197+0000 7f47a6651640 1 --2- 172.21.15.43:0/110453968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47a0100a60 0x7f47a0197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:12.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.197+0000 7f47a6651640 1 -- 172.21.15.43:0/110453968 >> 172.21.15.43:0/110453968 conn(0x7f47a00fc780 msgr2=0x7f47a00fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:12.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.197+0000 7f47a6651640 1 -- 172.21.15.43:0/110453968 shutdown_connections 2024-09-11T23:19:12.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:12.197+0000 7f47a6651640 1 -- 172.21.15.43:0/110453968 wait complete. 2024-09-11T23:19:12.202 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:11 smithi097 ceph-mon[35562]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:12.961 INFO:tasks.cephadm:Adding mds.a on smithi043 2024-09-11T23:19:12.961 INFO:tasks.cephadm:Adding mds.d on smithi043 2024-09-11T23:19:12.962 INFO:tasks.cephadm:Adding mds.g on smithi043 2024-09-11T23:19:12.962 INFO:tasks.cephadm:Adding mds.j on smithi043 2024-09-11T23:19:12.962 INFO:tasks.cephadm:Adding mds.b on smithi090 2024-09-11T23:19:12.962 INFO:tasks.cephadm:Adding mds.e on smithi090 2024-09-11T23:19:12.962 INFO:tasks.cephadm:Adding mds.h on smithi090 2024-09-11T23:19:12.962 INFO:tasks.cephadm:Adding mds.k on smithi090 2024-09-11T23:19:12.962 INFO:tasks.cephadm:Adding mds.c on smithi097 2024-09-11T23:19:12.962 INFO:tasks.cephadm:Adding mds.f on smithi097 2024-09-11T23:19:12.962 INFO:tasks.cephadm:Adding mds.i on smithi097 2024-09-11T23:19:12.962 INFO:tasks.cephadm:Adding mds.l on smithi097 2024-09-11T23:19:12.962 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi097=c;smithi097=f;smithi097=i;smithi097=l' 2024-09-11T23:19:13.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:12 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/110453968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:19:13.212 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:19:13.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:12 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/110453968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:19:13.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:12 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/110453968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:19:13.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.703+0000 7fb22986a640 1 -- 172.21.15.97:0/1506016243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2241078c0 msgr2=0x7fb224109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:13.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.703+0000 7fb22986a640 1 --2- 172.21.15.97:0/1506016243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2241078c0 0x7fb224109d10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009ca0 tx=0x7fb21c02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:19:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.703+0000 7fb22986a640 1 -- 172.21.15.97:0/1506016243 shutdown_connections 2024-09-11T23:19:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.703+0000 7fb22986a640 1 --2- 172.21.15.97:0/1506016243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2241078c0 0x7fb224109d10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.703+0000 7fb22986a640 1 --2- 172.21.15.97:0/1506016243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb224104f90 0x7fb224107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.703+0000 7fb22986a640 1 --2- 172.21.15.97:0/1506016243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb224102660 0x7fb224104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.703+0000 7fb22986a640 1 -- 172.21.15.97:0/1506016243 >> 172.21.15.97:0/1506016243 conn(0x7fb2240fc280 msgr2=0x7fb2240fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.703+0000 7fb22986a640 1 -- 172.21.15.97:0/1506016243 shutdown_connections 2024-09-11T23:19:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.703+0000 7fb22986a640 1 -- 172.21.15.97:0/1506016243 wait complete. 2024-09-11T23:19:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.703+0000 7fb22986a640 1 Processor -- start 2024-09-11T23:19:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.704+0000 7fb22986a640 1 -- start start 2024-09-11T23:19:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.704+0000 7fb22986a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb224102660 0x7fb224197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.704+0000 7fb22986a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb224104f90 0x7fb224198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.704+0000 7fb22986a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2241078c0 0x7fb22419f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:13.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.704+0000 7fb22986a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb22410d860 con 0x7fb2241078c0 2024-09-11T23:19:13.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.704+0000 7fb22986a640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb22410d6e0 con 0x7fb224102660 2024-09-11T23:19:13.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.704+0000 7fb22986a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb22410d9e0 con 0x7fb224104f90 2024-09-11T23:19:13.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.704+0000 7fb2227fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb224104f90 0x7fb224198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:13.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.704+0000 7fb2227fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb224104f90 0x7fb224198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58786/0 (socket says 172.21.15.97:58786) 2024-09-11T23:19:13.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.704+0000 7fb2227fc640 1 -- 172.21.15.97:0/2076434656 learned_addr learned my addr 172.21.15.97:0/2076434656 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-11T23:19:13.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.704+0000 7fb2237fe640 1 --2- 172.21.15.97:0/2076434656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2241078c0 0x7fb22419f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:13.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.704+0000 7fb2227fc640 1 -- 172.21.15.97:0/2076434656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb224102660 msgr2=0x7fb224197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:13.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.704+0000 7fb2227fc640 1 --2- 172.21.15.97:0/2076434656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb224102660 0x7fb224197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:13.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.704+0000 7fb2227fc640 1 -- 172.21.15.97:0/2076434656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2241078c0 msgr2=0x7fb22419f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:13.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.704+0000 7fb2227fc640 1 --2- 172.21.15.97:0/2076434656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2241078c0 0x7fb22419f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:13.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.704+0000 7fb2227fc640 1 -- 172.21.15.97:0/2076434656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22419faf0 con 0x7fb224104f90 2024-09-11T23:19:13.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.704+0000 7fb2237fe640 1 --2- 172.21.15.97:0/2076434656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2241078c0 0x7fb22419f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:19:13.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.704+0000 7fb2227fc640 1 --2- 172.21.15.97:0/2076434656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb224104f90 0x7fb224198400 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb21400c970 tx=0x7fb21400ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:13.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.705+0000 7fb228868640 1 -- 172.21.15.97:0/2076434656 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214007bb0 con 0x7fb224104f90 2024-09-11T23:19:13.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.705+0000 7fb228868640 1 -- 172.21.15.97:0/2076434656 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb214007d50 con 0x7fb224104f90 2024-09-11T23:19:13.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.705+0000 7fb22986a640 1 -- 172.21.15.97:0/2076434656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb22419fde0 con 0x7fb224104f90 2024-09-11T23:19:13.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.705+0000 7fb228868640 1 -- 172.21.15.97:0/2076434656 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21400f450 con 0x7fb224104f90 2024-09-11T23:19:13.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.705+0000 7fb22986a640 1 -- 172.21.15.97:0/2076434656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2241a0290 con 0x7fb224104f90 2024-09-11T23:19:13.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.706+0000 7fb22986a640 1 -- 172.21.15.97:0/2076434656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2240fdd70 con 0x7fb224104f90 2024-09-11T23:19:13.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.707+0000 7fb228868640 1 -- 172.21.15.97:0/2076434656 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb214004f70 con 0x7fb224104f90 2024-09-11T23:19:13.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.708+0000 7fb228868640 1 --2- 172.21.15.97:0/2076434656 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb2100a2bb0 0x7fb2100a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:13.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.708+0000 7fb228868640 1 -- 172.21.15.97:0/2076434656 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fb2140cec60 con 0x7fb224104f90 2024-09-11T23:19:13.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.708+0000 7fb222ffd640 1 --2- 172.21.15.97:0/2076434656 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb2100a2bb0 0x7fb2100a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:13.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.708+0000 7fb222ffd640 1 --2- 172.21.15.97:0/2076434656 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb2100a2bb0 0x7fb2100a5070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00ba80 tx=0x7fb20c005ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:13.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.711+0000 7fb228868640 1 -- 172.21.15.97:0/2076434656 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb214014030 con 0x7fb224104f90 2024-09-11T23:19:13.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.887+0000 7fb22986a640 1 -- 172.21.15.97:0/2076434656 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi097=c;smithi097=f;smithi097=i;smithi097=l", "target": ["mon-mgr", ""]}) -- 0x7fb224062730 con 0x7fb2100a2bb0 2024-09-11T23:19:13.887 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:13 smithi097 ceph-mon[35562]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:13.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.894+0000 7fb228868640 1 -- 172.21.15.97:0/2076434656 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fb224062730 con 0x7fb2100a2bb0 2024-09-11T23:19:13.895 INFO:teuthology.orchestra.run.smithi097.stdout:Scheduled mds.all update... 2024-09-11T23:19:13.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.896+0000 7fb22986a640 1 -- 172.21.15.97:0/2076434656 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb2100a2bb0 msgr2=0x7fb2100a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:13.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.896+0000 7fb22986a640 1 --2- 172.21.15.97:0/2076434656 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb2100a2bb0 0x7fb2100a5070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00ba80 tx=0x7fb20c005ed0 comp rx=0 tx=0).stop 2024-09-11T23:19:13.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.896+0000 7fb22986a640 1 -- 172.21.15.97:0/2076434656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb224104f90 msgr2=0x7fb224198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:13.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.896+0000 7fb22986a640 1 --2- 172.21.15.97:0/2076434656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb224104f90 0x7fb224198400 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb21400c970 tx=0x7fb21400ce30 comp rx=0 tx=0).stop 2024-09-11T23:19:13.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.896+0000 7fb22986a640 1 -- 172.21.15.97:0/2076434656 shutdown_connections 2024-09-11T23:19:13.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.896+0000 7fb22986a640 1 --2- 172.21.15.97:0/2076434656 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb2100a2bb0 0x7fb2100a5070 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:13.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.896+0000 7fb22986a640 1 --2- 172.21.15.97:0/2076434656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2241078c0 0x7fb22419f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:13.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.896+0000 7fb22986a640 1 --2- 172.21.15.97:0/2076434656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb224104f90 0x7fb224198400 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:13.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.896+0000 7fb22986a640 1 --2- 172.21.15.97:0/2076434656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb224102660 0x7fb224197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:13.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.896+0000 7fb22986a640 1 -- 172.21.15.97:0/2076434656 >> 172.21.15.97:0/2076434656 conn(0x7fb2240fc280 msgr2=0x7fb224102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:13.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.896+0000 7fb22986a640 1 -- 172.21.15.97:0/2076434656 shutdown_connections 2024-09-11T23:19:13.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-11T23:19:13.896+0000 7fb22986a640 1 -- 172.21.15.97:0/2076434656 wait complete. 2024-09-11T23:19:14.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:13 smithi043 ceph-mon[30950]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:14.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:13 smithi090 ceph-mon[35130]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:14.656 DEBUG:teuthology.orchestra.run.smithi043:mds.a> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.a.service 2024-09-11T23:19:14.659 DEBUG:teuthology.orchestra.run.smithi043:mds.d> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.d.service 2024-09-11T23:19:14.702 DEBUG:teuthology.orchestra.run.smithi043:mds.g> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.g.service 2024-09-11T23:19:14.745 DEBUG:teuthology.orchestra.run.smithi043:mds.j> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.j.service 2024-09-11T23:19:14.788 DEBUG:teuthology.orchestra.run.smithi090:mds.b> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.b.service 2024-09-11T23:19:14.790 DEBUG:teuthology.orchestra.run.smithi090:mds.e> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.e.service 2024-09-11T23:19:14.835 DEBUG:teuthology.orchestra.run.smithi090:mds.h> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.h.service 2024-09-11T23:19:14.877 DEBUG:teuthology.orchestra.run.smithi090:mds.k> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.k.service 2024-09-11T23:19:14.921 DEBUG:teuthology.orchestra.run.smithi097:mds.c> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.c.service 2024-09-11T23:19:14.923 DEBUG:teuthology.orchestra.run.smithi097:mds.f> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.f.service 2024-09-11T23:19:14.966 DEBUG:teuthology.orchestra.run.smithi097:mds.i> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.i.service 2024-09-11T23:19:15.009 DEBUG:teuthology.orchestra.run.smithi097:mds.l> sudo journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.l.service 2024-09-11T23:19:15.052 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-11T23:19:15.052 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:19:15.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:14 smithi043 ceph-mon[30950]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi097=c;smithi097=f;smithi097=i;smithi097=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:19:15.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:14 smithi043 ceph-mon[30950]: Saving service mds.all spec with placement smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi097=c;smithi097=f;smithi097=i;smithi097=l;count:12 2024-09-11T23:19:15.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:14 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:15.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:14 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:19:15.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:14 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:15.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:14 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:19:15.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:14 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:15.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:14 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:15.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:14 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:15.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:14 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:15.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:14 smithi043 ceph-mon[30950]: Deploying daemon mds.l on smithi097 2024-09-11T23:19:15.180 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:14 smithi097 ceph-mon[35562]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi097=c;smithi097=f;smithi097=i;smithi097=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:19:15.180 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:14 smithi097 ceph-mon[35562]: Saving service mds.all spec with placement smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi097=c;smithi097=f;smithi097=i;smithi097=l;count:12 2024-09-11T23:19:15.180 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:14 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:15.180 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:14 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:19:15.180 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:14 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:15.180 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:14 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:19:15.180 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:14 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:15.180 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:14 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:15.181 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:14 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:15.181 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:14 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:15.181 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:14 smithi097 ceph-mon[35562]: Deploying daemon mds.l on smithi097 2024-09-11T23:19:15.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:14 smithi090 ceph-mon[35130]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi097=c;smithi097=f;smithi097=i;smithi097=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:19:15.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:14 smithi090 ceph-mon[35130]: Saving service mds.all spec with placement smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi097=c;smithi097=f;smithi097=i;smithi097=l;count:12 2024-09-11T23:19:15.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:14 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:15.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:14 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:19:15.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:14 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:15.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:14 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:19:15.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:14 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:15.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:14 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:15.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:14 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:15.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:14 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:15.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:14 smithi090 ceph-mon[35130]: Deploying daemon mds.l on smithi097 2024-09-11T23:19:15.311 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:15.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.790+0000 7f195e535640 1 -- 172.21.15.43:0/832945299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1958101de0 msgr2=0x7f19581021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:15.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.790+0000 7f195e535640 1 --2- 172.21.15.43:0/832945299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1958101de0 0x7f19581021c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1948009900 tx=0x7f194802a3f0 comp rx=0 tx=0).stop 2024-09-11T23:19:15.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.790+0000 7f195e535640 1 -- 172.21.15.43:0/832945299 shutdown_connections 2024-09-11T23:19:15.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.790+0000 7f195e535640 1 --2- 172.21.15.43:0/832945299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1958106f10 0x7f195810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:15.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.790+0000 7f195e535640 1 --2- 172.21.15.43:0/832945299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1958102790 0x7f19581067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:15.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.790+0000 7f195e535640 1 --2- 172.21.15.43:0/832945299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1958101de0 0x7f19581021c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:15.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.790+0000 7f195e535640 1 -- 172.21.15.43:0/832945299 >> 172.21.15.43:0/832945299 conn(0x7f19580fd8c0 msgr2=0x7f19580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:15.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.790+0000 7f195e535640 1 -- 172.21.15.43:0/832945299 shutdown_connections 2024-09-11T23:19:15.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.790+0000 7f195e535640 1 -- 172.21.15.43:0/832945299 wait complete. 2024-09-11T23:19:15.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.791+0000 7f195e535640 1 Processor -- start 2024-09-11T23:19:15.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.791+0000 7f195e535640 1 -- start start 2024-09-11T23:19:15.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.791+0000 7f195e535640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1958101de0 0x7f1958198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:15.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.791+0000 7f195e535640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1958102790 0x7f19581988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:15.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.791+0000 7f195e535640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1958106f10 0x7f195819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:15.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.791+0000 7f195e535640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1958111f70 con 0x7f1958101de0 2024-09-11T23:19:15.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.791+0000 7f195e535640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1958111df0 con 0x7f1958106f10 2024-09-11T23:19:15.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.791+0000 7f195e535640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f19581120f0 con 0x7f1958102790 2024-09-11T23:19:15.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.791+0000 7f1957fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1958101de0 0x7f1958198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:15.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.792+0000 7f1957fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1958101de0 0x7f1958198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58604/0 (socket says 172.21.15.43:58604) 2024-09-11T23:19:15.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.792+0000 7f1957fff640 1 -- 172.21.15.43:0/571505460 learned_addr learned my addr 172.21.15.43:0/571505460 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:19:15.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.792+0000 7f19577fe640 1 --2- 172.21.15.43:0/571505460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1958102790 0x7f19581988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:15.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.792+0000 7f195cad2640 1 --2- 172.21.15.43:0/571505460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1958106f10 0x7f195819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:15.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.792+0000 7f1957fff640 1 -- 172.21.15.43:0/571505460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1958102790 msgr2=0x7f19581988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:15.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.792+0000 7f1957fff640 1 --2- 172.21.15.43:0/571505460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1958102790 0x7f19581988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:15.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.792+0000 7f1957fff640 1 -- 172.21.15.43:0/571505460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1958106f10 msgr2=0x7f195819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:15.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.792+0000 7f1957fff640 1 --2- 172.21.15.43:0/571505460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1958106f10 0x7f195819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:15.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.792+0000 7f1957fff640 1 -- 172.21.15.43:0/571505460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195819d320 con 0x7f1958101de0 2024-09-11T23:19:15.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.792+0000 7f1957fff640 1 --2- 172.21.15.43:0/571505460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1958101de0 0x7f1958198370 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1948006010 tx=0x7f19480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:15.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.792+0000 7f19557fa640 1 -- 172.21.15.43:0/571505460 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194803f030 con 0x7f1958101de0 2024-09-11T23:19:15.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.792+0000 7f19557fa640 1 -- 172.21.15.43:0/571505460 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1948003ce0 con 0x7f1958101de0 2024-09-11T23:19:15.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.792+0000 7f195e535640 1 -- 172.21.15.43:0/571505460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f195819d5b0 con 0x7f1958101de0 2024-09-11T23:19:15.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.792+0000 7f19557fa640 1 -- 172.21.15.43:0/571505460 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948035070 con 0x7f1958101de0 2024-09-11T23:19:15.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.792+0000 7f195e535640 1 -- 172.21.15.43:0/571505460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19581a4ef0 con 0x7f1958101de0 2024-09-11T23:19:15.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.793+0000 7f195e535640 1 -- 172.21.15.43:0/571505460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19581032c0 con 0x7f1958101de0 2024-09-11T23:19:15.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.795+0000 7f19557fa640 1 -- 172.21.15.43:0/571505460 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1948042030 con 0x7f1958101de0 2024-09-11T23:19:15.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.795+0000 7f19557fa640 1 --2- 172.21.15.43:0/571505460 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f19400a2b80 0x7f19400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:15.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.795+0000 7f19577fe640 1 --2- 172.21.15.43:0/571505460 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f19400a2b80 0x7f19400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:15.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.795+0000 7f19557fa640 1 -- 172.21.15.43:0/571505460 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f19480ef180 con 0x7f1958101de0 2024-09-11T23:19:15.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.796+0000 7f19577fe640 1 --2- 172.21.15.43:0/571505460 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f19400a2b80 0x7f19400a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f194c005e10 tx=0x7f194c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:15.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:15.800+0000 7f19557fa640 1 -- 172.21.15.43:0/571505460 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19480bc620 con 0x7f1958101de0 2024-09-11T23:19:16.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:16.011+0000 7f195e535640 1 -- 172.21.15.43:0/571505460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f195810a0f0 con 0x7f1958101de0 2024-09-11T23:19:16.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:16.012+0000 7f19557fa640 1 -- 172.21.15.43:0/571505460 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f19480bbf20 con 0x7f1958101de0 2024-09-11T23:19:16.012 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:19:16.012 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"btime":"2024-09-11T23:12:03:504233+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-11T23:19:16.012 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 1 2024-09-11T23:19:16.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:16.013+0000 7f195e535640 1 -- 172.21.15.43:0/571505460 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f19400a2b80 msgr2=0x7f19400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:16.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:16.013+0000 7f195e535640 1 --2- 172.21.15.43:0/571505460 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f19400a2b80 0x7f19400a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f194c005e10 tx=0x7f194c005da0 comp rx=0 tx=0).stop 2024-09-11T23:19:16.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:16.014+0000 7f195e535640 1 -- 172.21.15.43:0/571505460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1958101de0 msgr2=0x7f1958198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:16.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:16.014+0000 7f195e535640 1 --2- 172.21.15.43:0/571505460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1958101de0 0x7f1958198370 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1948006010 tx=0x7f19480042f0 comp rx=0 tx=0).stop 2024-09-11T23:19:16.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:16.014+0000 7f195e535640 1 -- 172.21.15.43:0/571505460 shutdown_connections 2024-09-11T23:19:16.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:16.014+0000 7f195e535640 1 --2- 172.21.15.43:0/571505460 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f19400a2b80 0x7f19400a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:16.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:16.014+0000 7f195e535640 1 --2- 172.21.15.43:0/571505460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1958106f10 0x7f195819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:16.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:16.014+0000 7f195e535640 1 --2- 172.21.15.43:0/571505460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1958102790 0x7f19581988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:16.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:16.014+0000 7f195e535640 1 --2- 172.21.15.43:0/571505460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1958101de0 0x7f1958198370 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:16.014+0000 7f195e535640 1 -- 172.21.15.43:0/571505460 >> 172.21.15.43:0/571505460 conn(0x7f19580fd8c0 msgr2=0x7f19580fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:16.014+0000 7f195e535640 1 -- 172.21.15.43:0/571505460 shutdown_connections 2024-09-11T23:19:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:16.014+0000 7f195e535640 1 -- 172.21.15.43:0/571505460 wait complete. 2024-09-11T23:19:16.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:16 smithi097 ceph-mon[35562]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:16.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:16 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/571505460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:16.552 INFO:journalctl@ceph.mds.l.smithi097.stdout:Sep 11 23:19:16 smithi097 systemd[1]: Starting Ceph mds.l for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:19:16.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:16 smithi043 ceph-mon[30950]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:16.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:16 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/571505460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:16.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:16 smithi090 ceph-mon[35130]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:16.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:16 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/571505460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:16.838 INFO:journalctl@ceph.mds.l.smithi097.stdout:Sep 11 23:19:16 smithi097 podman[64840]: 2024-09-11 23:19:16.466161165 +0000 UTC m=+0.015742131 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:19:16.838 INFO:journalctl@ceph.mds.l.smithi097.stdout:Sep 11 23:19:16 smithi097 podman[64840]: 2024-09-11 23:19:16.594276793 +0000 UTC m=+0.143857750 container create a21dd9902d79295cb1c8c769a1528139502b09f937ca0732781e7d37fd02aec2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-11T23:19:16.838 INFO:journalctl@ceph.mds.l.smithi097.stdout:Sep 11 23:19:16 smithi097 podman[64840]: 2024-09-11 23:19:16.838627676 +0000 UTC m=+0.388208642 container init a21dd9902d79295cb1c8c769a1528139502b09f937ca0732781e7d37fd02aec2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-l, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T23:19:17.301 INFO:journalctl@ceph.mds.l.smithi097.stdout:Sep 11 23:19:16 smithi097 podman[64840]: 2024-09-11 23:19:16.841641503 +0000 UTC m=+0.391222461 container start a21dd9902d79295cb1c8c769a1528139502b09f937ca0732781e7d37fd02aec2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-11T23:19:17.301 INFO:journalctl@ceph.mds.l.smithi097.stdout:Sep 11 23:19:16 smithi097 ceph-mds[64859]: -- 172.21.15.97:0/3946301802 <== mon.1 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a0edb77860 con 0x55a0edb55400 2024-09-11T23:19:17.301 INFO:journalctl@ceph.mds.l.smithi097.stdout:Sep 11 23:19:16 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-l[64855]: starting mds.l at 2024-09-11T23:19:17.301 INFO:journalctl@ceph.mds.l.smithi097.stdout:Sep 11 23:19:16 smithi097 bash[64840]: a21dd9902d79295cb1c8c769a1528139502b09f937ca0732781e7d37fd02aec2 2024-09-11T23:19:17.302 INFO:journalctl@ceph.mds.l.smithi097.stdout:Sep 11 23:19:16 smithi097 systemd[1]: Started Ceph mds.l for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:19:18.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:17 smithi090 ceph-mon[35130]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:18.038 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:17 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.97:6832/2769852264,v1:172.21.15.97:6833/2769852264] up:boot 2024-09-11T23:19:18.038 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:17 smithi090 ceph-mon[35130]: fsmap 1 up:standby 2024-09-11T23:19:18.039 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:17 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:19:18.039 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:17 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:18.039 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:17 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:18.039 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:17 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:18.039 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:17 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:18.039 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:17 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:18.039 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:17 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:18.039 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:17 smithi090 ceph-mon[35130]: Deploying daemon mds.k on smithi090 2024-09-11T23:19:18.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:17 smithi097 ceph-mon[35562]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:18.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:17 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.97:6832/2769852264,v1:172.21.15.97:6833/2769852264] up:boot 2024-09-11T23:19:18.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:17 smithi097 ceph-mon[35562]: fsmap 1 up:standby 2024-09-11T23:19:18.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:17 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:19:18.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:17 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:18.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:17 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:18.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:17 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:18.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:17 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:18.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:17 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:18.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:17 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:18.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:17 smithi097 ceph-mon[35562]: Deploying daemon mds.k on smithi090 2024-09-11T23:19:18.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:17 smithi043 ceph-mon[30950]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:18.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:17 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.97:6832/2769852264,v1:172.21.15.97:6833/2769852264] up:boot 2024-09-11T23:19:18.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:17 smithi043 ceph-mon[30950]: fsmap 1 up:standby 2024-09-11T23:19:18.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:17 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:19:18.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:17 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:18.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:17 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:18.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:17 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:18.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:17 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:18.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:17 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:18.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:17 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:18.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:17 smithi043 ceph-mon[30950]: Deploying daemon mds.k on smithi090 2024-09-11T23:19:19.083 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:19:19.270 INFO:journalctl@ceph.mds.k.smithi090.stdout:Sep 11 23:19:19 smithi090 systemd[1]: Starting Ceph mds.k for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:19:19.350 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:19.662 INFO:journalctl@ceph.mds.k.smithi090.stdout:Sep 11 23:19:19 smithi090 podman[64236]: 2024-09-11 23:19:19.290454931 +0000 UTC m=+0.015957756 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:19:19.662 INFO:journalctl@ceph.mds.k.smithi090.stdout:Sep 11 23:19:19 smithi090 podman[64236]: 2024-09-11 23:19:19.408529746 +0000 UTC m=+0.134032608 container create c753dd3ba9cff24311b537dd46ba63726ac18d892539ec701922af3337b6ebd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-11T23:19:19.662 INFO:journalctl@ceph.mds.k.smithi090.stdout:Sep 11 23:19:19 smithi090 podman[64236]: 2024-09-11 23:19:19.662825545 +0000 UTC m=+0.388328362 container init c753dd3ba9cff24311b537dd46ba63726ac18d892539ec701922af3337b6ebd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-k, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-11T23:19:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.852+0000 7fb8c3d47640 1 -- 172.21.15.43:0/3657676749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8bc105ad0 msgr2=0x7fb8bc105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.852+0000 7fb8c3d47640 1 --2- 172.21.15.43:0/3657676749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8bc105ad0 0x7fb8bc105eb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009900 tx=0x7fb8ac02a410 comp rx=0 tx=0).stop 2024-09-11T23:19:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.853+0000 7fb8c3d47640 1 -- 172.21.15.43:0/3657676749 shutdown_connections 2024-09-11T23:19:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.853+0000 7fb8c3d47640 1 --2- 172.21.15.43:0/3657676749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8bc101ba0 0x7fb8bc10d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.853+0000 7fb8c3d47640 1 --2- 172.21.15.43:0/3657676749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8bc101200 0x7fb8bc101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.853+0000 7fb8c3d47640 1 --2- 172.21.15.43:0/3657676749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8bc105ad0 0x7fb8bc105eb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.853+0000 7fb8c3d47640 1 -- 172.21.15.43:0/3657676749 >> 172.21.15.43:0/3657676749 conn(0x7fb8bc077210 msgr2=0x7fb8bc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.853+0000 7fb8c3d47640 1 -- 172.21.15.43:0/3657676749 shutdown_connections 2024-09-11T23:19:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.853+0000 7fb8c3d47640 1 -- 172.21.15.43:0/3657676749 wait complete. 2024-09-11T23:19:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.853+0000 7fb8c3d47640 1 Processor -- start 2024-09-11T23:19:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.853+0000 7fb8c3d47640 1 -- start start 2024-09-11T23:19:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.854+0000 7fb8c3d47640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8bc101200 0x7fb8bc0fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.854+0000 7fb8c3d47640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8bc101ba0 0x7fb8bc0fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.854+0000 7fb8c3d47640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8bc105ad0 0x7fb8bc0fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.854+0000 7fb8c3d47640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb8bc110eb0 con 0x7fb8bc101ba0 2024-09-11T23:19:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.854+0000 7fb8c3d47640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb8bc110d30 con 0x7fb8bc101200 2024-09-11T23:19:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.854+0000 7fb8c3d47640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb8bc111030 con 0x7fb8bc105ad0 2024-09-11T23:19:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.854+0000 7fb8c12e2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8bc101ba0 0x7fb8bc0fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.854+0000 7fb8c12e2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8bc101ba0 0x7fb8bc0fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58618/0 (socket says 172.21.15.43:58618) 2024-09-11T23:19:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.854+0000 7fb8c12e2640 1 -- 172.21.15.43:0/2257036845 learned_addr learned my addr 172.21.15.43:0/2257036845 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:19:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.854+0000 7fb8c22e4640 1 --2- 172.21.15.43:0/2257036845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8bc105ad0 0x7fb8bc0fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.854+0000 7fb8c1ae3640 1 --2- 172.21.15.43:0/2257036845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8bc101200 0x7fb8bc0fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.854+0000 7fb8c12e2640 1 -- 172.21.15.43:0/2257036845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8bc105ad0 msgr2=0x7fb8bc0fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.854+0000 7fb8c12e2640 1 --2- 172.21.15.43:0/2257036845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8bc105ad0 0x7fb8bc0fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.854+0000 7fb8c12e2640 1 -- 172.21.15.43:0/2257036845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8bc101200 msgr2=0x7fb8bc0fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.854+0000 7fb8c12e2640 1 --2- 172.21.15.43:0/2257036845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8bc101200 0x7fb8bc0fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.854+0000 7fb8c12e2640 1 -- 172.21.15.43:0/2257036845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8bc1a6910 con 0x7fb8bc101ba0 2024-09-11T23:19:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.854+0000 7fb8c12e2640 1 --2- 172.21.15.43:0/2257036845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8bc101ba0 0x7fb8bc0fcbd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800d950 tx=0x7fb8b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:19.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.854+0000 7fb8b2ffd640 1 -- 172.21.15.43:0/2257036845 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8017070 con 0x7fb8bc101ba0 2024-09-11T23:19:19.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.854+0000 7fb8c3d47640 1 -- 172.21.15.43:0/2257036845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8bc1a6c00 con 0x7fb8bc101ba0 2024-09-11T23:19:19.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.854+0000 7fb8b2ffd640 1 -- 172.21.15.43:0/2257036845 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8b8009c30 con 0x7fb8bc101ba0 2024-09-11T23:19:19.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.854+0000 7fb8b2ffd640 1 -- 172.21.15.43:0/2257036845 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b800f6b0 con 0x7fb8bc101ba0 2024-09-11T23:19:19.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.855+0000 7fb8c3d47640 1 -- 172.21.15.43:0/2257036845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8bc1a7190 con 0x7fb8bc101ba0 2024-09-11T23:19:19.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.856+0000 7fb8c3d47640 1 -- 172.21.15.43:0/2257036845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8bc101660 con 0x7fb8bc101ba0 2024-09-11T23:19:19.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.857+0000 7fb8b2ffd640 1 -- 172.21.15.43:0/2257036845 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb8b80109d0 con 0x7fb8bc101ba0 2024-09-11T23:19:19.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.858+0000 7fb8b2ffd640 1 --2- 172.21.15.43:0/2257036845 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb8a80a2c00 0x7fb8a80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:19.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.858+0000 7fb8c1ae3640 1 --2- 172.21.15.43:0/2257036845 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb8a80a2c00 0x7fb8a80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:19.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.858+0000 7fb8b2ffd640 1 -- 172.21.15.43:0/2257036845 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fb8b80ceaf0 con 0x7fb8bc101ba0 2024-09-11T23:19:19.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.858+0000 7fb8c1ae3640 1 --2- 172.21.15.43:0/2257036845 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb8a80a2c00 0x7fb8a80a50c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac005ab0 tx=0x7fb8ac005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:19.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:19.862+0000 7fb8b2ffd640 1 -- 172.21.15.43:0/2257036845 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb8b8009de0 con 0x7fb8bc101ba0 2024-09-11T23:19:20.020 INFO:journalctl@ceph.mds.k.smithi090.stdout:Sep 11 23:19:19 smithi090 podman[64236]: 2024-09-11 23:19:19.666137407 +0000 UTC m=+0.391640224 container start c753dd3ba9cff24311b537dd46ba63726ac18d892539ec701922af3337b6ebd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-k, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-11T23:19:20.020 INFO:journalctl@ceph.mds.k.smithi090.stdout:Sep 11 23:19:19 smithi090 ceph-mds[64255]: -- 172.21.15.90:0/773855986 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56376326f860 con 0x56376324c800 2024-09-11T23:19:20.020 INFO:journalctl@ceph.mds.k.smithi090.stdout:Sep 11 23:19:19 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-k[64251]: starting mds.k at 2024-09-11T23:19:20.020 INFO:journalctl@ceph.mds.k.smithi090.stdout:Sep 11 23:19:19 smithi090 bash[64236]: c753dd3ba9cff24311b537dd46ba63726ac18d892539ec701922af3337b6ebd5 2024-09-11T23:19:20.020 INFO:journalctl@ceph.mds.k.smithi090.stdout:Sep 11 23:19:19 smithi090 systemd[1]: Started Ceph mds.k for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:19:20.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:19 smithi090 ceph-mon[35130]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:20.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:19 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:20.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:19 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:20.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:19 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:20.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:19 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:20.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:19 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:20.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:19 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:20.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:20.086+0000 7fb8c3d47640 1 -- 172.21.15.43:0/2257036845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8bc0fdbc0 con 0x7fb8bc101ba0 2024-09-11T23:19:20.086 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:19 smithi043 ceph-mon[30950]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:20.086 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:19 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:20.086 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:19 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:20.087 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:19 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:20.087 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:19 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:20.087 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:19 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:20.087 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:19 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:20.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:20.086+0000 7fb8b2ffd640 1 -- 172.21.15.43:0/2257036845 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2260 (secure 0 0 0) 0x7fb8bc0fdbc0 con 0x7fb8bc101ba0 2024-09-11T23:19:20.088 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:19:20.088 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":3,"btime":"2024-09-11T23:19:19:873349+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-11T23:19:20.088 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 3 2024-09-11T23:19:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:20.089+0000 7fb8c3d47640 1 -- 172.21.15.43:0/2257036845 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb8a80a2c00 msgr2=0x7fb8a80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:20.089+0000 7fb8c3d47640 1 --2- 172.21.15.43:0/2257036845 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb8a80a2c00 0x7fb8a80a50c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac005ab0 tx=0x7fb8ac005a20 comp rx=0 tx=0).stop 2024-09-11T23:19:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:20.089+0000 7fb8c3d47640 1 -- 172.21.15.43:0/2257036845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8bc101ba0 msgr2=0x7fb8bc0fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:20.089+0000 7fb8c3d47640 1 --2- 172.21.15.43:0/2257036845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8bc101ba0 0x7fb8bc0fcbd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800d950 tx=0x7fb8b800de10 comp rx=0 tx=0).stop 2024-09-11T23:19:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:20.089+0000 7fb8c3d47640 1 -- 172.21.15.43:0/2257036845 shutdown_connections 2024-09-11T23:19:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:20.089+0000 7fb8c3d47640 1 --2- 172.21.15.43:0/2257036845 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb8a80a2c00 0x7fb8a80a50c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:20.089+0000 7fb8c3d47640 1 --2- 172.21.15.43:0/2257036845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8bc105ad0 0x7fb8bc0fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:20.089+0000 7fb8c3d47640 1 --2- 172.21.15.43:0/2257036845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8bc101ba0 0x7fb8bc0fcbd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:20.089+0000 7fb8c3d47640 1 --2- 172.21.15.43:0/2257036845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8bc101200 0x7fb8bc0fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:20.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:20.089+0000 7fb8c3d47640 1 -- 172.21.15.43:0/2257036845 >> 172.21.15.43:0/2257036845 conn(0x7fb8bc077210 msgr2=0x7fb8bc100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:20.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:20.090+0000 7fb8c3d47640 1 -- 172.21.15.43:0/2257036845 shutdown_connections 2024-09-11T23:19:20.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:20.090+0000 7fb8c3d47640 1 -- 172.21.15.43:0/2257036845 wait complete. 2024-09-11T23:19:20.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:19 smithi097 ceph-mon[35562]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:20.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:19 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:20.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:19 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:20.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:19 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:20.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:19 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:20.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:19 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:20.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:19 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:21.223 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:20 smithi043 ceph-mon[30950]: Deploying daemon mds.j on smithi043 2024-09-11T23:19:21.224 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:20 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.90:6832/2266906537,v1:172.21.15.90:6833/2266906537] up:boot 2024-09-11T23:19:21.224 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:20 smithi043 ceph-mon[30950]: fsmap 2 up:standby 2024-09-11T23:19:21.224 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:20 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:19:21.224 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:20 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2257036845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:21.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:20 smithi090 ceph-mon[35130]: Deploying daemon mds.j on smithi043 2024-09-11T23:19:21.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:20 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.90:6832/2266906537,v1:172.21.15.90:6833/2266906537] up:boot 2024-09-11T23:19:21.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:20 smithi090 ceph-mon[35130]: fsmap 2 up:standby 2024-09-11T23:19:21.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:20 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:19:21.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:20 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2257036845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:21.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:20 smithi097 ceph-mon[35562]: Deploying daemon mds.j on smithi043 2024-09-11T23:19:21.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:20 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.90:6832/2266906537,v1:172.21.15.90:6833/2266906537] up:boot 2024-09-11T23:19:21.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:20 smithi097 ceph-mon[35562]: fsmap 2 up:standby 2024-09-11T23:19:21.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:20 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:19:21.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:20 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2257036845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:22.083 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:21 smithi090 ceph-mon[35130]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:22.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:21 smithi097 ceph-mon[35562]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:22.350 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:21 smithi043 ceph-mon[30950]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:23.054 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:19:23.055 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 11 23:19:22 smithi043 systemd[1]: Starting Ceph mds.j for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:19:23.331 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 11 23:19:23 smithi043 podman[70121]: 2024-09-11 23:19:22.956837443 +0000 UTC m=+0.016324102 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:19:23.331 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 11 23:19:23 smithi043 podman[70121]: 2024-09-11 23:19:23.202939599 +0000 UTC m=+0.262426247 container create fa72d4c80cdc7e4f02fa21eb47aecbaf10bae1607e1ef1f562647bbe87af75ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-j, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-11T23:19:23.591 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:23.717 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 11 23:19:23 smithi043 podman[70121]: 2024-09-11 23:19:23.447551397 +0000 UTC m=+0.507038058 container init fa72d4c80cdc7e4f02fa21eb47aecbaf10bae1607e1ef1f562647bbe87af75ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-11T23:19:23.717 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 11 23:19:23 smithi043 podman[70121]: 2024-09-11 23:19:23.450621108 +0000 UTC m=+0.510107754 container start fa72d4c80cdc7e4f02fa21eb47aecbaf10bae1607e1ef1f562647bbe87af75ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:19:23.717 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 11 23:19:23 smithi043 ceph-mds[70183]: -- 172.21.15.43:0/3333007974 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556fd13d9c20 con 0x556fd13b7400 2024-09-11T23:19:23.717 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 11 23:19:23 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-j[70178]: starting mds.j at 2024-09-11T23:19:23.717 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 11 23:19:23 smithi043 bash[70121]: fa72d4c80cdc7e4f02fa21eb47aecbaf10bae1607e1ef1f562647bbe87af75ee 2024-09-11T23:19:23.718 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 11 23:19:23 smithi043 systemd[1]: Started Ceph mds.j for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:19:24.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:23 smithi043 ceph-mon[30950]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:24.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:24.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:24.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:24.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:24.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:24.001 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:23 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:24.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.084+0000 7f75f1384640 1 -- 172.21.15.43:0/527256139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75ec103780 msgr2=0x7f75ec107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.084+0000 7f75f1384640 1 --2- 172.21.15.43:0/527256139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75ec103780 0x7f75ec107bc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f75d4009930 tx=0x7f75d402a420 comp rx=0 tx=0).stop 2024-09-11T23:19:24.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.084+0000 7f75f1384640 1 -- 172.21.15.43:0/527256139 shutdown_connections 2024-09-11T23:19:24.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.084+0000 7f75f1384640 1 --2- 172.21.15.43:0/527256139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec108300 0x7f75ec10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.084+0000 7f75f1384640 1 --2- 172.21.15.43:0/527256139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75ec103780 0x7f75ec107bc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.084+0000 7f75f1384640 1 --2- 172.21.15.43:0/527256139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75ec102dd0 0x7f75ec1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.084+0000 7f75f1384640 1 -- 172.21.15.43:0/527256139 >> 172.21.15.43:0/527256139 conn(0x7f75ec0fc7d0 msgr2=0x7f75ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:24.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.084+0000 7f75f1384640 1 -- 172.21.15.43:0/527256139 shutdown_connections 2024-09-11T23:19:24.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.084+0000 7f75f1384640 1 -- 172.21.15.43:0/527256139 wait complete. 2024-09-11T23:19:24.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.084+0000 7f75f1384640 1 Processor -- start 2024-09-11T23:19:24.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.085+0000 7f75f1384640 1 -- start start 2024-09-11T23:19:24.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.085+0000 7f75f1384640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75ec102dd0 0x7f75ec198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.085+0000 7f75f1384640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec103780 0x7f75ec198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.085+0000 7f75f1384640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75ec108300 0x7f75ec19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.085+0000 7f75f1384640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec112050 con 0x7f75ec108300 2024-09-11T23:19:24.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.085+0000 7f75f1384640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec111ed0 con 0x7f75ec103780 2024-09-11T23:19:24.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.085+0000 7f75f1384640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec1121d0 con 0x7f75ec102dd0 2024-09-11T23:19:24.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.085+0000 7f75eb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75ec108300 0x7f75ec19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.085+0000 7f75eb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75ec108300 0x7f75ec19ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55022/0 (socket says 172.21.15.43:55022) 2024-09-11T23:19:24.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.085+0000 7f75eb7fe640 1 -- 172.21.15.43:0/2121620058 learned_addr learned my addr 172.21.15.43:0/2121620058 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:19:24.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.085+0000 7f75eaffd640 1 --2- 172.21.15.43:0/2121620058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75ec102dd0 0x7f75ec198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.085+0000 7f75ea7fc640 1 --2- 172.21.15.43:0/2121620058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec103780 0x7f75ec198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.085+0000 7f75eb7fe640 1 -- 172.21.15.43:0/2121620058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75ec102dd0 msgr2=0x7f75ec198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.085+0000 7f75eb7fe640 1 --2- 172.21.15.43:0/2121620058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75ec102dd0 0x7f75ec198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.085+0000 7f75eb7fe640 1 -- 172.21.15.43:0/2121620058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec103780 msgr2=0x7f75ec198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.085+0000 7f75eb7fe640 1 --2- 172.21.15.43:0/2121620058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec103780 0x7f75ec198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.085+0000 7f75eb7fe640 1 -- 172.21.15.43:0/2121620058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ec19d500 con 0x7f75ec108300 2024-09-11T23:19:24.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.086+0000 7f75eb7fe640 1 --2- 172.21.15.43:0/2121620058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75ec108300 0x7f75ec19ce20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f75e400cce0 tx=0x7f75e400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:24.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.086+0000 7f75e3fff640 1 -- 172.21.15.43:0/2121620058 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e400f450 con 0x7f75ec108300 2024-09-11T23:19:24.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.086+0000 7f75f1384640 1 -- 172.21.15.43:0/2121620058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75ec19d7f0 con 0x7f75ec108300 2024-09-11T23:19:24.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.086+0000 7f75e3fff640 1 -- 172.21.15.43:0/2121620058 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75e400fad0 con 0x7f75ec108300 2024-09-11T23:19:24.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.086+0000 7f75e3fff640 1 -- 172.21.15.43:0/2121620058 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e4018b60 con 0x7f75ec108300 2024-09-11T23:19:24.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.086+0000 7f75f1384640 1 -- 172.21.15.43:0/2121620058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75ec1a50d0 con 0x7f75ec108300 2024-09-11T23:19:24.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.087+0000 7f75f1384640 1 -- 172.21.15.43:0/2121620058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75ec1031b0 con 0x7f75ec108300 2024-09-11T23:19:24.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.088+0000 7f75e3fff640 1 -- 172.21.15.43:0/2121620058 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f75e400fc80 con 0x7f75ec108300 2024-09-11T23:19:24.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.089+0000 7f75e3fff640 1 --2- 172.21.15.43:0/2121620058 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f75d80a2b80 0x7f75d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:24.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.089+0000 7f75eaffd640 1 --2- 172.21.15.43:0/2121620058 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f75d80a2b80 0x7f75d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:24.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.089+0000 7f75e3fff640 1 -- 172.21.15.43:0/2121620058 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f75e40cf440 con 0x7f75ec108300 2024-09-11T23:19:24.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.089+0000 7f75eaffd640 1 --2- 172.21.15.43:0/2121620058 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f75d80a2b80 0x7f75d80a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00abe0 tx=0x7f75dc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:24.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.092+0000 7f75e3fff640 1 -- 172.21.15.43:0/2121620058 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f75e4014030 con 0x7f75ec108300 2024-09-11T23:19:24.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:23 smithi090 ceph-mon[35130]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:24.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:23 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:24.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:23 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:24.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:23 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:24.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:23 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:24.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:23 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:24.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:23 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:24.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:23 smithi097 ceph-mon[35562]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:24.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:23 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:24.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:23 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:24.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:23 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:24.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:23 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:24.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:23 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:24.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:23 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:24.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.306+0000 7f75f1384640 1 -- 172.21.15.43:0/2121620058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75ec10a290 con 0x7f75ec108300 2024-09-11T23:19:24.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.306+0000 7f75e3fff640 1 -- 172.21.15.43:0/2121620058 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3082 (secure 0 0 0) 0x7f75e409c840 con 0x7f75ec108300 2024-09-11T23:19:24.306 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:19:24.306 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":4,"btime":"2024-09-11T23:19:23:889396+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-11T23:19:24.307 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 4 2024-09-11T23:19:24.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.308+0000 7f75f1384640 1 -- 172.21.15.43:0/2121620058 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f75d80a2b80 msgr2=0x7f75d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.308+0000 7f75f1384640 1 --2- 172.21.15.43:0/2121620058 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f75d80a2b80 0x7f75d80a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00abe0 tx=0x7f75dc00a2d0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.308+0000 7f75f1384640 1 -- 172.21.15.43:0/2121620058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75ec108300 msgr2=0x7f75ec19ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:24.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.308+0000 7f75f1384640 1 --2- 172.21.15.43:0/2121620058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75ec108300 0x7f75ec19ce20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f75e400cce0 tx=0x7f75e400a620 comp rx=0 tx=0).stop 2024-09-11T23:19:24.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.308+0000 7f75f1384640 1 -- 172.21.15.43:0/2121620058 shutdown_connections 2024-09-11T23:19:24.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.308+0000 7f75f1384640 1 --2- 172.21.15.43:0/2121620058 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f75d80a2b80 0x7f75d80a5040 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.308+0000 7f75f1384640 1 --2- 172.21.15.43:0/2121620058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75ec108300 0x7f75ec19ce20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.308+0000 7f75f1384640 1 --2- 172.21.15.43:0/2121620058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec103780 0x7f75ec198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.308+0000 7f75f1384640 1 --2- 172.21.15.43:0/2121620058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75ec102dd0 0x7f75ec198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:24.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.308+0000 7f75f1384640 1 -- 172.21.15.43:0/2121620058 >> 172.21.15.43:0/2121620058 conn(0x7f75ec0fc7d0 msgr2=0x7f75ec1063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:24.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.308+0000 7f75f1384640 1 -- 172.21.15.43:0/2121620058 shutdown_connections 2024-09-11T23:19:24.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:24.308+0000 7f75f1384640 1 -- 172.21.15.43:0/2121620058 wait complete. 2024-09-11T23:19:25.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:24 smithi043 ceph-mon[30950]: Deploying daemon mds.d on smithi043 2024-09-11T23:19:25.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:24 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.43:6834/108369845,v1:172.21.15.43:6835/108369845] up:boot 2024-09-11T23:19:25.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:24 smithi043 ceph-mon[30950]: fsmap 3 up:standby 2024-09-11T23:19:25.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:24 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:19:25.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:24 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2121620058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:25.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:24 smithi090 ceph-mon[35130]: Deploying daemon mds.d on smithi043 2024-09-11T23:19:25.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:24 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.43:6834/108369845,v1:172.21.15.43:6835/108369845] up:boot 2024-09-11T23:19:25.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:24 smithi090 ceph-mon[35130]: fsmap 3 up:standby 2024-09-11T23:19:25.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:24 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:19:25.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:24 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2121620058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:25.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:24 smithi097 ceph-mon[35562]: Deploying daemon mds.d on smithi043 2024-09-11T23:19:25.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:24 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.43:6834/108369845,v1:172.21.15.43:6835/108369845] up:boot 2024-09-11T23:19:25.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:24 smithi097 ceph-mon[35562]: fsmap 3 up:standby 2024-09-11T23:19:25.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:24 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:19:25.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:24 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2121620058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:26.217 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:25 smithi043 ceph-mon[30950]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:26.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:25 smithi090 ceph-mon[35130]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:26.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:25 smithi097 ceph-mon[35562]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:26.894 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 11 23:19:26 smithi043 podman[70536]: 2024-09-11 23:19:26.75445822 +0000 UTC m=+0.018433843 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:19:26.895 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 11 23:19:26 smithi043 podman[70536]: 2024-09-11 23:19:26.894178408 +0000 UTC m=+0.158154010 container create 03d0b56e736e1dce4e96ca0011f57ecd8a34b64f143f0687a2d102b9b2d195a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T23:19:27.165 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 11 23:19:27 smithi043 podman[70536]: 2024-09-11 23:19:27.138629637 +0000 UTC m=+0.402605245 container init 03d0b56e736e1dce4e96ca0011f57ecd8a34b64f143f0687a2d102b9b2d195a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-09-11T23:19:27.166 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 11 23:19:27 smithi043 podman[70536]: 2024-09-11 23:19:27.14183953 +0000 UTC m=+0.405815133 container start 03d0b56e736e1dce4e96ca0011f57ecd8a34b64f143f0687a2d102b9b2d195a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3) 2024-09-11T23:19:27.610 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 11 23:19:27 smithi043 ceph-mds[70563]: -- 172.21.15.43:0/1995146152 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a593d17860 con 0x55a593cf4800 2024-09-11T23:19:27.610 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 11 23:19:27 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-d[70559]: starting mds.d at 2024-09-11T23:19:27.610 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 11 23:19:27 smithi043 bash[70536]: 03d0b56e736e1dce4e96ca0011f57ecd8a34b64f143f0687a2d102b9b2d195a5 2024-09-11T23:19:27.610 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 11 23:19:27 smithi043 systemd[1]: Started Ceph mds.d for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:19:27.629 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:19:27.889 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:28.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.425+0000 7f2abdfe1640 1 -- 172.21.15.43:0/244137518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ab8102d40 msgr2=0x7f2ab8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:28.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.425+0000 7f2abdfe1640 1 --2- 172.21.15.43:0/244137518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ab8102d40 0x7f2ab8103120 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009900 tx=0x7f2aa802a480 comp rx=0 tx=0).stop 2024-09-11T23:19:28.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.426+0000 7f2abdfe1640 1 -- 172.21.15.43:0/244137518 shutdown_connections 2024-09-11T23:19:28.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.426+0000 7f2abdfe1640 1 --2- 172.21.15.43:0/244137518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ab810c170 0x7f2ab810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:28.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.426+0000 7f2abdfe1640 1 --2- 172.21.15.43:0/244137518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab8103660 0x7f2ab810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.426+0000 7f2abdfe1640 1 --2- 172.21.15.43:0/244137518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ab8102d40 0x7f2ab8103120 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.426+0000 7f2abdfe1640 1 -- 172.21.15.43:0/244137518 >> 172.21.15.43:0/244137518 conn(0x7f2ab80fc7a0 msgr2=0x7f2ab80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.426+0000 7f2abdfe1640 1 -- 172.21.15.43:0/244137518 shutdown_connections 2024-09-11T23:19:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.426+0000 7f2abdfe1640 1 -- 172.21.15.43:0/244137518 wait complete. 2024-09-11T23:19:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.426+0000 7f2abdfe1640 1 Processor -- start 2024-09-11T23:19:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.426+0000 7f2abdfe1640 1 -- start start 2024-09-11T23:19:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.427+0000 7f2abdfe1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ab8102d40 0x7f2ab8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.427+0000 7f2abdfe1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab8103660 0x7f2ab8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.427+0000 7f2abdfe1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ab810c170 0x7f2ab819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.427+0000 7f2abdfe1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab8111f70 con 0x7f2ab810c170 2024-09-11T23:19:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.427+0000 7f2abdfe1640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab8111df0 con 0x7f2ab8103660 2024-09-11T23:19:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.427+0000 7f2abdfe1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab81120f0 con 0x7f2ab8102d40 2024-09-11T23:19:28.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.427+0000 7f2ab7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ab810c170 0x7f2ab819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:28.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.427+0000 7f2ab7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ab810c170 0x7f2ab819cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55054/0 (socket says 172.21.15.43:55054) 2024-09-11T23:19:28.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.427+0000 7f2ab7fff640 1 -- 172.21.15.43:0/317479311 learned_addr learned my addr 172.21.15.43:0/317479311 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:19:28.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.427+0000 7f2ab77fe640 1 --2- 172.21.15.43:0/317479311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ab8102d40 0x7f2ab8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:28.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.427+0000 7f2ab6ffd640 1 --2- 172.21.15.43:0/317479311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab8103660 0x7f2ab8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:28.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.427+0000 7f2ab7fff640 1 -- 172.21.15.43:0/317479311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ab8102d40 msgr2=0x7f2ab8198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:28.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.427+0000 7f2ab7fff640 1 --2- 172.21.15.43:0/317479311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ab8102d40 0x7f2ab8198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:28.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.427+0000 7f2ab7fff640 1 -- 172.21.15.43:0/317479311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab8103660 msgr2=0x7f2ab8198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:28.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.427+0000 7f2ab7fff640 1 --2- 172.21.15.43:0/317479311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab8103660 0x7f2ab8198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:28.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.427+0000 7f2ab7fff640 1 -- 172.21.15.43:0/317479311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab819d400 con 0x7f2ab810c170 2024-09-11T23:19:28.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.427+0000 7f2ab7fff640 1 --2- 172.21.15.43:0/317479311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ab810c170 0x7f2ab819cd20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000eb10 tx=0x7f2ab000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.427+0000 7f2ab4ff9640 1 -- 172.21.15.43:0/317479311 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0004b20 con 0x7f2ab810c170 2024-09-11T23:19:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.428+0000 7f2ab4ff9640 1 -- 172.21.15.43:0/317479311 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ab0004cc0 con 0x7f2ab810c170 2024-09-11T23:19:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.428+0000 7f2ab4ff9640 1 -- 172.21.15.43:0/317479311 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0010760 con 0x7f2ab810c170 2024-09-11T23:19:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.428+0000 7f2abdfe1640 1 -- 172.21.15.43:0/317479311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab819d6f0 con 0x7f2ab810c170 2024-09-11T23:19:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.428+0000 7f2abdfe1640 1 -- 172.21.15.43:0/317479311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab81a4fd0 con 0x7f2ab810c170 2024-09-11T23:19:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.429+0000 7f2abdfe1640 1 -- 172.21.15.43:0/317479311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab8103120 con 0x7f2ab810c170 2024-09-11T23:19:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.430+0000 7f2ab4ff9640 1 -- 172.21.15.43:0/317479311 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2ab0005000 con 0x7f2ab810c170 2024-09-11T23:19:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.430+0000 7f2ab4ff9640 1 --2- 172.21.15.43:0/317479311 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f2aa00a2b80 0x7f2aa00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.431+0000 7f2ab77fe640 1 --2- 172.21.15.43:0/317479311 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f2aa00a2b80 0x7f2aa00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.431+0000 7f2ab4ff9640 1 -- 172.21.15.43:0/317479311 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f2ab00cf5e0 con 0x7f2ab810c170 2024-09-11T23:19:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.431+0000 7f2ab77fe640 1 --2- 172.21.15.43:0/317479311 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f2aa00a2b80 0x7f2aa00a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8006010 tx=0x7f2aa80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.434+0000 7f2ab4ff9640 1 -- 172.21.15.43:0/317479311 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ab009c9e0 con 0x7f2ab810c170 2024-09-11T23:19:28.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:28 smithi097 ceph-mon[35562]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:28.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:28 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:28.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:28 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:28.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:28 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:28.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:28 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:28.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:28 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:28.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:28 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:28.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:28 smithi097 ceph-mon[35562]: Deploying daemon mds.e on smithi090 2024-09-11T23:19:28.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:28 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.43:6836/3739466479,v1:172.21.15.43:6837/3739466479] up:boot 2024-09-11T23:19:28.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:28 smithi097 ceph-mon[35562]: fsmap 4 up:standby 2024-09-11T23:19:28.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:28 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:19:28.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:28 smithi043 ceph-mon[30950]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:28.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:28 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:28.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:28 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:28.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:28 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:28.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:28 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:28.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:28 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:28.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:28 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:28.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:28 smithi043 ceph-mon[30950]: Deploying daemon mds.e on smithi090 2024-09-11T23:19:28.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:28 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.43:6836/3739466479,v1:172.21.15.43:6837/3739466479] up:boot 2024-09-11T23:19:28.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:28 smithi043 ceph-mon[30950]: fsmap 4 up:standby 2024-09-11T23:19:28.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:28 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:19:28.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.648+0000 7f2abdfe1640 1 -- 172.21.15.43:0/317479311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ab81997b0 con 0x7f2ab810c170 2024-09-11T23:19:28.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.648+0000 7f2ab4ff9640 1 -- 172.21.15.43:0/317479311 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3907 (secure 0 0 0) 0x7f2ab81997b0 con 0x7f2ab810c170 2024-09-11T23:19:28.651 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:19:28.652 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":5,"btime":"2024-09-11T23:19:27:902125+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-11T23:19:28.652 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 5 2024-09-11T23:19:28.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.651+0000 7f2abdfe1640 1 -- 172.21.15.43:0/317479311 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f2aa00a2b80 msgr2=0x7f2aa00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.651+0000 7f2abdfe1640 1 --2- 172.21.15.43:0/317479311 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f2aa00a2b80 0x7f2aa00a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8006010 tx=0x7f2aa80058d0 comp rx=0 tx=0).stop 2024-09-11T23:19:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.651+0000 7f2abdfe1640 1 -- 172.21.15.43:0/317479311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ab810c170 msgr2=0x7f2ab819cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.651+0000 7f2abdfe1640 1 --2- 172.21.15.43:0/317479311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ab810c170 0x7f2ab819cd20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000eb10 tx=0x7f2ab000c2d0 comp rx=0 tx=0).stop 2024-09-11T23:19:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.651+0000 7f2abdfe1640 1 -- 172.21.15.43:0/317479311 shutdown_connections 2024-09-11T23:19:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.651+0000 7f2abdfe1640 1 --2- 172.21.15.43:0/317479311 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f2aa00a2b80 0x7f2aa00a5040 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.651+0000 7f2abdfe1640 1 --2- 172.21.15.43:0/317479311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ab810c170 0x7f2ab819cd20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.651+0000 7f2abdfe1640 1 --2- 172.21.15.43:0/317479311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab8103660 0x7f2ab8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.651+0000 7f2abdfe1640 1 --2- 172.21.15.43:0/317479311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ab8102d40 0x7f2ab8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.651+0000 7f2abdfe1640 1 -- 172.21.15.43:0/317479311 >> 172.21.15.43:0/317479311 conn(0x7f2ab80fc7a0 msgr2=0x7f2ab80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.651+0000 7f2abdfe1640 1 -- 172.21.15.43:0/317479311 shutdown_connections 2024-09-11T23:19:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:28.651+0000 7f2abdfe1640 1 -- 172.21.15.43:0/317479311 wait complete. 2024-09-11T23:19:28.743 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:28 smithi090 ceph-mon[35130]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:28.743 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:28 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:28.743 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:28 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:28.743 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:28 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:28.743 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:28 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:28.743 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:28 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:28.743 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:28 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:28.743 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:28 smithi090 ceph-mon[35130]: Deploying daemon mds.e on smithi090 2024-09-11T23:19:28.744 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:28 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.43:6836/3739466479,v1:172.21.15.43:6837/3739466479] up:boot 2024-09-11T23:19:28.744 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:28 smithi090 ceph-mon[35130]: fsmap 4 up:standby 2024-09-11T23:19:28.744 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:28 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:19:29.440 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:29 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/317479311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:29.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:29 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/317479311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:29.770 INFO:journalctl@ceph.mds.e.smithi090.stdout:Sep 11 23:19:29 smithi090 podman[64551]: 2024-09-11 23:19:29.673066089 +0000 UTC m=+0.132464987 container create 53810d43153a5c8a33fb0bb1a260c2f97e6bc32eba03e8c46b9eca314bc1f2ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-e, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:19:29.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:29 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/317479311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:30.270 INFO:journalctl@ceph.mds.e.smithi090.stdout:Sep 11 23:19:29 smithi090 podman[64551]: 2024-09-11 23:19:29.918157653 +0000 UTC m=+0.377556552 container init 53810d43153a5c8a33fb0bb1a260c2f97e6bc32eba03e8c46b9eca314bc1f2ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T23:19:30.270 INFO:journalctl@ceph.mds.e.smithi090.stdout:Sep 11 23:19:29 smithi090 podman[64551]: 2024-09-11 23:19:29.920908916 +0000 UTC m=+0.380307815 container start 53810d43153a5c8a33fb0bb1a260c2f97e6bc32eba03e8c46b9eca314bc1f2ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-e, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T23:19:30.270 INFO:journalctl@ceph.mds.e.smithi090.stdout:Sep 11 23:19:29 smithi090 ceph-mds[64570]: -- 172.21.15.90:0/2596638214 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bd3b43f860 con 0x55bd3b41d000 2024-09-11T23:19:30.270 INFO:journalctl@ceph.mds.e.smithi090.stdout:Sep 11 23:19:29 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-e[64566]: starting mds.e at 2024-09-11T23:19:30.270 INFO:journalctl@ceph.mds.e.smithi090.stdout:Sep 11 23:19:30 smithi090 bash[64551]: 53810d43153a5c8a33fb0bb1a260c2f97e6bc32eba03e8c46b9eca314bc1f2ed 2024-09-11T23:19:30.270 INFO:journalctl@ceph.mds.e.smithi090.stdout:Sep 11 23:19:30 smithi090 systemd[1]: Started Ceph mds.e for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:19:30.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:30 smithi043 ceph-mon[30950]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:30.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:30 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:30.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:30 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:30.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:30 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:30.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:30 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:30.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:30 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:30.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:30 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:30.646 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:30 smithi097 ceph-mon[35562]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:30.646 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:30 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:30.647 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:30 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:30.647 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:30 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:30.647 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:30 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:30.647 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:30 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:30.647 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:30 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:30.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:30 smithi090 ceph-mon[35130]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:30.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:30 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:30.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:30 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:30.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:30 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:30.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:30 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:30.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:30 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:30.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:30 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:31.314 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:19:31.583 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:31.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:31 smithi043 ceph-mon[30950]: Deploying daemon mds.f on smithi097 2024-09-11T23:19:31.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:31 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.90:6834/4233140081,v1:172.21.15.90:6835/4233140081] up:boot 2024-09-11T23:19:31.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:31 smithi043 ceph-mon[30950]: fsmap 5 up:standby 2024-09-11T23:19:31.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:31 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:19:31.613 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:31 smithi097 ceph-mon[35562]: Deploying daemon mds.f on smithi097 2024-09-11T23:19:31.613 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:31 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.90:6834/4233140081,v1:172.21.15.90:6835/4233140081] up:boot 2024-09-11T23:19:31.613 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:31 smithi097 ceph-mon[35562]: fsmap 5 up:standby 2024-09-11T23:19:31.613 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:31 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:19:31.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:31 smithi090 ceph-mon[35130]: Deploying daemon mds.f on smithi097 2024-09-11T23:19:31.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:31 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.90:6834/4233140081,v1:172.21.15.90:6835/4233140081] up:boot 2024-09-11T23:19:31.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:31 smithi090 ceph-mon[35130]: fsmap 5 up:standby 2024-09-11T23:19:31.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:31 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:19:31.998 INFO:journalctl@ceph.mds.f.smithi097.stdout:Sep 11 23:19:31 smithi097 systemd[1]: Starting Ceph mds.f for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:19:32.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.191+0000 7fe4d0845640 1 -- 172.21.15.43:0/3066865826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4cc101de0 msgr2=0x7fe4cc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:32.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.191+0000 7fe4d0845640 1 --2- 172.21.15.43:0/3066865826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4cc101de0 0x7fe4cc1021c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009900 tx=0x7fe4bc02a410 comp rx=0 tx=0).stop 2024-09-11T23:19:32.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.191+0000 7fe4d0845640 1 -- 172.21.15.43:0/3066865826 shutdown_connections 2024-09-11T23:19:32.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.191+0000 7fe4d0845640 1 --2- 172.21.15.43:0/3066865826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4cc106f10 0x7fe4cc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:32.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.191+0000 7fe4d0845640 1 --2- 172.21.15.43:0/3066865826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4cc102790 0x7fe4cc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:32.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.191+0000 7fe4d0845640 1 --2- 172.21.15.43:0/3066865826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4cc101de0 0x7fe4cc1021c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:32.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.191+0000 7fe4d0845640 1 -- 172.21.15.43:0/3066865826 >> 172.21.15.43:0/3066865826 conn(0x7fe4cc0fd8c0 msgr2=0x7fe4cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:32.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.191+0000 7fe4d0845640 1 -- 172.21.15.43:0/3066865826 shutdown_connections 2024-09-11T23:19:32.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.191+0000 7fe4d0845640 1 -- 172.21.15.43:0/3066865826 wait complete. 2024-09-11T23:19:32.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.192+0000 7fe4d0845640 1 Processor -- start 2024-09-11T23:19:32.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.192+0000 7fe4d0845640 1 -- start start 2024-09-11T23:19:32.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.192+0000 7fe4d0845640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4cc101de0 0x7fe4cc198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:32.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.192+0000 7fe4d0845640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4cc102790 0x7fe4cc198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:32.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.192+0000 7fe4d0845640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4cc106f10 0x7fe4cc19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:32.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.192+0000 7fe4d0845640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc111f30 con 0x7fe4cc102790 2024-09-11T23:19:32.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.192+0000 7fe4d0845640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc111db0 con 0x7fe4cc101de0 2024-09-11T23:19:32.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.192+0000 7fe4d0845640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc1120b0 con 0x7fe4cc106f10 2024-09-11T23:19:32.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.192+0000 7fe4c9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4cc102790 0x7fe4cc198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:32.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.192+0000 7fe4c9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4cc102790 0x7fe4cc198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33154/0 (socket says 172.21.15.43:33154) 2024-09-11T23:19:32.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.192+0000 7fe4c9d9b640 1 -- 172.21.15.43:0/107448870 learned_addr learned my addr 172.21.15.43:0/107448870 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:19:32.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.192+0000 7fe4ca59c640 1 --2- 172.21.15.43:0/107448870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4cc101de0 0x7fe4cc198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:32.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.192+0000 7fe4cad9d640 1 --2- 172.21.15.43:0/107448870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4cc106f10 0x7fe4cc19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:32.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.193+0000 7fe4c9d9b640 1 -- 172.21.15.43:0/107448870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4cc106f10 msgr2=0x7fe4cc19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:32.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.193+0000 7fe4c9d9b640 1 --2- 172.21.15.43:0/107448870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4cc106f10 0x7fe4cc19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:32.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.193+0000 7fe4c9d9b640 1 -- 172.21.15.43:0/107448870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4cc101de0 msgr2=0x7fe4cc198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:32.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.193+0000 7fe4c9d9b640 1 --2- 172.21.15.43:0/107448870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4cc101de0 0x7fe4cc198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:32.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.193+0000 7fe4c9d9b640 1 -- 172.21.15.43:0/107448870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4cc19d3f0 con 0x7fe4cc102790 2024-09-11T23:19:32.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.193+0000 7fe4c9d9b640 1 --2- 172.21.15.43:0/107448870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4cc102790 0x7fe4cc198980 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000d950 tx=0x7fe4c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:32.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.193+0000 7fe4bb7fe640 1 -- 172.21.15.43:0/107448870 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0017030 con 0x7fe4cc102790 2024-09-11T23:19:32.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.193+0000 7fe4bb7fe640 1 -- 172.21.15.43:0/107448870 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4c0004020 con 0x7fe4cc102790 2024-09-11T23:19:32.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.193+0000 7fe4d0845640 1 -- 172.21.15.43:0/107448870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4cc19d6e0 con 0x7fe4cc102790 2024-09-11T23:19:32.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.193+0000 7fe4bb7fe640 1 -- 172.21.15.43:0/107448870 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c00047a0 con 0x7fe4cc102790 2024-09-11T23:19:32.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.193+0000 7fe4d0845640 1 -- 172.21.15.43:0/107448870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4cc1a4fc0 con 0x7fe4cc102790 2024-09-11T23:19:32.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.194+0000 7fe4d0845640 1 -- 172.21.15.43:0/107448870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4cc1021c0 con 0x7fe4cc102790 2024-09-11T23:19:32.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.195+0000 7fe4bb7fe640 1 -- 172.21.15.43:0/107448870 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe4c00041d0 con 0x7fe4cc102790 2024-09-11T23:19:32.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.196+0000 7fe4bb7fe640 1 --2- 172.21.15.43:0/107448870 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe4b40a2c00 0x7fe4b40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:32.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.196+0000 7fe4ca59c640 1 --2- 172.21.15.43:0/107448870 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe4b40a2c00 0x7fe4b40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:32.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.196+0000 7fe4bb7fe640 1 -- 172.21.15.43:0/107448870 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fe4c00ce510 con 0x7fe4cc102790 2024-09-11T23:19:32.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.196+0000 7fe4ca59c640 1 --2- 172.21.15.43:0/107448870 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe4b40a2c00 0x7fe4b40a50c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009230 tx=0x7fe4bc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:32.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.200+0000 7fe4bb7fe640 1 -- 172.21.15.43:0/107448870 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe4c009b910 con 0x7fe4cc102790 2024-09-11T23:19:32.301 INFO:journalctl@ceph.mds.f.smithi097.stdout:Sep 11 23:19:32 smithi097 podman[65139]: 2024-09-11 23:19:32.110921531 +0000 UTC m=+0.015764092 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:19:32.301 INFO:journalctl@ceph.mds.f.smithi097.stdout:Sep 11 23:19:32 smithi097 podman[65139]: 2024-09-11 23:19:32.224810446 +0000 UTC m=+0.129653000 container create 1c28dd39f10a21c7749a5e73b8aeea50859afe41cf94bb42edba1bf429c81edc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:19:32.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:32 smithi043 ceph-mon[30950]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:32.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.410+0000 7fe4d0845640 1 -- 172.21.15.43:0/107448870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4cc02d7c0 con 0x7fe4cc102790 2024-09-11T23:19:32.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.411+0000 7fe4bb7fe640 1 -- 172.21.15.43:0/107448870 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4732 (secure 0 0 0) 0x7fe4c009b210 con 0x7fe4cc102790 2024-09-11T23:19:32.411 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:19:32.411 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":6,"btime":"2024-09-11T23:19:30:306333+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-11T23:19:32.412 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 6 2024-09-11T23:19:32.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.413+0000 7fe4d0845640 1 -- 172.21.15.43:0/107448870 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe4b40a2c00 msgr2=0x7fe4b40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:32.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.413+0000 7fe4d0845640 1 --2- 172.21.15.43:0/107448870 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe4b40a2c00 0x7fe4b40a50c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009230 tx=0x7fe4bc0058d0 comp rx=0 tx=0).stop 2024-09-11T23:19:32.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.413+0000 7fe4d0845640 1 -- 172.21.15.43:0/107448870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4cc102790 msgr2=0x7fe4cc198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:32.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.413+0000 7fe4d0845640 1 --2- 172.21.15.43:0/107448870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4cc102790 0x7fe4cc198980 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000d950 tx=0x7fe4c000de10 comp rx=0 tx=0).stop 2024-09-11T23:19:32.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.413+0000 7fe4d0845640 1 -- 172.21.15.43:0/107448870 shutdown_connections 2024-09-11T23:19:32.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.413+0000 7fe4d0845640 1 --2- 172.21.15.43:0/107448870 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe4b40a2c00 0x7fe4b40a50c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:32.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.413+0000 7fe4d0845640 1 --2- 172.21.15.43:0/107448870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4cc106f10 0x7fe4cc19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:32.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.413+0000 7fe4d0845640 1 --2- 172.21.15.43:0/107448870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4cc102790 0x7fe4cc198980 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:32.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.413+0000 7fe4d0845640 1 --2- 172.21.15.43:0/107448870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4cc101de0 0x7fe4cc198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:32.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.413+0000 7fe4d0845640 1 -- 172.21.15.43:0/107448870 >> 172.21.15.43:0/107448870 conn(0x7fe4cc0fd8c0 msgr2=0x7fe4cc0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:32.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.413+0000 7fe4d0845640 1 -- 172.21.15.43:0/107448870 shutdown_connections 2024-09-11T23:19:32.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:32.413+0000 7fe4d0845640 1 -- 172.21.15.43:0/107448870 wait complete. 2024-09-11T23:19:32.577 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:32 smithi097 ceph-mon[35562]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:32.577 INFO:journalctl@ceph.mds.f.smithi097.stdout:Sep 11 23:19:32 smithi097 podman[65139]: 2024-09-11 23:19:32.477691579 +0000 UTC m=+0.382534130 container init 1c28dd39f10a21c7749a5e73b8aeea50859afe41cf94bb42edba1bf429c81edc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-11T23:19:32.577 INFO:journalctl@ceph.mds.f.smithi097.stdout:Sep 11 23:19:32 smithi097 podman[65139]: 2024-09-11 23:19:32.480465989 +0000 UTC m=+0.385308540 container start 1c28dd39f10a21c7749a5e73b8aeea50859afe41cf94bb42edba1bf429c81edc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-11T23:19:32.578 INFO:journalctl@ceph.mds.f.smithi097.stdout:Sep 11 23:19:32 smithi097 ceph-mds[65158]: -- 172.21.15.97:0/1125290817 <== mon.1 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cfe3d65860 con 0x55cfe3d43000 2024-09-11T23:19:32.578 INFO:journalctl@ceph.mds.f.smithi097.stdout:Sep 11 23:19:32 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-f[65154]: starting mds.f at 2024-09-11T23:19:32.578 INFO:journalctl@ceph.mds.f.smithi097.stdout:Sep 11 23:19:32 smithi097 bash[65139]: 1c28dd39f10a21c7749a5e73b8aeea50859afe41cf94bb42edba1bf429c81edc 2024-09-11T23:19:32.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:32 smithi090 ceph-mon[35130]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:33.049 INFO:journalctl@ceph.mds.f.smithi097.stdout:Sep 11 23:19:32 smithi097 systemd[1]: Started Ceph mds.f for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:19:33.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:33 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/107448870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:33.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:33 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:33.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:33 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:33.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:33 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:33.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:33 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:33.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:33 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:33.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:33 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:33.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:33 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/107448870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:33.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:33 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:33.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:33 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:33.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:33 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:33.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:33 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:33.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:33 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:33.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:33 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:33.768 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:33 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/107448870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:33.768 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:33 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:33.768 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:33 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:33.768 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:33 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:33.768 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:33 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:33.768 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:33 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:33.768 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:33 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:34.395 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:34 smithi097 ceph-mon[35562]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:34.395 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:34 smithi097 ceph-mon[35562]: Deploying daemon mds.c on smithi097 2024-09-11T23:19:34.396 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:34 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.97:6834/1640511441,v1:172.21.15.97:6835/1640511441] up:boot 2024-09-11T23:19:34.396 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:34 smithi097 ceph-mon[35562]: fsmap 6 up:standby 2024-09-11T23:19:34.396 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:34 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:19:34.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:34 smithi043 ceph-mon[30950]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:34.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:34 smithi043 ceph-mon[30950]: Deploying daemon mds.c on smithi097 2024-09-11T23:19:34.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:34 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.97:6834/1640511441,v1:172.21.15.97:6835/1640511441] up:boot 2024-09-11T23:19:34.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:34 smithi043 ceph-mon[30950]: fsmap 6 up:standby 2024-09-11T23:19:34.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:34 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:19:34.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:34 smithi090 ceph-mon[35130]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:34.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:34 smithi090 ceph-mon[35130]: Deploying daemon mds.c on smithi097 2024-09-11T23:19:34.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:34 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.97:6834/1640511441,v1:172.21.15.97:6835/1640511441] up:boot 2024-09-11T23:19:34.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:34 smithi090 ceph-mon[35130]: fsmap 6 up:standby 2024-09-11T23:19:34.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:34 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:19:34.796 INFO:journalctl@ceph.mds.c.smithi097.stdout:Sep 11 23:19:34 smithi097 systemd[1]: Starting Ceph mds.c for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:19:35.051 INFO:journalctl@ceph.mds.c.smithi097.stdout:Sep 11 23:19:34 smithi097 podman[65441]: 2024-09-11 23:19:34.698076309 +0000 UTC m=+0.015737650 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:19:35.051 INFO:journalctl@ceph.mds.c.smithi097.stdout:Sep 11 23:19:34 smithi097 podman[65441]: 2024-09-11 23:19:34.830373258 +0000 UTC m=+0.148034629 container create 9eaae0bf15046461d7cb6a5331a3ea3198f61264d23dd43512df0392a366d5c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-11T23:19:35.069 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:19:35.347 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:35.551 INFO:journalctl@ceph.mds.c.smithi097.stdout:Sep 11 23:19:35 smithi097 podman[65441]: 2024-09-11 23:19:35.091298503 +0000 UTC m=+0.408959838 container init 9eaae0bf15046461d7cb6a5331a3ea3198f61264d23dd43512df0392a366d5c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:19:35.551 INFO:journalctl@ceph.mds.c.smithi097.stdout:Sep 11 23:19:35 smithi097 podman[65441]: 2024-09-11 23:19:35.094102693 +0000 UTC m=+0.411764021 container start 9eaae0bf15046461d7cb6a5331a3ea3198f61264d23dd43512df0392a366d5c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-c, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T23:19:35.551 INFO:journalctl@ceph.mds.c.smithi097.stdout:Sep 11 23:19:35 smithi097 ceph-mds[65460]: -- 172.21.15.97:0/3133728444 <== mon.1 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5644174e7860 con 0x5644174c4800 2024-09-11T23:19:35.551 INFO:journalctl@ceph.mds.c.smithi097.stdout:Sep 11 23:19:35 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-c[65456]: starting mds.c at 2024-09-11T23:19:35.551 INFO:journalctl@ceph.mds.c.smithi097.stdout:Sep 11 23:19:35 smithi097 bash[65441]: 9eaae0bf15046461d7cb6a5331a3ea3198f61264d23dd43512df0392a366d5c4 2024-09-11T23:19:35.551 INFO:journalctl@ceph.mds.c.smithi097.stdout:Sep 11 23:19:35 smithi097 systemd[1]: Started Ceph mds.c for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:19:35.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.842+0000 7fd6ab712640 1 -- 172.21.15.43:0/3141638408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6a4102790 msgr2=0x7fd6a41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:35.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.842+0000 7fd6ab712640 1 --2- 172.21.15.43:0/3141638408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6a4102790 0x7fd6a41067e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd694009900 tx=0x7fd69402a410 comp rx=0 tx=0).stop 2024-09-11T23:19:35.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.842+0000 7fd6ab712640 1 -- 172.21.15.43:0/3141638408 shutdown_connections 2024-09-11T23:19:35.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.842+0000 7fd6ab712640 1 --2- 172.21.15.43:0/3141638408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6a4106f10 0x7fd6a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.842+0000 7fd6ab712640 1 --2- 172.21.15.43:0/3141638408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6a4102790 0x7fd6a41067e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.842+0000 7fd6ab712640 1 --2- 172.21.15.43:0/3141638408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6a4101de0 0x7fd6a41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.842+0000 7fd6ab712640 1 -- 172.21.15.43:0/3141638408 >> 172.21.15.43:0/3141638408 conn(0x7fd6a40fd8c0 msgr2=0x7fd6a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.842+0000 7fd6ab712640 1 -- 172.21.15.43:0/3141638408 shutdown_connections 2024-09-11T23:19:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.842+0000 7fd6ab712640 1 -- 172.21.15.43:0/3141638408 wait complete. 2024-09-11T23:19:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.843+0000 7fd6ab712640 1 Processor -- start 2024-09-11T23:19:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.843+0000 7fd6ab712640 1 -- start start 2024-09-11T23:19:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.844+0000 7fd6ab712640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6a4101de0 0x7fd6a4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.844+0000 7fd6ab712640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6a4102790 0x7fd6a41989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.844+0000 7fd6ab712640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6a4106f10 0x7fd6a419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.844+0000 7fd6ab712640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a4111f70 con 0x7fd6a4101de0 2024-09-11T23:19:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.844+0000 7fd6ab712640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a4111df0 con 0x7fd6a4106f10 2024-09-11T23:19:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.844+0000 7fd6ab712640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a41120f0 con 0x7fd6a4102790 2024-09-11T23:19:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.844+0000 7fd6a94ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6a4101de0 0x7fd6a4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.844+0000 7fd6a94ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6a4101de0 0x7fd6a4198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33166/0 (socket says 172.21.15.43:33166) 2024-09-11T23:19:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.844+0000 7fd6a94ae640 1 -- 172.21.15.43:0/2497633917 learned_addr learned my addr 172.21.15.43:0/2497633917 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:19:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.844+0000 7fd6a8cad640 1 --2- 172.21.15.43:0/2497633917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6a4102790 0x7fd6a41989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.844+0000 7fd6a9caf640 1 --2- 172.21.15.43:0/2497633917 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6a4106f10 0x7fd6a419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.844+0000 7fd6a94ae640 1 -- 172.21.15.43:0/2497633917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6a4102790 msgr2=0x7fd6a41989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.844+0000 7fd6a94ae640 1 --2- 172.21.15.43:0/2497633917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6a4102790 0x7fd6a41989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.844+0000 7fd6a94ae640 1 -- 172.21.15.43:0/2497633917 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6a4106f10 msgr2=0x7fd6a419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.844+0000 7fd6a94ae640 1 --2- 172.21.15.43:0/2497633917 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6a4106f10 0x7fd6a419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.844+0000 7fd6a94ae640 1 -- 172.21.15.43:0/2497633917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a419d440 con 0x7fd6a4101de0 2024-09-11T23:19:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.844+0000 7fd6a94ae640 1 --2- 172.21.15.43:0/2497633917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6a4101de0 0x7fd6a4198490 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000d950 tx=0x7fd6a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.844+0000 7fd69a7fc640 1 -- 172.21.15.43:0/2497633917 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0017070 con 0x7fd6a4101de0 2024-09-11T23:19:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.845+0000 7fd69a7fc640 1 -- 172.21.15.43:0/2497633917 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6a0009c30 con 0x7fd6a4101de0 2024-09-11T23:19:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.845+0000 7fd6ab712640 1 -- 172.21.15.43:0/2497633917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a419d730 con 0x7fd6a4101de0 2024-09-11T23:19:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.845+0000 7fd69a7fc640 1 -- 172.21.15.43:0/2497633917 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a000f6b0 con 0x7fd6a4101de0 2024-09-11T23:19:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.845+0000 7fd6ab712640 1 -- 172.21.15.43:0/2497633917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a41a5010 con 0x7fd6a4101de0 2024-09-11T23:19:35.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.845+0000 7fd6ab712640 1 -- 172.21.15.43:0/2497633917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a41021c0 con 0x7fd6a4101de0 2024-09-11T23:19:35.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.847+0000 7fd69a7fc640 1 -- 172.21.15.43:0/2497633917 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd6a00109d0 con 0x7fd6a4101de0 2024-09-11T23:19:35.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.848+0000 7fd69a7fc640 1 --2- 172.21.15.43:0/2497633917 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd6900a2b80 0x7fd6900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:35.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.848+0000 7fd6a8cad640 1 --2- 172.21.15.43:0/2497633917 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd6900a2b80 0x7fd6900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:35.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.848+0000 7fd69a7fc640 1 -- 172.21.15.43:0/2497633917 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fd6a00ce730 con 0x7fd6a4101de0 2024-09-11T23:19:35.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.849+0000 7fd6a8cad640 1 --2- 172.21.15.43:0/2497633917 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd6900a2b80 0x7fd6900a5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd694009230 tx=0x7fd6940058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:35.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:35.851+0000 7fd69a7fc640 1 -- 172.21.15.43:0/2497633917 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd6a0009de0 con 0x7fd6a4101de0 2024-09-11T23:19:36.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:36.067+0000 7fd6ab712640 1 -- 172.21.15.43:0/2497633917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6a41997f0 con 0x7fd6a4101de0 2024-09-11T23:19:36.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:36.068+0000 7fd69a7fc640 1 -- 172.21.15.43:0/2497633917 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6379 (secure 0 0 0) 0x7fd6a009b4e0 con 0x7fd6a4101de0 2024-09-11T23:19:36.068 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:19:36.068 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":8,"btime":"2024-09-11T23:19:35:325190+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-11T23:19:36.069 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 8 2024-09-11T23:19:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:36.070+0000 7fd6ab712640 1 -- 172.21.15.43:0/2497633917 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd6900a2b80 msgr2=0x7fd6900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:36.070+0000 7fd6ab712640 1 --2- 172.21.15.43:0/2497633917 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd6900a2b80 0x7fd6900a5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd694009230 tx=0x7fd6940058d0 comp rx=0 tx=0).stop 2024-09-11T23:19:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:36.070+0000 7fd6ab712640 1 -- 172.21.15.43:0/2497633917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6a4101de0 msgr2=0x7fd6a4198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:36.071+0000 7fd6ab712640 1 --2- 172.21.15.43:0/2497633917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6a4101de0 0x7fd6a4198490 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000d950 tx=0x7fd6a000de10 comp rx=0 tx=0).stop 2024-09-11T23:19:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:36.071+0000 7fd6ab712640 1 -- 172.21.15.43:0/2497633917 shutdown_connections 2024-09-11T23:19:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:36.071+0000 7fd6ab712640 1 --2- 172.21.15.43:0/2497633917 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd6900a2b80 0x7fd6900a5040 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:36.071+0000 7fd6ab712640 1 --2- 172.21.15.43:0/2497633917 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6a4106f10 0x7fd6a419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:36.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:36.071+0000 7fd6ab712640 1 --2- 172.21.15.43:0/2497633917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6a4102790 0x7fd6a41989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:36.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:36.071+0000 7fd6ab712640 1 --2- 172.21.15.43:0/2497633917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6a4101de0 0x7fd6a4198490 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:36.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:36.071+0000 7fd6ab712640 1 -- 172.21.15.43:0/2497633917 >> 172.21.15.43:0/2497633917 conn(0x7fd6a40fd8c0 msgr2=0x7fd6a40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:36.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:36.071+0000 7fd6ab712640 1 -- 172.21.15.43:0/2497633917 shutdown_connections 2024-09-11T23:19:36.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:36.071+0000 7fd6ab712640 1 -- 172.21.15.43:0/2497633917 wait complete. 2024-09-11T23:19:36.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:36 smithi043 ceph-mon[30950]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:36.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:36 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:36.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:36 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:36.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:36 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:36.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:36 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:36.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:36 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:36.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:36 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:36.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:36 smithi043 ceph-mon[30950]: Deploying daemon mds.h on smithi090 2024-09-11T23:19:36.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:36 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.97:6836/191447096,v1:172.21.15.97:6837/191447096] up:boot 2024-09-11T23:19:36.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:36 smithi043 ceph-mon[30950]: fsmap 7 up:standby 2024-09-11T23:19:36.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:36 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:19:36.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:36 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2497633917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:36.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:36 smithi090 ceph-mon[35130]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:36.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:36 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:36.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:36 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:36.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:36 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:36.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:36 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:36.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:36 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:36.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:36 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:36.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:36 smithi090 ceph-mon[35130]: Deploying daemon mds.h on smithi090 2024-09-11T23:19:36.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:36 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.97:6836/191447096,v1:172.21.15.97:6837/191447096] up:boot 2024-09-11T23:19:36.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:36 smithi090 ceph-mon[35130]: fsmap 7 up:standby 2024-09-11T23:19:36.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:36 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:19:36.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:36 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2497633917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:36.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:36 smithi097 ceph-mon[35562]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:36.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:36 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:36.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:36 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:36.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:36 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:36.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:36 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:36.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:36 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:36.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:36 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:36.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:36 smithi097 ceph-mon[35562]: Deploying daemon mds.h on smithi090 2024-09-11T23:19:36.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:36 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.97:6836/191447096,v1:172.21.15.97:6837/191447096] up:boot 2024-09-11T23:19:36.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:36 smithi097 ceph-mon[35562]: fsmap 7 up:standby 2024-09-11T23:19:36.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:36 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:19:36.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:36 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2497633917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:37.520 INFO:journalctl@ceph.mds.h.smithi090.stdout:Sep 11 23:19:37 smithi090 systemd[1]: Starting Ceph mds.h for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:19:37.821 INFO:journalctl@ceph.mds.h.smithi090.stdout:Sep 11 23:19:37 smithi090 podman[64853]: 2024-09-11 23:19:37.449421906 +0000 UTC m=+0.015744035 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:19:37.821 INFO:journalctl@ceph.mds.h.smithi090.stdout:Sep 11 23:19:37 smithi090 podman[64853]: 2024-09-11 23:19:37.56847604 +0000 UTC m=+0.134798161 container create a55012e7c4519e7332fc5e9163b992db7b3f58367436666ef4cbe9c5ff98cc21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-11T23:19:37.822 INFO:journalctl@ceph.mds.h.smithi090.stdout:Sep 11 23:19:37 smithi090 podman[64853]: 2024-09-11 23:19:37.822119842 +0000 UTC m=+0.388441971 container init a55012e7c4519e7332fc5e9163b992db7b3f58367436666ef4cbe9c5ff98cc21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-11T23:19:38.250 INFO:journalctl@ceph.mds.h.smithi090.stdout:Sep 11 23:19:37 smithi090 podman[64853]: 2024-09-11 23:19:37.824971819 +0000 UTC m=+0.391293946 container start a55012e7c4519e7332fc5e9163b992db7b3f58367436666ef4cbe9c5ff98cc21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-h, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T23:19:38.250 INFO:journalctl@ceph.mds.h.smithi090.stdout:Sep 11 23:19:37 smithi090 ceph-mds[64872]: -- 172.21.15.90:0/716664726 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555b8edb7860 con 0x555b8ed95400 2024-09-11T23:19:38.250 INFO:journalctl@ceph.mds.h.smithi090.stdout:Sep 11 23:19:37 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-h[64868]: starting mds.h at 2024-09-11T23:19:38.250 INFO:journalctl@ceph.mds.h.smithi090.stdout:Sep 11 23:19:37 smithi090 bash[64853]: a55012e7c4519e7332fc5e9163b992db7b3f58367436666ef4cbe9c5ff98cc21 2024-09-11T23:19:38.250 INFO:journalctl@ceph.mds.h.smithi090.stdout:Sep 11 23:19:37 smithi090 systemd[1]: Started Ceph mds.h for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:19:38.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:38 smithi090 ceph-mon[35130]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:38.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:38 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:38.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:38 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:38.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:38 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:38.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:38 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:38.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:38 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:38.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:38 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:38.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:38 smithi097 ceph-mon[35562]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:38.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:38 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:38.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:38 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:38.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:38 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:38.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:38 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:38.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:38 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:38.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:38 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:38.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:38 smithi043 ceph-mon[30950]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:38.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:38 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:38.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:38 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:38.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:38 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:38.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:38 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:38.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:38 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:38.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:38 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:38.735 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:19:39.005 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:39.459 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:39 smithi043 ceph-mon[30950]: Deploying daemon mds.i on smithi097 2024-09-11T23:19:39.459 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:39 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] up:boot 2024-09-11T23:19:39.459 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:39 smithi043 ceph-mon[30950]: fsmap 8 up:standby 2024-09-11T23:19:39.459 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:39 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:19:39.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:39 smithi090 ceph-mon[35130]: Deploying daemon mds.i on smithi097 2024-09-11T23:19:39.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:39 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] up:boot 2024-09-11T23:19:39.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:39 smithi090 ceph-mon[35130]: fsmap 8 up:standby 2024-09-11T23:19:39.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:39 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:19:39.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:39 smithi097 ceph-mon[35562]: Deploying daemon mds.i on smithi097 2024-09-11T23:19:39.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:39 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] up:boot 2024-09-11T23:19:39.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:39 smithi097 ceph-mon[35562]: fsmap 8 up:standby 2024-09-11T23:19:39.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:39 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:19:39.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.540+0000 7f9cf84b4640 1 -- 172.21.15.43:0/3275437200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0100480 msgr2=0x7f9cf0100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:39.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.540+0000 7f9cf84b4640 1 --2- 172.21.15.43:0/3275437200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0100480 0x7f9cf0100860 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009900 tx=0x7f9cec02a410 comp rx=0 tx=0).stop 2024-09-11T23:19:39.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.540+0000 7f9cf84b4640 1 -- 172.21.15.43:0/3275437200 shutdown_connections 2024-09-11T23:19:39.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.540+0000 7f9cf84b4640 1 --2- 172.21.15.43:0/3275437200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cf01055b0 0x7f9cf010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:39.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.540+0000 7f9cf84b4640 1 --2- 172.21.15.43:0/3275437200 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cf0100e30 0x7f9cf0104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:39.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.540+0000 7f9cf84b4640 1 --2- 172.21.15.43:0/3275437200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0100480 0x7f9cf0100860 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:39.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.540+0000 7f9cf84b4640 1 -- 172.21.15.43:0/3275437200 >> 172.21.15.43:0/3275437200 conn(0x7f9cf0077210 msgr2=0x7f9cf00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:39.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.540+0000 7f9cf84b4640 1 -- 172.21.15.43:0/3275437200 shutdown_connections 2024-09-11T23:19:39.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.540+0000 7f9cf84b4640 1 -- 172.21.15.43:0/3275437200 wait complete. 2024-09-11T23:19:39.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.541+0000 7f9cf84b4640 1 Processor -- start 2024-09-11T23:19:39.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.541+0000 7f9cf84b4640 1 -- start start 2024-09-11T23:19:39.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.541+0000 7f9cf84b4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0100480 0x7f9cf010e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:39.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.541+0000 7f9cf84b4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cf0100e30 0x7f9cf010ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:39.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.541+0000 7f9cf84b4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cf01055b0 0x7f9cf010f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:39.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.541+0000 7f9cf84b4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf0111f90 con 0x7f9cf0100480 2024-09-11T23:19:39.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.541+0000 7f9cf84b4640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf0111e10 con 0x7f9cf0100e30 2024-09-11T23:19:39.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.541+0000 7f9cf84b4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf0112110 con 0x7f9cf01055b0 2024-09-11T23:19:39.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.541+0000 7f9cf6250640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0100480 0x7f9cf010e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:39.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.541+0000 7f9cf6250640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0100480 0x7f9cf010e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33192/0 (socket says 172.21.15.43:33192) 2024-09-11T23:19:39.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.541+0000 7f9cf6250640 1 -- 172.21.15.43:0/3320783318 learned_addr learned my addr 172.21.15.43:0/3320783318 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:19:39.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.542+0000 7f9cf6250640 1 -- 172.21.15.43:0/3320783318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cf01055b0 msgr2=0x7f9cf010f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:39.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.542+0000 7f9cf6250640 1 --2- 172.21.15.43:0/3320783318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cf01055b0 0x7f9cf010f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:39.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.542+0000 7f9cf6250640 1 -- 172.21.15.43:0/3320783318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cf0100e30 msgr2=0x7f9cf010ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:39.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.542+0000 7f9cf5a4f640 1 --2- 172.21.15.43:0/3320783318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cf0100e30 0x7f9cf010ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:39.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.542+0000 7f9cf6250640 1 --2- 172.21.15.43:0/3320783318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cf0100e30 0x7f9cf010ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:39.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.542+0000 7f9cf6250640 1 -- 172.21.15.43:0/3320783318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf01a6910 con 0x7f9cf0100480 2024-09-11T23:19:39.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.542+0000 7f9cf5a4f640 1 --2- 172.21.15.43:0/3320783318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cf0100e30 0x7f9cf010ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:19:39.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.542+0000 7f9cf6250640 1 --2- 172.21.15.43:0/3320783318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0100480 0x7f9cf010e850 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9cec006010 tx=0x7f9cec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:39.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.542+0000 7f9ce77fe640 1 -- 172.21.15.43:0/3320783318 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec03f030 con 0x7f9cf0100480 2024-09-11T23:19:39.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.542+0000 7f9cf84b4640 1 -- 172.21.15.43:0/3320783318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf01a6ba0 con 0x7f9cf0100480 2024-09-11T23:19:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.542+0000 7f9ce77fe640 1 -- 172.21.15.43:0/3320783318 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cec003ce0 con 0x7f9cf0100480 2024-09-11T23:19:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.542+0000 7f9ce77fe640 1 -- 172.21.15.43:0/3320783318 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec035070 con 0x7f9cf0100480 2024-09-11T23:19:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.542+0000 7f9cf84b4640 1 -- 172.21.15.43:0/3320783318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf01a70d0 con 0x7f9cf0100480 2024-09-11T23:19:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.543+0000 7f9cf84b4640 1 -- 172.21.15.43:0/3320783318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cf0100860 con 0x7f9cf0100480 2024-09-11T23:19:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.544+0000 7f9ce77fe640 1 -- 172.21.15.43:0/3320783318 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9cec042030 con 0x7f9cf0100480 2024-09-11T23:19:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.545+0000 7f9ce77fe640 1 --2- 172.21.15.43:0/3320783318 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9cdc0a2b80 0x7f9cdc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.545+0000 7f9cf5a4f640 1 --2- 172.21.15.43:0/3320783318 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9cdc0a2b80 0x7f9cdc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.545+0000 7f9ce77fe640 1 -- 172.21.15.43:0/3320783318 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f9cec0ef4b0 con 0x7f9cf0100480 2024-09-11T23:19:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.545+0000 7f9cf5a4f640 1 --2- 172.21.15.43:0/3320783318 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9cdc0a2b80 0x7f9cdc0a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0005d00 tx=0x7f9ce0009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.548+0000 7f9ce77fe640 1 -- 172.21.15.43:0/3320783318 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9cec0bc970 con 0x7f9cf0100480 2024-09-11T23:19:39.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.758+0000 7f9cf84b4640 1 -- 172.21.15.43:0/3320783318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cf010fd80 con 0x7f9cf0100480 2024-09-11T23:19:39.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.759+0000 7f9ce77fe640 1 -- 172.21.15.43:0/3320783318 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7204 (secure 0 0 0) 0x7f9cf010fd80 con 0x7f9cf0100480 2024-09-11T23:19:39.760 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:19:39.760 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"btime":"2024-09-11T23:19:38:241856+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-11T23:19:39.760 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2024-09-11T23:19:39.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.761+0000 7f9cf84b4640 1 -- 172.21.15.43:0/3320783318 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9cdc0a2b80 msgr2=0x7f9cdc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:39.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.761+0000 7f9cf84b4640 1 --2- 172.21.15.43:0/3320783318 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9cdc0a2b80 0x7f9cdc0a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0005d00 tx=0x7f9ce0009550 comp rx=0 tx=0).stop 2024-09-11T23:19:39.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.761+0000 7f9cf84b4640 1 -- 172.21.15.43:0/3320783318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0100480 msgr2=0x7f9cf010e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:39.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.761+0000 7f9cf84b4640 1 --2- 172.21.15.43:0/3320783318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0100480 0x7f9cf010e850 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9cec006010 tx=0x7f9cec0042f0 comp rx=0 tx=0).stop 2024-09-11T23:19:39.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.761+0000 7f9cf84b4640 1 -- 172.21.15.43:0/3320783318 shutdown_connections 2024-09-11T23:19:39.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.761+0000 7f9cf84b4640 1 --2- 172.21.15.43:0/3320783318 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9cdc0a2b80 0x7f9cdc0a5040 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:39.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.761+0000 7f9cf84b4640 1 --2- 172.21.15.43:0/3320783318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cf01055b0 0x7f9cf010f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:39.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.761+0000 7f9cf84b4640 1 --2- 172.21.15.43:0/3320783318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cf0100e30 0x7f9cf010ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:39.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.761+0000 7f9cf84b4640 1 --2- 172.21.15.43:0/3320783318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0100480 0x7f9cf010e850 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:39.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.761+0000 7f9cf84b4640 1 -- 172.21.15.43:0/3320783318 >> 172.21.15.43:0/3320783318 conn(0x7f9cf0077210 msgr2=0x7f9cf0103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:39.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.762+0000 7f9cf84b4640 1 -- 172.21.15.43:0/3320783318 shutdown_connections 2024-09-11T23:19:39.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:39.762+0000 7f9cf84b4640 1 -- 172.21.15.43:0/3320783318 wait complete. 2024-09-11T23:19:39.948 INFO:journalctl@ceph.mds.i.smithi097.stdout:Sep 11 23:19:39 smithi097 systemd[1]: Starting Ceph mds.i for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:19:40.256 INFO:journalctl@ceph.mds.i.smithi097.stdout:Sep 11 23:19:40 smithi097 podman[65744]: 2024-09-11 23:19:40.05970445 +0000 UTC m=+0.015713224 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:19:40.256 INFO:journalctl@ceph.mds.i.smithi097.stdout:Sep 11 23:19:40 smithi097 podman[65744]: 2024-09-11 23:19:40.193900323 +0000 UTC m=+0.149909087 container create 47fd2598a74d92817b10daf8ed346baa155827baa7ea36158642ce0424b2cc97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-i, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-11T23:19:40.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:40 smithi043 ceph-mon[30950]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:40.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:40 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3320783318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:40.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:40 smithi090 ceph-mon[35130]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:40.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:40 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3320783318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:40.546 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:40 smithi097 ceph-mon[35562]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:40.546 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:40 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3320783318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:40.546 INFO:journalctl@ceph.mds.i.smithi097.stdout:Sep 11 23:19:40 smithi097 podman[65744]: 2024-09-11 23:19:40.446593464 +0000 UTC m=+0.402602228 container init 47fd2598a74d92817b10daf8ed346baa155827baa7ea36158642ce0424b2cc97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T23:19:40.546 INFO:journalctl@ceph.mds.i.smithi097.stdout:Sep 11 23:19:40 smithi097 podman[65744]: 2024-09-11 23:19:40.449408013 +0000 UTC m=+0.405416778 container start 47fd2598a74d92817b10daf8ed346baa155827baa7ea36158642ce0424b2cc97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-i, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-09-11T23:19:40.546 INFO:journalctl@ceph.mds.i.smithi097.stdout:Sep 11 23:19:40 smithi097 ceph-mds[65764]: -- 172.21.15.97:0/4110408126 <== mon.1 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558301ddb860 con 0x558301db9400 2024-09-11T23:19:40.546 INFO:journalctl@ceph.mds.i.smithi097.stdout:Sep 11 23:19:40 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-i[65760]: starting mds.i at 2024-09-11T23:19:40.546 INFO:journalctl@ceph.mds.i.smithi097.stdout:Sep 11 23:19:40 smithi097 bash[65744]: 47fd2598a74d92817b10daf8ed346baa155827baa7ea36158642ce0424b2cc97 2024-09-11T23:19:40.801 INFO:journalctl@ceph.mds.i.smithi097.stdout:Sep 11 23:19:40 smithi097 systemd[1]: Started Ceph mds.i for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:19:41.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:41.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:41.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:41.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:41.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:41.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:41.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:41 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] up:boot 2024-09-11T23:19:41.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:41 smithi043 ceph-mon[30950]: fsmap 9 up:standby 2024-09-11T23:19:41.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:41 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:19:42.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:42.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:42.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:42.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:42.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:42.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:42.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:41 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] up:boot 2024-09-11T23:19:42.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:41 smithi090 ceph-mon[35130]: fsmap 9 up:standby 2024-09-11T23:19:42.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:41 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:19:42.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:42.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:42.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:42.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:42.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:42.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:42.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:41 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] up:boot 2024-09-11T23:19:42.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:41 smithi097 ceph-mon[35562]: fsmap 9 up:standby 2024-09-11T23:19:42.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:41 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:19:42.393 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:19:42.823 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:42 smithi043 ceph-mon[30950]: Deploying daemon mds.g on smithi043 2024-09-11T23:19:42.823 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:42 smithi043 ceph-mon[30950]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:42.823 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 11 23:19:42 smithi043 systemd[1]: Starting Ceph mds.g for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:19:42.952 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:42 smithi097 ceph-mon[35562]: Deploying daemon mds.g on smithi043 2024-09-11T23:19:42.952 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:42 smithi097 ceph-mon[35562]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:43.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:42 smithi090 ceph-mon[35130]: Deploying daemon mds.g on smithi043 2024-09-11T23:19:43.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:42 smithi090 ceph-mon[35130]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:43.110 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 11 23:19:42 smithi043 podman[71325]: 2024-09-11 23:19:42.726794033 +0000 UTC m=+0.016729277 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:19:43.110 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 11 23:19:42 smithi043 podman[71325]: 2024-09-11 23:19:42.866568128 +0000 UTC m=+0.156503371 container create e7f1f9ed7f3cde862ea82ce0b67471ae715698dd5477d1133d3cf51589f02e84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-g, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-11T23:19:43.259 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:43.387 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 11 23:19:43 smithi043 podman[71325]: 2024-09-11 23:19:43.13658219 +0000 UTC m=+0.426517445 container init e7f1f9ed7f3cde862ea82ce0b67471ae715698dd5477d1133d3cf51589f02e84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-09-11T23:19:43.387 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 11 23:19:43 smithi043 podman[71325]: 2024-09-11 23:19:43.139754098 +0000 UTC m=+0.429689342 container start e7f1f9ed7f3cde862ea82ce0b67471ae715698dd5477d1133d3cf51589f02e84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-g, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-11T23:19:43.387 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 11 23:19:43 smithi043 ceph-mds[71367]: -- 172.21.15.43:0/829217417 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563674c81860 con 0x563674c5e800 2024-09-11T23:19:43.387 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 11 23:19:43 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-g[71351]: starting mds.g at 2024-09-11T23:19:43.387 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 11 23:19:43 smithi043 bash[71325]: e7f1f9ed7f3cde862ea82ce0b67471ae715698dd5477d1133d3cf51589f02e84 2024-09-11T23:19:43.387 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 11 23:19:43 smithi043 systemd[1]: Started Ceph mds.g for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:19:43.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.750+0000 7f54483ed640 1 -- 172.21.15.43:0/3255034601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5440103670 msgr2=0x7f544010bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:43.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.750+0000 7f54483ed640 1 --2- 172.21.15.43:0/3255034601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5440103670 0x7f544010bb60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f543c009900 tx=0x7f543c02a480 comp rx=0 tx=0).stop 2024-09-11T23:19:43.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.750+0000 7f54483ed640 1 -- 172.21.15.43:0/3255034601 shutdown_connections 2024-09-11T23:19:43.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.750+0000 7f54483ed640 1 --2- 172.21.15.43:0/3255034601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f544010c180 0x7f544010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:43.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.750+0000 7f54483ed640 1 --2- 172.21.15.43:0/3255034601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5440103670 0x7f544010bb60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:43.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.750+0000 7f54483ed640 1 --2- 172.21.15.43:0/3255034601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5440102d50 0x7f5440103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:43.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.750+0000 7f54483ed640 1 -- 172.21.15.43:0/3255034601 >> 172.21.15.43:0/3255034601 conn(0x7f54400fc7d0 msgr2=0x7f54400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:43.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.751+0000 7f54483ed640 1 -- 172.21.15.43:0/3255034601 shutdown_connections 2024-09-11T23:19:43.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.751+0000 7f54483ed640 1 -- 172.21.15.43:0/3255034601 wait complete. 2024-09-11T23:19:43.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.751+0000 7f54483ed640 1 Processor -- start 2024-09-11T23:19:43.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.751+0000 7f54483ed640 1 -- start start 2024-09-11T23:19:43.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.751+0000 7f54483ed640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5440102d50 0x7f54401984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:43.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.751+0000 7f54483ed640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5440103670 0x7f5440198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:43.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.751+0000 7f54483ed640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f544010c180 0x7f544019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:43.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.751+0000 7f54483ed640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5440111f80 con 0x7f544010c180 2024-09-11T23:19:43.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.751+0000 7f54483ed640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5440111e00 con 0x7f5440103670 2024-09-11T23:19:43.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.751+0000 7f54483ed640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5440112100 con 0x7f5440102d50 2024-09-11T23:19:43.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.752+0000 7f544698a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f544010c180 0x7f544019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:43.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.752+0000 7f544698a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f544010c180 0x7f544019cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56384/0 (socket says 172.21.15.43:56384) 2024-09-11T23:19:43.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.752+0000 7f544698a640 1 -- 172.21.15.43:0/237311112 learned_addr learned my addr 172.21.15.43:0/237311112 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:19:43.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.752+0000 7f5445988640 1 --2- 172.21.15.43:0/237311112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5440103670 0x7f5440198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:43.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.752+0000 7f5446189640 1 --2- 172.21.15.43:0/237311112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5440102d50 0x7f54401984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:43.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.752+0000 7f544698a640 1 -- 172.21.15.43:0/237311112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5440102d50 msgr2=0x7f54401984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:43.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.752+0000 7f544698a640 1 --2- 172.21.15.43:0/237311112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5440102d50 0x7f54401984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:43.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.752+0000 7f544698a640 1 -- 172.21.15.43:0/237311112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5440103670 msgr2=0x7f5440198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:43.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.752+0000 7f544698a640 1 --2- 172.21.15.43:0/237311112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5440103670 0x7f5440198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:43.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.752+0000 7f544698a640 1 -- 172.21.15.43:0/237311112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544019d490 con 0x7f544010c180 2024-09-11T23:19:43.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.752+0000 7f5446189640 1 --2- 172.21.15.43:0/237311112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5440102d50 0x7f54401984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:19:43.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.752+0000 7f544698a640 1 --2- 172.21.15.43:0/237311112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f544010c180 0x7f544019cdb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5438009fd0 tx=0x7f543800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:43.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.752+0000 7f54377fe640 1 -- 172.21.15.43:0/237311112 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438017070 con 0x7f544010c180 2024-09-11T23:19:43.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.752+0000 7f54377fe640 1 -- 172.21.15.43:0/237311112 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5438003e50 con 0x7f544010c180 2024-09-11T23:19:43.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.752+0000 7f54377fe640 1 -- 172.21.15.43:0/237311112 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54380106c0 con 0x7f544010c180 2024-09-11T23:19:43.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.752+0000 7f54483ed640 1 -- 172.21.15.43:0/237311112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f544019d780 con 0x7f544010c180 2024-09-11T23:19:43.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.752+0000 7f54483ed640 1 -- 172.21.15.43:0/237311112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54401a5060 con 0x7f544010c180 2024-09-11T23:19:43.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.753+0000 7f54483ed640 1 -- 172.21.15.43:0/237311112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5440103130 con 0x7f544010c180 2024-09-11T23:19:43.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.755+0000 7f54377fe640 1 -- 172.21.15.43:0/237311112 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5438003970 con 0x7f544010c180 2024-09-11T23:19:43.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.755+0000 7f54377fe640 1 --2- 172.21.15.43:0/237311112 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f542c0a2c00 0x7f542c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:43.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.755+0000 7f54377fe640 1 -- 172.21.15.43:0/237311112 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f54380ce800 con 0x7f544010c180 2024-09-11T23:19:43.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.755+0000 7f5446189640 1 --2- 172.21.15.43:0/237311112 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f542c0a2c00 0x7f542c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:43.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.756+0000 7f5446189640 1 --2- 172.21.15.43:0/237311112 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f542c0a2c00 0x7f542c0a50c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f54300058c0 tx=0x7f5430005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:43.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.759+0000 7f54377fe640 1 -- 172.21.15.43:0/237311112 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f543809bc80 con 0x7f544010c180 2024-09-11T23:19:43.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.974+0000 7f54483ed640 1 -- 172.21.15.43:0/237311112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5440199840 con 0x7f544010c180 2024-09-11T23:19:43.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.975+0000 7f54377fe640 1 -- 172.21.15.43:0/237311112 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8857 (secure 0 0 0) 0x7f543809b580 con 0x7f544010c180 2024-09-11T23:19:43.975 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:19:43.976 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":11,"btime":"2024-09-11T23:19:43:597553+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-11T23:19:43.976 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 11 2024-09-11T23:19:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.977+0000 7f54483ed640 1 -- 172.21.15.43:0/237311112 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f542c0a2c00 msgr2=0x7f542c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.977+0000 7f54483ed640 1 --2- 172.21.15.43:0/237311112 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f542c0a2c00 0x7f542c0a50c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f54300058c0 tx=0x7f5430005850 comp rx=0 tx=0).stop 2024-09-11T23:19:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.977+0000 7f54483ed640 1 -- 172.21.15.43:0/237311112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f544010c180 msgr2=0x7f544019cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.977+0000 7f54483ed640 1 --2- 172.21.15.43:0/237311112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f544010c180 0x7f544019cdb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5438009fd0 tx=0x7f543800b260 comp rx=0 tx=0).stop 2024-09-11T23:19:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.977+0000 7f54483ed640 1 -- 172.21.15.43:0/237311112 shutdown_connections 2024-09-11T23:19:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.977+0000 7f54483ed640 1 --2- 172.21.15.43:0/237311112 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f542c0a2c00 0x7f542c0a50c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.977+0000 7f54483ed640 1 --2- 172.21.15.43:0/237311112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f544010c180 0x7f544019cdb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.977+0000 7f54483ed640 1 --2- 172.21.15.43:0/237311112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5440103670 0x7f5440198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.977+0000 7f54483ed640 1 --2- 172.21.15.43:0/237311112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5440102d50 0x7f54401984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.977+0000 7f54483ed640 1 -- 172.21.15.43:0/237311112 >> 172.21.15.43:0/237311112 conn(0x7f54400fc7d0 msgr2=0x7f54400fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.977+0000 7f54483ed640 1 -- 172.21.15.43:0/237311112 shutdown_connections 2024-09-11T23:19:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:43.978+0000 7f54483ed640 1 -- 172.21.15.43:0/237311112 wait complete. 2024-09-11T23:19:44.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:44 smithi090 ceph-mon[35130]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:44.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:44 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:44.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:44 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:44.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:44 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:44.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:44 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:44.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:44 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:44.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:44 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:44.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:44 smithi090 ceph-mon[35130]: Deploying daemon mds.a on smithi043 2024-09-11T23:19:44.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:44 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.43:6838/2165493077,v1:172.21.15.43:6839/2165493077] up:boot 2024-09-11T23:19:44.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:44 smithi090 ceph-mon[35130]: fsmap 10 up:standby 2024-09-11T23:19:44.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:44 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:19:44.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:44 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/237311112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:44.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:44 smithi097 ceph-mon[35562]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:44.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:44 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:44.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:44 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:44.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:44 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:44.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:44 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:44.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:44 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:44.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:44 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:44.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:44 smithi097 ceph-mon[35562]: Deploying daemon mds.a on smithi043 2024-09-11T23:19:44.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:44 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.43:6838/2165493077,v1:172.21.15.43:6839/2165493077] up:boot 2024-09-11T23:19:44.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:44 smithi097 ceph-mon[35562]: fsmap 10 up:standby 2024-09-11T23:19:44.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:44 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:19:44.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:44 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/237311112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:44.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:44 smithi043 ceph-mon[30950]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:44.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:44 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:44.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:44 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:44.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:44 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:44 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:44 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:44 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:44 smithi043 ceph-mon[30950]: Deploying daemon mds.a on smithi043 2024-09-11T23:19:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:44 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.43:6838/2165493077,v1:172.21.15.43:6839/2165493077] up:boot 2024-09-11T23:19:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:44 smithi043 ceph-mon[30950]: fsmap 10 up:standby 2024-09-11T23:19:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:44 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:19:44.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:44 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/237311112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:46.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:46 smithi043 ceph-mon[30950]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:46.360 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 11 23:19:46 smithi043 systemd[1]: Starting Ceph mds.a for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:19:46.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:46 smithi090 ceph-mon[35130]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:46.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:46 smithi097 ceph-mon[35562]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:46.763 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:19:46.788 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 11 23:19:46 smithi043 podman[71722]: 2024-09-11 23:19:46.398636338 +0000 UTC m=+0.016626259 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:19:46.788 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 11 23:19:46 smithi043 podman[71722]: 2024-09-11 23:19:46.522618437 +0000 UTC m=+0.140608348 container create 0697bba84f7eed955a6b3a171ada4b947d183695cf5bb094fd16dd7f7cb8e995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-a, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-11T23:19:47.031 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:47.110 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 11 23:19:46 smithi043 podman[71722]: 2024-09-11 23:19:46.810094359 +0000 UTC m=+0.428084281 container init 0697bba84f7eed955a6b3a171ada4b947d183695cf5bb094fd16dd7f7cb8e995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-a, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-11T23:19:47.110 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 11 23:19:46 smithi043 podman[71722]: 2024-09-11 23:19:46.813891413 +0000 UTC m=+0.431881326 container start 0697bba84f7eed955a6b3a171ada4b947d183695cf5bb094fd16dd7f7cb8e995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True) 2024-09-11T23:19:47.110 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 11 23:19:46 smithi043 ceph-mds[71759]: -- 172.21.15.43:0/2877984297 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d3223dd860 con 0x55d3223ba800 2024-09-11T23:19:47.110 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 11 23:19:46 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-a[71737]: starting mds.a at 2024-09-11T23:19:47.111 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 11 23:19:46 smithi043 bash[71722]: 0697bba84f7eed955a6b3a171ada4b947d183695cf5bb094fd16dd7f7cb8e995 2024-09-11T23:19:47.111 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 11 23:19:46 smithi043 systemd[1]: Started Ceph mds.a for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:19:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.537+0000 7efe4a99b640 1 -- 172.21.15.43:0/2336354028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe4410c170 msgr2=0x7efe4410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.537+0000 7efe4a99b640 1 --2- 172.21.15.43:0/2336354028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe4410c170 0x7efe4410e610 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009ca0 tx=0x7efe3c02a950 comp rx=0 tx=0).stop 2024-09-11T23:19:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.537+0000 7efe4a99b640 1 -- 172.21.15.43:0/2336354028 shutdown_connections 2024-09-11T23:19:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.537+0000 7efe4a99b640 1 --2- 172.21.15.43:0/2336354028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe4410c170 0x7efe4410e610 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.537+0000 7efe4a99b640 1 --2- 172.21.15.43:0/2336354028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe44103660 0x7efe4410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.537+0000 7efe4a99b640 1 --2- 172.21.15.43:0/2336354028 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe44102d40 0x7efe44103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.537+0000 7efe4a99b640 1 -- 172.21.15.43:0/2336354028 >> 172.21.15.43:0/2336354028 conn(0x7efe440fc7a0 msgr2=0x7efe440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.537+0000 7efe4a99b640 1 -- 172.21.15.43:0/2336354028 shutdown_connections 2024-09-11T23:19:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.537+0000 7efe4a99b640 1 -- 172.21.15.43:0/2336354028 wait complete. 2024-09-11T23:19:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.538+0000 7efe4a99b640 1 Processor -- start 2024-09-11T23:19:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.538+0000 7efe4a99b640 1 -- start start 2024-09-11T23:19:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe4a99b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe44102d40 0x7efe441009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe4a99b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe44103660 0x7efe44100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe4a99b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe4410c170 0x7efe44101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe4a99b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efe44111f50 con 0x7efe44102d40 2024-09-11T23:19:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe4a99b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efe44111dd0 con 0x7efe44103660 2024-09-11T23:19:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe4a99b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efe441120d0 con 0x7efe4410c170 2024-09-11T23:19:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe43fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe44102d40 0x7efe441009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe43fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe44102d40 0x7efe441009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56438/0 (socket says 172.21.15.43:56438) 2024-09-11T23:19:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe43fff640 1 -- 172.21.15.43:0/1095494067 learned_addr learned my addr 172.21.15.43:0/1095494067 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:19:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe437fe640 1 --2- 172.21.15.43:0/1095494067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe44103660 0x7efe44100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe48f38640 1 --2- 172.21.15.43:0/1095494067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe4410c170 0x7efe44101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe43fff640 1 -- 172.21.15.43:0/1095494067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe4410c170 msgr2=0x7efe44101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe43fff640 1 --2- 172.21.15.43:0/1095494067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe4410c170 0x7efe44101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:47.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe43fff640 1 -- 172.21.15.43:0/1095494067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe44103660 msgr2=0x7efe44100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:47.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe43fff640 1 --2- 172.21.15.43:0/1095494067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe44103660 0x7efe44100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:47.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe43fff640 1 -- 172.21.15.43:0/1095494067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe441a6920 con 0x7efe44102d40 2024-09-11T23:19:47.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe437fe640 1 --2- 172.21.15.43:0/1095494067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe44103660 0x7efe44100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:47.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe43fff640 1 --2- 172.21.15.43:0/1095494067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe44102d40 0x7efe441009a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efe3400d950 tx=0x7efe3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:47.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe417fa640 1 -- 172.21.15.43:0/1095494067 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe34017070 con 0x7efe44102d40 2024-09-11T23:19:47.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe417fa640 1 -- 172.21.15.43:0/1095494067 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe34009c30 con 0x7efe44102d40 2024-09-11T23:19:47.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe4a99b640 1 -- 172.21.15.43:0/1095494067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe441a6c10 con 0x7efe44102d40 2024-09-11T23:19:47.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.539+0000 7efe417fa640 1 -- 172.21.15.43:0/1095494067 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3400f6b0 con 0x7efe44102d40 2024-09-11T23:19:47.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.540+0000 7efe4a99b640 1 -- 172.21.15.43:0/1095494067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe441a7150 con 0x7efe44102d40 2024-09-11T23:19:47.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.540+0000 7efe4a99b640 1 -- 172.21.15.43:0/1095494067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe44103120 con 0x7efe44102d40 2024-09-11T23:19:47.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.542+0000 7efe417fa640 1 -- 172.21.15.43:0/1095494067 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efe340109d0 con 0x7efe44102d40 2024-09-11T23:19:47.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.542+0000 7efe417fa640 1 --2- 172.21.15.43:0/1095494067 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7efe2c0a2b80 0x7efe2c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:47.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.542+0000 7efe417fa640 1 -- 172.21.15.43:0/1095494067 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7efe340ce7b0 con 0x7efe44102d40 2024-09-11T23:19:47.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.542+0000 7efe437fe640 1 --2- 172.21.15.43:0/1095494067 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7efe2c0a2b80 0x7efe2c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:47.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.543+0000 7efe437fe640 1 --2- 172.21.15.43:0/1095494067 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7efe2c0a2b80 0x7efe2c0a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efe44102430 tx=0x7efe38005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:47.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.546+0000 7efe417fa640 1 -- 172.21.15.43:0/1095494067 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe34009de0 con 0x7efe44102d40 2024-09-11T23:19:47.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.757+0000 7efe4a99b640 1 -- 172.21.15.43:0/1095494067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe44107210 con 0x7efe44102d40 2024-09-11T23:19:47.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.758+0000 7efe417fa640 1 -- 172.21.15.43:0/1095494067 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9683 (secure 0 0 0) 0x7efe3409bbb0 con 0x7efe44102d40 2024-09-11T23:19:47.758 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:19:47.759 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":12,"btime":"2024-09-11T23:19:47:269194+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-11T23:19:47.759 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 12 2024-09-11T23:19:47.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.760+0000 7efe4a99b640 1 -- 172.21.15.43:0/1095494067 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7efe2c0a2b80 msgr2=0x7efe2c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:47.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.760+0000 7efe4a99b640 1 --2- 172.21.15.43:0/1095494067 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7efe2c0a2b80 0x7efe2c0a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efe44102430 tx=0x7efe38005f90 comp rx=0 tx=0).stop 2024-09-11T23:19:47.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.760+0000 7efe4a99b640 1 -- 172.21.15.43:0/1095494067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe44102d40 msgr2=0x7efe441009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:47.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.760+0000 7efe4a99b640 1 --2- 172.21.15.43:0/1095494067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe44102d40 0x7efe441009a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efe3400d950 tx=0x7efe3400de10 comp rx=0 tx=0).stop 2024-09-11T23:19:47.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.760+0000 7efe4a99b640 1 -- 172.21.15.43:0/1095494067 shutdown_connections 2024-09-11T23:19:47.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.760+0000 7efe4a99b640 1 --2- 172.21.15.43:0/1095494067 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7efe2c0a2b80 0x7efe2c0a5040 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:47.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.760+0000 7efe4a99b640 1 --2- 172.21.15.43:0/1095494067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe4410c170 0x7efe44101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:47.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.760+0000 7efe4a99b640 1 --2- 172.21.15.43:0/1095494067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe44103660 0x7efe44100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:47.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.760+0000 7efe4a99b640 1 --2- 172.21.15.43:0/1095494067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe44102d40 0x7efe441009a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:47.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.760+0000 7efe4a99b640 1 -- 172.21.15.43:0/1095494067 >> 172.21.15.43:0/1095494067 conn(0x7efe440fc7a0 msgr2=0x7efe440fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:47.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.760+0000 7efe4a99b640 1 -- 172.21.15.43:0/1095494067 shutdown_connections 2024-09-11T23:19:47.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:47.760+0000 7efe4a99b640 1 -- 172.21.15.43:0/1095494067 wait complete. 2024-09-11T23:19:48.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:47 smithi043 ceph-mon[30950]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:48.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:47 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:48.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:47 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:48.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:47 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:48.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:47 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:47 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:47 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:47 smithi043 ceph-mon[30950]: Deploying daemon mds.b on smithi090 2024-09-11T23:19:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:47 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.43:6840/3592420960,v1:172.21.15.43:6841/3592420960] up:boot 2024-09-11T23:19:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:47 smithi043 ceph-mon[30950]: fsmap 11 up:standby 2024-09-11T23:19:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:47 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:19:48.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:47 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1095494067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:48.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:47 smithi090 ceph-mon[35130]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:48.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:47 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:48.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:47 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:48.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:47 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:48.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:47 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:48.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:47 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:48.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:47 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:48.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:47 smithi090 ceph-mon[35130]: Deploying daemon mds.b on smithi090 2024-09-11T23:19:48.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:47 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.43:6840/3592420960,v1:172.21.15.43:6841/3592420960] up:boot 2024-09-11T23:19:48.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:47 smithi090 ceph-mon[35130]: fsmap 11 up:standby 2024-09-11T23:19:48.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:47 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:19:48.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:47 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1095494067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:48.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:47 smithi097 ceph-mon[35562]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:48.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:47 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:48.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:47 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:48.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:47 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:48.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:47 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T23:19:48.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:47 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T23:19:48.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:47 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:48.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:47 smithi097 ceph-mon[35562]: Deploying daemon mds.b on smithi090 2024-09-11T23:19:48.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:47 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.43:6840/3592420960,v1:172.21.15.43:6841/3592420960] up:boot 2024-09-11T23:19:48.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:47 smithi097 ceph-mon[35562]: fsmap 11 up:standby 2024-09-11T23:19:48.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:47 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:19:48.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:47 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1095494067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:49.425 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:19:49 smithi090 systemd[1]: Starting Ceph mds.b for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:19:49.685 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:19:49 smithi090 podman[65163]: 2024-09-11 23:19:49.327726982 +0000 UTC m=+0.016145664 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:19:49.685 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:19:49 smithi090 podman[65163]: 2024-09-11 23:19:49.432304022 +0000 UTC m=+0.120722733 container create de004e4ae010e8cf6dd3c3c0c5065f1151ba789a005747f71328c46a8290f763 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:19:49.982 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:19:49 smithi090 podman[65163]: 2024-09-11 23:19:49.685446429 +0000 UTC m=+0.373865097 container init de004e4ae010e8cf6dd3c3c0c5065f1151ba789a005747f71328c46a8290f763 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-09-11T23:19:49.983 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:19:49 smithi090 podman[65163]: 2024-09-11 23:19:49.688768143 +0000 UTC m=+0.377186813 container start de004e4ae010e8cf6dd3c3c0c5065f1151ba789a005747f71328c46a8290f763 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3) 2024-09-11T23:19:49.983 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:19:49 smithi090 ceph-mds[65182]: -- 172.21.15.90:0/2931096611 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b6edc57c20 con 0x55b6edc35000 2024-09-11T23:19:49.983 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:19:49 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-b[65178]: starting mds.b at 2024-09-11T23:19:49.983 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:19:49 smithi090 bash[65163]: de004e4ae010e8cf6dd3c3c0c5065f1151ba789a005747f71328c46a8290f763 2024-09-11T23:19:49.983 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:19:49 smithi090 systemd[1]: Started Ceph mds.b for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:19:50.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:49 smithi090 ceph-mon[35130]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:50.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:50.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:50.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:50.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:50.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:50.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:49 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:19:50.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:49 smithi097 ceph-mon[35562]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:50.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:50.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:50.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:50.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:50.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:50.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:49 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:19:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:49 smithi043 ceph-mon[30950]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:49 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:19:50.478 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:19:51.109 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:51.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:50 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] up:boot 2024-09-11T23:19:51.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:50 smithi043 ceph-mon[30950]: fsmap 12 up:standby 2024-09-11T23:19:51.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:50 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:19:51.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:50 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:51.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:50 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] up:boot 2024-09-11T23:19:51.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:50 smithi090 ceph-mon[35130]: fsmap 12 up:standby 2024-09-11T23:19:51.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:50 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:19:51.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:50 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:51.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:50 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] up:boot 2024-09-11T23:19:51.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:50 smithi097 ceph-mon[35562]: fsmap 12 up:standby 2024-09-11T23:19:51.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:50 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:19:51.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:50 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:51.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.607+0000 7fb37eb7c640 1 -- 172.21.15.43:0/4180172664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3781055b0 msgr2=0x7fb37810d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:51.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.607+0000 7fb37eb7c640 1 --2- 172.21.15.43:0/4180172664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3781055b0 0x7fb37810d130 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb370009ca0 tx=0x7fb37002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:19:51.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.607+0000 7fb37eb7c640 1 -- 172.21.15.43:0/4180172664 shutdown_connections 2024-09-11T23:19:51.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.607+0000 7fb37eb7c640 1 --2- 172.21.15.43:0/4180172664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3781055b0 0x7fb37810d130 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:51.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.607+0000 7fb37eb7c640 1 --2- 172.21.15.43:0/4180172664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb378100e30 0x7fb378104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:51.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.607+0000 7fb37eb7c640 1 --2- 172.21.15.43:0/4180172664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb378100480 0x7fb378100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:51.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.607+0000 7fb37eb7c640 1 -- 172.21.15.43:0/4180172664 >> 172.21.15.43:0/4180172664 conn(0x7fb378077210 msgr2=0x7fb3780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:51.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.607+0000 7fb37eb7c640 1 -- 172.21.15.43:0/4180172664 shutdown_connections 2024-09-11T23:19:51.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.607+0000 7fb37eb7c640 1 -- 172.21.15.43:0/4180172664 wait complete. 2024-09-11T23:19:51.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.608+0000 7fb37eb7c640 1 Processor -- start 2024-09-11T23:19:51.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.608+0000 7fb37eb7c640 1 -- start start 2024-09-11T23:19:51.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.608+0000 7fb37eb7c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb378100480 0x7fb37819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:51.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.608+0000 7fb37eb7c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb378100e30 0x7fb37819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:51.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.608+0000 7fb37eb7c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3781055b0 0x7fb37819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:51.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.608+0000 7fb37eb7c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb378112020 con 0x7fb378100e30 2024-09-11T23:19:51.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.608+0000 7fb37eb7c640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb378111ea0 con 0x7fb378100480 2024-09-11T23:19:51.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.608+0000 7fb37eb7c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb3781121a0 con 0x7fb3781055b0 2024-09-11T23:19:51.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.608+0000 7fb377fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb378100e30 0x7fb37819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:51.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.608+0000 7fb377fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb378100e30 0x7fb37819abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51438/0 (socket says 172.21.15.43:51438) 2024-09-11T23:19:51.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.608+0000 7fb377fff640 1 -- 172.21.15.43:0/3366712769 learned_addr learned my addr 172.21.15.43:0/3366712769 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:19:51.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.608+0000 7fb37d119640 1 --2- 172.21.15.43:0/3366712769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3781055b0 0x7fb37819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:51.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.608+0000 7fb37c918640 1 --2- 172.21.15.43:0/3366712769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb378100480 0x7fb37819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:51.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.608+0000 7fb377fff640 1 -- 172.21.15.43:0/3366712769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3781055b0 msgr2=0x7fb37819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:51.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.608+0000 7fb377fff640 1 --2- 172.21.15.43:0/3366712769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3781055b0 0x7fb37819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:51.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.608+0000 7fb377fff640 1 -- 172.21.15.43:0/3366712769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb378100480 msgr2=0x7fb37819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:51.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.608+0000 7fb377fff640 1 --2- 172.21.15.43:0/3366712769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb378100480 0x7fb37819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:51.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.608+0000 7fb377fff640 1 -- 172.21.15.43:0/3366712769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37819f630 con 0x7fb378100e30 2024-09-11T23:19:51.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.609+0000 7fb377fff640 1 --2- 172.21.15.43:0/3366712769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb378100e30 0x7fb37819abc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb36000d770 tx=0x7fb36000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:51.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.609+0000 7fb375ffb640 1 -- 172.21.15.43:0/3366712769 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360017070 con 0x7fb378100e30 2024-09-11T23:19:51.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.609+0000 7fb375ffb640 1 -- 172.21.15.43:0/3366712769 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb360009820 con 0x7fb378100e30 2024-09-11T23:19:51.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.609+0000 7fb37eb7c640 1 -- 172.21.15.43:0/3366712769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37819f920 con 0x7fb378100e30 2024-09-11T23:19:51.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.609+0000 7fb375ffb640 1 -- 172.21.15.43:0/3366712769 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360004cd0 con 0x7fb378100e30 2024-09-11T23:19:51.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.609+0000 7fb37eb7c640 1 -- 172.21.15.43:0/3366712769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3781a7200 con 0x7fb378100e30 2024-09-11T23:19:51.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.610+0000 7fb37eb7c640 1 -- 172.21.15.43:0/3366712769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb378100860 con 0x7fb378100e30 2024-09-11T23:19:51.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.611+0000 7fb375ffb640 1 -- 172.21.15.43:0/3366712769 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb3600099d0 con 0x7fb378100e30 2024-09-11T23:19:51.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.612+0000 7fb375ffb640 1 --2- 172.21.15.43:0/3366712769 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb3640a2c00 0x7fb3640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:51.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.612+0000 7fb37c918640 1 --2- 172.21.15.43:0/3366712769 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb3640a2c00 0x7fb3640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:51.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.612+0000 7fb375ffb640 1 -- 172.21.15.43:0/3366712769 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fb3600ce4e0 con 0x7fb378100e30 2024-09-11T23:19:51.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.612+0000 7fb37c918640 1 --2- 172.21.15.43:0/3366712769 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb3640a2c00 0x7fb3640a50c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb368009990 tx=0x7fb368009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:51.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.616+0000 7fb375ffb640 1 -- 172.21.15.43:0/3366712769 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb36009b8e0 con 0x7fb378100e30 2024-09-11T23:19:51.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.842+0000 7fb37eb7c640 1 -- 172.21.15.43:0/3366712769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3781a7530 con 0x7fb378100e30 2024-09-11T23:19:51.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.843+0000 7fb375ffb640 1 -- 172.21.15.43:0/3366712769 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10509 (secure 0 0 0) 0x7fb3600b9e20 con 0x7fb378100e30 2024-09-11T23:19:51.843 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:19:51.843 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":13,"btime":"2024-09-11T23:19:49:975831+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24497,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-11T23:19:51.844 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 13 2024-09-11T23:19:51.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.846+0000 7fb37eb7c640 1 -- 172.21.15.43:0/3366712769 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb3640a2c00 msgr2=0x7fb3640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:51.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.846+0000 7fb37eb7c640 1 --2- 172.21.15.43:0/3366712769 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb3640a2c00 0x7fb3640a50c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb368009990 tx=0x7fb368009110 comp rx=0 tx=0).stop 2024-09-11T23:19:51.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.846+0000 7fb37eb7c640 1 -- 172.21.15.43:0/3366712769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb378100e30 msgr2=0x7fb37819abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:51.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.846+0000 7fb37eb7c640 1 --2- 172.21.15.43:0/3366712769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb378100e30 0x7fb37819abc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb36000d770 tx=0x7fb36000db70 comp rx=0 tx=0).stop 2024-09-11T23:19:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.846+0000 7fb37eb7c640 1 -- 172.21.15.43:0/3366712769 shutdown_connections 2024-09-11T23:19:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.846+0000 7fb37eb7c640 1 --2- 172.21.15.43:0/3366712769 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb3640a2c00 0x7fb3640a50c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.846+0000 7fb37eb7c640 1 --2- 172.21.15.43:0/3366712769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3781055b0 0x7fb37819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.846+0000 7fb37eb7c640 1 --2- 172.21.15.43:0/3366712769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb378100e30 0x7fb37819abc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.846+0000 7fb37eb7c640 1 --2- 172.21.15.43:0/3366712769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb378100480 0x7fb37819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.846+0000 7fb37eb7c640 1 -- 172.21.15.43:0/3366712769 >> 172.21.15.43:0/3366712769 conn(0x7fb378077210 msgr2=0x7fb378103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.846+0000 7fb37eb7c640 1 -- 172.21.15.43:0/3366712769 shutdown_connections 2024-09-11T23:19:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:51.846+0000 7fb37eb7c640 1 -- 172.21.15.43:0/3366712769 wait complete. 2024-09-11T23:19:52.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:52 smithi043 ceph-mon[30950]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:52.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:52 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:52.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:52 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:52.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:52 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:52.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:52 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:52.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:52 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3366712769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:52.520 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-11T23:19:52.743 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:52 smithi090 ceph-mon[35130]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:52.743 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:52 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:52.743 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:52 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:52.743 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:52 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:52.743 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:52 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:52.743 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:52 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3366712769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:52.800 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:52.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:52 smithi097 ceph-mon[35562]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:52.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:52 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:52.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:52 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:52.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:52 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:52.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:52 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:52.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:52 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3366712769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:19:53.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.332+0000 7fe8416ab640 1 -- 172.21.15.43:0/2195848130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c100a80 msgr2=0x7fe83c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:53.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.332+0000 7fe8416ab640 1 --2- 172.21.15.43:0/2195848130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c100a80 0x7fe83c100ee0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe830009930 tx=0x7fe83002a420 comp rx=0 tx=0).stop 2024-09-11T23:19:53.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.332+0000 7fe8416ab640 1 -- 172.21.15.43:0/2195848130 shutdown_connections 2024-09-11T23:19:53.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.332+0000 7fe8416ab640 1 --2- 172.21.15.43:0/2195848130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe83c101420 0x7fe83c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:53.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.332+0000 7fe8416ab640 1 --2- 172.21.15.43:0/2195848130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c100a80 0x7fe83c100ee0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:53.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.332+0000 7fe8416ab640 1 --2- 172.21.15.43:0/2195848130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe83c106a80 0x7fe83c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:53.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.332+0000 7fe8416ab640 1 -- 172.21.15.43:0/2195848130 >> 172.21.15.43:0/2195848130 conn(0x7fe83c0fc7a0 msgr2=0x7fe83c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:53.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.332+0000 7fe8416ab640 1 -- 172.21.15.43:0/2195848130 shutdown_connections 2024-09-11T23:19:53.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.332+0000 7fe8416ab640 1 -- 172.21.15.43:0/2195848130 wait complete. 2024-09-11T23:19:53.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.333+0000 7fe8416ab640 1 Processor -- start 2024-09-11T23:19:53.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.333+0000 7fe8416ab640 1 -- start start 2024-09-11T23:19:53.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.333+0000 7fe8416ab640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe83c100a80 0x7fe83c0fce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:53.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.333+0000 7fe8416ab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c101420 0x7fe83c0fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:53.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.333+0000 7fe8416ab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe83c106a80 0x7fe83c0fd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:53.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.333+0000 7fe8416ab640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c110e80 con 0x7fe83c101420 2024-09-11T23:19:53.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.333+0000 7fe8416ab640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c110d00 con 0x7fe83c100a80 2024-09-11T23:19:53.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.333+0000 7fe8416ab640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c111000 con 0x7fe83c106a80 2024-09-11T23:19:53.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.334+0000 7fe83a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c101420 0x7fe83c0fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:53.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.334+0000 7fe83affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe83c100a80 0x7fe83c0fce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:53.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.334+0000 7fe83affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe83c100a80 0x7fe83c0fce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:60754/0 (socket says 172.21.15.43:60754) 2024-09-11T23:19:53.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.334+0000 7fe83a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c101420 0x7fe83c0fd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51452/0 (socket says 172.21.15.43:51452) 2024-09-11T23:19:53.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.334+0000 7fe83affd640 1 -- 172.21.15.43:0/3110313662 learned_addr learned my addr 172.21.15.43:0/3110313662 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:19:53.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.334+0000 7fe83b7fe640 1 --2- 172.21.15.43:0/3110313662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe83c106a80 0x7fe83c0fd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:53.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.334+0000 7fe83a7fc640 1 -- 172.21.15.43:0/3110313662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe83c106a80 msgr2=0x7fe83c0fd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:53.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.334+0000 7fe83a7fc640 1 --2- 172.21.15.43:0/3110313662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe83c106a80 0x7fe83c0fd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:53.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.334+0000 7fe83a7fc640 1 -- 172.21.15.43:0/3110313662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe83c100a80 msgr2=0x7fe83c0fce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:53.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.334+0000 7fe83a7fc640 1 --2- 172.21.15.43:0/3110313662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe83c100a80 0x7fe83c0fce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:53.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.334+0000 7fe83a7fc640 1 -- 172.21.15.43:0/3110313662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83c1a6930 con 0x7fe83c101420 2024-09-11T23:19:53.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.334+0000 7fe83affd640 1 --2- 172.21.15.43:0/3110313662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe83c100a80 0x7fe83c0fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:19:53.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.334+0000 7fe83a7fc640 1 --2- 172.21.15.43:0/3110313662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c101420 0x7fe83c0fd3d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe830005940 tx=0x7fe8300040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:53.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.334+0000 7fe823fff640 1 -- 172.21.15.43:0/3110313662 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83003f070 con 0x7fe83c101420 2024-09-11T23:19:53.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.334+0000 7fe8416ab640 1 -- 172.21.15.43:0/3110313662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83c1a6bc0 con 0x7fe83c101420 2024-09-11T23:19:53.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.334+0000 7fe823fff640 1 -- 172.21.15.43:0/3110313662 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8300044c0 con 0x7fe83c101420 2024-09-11T23:19:53.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.334+0000 7fe823fff640 1 -- 172.21.15.43:0/3110313662 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830035070 con 0x7fe83c101420 2024-09-11T23:19:53.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.334+0000 7fe8416ab640 1 -- 172.21.15.43:0/3110313662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe83c1a70f0 con 0x7fe83c101420 2024-09-11T23:19:53.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.335+0000 7fe8416ab640 1 -- 172.21.15.43:0/3110313662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe83c100ee0 con 0x7fe83c101420 2024-09-11T23:19:53.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.336+0000 7fe823fff640 1 -- 172.21.15.43:0/3110313662 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe83003b460 con 0x7fe83c101420 2024-09-11T23:19:53.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.337+0000 7fe823fff640 1 --2- 172.21.15.43:0/3110313662 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe8240a2b80 0x7fe8240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:53.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.337+0000 7fe83affd640 1 --2- 172.21.15.43:0/3110313662 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe8240a2b80 0x7fe8240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:53.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.337+0000 7fe823fff640 1 -- 172.21.15.43:0/3110313662 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fe8300ef220 con 0x7fe83c101420 2024-09-11T23:19:53.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.338+0000 7fe83affd640 1 --2- 172.21.15.43:0/3110313662 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe8240a2b80 0x7fe8240a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe82c011490 tx=0x7fe82c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:53.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.340+0000 7fe823fff640 1 -- 172.21.15.43:0/3110313662 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8300bbfd0 con 0x7fe83c101420 2024-09-11T23:19:53.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.530+0000 7fe8416ab640 1 -- 172.21.15.43:0/3110313662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe83c10a000 con 0x7fe83c101420 2024-09-11T23:19:53.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.531+0000 7fe823fff640 1 -- 172.21.15.43:0/3110313662 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) ==== 84+0+65 (secure 0 0 0) 0x7fe8300c02d0 con 0x7fe83c101420 2024-09-11T23:19:53.531 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:19:53.531 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-09-11T23:19:53.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:19:53.531 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-09-11T23:19:53.531 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-09-11T23:19:53.531 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:19:53.532 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-09-11T23:19:53.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.533+0000 7fe8416ab640 1 -- 172.21.15.43:0/3110313662 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe8240a2b80 msgr2=0x7fe8240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:53.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.533+0000 7fe8416ab640 1 --2- 172.21.15.43:0/3110313662 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe8240a2b80 0x7fe8240a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe82c011490 tx=0x7fe82c005f90 comp rx=0 tx=0).stop 2024-09-11T23:19:53.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.533+0000 7fe8416ab640 1 -- 172.21.15.43:0/3110313662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c101420 msgr2=0x7fe83c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:53.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.533+0000 7fe8416ab640 1 --2- 172.21.15.43:0/3110313662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c101420 0x7fe83c0fd3d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe830005940 tx=0x7fe8300040c0 comp rx=0 tx=0).stop 2024-09-11T23:19:53.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.533+0000 7fe8416ab640 1 -- 172.21.15.43:0/3110313662 shutdown_connections 2024-09-11T23:19:53.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.533+0000 7fe8416ab640 1 --2- 172.21.15.43:0/3110313662 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe8240a2b80 0x7fe8240a5040 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:53.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.533+0000 7fe8416ab640 1 --2- 172.21.15.43:0/3110313662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe83c106a80 0x7fe83c0fd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:53.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.533+0000 7fe8416ab640 1 --2- 172.21.15.43:0/3110313662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c101420 0x7fe83c0fd3d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:53.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.533+0000 7fe8416ab640 1 --2- 172.21.15.43:0/3110313662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe83c100a80 0x7fe83c0fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:53.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.533+0000 7fe8416ab640 1 -- 172.21.15.43:0/3110313662 >> 172.21.15.43:0/3110313662 conn(0x7fe83c0fc7a0 msgr2=0x7fe83c06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:53.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.533+0000 7fe8416ab640 1 -- 172.21.15.43:0/3110313662 shutdown_connections 2024-09-11T23:19:53.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:53.533+0000 7fe8416ab640 1 -- 172.21.15.43:0/3110313662 wait complete. 2024-09-11T23:19:53.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:53 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:53.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:53 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:53.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:53 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:53.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:53 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:19:53.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:53 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:53.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:53 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:19:53.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:53 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3110313662' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T23:19:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:53 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:53 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:53 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:53 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:19:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:53 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:53 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:19:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:53 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3110313662' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T23:19:54.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:53 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:54.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:53 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:54.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:53 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:54.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:53 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:19:54.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:53 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:54.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:53 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:19:54.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:53 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3110313662' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T23:19:54.650 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T23:19:54.650 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T23:19:54.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:54 smithi043 ceph-mon[30950]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:54.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:54 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:54.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:54 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:54.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:54 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:54.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:54 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:55.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:54 smithi090 ceph-mon[35130]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:55.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:54 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:55.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:54 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:55.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:54 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:55.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:54 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:55.049 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:55.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:54 smithi097 ceph-mon[35562]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:55.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:54 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:55.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:54 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:55.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:54 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:55.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:54 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:55.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.575+0000 7f9141018640 1 -- 172.21.15.43:0/2863141844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f913c101420 msgr2=0x7f913c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:55.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.575+0000 7f9141018640 1 --2- 172.21.15.43:0/2863141844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f913c101420 0x7f913c10d950 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9134009cd0 tx=0x7f913402a8e0 comp rx=0 tx=0).stop 2024-09-11T23:19:55.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.575+0000 7f9141018640 1 -- 172.21.15.43:0/2863141844 shutdown_connections 2024-09-11T23:19:55.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.575+0000 7f9141018640 1 --2- 172.21.15.43:0/2863141844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f913c101420 0x7f913c10d950 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:55.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.575+0000 7f9141018640 1 --2- 172.21.15.43:0/2863141844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f913c100a80 0x7f913c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:55.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.575+0000 7f9141018640 1 --2- 172.21.15.43:0/2863141844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f913c106a80 0x7f913c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:55.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.575+0000 7f9141018640 1 -- 172.21.15.43:0/2863141844 >> 172.21.15.43:0/2863141844 conn(0x7f913c0fc7a0 msgr2=0x7f913c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:55.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.575+0000 7f9141018640 1 -- 172.21.15.43:0/2863141844 shutdown_connections 2024-09-11T23:19:55.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.575+0000 7f9141018640 1 -- 172.21.15.43:0/2863141844 wait complete. 2024-09-11T23:19:55.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.576+0000 7f9141018640 1 Processor -- start 2024-09-11T23:19:55.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.577+0000 7f9141018640 1 -- start start 2024-09-11T23:19:55.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.577+0000 7f9141018640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f913c100a80 0x7f913c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:55.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.577+0000 7f9141018640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f913c101420 0x7f913c199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:55.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.577+0000 7f9141018640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f913c106a80 0x7f913c1a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.577+0000 7f9141018640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f913c110ea0 con 0x7f913c101420 2024-09-11T23:19:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.577+0000 7f9141018640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f913c110d20 con 0x7f913c100a80 2024-09-11T23:19:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.577+0000 7f9141018640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f913c111020 con 0x7f913c106a80 2024-09-11T23:19:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.577+0000 7f913a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f913c101420 0x7f913c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.577+0000 7f913a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f913c101420 0x7f913c199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51462/0 (socket says 172.21.15.43:51462) 2024-09-11T23:19:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.577+0000 7f913a59c640 1 -- 172.21.15.43:0/1193046166 learned_addr learned my addr 172.21.15.43:0/1193046166 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:19:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.577+0000 7f913b59e640 1 --2- 172.21.15.43:0/1193046166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f913c106a80 0x7f913c1a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.577+0000 7f913ad9d640 1 --2- 172.21.15.43:0/1193046166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f913c100a80 0x7f913c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.577+0000 7f913a59c640 1 -- 172.21.15.43:0/1193046166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f913c106a80 msgr2=0x7f913c1a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.577+0000 7f913a59c640 1 --2- 172.21.15.43:0/1193046166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f913c106a80 0x7f913c1a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.577+0000 7f913a59c640 1 -- 172.21.15.43:0/1193046166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f913c100a80 msgr2=0x7f913c1992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.577+0000 7f913a59c640 1 --2- 172.21.15.43:0/1193046166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f913c100a80 0x7f913c1992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:55.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.577+0000 7f913a59c640 1 -- 172.21.15.43:0/1193046166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913c1a4890 con 0x7f913c101420 2024-09-11T23:19:55.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.577+0000 7f913b59e640 1 --2- 172.21.15.43:0/1193046166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f913c106a80 0x7f913c1a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:19:55.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.577+0000 7f913ad9d640 1 --2- 172.21.15.43:0/1193046166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f913c100a80 0x7f913c1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:19:55.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.577+0000 7f913a59c640 1 --2- 172.21.15.43:0/1193046166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f913c101420 0x7f913c199810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f913000d950 tx=0x7f913000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:55.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.578+0000 7f9123fff640 1 -- 172.21.15.43:0/1193046166 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130017070 con 0x7f913c101420 2024-09-11T23:19:55.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.578+0000 7f9141018640 1 -- 172.21.15.43:0/1193046166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f913c1a4a60 con 0x7f913c101420 2024-09-11T23:19:55.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.578+0000 7f9123fff640 1 -- 172.21.15.43:0/1193046166 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9130009c30 con 0x7f913c101420 2024-09-11T23:19:55.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.578+0000 7f9123fff640 1 -- 172.21.15.43:0/1193046166 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913000f6b0 con 0x7f913c101420 2024-09-11T23:19:55.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.578+0000 7f9141018640 1 -- 172.21.15.43:0/1193046166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f913c1a4fa0 con 0x7f913c101420 2024-09-11T23:19:55.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.579+0000 7f9141018640 1 -- 172.21.15.43:0/1193046166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f913c100ee0 con 0x7f913c101420 2024-09-11T23:19:55.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.580+0000 7f9123fff640 1 -- 172.21.15.43:0/1193046166 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f91300109d0 con 0x7f913c101420 2024-09-11T23:19:55.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.580+0000 7f9123fff640 1 --2- 172.21.15.43:0/1193046166 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f91240a2b80 0x7f91240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:55.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.581+0000 7f913ad9d640 1 --2- 172.21.15.43:0/1193046166 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f91240a2b80 0x7f91240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:55.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.581+0000 7f9123fff640 1 -- 172.21.15.43:0/1193046166 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f91300ce8c0 con 0x7f913c101420 2024-09-11T23:19:55.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.581+0000 7f913ad9d640 1 --2- 172.21.15.43:0/1193046166 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f91240a2b80 0x7f91240a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f912c006fd0 tx=0x7f912c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:55.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.585+0000 7f9123fff640 1 -- 172.21.15.43:0/1193046166 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9130009de0 con 0x7f913c101420 2024-09-11T23:19:55.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:55.765+0000 7f9141018640 1 -- 172.21.15.43:0/1193046166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f913c1a51d0 con 0x7f913c101420 2024-09-11T23:19:56.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:56.595+0000 7f9123fff640 1 -- 172.21.15.43:0/1193046166 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) ==== 138+0+0 (secure 0 0 0) 0x7f913009bcc0 con 0x7f913c101420 2024-09-11T23:19:56.597 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-09-11T23:19:56.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:56.598+0000 7f9141018640 1 -- 172.21.15.43:0/1193046166 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f91240a2b80 msgr2=0x7f91240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:56.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:56.598+0000 7f9141018640 1 --2- 172.21.15.43:0/1193046166 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f91240a2b80 0x7f91240a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f912c006fd0 tx=0x7f912c008040 comp rx=0 tx=0).stop 2024-09-11T23:19:56.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:56.598+0000 7f9141018640 1 -- 172.21.15.43:0/1193046166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f913c101420 msgr2=0x7f913c199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:56.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:56.598+0000 7f9141018640 1 --2- 172.21.15.43:0/1193046166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f913c101420 0x7f913c199810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f913000d950 tx=0x7f913000de10 comp rx=0 tx=0).stop 2024-09-11T23:19:56.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:56.599+0000 7f9141018640 1 -- 172.21.15.43:0/1193046166 shutdown_connections 2024-09-11T23:19:56.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:56.599+0000 7f9141018640 1 --2- 172.21.15.43:0/1193046166 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f91240a2b80 0x7f91240a5040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:56.599+0000 7f9141018640 1 --2- 172.21.15.43:0/1193046166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f913c106a80 0x7f913c1a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:56.599+0000 7f9141018640 1 --2- 172.21.15.43:0/1193046166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f913c101420 0x7f913c199810 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:56.599+0000 7f9141018640 1 --2- 172.21.15.43:0/1193046166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f913c100a80 0x7f913c1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:56.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:56.599+0000 7f9141018640 1 -- 172.21.15.43:0/1193046166 >> 172.21.15.43:0/1193046166 conn(0x7f913c0fc7a0 msgr2=0x7f913c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:56.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:56.599+0000 7f9141018640 1 -- 172.21.15.43:0/1193046166 shutdown_connections 2024-09-11T23:19:56.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:56.600+0000 7f9141018640 1 -- 172.21.15.43:0/1193046166 wait complete. 2024-09-11T23:19:56.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:56 smithi043 ceph-mon[30950]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:56.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:56 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1193046166' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T23:19:56.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:56 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:56.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:56 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:56.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:56 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:56.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:56 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:19:56.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:56 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:57.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:56 smithi090 ceph-mon[35130]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:57.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:56 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1193046166' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T23:19:57.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:56 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:57.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:56 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:57.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:56 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:57.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:56 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:19:57.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:56 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:57.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:56 smithi097 ceph-mon[35562]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:57.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:56 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1193046166' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T23:19:57.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:56 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:57.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:56 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:57.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:56 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:19:57.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:56 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:19:57.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:56 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:19:57.272 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T23:19:57.538 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:57.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:57 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1193046166' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T23:19:57.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:57 smithi043 ceph-mon[30950]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T23:19:58.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:57 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1193046166' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T23:19:58.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:57 smithi090 ceph-mon[35130]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T23:19:58.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:57 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1193046166' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T23:19:58.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:57 smithi097 ceph-mon[35562]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T23:19:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.086+0000 7f92ba42c640 1 -- 172.21.15.43:0/1897725683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b410c170 msgr2=0x7f92b410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.086+0000 7f92ba42c640 1 --2- 172.21.15.43:0/1897725683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b410c170 0x7f92b410e610 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009ca0 tx=0x7f92ac02a950 comp rx=0 tx=0).stop 2024-09-11T23:19:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.086+0000 7f92ba42c640 1 -- 172.21.15.43:0/1897725683 shutdown_connections 2024-09-11T23:19:58.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.086+0000 7f92ba42c640 1 --2- 172.21.15.43:0/1897725683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b410c170 0x7f92b410e610 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:58.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.086+0000 7f92ba42c640 1 --2- 172.21.15.43:0/1897725683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b4103660 0x7f92b410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:58.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.086+0000 7f92ba42c640 1 --2- 172.21.15.43:0/1897725683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92b4102d40 0x7f92b4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:58.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.086+0000 7f92ba42c640 1 -- 172.21.15.43:0/1897725683 >> 172.21.15.43:0/1897725683 conn(0x7f92b40fc7a0 msgr2=0x7f92b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:58.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.086+0000 7f92ba42c640 1 -- 172.21.15.43:0/1897725683 shutdown_connections 2024-09-11T23:19:58.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.086+0000 7f92ba42c640 1 -- 172.21.15.43:0/1897725683 wait complete. 2024-09-11T23:19:58.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.087+0000 7f92ba42c640 1 Processor -- start 2024-09-11T23:19:58.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.087+0000 7f92ba42c640 1 -- start start 2024-09-11T23:19:58.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.087+0000 7f92ba42c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b4102d40 0x7f92b4198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:58.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.087+0000 7f92ba42c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b4103660 0x7f92b4198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:58.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.087+0000 7f92ba42c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92b410c170 0x7f92b419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:58.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.087+0000 7f92ba42c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4112000 con 0x7f92b4102d40 2024-09-11T23:19:58.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.087+0000 7f92ba42c640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4111e80 con 0x7f92b410c170 2024-09-11T23:19:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.087+0000 7f92ba42c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4112180 con 0x7f92b4103660 2024-09-11T23:19:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.088+0000 7f92b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b4102d40 0x7f92b4198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.088+0000 7f92b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b4102d40 0x7f92b4198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51484/0 (socket says 172.21.15.43:51484) 2024-09-11T23:19:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.088+0000 7f92b3fff640 1 -- 172.21.15.43:0/3976639867 learned_addr learned my addr 172.21.15.43:0/3976639867 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:19:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.088+0000 7f92b89c9640 1 --2- 172.21.15.43:0/3976639867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92b410c170 0x7f92b419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.088+0000 7f92b37fe640 1 --2- 172.21.15.43:0/3976639867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b4103660 0x7f92b4198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.088+0000 7f92b3fff640 1 -- 172.21.15.43:0/3976639867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b4103660 msgr2=0x7f92b4198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.088+0000 7f92b3fff640 1 --2- 172.21.15.43:0/3976639867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b4103660 0x7f92b4198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.088+0000 7f92b3fff640 1 -- 172.21.15.43:0/3976639867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92b410c170 msgr2=0x7f92b419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.088+0000 7f92b3fff640 1 --2- 172.21.15.43:0/3976639867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92b410c170 0x7f92b419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.088+0000 7f92b3fff640 1 -- 172.21.15.43:0/3976639867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b419d3f0 con 0x7f92b4102d40 2024-09-11T23:19:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.088+0000 7f92b37fe640 1 --2- 172.21.15.43:0/3976639867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b4103660 0x7f92b4198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:19:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.088+0000 7f92b3fff640 1 --2- 172.21.15.43:0/3976639867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b4102d40 0x7f92b4198440 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f92a400d950 tx=0x7f92a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.088+0000 7f92b17fa640 1 -- 172.21.15.43:0/3976639867 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4017030 con 0x7f92b4102d40 2024-09-11T23:19:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.088+0000 7f92b17fa640 1 -- 172.21.15.43:0/3976639867 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92a4004020 con 0x7f92b4102d40 2024-09-11T23:19:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.088+0000 7f92ba42c640 1 -- 172.21.15.43:0/3976639867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b419d6e0 con 0x7f92b4102d40 2024-09-11T23:19:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.088+0000 7f92b17fa640 1 -- 172.21.15.43:0/3976639867 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a40047a0 con 0x7f92b4102d40 2024-09-11T23:19:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.088+0000 7f92ba42c640 1 -- 172.21.15.43:0/3976639867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b41a4fc0 con 0x7f92b4102d40 2024-09-11T23:19:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.089+0000 7f92ba42c640 1 -- 172.21.15.43:0/3976639867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b4103120 con 0x7f92b4102d40 2024-09-11T23:19:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.091+0000 7f92b17fa640 1 -- 172.21.15.43:0/3976639867 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f92a40041d0 con 0x7f92b4102d40 2024-09-11T23:19:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.091+0000 7f92b17fa640 1 --2- 172.21.15.43:0/3976639867 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f929c0a2c00 0x7f929c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:19:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.091+0000 7f92b37fe640 1 --2- 172.21.15.43:0/3976639867 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f929c0a2c00 0x7f929c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:19:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.091+0000 7f92b17fa640 1 -- 172.21.15.43:0/3976639867 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7497+0+0 (secure 0 0 0) 0x7f92a40ced10 con 0x7f92b4102d40 2024-09-11T23:19:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.092+0000 7f92b37fe640 1 --2- 172.21.15.43:0/3976639867 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f929c0a2c00 0x7f929c0a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f92a8006fd0 tx=0x7f92a8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:19:58.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.095+0000 7f92b17fa640 1 -- 172.21.15.43:0/3976639867 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f92a409c020 con 0x7f92b4102d40 2024-09-11T23:19:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.271+0000 7f92ba42c640 1 -- 172.21.15.43:0/3976639867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f92b41a52c0 con 0x7f92b4102d40 2024-09-11T23:19:58.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.712+0000 7f92b17fa640 1 -- 172.21.15.43:0/3976639867 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) ==== 170+0+0 (secure 0 0 0) 0x7f92a409b920 con 0x7f92b4102d40 2024-09-11T23:19:58.714 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-09-11T23:19:58.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.715+0000 7f92ba42c640 1 -- 172.21.15.43:0/3976639867 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f929c0a2c00 msgr2=0x7f929c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:58.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.715+0000 7f92ba42c640 1 --2- 172.21.15.43:0/3976639867 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f929c0a2c00 0x7f929c0a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f92a8006fd0 tx=0x7f92a8008040 comp rx=0 tx=0).stop 2024-09-11T23:19:58.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.715+0000 7f92ba42c640 1 -- 172.21.15.43:0/3976639867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b4102d40 msgr2=0x7f92b4198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:19:58.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.715+0000 7f92ba42c640 1 --2- 172.21.15.43:0/3976639867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b4102d40 0x7f92b4198440 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f92a400d950 tx=0x7f92a400de10 comp rx=0 tx=0).stop 2024-09-11T23:19:58.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.715+0000 7f92ba42c640 1 -- 172.21.15.43:0/3976639867 shutdown_connections 2024-09-11T23:19:58.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.715+0000 7f92ba42c640 1 --2- 172.21.15.43:0/3976639867 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f929c0a2c00 0x7f929c0a50c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:58.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.715+0000 7f92ba42c640 1 --2- 172.21.15.43:0/3976639867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92b410c170 0x7f92b419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:58.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.715+0000 7f92ba42c640 1 --2- 172.21.15.43:0/3976639867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b4103660 0x7f92b4198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:58.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.715+0000 7f92ba42c640 1 --2- 172.21.15.43:0/3976639867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b4102d40 0x7f92b4198440 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:19:58.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.715+0000 7f92ba42c640 1 -- 172.21.15.43:0/3976639867 >> 172.21.15.43:0/3976639867 conn(0x7f92b40fc7a0 msgr2=0x7f92b40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:19:58.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.715+0000 7f92ba42c640 1 -- 172.21.15.43:0/3976639867 shutdown_connections 2024-09-11T23:19:58.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:19:58.715+0000 7f92ba42c640 1 -- 172.21.15.43:0/3976639867 wait complete. 2024-09-11T23:19:59.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:58 smithi090 ceph-mon[35130]: pgmap v276: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:59.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:58 smithi090 ceph-mon[35130]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T23:19:59.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:58 smithi090 ceph-mon[35130]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T23:19:59.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:58 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3976639867' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T23:19:59.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:58 smithi097 ceph-mon[35562]: pgmap v276: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:59.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:58 smithi097 ceph-mon[35562]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T23:19:59.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:58 smithi097 ceph-mon[35562]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T23:19:59.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:58 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3976639867' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T23:19:59.101 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:58 smithi043 ceph-mon[30950]: pgmap v276: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:59.101 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:58 smithi043 ceph-mon[30950]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T23:19:59.101 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:58 smithi043 ceph-mon[30950]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T23:19:59.102 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:58 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3976639867' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T23:19:59.442 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T23:19:59.701 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:19:59.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:59 smithi043 ceph-mon[30950]: pgmap v278: 33 pgs: 3 creating+peering, 18 unknown, 12 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:19:59.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:59 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3976639867' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T23:19:59.998 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:19:59 smithi043 ceph-mon[30950]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T23:20:00.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:59 smithi090 ceph-mon[35130]: pgmap v278: 33 pgs: 3 creating+peering, 18 unknown, 12 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:20:00.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:59 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3976639867' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T23:20:00.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:19:59 smithi090 ceph-mon[35130]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T23:20:00.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:59 smithi097 ceph-mon[35562]: pgmap v278: 33 pgs: 3 creating+peering, 18 unknown, 12 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:20:00.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:59 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3976639867' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T23:20:00.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:19:59 smithi097 ceph-mon[35562]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T23:20:00.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.242+0000 7f4b56087640 1 -- 172.21.15.43:0/1374760088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b50102790 msgr2=0x7f4b501067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:00.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.242+0000 7f4b56087640 1 --2- 172.21.15.43:0/1374760088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b50102790 0x7f4b501067e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009900 tx=0x7f4b3802a410 comp rx=0 tx=0).stop 2024-09-11T23:20:00.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.242+0000 7f4b56087640 1 -- 172.21.15.43:0/1374760088 shutdown_connections 2024-09-11T23:20:00.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.242+0000 7f4b56087640 1 --2- 172.21.15.43:0/1374760088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b50106f10 0x7f4b5010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:00.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.242+0000 7f4b56087640 1 --2- 172.21.15.43:0/1374760088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b50102790 0x7f4b501067e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:00.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.242+0000 7f4b56087640 1 --2- 172.21.15.43:0/1374760088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b50101de0 0x7f4b501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:00.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.242+0000 7f4b56087640 1 -- 172.21.15.43:0/1374760088 >> 172.21.15.43:0/1374760088 conn(0x7f4b500fd8c0 msgr2=0x7f4b500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:00.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.242+0000 7f4b56087640 1 -- 172.21.15.43:0/1374760088 shutdown_connections 2024-09-11T23:20:00.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.242+0000 7f4b56087640 1 -- 172.21.15.43:0/1374760088 wait complete. 2024-09-11T23:20:00.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.242+0000 7f4b56087640 1 Processor -- start 2024-09-11T23:20:00.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.242+0000 7f4b56087640 1 -- start start 2024-09-11T23:20:00.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.243+0000 7f4b56087640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b50101de0 0x7f4b501985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:00.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.243+0000 7f4b56087640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b50102790 0x7f4b50198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:00.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.243+0000 7f4b56087640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b50106f10 0x7f4b5019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:00.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.243+0000 7f4b56087640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4b50112010 con 0x7f4b50106f10 2024-09-11T23:20:00.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.243+0000 7f4b56087640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4b50111e90 con 0x7f4b50101de0 2024-09-11T23:20:00.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.243+0000 7f4b56087640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4b50112190 con 0x7f4b50102790 2024-09-11T23:20:00.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.243+0000 7f4b4ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b50106f10 0x7f4b5019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:00.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.243+0000 7f4b4ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b50106f10 0x7f4b5019ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39080/0 (socket says 172.21.15.43:39080) 2024-09-11T23:20:00.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.243+0000 7f4b4ffff640 1 -- 172.21.15.43:0/4080288518 learned_addr learned my addr 172.21.15.43:0/4080288518 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:00.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.243+0000 7f4b4f7fe640 1 --2- 172.21.15.43:0/4080288518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b50101de0 0x7f4b501985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:00.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.243+0000 7f4b46ffd640 1 --2- 172.21.15.43:0/4080288518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b50102790 0x7f4b50198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:00.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.243+0000 7f4b4ffff640 1 -- 172.21.15.43:0/4080288518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b50102790 msgr2=0x7f4b50198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:00.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.243+0000 7f4b4ffff640 1 --2- 172.21.15.43:0/4080288518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b50102790 0x7f4b50198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:00.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.243+0000 7f4b4ffff640 1 -- 172.21.15.43:0/4080288518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b50101de0 msgr2=0x7f4b501985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:00.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.243+0000 7f4b4ffff640 1 --2- 172.21.15.43:0/4080288518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b50101de0 0x7f4b501985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:00.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.243+0000 7f4b4ffff640 1 -- 172.21.15.43:0/4080288518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5019d560 con 0x7f4b50106f10 2024-09-11T23:20:00.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.243+0000 7f4b4ffff640 1 --2- 172.21.15.43:0/4080288518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b50106f10 0x7f4b5019ce80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800b970 tx=0x7f4b4800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:00.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.244+0000 7f4b4d7fa640 1 -- 172.21.15.43:0/4080288518 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48019070 con 0x7f4b50106f10 2024-09-11T23:20:00.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.244+0000 7f4b4d7fa640 1 -- 172.21.15.43:0/4080288518 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b48014440 con 0x7f4b50106f10 2024-09-11T23:20:00.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.244+0000 7f4b4d7fa640 1 -- 172.21.15.43:0/4080288518 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b480075d0 con 0x7f4b50106f10 2024-09-11T23:20:00.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.244+0000 7f4b56087640 1 -- 172.21.15.43:0/4080288518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5019d850 con 0x7f4b50106f10 2024-09-11T23:20:00.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.244+0000 7f4b56087640 1 -- 172.21.15.43:0/4080288518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b501a5130 con 0x7f4b50106f10 2024-09-11T23:20:00.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.245+0000 7f4b56087640 1 -- 172.21.15.43:0/4080288518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b50102240 con 0x7f4b50106f10 2024-09-11T23:20:00.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.246+0000 7f4b4d7fa640 1 -- 172.21.15.43:0/4080288518 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4b4801c070 con 0x7f4b50106f10 2024-09-11T23:20:00.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.247+0000 7f4b4d7fa640 1 --2- 172.21.15.43:0/4080288518 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f4b3c0a2c00 0x7f4b3c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:00.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.247+0000 7f4b4f7fe640 1 --2- 172.21.15.43:0/4080288518 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f4b3c0a2c00 0x7f4b3c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:00.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.247+0000 7f4b4d7fa640 1 -- 172.21.15.43:0/4080288518 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7879+0+0 (secure 0 0 0) 0x7f4b480ce9d0 con 0x7f4b50106f10 2024-09-11T23:20:00.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.247+0000 7f4b4f7fe640 1 --2- 172.21.15.43:0/4080288518 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f4b3c0a2c00 0x7f4b3c0a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009990 tx=0x7f4b40009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:00.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.251+0000 7f4b4d7fa640 1 -- 172.21.15.43:0/4080288518 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4b4809ba40 con 0x7f4b50106f10 2024-09-11T23:20:00.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.463+0000 7f4b56087640 1 -- 172.21.15.43:0/4080288518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4b50109b60 con 0x7f4b50106f10 2024-09-11T23:20:00.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.731+0000 7f4b4d7fa640 1 -- 172.21.15.43:0/4080288518 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f4b4809b340 con 0x7f4b50106f10 2024-09-11T23:20:00.733 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 2 and data pool 3 2024-09-11T23:20:00.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.734+0000 7f4b56087640 1 -- 172.21.15.43:0/4080288518 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f4b3c0a2c00 msgr2=0x7f4b3c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:00.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.734+0000 7f4b56087640 1 --2- 172.21.15.43:0/4080288518 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f4b3c0a2c00 0x7f4b3c0a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009990 tx=0x7f4b40009110 comp rx=0 tx=0).stop 2024-09-11T23:20:00.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.734+0000 7f4b56087640 1 -- 172.21.15.43:0/4080288518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b50106f10 msgr2=0x7f4b5019ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:00.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.734+0000 7f4b56087640 1 --2- 172.21.15.43:0/4080288518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b50106f10 0x7f4b5019ce80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800b970 tx=0x7f4b4800bd70 comp rx=0 tx=0).stop 2024-09-11T23:20:00.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.734+0000 7f4b56087640 1 -- 172.21.15.43:0/4080288518 shutdown_connections 2024-09-11T23:20:00.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.734+0000 7f4b56087640 1 --2- 172.21.15.43:0/4080288518 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f4b3c0a2c00 0x7f4b3c0a50c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:00.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.734+0000 7f4b56087640 1 --2- 172.21.15.43:0/4080288518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b50106f10 0x7f4b5019ce80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:00.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.734+0000 7f4b56087640 1 --2- 172.21.15.43:0/4080288518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b50102790 0x7f4b50198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:00.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.734+0000 7f4b56087640 1 --2- 172.21.15.43:0/4080288518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b50101de0 0x7f4b501985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:00.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.734+0000 7f4b56087640 1 -- 172.21.15.43:0/4080288518 >> 172.21.15.43:0/4080288518 conn(0x7f4b500fd8c0 msgr2=0x7f4b500fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:00.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.734+0000 7f4b56087640 1 -- 172.21.15.43:0/4080288518 shutdown_connections 2024-09-11T23:20:00.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:00.734+0000 7f4b56087640 1 -- 172.21.15.43:0/4080288518 wait complete. 2024-09-11T23:20:01.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:00 smithi090 ceph-mon[35130]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T23:20:01.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:00 smithi090 ceph-mon[35130]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-11T23:20:01.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:00 smithi090 ceph-mon[35130]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-11T23:20:01.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:00 smithi090 ceph-mon[35130]: application not enabled on pool 'cephfs_metadata' 2024-09-11T23:20:01.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:00 smithi090 ceph-mon[35130]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-11T23:20:01.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:00 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/4080288518' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T23:20:01.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:00 smithi090 ceph-mon[35130]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T23:20:01.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:00 smithi097 ceph-mon[35562]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T23:20:01.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:00 smithi097 ceph-mon[35562]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-11T23:20:01.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:00 smithi097 ceph-mon[35562]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-11T23:20:01.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:00 smithi097 ceph-mon[35562]: application not enabled on pool 'cephfs_metadata' 2024-09-11T23:20:01.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:00 smithi097 ceph-mon[35562]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-11T23:20:01.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:00 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/4080288518' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T23:20:01.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:00 smithi097 ceph-mon[35562]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T23:20:01.060 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:00 smithi043 ceph-mon[30950]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T23:20:01.060 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:00 smithi043 ceph-mon[30950]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-11T23:20:01.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:00 smithi043 ceph-mon[30950]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-11T23:20:01.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:00 smithi043 ceph-mon[30950]: application not enabled on pool 'cephfs_metadata' 2024-09-11T23:20:01.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:00 smithi043 ceph-mon[30950]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-11T23:20:01.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:00 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/4080288518' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T23:20:01.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:00 smithi043 ceph-mon[30950]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T23:20:01.415 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:20:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:02.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:01 smithi090 ceph-mon[35130]: pgmap v281: 97 pgs: 22 creating+peering, 54 unknown, 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:20:02.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:01 smithi090 ceph-mon[35130]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T23:20:02.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:01 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/4080288518' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T23:20:02.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:01 smithi090 ceph-mon[35130]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T23:20:02.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:01 smithi090 ceph-mon[35130]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T23:20:02.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:01 smithi097 ceph-mon[35562]: pgmap v281: 97 pgs: 22 creating+peering, 54 unknown, 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:20:02.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:01 smithi097 ceph-mon[35562]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T23:20:02.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:01 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/4080288518' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T23:20:02.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:01 smithi097 ceph-mon[35562]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T23:20:02.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:01 smithi097 ceph-mon[35562]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T23:20:02.073 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:01 smithi043 ceph-mon[30950]: pgmap v281: 97 pgs: 22 creating+peering, 54 unknown, 21 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:20:02.073 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:01 smithi043 ceph-mon[30950]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T23:20:02.073 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:01 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/4080288518' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T23:20:02.073 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:01 smithi043 ceph-mon[30950]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T23:20:02.073 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:01 smithi043 ceph-mon[30950]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T23:20:02.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.154+0000 7fd10b2f8640 1 -- 172.21.15.43:0/3424680129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd104103740 msgr2=0x7fd104107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:02.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.154+0000 7fd10b2f8640 1 --2- 172.21.15.43:0/3424680129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd104103740 0x7fd104107b80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009930 tx=0x7fd0f402a420 comp rx=0 tx=0).stop 2024-09-11T23:20:02.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.154+0000 7fd10b2f8640 1 -- 172.21.15.43:0/3424680129 shutdown_connections 2024-09-11T23:20:02.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.154+0000 7fd10b2f8640 1 --2- 172.21.15.43:0/3424680129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd104108290 0x7fd10410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:02.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.154+0000 7fd10b2f8640 1 --2- 172.21.15.43:0/3424680129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd104103740 0x7fd104107b80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:02.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.154+0000 7fd10b2f8640 1 --2- 172.21.15.43:0/3424680129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd104102d90 0x7fd104103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:02.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.154+0000 7fd10b2f8640 1 -- 172.21.15.43:0/3424680129 >> 172.21.15.43:0/3424680129 conn(0x7fd1040fc7d0 msgr2=0x7fd1040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:02.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.154+0000 7fd10b2f8640 1 -- 172.21.15.43:0/3424680129 shutdown_connections 2024-09-11T23:20:02.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.154+0000 7fd10b2f8640 1 -- 172.21.15.43:0/3424680129 wait complete. 2024-09-11T23:20:02.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.155+0000 7fd10b2f8640 1 Processor -- start 2024-09-11T23:20:02.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.155+0000 7fd10b2f8640 1 -- start start 2024-09-11T23:20:02.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.155+0000 7fd10b2f8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd104102d90 0x7fd10419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:02.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.155+0000 7fd10b2f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd104103740 0x7fd10419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:02.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.155+0000 7fd10b2f8640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd104108290 0x7fd1041985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:02.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.155+0000 7fd10b2f8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd104112130 con 0x7fd104103740 2024-09-11T23:20:02.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.155+0000 7fd10b2f8640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd104111fb0 con 0x7fd104108290 2024-09-11T23:20:02.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.155+0000 7fd10b2f8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041122b0 con 0x7fd104102d90 2024-09-11T23:20:02.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.155+0000 7fd108893640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd104103740 0x7fd10419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:02.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.156+0000 7fd108893640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd104103740 0x7fd10419e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39100/0 (socket says 172.21.15.43:39100) 2024-09-11T23:20:02.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.156+0000 7fd108893640 1 -- 172.21.15.43:0/1571785673 learned_addr learned my addr 172.21.15.43:0/1571785673 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:02.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.156+0000 7fd109895640 1 --2- 172.21.15.43:0/1571785673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd104108290 0x7fd1041985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:02.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.156+0000 7fd109094640 1 --2- 172.21.15.43:0/1571785673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd104102d90 0x7fd10419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:02.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.156+0000 7fd108893640 1 -- 172.21.15.43:0/1571785673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd104102d90 msgr2=0x7fd10419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:02.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.156+0000 7fd108893640 1 --2- 172.21.15.43:0/1571785673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd104102d90 0x7fd10419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:02.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.156+0000 7fd108893640 1 -- 172.21.15.43:0/1571785673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd104108290 msgr2=0x7fd1041985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:02.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.156+0000 7fd108893640 1 --2- 172.21.15.43:0/1571785673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd104108290 0x7fd1041985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:02.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.156+0000 7fd108893640 1 -- 172.21.15.43:0/1571785673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd104198ce0 con 0x7fd104103740 2024-09-11T23:20:02.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.156+0000 7fd108893640 1 --2- 172.21.15.43:0/1571785673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd104103740 0x7fd10419e930 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009230 tx=0x7fd0f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:02.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.156+0000 7fd0fa7fc640 1 -- 172.21.15.43:0/1571785673 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f403f030 con 0x7fd104103740 2024-09-11T23:20:02.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.156+0000 7fd10b2f8640 1 -- 172.21.15.43:0/1571785673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd104198f70 con 0x7fd104103740 2024-09-11T23:20:02.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.156+0000 7fd0fa7fc640 1 -- 172.21.15.43:0/1571785673 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f4003ce0 con 0x7fd104103740 2024-09-11T23:20:02.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.156+0000 7fd0fa7fc640 1 -- 172.21.15.43:0/1571785673 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4035070 con 0x7fd104103740 2024-09-11T23:20:02.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.156+0000 7fd10b2f8640 1 -- 172.21.15.43:0/1571785673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1041a52c0 con 0x7fd104103740 2024-09-11T23:20:02.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.157+0000 7fd10b2f8640 1 -- 172.21.15.43:0/1571785673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd10406a7f0 con 0x7fd104103740 2024-09-11T23:20:02.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.159+0000 7fd0fa7fc640 1 -- 172.21.15.43:0/1571785673 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd0f4042030 con 0x7fd104103740 2024-09-11T23:20:02.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.159+0000 7fd0fa7fc640 1 --2- 172.21.15.43:0/1571785673 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd0f00a2b80 0x7fd0f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:02.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.159+0000 7fd109094640 1 --2- 172.21.15.43:0/1571785673 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd0f00a2b80 0x7fd0f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:02.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.159+0000 7fd0fa7fc640 1 -- 172.21.15.43:0/1571785673 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fd0f40ef8a0 con 0x7fd104103740 2024-09-11T23:20:02.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.160+0000 7fd109094640 1 --2- 172.21.15.43:0/1571785673 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd0f00a2b80 0x7fd0f00a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd100009890 tx=0x7fd100008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:02.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.162+0000 7fd0fa7fc640 1 -- 172.21.15.43:0/1571785673 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd0f40bc950 con 0x7fd104103740 2024-09-11T23:20:02.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.338+0000 7fd10b2f8640 1 -- 172.21.15.43:0/1571785673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd104199ae0 con 0x7fd104103740 2024-09-11T23:20:02.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.340+0000 7fd0fa7fc640 1 -- 172.21.15.43:0/1571785673 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22145 (secure 0 0 0) 0x7fd0f40bc250 con 0x7fd104103740 2024-09-11T23:20:02.341 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:20:02.341 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":82,"fsid":"25c0cd84-7093-11ef-bcea-c7b262605968","created":"2024-09-11T23:12:03.504406+0000","modified":"2024-09-11T23:20:00.465617+0000","last_up_change":"2024-09-11T23:19:07.797873+0000","last_in_change":"2024-09-11T23:18:44.595176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:15:17.058713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:19:56.221406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:19:58.705875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24076596-3909-4c87-85f1-cbf168070dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6803","nonce":43839997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6805","nonce":43839997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6809","nonce":43839997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6807","nonce":43839997}]},"public_addr":"172.21.15.43:6803/43839997","cluster_addr":"172.21.15.43:6805/43839997","heartbeat_back_addr":"172.21.15.43:6809/43839997","heartbeat_front_addr":"172.21.15.43:6807/43839997","state":["exists","up"]},{"osd":1,"uuid":"a0472a57-243c-489d-a247-972ee10a71f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6801","nonce":1825149246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6803","nonce":1825149246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6807","nonce":1825149246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6805","nonce":1825149246}]},"public_addr":"172.21.15.90:6801/1825149246","cluster_addr":"172.21.15.90:6803/1825149246","heartbeat_back_addr":"172.21.15.90:6807/1825149246","heartbeat_front_addr":"172.21.15.90:6805/1825149246","state":["exists","up"]},{"osd":2,"uuid":"40366536-0fcf-45a0-b8e3-49178401dc34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6801","nonce":1456846443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6803","nonce":1456846443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6807","nonce":1456846443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6805","nonce":1456846443}]},"public_addr":"172.21.15.97:6801/1456846443","cluster_addr":"172.21.15.97:6803/1456846443","heartbeat_back_addr":"172.21.15.97:6807/1456846443","heartbeat_front_addr":"172.21.15.97:6805/1456846443","state":["exists","up"]},{"osd":3,"uuid":"3062133e-000b-4421-9d53-ac434b9ed9b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6811","nonce":1962837804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6813","nonce":1962837804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6817","nonce":1962837804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6815","nonce":1962837804}]},"public_addr":"172.21.15.43:6811/1962837804","cluster_addr":"172.21.15.43:6813/1962837804","heartbeat_back_addr":"172.21.15.43:6817/1962837804","heartbeat_front_addr":"172.21.15.43:6815/1962837804","state":["exists","up"]},{"osd":4,"uuid":"2e73fa4b-f22c-4319-828d-87b91b9f19f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6809","nonce":181614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6811","nonce":181614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6815","nonce":181614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6813","nonce":181614173}]},"public_addr":"172.21.15.90:6809/181614173","cluster_addr":"172.21.15.90:6811/181614173","heartbeat_back_addr":"172.21.15.90:6815/181614173","heartbeat_front_addr":"172.21.15.90:6813/181614173","state":["exists","up"]},{"osd":5,"uuid":"1e1ad4f9-e724-440f-b515-3ec1ad97d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6809","nonce":1540039543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6811","nonce":1540039543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6815","nonce":1540039543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6813","nonce":1540039543}]},"public_addr":"172.21.15.97:6809/1540039543","cluster_addr":"172.21.15.97:6811/1540039543","heartbeat_back_addr":"172.21.15.97:6815/1540039543","heartbeat_front_addr":"172.21.15.97:6813/1540039543","state":["exists","up"]},{"osd":6,"uuid":"d34df8df-f40d-4159-8b29-5401b1b0e431","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6819","nonce":3785677637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6821","nonce":3785677637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6825","nonce":3785677637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6823","nonce":3785677637}]},"public_addr":"172.21.15.43:6819/3785677637","cluster_addr":"172.21.15.43:6821/3785677637","heartbeat_back_addr":"172.21.15.43:6825/3785677637","heartbeat_front_addr":"172.21.15.43:6823/3785677637","state":["exists","up"]},{"osd":7,"uuid":"b2e01078-528b-49fd-b000-9f584f784b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6817","nonce":236874962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6819","nonce":236874962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6823","nonce":236874962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6821","nonce":236874962}]},"public_addr":"172.21.15.90:6817/236874962","cluster_addr":"172.21.15.90:6819/236874962","heartbeat_back_addr":"172.21.15.90:6823/236874962","heartbeat_front_addr":"172.21.15.90:6821/236874962","state":["exists","up"]},{"osd":8,"uuid":"abef8f27-bf9b-4d63-bfbe-f967fc3d8b17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6817","nonce":1707327301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6819","nonce":1707327301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6823","nonce":1707327301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6821","nonce":1707327301}]},"public_addr":"172.21.15.97:6817/1707327301","cluster_addr":"172.21.15.97:6819/1707327301","heartbeat_back_addr":"172.21.15.97:6823/1707327301","heartbeat_front_addr":"172.21.15.97:6821/1707327301","state":["exists","up"]},{"osd":9,"uuid":"190ab8b2-a835-47e6-9b39-16a58331c58a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6827","nonce":3170306011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6829","nonce":3170306011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6833","nonce":3170306011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6831","nonce":3170306011}]},"public_addr":"172.21.15.43:6827/3170306011","cluster_addr":"172.21.15.43:6829/3170306011","heartbeat_back_addr":"172.21.15.43:6833/3170306011","heartbeat_front_addr":"172.21.15.43:6831/3170306011","state":["exists","up"]},{"osd":10,"uuid":"b32ab5fe-466f-4c8e-8696-cd9f820a3263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6825","nonce":59487041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6827","nonce":59487041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6831","nonce":59487041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6829","nonce":59487041}]},"public_addr":"172.21.15.90:6825/59487041","cluster_addr":"172.21.15.90:6827/59487041","heartbeat_back_addr":"172.21.15.90:6831/59487041","heartbeat_front_addr":"172.21.15.90:6829/59487041","state":["exists","up"]},{"osd":11,"uuid":"d0faa589-21c8-4a9f-8a73-813aafe6d33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6825","nonce":194699315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6827","nonce":194699315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6831","nonce":194699315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6829","nonce":194699315}]},"public_addr":"172.21.15.97:6825/194699315","cluster_addr":"172.21.15.97:6827/194699315","heartbeat_back_addr":"172.21.15.97:6831/194699315","heartbeat_front_addr":"172.21.15.97:6829/194699315","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:23.381247+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:48.843190+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:13.538979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:40.136983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:05.513790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:31.247090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:56.635221+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:22.667327+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:47.910422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:14.353914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:40.584503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:19:06.476766+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/76575371":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/4006124758":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/120495444":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6800/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:6800/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/206044413":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1512201905":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1375379108":"2024-09-12T23:12:35.756598+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:20:02.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.342+0000 7fd10b2f8640 1 -- 172.21.15.43:0/1571785673 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd0f00a2b80 msgr2=0x7fd0f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:02.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.342+0000 7fd10b2f8640 1 --2- 172.21.15.43:0/1571785673 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd0f00a2b80 0x7fd0f00a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd100009890 tx=0x7fd100008040 comp rx=0 tx=0).stop 2024-09-11T23:20:02.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.342+0000 7fd10b2f8640 1 -- 172.21.15.43:0/1571785673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd104103740 msgr2=0x7fd10419e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:02.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.342+0000 7fd10b2f8640 1 --2- 172.21.15.43:0/1571785673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd104103740 0x7fd10419e930 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009230 tx=0x7fd0f40042f0 comp rx=0 tx=0).stop 2024-09-11T23:20:02.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.343+0000 7fd10b2f8640 1 -- 172.21.15.43:0/1571785673 shutdown_connections 2024-09-11T23:20:02.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.343+0000 7fd10b2f8640 1 --2- 172.21.15.43:0/1571785673 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd0f00a2b80 0x7fd0f00a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:02.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.343+0000 7fd10b2f8640 1 --2- 172.21.15.43:0/1571785673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd104108290 0x7fd1041985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:02.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.343+0000 7fd10b2f8640 1 --2- 172.21.15.43:0/1571785673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd104103740 0x7fd10419e930 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:02.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.343+0000 7fd10b2f8640 1 --2- 172.21.15.43:0/1571785673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd104102d90 0x7fd10419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:02.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.343+0000 7fd10b2f8640 1 -- 172.21.15.43:0/1571785673 >> 172.21.15.43:0/1571785673 conn(0x7fd1040fc7d0 msgr2=0x7fd104105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:02.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.343+0000 7fd10b2f8640 1 -- 172.21.15.43:0/1571785673 shutdown_connections 2024-09-11T23:20:02.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:02.343+0000 7fd10b2f8640 1 -- 172.21.15.43:0/1571785673 wait complete. 2024-09-11T23:20:03.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:02 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] up:active 2024-09-11T23:20:03.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:02 smithi090 ceph-mon[35130]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:20:03.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:02 smithi090 ceph-mon[35130]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:20:03.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:02 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1571785673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:20:03.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:02 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] up:active 2024-09-11T23:20:03.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:02 smithi097 ceph-mon[35562]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:20:03.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:02 smithi097 ceph-mon[35562]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:20:03.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:02 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1571785673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:20:03.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:02 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] up:active 2024-09-11T23:20:03.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:02 smithi043 ceph-mon[30950]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:20:03.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:02 smithi043 ceph-mon[30950]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:20:03.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:02 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1571785673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:20:03.161 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:20:03.421 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:03.823 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:03 smithi043 ceph-mon[30950]: pgmap v283: 97 pgs: 19 creating+peering, 13 unknown, 65 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 2 op/s 2024-09-11T23:20:03.824 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:03 smithi043 ceph-mon[30950]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T23:20:03.824 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:03 smithi043 ceph-mon[30950]: Cluster is now healthy 2024-09-11T23:20:03.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.904+0000 7f695d19e640 1 -- 172.21.15.43:0/1942924666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6958102380 msgr2=0x7f695810e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.904+0000 7f695d19e640 1 --2- 172.21.15.43:0/1942924666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6958102380 0x7f695810e9c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6950009cd0 tx=0x7f695002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.904+0000 7f695d19e640 1 -- 172.21.15.43:0/1942924666 shutdown_connections 2024-09-11T23:20:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.904+0000 7f695d19e640 1 --2- 172.21.15.43:0/1942924666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6958102380 0x7f695810e9c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.904+0000 7f695d19e640 1 --2- 172.21.15.43:0/1942924666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69581019e0 0x7f6958101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.904+0000 7f695d19e640 1 --2- 172.21.15.43:0/1942924666 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6958107af0 0x7f6958107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.904+0000 7f695d19e640 1 -- 172.21.15.43:0/1942924666 >> 172.21.15.43:0/1942924666 conn(0x7f69580fd700 msgr2=0x7f69580ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.904+0000 7f695d19e640 1 -- 172.21.15.43:0/1942924666 shutdown_connections 2024-09-11T23:20:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.904+0000 7f695d19e640 1 -- 172.21.15.43:0/1942924666 wait complete. 2024-09-11T23:20:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.905+0000 7f695d19e640 1 Processor -- start 2024-09-11T23:20:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.905+0000 7f695d19e640 1 -- start start 2024-09-11T23:20:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.906+0000 7f695d19e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69581019e0 0x7f6958198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.906+0000 7f695d19e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6958102380 0x7f6958198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.906+0000 7f695d19e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6958107af0 0x7f695819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.906+0000 7f695d19e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6958111ef0 con 0x7f6958107af0 2024-09-11T23:20:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.906+0000 7f695d19e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6958111d70 con 0x7f69581019e0 2024-09-11T23:20:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.906+0000 7f695d19e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6958112070 con 0x7f6958102380 2024-09-11T23:20:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.906+0000 7f695c99d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6958107af0 0x7f695819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.906+0000 7f695c99d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6958107af0 0x7f695819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39122/0 (socket says 172.21.15.43:39122) 2024-09-11T23:20:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.906+0000 7f695c99d640 1 -- 172.21.15.43:0/3645469677 learned_addr learned my addr 172.21.15.43:0/3645469677 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.906+0000 7f6957fff640 1 --2- 172.21.15.43:0/3645469677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69581019e0 0x7f6958198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.906+0000 7f69577fe640 1 --2- 172.21.15.43:0/3645469677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6958102380 0x7f6958198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.906+0000 7f695c99d640 1 -- 172.21.15.43:0/3645469677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6958102380 msgr2=0x7f6958198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.906+0000 7f695c99d640 1 --2- 172.21.15.43:0/3645469677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6958102380 0x7f6958198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.906+0000 7f695c99d640 1 -- 172.21.15.43:0/3645469677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69581019e0 msgr2=0x7f6958198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:03.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.906+0000 7f695c99d640 1 --2- 172.21.15.43:0/3645469677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69581019e0 0x7f6958198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.906+0000 7f695c99d640 1 -- 172.21.15.43:0/3645469677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695819d2e0 con 0x7f6958107af0 2024-09-11T23:20:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.906+0000 7f69577fe640 1 --2- 172.21.15.43:0/3645469677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6958102380 0x7f6958198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.906+0000 7f695c99d640 1 --2- 172.21.15.43:0/3645469677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6958107af0 0x7f695819cc00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6950005cc0 tx=0x7f6950003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.907+0000 7f69557fa640 1 -- 172.21.15.43:0/3645469677 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950004040 con 0x7f6958107af0 2024-09-11T23:20:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.907+0000 7f69557fa640 1 -- 172.21.15.43:0/3645469677 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69500046c0 con 0x7f6958107af0 2024-09-11T23:20:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.907+0000 7f695d19e640 1 -- 172.21.15.43:0/3645469677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f695819d570 con 0x7f6958107af0 2024-09-11T23:20:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.907+0000 7f69557fa640 1 -- 172.21.15.43:0/3645469677 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950046480 con 0x7f6958107af0 2024-09-11T23:20:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.907+0000 7f695d19e640 1 -- 172.21.15.43:0/3645469677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69581a4eb0 con 0x7f6958107af0 2024-09-11T23:20:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.908+0000 7f695d19e640 1 -- 172.21.15.43:0/3645469677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6958101e40 con 0x7f6958107af0 2024-09-11T23:20:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.909+0000 7f69557fa640 1 -- 172.21.15.43:0/3645469677 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f69500305a0 con 0x7f6958107af0 2024-09-11T23:20:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.910+0000 7f69557fa640 1 --2- 172.21.15.43:0/3645469677 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f69400a2c00 0x7f69400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.910+0000 7f6957fff640 1 --2- 172.21.15.43:0/3645469677 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f69400a2c00 0x7f69400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.910+0000 7f69557fa640 1 -- 172.21.15.43:0/3645469677 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f695002e030 con 0x7f6958107af0 2024-09-11T23:20:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.910+0000 7f6957fff640 1 --2- 172.21.15.43:0/3645469677 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f69400a2c00 0x7f69400a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6948005d00 tx=0x7f694800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:03.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:03.914+0000 7f69557fa640 1 -- 172.21.15.43:0/3645469677 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f695004b060 con 0x7f6958107af0 2024-09-11T23:20:04.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:03 smithi090 ceph-mon[35130]: pgmap v283: 97 pgs: 19 creating+peering, 13 unknown, 65 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 2 op/s 2024-09-11T23:20:04.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:03 smithi090 ceph-mon[35130]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T23:20:04.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:03 smithi090 ceph-mon[35130]: Cluster is now healthy 2024-09-11T23:20:04.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:03 smithi097 ceph-mon[35562]: pgmap v283: 97 pgs: 19 creating+peering, 13 unknown, 65 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 2 op/s 2024-09-11T23:20:04.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:03 smithi097 ceph-mon[35562]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T23:20:04.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:03 smithi097 ceph-mon[35562]: Cluster is now healthy 2024-09-11T23:20:04.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:04.092+0000 7f695d19e640 1 -- 172.21.15.43:0/3645469677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6958103270 con 0x7f6958107af0 2024-09-11T23:20:04.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:04.093+0000 7f69557fa640 1 -- 172.21.15.43:0/3645469677 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22145 (secure 0 0 0) 0x7f69500bd780 con 0x7f6958107af0 2024-09-11T23:20:04.095 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:20:04.095 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":82,"fsid":"25c0cd84-7093-11ef-bcea-c7b262605968","created":"2024-09-11T23:12:03.504406+0000","modified":"2024-09-11T23:20:00.465617+0000","last_up_change":"2024-09-11T23:19:07.797873+0000","last_in_change":"2024-09-11T23:18:44.595176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:15:17.058713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:19:56.221406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:19:58.705875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24076596-3909-4c87-85f1-cbf168070dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6803","nonce":43839997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6805","nonce":43839997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6809","nonce":43839997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6807","nonce":43839997}]},"public_addr":"172.21.15.43:6803/43839997","cluster_addr":"172.21.15.43:6805/43839997","heartbeat_back_addr":"172.21.15.43:6809/43839997","heartbeat_front_addr":"172.21.15.43:6807/43839997","state":["exists","up"]},{"osd":1,"uuid":"a0472a57-243c-489d-a247-972ee10a71f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6801","nonce":1825149246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6803","nonce":1825149246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6807","nonce":1825149246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6805","nonce":1825149246}]},"public_addr":"172.21.15.90:6801/1825149246","cluster_addr":"172.21.15.90:6803/1825149246","heartbeat_back_addr":"172.21.15.90:6807/1825149246","heartbeat_front_addr":"172.21.15.90:6805/1825149246","state":["exists","up"]},{"osd":2,"uuid":"40366536-0fcf-45a0-b8e3-49178401dc34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6801","nonce":1456846443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6803","nonce":1456846443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6807","nonce":1456846443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6805","nonce":1456846443}]},"public_addr":"172.21.15.97:6801/1456846443","cluster_addr":"172.21.15.97:6803/1456846443","heartbeat_back_addr":"172.21.15.97:6807/1456846443","heartbeat_front_addr":"172.21.15.97:6805/1456846443","state":["exists","up"]},{"osd":3,"uuid":"3062133e-000b-4421-9d53-ac434b9ed9b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6811","nonce":1962837804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6813","nonce":1962837804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6817","nonce":1962837804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6815","nonce":1962837804}]},"public_addr":"172.21.15.43:6811/1962837804","cluster_addr":"172.21.15.43:6813/1962837804","heartbeat_back_addr":"172.21.15.43:6817/1962837804","heartbeat_front_addr":"172.21.15.43:6815/1962837804","state":["exists","up"]},{"osd":4,"uuid":"2e73fa4b-f22c-4319-828d-87b91b9f19f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6809","nonce":181614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6811","nonce":181614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6815","nonce":181614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6813","nonce":181614173}]},"public_addr":"172.21.15.90:6809/181614173","cluster_addr":"172.21.15.90:6811/181614173","heartbeat_back_addr":"172.21.15.90:6815/181614173","heartbeat_front_addr":"172.21.15.90:6813/181614173","state":["exists","up"]},{"osd":5,"uuid":"1e1ad4f9-e724-440f-b515-3ec1ad97d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6809","nonce":1540039543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6811","nonce":1540039543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6815","nonce":1540039543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6813","nonce":1540039543}]},"public_addr":"172.21.15.97:6809/1540039543","cluster_addr":"172.21.15.97:6811/1540039543","heartbeat_back_addr":"172.21.15.97:6815/1540039543","heartbeat_front_addr":"172.21.15.97:6813/1540039543","state":["exists","up"]},{"osd":6,"uuid":"d34df8df-f40d-4159-8b29-5401b1b0e431","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6819","nonce":3785677637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6821","nonce":3785677637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6825","nonce":3785677637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6823","nonce":3785677637}]},"public_addr":"172.21.15.43:6819/3785677637","cluster_addr":"172.21.15.43:6821/3785677637","heartbeat_back_addr":"172.21.15.43:6825/3785677637","heartbeat_front_addr":"172.21.15.43:6823/3785677637","state":["exists","up"]},{"osd":7,"uuid":"b2e01078-528b-49fd-b000-9f584f784b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6817","nonce":236874962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6819","nonce":236874962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6823","nonce":236874962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6821","nonce":236874962}]},"public_addr":"172.21.15.90:6817/236874962","cluster_addr":"172.21.15.90:6819/236874962","heartbeat_back_addr":"172.21.15.90:6823/236874962","heartbeat_front_addr":"172.21.15.90:6821/236874962","state":["exists","up"]},{"osd":8,"uuid":"abef8f27-bf9b-4d63-bfbe-f967fc3d8b17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6817","nonce":1707327301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6819","nonce":1707327301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6823","nonce":1707327301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6821","nonce":1707327301}]},"public_addr":"172.21.15.97:6817/1707327301","cluster_addr":"172.21.15.97:6819/1707327301","heartbeat_back_addr":"172.21.15.97:6823/1707327301","heartbeat_front_addr":"172.21.15.97:6821/1707327301","state":["exists","up"]},{"osd":9,"uuid":"190ab8b2-a835-47e6-9b39-16a58331c58a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6827","nonce":3170306011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6829","nonce":3170306011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6833","nonce":3170306011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6831","nonce":3170306011}]},"public_addr":"172.21.15.43:6827/3170306011","cluster_addr":"172.21.15.43:6829/3170306011","heartbeat_back_addr":"172.21.15.43:6833/3170306011","heartbeat_front_addr":"172.21.15.43:6831/3170306011","state":["exists","up"]},{"osd":10,"uuid":"b32ab5fe-466f-4c8e-8696-cd9f820a3263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6825","nonce":59487041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6827","nonce":59487041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6831","nonce":59487041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6829","nonce":59487041}]},"public_addr":"172.21.15.90:6825/59487041","cluster_addr":"172.21.15.90:6827/59487041","heartbeat_back_addr":"172.21.15.90:6831/59487041","heartbeat_front_addr":"172.21.15.90:6829/59487041","state":["exists","up"]},{"osd":11,"uuid":"d0faa589-21c8-4a9f-8a73-813aafe6d33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6825","nonce":194699315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6827","nonce":194699315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6831","nonce":194699315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6829","nonce":194699315}]},"public_addr":"172.21.15.97:6825/194699315","cluster_addr":"172.21.15.97:6827/194699315","heartbeat_back_addr":"172.21.15.97:6831/194699315","heartbeat_front_addr":"172.21.15.97:6829/194699315","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:23.381247+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:48.843190+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:13.538979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:40.136983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:05.513790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:31.247090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:56.635221+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:22.667327+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:47.910422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:14.353914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:40.584503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:19:06.476766+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/76575371":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/4006124758":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/120495444":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6800/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:6800/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/206044413":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1512201905":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1375379108":"2024-09-12T23:12:35.756598+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:20:04.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:04.096+0000 7f695d19e640 1 -- 172.21.15.43:0/3645469677 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f69400a2c00 msgr2=0x7f69400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:04.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:04.096+0000 7f695d19e640 1 --2- 172.21.15.43:0/3645469677 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f69400a2c00 0x7f69400a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6948005d00 tx=0x7f694800b410 comp rx=0 tx=0).stop 2024-09-11T23:20:04.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:04.096+0000 7f695d19e640 1 -- 172.21.15.43:0/3645469677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6958107af0 msgr2=0x7f695819cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:04.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:04.096+0000 7f695d19e640 1 --2- 172.21.15.43:0/3645469677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6958107af0 0x7f695819cc00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6950005cc0 tx=0x7f6950003970 comp rx=0 tx=0).stop 2024-09-11T23:20:04.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:04.096+0000 7f695d19e640 1 -- 172.21.15.43:0/3645469677 shutdown_connections 2024-09-11T23:20:04.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:04.096+0000 7f695d19e640 1 --2- 172.21.15.43:0/3645469677 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f69400a2c00 0x7f69400a50c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:04.096+0000 7f695d19e640 1 --2- 172.21.15.43:0/3645469677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6958107af0 0x7f695819cc00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:04.096+0000 7f695d19e640 1 --2- 172.21.15.43:0/3645469677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6958102380 0x7f6958198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:04.096+0000 7f695d19e640 1 --2- 172.21.15.43:0/3645469677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69581019e0 0x7f6958198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:04.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:04.096+0000 7f695d19e640 1 -- 172.21.15.43:0/3645469677 >> 172.21.15.43:0/3645469677 conn(0x7f69580fd700 msgr2=0x7f69580fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:04.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:04.096+0000 7f695d19e640 1 -- 172.21.15.43:0/3645469677 shutdown_connections 2024-09-11T23:20:04.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:04.096+0000 7f695d19e640 1 -- 172.21.15.43:0/3645469677 wait complete. 2024-09-11T23:20:04.793 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-11T23:20:04.834 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:04 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3645469677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:20:05.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:04 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3645469677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:20:05.020 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:20:04 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-b[65178]: 2024-09-11T23:20:04.983+0000 7fa492c14640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T23:20:05.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:04 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3645469677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:20:05.069 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:05.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.574+0000 7fd461905640 1 -- 172.21.15.43:0/2195767230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd45c101de0 msgr2=0x7fd45c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:05.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.574+0000 7fd461905640 1 --2- 172.21.15.43:0/2195767230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd45c101de0 0x7fd45c1021c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009900 tx=0x7fd44c02a410 comp rx=0 tx=0).stop 2024-09-11T23:20:05.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.574+0000 7fd461905640 1 -- 172.21.15.43:0/2195767230 shutdown_connections 2024-09-11T23:20:05.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.574+0000 7fd461905640 1 --2- 172.21.15.43:0/2195767230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd45c106f10 0x7fd45c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:05.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.574+0000 7fd461905640 1 --2- 172.21.15.43:0/2195767230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd45c102790 0x7fd45c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:05.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.574+0000 7fd461905640 1 --2- 172.21.15.43:0/2195767230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd45c101de0 0x7fd45c1021c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:05.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.574+0000 7fd461905640 1 -- 172.21.15.43:0/2195767230 >> 172.21.15.43:0/2195767230 conn(0x7fd45c0fd8c0 msgr2=0x7fd45c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:05.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.574+0000 7fd461905640 1 -- 172.21.15.43:0/2195767230 shutdown_connections 2024-09-11T23:20:05.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.574+0000 7fd461905640 1 -- 172.21.15.43:0/2195767230 wait complete. 2024-09-11T23:20:05.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.575+0000 7fd461905640 1 Processor -- start 2024-09-11T23:20:05.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.575+0000 7fd461905640 1 -- start start 2024-09-11T23:20:05.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.575+0000 7fd461905640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd45c101de0 0x7fd45c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:05.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.575+0000 7fd461905640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd45c102790 0x7fd45c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:05.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.575+0000 7fd461905640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd45c106f10 0x7fd45c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:05.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.575+0000 7fd461905640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c111f70 con 0x7fd45c101de0 2024-09-11T23:20:05.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.575+0000 7fd461905640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c111df0 con 0x7fd45c102790 2024-09-11T23:20:05.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.575+0000 7fd461905640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c1120f0 con 0x7fd45c106f10 2024-09-11T23:20:05.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.575+0000 7fd45affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd45c101de0 0x7fd45c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:05.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.575+0000 7fd45affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd45c101de0 0x7fd45c198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39138/0 (socket says 172.21.15.43:39138) 2024-09-11T23:20:05.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.575+0000 7fd45affd640 1 -- 172.21.15.43:0/3722431864 learned_addr learned my addr 172.21.15.43:0/3722431864 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:05.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.575+0000 7fd45b7fe640 1 --2- 172.21.15.43:0/3722431864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd45c106f10 0x7fd45c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:05.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.576+0000 7fd45a7fc640 1 --2- 172.21.15.43:0/3722431864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd45c102790 0x7fd45c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:05.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.576+0000 7fd45affd640 1 -- 172.21.15.43:0/3722431864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd45c106f10 msgr2=0x7fd45c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:05.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.576+0000 7fd45affd640 1 --2- 172.21.15.43:0/3722431864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd45c106f10 0x7fd45c19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:05.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.576+0000 7fd45affd640 1 -- 172.21.15.43:0/3722431864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd45c102790 msgr2=0x7fd45c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:05.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.576+0000 7fd45affd640 1 --2- 172.21.15.43:0/3722431864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd45c102790 0x7fd45c1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:05.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.576+0000 7fd45affd640 1 -- 172.21.15.43:0/3722431864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45c19d320 con 0x7fd45c101de0 2024-09-11T23:20:05.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.576+0000 7fd45affd640 1 --2- 172.21.15.43:0/3722431864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd45c101de0 0x7fd45c198370 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd44c006010 tx=0x7fd44c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:05.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.576+0000 7fd460903640 1 -- 172.21.15.43:0/3722431864 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c03f030 con 0x7fd45c101de0 2024-09-11T23:20:05.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.576+0000 7fd460903640 1 -- 172.21.15.43:0/3722431864 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd44c003ce0 con 0x7fd45c101de0 2024-09-11T23:20:05.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.576+0000 7fd461905640 1 -- 172.21.15.43:0/3722431864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd45c19d5b0 con 0x7fd45c101de0 2024-09-11T23:20:05.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.576+0000 7fd460903640 1 -- 172.21.15.43:0/3722431864 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c035070 con 0x7fd45c101de0 2024-09-11T23:20:05.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.576+0000 7fd461905640 1 -- 172.21.15.43:0/3722431864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd45c1a4ef0 con 0x7fd45c101de0 2024-09-11T23:20:05.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.577+0000 7fd461905640 1 -- 172.21.15.43:0/3722431864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd45c1032c0 con 0x7fd45c101de0 2024-09-11T23:20:05.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.578+0000 7fd460903640 1 -- 172.21.15.43:0/3722431864 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd44c042030 con 0x7fd45c101de0 2024-09-11T23:20:05.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.579+0000 7fd460903640 1 --2- 172.21.15.43:0/3722431864 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd4440a2b80 0x7fd4440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:05.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.579+0000 7fd45a7fc640 1 --2- 172.21.15.43:0/3722431864 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd4440a2b80 0x7fd4440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:05.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.579+0000 7fd460903640 1 -- 172.21.15.43:0/3722431864 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fd44c0ef5d0 con 0x7fd45c101de0 2024-09-11T23:20:05.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.580+0000 7fd45a7fc640 1 --2- 172.21.15.43:0/3722431864 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd4440a2b80 0x7fd4440a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd45c1088e0 tx=0x7fd450005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:05.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.583+0000 7fd460903640 1 -- 172.21.15.43:0/3722431864 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd44c0bc720 con 0x7fd45c101de0 2024-09-11T23:20:05.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:05.798+0000 7fd461905640 1 -- 172.21.15.43:0/3722431864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd45c10a140 con 0x7fd45c101de0 2024-09-11T23:20:06.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:05 smithi090 ceph-mon[35130]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-11T23:20:06.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:05 smithi097 ceph-mon[35562]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-11T23:20:06.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:05 smithi043 ceph-mon[30950]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-11T23:20:06.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:06.757+0000 7fd460903640 1 -- 172.21.15.43:0/3722431864 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fd45c10a140 con 0x7fd45c101de0 2024-09-11T23:20:06.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:06.761+0000 7fd461905640 1 -- 172.21.15.43:0/3722431864 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd4440a2b80 msgr2=0x7fd4440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:06.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:06.761+0000 7fd461905640 1 --2- 172.21.15.43:0/3722431864 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd4440a2b80 0x7fd4440a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd45c1088e0 tx=0x7fd450005cc0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:06.761+0000 7fd461905640 1 -- 172.21.15.43:0/3722431864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd45c101de0 msgr2=0x7fd45c198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:06.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:06.761+0000 7fd461905640 1 --2- 172.21.15.43:0/3722431864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd45c101de0 0x7fd45c198370 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd44c006010 tx=0x7fd44c0042f0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:06.761+0000 7fd461905640 1 -- 172.21.15.43:0/3722431864 shutdown_connections 2024-09-11T23:20:06.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:06.761+0000 7fd461905640 1 --2- 172.21.15.43:0/3722431864 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd4440a2b80 0x7fd4440a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:06.761+0000 7fd461905640 1 --2- 172.21.15.43:0/3722431864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd45c106f10 0x7fd45c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:06.761+0000 7fd461905640 1 --2- 172.21.15.43:0/3722431864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd45c102790 0x7fd45c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:06.761+0000 7fd461905640 1 --2- 172.21.15.43:0/3722431864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd45c101de0 0x7fd45c198370 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:06.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:06.761+0000 7fd461905640 1 -- 172.21.15.43:0/3722431864 >> 172.21.15.43:0/3722431864 conn(0x7fd45c0fd8c0 msgr2=0x7fd45c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:06.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:06.762+0000 7fd461905640 1 -- 172.21.15.43:0/3722431864 shutdown_connections 2024-09-11T23:20:06.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:06.762+0000 7fd461905640 1 -- 172.21.15.43:0/3722431864 wait complete. 2024-09-11T23:20:07.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:06 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3722431864' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T23:20:07.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:06 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3722431864' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T23:20:07.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:06 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3722431864' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T23:20:07.452 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-11T23:20:07.453 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-11T23:20:07.757 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:08.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:07 smithi090 ceph-mon[35130]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-11T23:20:08.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:07 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3722431864' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T23:20:08.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:07 smithi090 ceph-mon[35130]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:20:08.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:07 smithi043 ceph-mon[30950]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-11T23:20:08.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:07 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3722431864' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T23:20:08.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:07 smithi043 ceph-mon[30950]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:20:08.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:07 smithi097 ceph-mon[35562]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-11T23:20:08.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:07 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3722431864' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T23:20:08.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:07 smithi097 ceph-mon[35562]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T23:20:08.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.290+0000 7f649da0f640 1 -- 172.21.15.43:0/45389594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6498106f10 msgr2=0x7f649810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:08.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.290+0000 7f649da0f640 1 --2- 172.21.15.43:0/45389594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6498106f10 0x7f649810ea90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6490009ca0 tx=0x7f649002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.290+0000 7f649da0f640 1 -- 172.21.15.43:0/45389594 shutdown_connections 2024-09-11T23:20:08.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.290+0000 7f649da0f640 1 --2- 172.21.15.43:0/45389594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6498106f10 0x7f649810ea90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.290+0000 7f649da0f640 1 --2- 172.21.15.43:0/45389594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6498102790 0x7f64981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.290+0000 7f649da0f640 1 --2- 172.21.15.43:0/45389594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6498101de0 0x7f64981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.290+0000 7f649da0f640 1 -- 172.21.15.43:0/45389594 >> 172.21.15.43:0/45389594 conn(0x7f64980fd8c0 msgr2=0x7f64980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:08.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.290+0000 7f649da0f640 1 -- 172.21.15.43:0/45389594 shutdown_connections 2024-09-11T23:20:08.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.290+0000 7f649da0f640 1 -- 172.21.15.43:0/45389594 wait complete. 2024-09-11T23:20:08.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.291+0000 7f649da0f640 1 Processor -- start 2024-09-11T23:20:08.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.291+0000 7f649da0f640 1 -- start start 2024-09-11T23:20:08.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.291+0000 7f649da0f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6498101de0 0x7f6498198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.291+0000 7f649da0f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6498102790 0x7f6498198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.291+0000 7f649da0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6498106f10 0x7f649819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.291+0000 7f649da0f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6498111f90 con 0x7f6498106f10 2024-09-11T23:20:08.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.291+0000 7f649da0f640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6498111e10 con 0x7f6498101de0 2024-09-11T23:20:08.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.291+0000 7f649da0f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6498112110 con 0x7f6498102790 2024-09-11T23:20:08.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.291+0000 7f64977fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6498106f10 0x7f649819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.291+0000 7f64977fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6498106f10 0x7f649819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39160/0 (socket says 172.21.15.43:39160) 2024-09-11T23:20:08.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.291+0000 7f64977fe640 1 -- 172.21.15.43:0/2716652325 learned_addr learned my addr 172.21.15.43:0/2716652325 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.291+0000 7f64967fc640 1 --2- 172.21.15.43:0/2716652325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6498102790 0x7f6498198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.291+0000 7f64977fe640 1 -- 172.21.15.43:0/2716652325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6498102790 msgr2=0x7f6498198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.291+0000 7f64977fe640 1 --2- 172.21.15.43:0/2716652325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6498102790 0x7f6498198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.291+0000 7f64977fe640 1 -- 172.21.15.43:0/2716652325 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6498101de0 msgr2=0x7f6498198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.291+0000 7f6496ffd640 1 --2- 172.21.15.43:0/2716652325 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6498101de0 0x7f6498198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.291+0000 7f64977fe640 1 --2- 172.21.15.43:0/2716652325 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6498101de0 0x7f6498198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.291+0000 7f64977fe640 1 -- 172.21.15.43:0/2716652325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649819d3c0 con 0x7f6498106f10 2024-09-11T23:20:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.292+0000 7f64977fe640 1 --2- 172.21.15.43:0/2716652325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6498106f10 0x7f649819cce0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6490031a60 tx=0x7f6490031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.292+0000 7f649ca0d640 1 -- 172.21.15.43:0/2716652325 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490003c30 con 0x7f6498106f10 2024-09-11T23:20:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.292+0000 7f649da0f640 1 -- 172.21.15.43:0/2716652325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f649819d650 con 0x7f6498106f10 2024-09-11T23:20:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.292+0000 7f649ca0d640 1 -- 172.21.15.43:0/2716652325 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6490003dd0 con 0x7f6498106f10 2024-09-11T23:20:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.292+0000 7f649ca0d640 1 -- 172.21.15.43:0/2716652325 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64900047e0 con 0x7f6498106f10 2024-09-11T23:20:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.292+0000 7f649da0f640 1 -- 172.21.15.43:0/2716652325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64981a4f90 con 0x7f6498106f10 2024-09-11T23:20:08.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.293+0000 7f649da0f640 1 -- 172.21.15.43:0/2716652325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64981021c0 con 0x7f6498106f10 2024-09-11T23:20:08.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.294+0000 7f649ca0d640 1 -- 172.21.15.43:0/2716652325 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f649002f790 con 0x7f6498106f10 2024-09-11T23:20:08.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.295+0000 7f649ca0d640 1 --2- 172.21.15.43:0/2716652325 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f64840a2b80 0x7f64840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:08.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.295+0000 7f6496ffd640 1 --2- 172.21.15.43:0/2716652325 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f64840a2b80 0x7f64840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:08.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.295+0000 7f649ca0d640 1 -- 172.21.15.43:0/2716652325 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f64900f0380 con 0x7f6498106f10 2024-09-11T23:20:08.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.295+0000 7f6496ffd640 1 --2- 172.21.15.43:0/2716652325 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f64840a2b80 0x7f64840a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f648800a810 tx=0x7f6488008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:08.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.299+0000 7f649ca0d640 1 -- 172.21.15.43:0/2716652325 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f64900bd430 con 0x7f6498106f10 2024-09-11T23:20:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.513+0000 7f649da0f640 1 -- 172.21.15.43:0/2716652325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f64981a51c0 con 0x7f6498106f10 2024-09-11T23:20:08.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.710+0000 7f649ca0d640 1 -- 172.21.15.43:0/2716652325 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f64900bcd30 con 0x7f6498106f10 2024-09-11T23:20:08.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.712+0000 7f649da0f640 1 -- 172.21.15.43:0/2716652325 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f64840a2b80 msgr2=0x7f64840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:08.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.712+0000 7f649da0f640 1 --2- 172.21.15.43:0/2716652325 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f64840a2b80 0x7f64840a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f648800a810 tx=0x7f6488008040 comp rx=0 tx=0).stop 2024-09-11T23:20:08.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.712+0000 7f649da0f640 1 -- 172.21.15.43:0/2716652325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6498106f10 msgr2=0x7f649819cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:08.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.712+0000 7f649da0f640 1 --2- 172.21.15.43:0/2716652325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6498106f10 0x7f649819cce0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6490031a60 tx=0x7f6490031a90 comp rx=0 tx=0).stop 2024-09-11T23:20:08.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.712+0000 7f649da0f640 1 -- 172.21.15.43:0/2716652325 shutdown_connections 2024-09-11T23:20:08.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.712+0000 7f649da0f640 1 --2- 172.21.15.43:0/2716652325 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f64840a2b80 0x7f64840a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.713+0000 7f649da0f640 1 --2- 172.21.15.43:0/2716652325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6498106f10 0x7f649819cce0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.713+0000 7f649da0f640 1 --2- 172.21.15.43:0/2716652325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6498102790 0x7f6498198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.713+0000 7f649da0f640 1 --2- 172.21.15.43:0/2716652325 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6498101de0 0x7f6498198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:08.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.713+0000 7f649da0f640 1 -- 172.21.15.43:0/2716652325 >> 172.21.15.43:0/2716652325 conn(0x7f64980fd8c0 msgr2=0x7f64980ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:08.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.713+0000 7f649da0f640 1 -- 172.21.15.43:0/2716652325 shutdown_connections 2024-09-11T23:20:08.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:08.713+0000 7f649da0f640 1 -- 172.21.15.43:0/2716652325 wait complete. 2024-09-11T23:20:09.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:08 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2716652325' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-11T23:20:09.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:08 smithi090 ceph-mon[35130]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T23:20:09.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:08 smithi090 ceph-mon[35130]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T23:20:09.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:08 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2716652325' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-11T23:20:09.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:08 smithi090 ceph-mon[35130]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-11T23:20:09.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:08 smithi090 ceph-mon[35130]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-11T23:20:09.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:08 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2716652325' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-11T23:20:09.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:08 smithi097 ceph-mon[35562]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T23:20:09.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:08 smithi097 ceph-mon[35562]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T23:20:09.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:08 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2716652325' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-11T23:20:09.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:08 smithi097 ceph-mon[35562]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-11T23:20:09.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:08 smithi097 ceph-mon[35562]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-11T23:20:09.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:08 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2716652325' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-11T23:20:09.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:08 smithi043 ceph-mon[30950]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T23:20:09.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:08 smithi043 ceph-mon[30950]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T23:20:09.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:08 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2716652325' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-11T23:20:09.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:08 smithi043 ceph-mon[30950]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-11T23:20:09.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:08 smithi043 ceph-mon[30950]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-11T23:20:09.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-11T23:20:09.768 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:10.020 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:20:09 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-b[65178]: 2024-09-11T23:20:09.984+0000 7fa492c14640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-11T23:20:10.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:09 smithi090 ceph-mon[35130]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-11T23:20:10.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:09 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] up:active 2024-09-11T23:20:10.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:09 smithi090 ceph-mon[35130]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T23:20:10.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:09 smithi090 ceph-mon[35130]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T23:20:10.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:09 smithi090 ceph-mon[35130]: Cluster is now healthy 2024-09-11T23:20:10.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:09 smithi090 ceph-mon[35130]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-11T23:20:10.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:09 smithi090 ceph-mon[35130]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-11T23:20:10.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:09 smithi090 ceph-mon[35130]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-11T23:20:10.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:09 smithi097 ceph-mon[35562]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-11T23:20:10.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:09 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] up:active 2024-09-11T23:20:10.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:09 smithi097 ceph-mon[35562]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T23:20:10.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:09 smithi097 ceph-mon[35562]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T23:20:10.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:09 smithi097 ceph-mon[35562]: Cluster is now healthy 2024-09-11T23:20:10.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:09 smithi097 ceph-mon[35562]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-11T23:20:10.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:09 smithi097 ceph-mon[35562]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-11T23:20:10.052 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:09 smithi097 ceph-mon[35562]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-11T23:20:10.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:09 smithi043 ceph-mon[30950]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-11T23:20:10.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:09 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] up:active 2024-09-11T23:20:10.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:09 smithi043 ceph-mon[30950]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T23:20:10.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:09 smithi043 ceph-mon[30950]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T23:20:10.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:09 smithi043 ceph-mon[30950]: Cluster is now healthy 2024-09-11T23:20:10.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:09 smithi043 ceph-mon[30950]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-11T23:20:10.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:09 smithi043 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-11T23:20:10.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:09 smithi043 ceph-mon[30950]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-11T23:20:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.292+0000 7fb4b3646640 1 -- 172.21.15.43:0/884606812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ac102790 msgr2=0x7fb4ac1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.292+0000 7fb4b3646640 1 --2- 172.21.15.43:0/884606812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ac102790 0x7fb4ac1067e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009900 tx=0x7fb49c02a410 comp rx=0 tx=0).stop 2024-09-11T23:20:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.292+0000 7fb4b3646640 1 -- 172.21.15.43:0/884606812 shutdown_connections 2024-09-11T23:20:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.292+0000 7fb4b3646640 1 --2- 172.21.15.43:0/884606812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4ac106f10 0x7fb4ac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.292+0000 7fb4b3646640 1 --2- 172.21.15.43:0/884606812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ac102790 0x7fb4ac1067e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.292+0000 7fb4b3646640 1 --2- 172.21.15.43:0/884606812 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4ac101de0 0x7fb4ac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.292+0000 7fb4b3646640 1 -- 172.21.15.43:0/884606812 >> 172.21.15.43:0/884606812 conn(0x7fb4ac0fd8c0 msgr2=0x7fb4ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.292+0000 7fb4b3646640 1 -- 172.21.15.43:0/884606812 shutdown_connections 2024-09-11T23:20:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.292+0000 7fb4b3646640 1 -- 172.21.15.43:0/884606812 wait complete. 2024-09-11T23:20:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.293+0000 7fb4b3646640 1 Processor -- start 2024-09-11T23:20:10.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.293+0000 7fb4b3646640 1 -- start start 2024-09-11T23:20:10.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.293+0000 7fb4b3646640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4ac101de0 0x7fb4ac1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.293+0000 7fb4b3646640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4ac102790 0x7fb4ac1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.293+0000 7fb4b3646640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ac106f10 0x7fb4ac19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.293+0000 7fb4b3646640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac111fb0 con 0x7fb4ac106f10 2024-09-11T23:20:10.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.293+0000 7fb4b3646640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac111e30 con 0x7fb4ac101de0 2024-09-11T23:20:10.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.293+0000 7fb4b3646640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac112130 con 0x7fb4ac102790 2024-09-11T23:20:10.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.293+0000 7fb4b1be3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ac106f10 0x7fb4ac19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.293+0000 7fb4b1be3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ac106f10 0x7fb4ac19cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56514/0 (socket says 172.21.15.43:56514) 2024-09-11T23:20:10.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.293+0000 7fb4b1be3640 1 -- 172.21.15.43:0/1328662473 learned_addr learned my addr 172.21.15.43:0/1328662473 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:10.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.293+0000 7fb4b13e2640 1 --2- 172.21.15.43:0/1328662473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4ac101de0 0x7fb4ac1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.293+0000 7fb4b0be1640 1 --2- 172.21.15.43:0/1328662473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4ac102790 0x7fb4ac1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.293+0000 7fb4b1be3640 1 -- 172.21.15.43:0/1328662473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4ac102790 msgr2=0x7fb4ac1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:10.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.293+0000 7fb4b1be3640 1 --2- 172.21.15.43:0/1328662473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4ac102790 0x7fb4ac1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.293+0000 7fb4b1be3640 1 -- 172.21.15.43:0/1328662473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4ac101de0 msgr2=0x7fb4ac1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:10.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.293+0000 7fb4b1be3640 1 --2- 172.21.15.43:0/1328662473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4ac101de0 0x7fb4ac1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.293+0000 7fb4b1be3640 1 -- 172.21.15.43:0/1328662473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4ac19d460 con 0x7fb4ac106f10 2024-09-11T23:20:10.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.294+0000 7fb4b1be3640 1 --2- 172.21.15.43:0/1328662473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ac106f10 0x7fb4ac19cd80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400eaa0 tx=0x7fb4a400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:10.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.294+0000 7fb4a27fc640 1 -- 172.21.15.43:0/1328662473 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4004d20 con 0x7fb4ac106f10 2024-09-11T23:20:10.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.294+0000 7fb4a27fc640 1 -- 172.21.15.43:0/1328662473 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a4005020 con 0x7fb4ac106f10 2024-09-11T23:20:10.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.294+0000 7fb4b3646640 1 -- 172.21.15.43:0/1328662473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4ac19d750 con 0x7fb4ac106f10 2024-09-11T23:20:10.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.294+0000 7fb4a27fc640 1 -- 172.21.15.43:0/1328662473 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a40116d0 con 0x7fb4ac106f10 2024-09-11T23:20:10.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.294+0000 7fb4b3646640 1 -- 172.21.15.43:0/1328662473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4ac1a5030 con 0x7fb4ac106f10 2024-09-11T23:20:10.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.295+0000 7fb4b3646640 1 -- 172.21.15.43:0/1328662473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4ac1021c0 con 0x7fb4ac106f10 2024-09-11T23:20:10.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.296+0000 7fb4a27fc640 1 -- 172.21.15.43:0/1328662473 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb4a40051d0 con 0x7fb4ac106f10 2024-09-11T23:20:10.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.297+0000 7fb4a27fc640 1 --2- 172.21.15.43:0/1328662473 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb4980a2b80 0x7fb4980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:10.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.297+0000 7fb4b13e2640 1 --2- 172.21.15.43:0/1328662473 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb4980a2b80 0x7fb4980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:10.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.297+0000 7fb4a27fc640 1 -- 172.21.15.43:0/1328662473 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fb4a40cf1d0 con 0x7fb4ac106f10 2024-09-11T23:20:10.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.297+0000 7fb4b13e2640 1 --2- 172.21.15.43:0/1328662473 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb4980a2b80 0x7fb4980a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009990 tx=0x7fb4a8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:10.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.301+0000 7fb4a27fc640 1 -- 172.21.15.43:0/1328662473 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb4a4016030 con 0x7fb4ac106f10 2024-09-11T23:20:10.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.516+0000 7fb4b3646640 1 -- 172.21.15.43:0/1328662473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fb4ac10a150 con 0x7fb4ac106f10 2024-09-11T23:20:10.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.717+0000 7fb4a27fc640 1 -- 172.21.15.43:0/1328662473 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7fb4a409c1e0 con 0x7fb4ac106f10 2024-09-11T23:20:10.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.720+0000 7fb4b3646640 1 -- 172.21.15.43:0/1328662473 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb4980a2b80 msgr2=0x7fb4980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:10.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.720+0000 7fb4b3646640 1 --2- 172.21.15.43:0/1328662473 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb4980a2b80 0x7fb4980a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009990 tx=0x7fb4a8009110 comp rx=0 tx=0).stop 2024-09-11T23:20:10.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.720+0000 7fb4b3646640 1 -- 172.21.15.43:0/1328662473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ac106f10 msgr2=0x7fb4ac19cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:10.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.720+0000 7fb4b3646640 1 --2- 172.21.15.43:0/1328662473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ac106f10 0x7fb4ac19cd80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400eaa0 tx=0x7fb4a400eea0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.720+0000 7fb4b3646640 1 -- 172.21.15.43:0/1328662473 shutdown_connections 2024-09-11T23:20:10.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.720+0000 7fb4b3646640 1 --2- 172.21.15.43:0/1328662473 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb4980a2b80 0x7fb4980a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.720+0000 7fb4b3646640 1 --2- 172.21.15.43:0/1328662473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4ac106f10 0x7fb4ac19cd80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.720+0000 7fb4b3646640 1 --2- 172.21.15.43:0/1328662473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4ac102790 0x7fb4ac1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.720+0000 7fb4b3646640 1 --2- 172.21.15.43:0/1328662473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4ac101de0 0x7fb4ac1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:10.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.720+0000 7fb4b3646640 1 -- 172.21.15.43:0/1328662473 >> 172.21.15.43:0/1328662473 conn(0x7fb4ac0fd8c0 msgr2=0x7fb4ac0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:10.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.720+0000 7fb4b3646640 1 -- 172.21.15.43:0/1328662473 shutdown_connections 2024-09-11T23:20:10.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:10.720+0000 7fb4b3646640 1 -- 172.21.15.43:0/1328662473 wait complete. 2024-09-11T23:20:11.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:10 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1328662473' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T23:20:11.046 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:10 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1328662473' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T23:20:11.046 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:10 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] up:active 2024-09-11T23:20:11.046 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:10 smithi043 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-11T23:20:11.046 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:10 smithi043 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:20:11.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:10 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1328662473' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T23:20:11.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:10 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1328662473' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T23:20:11.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:10 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] up:active 2024-09-11T23:20:11.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:10 smithi097 ceph-mon[35562]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-11T23:20:11.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:10 smithi097 ceph-mon[35562]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:20:11.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:10 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1328662473' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T23:20:11.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:10 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1328662473' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T23:20:11.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:10 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] up:active 2024-09-11T23:20:11.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:10 smithi090 ceph-mon[35130]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-11T23:20:11.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:10 smithi090 ceph-mon[35130]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:20:11.383 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-11T23:20:11.641 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:12.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:11 smithi097 ceph-mon[35562]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-11T23:20:12.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:11 smithi043 ceph-mon[30950]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-11T23:20:12.171 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:11 smithi090 ceph-mon[35130]: pgmap v287: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-11T23:20:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.217+0000 7fe947e48640 1 -- 172.21.15.43:0/3179166929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe940102790 msgr2=0x7fe9401067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.217+0000 7fe947e48640 1 --2- 172.21.15.43:0/3179166929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe940102790 0x7fe9401067e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe930009900 tx=0x7fe93002a410 comp rx=0 tx=0).stop 2024-09-11T23:20:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.217+0000 7fe947e48640 1 -- 172.21.15.43:0/3179166929 shutdown_connections 2024-09-11T23:20:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.217+0000 7fe947e48640 1 --2- 172.21.15.43:0/3179166929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe940106f10 0x7fe94010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.217+0000 7fe947e48640 1 --2- 172.21.15.43:0/3179166929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe940102790 0x7fe9401067e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.217+0000 7fe947e48640 1 --2- 172.21.15.43:0/3179166929 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe940101de0 0x7fe9401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.217+0000 7fe947e48640 1 -- 172.21.15.43:0/3179166929 >> 172.21.15.43:0/3179166929 conn(0x7fe9400fd8c0 msgr2=0x7fe9400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.217+0000 7fe947e48640 1 -- 172.21.15.43:0/3179166929 shutdown_connections 2024-09-11T23:20:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.217+0000 7fe947e48640 1 -- 172.21.15.43:0/3179166929 wait complete. 2024-09-11T23:20:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.218+0000 7fe947e48640 1 Processor -- start 2024-09-11T23:20:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.218+0000 7fe947e48640 1 -- start start 2024-09-11T23:20:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.218+0000 7fe947e48640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe940101de0 0x7fe940198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.218+0000 7fe947e48640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe940102790 0x7fe940198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.218+0000 7fe947e48640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe940106f10 0x7fe94019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.218+0000 7fe947e48640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe940111f90 con 0x7fe940102790 2024-09-11T23:20:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.218+0000 7fe947e48640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe940111e10 con 0x7fe940101de0 2024-09-11T23:20:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.218+0000 7fe947e48640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe940112110 con 0x7fe940106f10 2024-09-11T23:20:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.218+0000 7fe9453e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe940102790 0x7fe940198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.218+0000 7fe9453e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe940102790 0x7fe940198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56528/0 (socket says 172.21.15.43:56528) 2024-09-11T23:20:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.218+0000 7fe9453e3640 1 -- 172.21.15.43:0/879613745 learned_addr learned my addr 172.21.15.43:0/879613745 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.218+0000 7fe945be4640 1 --2- 172.21.15.43:0/879613745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe940101de0 0x7fe940198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.218+0000 7fe9463e5640 1 --2- 172.21.15.43:0/879613745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe940106f10 0x7fe94019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.219+0000 7fe9453e3640 1 -- 172.21.15.43:0/879613745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe940106f10 msgr2=0x7fe94019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:12.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.219+0000 7fe9453e3640 1 --2- 172.21.15.43:0/879613745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe940106f10 0x7fe94019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:12.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.219+0000 7fe9453e3640 1 -- 172.21.15.43:0/879613745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe940101de0 msgr2=0x7fe940198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:12.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.219+0000 7fe9453e3640 1 --2- 172.21.15.43:0/879613745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe940101de0 0x7fe940198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:12.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.219+0000 7fe9453e3640 1 -- 172.21.15.43:0/879613745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94019d3d0 con 0x7fe940102790 2024-09-11T23:20:12.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.219+0000 7fe9453e3640 1 --2- 172.21.15.43:0/879613745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe940102790 0x7fe940198960 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe930005ab0 tx=0x7fe9300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:12.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.219+0000 7fe936ffd640 1 -- 172.21.15.43:0/879613745 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93003f030 con 0x7fe940102790 2024-09-11T23:20:12.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.219+0000 7fe936ffd640 1 -- 172.21.15.43:0/879613745 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe930003ce0 con 0x7fe940102790 2024-09-11T23:20:12.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.219+0000 7fe947e48640 1 -- 172.21.15.43:0/879613745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe94019d660 con 0x7fe940102790 2024-09-11T23:20:12.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.219+0000 7fe936ffd640 1 -- 172.21.15.43:0/879613745 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe930035070 con 0x7fe940102790 2024-09-11T23:20:12.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.219+0000 7fe947e48640 1 -- 172.21.15.43:0/879613745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9401a4fa0 con 0x7fe940102790 2024-09-11T23:20:12.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.220+0000 7fe947e48640 1 -- 172.21.15.43:0/879613745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9401032c0 con 0x7fe940102790 2024-09-11T23:20:12.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.221+0000 7fe936ffd640 1 -- 172.21.15.43:0/879613745 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe930042030 con 0x7fe940102790 2024-09-11T23:20:12.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.222+0000 7fe936ffd640 1 --2- 172.21.15.43:0/879613745 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe92c0a2b80 0x7fe92c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:12.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.222+0000 7fe945be4640 1 --2- 172.21.15.43:0/879613745 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe92c0a2b80 0x7fe92c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:12.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.222+0000 7fe936ffd640 1 -- 172.21.15.43:0/879613745 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fe9300ef7c0 con 0x7fe940102790 2024-09-11T23:20:12.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.223+0000 7fe945be4640 1 --2- 172.21.15.43:0/879613745 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe92c0a2b80 0x7fe92c0a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00abe0 tx=0x7fe93c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:12.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.226+0000 7fe936ffd640 1 -- 172.21.15.43:0/879613745 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9300bc870 con 0x7fe940102790 2024-09-11T23:20:12.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.437+0000 7fe947e48640 1 -- 172.21.15.43:0/879613745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fe94010a0f0 con 0x7fe940102790 2024-09-11T23:20:12.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.777+0000 7fe936ffd640 1 -- 172.21.15.43:0/879613745 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7fe9300bc170 con 0x7fe940102790 2024-09-11T23:20:12.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.780+0000 7fe947e48640 1 -- 172.21.15.43:0/879613745 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe92c0a2b80 msgr2=0x7fe92c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:12.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.780+0000 7fe947e48640 1 --2- 172.21.15.43:0/879613745 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe92c0a2b80 0x7fe92c0a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00abe0 tx=0x7fe93c00a2d0 comp rx=0 tx=0).stop 2024-09-11T23:20:12.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.780+0000 7fe947e48640 1 -- 172.21.15.43:0/879613745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe940102790 msgr2=0x7fe940198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:12.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.780+0000 7fe947e48640 1 --2- 172.21.15.43:0/879613745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe940102790 0x7fe940198960 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe930005ab0 tx=0x7fe9300042f0 comp rx=0 tx=0).stop 2024-09-11T23:20:12.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.780+0000 7fe947e48640 1 -- 172.21.15.43:0/879613745 shutdown_connections 2024-09-11T23:20:12.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.780+0000 7fe947e48640 1 --2- 172.21.15.43:0/879613745 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe92c0a2b80 0x7fe92c0a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:12.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.780+0000 7fe947e48640 1 --2- 172.21.15.43:0/879613745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe940106f10 0x7fe94019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:12.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.780+0000 7fe947e48640 1 --2- 172.21.15.43:0/879613745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe940102790 0x7fe940198960 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:12.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.780+0000 7fe947e48640 1 --2- 172.21.15.43:0/879613745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe940101de0 0x7fe940198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:12.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.780+0000 7fe947e48640 1 -- 172.21.15.43:0/879613745 >> 172.21.15.43:0/879613745 conn(0x7fe9400fd8c0 msgr2=0x7fe9400fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:12.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.780+0000 7fe947e48640 1 -- 172.21.15.43:0/879613745 shutdown_connections 2024-09-11T23:20:12.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:12.780+0000 7fe947e48640 1 -- 172.21.15.43:0/879613745 wait complete. 2024-09-11T23:20:13.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:12 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/879613745' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T23:20:13.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:12 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/879613745' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T23:20:13.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:12 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/879613745' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T23:20:13.579 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-11T23:20:13.579 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-11T23:20:13.841 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:13.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:13 smithi043 ceph-mon[30950]: pgmap v288: 97 pgs: 97 active+clean; 580 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 601 B/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-11T23:20:13.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:13 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/879613745' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T23:20:13.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:13 smithi043 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:20:14.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:13 smithi097 ceph-mon[35562]: pgmap v288: 97 pgs: 97 active+clean; 580 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 601 B/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-11T23:20:14.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:13 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/879613745' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T23:20:14.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:13 smithi097 ceph-mon[35562]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:20:14.273 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:13 smithi090 ceph-mon[35130]: pgmap v288: 97 pgs: 97 active+clean; 580 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 601 B/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-11T23:20:14.273 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:13 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/879613745' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T23:20:14.273 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:13 smithi090 ceph-mon[35130]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:20:14.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.376+0000 7f76ed5dc640 1 -- 172.21.15.43:0/1061163182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76e8102d30 msgr2=0x7f76e8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:14.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.376+0000 7f76ed5dc640 1 --2- 172.21.15.43:0/1061163182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76e8102d30 0x7f76e8103110 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009cd0 tx=0x7f76d802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:20:14.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.376+0000 7f76ed5dc640 1 -- 172.21.15.43:0/1061163182 shutdown_connections 2024-09-11T23:20:14.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.376+0000 7f76ed5dc640 1 --2- 172.21.15.43:0/1061163182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76e81082a0 0x7f76e810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:14.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.376+0000 7f76ed5dc640 1 --2- 172.21.15.43:0/1061163182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76e81036e0 0x7f76e8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:14.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.376+0000 7f76ed5dc640 1 --2- 172.21.15.43:0/1061163182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76e8102d30 0x7f76e8103110 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:14.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.376+0000 7f76ed5dc640 1 -- 172.21.15.43:0/1061163182 >> 172.21.15.43:0/1061163182 conn(0x7f76e80fc7d0 msgr2=0x7f76e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:14.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.376+0000 7f76ed5dc640 1 -- 172.21.15.43:0/1061163182 shutdown_connections 2024-09-11T23:20:14.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.376+0000 7f76ed5dc640 1 -- 172.21.15.43:0/1061163182 wait complete. 2024-09-11T23:20:14.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.377+0000 7f76ed5dc640 1 Processor -- start 2024-09-11T23:20:14.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.377+0000 7f76ed5dc640 1 -- start start 2024-09-11T23:20:14.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.377+0000 7f76ed5dc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76e8102d30 0x7f76e8198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:14.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.377+0000 7f76ed5dc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76e81036e0 0x7f76e8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:14.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.377+0000 7f76ed5dc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76e81082a0 0x7f76e819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:14.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.378+0000 7f76ed5dc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f76e81120a0 con 0x7f76e81082a0 2024-09-11T23:20:14.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.378+0000 7f76ed5dc640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8111f20 con 0x7f76e81036e0 2024-09-11T23:20:14.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.378+0000 7f76ed5dc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8112220 con 0x7f76e8102d30 2024-09-11T23:20:14.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.378+0000 7f76e77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76e81082a0 0x7f76e819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:14.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.378+0000 7f76e77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76e81082a0 0x7f76e819cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56544/0 (socket says 172.21.15.43:56544) 2024-09-11T23:20:14.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.378+0000 7f76e6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76e8102d30 0x7f76e8198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:14.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.378+0000 7f76e77fe640 1 -- 172.21.15.43:0/3179485834 learned_addr learned my addr 172.21.15.43:0/3179485834 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:14.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.378+0000 7f76e67fc640 1 --2- 172.21.15.43:0/3179485834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76e81036e0 0x7f76e8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:14.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.378+0000 7f76e77fe640 1 -- 172.21.15.43:0/3179485834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76e8102d30 msgr2=0x7f76e8198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:14.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.378+0000 7f76e77fe640 1 --2- 172.21.15.43:0/3179485834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76e8102d30 0x7f76e8198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:14.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.378+0000 7f76e77fe640 1 -- 172.21.15.43:0/3179485834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76e81036e0 msgr2=0x7f76e8198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:14.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.378+0000 7f76e77fe640 1 --2- 172.21.15.43:0/3179485834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76e81036e0 0x7f76e8198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:14.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.378+0000 7f76e77fe640 1 -- 172.21.15.43:0/3179485834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e819d4d0 con 0x7f76e81082a0 2024-09-11T23:20:14.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.378+0000 7f76e77fe640 1 --2- 172.21.15.43:0/3179485834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76e81082a0 0x7f76e819cdf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f76e000c970 tx=0x7f76e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:14.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.378+0000 7f76cffff640 1 -- 172.21.15.43:0/3179485834 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0007bb0 con 0x7f76e81082a0 2024-09-11T23:20:14.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.378+0000 7f76ed5dc640 1 -- 172.21.15.43:0/3179485834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e819d7c0 con 0x7f76e81082a0 2024-09-11T23:20:14.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.378+0000 7f76cffff640 1 -- 172.21.15.43:0/3179485834 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e0007d50 con 0x7f76e81082a0 2024-09-11T23:20:14.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.378+0000 7f76cffff640 1 -- 172.21.15.43:0/3179485834 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0018800 con 0x7f76e81082a0 2024-09-11T23:20:14.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.378+0000 7f76ed5dc640 1 -- 172.21.15.43:0/3179485834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e81a50a0 con 0x7f76e81082a0 2024-09-11T23:20:14.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.379+0000 7f76ed5dc640 1 -- 172.21.15.43:0/3179485834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e806a850 con 0x7f76e81082a0 2024-09-11T23:20:14.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.381+0000 7f76cffff640 1 -- 172.21.15.43:0/3179485834 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f76e000fa50 con 0x7f76e81082a0 2024-09-11T23:20:14.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.381+0000 7f76cffff640 1 --2- 172.21.15.43:0/3179485834 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f76d00a2b80 0x7f76d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:14.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.381+0000 7f76e6ffd640 1 --2- 172.21.15.43:0/3179485834 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f76d00a2b80 0x7f76d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:14.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.381+0000 7f76cffff640 1 -- 172.21.15.43:0/3179485834 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f76e00cee80 con 0x7f76e81082a0 2024-09-11T23:20:14.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.382+0000 7f76e6ffd640 1 --2- 172.21.15.43:0/3179485834 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f76d00a2b80 0x7f76d00a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f76d8006010 tx=0x7f76d8009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:14.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.385+0000 7f76cffff640 1 -- 172.21.15.43:0/3179485834 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76e0014030 con 0x7f76e81082a0 2024-09-11T23:20:14.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.586+0000 7f76ed5dc640 1 -- 172.21.15.43:0/3179485834 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f76e8107420 con 0x7f76d00a2b80 2024-09-11T23:20:14.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.611+0000 7f76cffff640 1 -- 172.21.15.43:0/3179485834 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f76e8107420 con 0x7f76d00a2b80 2024-09-11T23:20:14.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.613+0000 7f76ed5dc640 1 -- 172.21.15.43:0/3179485834 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f76d00a2b80 msgr2=0x7f76d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:14.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.613+0000 7f76ed5dc640 1 --2- 172.21.15.43:0/3179485834 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f76d00a2b80 0x7f76d00a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f76d8006010 tx=0x7f76d8009360 comp rx=0 tx=0).stop 2024-09-11T23:20:14.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.613+0000 7f76ed5dc640 1 -- 172.21.15.43:0/3179485834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76e81082a0 msgr2=0x7f76e819cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:14.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.613+0000 7f76ed5dc640 1 --2- 172.21.15.43:0/3179485834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76e81082a0 0x7f76e819cdf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f76e000c970 tx=0x7f76e000ce30 comp rx=0 tx=0).stop 2024-09-11T23:20:14.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.613+0000 7f76ed5dc640 1 -- 172.21.15.43:0/3179485834 shutdown_connections 2024-09-11T23:20:14.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.613+0000 7f76ed5dc640 1 --2- 172.21.15.43:0/3179485834 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f76d00a2b80 0x7f76d00a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:14.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.613+0000 7f76ed5dc640 1 --2- 172.21.15.43:0/3179485834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76e81082a0 0x7f76e819cdf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:14.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.613+0000 7f76ed5dc640 1 --2- 172.21.15.43:0/3179485834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76e81036e0 0x7f76e8198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:14.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.613+0000 7f76ed5dc640 1 --2- 172.21.15.43:0/3179485834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76e8102d30 0x7f76e8198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:14.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.613+0000 7f76ed5dc640 1 -- 172.21.15.43:0/3179485834 >> 172.21.15.43:0/3179485834 conn(0x7f76e80fc7d0 msgr2=0x7f76e8105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:14.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.613+0000 7f76ed5dc640 1 -- 172.21.15.43:0/3179485834 shutdown_connections 2024-09-11T23:20:14.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:14.613+0000 7f76ed5dc640 1 -- 172.21.15.43:0/3179485834 wait complete. 2024-09-11T23:20:14.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:14 smithi043 ceph-mon[30950]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:20:14.860 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:20:14 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:20:14.591+0000 7f5b0777d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:20:14.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:20:14 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:20:14.591+0000 7f5b0777d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:20:14.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:20:14 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:20:14.591+0000 7f5b0777d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:20:14.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:20:14 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:20:14.591+0000 7f5b0777d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:20:14.861 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:20:14 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:20:14.591+0000 7f5b0777d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:20:15.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:14 smithi097 ceph-mon[35562]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:20:15.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:14 smithi090 ceph-mon[35130]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:20:15.334 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-11T23:20:15.590 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:15.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:15 smithi043 ceph-mon[30950]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-11T23:20:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:15 smithi043 ceph-mon[30950]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-11T23:20:16.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:15 smithi097 ceph-mon[35562]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-11T23:20:16.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:15 smithi097 ceph-mon[35562]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-11T23:20:16.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.070+0000 7f9abb4ce640 1 -- 172.21.15.43:0/51266193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ab41008d0 msgr2=0x7f9ab4100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:16.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.070+0000 7f9abb4ce640 1 --2- 172.21.15.43:0/51266193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ab41008d0 0x7f9ab4100cb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0009900 tx=0x7f9ab002a410 comp rx=0 tx=0).stop 2024-09-11T23:20:16.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.070+0000 7f9abb4ce640 1 -- 172.21.15.43:0/51266193 shutdown_connections 2024-09-11T23:20:16.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.070+0000 7f9abb4ce640 1 --2- 172.21.15.43:0/51266193 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ab410e480 0x7f9ab4110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:16.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.070+0000 7f9abb4ce640 1 --2- 172.21.15.43:0/51266193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ab4101280 0x7f9ab410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:16.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.070+0000 7f9abb4ce640 1 --2- 172.21.15.43:0/51266193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ab41008d0 0x7f9ab4100cb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:16.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.070+0000 7f9abb4ce640 1 -- 172.21.15.43:0/51266193 >> 172.21.15.43:0/51266193 conn(0x7f9ab40fc7a0 msgr2=0x7f9ab40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:16.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.070+0000 7f9abb4ce640 1 -- 172.21.15.43:0/51266193 shutdown_connections 2024-09-11T23:20:16.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.070+0000 7f9abb4ce640 1 -- 172.21.15.43:0/51266193 wait complete. 2024-09-11T23:20:16.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.071+0000 7f9abb4ce640 1 Processor -- start 2024-09-11T23:20:16.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.071+0000 7f9abb4ce640 1 -- start start 2024-09-11T23:20:16.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.071+0000 7f9abb4ce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ab41008d0 0x7f9ab419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:16.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.071+0000 7f9abb4ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ab4101280 0x7f9ab419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:16.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.071+0000 7f9abb4ce640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ab410e480 0x7f9ab419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:16.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.071+0000 7f9abb4ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab4114190 con 0x7f9ab4101280 2024-09-11T23:20:16.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.071+0000 7f9abb4ce640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab4114010 con 0x7f9ab410e480 2024-09-11T23:20:16.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.071+0000 7f9abb4ce640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab4114310 con 0x7f9ab41008d0 2024-09-11T23:20:16.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.071+0000 7f9ab8a69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ab4101280 0x7f9ab419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:16.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.071+0000 7f9ab8a69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ab4101280 0x7f9ab419ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56580/0 (socket says 172.21.15.43:56580) 2024-09-11T23:20:16.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.071+0000 7f9ab8a69640 1 -- 172.21.15.43:0/3652876662 learned_addr learned my addr 172.21.15.43:0/3652876662 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:16.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.071+0000 7f9ab926a640 1 --2- 172.21.15.43:0/3652876662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ab41008d0 0x7f9ab419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:16.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.071+0000 7f9ab9a6b640 1 --2- 172.21.15.43:0/3652876662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ab410e480 0x7f9ab419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:16.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.072+0000 7f9ab8a69640 1 -- 172.21.15.43:0/3652876662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ab41008d0 msgr2=0x7f9ab419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:16.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.072+0000 7f9ab8a69640 1 --2- 172.21.15.43:0/3652876662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ab41008d0 0x7f9ab419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:16.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.072+0000 7f9ab8a69640 1 -- 172.21.15.43:0/3652876662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ab410e480 msgr2=0x7f9ab419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:16.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.072+0000 7f9ab8a69640 1 --2- 172.21.15.43:0/3652876662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ab410e480 0x7f9ab419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:16.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.072+0000 7f9ab8a69640 1 -- 172.21.15.43:0/3652876662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab419f5b0 con 0x7f9ab4101280 2024-09-11T23:20:16.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.072+0000 7f9ab8a69640 1 --2- 172.21.15.43:0/3652876662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ab4101280 0x7f9ab419ab40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400c920 tx=0x7f9aa400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:16.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.072+0000 7f9aaa7fc640 1 -- 172.21.15.43:0/3652876662 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4007c70 con 0x7f9ab4101280 2024-09-11T23:20:16.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.072+0000 7f9aaa7fc640 1 -- 172.21.15.43:0/3652876662 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aa4007e10 con 0x7f9ab4101280 2024-09-11T23:20:16.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.072+0000 7f9abb4ce640 1 -- 172.21.15.43:0/3652876662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab419f8a0 con 0x7f9ab4101280 2024-09-11T23:20:16.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.072+0000 7f9aaa7fc640 1 -- 172.21.15.43:0/3652876662 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4018890 con 0x7f9ab4101280 2024-09-11T23:20:16.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.072+0000 7f9abb4ce640 1 -- 172.21.15.43:0/3652876662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab41a7180 con 0x7f9ab4101280 2024-09-11T23:20:16.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.073+0000 7f9abb4ce640 1 -- 172.21.15.43:0/3652876662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab4100cb0 con 0x7f9ab4101280 2024-09-11T23:20:16.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.074+0000 7f9aaa7fc640 1 -- 172.21.15.43:0/3652876662 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9aa4018a70 con 0x7f9ab4101280 2024-09-11T23:20:16.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.075+0000 7f9aaa7fc640 1 --2- 172.21.15.43:0/3652876662 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9aa00a2c10 0x7f9aa00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:16.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.075+0000 7f9ab926a640 1 --2- 172.21.15.43:0/3652876662 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9aa00a2c10 0x7f9aa00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:16.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.075+0000 7f9aaa7fc640 1 -- 172.21.15.43:0/3652876662 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f9aa40cf220 con 0x7f9ab4101280 2024-09-11T23:20:16.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.075+0000 7f9ab926a640 1 --2- 172.21.15.43:0/3652876662 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9aa00a2c10 0x7f9aa00a50d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0005ab0 tx=0x7f9ab0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:16.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.079+0000 7f9aaa7fc640 1 -- 172.21.15.43:0/3652876662 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9aa4014030 con 0x7f9ab4101280 2024-09-11T23:20:16.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:15 smithi090 ceph-mon[35130]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-11T23:20:16.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:15 smithi090 ceph-mon[35130]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-11T23:20:16.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.282+0000 7f9abb4ce640 1 -- 172.21.15.43:0/3652876662 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9ab4062630 con 0x7f9aa00a2c10 2024-09-11T23:20:16.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.308+0000 7f9aaa7fc640 1 -- 172.21.15.43:0/3652876662 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9ab4062630 con 0x7f9aa00a2c10 2024-09-11T23:20:16.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.310+0000 7f9abb4ce640 1 -- 172.21.15.43:0/3652876662 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9aa00a2c10 msgr2=0x7f9aa00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:16.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.310+0000 7f9abb4ce640 1 --2- 172.21.15.43:0/3652876662 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9aa00a2c10 0x7f9aa00a50d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0005ab0 tx=0x7f9ab0005a20 comp rx=0 tx=0).stop 2024-09-11T23:20:16.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.310+0000 7f9abb4ce640 1 -- 172.21.15.43:0/3652876662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ab4101280 msgr2=0x7f9ab419ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:16.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.310+0000 7f9abb4ce640 1 --2- 172.21.15.43:0/3652876662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ab4101280 0x7f9ab419ab40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400c920 tx=0x7f9aa400cde0 comp rx=0 tx=0).stop 2024-09-11T23:20:16.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.310+0000 7f9abb4ce640 1 -- 172.21.15.43:0/3652876662 shutdown_connections 2024-09-11T23:20:16.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.310+0000 7f9abb4ce640 1 --2- 172.21.15.43:0/3652876662 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9aa00a2c10 0x7f9aa00a50d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:16.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.310+0000 7f9abb4ce640 1 --2- 172.21.15.43:0/3652876662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ab410e480 0x7f9ab419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:16.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.310+0000 7f9abb4ce640 1 --2- 172.21.15.43:0/3652876662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ab4101280 0x7f9ab419ab40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:16.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.310+0000 7f9abb4ce640 1 --2- 172.21.15.43:0/3652876662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ab41008d0 0x7f9ab419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:16.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.310+0000 7f9abb4ce640 1 -- 172.21.15.43:0/3652876662 >> 172.21.15.43:0/3652876662 conn(0x7f9ab40fc7a0 msgr2=0x7f9ab40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:16.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.310+0000 7f9abb4ce640 1 -- 172.21.15.43:0/3652876662 shutdown_connections 2024-09-11T23:20:16.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:16.310+0000 7f9abb4ce640 1 -- 172.21.15.43:0/3652876662 wait complete. 2024-09-11T23:20:17.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:16 smithi097 ceph-mon[35562]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:20:17.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:16 smithi043 ceph-mon[30950]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:20:17.060 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-11T23:20:17.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:16 smithi090 ceph-mon[35130]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:20:17.316 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:17.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.835+0000 7f39e4685640 1 -- 172.21.15.43:0/2041912120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39dc102d30 msgr2=0x7f39dc103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:17.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.835+0000 7f39e4685640 1 --2- 172.21.15.43:0/2041912120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39dc102d30 0x7f39dc103110 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009cd0 tx=0x7f39cc02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.835+0000 7f39e4685640 1 -- 172.21.15.43:0/2041912120 shutdown_connections 2024-09-11T23:20:17.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.835+0000 7f39e4685640 1 --2- 172.21.15.43:0/2041912120 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39dc1082a0 0x7f39dc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.835+0000 7f39e4685640 1 --2- 172.21.15.43:0/2041912120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc1036e0 0x7f39dc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.835+0000 7f39e4685640 1 --2- 172.21.15.43:0/2041912120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39dc102d30 0x7f39dc103110 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.835+0000 7f39e4685640 1 -- 172.21.15.43:0/2041912120 >> 172.21.15.43:0/2041912120 conn(0x7f39dc0fc7d0 msgr2=0x7f39dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:17.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.835+0000 7f39e4685640 1 -- 172.21.15.43:0/2041912120 shutdown_connections 2024-09-11T23:20:17.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.835+0000 7f39e4685640 1 -- 172.21.15.43:0/2041912120 wait complete. 2024-09-11T23:20:17.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.835+0000 7f39e4685640 1 Processor -- start 2024-09-11T23:20:17.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.835+0000 7f39e4685640 1 -- start start 2024-09-11T23:20:17.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.836+0000 7f39e4685640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39dc102d30 0x7f39dc1a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.836+0000 7f39e4685640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc1036e0 0x7f39dc1a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.836+0000 7f39e4685640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39dc1082a0 0x7f39dc19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.836+0000 7f39e4685640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1120a0 con 0x7f39dc1082a0 2024-09-11T23:20:17.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.836+0000 7f39e4685640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc111f20 con 0x7f39dc102d30 2024-09-11T23:20:17.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.836+0000 7f39e4685640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc112220 con 0x7f39dc1036e0 2024-09-11T23:20:17.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.836+0000 7f39e2c22640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39dc1082a0 0x7f39dc19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.836+0000 7f39e2c22640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39dc1082a0 0x7f39dc19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56610/0 (socket says 172.21.15.43:56610) 2024-09-11T23:20:17.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.836+0000 7f39e2c22640 1 -- 172.21.15.43:0/3802098609 learned_addr learned my addr 172.21.15.43:0/3802098609 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:17.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.836+0000 7f39e1c20640 1 --2- 172.21.15.43:0/3802098609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc1036e0 0x7f39dc1a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.836+0000 7f39e2421640 1 --2- 172.21.15.43:0/3802098609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39dc102d30 0x7f39dc1a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.836+0000 7f39e2c22640 1 -- 172.21.15.43:0/3802098609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc1036e0 msgr2=0x7f39dc1a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:17.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.836+0000 7f39e2c22640 1 --2- 172.21.15.43:0/3802098609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc1036e0 0x7f39dc1a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.836+0000 7f39e2c22640 1 -- 172.21.15.43:0/3802098609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39dc102d30 msgr2=0x7f39dc1a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:17.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.836+0000 7f39e2c22640 1 --2- 172.21.15.43:0/3802098609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39dc102d30 0x7f39dc1a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:17.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.836+0000 7f39e2c22640 1 -- 172.21.15.43:0/3802098609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39dc19ad20 con 0x7f39dc1082a0 2024-09-11T23:20:17.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.836+0000 7f39e2c22640 1 --2- 172.21.15.43:0/3802098609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39dc1082a0 0x7f39dc19a5f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f39d400d750 tx=0x7f39d400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:17.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.837+0000 7f39db7fe640 1 -- 172.21.15.43:0/3802098609 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4017070 con 0x7f39dc1082a0 2024-09-11T23:20:17.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.837+0000 7f39db7fe640 1 -- 172.21.15.43:0/3802098609 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d4005500 con 0x7f39dc1082a0 2024-09-11T23:20:17.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.837+0000 7f39e4685640 1 -- 172.21.15.43:0/3802098609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39dc19b010 con 0x7f39dc1082a0 2024-09-11T23:20:17.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.837+0000 7f39db7fe640 1 -- 172.21.15.43:0/3802098609 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4004d90 con 0x7f39dc1082a0 2024-09-11T23:20:17.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.837+0000 7f39e4685640 1 -- 172.21.15.43:0/3802098609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39dc1a7300 con 0x7f39dc1082a0 2024-09-11T23:20:17.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.838+0000 7f39e4685640 1 -- 172.21.15.43:0/3802098609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39dc06a850 con 0x7f39dc1082a0 2024-09-11T23:20:17.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.839+0000 7f39db7fe640 1 -- 172.21.15.43:0/3802098609 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f39d4004f30 con 0x7f39dc1082a0 2024-09-11T23:20:17.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.839+0000 7f39db7fe640 1 --2- 172.21.15.43:0/3802098609 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f39c40a2c90 0x7f39c40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:17.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.839+0000 7f39e2421640 1 --2- 172.21.15.43:0/3802098609 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f39c40a2c90 0x7f39c40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:17.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.840+0000 7f39db7fe640 1 -- 172.21.15.43:0/3802098609 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f39d40ce900 con 0x7f39dc1082a0 2024-09-11T23:20:17.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.840+0000 7f39e2421640 1 --2- 172.21.15.43:0/3802098609 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f39c40a2c90 0x7f39c40a5150 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f39cc006010 tx=0x7f39cc009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:17.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:17.844+0000 7f39db7fe640 1 -- 172.21.15.43:0/3802098609 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39d409b1e0 con 0x7f39dc1082a0 2024-09-11T23:20:18.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:18.047+0000 7f39e4685640 1 -- 172.21.15.43:0/3802098609 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f39dc062630 con 0x7f39c40a2c90 2024-09-11T23:20:18.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:17 smithi043 ceph-mon[30950]: pgmap v290: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-11T23:20:18.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:17 smithi097 ceph-mon[35562]: pgmap v290: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-11T23:20:18.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:18.053+0000 7f39db7fe640 1 -- 172.21.15.43:0/3802098609 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f39dc062630 con 0x7f39c40a2c90 2024-09-11T23:20:18.053 INFO:teuthology.orchestra.run.smithi043.stdout:/volumes/qa/sv_0/27d842cf-4fef-41d3-b9e8-c86cc7a599db 2024-09-11T23:20:18.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:18.055+0000 7f39e4685640 1 -- 172.21.15.43:0/3802098609 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f39c40a2c90 msgr2=0x7f39c40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:18.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:18.055+0000 7f39e4685640 1 --2- 172.21.15.43:0/3802098609 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f39c40a2c90 0x7f39c40a5150 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f39cc006010 tx=0x7f39cc009360 comp rx=0 tx=0).stop 2024-09-11T23:20:18.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:18.055+0000 7f39e4685640 1 -- 172.21.15.43:0/3802098609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39dc1082a0 msgr2=0x7f39dc19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:18.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:18.055+0000 7f39e4685640 1 --2- 172.21.15.43:0/3802098609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39dc1082a0 0x7f39dc19a5f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f39d400d750 tx=0x7f39d400dc10 comp rx=0 tx=0).stop 2024-09-11T23:20:18.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:18.055+0000 7f39e4685640 1 -- 172.21.15.43:0/3802098609 shutdown_connections 2024-09-11T23:20:18.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:18.055+0000 7f39e4685640 1 --2- 172.21.15.43:0/3802098609 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f39c40a2c90 0x7f39c40a5150 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:18.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:18.055+0000 7f39e4685640 1 --2- 172.21.15.43:0/3802098609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39dc1082a0 0x7f39dc19a5f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:18.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:18.055+0000 7f39e4685640 1 --2- 172.21.15.43:0/3802098609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc1036e0 0x7f39dc1a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:18.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:18.055+0000 7f39e4685640 1 --2- 172.21.15.43:0/3802098609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39dc102d30 0x7f39dc1a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:18.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:18.055+0000 7f39e4685640 1 -- 172.21.15.43:0/3802098609 >> 172.21.15.43:0/3802098609 conn(0x7f39dc0fc7d0 msgr2=0x7f39dc100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:18.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:18.055+0000 7f39e4685640 1 -- 172.21.15.43:0/3802098609 shutdown_connections 2024-09-11T23:20:18.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:18.055+0000 7f39e4685640 1 -- 172.21.15.43:0/3802098609 wait complete. 2024-09-11T23:20:18.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:17 smithi090 ceph-mon[35130]: pgmap v290: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-11T23:20:18.742 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-11T23:20:19.003 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:19.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:18 smithi097 ceph-mon[35562]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:20:19.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:18 smithi043 ceph-mon[30950]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:20:19.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:18 smithi090 ceph-mon[35130]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:20:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.498+0000 7f104af51640 1 -- 172.21.15.43:0/2724085839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1044101420 msgr2=0x7f104410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.498+0000 7f104af51640 1 --2- 172.21.15.43:0/2724085839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1044101420 0x7f104410d950 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f103c009cd0 tx=0x7f103c02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:20:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.499+0000 7f104af51640 1 -- 172.21.15.43:0/2724085839 shutdown_connections 2024-09-11T23:20:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.499+0000 7f104af51640 1 --2- 172.21.15.43:0/2724085839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1044101420 0x7f104410d950 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.499+0000 7f104af51640 1 --2- 172.21.15.43:0/2724085839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1044100a80 0x7f1044100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.499+0000 7f104af51640 1 --2- 172.21.15.43:0/2724085839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1044106a80 0x7f1044106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.499+0000 7f104af51640 1 -- 172.21.15.43:0/2724085839 >> 172.21.15.43:0/2724085839 conn(0x7f10440fc7a0 msgr2=0x7f10440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.499+0000 7f104af51640 1 -- 172.21.15.43:0/2724085839 shutdown_connections 2024-09-11T23:20:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.499+0000 7f104af51640 1 -- 172.21.15.43:0/2724085839 wait complete. 2024-09-11T23:20:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.499+0000 7f104af51640 1 Processor -- start 2024-09-11T23:20:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.499+0000 7f104af51640 1 -- start start 2024-09-11T23:20:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.499+0000 7f104af51640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1044100a80 0x7f1044102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.499+0000 7f104af51640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1044101420 0x7f1044102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.499+0000 7f104af51640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1044106a80 0x7f10441030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.499+0000 7f104af51640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1044110ea0 con 0x7f1044101420 2024-09-11T23:20:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.499+0000 7f104af51640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1044110d20 con 0x7f1044106a80 2024-09-11T23:20:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.499+0000 7f104af51640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1044111020 con 0x7f1044100a80 2024-09-11T23:20:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.500+0000 7f1043fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1044101420 0x7f1044102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.500+0000 7f1043fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1044101420 0x7f1044102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56626/0 (socket says 172.21.15.43:56626) 2024-09-11T23:20:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.500+0000 7f1043fff640 1 -- 172.21.15.43:0/685776294 learned_addr learned my addr 172.21.15.43:0/685776294 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.500+0000 7f1048ced640 1 --2- 172.21.15.43:0/685776294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1044100a80 0x7f1044102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.500+0000 7f10494ee640 1 --2- 172.21.15.43:0/685776294 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1044106a80 0x7f10441030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.500+0000 7f1043fff640 1 -- 172.21.15.43:0/685776294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1044100a80 msgr2=0x7f1044102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.500+0000 7f1043fff640 1 --2- 172.21.15.43:0/685776294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1044100a80 0x7f1044102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.500+0000 7f1043fff640 1 -- 172.21.15.43:0/685776294 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1044106a80 msgr2=0x7f10441030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.500+0000 7f1043fff640 1 --2- 172.21.15.43:0/685776294 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1044106a80 0x7f10441030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.500+0000 7f1043fff640 1 -- 172.21.15.43:0/685776294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10441a6890 con 0x7f1044101420 2024-09-11T23:20:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.500+0000 7f10494ee640 1 --2- 172.21.15.43:0/685776294 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1044106a80 0x7f10441030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:20:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.500+0000 7f1048ced640 1 --2- 172.21.15.43:0/685776294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1044100a80 0x7f1044102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.500+0000 7f1043fff640 1 --2- 172.21.15.43:0/685776294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1044101420 0x7f1044102b90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f102c00b4a0 tx=0x7f102c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.500+0000 7f1041ffb640 1 -- 172.21.15.43:0/685776294 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c014070 con 0x7f1044101420 2024-09-11T23:20:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.500+0000 7f1041ffb640 1 -- 172.21.15.43:0/685776294 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f102c010af0 con 0x7f1044101420 2024-09-11T23:20:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.500+0000 7f104af51640 1 -- 172.21.15.43:0/685776294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10441a6b80 con 0x7f1044101420 2024-09-11T23:20:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.500+0000 7f1041ffb640 1 -- 172.21.15.43:0/685776294 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c017d10 con 0x7f1044101420 2024-09-11T23:20:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.500+0000 7f104af51640 1 -- 172.21.15.43:0/685776294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10441a70c0 con 0x7f1044101420 2024-09-11T23:20:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.501+0000 7f104af51640 1 -- 172.21.15.43:0/685776294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1044100ee0 con 0x7f1044101420 2024-09-11T23:20:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.502+0000 7f1041ffb640 1 -- 172.21.15.43:0/685776294 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f102c017070 con 0x7f1044101420 2024-09-11T23:20:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.503+0000 7f1041ffb640 1 --2- 172.21.15.43:0/685776294 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f10300a2c10 0x7f10300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.503+0000 7f1048ced640 1 --2- 172.21.15.43:0/685776294 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f10300a2c10 0x7f10300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.504+0000 7f1041ffb640 1 -- 172.21.15.43:0/685776294 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f102c0ce330 con 0x7f1044101420 2024-09-11T23:20:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.504+0000 7f1048ced640 1 --2- 172.21.15.43:0/685776294 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f10300a2c10 0x7f10300a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f103400abe0 tx=0x7f103400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.508+0000 7f1041ffb640 1 -- 172.21.15.43:0/685776294 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f102c09b3f0 con 0x7f1044101420 2024-09-11T23:20:19.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.709+0000 7f104af51640 1 -- 172.21.15.43:0/685776294 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f10440fcc60 con 0x7f10300a2c10 2024-09-11T23:20:19.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.736+0000 7f1041ffb640 1 -- 172.21.15.43:0/685776294 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f10440fcc60 con 0x7f10300a2c10 2024-09-11T23:20:19.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.738+0000 7f104af51640 1 -- 172.21.15.43:0/685776294 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f10300a2c10 msgr2=0x7f10300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:19.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.738+0000 7f104af51640 1 --2- 172.21.15.43:0/685776294 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f10300a2c10 0x7f10300a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f103400abe0 tx=0x7f103400a5c0 comp rx=0 tx=0).stop 2024-09-11T23:20:19.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.738+0000 7f104af51640 1 -- 172.21.15.43:0/685776294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1044101420 msgr2=0x7f1044102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:19.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.738+0000 7f104af51640 1 --2- 172.21.15.43:0/685776294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1044101420 0x7f1044102b90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f102c00b4a0 tx=0x7f102c00b960 comp rx=0 tx=0).stop 2024-09-11T23:20:19.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.738+0000 7f104af51640 1 -- 172.21.15.43:0/685776294 shutdown_connections 2024-09-11T23:20:19.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.738+0000 7f104af51640 1 --2- 172.21.15.43:0/685776294 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f10300a2c10 0x7f10300a50d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:19.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.738+0000 7f104af51640 1 --2- 172.21.15.43:0/685776294 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1044106a80 0x7f10441030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:19.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.738+0000 7f104af51640 1 --2- 172.21.15.43:0/685776294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1044101420 0x7f1044102b90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:19.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.738+0000 7f104af51640 1 --2- 172.21.15.43:0/685776294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1044100a80 0x7f1044102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:19.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.738+0000 7f104af51640 1 -- 172.21.15.43:0/685776294 >> 172.21.15.43:0/685776294 conn(0x7f10440fc7a0 msgr2=0x7f1044104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:19.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.738+0000 7f104af51640 1 -- 172.21.15.43:0/685776294 shutdown_connections 2024-09-11T23:20:19.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:19.738+0000 7f104af51640 1 -- 172.21.15.43:0/685776294 wait complete. 2024-09-11T23:20:20.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:19 smithi043 ceph-mon[30950]: pgmap v291: 97 pgs: 97 active+clean; 602 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-11T23:20:20.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:19 smithi090 ceph-mon[35130]: pgmap v291: 97 pgs: 97 active+clean; 602 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-11T23:20:20.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:19 smithi097 ceph-mon[35562]: pgmap v291: 97 pgs: 97 active+clean; 602 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-11T23:20:20.437 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-11T23:20:20.692 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:21.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:20 smithi043 ceph-mon[30950]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:20:21.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.207+0000 7fd3b422d640 1 -- 172.21.15.43:0/756229621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3ac100eb0 msgr2=0x7fd3ac106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:21.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.207+0000 7fd3b422d640 1 --2- 172.21.15.43:0/756229621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3ac100eb0 0x7fd3ac106840 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009900 tx=0x7fd39c02a410 comp rx=0 tx=0).stop 2024-09-11T23:20:21.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.207+0000 7fd3b422d640 1 -- 172.21.15.43:0/756229621 shutdown_connections 2024-09-11T23:20:21.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.207+0000 7fd3b422d640 1 --2- 172.21.15.43:0/756229621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3ac106f70 0x7fd3ac10eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:21.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.207+0000 7fd3b422d640 1 --2- 172.21.15.43:0/756229621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3ac100eb0 0x7fd3ac106840 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:21.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.207+0000 7fd3b422d640 1 --2- 172.21.15.43:0/756229621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3ac100500 0x7fd3ac1008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:21.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.207+0000 7fd3b422d640 1 -- 172.21.15.43:0/756229621 >> 172.21.15.43:0/756229621 conn(0x7fd3ac0fc5c0 msgr2=0x7fd3ac0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:21.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.207+0000 7fd3b422d640 1 -- 172.21.15.43:0/756229621 shutdown_connections 2024-09-11T23:20:21.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.207+0000 7fd3b422d640 1 -- 172.21.15.43:0/756229621 wait complete. 2024-09-11T23:20:21.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.208+0000 7fd3b422d640 1 Processor -- start 2024-09-11T23:20:21.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.208+0000 7fd3b422d640 1 -- start start 2024-09-11T23:20:21.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.208+0000 7fd3b422d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3ac100500 0x7fd3ac101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:21.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.208+0000 7fd3b422d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3ac100eb0 0x7fd3ac1019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:21.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.208+0000 7fd3b422d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3ac106f70 0x7fd3ac101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:21.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.208+0000 7fd3b422d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac111f90 con 0x7fd3ac106f70 2024-09-11T23:20:21.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.208+0000 7fd3b422d640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac111e10 con 0x7fd3ac100eb0 2024-09-11T23:20:21.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.208+0000 7fd3b422d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac112110 con 0x7fd3ac100500 2024-09-11T23:20:21.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.208+0000 7fd3b27ca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3ac106f70 0x7fd3ac101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:21.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.208+0000 7fd3b27ca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3ac106f70 0x7fd3ac101f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39906/0 (socket says 172.21.15.43:39906) 2024-09-11T23:20:21.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.208+0000 7fd3b27ca640 1 -- 172.21.15.43:0/1826774920 learned_addr learned my addr 172.21.15.43:0/1826774920 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:21.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.208+0000 7fd3b1fc9640 1 --2- 172.21.15.43:0/1826774920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3ac100500 0x7fd3ac101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:21.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.209+0000 7fd3b17c8640 1 --2- 172.21.15.43:0/1826774920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3ac100eb0 0x7fd3ac1019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:21.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.209+0000 7fd3b27ca640 1 -- 172.21.15.43:0/1826774920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3ac100500 msgr2=0x7fd3ac101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:21.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.209+0000 7fd3b27ca640 1 --2- 172.21.15.43:0/1826774920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3ac100500 0x7fd3ac101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:21.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.209+0000 7fd3b27ca640 1 -- 172.21.15.43:0/1826774920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3ac100eb0 msgr2=0x7fd3ac1019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:21.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.209+0000 7fd3b27ca640 1 --2- 172.21.15.43:0/1826774920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3ac100eb0 0x7fd3ac1019c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:21.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.209+0000 7fd3b27ca640 1 -- 172.21.15.43:0/1826774920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ac1a68b0 con 0x7fd3ac106f70 2024-09-11T23:20:21.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.209+0000 7fd3b27ca640 1 --2- 172.21.15.43:0/1826774920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3ac106f70 0x7fd3ac101f00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400cad0 tx=0x7fd3a400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:21.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.209+0000 7fd3a2ffd640 1 -- 172.21.15.43:0/1826774920 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a400ae20 con 0x7fd3ac106f70 2024-09-11T23:20:21.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.209+0000 7fd3a2ffd640 1 -- 172.21.15.43:0/1826774920 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a4005020 con 0x7fd3ac106f70 2024-09-11T23:20:21.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.209+0000 7fd3b422d640 1 -- 172.21.15.43:0/1826774920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3ac1a6ba0 con 0x7fd3ac106f70 2024-09-11T23:20:21.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.209+0000 7fd3a2ffd640 1 -- 172.21.15.43:0/1826774920 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a400f6f0 con 0x7fd3ac106f70 2024-09-11T23:20:21.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.209+0000 7fd3b422d640 1 -- 172.21.15.43:0/1826774920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3ac1a7100 con 0x7fd3ac106f70 2024-09-11T23:20:21.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.210+0000 7fd3b422d640 1 -- 172.21.15.43:0/1826774920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3ac1008e0 con 0x7fd3ac106f70 2024-09-11T23:20:21.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.211+0000 7fd3a2ffd640 1 -- 172.21.15.43:0/1826774920 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd3a40051d0 con 0x7fd3ac106f70 2024-09-11T23:20:21.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.212+0000 7fd3a2ffd640 1 --2- 172.21.15.43:0/1826774920 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd3980a2c90 0x7fd3980a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:21.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.213+0000 7fd3b1fc9640 1 --2- 172.21.15.43:0/1826774920 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd3980a2c90 0x7fd3980a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:21.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.213+0000 7fd3a2ffd640 1 -- 172.21.15.43:0/1826774920 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fd3a40cf2a0 con 0x7fd3ac106f70 2024-09-11T23:20:21.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.213+0000 7fd3b1fc9640 1 --2- 172.21.15.43:0/1826774920 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd3980a2c90 0x7fd3980a5150 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac069fc0 tx=0x7fd3a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:21.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.216+0000 7fd3a2ffd640 1 -- 172.21.15.43:0/1826774920 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd3a4014030 con 0x7fd3ac106f70 2024-09-11T23:20:21.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:20 smithi090 ceph-mon[35130]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:20:21.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:20 smithi097 ceph-mon[35562]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:20:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.424+0000 7fd3b422d640 1 -- 172.21.15.43:0/1826774920 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fd3ac062630 con 0x7fd3980a2c90 2024-09-11T23:20:21.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.429+0000 7fd3a2ffd640 1 -- 172.21.15.43:0/1826774920 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd3ac062630 con 0x7fd3980a2c90 2024-09-11T23:20:21.429 INFO:teuthology.orchestra.run.smithi043.stdout:/volumes/qa/sv_1/2b469744-514b-43ef-9c93-a53c45581893 2024-09-11T23:20:21.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.431+0000 7fd3b422d640 1 -- 172.21.15.43:0/1826774920 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd3980a2c90 msgr2=0x7fd3980a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:21.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.431+0000 7fd3b422d640 1 --2- 172.21.15.43:0/1826774920 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd3980a2c90 0x7fd3980a5150 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac069fc0 tx=0x7fd3a8005f90 comp rx=0 tx=0).stop 2024-09-11T23:20:21.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.431+0000 7fd3b422d640 1 -- 172.21.15.43:0/1826774920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3ac106f70 msgr2=0x7fd3ac101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:21.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.431+0000 7fd3b422d640 1 --2- 172.21.15.43:0/1826774920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3ac106f70 0x7fd3ac101f00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400cad0 tx=0x7fd3a400cf90 comp rx=0 tx=0).stop 2024-09-11T23:20:21.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.431+0000 7fd3b422d640 1 -- 172.21.15.43:0/1826774920 shutdown_connections 2024-09-11T23:20:21.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.431+0000 7fd3b422d640 1 --2- 172.21.15.43:0/1826774920 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd3980a2c90 0x7fd3980a5150 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:21.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.431+0000 7fd3b422d640 1 --2- 172.21.15.43:0/1826774920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3ac106f70 0x7fd3ac101f00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:21.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.431+0000 7fd3b422d640 1 --2- 172.21.15.43:0/1826774920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3ac100eb0 0x7fd3ac1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:21.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.431+0000 7fd3b422d640 1 --2- 172.21.15.43:0/1826774920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3ac100500 0x7fd3ac101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:21.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.431+0000 7fd3b422d640 1 -- 172.21.15.43:0/1826774920 >> 172.21.15.43:0/1826774920 conn(0x7fd3ac0fc5c0 msgr2=0x7fd3ac105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:21.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.431+0000 7fd3b422d640 1 -- 172.21.15.43:0/1826774920 shutdown_connections 2024-09-11T23:20:21.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:21.431+0000 7fd3b422d640 1 -- 172.21.15.43:0/1826774920 wait complete. 2024-09-11T23:20:22.109 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:21 smithi043 ceph-mon[30950]: pgmap v292: 97 pgs: 97 active+clean; 602 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-11T23:20:22.109 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:21 smithi043 ceph-mon[30950]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:20:22.141 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:20:22.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:21 smithi090 ceph-mon[35130]: pgmap v292: 97 pgs: 97 active+clean; 602 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-11T23:20:22.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:21 smithi090 ceph-mon[35130]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:20:22.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:21 smithi097 ceph-mon[35562]: pgmap v292: 97 pgs: 97 active+clean; 602 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-11T23:20:22.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:21 smithi097 ceph-mon[35562]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:20:22.400 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:22.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.911+0000 7fafb9fad640 1 -- 172.21.15.43:0/2204702152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb4102d40 msgr2=0x7fafb4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:22.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.911+0000 7fafb9fad640 1 --2- 172.21.15.43:0/2204702152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb4102d40 0x7fafb4103120 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009900 tx=0x7fafa402a480 comp rx=0 tx=0).stop 2024-09-11T23:20:22.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.911+0000 7fafb9fad640 1 -- 172.21.15.43:0/2204702152 shutdown_connections 2024-09-11T23:20:22.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.911+0000 7fafb9fad640 1 --2- 172.21.15.43:0/2204702152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafb410c170 0x7fafb410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:22.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.911+0000 7fafb9fad640 1 --2- 172.21.15.43:0/2204702152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb4103660 0x7fafb410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:22.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.911+0000 7fafb9fad640 1 --2- 172.21.15.43:0/2204702152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb4102d40 0x7fafb4103120 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:22.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.911+0000 7fafb9fad640 1 -- 172.21.15.43:0/2204702152 >> 172.21.15.43:0/2204702152 conn(0x7fafb40fc7a0 msgr2=0x7fafb40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:22.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.911+0000 7fafb9fad640 1 -- 172.21.15.43:0/2204702152 shutdown_connections 2024-09-11T23:20:22.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.911+0000 7fafb9fad640 1 -- 172.21.15.43:0/2204702152 wait complete. 2024-09-11T23:20:22.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.912+0000 7fafb9fad640 1 Processor -- start 2024-09-11T23:20:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.912+0000 7fafb9fad640 1 -- start start 2024-09-11T23:20:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.912+0000 7fafb9fad640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafb4102d40 0x7fafb4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.912+0000 7fafb9fad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb4103660 0x7fafb41989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.912+0000 7fafb9fad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb410c170 0x7fafb419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.912+0000 7fafb9fad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fafb4111f70 con 0x7fafb410c170 2024-09-11T23:20:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.912+0000 7fafb9fad640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fafb4111df0 con 0x7fafb4102d40 2024-09-11T23:20:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.912+0000 7fafb9fad640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fafb41120f0 con 0x7fafb4103660 2024-09-11T23:20:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.912+0000 7fafb3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb410c170 0x7fafb419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.912+0000 7fafb3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb410c170 0x7fafb419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39922/0 (socket says 172.21.15.43:39922) 2024-09-11T23:20:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.912+0000 7fafb3fff640 1 -- 172.21.15.43:0/1376753852 learned_addr learned my addr 172.21.15.43:0/1376753852 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.912+0000 7fafb37fe640 1 --2- 172.21.15.43:0/1376753852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafb4102d40 0x7fafb4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.913+0000 7fafb2ffd640 1 --2- 172.21.15.43:0/1376753852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb4103660 0x7fafb41989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.913+0000 7fafb3fff640 1 -- 172.21.15.43:0/1376753852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb4103660 msgr2=0x7fafb41989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.913+0000 7fafb3fff640 1 --2- 172.21.15.43:0/1376753852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb4103660 0x7fafb41989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.913+0000 7fafb3fff640 1 -- 172.21.15.43:0/1376753852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafb4102d40 msgr2=0x7fafb4198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.913+0000 7fafb3fff640 1 --2- 172.21.15.43:0/1376753852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafb4102d40 0x7fafb4198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.913+0000 7fafb3fff640 1 -- 172.21.15.43:0/1376753852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb419d420 con 0x7fafb410c170 2024-09-11T23:20:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.913+0000 7fafb2ffd640 1 --2- 172.21.15.43:0/1376753852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb4103660 0x7fafb41989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:20:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.913+0000 7fafb3fff640 1 --2- 172.21.15.43:0/1376753852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb410c170 0x7fafb419cd40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fafac00eaa0 tx=0x7fafac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.913+0000 7fafb0ff9640 1 -- 172.21.15.43:0/1376753852 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac00cdb0 con 0x7fafb410c170 2024-09-11T23:20:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.913+0000 7fafb0ff9640 1 -- 172.21.15.43:0/1376753852 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafac005020 con 0x7fafb410c170 2024-09-11T23:20:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.913+0000 7fafb9fad640 1 -- 172.21.15.43:0/1376753852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb419d710 con 0x7fafb410c170 2024-09-11T23:20:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.913+0000 7fafb0ff9640 1 -- 172.21.15.43:0/1376753852 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac0116b0 con 0x7fafb410c170 2024-09-11T23:20:22.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.913+0000 7fafb9fad640 1 -- 172.21.15.43:0/1376753852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb41a4ff0 con 0x7fafb410c170 2024-09-11T23:20:22.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.914+0000 7fafb9fad640 1 -- 172.21.15.43:0/1376753852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb4103120 con 0x7fafb410c170 2024-09-11T23:20:22.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.915+0000 7fafb0ff9640 1 -- 172.21.15.43:0/1376753852 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fafac005560 con 0x7fafb410c170 2024-09-11T23:20:22.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.916+0000 7fafb0ff9640 1 --2- 172.21.15.43:0/1376753852 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7faf9c0a2c10 0x7faf9c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:22.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.916+0000 7fafb37fe640 1 --2- 172.21.15.43:0/1376753852 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7faf9c0a2c10 0x7faf9c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:22.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.917+0000 7fafb0ff9640 1 -- 172.21.15.43:0/1376753852 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fafac0cf210 con 0x7fafb410c170 2024-09-11T23:20:22.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.917+0000 7fafb37fe640 1 --2- 172.21.15.43:0/1376753852 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7faf9c0a2c10 0x7faf9c0a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009230 tx=0x7fafa40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:22.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:22.921+0000 7fafb0ff9640 1 -- 172.21.15.43:0/1376753852 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fafac016030 con 0x7fafb410c170 2024-09-11T23:20:23.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:23.138+0000 7fafb9fad640 1 -- 172.21.15.43:0/1376753852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafb402d7c0 con 0x7fafb410c170 2024-09-11T23:20:23.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:23.138+0000 7fafb0ff9640 1 -- 172.21.15.43:0/1376753852 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11896 (secure 0 0 0) 0x7fafac09c1c0 con 0x7fafb410c170 2024-09-11T23:20:23.139 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:20:23.139 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":23,"btime":"2024-09-11T23:20:12:774166+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:20:12.440327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24457,24470,24497]},"id":1}]} 2024-09-11T23:20:23.139 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 23 2024-09-11T23:20:23.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:23.141+0000 7fafb9fad640 1 -- 172.21.15.43:0/1376753852 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7faf9c0a2c10 msgr2=0x7faf9c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:23.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:23.141+0000 7fafb9fad640 1 --2- 172.21.15.43:0/1376753852 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7faf9c0a2c10 0x7faf9c0a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009230 tx=0x7fafa40058d0 comp rx=0 tx=0).stop 2024-09-11T23:20:23.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:23.141+0000 7fafb9fad640 1 -- 172.21.15.43:0/1376753852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb410c170 msgr2=0x7fafb419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:23.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:23.141+0000 7fafb9fad640 1 --2- 172.21.15.43:0/1376753852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb410c170 0x7fafb419cd40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fafac00eaa0 tx=0x7fafac00ef60 comp rx=0 tx=0).stop 2024-09-11T23:20:23.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:23.141+0000 7fafb9fad640 1 -- 172.21.15.43:0/1376753852 shutdown_connections 2024-09-11T23:20:23.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:23.141+0000 7fafb9fad640 1 --2- 172.21.15.43:0/1376753852 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7faf9c0a2c10 0x7faf9c0a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:23.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:23.141+0000 7fafb9fad640 1 --2- 172.21.15.43:0/1376753852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb410c170 0x7fafb419cd40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:23.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:23.141+0000 7fafb9fad640 1 --2- 172.21.15.43:0/1376753852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafb4103660 0x7fafb41989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:23.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:23.141+0000 7fafb9fad640 1 --2- 172.21.15.43:0/1376753852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafb4102d40 0x7fafb4198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:23.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:23.141+0000 7fafb9fad640 1 -- 172.21.15.43:0/1376753852 >> 172.21.15.43:0/1376753852 conn(0x7fafb40fc7a0 msgr2=0x7fafb40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:23.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:23.141+0000 7fafb9fad640 1 -- 172.21.15.43:0/1376753852 shutdown_connections 2024-09-11T23:20:23.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:23.141+0000 7fafb9fad640 1 -- 172.21.15.43:0/1376753852 wait complete. 2024-09-11T23:20:23.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:20:24.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:23 smithi097 ceph-mon[35562]: pgmap v293: 97 pgs: 97 active+clean; 622 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-09-11T23:20:24.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:23 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1376753852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:20:24.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:23 smithi043 ceph-mon[30950]: pgmap v293: 97 pgs: 97 active+clean; 622 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-09-11T23:20:24.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:23 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1376753852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:20:24.207 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:24.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:23 smithi090 ceph-mon[35130]: pgmap v293: 97 pgs: 97 active+clean; 622 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-09-11T23:20:24.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:23 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1376753852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:20:24.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.695+0000 7ff148ea7640 1 -- 172.21.15.43:0/1040651845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff144102d40 msgr2=0x7ff144103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:24.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.695+0000 7ff148ea7640 1 --2- 172.21.15.43:0/1040651845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff144102d40 0x7ff144103120 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff134009860 tx=0x7ff13402a3d0 comp rx=0 tx=0).stop 2024-09-11T23:20:24.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.695+0000 7ff148ea7640 1 -- 172.21.15.43:0/1040651845 shutdown_connections 2024-09-11T23:20:24.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.695+0000 7ff148ea7640 1 --2- 172.21.15.43:0/1040651845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff14410c170 0x7ff14410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:24.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.695+0000 7ff148ea7640 1 --2- 172.21.15.43:0/1040651845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff144103660 0x7ff14410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:24.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.695+0000 7ff148ea7640 1 --2- 172.21.15.43:0/1040651845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff144102d40 0x7ff144103120 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:24.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.695+0000 7ff148ea7640 1 -- 172.21.15.43:0/1040651845 >> 172.21.15.43:0/1040651845 conn(0x7ff1440fc7a0 msgr2=0x7ff1440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:24.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.695+0000 7ff148ea7640 1 -- 172.21.15.43:0/1040651845 shutdown_connections 2024-09-11T23:20:24.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.695+0000 7ff148ea7640 1 -- 172.21.15.43:0/1040651845 wait complete. 2024-09-11T23:20:24.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.696+0000 7ff148ea7640 1 Processor -- start 2024-09-11T23:20:24.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.696+0000 7ff148ea7640 1 -- start start 2024-09-11T23:20:24.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.696+0000 7ff148ea7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff144102d40 0x7ff144198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:24.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.696+0000 7ff148ea7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff144103660 0x7ff144198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:24.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.696+0000 7ff148ea7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff14410c170 0x7ff14419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:24.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.696+0000 7ff148ea7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff144111f70 con 0x7ff144102d40 2024-09-11T23:20:24.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.696+0000 7ff148ea7640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff144111df0 con 0x7ff14410c170 2024-09-11T23:20:24.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.696+0000 7ff148ea7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff1441120f0 con 0x7ff144103660 2024-09-11T23:20:24.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.697+0000 7ff14259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff144102d40 0x7ff144198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:24.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.697+0000 7ff14259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff144102d40 0x7ff144198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39942/0 (socket says 172.21.15.43:39942) 2024-09-11T23:20:24.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.697+0000 7ff14259c640 1 -- 172.21.15.43:0/2315046557 learned_addr learned my addr 172.21.15.43:0/2315046557 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:24.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.697+0000 7ff141d9b640 1 --2- 172.21.15.43:0/2315046557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff144103660 0x7ff144198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:24.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.697+0000 7ff142d9d640 1 --2- 172.21.15.43:0/2315046557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff14410c170 0x7ff14419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:24.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.697+0000 7ff14259c640 1 -- 172.21.15.43:0/2315046557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff144103660 msgr2=0x7ff144198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:24.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.697+0000 7ff14259c640 1 --2- 172.21.15.43:0/2315046557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff144103660 0x7ff144198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:24.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.697+0000 7ff14259c640 1 -- 172.21.15.43:0/2315046557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff14410c170 msgr2=0x7ff14419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:24.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.697+0000 7ff14259c640 1 --2- 172.21.15.43:0/2315046557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff14410c170 0x7ff14419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:24.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.697+0000 7ff14259c640 1 -- 172.21.15.43:0/2315046557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14419d400 con 0x7ff144102d40 2024-09-11T23:20:24.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.697+0000 7ff14259c640 1 --2- 172.21.15.43:0/2315046557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff144102d40 0x7ff144198450 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff13402a8f0 tx=0x7ff134030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:24.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.697+0000 7ff1337fe640 1 -- 172.21.15.43:0/2315046557 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff134040020 con 0x7ff144102d40 2024-09-11T23:20:24.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.697+0000 7ff1337fe640 1 -- 172.21.15.43:0/2315046557 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff134004460 con 0x7ff144102d40 2024-09-11T23:20:24.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.697+0000 7ff148ea7640 1 -- 172.21.15.43:0/2315046557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14419d690 con 0x7ff144102d40 2024-09-11T23:20:24.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.697+0000 7ff1337fe640 1 -- 172.21.15.43:0/2315046557 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1340046c0 con 0x7ff144102d40 2024-09-11T23:20:24.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.697+0000 7ff148ea7640 1 -- 172.21.15.43:0/2315046557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1441a4fd0 con 0x7ff144102d40 2024-09-11T23:20:24.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.698+0000 7ff148ea7640 1 -- 172.21.15.43:0/2315046557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff14406b070 con 0x7ff144102d40 2024-09-11T23:20:24.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.700+0000 7ff1337fe640 1 -- 172.21.15.43:0/2315046557 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff134043020 con 0x7ff144102d40 2024-09-11T23:20:24.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.700+0000 7ff1337fe640 1 --2- 172.21.15.43:0/2315046557 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff12c0a2c10 0x7ff12c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:24.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.700+0000 7ff141d9b640 1 --2- 172.21.15.43:0/2315046557 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff12c0a2c10 0x7ff12c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:24.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.700+0000 7ff1337fe640 1 -- 172.21.15.43:0/2315046557 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7ff1340eff40 con 0x7ff144102d40 2024-09-11T23:20:24.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.701+0000 7ff141d9b640 1 --2- 172.21.15.43:0/2315046557 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff12c0a2c10 0x7ff12c0a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff1440feda0 tx=0x7ff138005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:24.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.705+0000 7ff1337fe640 1 -- 172.21.15.43:0/2315046557 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff13404a020 con 0x7ff144102d40 2024-09-11T23:20:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.882+0000 7ff148ea7640 1 -- 172.21.15.43:0/2315046557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1441071e0 con 0x7ff144102d40 2024-09-11T23:20:24.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.883+0000 7ff1337fe640 1 -- 172.21.15.43:0/2315046557 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22145 (secure 0 0 0) 0x7ff1340bcef0 con 0x7ff144102d40 2024-09-11T23:20:24.884 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:20:24.884 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":82,"fsid":"25c0cd84-7093-11ef-bcea-c7b262605968","created":"2024-09-11T23:12:03.504406+0000","modified":"2024-09-11T23:20:00.465617+0000","last_up_change":"2024-09-11T23:19:07.797873+0000","last_in_change":"2024-09-11T23:18:44.595176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:15:17.058713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:19:56.221406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:19:58.705875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24076596-3909-4c87-85f1-cbf168070dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6803","nonce":43839997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6805","nonce":43839997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6809","nonce":43839997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6807","nonce":43839997}]},"public_addr":"172.21.15.43:6803/43839997","cluster_addr":"172.21.15.43:6805/43839997","heartbeat_back_addr":"172.21.15.43:6809/43839997","heartbeat_front_addr":"172.21.15.43:6807/43839997","state":["exists","up"]},{"osd":1,"uuid":"a0472a57-243c-489d-a247-972ee10a71f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6801","nonce":1825149246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6803","nonce":1825149246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6807","nonce":1825149246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6805","nonce":1825149246}]},"public_addr":"172.21.15.90:6801/1825149246","cluster_addr":"172.21.15.90:6803/1825149246","heartbeat_back_addr":"172.21.15.90:6807/1825149246","heartbeat_front_addr":"172.21.15.90:6805/1825149246","state":["exists","up"]},{"osd":2,"uuid":"40366536-0fcf-45a0-b8e3-49178401dc34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6801","nonce":1456846443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6803","nonce":1456846443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6807","nonce":1456846443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6805","nonce":1456846443}]},"public_addr":"172.21.15.97:6801/1456846443","cluster_addr":"172.21.15.97:6803/1456846443","heartbeat_back_addr":"172.21.15.97:6807/1456846443","heartbeat_front_addr":"172.21.15.97:6805/1456846443","state":["exists","up"]},{"osd":3,"uuid":"3062133e-000b-4421-9d53-ac434b9ed9b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6811","nonce":1962837804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6813","nonce":1962837804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6817","nonce":1962837804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6815","nonce":1962837804}]},"public_addr":"172.21.15.43:6811/1962837804","cluster_addr":"172.21.15.43:6813/1962837804","heartbeat_back_addr":"172.21.15.43:6817/1962837804","heartbeat_front_addr":"172.21.15.43:6815/1962837804","state":["exists","up"]},{"osd":4,"uuid":"2e73fa4b-f22c-4319-828d-87b91b9f19f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6809","nonce":181614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6811","nonce":181614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6815","nonce":181614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6813","nonce":181614173}]},"public_addr":"172.21.15.90:6809/181614173","cluster_addr":"172.21.15.90:6811/181614173","heartbeat_back_addr":"172.21.15.90:6815/181614173","heartbeat_front_addr":"172.21.15.90:6813/181614173","state":["exists","up"]},{"osd":5,"uuid":"1e1ad4f9-e724-440f-b515-3ec1ad97d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6809","nonce":1540039543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6811","nonce":1540039543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6815","nonce":1540039543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6813","nonce":1540039543}]},"public_addr":"172.21.15.97:6809/1540039543","cluster_addr":"172.21.15.97:6811/1540039543","heartbeat_back_addr":"172.21.15.97:6815/1540039543","heartbeat_front_addr":"172.21.15.97:6813/1540039543","state":["exists","up"]},{"osd":6,"uuid":"d34df8df-f40d-4159-8b29-5401b1b0e431","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6819","nonce":3785677637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6821","nonce":3785677637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6825","nonce":3785677637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6823","nonce":3785677637}]},"public_addr":"172.21.15.43:6819/3785677637","cluster_addr":"172.21.15.43:6821/3785677637","heartbeat_back_addr":"172.21.15.43:6825/3785677637","heartbeat_front_addr":"172.21.15.43:6823/3785677637","state":["exists","up"]},{"osd":7,"uuid":"b2e01078-528b-49fd-b000-9f584f784b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6817","nonce":236874962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6819","nonce":236874962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6823","nonce":236874962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6821","nonce":236874962}]},"public_addr":"172.21.15.90:6817/236874962","cluster_addr":"172.21.15.90:6819/236874962","heartbeat_back_addr":"172.21.15.90:6823/236874962","heartbeat_front_addr":"172.21.15.90:6821/236874962","state":["exists","up"]},{"osd":8,"uuid":"abef8f27-bf9b-4d63-bfbe-f967fc3d8b17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6817","nonce":1707327301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6819","nonce":1707327301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6823","nonce":1707327301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6821","nonce":1707327301}]},"public_addr":"172.21.15.97:6817/1707327301","cluster_addr":"172.21.15.97:6819/1707327301","heartbeat_back_addr":"172.21.15.97:6823/1707327301","heartbeat_front_addr":"172.21.15.97:6821/1707327301","state":["exists","up"]},{"osd":9,"uuid":"190ab8b2-a835-47e6-9b39-16a58331c58a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6827","nonce":3170306011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6829","nonce":3170306011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6833","nonce":3170306011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6831","nonce":3170306011}]},"public_addr":"172.21.15.43:6827/3170306011","cluster_addr":"172.21.15.43:6829/3170306011","heartbeat_back_addr":"172.21.15.43:6833/3170306011","heartbeat_front_addr":"172.21.15.43:6831/3170306011","state":["exists","up"]},{"osd":10,"uuid":"b32ab5fe-466f-4c8e-8696-cd9f820a3263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6825","nonce":59487041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6827","nonce":59487041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6831","nonce":59487041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6829","nonce":59487041}]},"public_addr":"172.21.15.90:6825/59487041","cluster_addr":"172.21.15.90:6827/59487041","heartbeat_back_addr":"172.21.15.90:6831/59487041","heartbeat_front_addr":"172.21.15.90:6829/59487041","state":["exists","up"]},{"osd":11,"uuid":"d0faa589-21c8-4a9f-8a73-813aafe6d33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6825","nonce":194699315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6827","nonce":194699315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6831","nonce":194699315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6829","nonce":194699315}]},"public_addr":"172.21.15.97:6825/194699315","cluster_addr":"172.21.15.97:6827/194699315","heartbeat_back_addr":"172.21.15.97:6831/194699315","heartbeat_front_addr":"172.21.15.97:6829/194699315","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:23.381247+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:48.843190+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:13.538979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:40.136983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:05.513790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:31.247090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:56.635221+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:22.667327+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:47.910422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:14.353914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:40.584503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:19:06.476766+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/76575371":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/4006124758":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/120495444":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6800/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:6800/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/206044413":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1512201905":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1375379108":"2024-09-12T23:12:35.756598+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:20:24.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.886+0000 7ff148ea7640 1 -- 172.21.15.43:0/2315046557 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff12c0a2c10 msgr2=0x7ff12c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:24.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.886+0000 7ff148ea7640 1 --2- 172.21.15.43:0/2315046557 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff12c0a2c10 0x7ff12c0a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff1440feda0 tx=0x7ff138005f90 comp rx=0 tx=0).stop 2024-09-11T23:20:24.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.886+0000 7ff148ea7640 1 -- 172.21.15.43:0/2315046557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff144102d40 msgr2=0x7ff144198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:24.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.886+0000 7ff148ea7640 1 --2- 172.21.15.43:0/2315046557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff144102d40 0x7ff144198450 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff13402a8f0 tx=0x7ff134030650 comp rx=0 tx=0).stop 2024-09-11T23:20:24.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.886+0000 7ff148ea7640 1 -- 172.21.15.43:0/2315046557 shutdown_connections 2024-09-11T23:20:24.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.886+0000 7ff148ea7640 1 --2- 172.21.15.43:0/2315046557 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff12c0a2c10 0x7ff12c0a50d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:24.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.886+0000 7ff148ea7640 1 --2- 172.21.15.43:0/2315046557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff14410c170 0x7ff14419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:24.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.886+0000 7ff148ea7640 1 --2- 172.21.15.43:0/2315046557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff144103660 0x7ff144198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:24.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.886+0000 7ff148ea7640 1 --2- 172.21.15.43:0/2315046557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff144102d40 0x7ff144198450 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:24.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.886+0000 7ff148ea7640 1 -- 172.21.15.43:0/2315046557 >> 172.21.15.43:0/2315046557 conn(0x7ff1440fc7a0 msgr2=0x7ff1440fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:24.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.886+0000 7ff148ea7640 1 -- 172.21.15.43:0/2315046557 shutdown_connections 2024-09-11T23:20:24.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:24.886+0000 7ff148ea7640 1 -- 172.21.15.43:0/2315046557 wait complete. 2024-09-11T23:20:25.682 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T23:20:25.682 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:20:25.948 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:26.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:25 smithi043 ceph-mon[30950]: pgmap v294: 97 pgs: 97 active+clean; 622 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-09-11T23:20:26.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:25 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2315046557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:20:26.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:25 smithi090 ceph-mon[35130]: pgmap v294: 97 pgs: 97 active+clean; 622 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-09-11T23:20:26.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:25 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2315046557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:20:26.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:25 smithi097 ceph-mon[35562]: pgmap v294: 97 pgs: 97 active+clean; 622 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-09-11T23:20:26.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:25 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2315046557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:20:26.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.476+0000 7f40c6172640 1 -- 172.21.15.43:0/3770509180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c01036e0 msgr2=0x7f40c0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:26.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.476+0000 7f40c6172640 1 --2- 172.21.15.43:0/3770509180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c01036e0 0x7f40c0107b90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009930 tx=0x7f40a802a420 comp rx=0 tx=0).stop 2024-09-11T23:20:26.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.476+0000 7f40c6172640 1 -- 172.21.15.43:0/3770509180 shutdown_connections 2024-09-11T23:20:26.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.476+0000 7f40c6172640 1 --2- 172.21.15.43:0/3770509180 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40c01082a0 0x7f40c010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:26.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.476+0000 7f40c6172640 1 --2- 172.21.15.43:0/3770509180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c01036e0 0x7f40c0107b90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:26.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.476+0000 7f40c6172640 1 --2- 172.21.15.43:0/3770509180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40c0102d30 0x7f40c0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:26.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.476+0000 7f40c6172640 1 -- 172.21.15.43:0/3770509180 >> 172.21.15.43:0/3770509180 conn(0x7f40c00fc7d0 msgr2=0x7f40c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:26.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.477+0000 7f40c6172640 1 -- 172.21.15.43:0/3770509180 shutdown_connections 2024-09-11T23:20:26.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.477+0000 7f40c6172640 1 -- 172.21.15.43:0/3770509180 wait complete. 2024-09-11T23:20:26.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.477+0000 7f40c6172640 1 Processor -- start 2024-09-11T23:20:26.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.477+0000 7f40c6172640 1 -- start start 2024-09-11T23:20:26.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.478+0000 7f40c6172640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40c0102d30 0x7f40c0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:26.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.478+0000 7f40c6172640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c01036e0 0x7f40c0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:26.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.478+0000 7f40c6172640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40c01082a0 0x7f40c0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:26.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.478+0000 7f40c6172640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f40c01120b0 con 0x7f40c01036e0 2024-09-11T23:20:26.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.478+0000 7f40c6172640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f40c0111f30 con 0x7f40c01082a0 2024-09-11T23:20:26.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.478+0000 7f40c6172640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f40c0112230 con 0x7f40c0102d30 2024-09-11T23:20:26.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.478+0000 7f40beffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c01036e0 0x7f40c0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:26.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.478+0000 7f40bf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40c0102d30 0x7f40c0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:26.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.478+0000 7f40beffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c01036e0 0x7f40c0100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39972/0 (socket says 172.21.15.43:39972) 2024-09-11T23:20:26.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.478+0000 7f40beffd640 1 -- 172.21.15.43:0/1607150912 learned_addr learned my addr 172.21.15.43:0/1607150912 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:26.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.478+0000 7f40bffff640 1 --2- 172.21.15.43:0/1607150912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40c01082a0 0x7f40c0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:26.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.478+0000 7f40beffd640 1 -- 172.21.15.43:0/1607150912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40c0102d30 msgr2=0x7f40c0100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:26.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.478+0000 7f40beffd640 1 --2- 172.21.15.43:0/1607150912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40c0102d30 0x7f40c0100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:26.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.478+0000 7f40beffd640 1 -- 172.21.15.43:0/1607150912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40c01082a0 msgr2=0x7f40c0101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:26.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.478+0000 7f40beffd640 1 --2- 172.21.15.43:0/1607150912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40c01082a0 0x7f40c0101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:26.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.478+0000 7f40beffd640 1 -- 172.21.15.43:0/1607150912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c01a6a70 con 0x7f40c01036e0 2024-09-11T23:20:26.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.478+0000 7f40beffd640 1 --2- 172.21.15.43:0/1607150912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c01036e0 0x7f40c0100ed0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f40a802a940 tx=0x7f40a8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:26.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.479+0000 7f40bcff9640 1 -- 172.21.15.43:0/1607150912 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a803f070 con 0x7f40c01036e0 2024-09-11T23:20:26.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.479+0000 7f40c6172640 1 -- 172.21.15.43:0/1607150912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c01a6d00 con 0x7f40c01036e0 2024-09-11T23:20:26.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.479+0000 7f40bcff9640 1 -- 172.21.15.43:0/1607150912 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40a8031c90 con 0x7f40c01036e0 2024-09-11T23:20:26.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.479+0000 7f40bcff9640 1 -- 172.21.15.43:0/1607150912 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8035030 con 0x7f40c01036e0 2024-09-11T23:20:26.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.479+0000 7f40c6172640 1 -- 172.21.15.43:0/1607150912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c01a71d0 con 0x7f40c01036e0 2024-09-11T23:20:26.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.480+0000 7f40c6172640 1 -- 172.21.15.43:0/1607150912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40c0103110 con 0x7f40c01036e0 2024-09-11T23:20:26.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.482+0000 7f40bcff9640 1 -- 172.21.15.43:0/1607150912 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f40a8042070 con 0x7f40c01036e0 2024-09-11T23:20:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.483+0000 7f40bcff9640 1 --2- 172.21.15.43:0/1607150912 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f40ac0a2c90 0x7f40ac0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.483+0000 7f40bf7fe640 1 --2- 172.21.15.43:0/1607150912 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f40ac0a2c90 0x7f40ac0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.483+0000 7f40bcff9640 1 -- 172.21.15.43:0/1607150912 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f40a80efb20 con 0x7f40c01036e0 2024-09-11T23:20:26.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.484+0000 7f40bf7fe640 1 --2- 172.21.15.43:0/1607150912 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f40ac0a2c90 0x7f40ac0a5150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f40b0005fd0 tx=0x7f40b000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:26.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.486+0000 7f40bcff9640 1 -- 172.21.15.43:0/1607150912 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40a80bc480 con 0x7f40c01036e0 2024-09-11T23:20:26.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.659+0000 7f40c6172640 1 -- 172.21.15.43:0/1607150912 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f40c0062630 con 0x7f40ac0a2c90 2024-09-11T23:20:26.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.674+0000 7f40bcff9640 1 -- 172.21.15.43:0/1607150912 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258862 (secure 0 0 0) 0x7f40c0062630 con 0x7f40ac0a2c90 2024-09-11T23:20:26.677 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:20:26.678 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-11T23:20:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.678+0000 7f40c6172640 1 -- 172.21.15.43:0/1607150912 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f40ac0a2c90 msgr2=0x7f40ac0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.678+0000 7f40c6172640 1 --2- 172.21.15.43:0/1607150912 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f40ac0a2c90 0x7f40ac0a5150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f40b0005fd0 tx=0x7f40b000b390 comp rx=0 tx=0).stop 2024-09-11T23:20:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.678+0000 7f40c6172640 1 -- 172.21.15.43:0/1607150912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c01036e0 msgr2=0x7f40c0100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.678+0000 7f40c6172640 1 --2- 172.21.15.43:0/1607150912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c01036e0 0x7f40c0100ed0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f40a802a940 tx=0x7f40a8003b90 comp rx=0 tx=0).stop 2024-09-11T23:20:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.678+0000 7f40c6172640 1 -- 172.21.15.43:0/1607150912 shutdown_connections 2024-09-11T23:20:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.678+0000 7f40c6172640 1 --2- 172.21.15.43:0/1607150912 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f40ac0a2c90 0x7f40ac0a5150 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.678+0000 7f40c6172640 1 --2- 172.21.15.43:0/1607150912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40c01082a0 0x7f40c0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.678+0000 7f40c6172640 1 --2- 172.21.15.43:0/1607150912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c01036e0 0x7f40c0100ed0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.678+0000 7f40c6172640 1 --2- 172.21.15.43:0/1607150912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40c0102d30 0x7f40c0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:26.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.678+0000 7f40c6172640 1 -- 172.21.15.43:0/1607150912 >> 172.21.15.43:0/1607150912 conn(0x7f40c00fc7d0 msgr2=0x7f40c0106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:26.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.678+0000 7f40c6172640 1 -- 172.21.15.43:0/1607150912 shutdown_connections 2024-09-11T23:20:26.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:26.678+0000 7f40c6172640 1 -- 172.21.15.43:0/1607150912 wait complete. 2024-09-11T23:20:27.499 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-09-11T23:20:26.638306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636808,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":232,"num_read_kb":216,"num_write":336,"num_write_kb":3027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368440,"kb_used_data":10872,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573384,"statfs":{"total":1151940427776,"available":1151563145216,"internally_reserved":0,"allocated":11132928,"data_stored":7023410,"data_compressed":108948,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41218,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":72,"num_write":30,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003176"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.731685+0000","last_change":"2024-09-11T23:19:59.734378+0000","last_active":"2024-09-11T23:20:00.731685+0000","last_peered":"2024-09-11T23:20:00.731685+0000","last_clean":"2024-09-11T23:20:00.731685+0000","last_became_active":"2024-09-11T23:19:59.734067+0000","last_became_peered":"2024-09-11T23:19:59.734067+0000","last_unstale":"2024-09-11T23:20:00.731685+0000","last_undegraded":"2024-09-11T23:20:00.731685+0000","last_fullsized":"2024-09-11T23:20:00.731685+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:28:18.384453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746459+0000","last_change":"2024-09-11T23:19:59.733892+0000","last_active":"2024-09-11T23:20:00.746459+0000","last_peered":"2024-09-11T23:20:00.746459+0000","last_clean":"2024-09-11T23:20:00.746459+0000","last_became_active":"2024-09-11T23:19:59.733540+0000","last_became_peered":"2024-09-11T23:19:59.733540+0000","last_unstale":"2024-09-11T23:20:00.746459+0000","last_undegraded":"2024-09-11T23:20:00.746459+0000","last_fullsized":"2024-09-11T23:20:00.746459+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:42.186458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746491+0000","last_change":"2024-09-11T23:19:59.739443+0000","last_active":"2024-09-11T23:20:00.746491+0000","last_peered":"2024-09-11T23:20:00.746491+0000","last_clean":"2024-09-11T23:20:00.746491+0000","last_became_active":"2024-09-11T23:19:59.739025+0000","last_became_peered":"2024-09-11T23:19:59.739025+0000","last_unstale":"2024-09-11T23:20:00.746491+0000","last_undegraded":"2024-09-11T23:20:00.746491+0000","last_fullsized":"2024-09-11T23:20:00.746491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:40.819321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.731171+0000","last_change":"2024-09-11T23:19:59.737076+0000","last_active":"2024-09-11T23:20:00.731171+0000","last_peered":"2024-09-11T23:20:00.731171+0000","last_clean":"2024-09-11T23:20:00.731171+0000","last_became_active":"2024-09-11T23:19:59.736222+0000","last_became_peered":"2024-09-11T23:19:59.736222+0000","last_unstale":"2024-09-11T23:20:00.731171+0000","last_undegraded":"2024-09-11T23:20:00.731171+0000","last_fullsized":"2024-09-11T23:20:00.731171+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:33:13.158825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744063+0000","last_change":"2024-09-11T23:19:59.738662+0000","last_active":"2024-09-11T23:20:00.744063+0000","last_peered":"2024-09-11T23:20:00.744063+0000","last_clean":"2024-09-11T23:20:00.744063+0000","last_became_active":"2024-09-11T23:19:59.738320+0000","last_became_peered":"2024-09-11T23:19:59.738320+0000","last_unstale":"2024-09-11T23:20:00.744063+0000","last_undegraded":"2024-09-11T23:20:00.744063+0000","last_fullsized":"2024-09-11T23:20:00.744063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:15.645179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.742212+0000","last_change":"2024-09-11T23:19:59.731442+0000","last_active":"2024-09-11T23:20:00.742212+0000","last_peered":"2024-09-11T23:20:00.742212+0000","last_clean":"2024-09-11T23:20:00.742212+0000","last_became_active":"2024-09-11T23:19:59.731079+0000","last_became_peered":"2024-09-11T23:19:59.731079+0000","last_unstale":"2024-09-11T23:20:00.742212+0000","last_undegraded":"2024-09-11T23:20:00.742212+0000","last_fullsized":"2024-09-11T23:20:00.742212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:32.764991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746867+0000","last_change":"2024-09-11T23:19:59.743734+0000","last_active":"2024-09-11T23:20:00.746867+0000","last_peered":"2024-09-11T23:20:00.746867+0000","last_clean":"2024-09-11T23:20:00.746867+0000","last_became_active":"2024-09-11T23:19:59.743387+0000","last_became_peered":"2024-09-11T23:19:59.743387+0000","last_unstale":"2024-09-11T23:20:00.746867+0000","last_undegraded":"2024-09-11T23:20:00.746867+0000","last_fullsized":"2024-09-11T23:20:00.746867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:20.515745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.743887+0000","last_change":"2024-09-11T23:19:59.740733+0000","last_active":"2024-09-11T23:20:00.743887+0000","last_peered":"2024-09-11T23:20:00.743887+0000","last_clean":"2024-09-11T23:20:00.743887+0000","last_became_active":"2024-09-11T23:19:59.740438+0000","last_became_peered":"2024-09-11T23:19:59.740438+0000","last_unstale":"2024-09-11T23:20:00.743887+0000","last_undegraded":"2024-09-11T23:20:00.743887+0000","last_fullsized":"2024-09-11T23:20:00.743887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:27.063305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746418+0000","last_change":"2024-09-11T23:19:59.742439+0000","last_active":"2024-09-11T23:20:00.746418+0000","last_peered":"2024-09-11T23:20:00.746418+0000","last_clean":"2024-09-11T23:20:00.746418+0000","last_became_active":"2024-09-11T23:19:59.741983+0000","last_became_peered":"2024-09-11T23:19:59.741983+0000","last_unstale":"2024-09-11T23:20:00.746418+0000","last_undegraded":"2024-09-11T23:20:00.746418+0000","last_fullsized":"2024-09-11T23:20:00.746418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:39.477167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.742710+0000","last_change":"2024-09-11T23:19:59.732544+0000","last_active":"2024-09-11T23:20:00.742710+0000","last_peered":"2024-09-11T23:20:00.742710+0000","last_clean":"2024-09-11T23:20:00.742710+0000","last_became_active":"2024-09-11T23:19:59.732205+0000","last_became_peered":"2024-09-11T23:19:59.732205+0000","last_unstale":"2024-09-11T23:20:00.742710+0000","last_undegraded":"2024-09-11T23:20:00.742710+0000","last_fullsized":"2024-09-11T23:20:00.742710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:22.494344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.747452+0000","last_change":"2024-09-11T23:19:59.741857+0000","last_active":"2024-09-11T23:20:00.747452+0000","last_peered":"2024-09-11T23:20:00.747452+0000","last_clean":"2024-09-11T23:20:00.747452+0000","last_became_active":"2024-09-11T23:19:59.741484+0000","last_became_peered":"2024-09-11T23:19:59.741484+0000","last_unstale":"2024-09-11T23:20:00.747452+0000","last_undegraded":"2024-09-11T23:20:00.747452+0000","last_fullsized":"2024-09-11T23:20:00.747452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:52.998258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.731332+0000","last_change":"2024-09-11T23:19:59.736042+0000","last_active":"2024-09-11T23:20:00.731332+0000","last_peered":"2024-09-11T23:20:00.731332+0000","last_clean":"2024-09-11T23:20:00.731332+0000","last_became_active":"2024-09-11T23:19:59.735731+0000","last_became_peered":"2024-09-11T23:19:59.735731+0000","last_unstale":"2024-09-11T23:20:00.731332+0000","last_undegraded":"2024-09-11T23:20:00.731332+0000","last_fullsized":"2024-09-11T23:20:00.731332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:19.085804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.728381+0000","last_change":"2024-09-11T23:19:59.742049+0000","last_active":"2024-09-11T23:20:00.728381+0000","last_peered":"2024-09-11T23:20:00.728381+0000","last_clean":"2024-09-11T23:20:00.728381+0000","last_became_active":"2024-09-11T23:19:59.740974+0000","last_became_peered":"2024-09-11T23:19:59.740974+0000","last_unstale":"2024-09-11T23:20:00.728381+0000","last_undegraded":"2024-09-11T23:20:00.728381+0000","last_fullsized":"2024-09-11T23:20:00.728381+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:10.443113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746606+0000","last_change":"2024-09-11T23:19:59.734270+0000","last_active":"2024-09-11T23:20:00.746606+0000","last_peered":"2024-09-11T23:20:00.746606+0000","last_clean":"2024-09-11T23:20:00.746606+0000","last_became_active":"2024-09-11T23:19:59.733967+0000","last_became_peered":"2024-09-11T23:19:59.733967+0000","last_unstale":"2024-09-11T23:20:00.746606+0000","last_undegraded":"2024-09-11T23:20:00.746606+0000","last_fullsized":"2024-09-11T23:20:00.746606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:02.185757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746434+0000","last_change":"2024-09-11T23:19:59.734152+0000","last_active":"2024-09-11T23:20:00.746434+0000","last_peered":"2024-09-11T23:20:00.746434+0000","last_clean":"2024-09-11T23:20:00.746434+0000","last_became_active":"2024-09-11T23:19:59.733163+0000","last_became_peered":"2024-09-11T23:19:59.733163+0000","last_unstale":"2024-09-11T23:20:00.746434+0000","last_undegraded":"2024-09-11T23:20:00.746434+0000","last_fullsized":"2024-09-11T23:20:00.746434+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:22.370591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744672+0000","last_change":"2024-09-11T23:19:59.735517+0000","last_active":"2024-09-11T23:20:00.744672+0000","last_peered":"2024-09-11T23:20:00.744672+0000","last_clean":"2024-09-11T23:20:00.744672+0000","last_became_active":"2024-09-11T23:19:59.735182+0000","last_became_peered":"2024-09-11T23:19:59.735182+0000","last_unstale":"2024-09-11T23:20:00.744672+0000","last_undegraded":"2024-09-11T23:20:00.744672+0000","last_fullsized":"2024-09-11T23:20:00.744672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:25.989416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.731699+0000","last_change":"2024-09-11T23:19:59.733579+0000","last_active":"2024-09-11T23:20:00.731699+0000","last_peered":"2024-09-11T23:20:00.731699+0000","last_clean":"2024-09-11T23:20:00.731699+0000","last_became_active":"2024-09-11T23:19:59.733245+0000","last_became_peered":"2024-09-11T23:19:59.733245+0000","last_unstale":"2024-09-11T23:20:00.731699+0000","last_undegraded":"2024-09-11T23:20:00.731699+0000","last_fullsized":"2024-09-11T23:20:00.731699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:22.361158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.742621+0000","last_change":"2024-09-11T23:19:59.734354+0000","last_active":"2024-09-11T23:20:00.742621+0000","last_peered":"2024-09-11T23:20:00.742621+0000","last_clean":"2024-09-11T23:20:00.742621+0000","last_became_active":"2024-09-11T23:19:59.733988+0000","last_became_peered":"2024-09-11T23:19:59.733988+0000","last_unstale":"2024-09-11T23:20:00.742621+0000","last_undegraded":"2024-09-11T23:20:00.742621+0000","last_fullsized":"2024-09-11T23:20:00.742621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:41.401496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744515+0000","last_change":"2024-09-11T23:19:59.730318+0000","last_active":"2024-09-11T23:20:00.744515+0000","last_peered":"2024-09-11T23:20:00.744515+0000","last_clean":"2024-09-11T23:20:00.744515+0000","last_became_active":"2024-09-11T23:19:59.730009+0000","last_became_peered":"2024-09-11T23:19:59.730009+0000","last_unstale":"2024-09-11T23:20:00.744515+0000","last_undegraded":"2024-09-11T23:20:00.744515+0000","last_fullsized":"2024-09-11T23:20:00.744515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:01:38.531967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.731289+0000","last_change":"2024-09-11T23:19:59.737254+0000","last_active":"2024-09-11T23:20:00.731289+0000","last_peered":"2024-09-11T23:20:00.731289+0000","last_clean":"2024-09-11T23:20:00.731289+0000","last_became_active":"2024-09-11T23:19:59.736936+0000","last_became_peered":"2024-09-11T23:19:59.736936+0000","last_unstale":"2024-09-11T23:20:00.731289+0000","last_undegraded":"2024-09-11T23:20:00.731289+0000","last_fullsized":"2024-09-11T23:20:00.731289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:00.361468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.728507+0000","last_change":"2024-09-11T23:19:59.741617+0000","last_active":"2024-09-11T23:20:00.728507+0000","last_peered":"2024-09-11T23:20:00.728507+0000","last_clean":"2024-09-11T23:20:00.728507+0000","last_became_active":"2024-09-11T23:19:59.740358+0000","last_became_peered":"2024-09-11T23:19:59.740358+0000","last_unstale":"2024-09-11T23:20:00.728507+0000","last_undegraded":"2024-09-11T23:20:00.728507+0000","last_fullsized":"2024-09-11T23:20:00.728507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:18.541105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.745052+0000","last_change":"2024-09-11T23:19:59.736437+0000","last_active":"2024-09-11T23:20:00.745052+0000","last_peered":"2024-09-11T23:20:00.745052+0000","last_clean":"2024-09-11T23:20:00.745052+0000","last_became_active":"2024-09-11T23:19:59.736029+0000","last_became_peered":"2024-09-11T23:19:59.736029+0000","last_unstale":"2024-09-11T23:20:00.745052+0000","last_undegraded":"2024-09-11T23:20:00.745052+0000","last_fullsized":"2024-09-11T23:20:00.745052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:29.226939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744096+0000","last_change":"2024-09-11T23:19:59.736104+0000","last_active":"2024-09-11T23:20:00.744096+0000","last_peered":"2024-09-11T23:20:00.744096+0000","last_clean":"2024-09-11T23:20:00.744096+0000","last_became_active":"2024-09-11T23:19:59.735760+0000","last_became_peered":"2024-09-11T23:19:59.735760+0000","last_unstale":"2024-09-11T23:20:00.744096+0000","last_undegraded":"2024-09-11T23:20:00.744096+0000","last_fullsized":"2024-09-11T23:20:00.744096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:56.554313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.743042+0000","last_change":"2024-09-11T23:19:59.735455+0000","last_active":"2024-09-11T23:20:00.743042+0000","last_peered":"2024-09-11T23:20:00.743042+0000","last_clean":"2024-09-11T23:20:00.743042+0000","last_became_active":"2024-09-11T23:19:59.735078+0000","last_became_peered":"2024-09-11T23:19:59.735078+0000","last_unstale":"2024-09-11T23:20:00.743042+0000","last_undegraded":"2024-09-11T23:20:00.743042+0000","last_fullsized":"2024-09-11T23:20:00.743042+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:21:16.336006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744454+0000","last_change":"2024-09-11T23:19:59.745997+0000","last_active":"2024-09-11T23:20:00.744454+0000","last_peered":"2024-09-11T23:20:00.744454+0000","last_clean":"2024-09-11T23:20:00.744454+0000","last_became_active":"2024-09-11T23:19:59.745715+0000","last_became_peered":"2024-09-11T23:19:59.745715+0000","last_unstale":"2024-09-11T23:20:00.744454+0000","last_undegraded":"2024-09-11T23:20:00.744454+0000","last_fullsized":"2024-09-11T23:20:00.744454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:34.776229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746016+0000","last_change":"2024-09-11T23:19:59.734181+0000","last_active":"2024-09-11T23:20:00.746016+0000","last_peered":"2024-09-11T23:20:00.746016+0000","last_clean":"2024-09-11T23:20:00.746016+0000","last_became_active":"2024-09-11T23:19:59.733793+0000","last_became_peered":"2024-09-11T23:19:59.733793+0000","last_unstale":"2024-09-11T23:20:00.746016+0000","last_undegraded":"2024-09-11T23:20:00.746016+0000","last_fullsized":"2024-09-11T23:20:00.746016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:36.906308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.731427+0000","last_change":"2024-09-11T23:19:59.738713+0000","last_active":"2024-09-11T23:20:00.731427+0000","last_peered":"2024-09-11T23:20:00.731427+0000","last_clean":"2024-09-11T23:20:00.731427+0000","last_became_active":"2024-09-11T23:19:59.737579+0000","last_became_peered":"2024-09-11T23:19:59.737579+0000","last_unstale":"2024-09-11T23:20:00.731427+0000","last_undegraded":"2024-09-11T23:20:00.731427+0000","last_fullsized":"2024-09-11T23:20:00.731427+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:47.135775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.728349+0000","last_change":"2024-09-11T23:19:59.741381+0000","last_active":"2024-09-11T23:20:00.728349+0000","last_peered":"2024-09-11T23:20:00.728349+0000","last_clean":"2024-09-11T23:20:00.728349+0000","last_became_active":"2024-09-11T23:19:59.740914+0000","last_became_peered":"2024-09-11T23:19:59.740914+0000","last_unstale":"2024-09-11T23:20:00.728349+0000","last_undegraded":"2024-09-11T23:20:00.728349+0000","last_fullsized":"2024-09-11T23:20:00.728349+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:24:04.365704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746606+0000","last_change":"2024-09-11T23:19:59.736840+0000","last_active":"2024-09-11T23:20:00.746606+0000","last_peered":"2024-09-11T23:20:00.746606+0000","last_clean":"2024-09-11T23:20:00.746606+0000","last_became_active":"2024-09-11T23:19:59.736554+0000","last_became_peered":"2024-09-11T23:19:59.736554+0000","last_unstale":"2024-09-11T23:20:00.746606+0000","last_undegraded":"2024-09-11T23:20:00.746606+0000","last_fullsized":"2024-09-11T23:20:00.746606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:48.508369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.743748+0000","last_change":"2024-09-11T23:19:59.735719+0000","last_active":"2024-09-11T23:20:00.743748+0000","last_peered":"2024-09-11T23:20:00.743748+0000","last_clean":"2024-09-11T23:20:00.743748+0000","last_became_active":"2024-09-11T23:19:59.735397+0000","last_became_peered":"2024-09-11T23:19:59.735397+0000","last_unstale":"2024-09-11T23:20:00.743748+0000","last_undegraded":"2024-09-11T23:20:00.743748+0000","last_fullsized":"2024-09-11T23:20:00.743748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:26:19.392730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.743571+0000","last_change":"2024-09-11T23:19:59.737704+0000","last_active":"2024-09-11T23:20:00.743571+0000","last_peered":"2024-09-11T23:20:00.743571+0000","last_clean":"2024-09-11T23:20:00.743571+0000","last_became_active":"2024-09-11T23:19:59.737176+0000","last_became_peered":"2024-09-11T23:19:59.737176+0000","last_unstale":"2024-09-11T23:20:00.743571+0000","last_undegraded":"2024-09-11T23:20:00.743571+0000","last_fullsized":"2024-09-11T23:20:00.743571+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:22.938171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.731303+0000","last_change":"2024-09-11T23:19:59.735499+0000","last_active":"2024-09-11T23:20:00.731303+0000","last_peered":"2024-09-11T23:20:00.731303+0000","last_clean":"2024-09-11T23:20:00.731303+0000","last_became_active":"2024-09-11T23:19:59.735088+0000","last_became_peered":"2024-09-11T23:19:59.735088+0000","last_unstale":"2024-09-11T23:20:00.731303+0000","last_undegraded":"2024-09-11T23:20:00.731303+0000","last_fullsized":"2024-09-11T23:20:00.731303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:12.633104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744008+0000","last_change":"2024-09-11T23:19:59.743972+0000","last_active":"2024-09-11T23:20:00.744008+0000","last_peered":"2024-09-11T23:20:00.744008+0000","last_clean":"2024-09-11T23:20:00.744008+0000","last_became_active":"2024-09-11T23:19:59.743675+0000","last_became_peered":"2024-09-11T23:19:59.743675+0000","last_unstale":"2024-09-11T23:20:00.744008+0000","last_undegraded":"2024-09-11T23:20:00.744008+0000","last_fullsized":"2024-09-11T23:20:00.744008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:46.645696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:21.748261+0000","last_change":"2024-09-11T23:19:57.615472+0000","last_active":"2024-09-11T23:20:21.748261+0000","last_peered":"2024-09-11T23:20:21.748261+0000","last_clean":"2024-09-11T23:20:21.748261+0000","last_became_active":"2024-09-11T23:19:57.615104+0000","last_became_peered":"2024-09-11T23:19:57.615104+0000","last_unstale":"2024-09-11T23:20:21.748261+0000","last_undegraded":"2024-09-11T23:20:21.748261+0000","last_fullsized":"2024-09-11T23:20:21.748261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:03.601363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.743721+0000","last_change":"2024-09-11T23:19:59.741027+0000","last_active":"2024-09-11T23:20:00.743721+0000","last_peered":"2024-09-11T23:20:00.743721+0000","last_clean":"2024-09-11T23:20:00.743721+0000","last_became_active":"2024-09-11T23:19:59.740681+0000","last_became_peered":"2024-09-11T23:19:59.740681+0000","last_unstale":"2024-09-11T23:20:00.743721+0000","last_undegraded":"2024-09-11T23:20:00.743721+0000","last_fullsized":"2024-09-11T23:20:00.743721+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:36.948147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:09.726955+0000","last_change":"2024-09-11T23:19:57.612730+0000","last_active":"2024-09-11T23:20:09.726955+0000","last_peered":"2024-09-11T23:20:09.726955+0000","last_clean":"2024-09-11T23:20:09.726955+0000","last_became_active":"2024-09-11T23:19:57.612369+0000","last_became_peered":"2024-09-11T23:19:57.612369+0000","last_unstale":"2024-09-11T23:20:09.726955+0000","last_undegraded":"2024-09-11T23:20:09.726955+0000","last_fullsized":"2024-09-11T23:20:09.726955+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:17.823616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:09.727988+0000","last_change":"2024-09-11T23:19:57.611470+0000","last_active":"2024-09-11T23:20:09.727988+0000","last_peered":"2024-09-11T23:20:09.727988+0000","last_clean":"2024-09-11T23:20:09.727988+0000","last_became_active":"2024-09-11T23:19:57.611141+0000","last_became_peered":"2024-09-11T23:19:57.611141+0000","last_unstale":"2024-09-11T23:20:09.727988+0000","last_undegraded":"2024-09-11T23:20:09.727988+0000","last_fullsized":"2024-09-11T23:20:09.727988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:43.173955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.743939+0000","last_change":"2024-09-11T23:19:59.732375+0000","last_active":"2024-09-11T23:20:00.743939+0000","last_peered":"2024-09-11T23:20:00.743939+0000","last_clean":"2024-09-11T23:20:00.743939+0000","last_became_active":"2024-09-11T23:19:59.731793+0000","last_became_peered":"2024-09-11T23:19:59.731793+0000","last_unstale":"2024-09-11T23:20:00.743939+0000","last_undegraded":"2024-09-11T23:20:00.743939+0000","last_fullsized":"2024-09-11T23:20:00.743939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:02:47.278646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.743426+0000","last_change":"2024-09-11T23:19:57.614062+0000","last_active":"2024-09-11T23:20:00.743426+0000","last_peered":"2024-09-11T23:20:00.743426+0000","last_clean":"2024-09-11T23:20:00.743426+0000","last_became_active":"2024-09-11T23:19:57.613749+0000","last_became_peered":"2024-09-11T23:19:57.613749+0000","last_unstale":"2024-09-11T23:20:00.743426+0000","last_undegraded":"2024-09-11T23:20:00.743426+0000","last_fullsized":"2024-09-11T23:20:00.743426+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:45:22.299379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.742304+0000","last_change":"2024-09-11T23:19:59.730758+0000","last_active":"2024-09-11T23:20:00.742304+0000","last_peered":"2024-09-11T23:20:00.742304+0000","last_clean":"2024-09-11T23:20:00.742304+0000","last_became_active":"2024-09-11T23:19:59.730423+0000","last_became_peered":"2024-09-11T23:19:59.730423+0000","last_unstale":"2024-09-11T23:20:00.742304+0000","last_undegraded":"2024-09-11T23:20:00.742304+0000","last_fullsized":"2024-09-11T23:20:00.742304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:30.679451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:19.744821+0000","last_change":"2024-09-11T23:19:57.615574+0000","last_active":"2024-09-11T23:20:19.744821+0000","last_peered":"2024-09-11T23:20:19.744821+0000","last_clean":"2024-09-11T23:20:19.744821+0000","last_became_active":"2024-09-11T23:19:57.615192+0000","last_became_peered":"2024-09-11T23:19:57.615192+0000","last_unstale":"2024-09-11T23:20:19.744821+0000","last_undegraded":"2024-09-11T23:20:19.744821+0000","last_fullsized":"2024-09-11T23:20:19.744821+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:29.186899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746162+0000","last_change":"2024-09-11T23:19:59.738433+0000","last_active":"2024-09-11T23:20:00.746162+0000","last_peered":"2024-09-11T23:20:00.746162+0000","last_clean":"2024-09-11T23:20:00.746162+0000","last_became_active":"2024-09-11T23:19:59.738081+0000","last_became_peered":"2024-09-11T23:19:59.738081+0000","last_unstale":"2024-09-11T23:20:00.746162+0000","last_undegraded":"2024-09-11T23:20:00.746162+0000","last_fullsized":"2024-09-11T23:20:00.746162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:10:46.786454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:08.723576+0000","last_change":"2024-09-11T23:19:57.613410+0000","last_active":"2024-09-11T23:20:08.723576+0000","last_peered":"2024-09-11T23:20:08.723576+0000","last_clean":"2024-09-11T23:20:08.723576+0000","last_became_active":"2024-09-11T23:19:57.613070+0000","last_became_peered":"2024-09-11T23:19:57.613070+0000","last_unstale":"2024-09-11T23:20:08.723576+0000","last_undegraded":"2024-09-11T23:20:08.723576+0000","last_fullsized":"2024-09-11T23:20:08.723576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:07:53.971089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746895+0000","last_change":"2024-09-11T23:19:59.741081+0000","last_active":"2024-09-11T23:20:00.746895+0000","last_peered":"2024-09-11T23:20:00.746895+0000","last_clean":"2024-09-11T23:20:00.746895+0000","last_became_active":"2024-09-11T23:19:59.739984+0000","last_became_peered":"2024-09-11T23:19:59.739984+0000","last_unstale":"2024-09-11T23:20:00.746895+0000","last_undegraded":"2024-09-11T23:20:00.746895+0000","last_fullsized":"2024-09-11T23:20:00.746895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:03.513121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:08.723433+0000","last_change":"2024-09-11T23:19:57.823272+0000","last_active":"2024-09-11T23:20:08.723433+0000","last_peered":"2024-09-11T23:20:08.723433+0000","last_clean":"2024-09-11T23:20:08.723433+0000","last_became_active":"2024-09-11T23:19:57.822957+0000","last_became_peered":"2024-09-11T23:19:57.822957+0000","last_unstale":"2024-09-11T23:20:08.723433+0000","last_undegraded":"2024-09-11T23:20:08.723433+0000","last_fullsized":"2024-09-11T23:20:08.723433+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:06.779385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.745482+0000","last_change":"2024-09-11T23:19:59.738398+0000","last_active":"2024-09-11T23:20:00.745482+0000","last_peered":"2024-09-11T23:20:00.745482+0000","last_clean":"2024-09-11T23:20:00.745482+0000","last_became_active":"2024-09-11T23:19:59.738042+0000","last_became_peered":"2024-09-11T23:19:59.738042+0000","last_unstale":"2024-09-11T23:20:00.745482+0000","last_undegraded":"2024-09-11T23:20:00.745482+0000","last_fullsized":"2024-09-11T23:20:00.745482+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:05.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'21","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:20.747384+0000","last_change":"2024-09-11T23:19:57.615036+0000","last_active":"2024-09-11T23:20:20.747384+0000","last_peered":"2024-09-11T23:20:20.747384+0000","last_clean":"2024-09-11T23:20:20.747384+0000","last_became_active":"2024-09-11T23:19:57.614641+0000","last_became_peered":"2024-09-11T23:19:57.614641+0000","last_unstale":"2024-09-11T23:20:20.747384+0000","last_undegraded":"2024-09-11T23:20:20.747384+0000","last_fullsized":"2024-09-11T23:20:20.747384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:05.430508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42203,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":27,"num_read_kb":54,"num_write":22,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.745280+0000","last_change":"2024-09-11T23:19:59.741597+0000","last_active":"2024-09-11T23:20:00.745280+0000","last_peered":"2024-09-11T23:20:00.745280+0000","last_clean":"2024-09-11T23:20:00.745280+0000","last_became_active":"2024-09-11T23:19:59.741296+0000","last_became_peered":"2024-09-11T23:19:59.741296+0000","last_unstale":"2024-09-11T23:20:00.745280+0000","last_undegraded":"2024-09-11T23:20:00.745280+0000","last_fullsized":"2024-09-11T23:20:00.745280+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:06:20.742916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:09.727457+0000","last_change":"2024-09-11T23:19:57.617550+0000","last_active":"2024-09-11T23:20:09.727457+0000","last_peered":"2024-09-11T23:20:09.727457+0000","last_clean":"2024-09-11T23:20:09.727457+0000","last_became_active":"2024-09-11T23:19:57.617211+0000","last_became_peered":"2024-09-11T23:19:57.617211+0000","last_unstale":"2024-09-11T23:20:09.727457+0000","last_undegraded":"2024-09-11T23:20:09.727457+0000","last_fullsized":"2024-09-11T23:20:09.727457+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:19.707742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.728298+0000","last_change":"2024-09-11T23:19:59.742070+0000","last_active":"2024-09-11T23:20:00.728298+0000","last_peered":"2024-09-11T23:20:00.728298+0000","last_clean":"2024-09-11T23:20:00.728298+0000","last_became_active":"2024-09-11T23:19:59.739367+0000","last_became_peered":"2024-09-11T23:19:59.739367+0000","last_unstale":"2024-09-11T23:20:00.728298+0000","last_undegraded":"2024-09-11T23:20:00.728298+0000","last_fullsized":"2024-09-11T23:20:00.728298+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:27.850136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:08.722144+0000","last_change":"2024-09-11T23:19:57.618054+0000","last_active":"2024-09-11T23:20:08.722144+0000","last_peered":"2024-09-11T23:20:08.722144+0000","last_clean":"2024-09-11T23:20:08.722144+0000","last_became_active":"2024-09-11T23:19:57.617702+0000","last_became_peered":"2024-09-11T23:19:57.617702+0000","last_unstale":"2024-09-11T23:20:08.722144+0000","last_undegraded":"2024-09-11T23:20:08.722144+0000","last_fullsized":"2024-09-11T23:20:08.722144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:40.171851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.747476+0000","last_change":"2024-09-11T23:19:59.744000+0000","last_active":"2024-09-11T23:20:00.747476+0000","last_peered":"2024-09-11T23:20:00.747476+0000","last_clean":"2024-09-11T23:20:00.747476+0000","last_became_active":"2024-09-11T23:19:59.743629+0000","last_became_peered":"2024-09-11T23:19:59.743629+0000","last_unstale":"2024-09-11T23:20:00.747476+0000","last_undegraded":"2024-09-11T23:20:00.747476+0000","last_fullsized":"2024-09-11T23:20:00.747476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:20:04.133258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:21.748521+0000","last_change":"2024-09-11T23:19:57.615117+0000","last_active":"2024-09-11T23:20:21.748521+0000","last_peered":"2024-09-11T23:20:21.748521+0000","last_clean":"2024-09-11T23:20:21.748521+0000","last_became_active":"2024-09-11T23:19:57.614758+0000","last_became_peered":"2024-09-11T23:19:57.614758+0000","last_unstale":"2024-09-11T23:20:21.748521+0000","last_undegraded":"2024-09-11T23:20:21.748521+0000","last_fullsized":"2024-09-11T23:20:21.748521+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:58.113986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.747353+0000","last_change":"2024-09-11T23:19:59.739403+0000","last_active":"2024-09-11T23:20:00.747353+0000","last_peered":"2024-09-11T23:20:00.747353+0000","last_clean":"2024-09-11T23:20:00.747353+0000","last_became_active":"2024-09-11T23:19:59.739078+0000","last_became_peered":"2024-09-11T23:19:59.739078+0000","last_unstale":"2024-09-11T23:20:00.747353+0000","last_undegraded":"2024-09-11T23:20:00.747353+0000","last_fullsized":"2024-09-11T23:20:00.747353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:46.168720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:09.729626+0000","last_change":"2024-09-11T23:19:57.823323+0000","last_active":"2024-09-11T23:20:09.729626+0000","last_peered":"2024-09-11T23:20:09.729626+0000","last_clean":"2024-09-11T23:20:09.729626+0000","last_became_active":"2024-09-11T23:19:57.822947+0000","last_became_peered":"2024-09-11T23:19:57.822947+0000","last_unstale":"2024-09-11T23:20:09.729626+0000","last_undegraded":"2024-09-11T23:20:09.729626+0000","last_fullsized":"2024-09-11T23:20:09.729626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:12.542911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.742898+0000","last_change":"2024-09-11T23:19:59.741163+0000","last_active":"2024-09-11T23:20:00.742898+0000","last_peered":"2024-09-11T23:20:00.742898+0000","last_clean":"2024-09-11T23:20:00.742898+0000","last_became_active":"2024-09-11T23:19:59.740633+0000","last_became_peered":"2024-09-11T23:19:59.740633+0000","last_unstale":"2024-09-11T23:20:00.742898+0000","last_undegraded":"2024-09-11T23:20:00.742898+0000","last_fullsized":"2024-09-11T23:20:00.742898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:26.301775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:10.724091+0000","last_change":"2024-09-11T23:19:57.615804+0000","last_active":"2024-09-11T23:20:10.724091+0000","last_peered":"2024-09-11T23:20:10.724091+0000","last_clean":"2024-09-11T23:20:10.724091+0000","last_became_active":"2024-09-11T23:19:57.615443+0000","last_became_peered":"2024-09-11T23:19:57.615443+0000","last_unstale":"2024-09-11T23:20:10.724091+0000","last_undegraded":"2024-09-11T23:20:10.724091+0000","last_fullsized":"2024-09-11T23:20:10.724091+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:39.296873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.731472+0000","last_change":"2024-09-11T23:19:59.735411+0000","last_active":"2024-09-11T23:20:00.731472+0000","last_peered":"2024-09-11T23:20:00.731472+0000","last_clean":"2024-09-11T23:20:00.731472+0000","last_became_active":"2024-09-11T23:19:59.735089+0000","last_became_peered":"2024-09-11T23:19:59.735089+0000","last_unstale":"2024-09-11T23:20:00.731472+0000","last_undegraded":"2024-09-11T23:20:00.731472+0000","last_fullsized":"2024-09-11T23:20:00.731472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:06.691739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.743223+0000","last_change":"2024-09-11T23:19:57.612873+0000","last_active":"2024-09-11T23:20:00.743223+0000","last_peered":"2024-09-11T23:20:00.743223+0000","last_clean":"2024-09-11T23:20:00.743223+0000","last_became_active":"2024-09-11T23:19:57.612566+0000","last_became_peered":"2024-09-11T23:19:57.612566+0000","last_unstale":"2024-09-11T23:20:00.743223+0000","last_undegraded":"2024-09-11T23:20:00.743223+0000","last_fullsized":"2024-09-11T23:20:00.743223+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:46.218212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.747753+0000","last_change":"2024-09-11T23:19:59.736352+0000","last_active":"2024-09-11T23:20:00.747753+0000","last_peered":"2024-09-11T23:20:00.747753+0000","last_clean":"2024-09-11T23:20:00.747753+0000","last_became_active":"2024-09-11T23:19:59.735402+0000","last_became_peered":"2024-09-11T23:19:59.735402+0000","last_unstale":"2024-09-11T23:20:00.747753+0000","last_undegraded":"2024-09-11T23:20:00.747753+0000","last_fullsized":"2024-09-11T23:20:00.747753+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:41.854526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.752591+0000","last_change":"2024-09-11T23:19:57.612439+0000","last_active":"2024-09-11T23:20:00.752591+0000","last_peered":"2024-09-11T23:20:00.752591+0000","last_clean":"2024-09-11T23:20:00.752591+0000","last_became_active":"2024-09-11T23:19:57.612030+0000","last_became_peered":"2024-09-11T23:19:57.612030+0000","last_unstale":"2024-09-11T23:20:00.752591+0000","last_undegraded":"2024-09-11T23:20:00.752591+0000","last_fullsized":"2024-09-11T23:20:00.752591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:36:24.579246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.747209+0000","last_change":"2024-09-11T23:19:59.741241+0000","last_active":"2024-09-11T23:20:00.747209+0000","last_peered":"2024-09-11T23:20:00.747209+0000","last_clean":"2024-09-11T23:20:00.747209+0000","last_became_active":"2024-09-11T23:19:59.740265+0000","last_became_peered":"2024-09-11T23:19:59.740265+0000","last_unstale":"2024-09-11T23:20:00.747209+0000","last_undegraded":"2024-09-11T23:20:00.747209+0000","last_fullsized":"2024-09-11T23:20:00.747209+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:00.676762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:09.728669+0000","last_change":"2024-09-11T23:19:57.825238+0000","last_active":"2024-09-11T23:20:09.728669+0000","last_peered":"2024-09-11T23:20:09.728669+0000","last_clean":"2024-09-11T23:20:09.728669+0000","last_became_active":"2024-09-11T23:19:57.824732+0000","last_became_peered":"2024-09-11T23:19:57.824732+0000","last_unstale":"2024-09-11T23:20:09.728669+0000","last_undegraded":"2024-09-11T23:20:09.728669+0000","last_fullsized":"2024-09-11T23:20:09.728669+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:24.465863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.747591+0000","last_change":"2024-09-11T23:19:59.741338+0000","last_active":"2024-09-11T23:20:00.747591+0000","last_peered":"2024-09-11T23:20:00.747591+0000","last_clean":"2024-09-11T23:20:00.747591+0000","last_became_active":"2024-09-11T23:19:59.741019+0000","last_became_peered":"2024-09-11T23:19:59.741019+0000","last_unstale":"2024-09-11T23:20:00.747591+0000","last_undegraded":"2024-09-11T23:20:00.747591+0000","last_fullsized":"2024-09-11T23:20:00.747591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:36:03.480006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":344,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.747469+0000","last_change":"2024-09-11T23:18:45.606646+0000","last_active":"2024-09-11T23:20:00.747469+0000","last_peered":"2024-09-11T23:20:00.747469+0000","last_clean":"2024-09-11T23:20:00.747469+0000","last_became_active":"2024-09-11T23:18:45.606065+0000","last_became_peered":"2024-09-11T23:18:45.606065+0000","last_unstale":"2024-09-11T23:20:00.747469+0000","last_undegraded":"2024-09-11T23:20:00.747469+0000","last_fullsized":"2024-09-11T23:20:00.747469+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:15:17.148562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:15:17.148562+0000","last_clean_scrub_stamp":"2024-09-11T23:15:17.148562+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:07.992971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:10.729494+0000","last_change":"2024-09-11T23:19:57.612584+0000","last_active":"2024-09-11T23:20:10.729494+0000","last_peered":"2024-09-11T23:20:10.729494+0000","last_clean":"2024-09-11T23:20:10.729494+0000","last_became_active":"2024-09-11T23:19:57.612211+0000","last_became_peered":"2024-09-11T23:19:57.612211+0000","last_unstale":"2024-09-11T23:20:10.729494+0000","last_undegraded":"2024-09-11T23:20:10.729494+0000","last_fullsized":"2024-09-11T23:20:10.729494+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:04.026233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744949+0000","last_change":"2024-09-11T23:19:59.734502+0000","last_active":"2024-09-11T23:20:00.744949+0000","last_peered":"2024-09-11T23:20:00.744949+0000","last_clean":"2024-09-11T23:20:00.744949+0000","last_became_active":"2024-09-11T23:19:59.734160+0000","last_became_peered":"2024-09-11T23:19:59.734160+0000","last_unstale":"2024-09-11T23:20:00.744949+0000","last_undegraded":"2024-09-11T23:20:00.744949+0000","last_fullsized":"2024-09-11T23:20:00.744949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:58.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:09.727469+0000","last_change":"2024-09-11T23:19:57.618283+0000","last_active":"2024-09-11T23:20:09.727469+0000","last_peered":"2024-09-11T23:20:09.727469+0000","last_clean":"2024-09-11T23:20:09.727469+0000","last_became_active":"2024-09-11T23:19:57.617942+0000","last_became_peered":"2024-09-11T23:19:57.617942+0000","last_unstale":"2024-09-11T23:20:09.727469+0000","last_undegraded":"2024-09-11T23:20:09.727469+0000","last_fullsized":"2024-09-11T23:20:09.727469+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:30.832602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.728041+0000","last_change":"2024-09-11T23:19:59.740730+0000","last_active":"2024-09-11T23:20:00.728041+0000","last_peered":"2024-09-11T23:20:00.728041+0000","last_clean":"2024-09-11T23:20:00.728041+0000","last_became_active":"2024-09-11T23:19:59.740369+0000","last_became_peered":"2024-09-11T23:19:59.740369+0000","last_unstale":"2024-09-11T23:20:00.728041+0000","last_undegraded":"2024-09-11T23:20:00.728041+0000","last_fullsized":"2024-09-11T23:20:00.728041+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:50.264644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:21.747869+0000","last_change":"2024-09-11T23:19:57.823253+0000","last_active":"2024-09-11T23:20:21.747869+0000","last_peered":"2024-09-11T23:20:21.747869+0000","last_clean":"2024-09-11T23:20:21.747869+0000","last_became_active":"2024-09-11T23:19:57.822895+0000","last_became_peered":"2024-09-11T23:19:57.822895+0000","last_unstale":"2024-09-11T23:20:21.747869+0000","last_undegraded":"2024-09-11T23:20:21.747869+0000","last_fullsized":"2024-09-11T23:20:21.747869+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:22.998371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.743210+0000","last_change":"2024-09-11T23:19:59.742248+0000","last_active":"2024-09-11T23:20:00.743210+0000","last_peered":"2024-09-11T23:20:00.743210+0000","last_clean":"2024-09-11T23:20:00.743210+0000","last_became_active":"2024-09-11T23:19:59.741810+0000","last_became_peered":"2024-09-11T23:19:59.741810+0000","last_unstale":"2024-09-11T23:20:00.743210+0000","last_undegraded":"2024-09-11T23:20:00.743210+0000","last_fullsized":"2024-09-11T23:20:00.743210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:19:52.720286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.747924+0000","last_change":"2024-09-11T23:19:57.616244+0000","last_active":"2024-09-11T23:20:00.747924+0000","last_peered":"2024-09-11T23:20:00.747924+0000","last_clean":"2024-09-11T23:20:00.747924+0000","last_became_active":"2024-09-11T23:19:57.615868+0000","last_became_peered":"2024-09-11T23:19:57.615868+0000","last_unstale":"2024-09-11T23:20:00.747924+0000","last_undegraded":"2024-09-11T23:20:00.747924+0000","last_fullsized":"2024-09-11T23:20:00.747924+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:42.510896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.742387+0000","last_change":"2024-09-11T23:19:59.731524+0000","last_active":"2024-09-11T23:20:00.742387+0000","last_peered":"2024-09-11T23:20:00.742387+0000","last_clean":"2024-09-11T23:20:00.742387+0000","last_became_active":"2024-09-11T23:19:59.731185+0000","last_became_peered":"2024-09-11T23:19:59.731185+0000","last_unstale":"2024-09-11T23:20:00.742387+0000","last_undegraded":"2024-09-11T23:20:00.742387+0000","last_fullsized":"2024-09-11T23:20:00.742387+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:56.443226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.742378+0000","last_change":"2024-09-11T23:19:57.824334+0000","last_active":"2024-09-11T23:20:00.742378+0000","last_peered":"2024-09-11T23:20:00.742378+0000","last_clean":"2024-09-11T23:20:00.742378+0000","last_became_active":"2024-09-11T23:19:57.823972+0000","last_became_peered":"2024-09-11T23:19:57.823972+0000","last_unstale":"2024-09-11T23:20:00.742378+0000","last_undegraded":"2024-09-11T23:20:00.742378+0000","last_fullsized":"2024-09-11T23:20:00.742378+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:39.183281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:01.696043+0000","last_change":"2024-09-11T23:19:59.728064+0000","last_active":"2024-09-11T23:20:01.696043+0000","last_peered":"2024-09-11T23:20:01.696043+0000","last_clean":"2024-09-11T23:20:01.696043+0000","last_became_active":"2024-09-11T23:19:59.727732+0000","last_became_peered":"2024-09-11T23:19:59.727732+0000","last_unstale":"2024-09-11T23:20:01.696043+0000","last_undegraded":"2024-09-11T23:20:01.696043+0000","last_fullsized":"2024-09-11T23:20:01.696043+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:15.383974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:08.721231+0000","last_change":"2024-09-11T23:19:57.824362+0000","last_active":"2024-09-11T23:20:08.721231+0000","last_peered":"2024-09-11T23:20:08.721231+0000","last_clean":"2024-09-11T23:20:08.721231+0000","last_became_active":"2024-09-11T23:19:57.823753+0000","last_became_peered":"2024-09-11T23:19:57.823753+0000","last_unstale":"2024-09-11T23:20:08.721231+0000","last_undegraded":"2024-09-11T23:20:08.721231+0000","last_fullsized":"2024-09-11T23:20:08.721231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:56.357268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.728474+0000","last_change":"2024-09-11T23:19:59.730623+0000","last_active":"2024-09-11T23:20:00.728474+0000","last_peered":"2024-09-11T23:20:00.728474+0000","last_clean":"2024-09-11T23:20:00.728474+0000","last_became_active":"2024-09-11T23:19:59.730307+0000","last_became_peered":"2024-09-11T23:19:59.730307+0000","last_unstale":"2024-09-11T23:20:00.728474+0000","last_undegraded":"2024-09-11T23:20:00.728474+0000","last_fullsized":"2024-09-11T23:20:00.728474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:52.874884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'10","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:24.743300+0000","last_change":"2024-09-11T23:19:57.611730+0000","last_active":"2024-09-11T23:20:24.743300+0000","last_peered":"2024-09-11T23:20:24.743300+0000","last_clean":"2024-09-11T23:20:24.743300+0000","last_became_active":"2024-09-11T23:19:57.611394+0000","last_became_peered":"2024-09-11T23:19:57.611394+0000","last_unstale":"2024-09-11T23:20:24.743300+0000","last_undegraded":"2024-09-11T23:20:24.743300+0000","last_fullsized":"2024-09-11T23:20:24.743300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:15.518573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744483+0000","last_change":"2024-09-11T23:19:59.741396+0000","last_active":"2024-09-11T23:20:00.744483+0000","last_peered":"2024-09-11T23:20:00.744483+0000","last_clean":"2024-09-11T23:20:00.744483+0000","last_became_active":"2024-09-11T23:19:59.741102+0000","last_became_peered":"2024-09-11T23:19:59.741102+0000","last_unstale":"2024-09-11T23:20:00.744483+0000","last_undegraded":"2024-09-11T23:20:00.744483+0000","last_fullsized":"2024-09-11T23:20:00.744483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:08.759512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:08.722269+0000","last_change":"2024-09-11T23:19:57.824812+0000","last_active":"2024-09-11T23:20:08.722269+0000","last_peered":"2024-09-11T23:20:08.722269+0000","last_clean":"2024-09-11T23:20:08.722269+0000","last_became_active":"2024-09-11T23:19:57.824454+0000","last_became_peered":"2024-09-11T23:19:57.824454+0000","last_unstale":"2024-09-11T23:20:08.722269+0000","last_undegraded":"2024-09-11T23:20:08.722269+0000","last_fullsized":"2024-09-11T23:20:08.722269+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:25.275573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746707+0000","last_change":"2024-09-11T23:19:59.735857+0000","last_active":"2024-09-11T23:20:00.746707+0000","last_peered":"2024-09-11T23:20:00.746707+0000","last_clean":"2024-09-11T23:20:00.746707+0000","last_became_active":"2024-09-11T23:19:59.735541+0000","last_became_peered":"2024-09-11T23:19:59.735541+0000","last_unstale":"2024-09-11T23:20:00.746707+0000","last_undegraded":"2024-09-11T23:20:00.746707+0000","last_fullsized":"2024-09-11T23:20:00.746707+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:21.600490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:08.722858+0000","last_change":"2024-09-11T23:19:57.823468+0000","last_active":"2024-09-11T23:20:08.722858+0000","last_peered":"2024-09-11T23:20:08.722858+0000","last_clean":"2024-09-11T23:20:08.722858+0000","last_became_active":"2024-09-11T23:19:57.823154+0000","last_became_peered":"2024-09-11T23:19:57.823154+0000","last_unstale":"2024-09-11T23:20:08.722858+0000","last_undegraded":"2024-09-11T23:20:08.722858+0000","last_fullsized":"2024-09-11T23:20:08.722858+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:28.770627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.742722+0000","last_change":"2024-09-11T23:19:59.741263+0000","last_active":"2024-09-11T23:20:00.742722+0000","last_peered":"2024-09-11T23:20:00.742722+0000","last_clean":"2024-09-11T23:20:00.742722+0000","last_became_active":"2024-09-11T23:19:59.740938+0000","last_became_peered":"2024-09-11T23:19:59.740938+0000","last_unstale":"2024-09-11T23:20:00.742722+0000","last_undegraded":"2024-09-11T23:20:00.742722+0000","last_fullsized":"2024-09-11T23:20:00.742722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:54.768652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:08.723486+0000","last_change":"2024-09-11T23:19:57.612517+0000","last_active":"2024-09-11T23:20:08.723486+0000","last_peered":"2024-09-11T23:20:08.723486+0000","last_clean":"2024-09-11T23:20:08.723486+0000","last_became_active":"2024-09-11T23:19:57.612188+0000","last_became_peered":"2024-09-11T23:19:57.612188+0000","last_unstale":"2024-09-11T23:20:08.723486+0000","last_undegraded":"2024-09-11T23:20:08.723486+0000","last_fullsized":"2024-09-11T23:20:08.723486+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:52.896413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.731401+0000","last_change":"2024-09-11T23:19:59.732520+0000","last_active":"2024-09-11T23:20:00.731401+0000","last_peered":"2024-09-11T23:20:00.731401+0000","last_clean":"2024-09-11T23:20:00.731401+0000","last_became_active":"2024-09-11T23:19:59.732173+0000","last_became_peered":"2024-09-11T23:19:59.732173+0000","last_unstale":"2024-09-11T23:20:00.731401+0000","last_undegraded":"2024-09-11T23:20:00.731401+0000","last_fullsized":"2024-09-11T23:20:00.731401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:04.626312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:01.159050+0000","last_change":"2024-09-11T23:19:57.613962+0000","last_active":"2024-09-11T23:20:01.159050+0000","last_peered":"2024-09-11T23:20:01.159050+0000","last_clean":"2024-09-11T23:20:01.159050+0000","last_became_active":"2024-09-11T23:19:57.613593+0000","last_became_peered":"2024-09-11T23:19:57.613593+0000","last_unstale":"2024-09-11T23:20:01.159050+0000","last_undegraded":"2024-09-11T23:20:01.159050+0000","last_fullsized":"2024-09-11T23:20:01.159050+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:12.400377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744905+0000","last_change":"2024-09-11T23:19:59.734667+0000","last_active":"2024-09-11T23:20:00.744905+0000","last_peered":"2024-09-11T23:20:00.744905+0000","last_clean":"2024-09-11T23:20:00.744905+0000","last_became_active":"2024-09-11T23:19:59.733754+0000","last_became_peered":"2024-09-11T23:19:59.733754+0000","last_unstale":"2024-09-11T23:20:00.744905+0000","last_undegraded":"2024-09-11T23:20:00.744905+0000","last_fullsized":"2024-09-11T23:20:00.744905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:57.392272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:08.722751+0000","last_change":"2024-09-11T23:19:57.614579+0000","last_active":"2024-09-11T23:20:08.722751+0000","last_peered":"2024-09-11T23:20:08.722751+0000","last_clean":"2024-09-11T23:20:08.722751+0000","last_became_active":"2024-09-11T23:19:57.614227+0000","last_became_peered":"2024-09-11T23:19:57.614227+0000","last_unstale":"2024-09-11T23:20:08.722751+0000","last_undegraded":"2024-09-11T23:20:08.722751+0000","last_fullsized":"2024-09-11T23:20:08.722751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:47.952920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.747070+0000","last_change":"2024-09-11T23:19:59.744219+0000","last_active":"2024-09-11T23:20:00.747070+0000","last_peered":"2024-09-11T23:20:00.747070+0000","last_clean":"2024-09-11T23:20:00.747070+0000","last_became_active":"2024-09-11T23:19:59.743914+0000","last_became_peered":"2024-09-11T23:19:59.743914+0000","last_unstale":"2024-09-11T23:20:00.747070+0000","last_undegraded":"2024-09-11T23:20:00.747070+0000","last_fullsized":"2024-09-11T23:20:00.747070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:52.149588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744576+0000","last_change":"2024-09-11T23:19:59.740502+0000","last_active":"2024-09-11T23:20:00.744576+0000","last_peered":"2024-09-11T23:20:00.744576+0000","last_clean":"2024-09-11T23:20:00.744576+0000","last_became_active":"2024-09-11T23:19:59.740122+0000","last_became_peered":"2024-09-11T23:19:59.740122+0000","last_unstale":"2024-09-11T23:20:00.744576+0000","last_undegraded":"2024-09-11T23:20:00.744576+0000","last_fullsized":"2024-09-11T23:20:00.744576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:36.446241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.752059+0000","last_change":"2024-09-11T23:19:57.610389+0000","last_active":"2024-09-11T23:20:00.752059+0000","last_peered":"2024-09-11T23:20:00.752059+0000","last_clean":"2024-09-11T23:20:00.752059+0000","last_became_active":"2024-09-11T23:19:57.609911+0000","last_became_peered":"2024-09-11T23:19:57.609911+0000","last_unstale":"2024-09-11T23:20:00.752059+0000","last_undegraded":"2024-09-11T23:20:00.752059+0000","last_fullsized":"2024-09-11T23:20:00.752059+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:23.949929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.745349+0000","last_change":"2024-09-11T23:19:59.736867+0000","last_active":"2024-09-11T23:20:00.745349+0000","last_peered":"2024-09-11T23:20:00.745349+0000","last_clean":"2024-09-11T23:20:00.745349+0000","last_became_active":"2024-09-11T23:19:59.736341+0000","last_became_peered":"2024-09-11T23:19:59.736341+0000","last_unstale":"2024-09-11T23:20:00.745349+0000","last_undegraded":"2024-09-11T23:20:00.745349+0000","last_fullsized":"2024-09-11T23:20:00.745349+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:57:51.015235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:09.727223+0000","last_change":"2024-09-11T23:19:57.615855+0000","last_active":"2024-09-11T23:20:09.727223+0000","last_peered":"2024-09-11T23:20:09.727223+0000","last_clean":"2024-09-11T23:20:09.727223+0000","last_became_active":"2024-09-11T23:19:57.615308+0000","last_became_peered":"2024-09-11T23:19:57.615308+0000","last_unstale":"2024-09-11T23:20:09.727223+0000","last_undegraded":"2024-09-11T23:20:09.727223+0000","last_fullsized":"2024-09-11T23:20:09.727223+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:29:16.143982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746858+0000","last_change":"2024-09-11T23:19:59.742049+0000","last_active":"2024-09-11T23:20:00.746858+0000","last_peered":"2024-09-11T23:20:00.746858+0000","last_clean":"2024-09-11T23:20:00.746858+0000","last_became_active":"2024-09-11T23:19:59.741763+0000","last_became_peered":"2024-09-11T23:19:59.741763+0000","last_unstale":"2024-09-11T23:20:00.746858+0000","last_undegraded":"2024-09-11T23:20:00.746858+0000","last_fullsized":"2024-09-11T23:20:00.746858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:48.698896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:01.232954+0000","last_change":"2024-09-11T23:19:57.822949+0000","last_active":"2024-09-11T23:20:01.232954+0000","last_peered":"2024-09-11T23:20:01.232954+0000","last_clean":"2024-09-11T23:20:01.232954+0000","last_became_active":"2024-09-11T23:19:57.822564+0000","last_became_peered":"2024-09-11T23:19:57.822564+0000","last_unstale":"2024-09-11T23:20:01.232954+0000","last_undegraded":"2024-09-11T23:20:01.232954+0000","last_fullsized":"2024-09-11T23:20:01.232954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:29.532109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744563+0000","last_change":"2024-09-11T23:19:59.735275+0000","last_active":"2024-09-11T23:20:00.744563+0000","last_peered":"2024-09-11T23:20:00.744563+0000","last_clean":"2024-09-11T23:20:00.744563+0000","last_became_active":"2024-09-11T23:19:59.734954+0000","last_became_peered":"2024-09-11T23:19:59.734954+0000","last_unstale":"2024-09-11T23:20:00.744563+0000","last_undegraded":"2024-09-11T23:20:00.744563+0000","last_fullsized":"2024-09-11T23:20:00.744563+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:55:43.511578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:09.729237+0000","last_change":"2024-09-11T23:19:57.613323+0000","last_active":"2024-09-11T23:20:09.729237+0000","last_peered":"2024-09-11T23:20:09.729237+0000","last_clean":"2024-09-11T23:20:09.729237+0000","last_became_active":"2024-09-11T23:19:57.612985+0000","last_became_peered":"2024-09-11T23:19:57.612985+0000","last_unstale":"2024-09-11T23:20:09.729237+0000","last_undegraded":"2024-09-11T23:20:09.729237+0000","last_fullsized":"2024-09-11T23:20:09.729237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:13:09.179547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":46206,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":96,"num_read_kb":97,"num_write":153,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":164120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285620,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514513,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":424539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":1058616,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906973,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":424775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168609,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":424695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":1016627,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561067,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":424164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790000,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1015918,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":424994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149755,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":905216,"data_stored":424475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":480261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":946176,"data_stored":478911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":425435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34680,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34680,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34680,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":44174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:20:27.504 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:20:27.761 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:28.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:27 smithi043 ceph-mon[30950]: pgmap v295: 97 pgs: 97 active+clean; 622 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-09-11T23:20:28.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:27 smithi043 ceph-mon[30950]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:20:28.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:27 smithi090 ceph-mon[35130]: pgmap v295: 97 pgs: 97 active+clean; 622 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-09-11T23:20:28.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:27 smithi090 ceph-mon[35130]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:20:28.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.274+0000 7eff4994c640 1 -- 172.21.15.43:0/451655480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff441036e0 msgr2=0x7eff44107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:28.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.274+0000 7eff4994c640 1 --2- 172.21.15.43:0/451655480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff441036e0 0x7eff44107b90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7eff38009930 tx=0x7eff3802a420 comp rx=0 tx=0).stop 2024-09-11T23:20:28.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.274+0000 7eff4994c640 1 -- 172.21.15.43:0/451655480 shutdown_connections 2024-09-11T23:20:28.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.274+0000 7eff4994c640 1 --2- 172.21.15.43:0/451655480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff441082a0 0x7eff4410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.274+0000 7eff4994c640 1 --2- 172.21.15.43:0/451655480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff441036e0 0x7eff44107b90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.274+0000 7eff4994c640 1 --2- 172.21.15.43:0/451655480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff44102d30 0x7eff44103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.274+0000 7eff4994c640 1 -- 172.21.15.43:0/451655480 >> 172.21.15.43:0/451655480 conn(0x7eff440fc7d0 msgr2=0x7eff440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.274+0000 7eff4994c640 1 -- 172.21.15.43:0/451655480 shutdown_connections 2024-09-11T23:20:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.274+0000 7eff4994c640 1 -- 172.21.15.43:0/451655480 wait complete. 2024-09-11T23:20:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.274+0000 7eff4994c640 1 Processor -- start 2024-09-11T23:20:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.274+0000 7eff4994c640 1 -- start start 2024-09-11T23:20:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.275+0000 7eff4994c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff44102d30 0x7eff44100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.275+0000 7eff4994c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff441036e0 0x7eff44100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.275+0000 7eff4994c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff441082a0 0x7eff44101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.275+0000 7eff4994c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7eff44112140 con 0x7eff441036e0 2024-09-11T23:20:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.275+0000 7eff4994c640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7eff44111fc0 con 0x7eff441082a0 2024-09-11T23:20:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.275+0000 7eff4994c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff441122c0 con 0x7eff44102d30 2024-09-11T23:20:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.275+0000 7eff427fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff441036e0 0x7eff44100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.275+0000 7eff427fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff441036e0 0x7eff44100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39990/0 (socket says 172.21.15.43:39990) 2024-09-11T23:20:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.275+0000 7eff427fc640 1 -- 172.21.15.43:0/4193864744 learned_addr learned my addr 172.21.15.43:0/4193864744 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.275+0000 7eff42ffd640 1 --2- 172.21.15.43:0/4193864744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff44102d30 0x7eff44100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.275+0000 7eff437fe640 1 --2- 172.21.15.43:0/4193864744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff441082a0 0x7eff44101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.275+0000 7eff427fc640 1 -- 172.21.15.43:0/4193864744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff44102d30 msgr2=0x7eff44100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.275+0000 7eff427fc640 1 --2- 172.21.15.43:0/4193864744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff44102d30 0x7eff44100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.275+0000 7eff427fc640 1 -- 172.21.15.43:0/4193864744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff441082a0 msgr2=0x7eff44101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.275+0000 7eff427fc640 1 --2- 172.21.15.43:0/4193864744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff441082a0 0x7eff44101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.275+0000 7eff427fc640 1 -- 172.21.15.43:0/4193864744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff441ab0b0 con 0x7eff441036e0 2024-09-11T23:20:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.275+0000 7eff427fc640 1 --2- 172.21.15.43:0/4193864744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff441036e0 0x7eff44100ed0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7eff38005940 tx=0x7eff380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.275+0000 7eff4894a640 1 -- 172.21.15.43:0/4193864744 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3803f030 con 0x7eff441036e0 2024-09-11T23:20:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.275+0000 7eff4894a640 1 -- 172.21.15.43:0/4193864744 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff38003ce0 con 0x7eff441036e0 2024-09-11T23:20:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.275+0000 7eff4994c640 1 -- 172.21.15.43:0/4193864744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff441ab280 con 0x7eff441036e0 2024-09-11T23:20:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.276+0000 7eff4894a640 1 -- 172.21.15.43:0/4193864744 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff38035070 con 0x7eff441036e0 2024-09-11T23:20:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.276+0000 7eff4994c640 1 -- 172.21.15.43:0/4193864744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff441ab7c0 con 0x7eff441036e0 2024-09-11T23:20:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.277+0000 7eff4994c640 1 -- 172.21.15.43:0/4193864744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff4406a850 con 0x7eff441036e0 2024-09-11T23:20:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.278+0000 7eff4894a640 1 -- 172.21.15.43:0/4193864744 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7eff38042030 con 0x7eff441036e0 2024-09-11T23:20:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.279+0000 7eff4894a640 1 --2- 172.21.15.43:0/4193864744 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7eff2c0a2c10 0x7eff2c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.279+0000 7eff42ffd640 1 --2- 172.21.15.43:0/4193864744 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7eff2c0a2c10 0x7eff2c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.279+0000 7eff4894a640 1 -- 172.21.15.43:0/4193864744 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7eff380ef330 con 0x7eff441036e0 2024-09-11T23:20:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.279+0000 7eff42ffd640 1 --2- 172.21.15.43:0/4193864744 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7eff2c0a2c10 0x7eff2c0a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7eff34009f20 tx=0x7eff340094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.283+0000 7eff4894a640 1 -- 172.21.15.43:0/4193864744 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7eff380bc2e0 con 0x7eff441036e0 2024-09-11T23:20:28.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:27 smithi097 ceph-mon[35562]: pgmap v295: 97 pgs: 97 active+clean; 622 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-09-11T23:20:28.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:27 smithi097 ceph-mon[35562]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:20:28.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.455+0000 7eff4994c640 1 -- 172.21.15.43:0/4193864744 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7eff44101ec0 con 0x7eff2c0a2c10 2024-09-11T23:20:28.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.461+0000 7eff4894a640 1 -- 172.21.15.43:0/4193864744 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258862 (secure 0 0 0) 0x7eff44101ec0 con 0x7eff2c0a2c10 2024-09-11T23:20:28.463 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:20:28.464 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-11T23:20:28.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.465+0000 7eff4994c640 1 -- 172.21.15.43:0/4193864744 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7eff2c0a2c10 msgr2=0x7eff2c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:28.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.465+0000 7eff4994c640 1 --2- 172.21.15.43:0/4193864744 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7eff2c0a2c10 0x7eff2c0a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7eff34009f20 tx=0x7eff340094f0 comp rx=0 tx=0).stop 2024-09-11T23:20:28.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.465+0000 7eff4994c640 1 -- 172.21.15.43:0/4193864744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff441036e0 msgr2=0x7eff44100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:28.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.465+0000 7eff4994c640 1 --2- 172.21.15.43:0/4193864744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff441036e0 0x7eff44100ed0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7eff38005940 tx=0x7eff380042f0 comp rx=0 tx=0).stop 2024-09-11T23:20:28.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.465+0000 7eff4994c640 1 -- 172.21.15.43:0/4193864744 shutdown_connections 2024-09-11T23:20:28.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.465+0000 7eff4994c640 1 --2- 172.21.15.43:0/4193864744 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7eff2c0a2c10 0x7eff2c0a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:28.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.465+0000 7eff4994c640 1 --2- 172.21.15.43:0/4193864744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff441082a0 0x7eff44101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:28.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.465+0000 7eff4994c640 1 --2- 172.21.15.43:0/4193864744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff441036e0 0x7eff44100ed0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:28.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.465+0000 7eff4994c640 1 --2- 172.21.15.43:0/4193864744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff44102d30 0x7eff44100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:28.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.465+0000 7eff4994c640 1 -- 172.21.15.43:0/4193864744 >> 172.21.15.43:0/4193864744 conn(0x7eff440fc7d0 msgr2=0x7eff44105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:28.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.465+0000 7eff4994c640 1 -- 172.21.15.43:0/4193864744 shutdown_connections 2024-09-11T23:20:28.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:28.465+0000 7eff4994c640 1 -- 172.21.15.43:0/4193864744 wait complete. 2024-09-11T23:20:29.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:28 smithi043 ceph-mon[30950]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:20:29.268 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-09-11T23:20:26.638306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636808,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":232,"num_read_kb":216,"num_write":336,"num_write_kb":3027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368440,"kb_used_data":10872,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573384,"statfs":{"total":1151940427776,"available":1151563145216,"internally_reserved":0,"allocated":11132928,"data_stored":7023410,"data_compressed":108948,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41218,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":72,"num_write":30,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003176"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.731685+0000","last_change":"2024-09-11T23:19:59.734378+0000","last_active":"2024-09-11T23:20:00.731685+0000","last_peered":"2024-09-11T23:20:00.731685+0000","last_clean":"2024-09-11T23:20:00.731685+0000","last_became_active":"2024-09-11T23:19:59.734067+0000","last_became_peered":"2024-09-11T23:19:59.734067+0000","last_unstale":"2024-09-11T23:20:00.731685+0000","last_undegraded":"2024-09-11T23:20:00.731685+0000","last_fullsized":"2024-09-11T23:20:00.731685+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:28:18.384453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746459+0000","last_change":"2024-09-11T23:19:59.733892+0000","last_active":"2024-09-11T23:20:00.746459+0000","last_peered":"2024-09-11T23:20:00.746459+0000","last_clean":"2024-09-11T23:20:00.746459+0000","last_became_active":"2024-09-11T23:19:59.733540+0000","last_became_peered":"2024-09-11T23:19:59.733540+0000","last_unstale":"2024-09-11T23:20:00.746459+0000","last_undegraded":"2024-09-11T23:20:00.746459+0000","last_fullsized":"2024-09-11T23:20:00.746459+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:42.186458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746491+0000","last_change":"2024-09-11T23:19:59.739443+0000","last_active":"2024-09-11T23:20:00.746491+0000","last_peered":"2024-09-11T23:20:00.746491+0000","last_clean":"2024-09-11T23:20:00.746491+0000","last_became_active":"2024-09-11T23:19:59.739025+0000","last_became_peered":"2024-09-11T23:19:59.739025+0000","last_unstale":"2024-09-11T23:20:00.746491+0000","last_undegraded":"2024-09-11T23:20:00.746491+0000","last_fullsized":"2024-09-11T23:20:00.746491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:40.819321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.731171+0000","last_change":"2024-09-11T23:19:59.737076+0000","last_active":"2024-09-11T23:20:00.731171+0000","last_peered":"2024-09-11T23:20:00.731171+0000","last_clean":"2024-09-11T23:20:00.731171+0000","last_became_active":"2024-09-11T23:19:59.736222+0000","last_became_peered":"2024-09-11T23:19:59.736222+0000","last_unstale":"2024-09-11T23:20:00.731171+0000","last_undegraded":"2024-09-11T23:20:00.731171+0000","last_fullsized":"2024-09-11T23:20:00.731171+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:33:13.158825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744063+0000","last_change":"2024-09-11T23:19:59.738662+0000","last_active":"2024-09-11T23:20:00.744063+0000","last_peered":"2024-09-11T23:20:00.744063+0000","last_clean":"2024-09-11T23:20:00.744063+0000","last_became_active":"2024-09-11T23:19:59.738320+0000","last_became_peered":"2024-09-11T23:19:59.738320+0000","last_unstale":"2024-09-11T23:20:00.744063+0000","last_undegraded":"2024-09-11T23:20:00.744063+0000","last_fullsized":"2024-09-11T23:20:00.744063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:15.645179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.742212+0000","last_change":"2024-09-11T23:19:59.731442+0000","last_active":"2024-09-11T23:20:00.742212+0000","last_peered":"2024-09-11T23:20:00.742212+0000","last_clean":"2024-09-11T23:20:00.742212+0000","last_became_active":"2024-09-11T23:19:59.731079+0000","last_became_peered":"2024-09-11T23:19:59.731079+0000","last_unstale":"2024-09-11T23:20:00.742212+0000","last_undegraded":"2024-09-11T23:20:00.742212+0000","last_fullsized":"2024-09-11T23:20:00.742212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:32.764991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746867+0000","last_change":"2024-09-11T23:19:59.743734+0000","last_active":"2024-09-11T23:20:00.746867+0000","last_peered":"2024-09-11T23:20:00.746867+0000","last_clean":"2024-09-11T23:20:00.746867+0000","last_became_active":"2024-09-11T23:19:59.743387+0000","last_became_peered":"2024-09-11T23:19:59.743387+0000","last_unstale":"2024-09-11T23:20:00.746867+0000","last_undegraded":"2024-09-11T23:20:00.746867+0000","last_fullsized":"2024-09-11T23:20:00.746867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:20.515745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.743887+0000","last_change":"2024-09-11T23:19:59.740733+0000","last_active":"2024-09-11T23:20:00.743887+0000","last_peered":"2024-09-11T23:20:00.743887+0000","last_clean":"2024-09-11T23:20:00.743887+0000","last_became_active":"2024-09-11T23:19:59.740438+0000","last_became_peered":"2024-09-11T23:19:59.740438+0000","last_unstale":"2024-09-11T23:20:00.743887+0000","last_undegraded":"2024-09-11T23:20:00.743887+0000","last_fullsized":"2024-09-11T23:20:00.743887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:27.063305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746418+0000","last_change":"2024-09-11T23:19:59.742439+0000","last_active":"2024-09-11T23:20:00.746418+0000","last_peered":"2024-09-11T23:20:00.746418+0000","last_clean":"2024-09-11T23:20:00.746418+0000","last_became_active":"2024-09-11T23:19:59.741983+0000","last_became_peered":"2024-09-11T23:19:59.741983+0000","last_unstale":"2024-09-11T23:20:00.746418+0000","last_undegraded":"2024-09-11T23:20:00.746418+0000","last_fullsized":"2024-09-11T23:20:00.746418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:39.477167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.742710+0000","last_change":"2024-09-11T23:19:59.732544+0000","last_active":"2024-09-11T23:20:00.742710+0000","last_peered":"2024-09-11T23:20:00.742710+0000","last_clean":"2024-09-11T23:20:00.742710+0000","last_became_active":"2024-09-11T23:19:59.732205+0000","last_became_peered":"2024-09-11T23:19:59.732205+0000","last_unstale":"2024-09-11T23:20:00.742710+0000","last_undegraded":"2024-09-11T23:20:00.742710+0000","last_fullsized":"2024-09-11T23:20:00.742710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:22.494344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.747452+0000","last_change":"2024-09-11T23:19:59.741857+0000","last_active":"2024-09-11T23:20:00.747452+0000","last_peered":"2024-09-11T23:20:00.747452+0000","last_clean":"2024-09-11T23:20:00.747452+0000","last_became_active":"2024-09-11T23:19:59.741484+0000","last_became_peered":"2024-09-11T23:19:59.741484+0000","last_unstale":"2024-09-11T23:20:00.747452+0000","last_undegraded":"2024-09-11T23:20:00.747452+0000","last_fullsized":"2024-09-11T23:20:00.747452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:52.998258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.731332+0000","last_change":"2024-09-11T23:19:59.736042+0000","last_active":"2024-09-11T23:20:00.731332+0000","last_peered":"2024-09-11T23:20:00.731332+0000","last_clean":"2024-09-11T23:20:00.731332+0000","last_became_active":"2024-09-11T23:19:59.735731+0000","last_became_peered":"2024-09-11T23:19:59.735731+0000","last_unstale":"2024-09-11T23:20:00.731332+0000","last_undegraded":"2024-09-11T23:20:00.731332+0000","last_fullsized":"2024-09-11T23:20:00.731332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:19.085804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.728381+0000","last_change":"2024-09-11T23:19:59.742049+0000","last_active":"2024-09-11T23:20:00.728381+0000","last_peered":"2024-09-11T23:20:00.728381+0000","last_clean":"2024-09-11T23:20:00.728381+0000","last_became_active":"2024-09-11T23:19:59.740974+0000","last_became_peered":"2024-09-11T23:19:59.740974+0000","last_unstale":"2024-09-11T23:20:00.728381+0000","last_undegraded":"2024-09-11T23:20:00.728381+0000","last_fullsized":"2024-09-11T23:20:00.728381+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:10.443113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746606+0000","last_change":"2024-09-11T23:19:59.734270+0000","last_active":"2024-09-11T23:20:00.746606+0000","last_peered":"2024-09-11T23:20:00.746606+0000","last_clean":"2024-09-11T23:20:00.746606+0000","last_became_active":"2024-09-11T23:19:59.733967+0000","last_became_peered":"2024-09-11T23:19:59.733967+0000","last_unstale":"2024-09-11T23:20:00.746606+0000","last_undegraded":"2024-09-11T23:20:00.746606+0000","last_fullsized":"2024-09-11T23:20:00.746606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:02.185757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746434+0000","last_change":"2024-09-11T23:19:59.734152+0000","last_active":"2024-09-11T23:20:00.746434+0000","last_peered":"2024-09-11T23:20:00.746434+0000","last_clean":"2024-09-11T23:20:00.746434+0000","last_became_active":"2024-09-11T23:19:59.733163+0000","last_became_peered":"2024-09-11T23:19:59.733163+0000","last_unstale":"2024-09-11T23:20:00.746434+0000","last_undegraded":"2024-09-11T23:20:00.746434+0000","last_fullsized":"2024-09-11T23:20:00.746434+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:22.370591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744672+0000","last_change":"2024-09-11T23:19:59.735517+0000","last_active":"2024-09-11T23:20:00.744672+0000","last_peered":"2024-09-11T23:20:00.744672+0000","last_clean":"2024-09-11T23:20:00.744672+0000","last_became_active":"2024-09-11T23:19:59.735182+0000","last_became_peered":"2024-09-11T23:19:59.735182+0000","last_unstale":"2024-09-11T23:20:00.744672+0000","last_undegraded":"2024-09-11T23:20:00.744672+0000","last_fullsized":"2024-09-11T23:20:00.744672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:25.989416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.731699+0000","last_change":"2024-09-11T23:19:59.733579+0000","last_active":"2024-09-11T23:20:00.731699+0000","last_peered":"2024-09-11T23:20:00.731699+0000","last_clean":"2024-09-11T23:20:00.731699+0000","last_became_active":"2024-09-11T23:19:59.733245+0000","last_became_peered":"2024-09-11T23:19:59.733245+0000","last_unstale":"2024-09-11T23:20:00.731699+0000","last_undegraded":"2024-09-11T23:20:00.731699+0000","last_fullsized":"2024-09-11T23:20:00.731699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:22.361158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.742621+0000","last_change":"2024-09-11T23:19:59.734354+0000","last_active":"2024-09-11T23:20:00.742621+0000","last_peered":"2024-09-11T23:20:00.742621+0000","last_clean":"2024-09-11T23:20:00.742621+0000","last_became_active":"2024-09-11T23:19:59.733988+0000","last_became_peered":"2024-09-11T23:19:59.733988+0000","last_unstale":"2024-09-11T23:20:00.742621+0000","last_undegraded":"2024-09-11T23:20:00.742621+0000","last_fullsized":"2024-09-11T23:20:00.742621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:41.401496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744515+0000","last_change":"2024-09-11T23:19:59.730318+0000","last_active":"2024-09-11T23:20:00.744515+0000","last_peered":"2024-09-11T23:20:00.744515+0000","last_clean":"2024-09-11T23:20:00.744515+0000","last_became_active":"2024-09-11T23:19:59.730009+0000","last_became_peered":"2024-09-11T23:19:59.730009+0000","last_unstale":"2024-09-11T23:20:00.744515+0000","last_undegraded":"2024-09-11T23:20:00.744515+0000","last_fullsized":"2024-09-11T23:20:00.744515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:01:38.531967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.731289+0000","last_change":"2024-09-11T23:19:59.737254+0000","last_active":"2024-09-11T23:20:00.731289+0000","last_peered":"2024-09-11T23:20:00.731289+0000","last_clean":"2024-09-11T23:20:00.731289+0000","last_became_active":"2024-09-11T23:19:59.736936+0000","last_became_peered":"2024-09-11T23:19:59.736936+0000","last_unstale":"2024-09-11T23:20:00.731289+0000","last_undegraded":"2024-09-11T23:20:00.731289+0000","last_fullsized":"2024-09-11T23:20:00.731289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:00.361468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.728507+0000","last_change":"2024-09-11T23:19:59.741617+0000","last_active":"2024-09-11T23:20:00.728507+0000","last_peered":"2024-09-11T23:20:00.728507+0000","last_clean":"2024-09-11T23:20:00.728507+0000","last_became_active":"2024-09-11T23:19:59.740358+0000","last_became_peered":"2024-09-11T23:19:59.740358+0000","last_unstale":"2024-09-11T23:20:00.728507+0000","last_undegraded":"2024-09-11T23:20:00.728507+0000","last_fullsized":"2024-09-11T23:20:00.728507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:18.541105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.745052+0000","last_change":"2024-09-11T23:19:59.736437+0000","last_active":"2024-09-11T23:20:00.745052+0000","last_peered":"2024-09-11T23:20:00.745052+0000","last_clean":"2024-09-11T23:20:00.745052+0000","last_became_active":"2024-09-11T23:19:59.736029+0000","last_became_peered":"2024-09-11T23:19:59.736029+0000","last_unstale":"2024-09-11T23:20:00.745052+0000","last_undegraded":"2024-09-11T23:20:00.745052+0000","last_fullsized":"2024-09-11T23:20:00.745052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:29.226939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744096+0000","last_change":"2024-09-11T23:19:59.736104+0000","last_active":"2024-09-11T23:20:00.744096+0000","last_peered":"2024-09-11T23:20:00.744096+0000","last_clean":"2024-09-11T23:20:00.744096+0000","last_became_active":"2024-09-11T23:19:59.735760+0000","last_became_peered":"2024-09-11T23:19:59.735760+0000","last_unstale":"2024-09-11T23:20:00.744096+0000","last_undegraded":"2024-09-11T23:20:00.744096+0000","last_fullsized":"2024-09-11T23:20:00.744096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:56.554313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.743042+0000","last_change":"2024-09-11T23:19:59.735455+0000","last_active":"2024-09-11T23:20:00.743042+0000","last_peered":"2024-09-11T23:20:00.743042+0000","last_clean":"2024-09-11T23:20:00.743042+0000","last_became_active":"2024-09-11T23:19:59.735078+0000","last_became_peered":"2024-09-11T23:19:59.735078+0000","last_unstale":"2024-09-11T23:20:00.743042+0000","last_undegraded":"2024-09-11T23:20:00.743042+0000","last_fullsized":"2024-09-11T23:20:00.743042+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:21:16.336006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744454+0000","last_change":"2024-09-11T23:19:59.745997+0000","last_active":"2024-09-11T23:20:00.744454+0000","last_peered":"2024-09-11T23:20:00.744454+0000","last_clean":"2024-09-11T23:20:00.744454+0000","last_became_active":"2024-09-11T23:19:59.745715+0000","last_became_peered":"2024-09-11T23:19:59.745715+0000","last_unstale":"2024-09-11T23:20:00.744454+0000","last_undegraded":"2024-09-11T23:20:00.744454+0000","last_fullsized":"2024-09-11T23:20:00.744454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:34.776229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746016+0000","last_change":"2024-09-11T23:19:59.734181+0000","last_active":"2024-09-11T23:20:00.746016+0000","last_peered":"2024-09-11T23:20:00.746016+0000","last_clean":"2024-09-11T23:20:00.746016+0000","last_became_active":"2024-09-11T23:19:59.733793+0000","last_became_peered":"2024-09-11T23:19:59.733793+0000","last_unstale":"2024-09-11T23:20:00.746016+0000","last_undegraded":"2024-09-11T23:20:00.746016+0000","last_fullsized":"2024-09-11T23:20:00.746016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:36.906308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.731427+0000","last_change":"2024-09-11T23:19:59.738713+0000","last_active":"2024-09-11T23:20:00.731427+0000","last_peered":"2024-09-11T23:20:00.731427+0000","last_clean":"2024-09-11T23:20:00.731427+0000","last_became_active":"2024-09-11T23:19:59.737579+0000","last_became_peered":"2024-09-11T23:19:59.737579+0000","last_unstale":"2024-09-11T23:20:00.731427+0000","last_undegraded":"2024-09-11T23:20:00.731427+0000","last_fullsized":"2024-09-11T23:20:00.731427+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:47.135775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.728349+0000","last_change":"2024-09-11T23:19:59.741381+0000","last_active":"2024-09-11T23:20:00.728349+0000","last_peered":"2024-09-11T23:20:00.728349+0000","last_clean":"2024-09-11T23:20:00.728349+0000","last_became_active":"2024-09-11T23:19:59.740914+0000","last_became_peered":"2024-09-11T23:19:59.740914+0000","last_unstale":"2024-09-11T23:20:00.728349+0000","last_undegraded":"2024-09-11T23:20:00.728349+0000","last_fullsized":"2024-09-11T23:20:00.728349+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:24:04.365704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746606+0000","last_change":"2024-09-11T23:19:59.736840+0000","last_active":"2024-09-11T23:20:00.746606+0000","last_peered":"2024-09-11T23:20:00.746606+0000","last_clean":"2024-09-11T23:20:00.746606+0000","last_became_active":"2024-09-11T23:19:59.736554+0000","last_became_peered":"2024-09-11T23:19:59.736554+0000","last_unstale":"2024-09-11T23:20:00.746606+0000","last_undegraded":"2024-09-11T23:20:00.746606+0000","last_fullsized":"2024-09-11T23:20:00.746606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:48.508369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.743748+0000","last_change":"2024-09-11T23:19:59.735719+0000","last_active":"2024-09-11T23:20:00.743748+0000","last_peered":"2024-09-11T23:20:00.743748+0000","last_clean":"2024-09-11T23:20:00.743748+0000","last_became_active":"2024-09-11T23:19:59.735397+0000","last_became_peered":"2024-09-11T23:19:59.735397+0000","last_unstale":"2024-09-11T23:20:00.743748+0000","last_undegraded":"2024-09-11T23:20:00.743748+0000","last_fullsized":"2024-09-11T23:20:00.743748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:26:19.392730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.743571+0000","last_change":"2024-09-11T23:19:59.737704+0000","last_active":"2024-09-11T23:20:00.743571+0000","last_peered":"2024-09-11T23:20:00.743571+0000","last_clean":"2024-09-11T23:20:00.743571+0000","last_became_active":"2024-09-11T23:19:59.737176+0000","last_became_peered":"2024-09-11T23:19:59.737176+0000","last_unstale":"2024-09-11T23:20:00.743571+0000","last_undegraded":"2024-09-11T23:20:00.743571+0000","last_fullsized":"2024-09-11T23:20:00.743571+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:22.938171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.731303+0000","last_change":"2024-09-11T23:19:59.735499+0000","last_active":"2024-09-11T23:20:00.731303+0000","last_peered":"2024-09-11T23:20:00.731303+0000","last_clean":"2024-09-11T23:20:00.731303+0000","last_became_active":"2024-09-11T23:19:59.735088+0000","last_became_peered":"2024-09-11T23:19:59.735088+0000","last_unstale":"2024-09-11T23:20:00.731303+0000","last_undegraded":"2024-09-11T23:20:00.731303+0000","last_fullsized":"2024-09-11T23:20:00.731303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:12.633104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744008+0000","last_change":"2024-09-11T23:19:59.743972+0000","last_active":"2024-09-11T23:20:00.744008+0000","last_peered":"2024-09-11T23:20:00.744008+0000","last_clean":"2024-09-11T23:20:00.744008+0000","last_became_active":"2024-09-11T23:19:59.743675+0000","last_became_peered":"2024-09-11T23:19:59.743675+0000","last_unstale":"2024-09-11T23:20:00.744008+0000","last_undegraded":"2024-09-11T23:20:00.744008+0000","last_fullsized":"2024-09-11T23:20:00.744008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:46.645696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:21.748261+0000","last_change":"2024-09-11T23:19:57.615472+0000","last_active":"2024-09-11T23:20:21.748261+0000","last_peered":"2024-09-11T23:20:21.748261+0000","last_clean":"2024-09-11T23:20:21.748261+0000","last_became_active":"2024-09-11T23:19:57.615104+0000","last_became_peered":"2024-09-11T23:19:57.615104+0000","last_unstale":"2024-09-11T23:20:21.748261+0000","last_undegraded":"2024-09-11T23:20:21.748261+0000","last_fullsized":"2024-09-11T23:20:21.748261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:03.601363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.743721+0000","last_change":"2024-09-11T23:19:59.741027+0000","last_active":"2024-09-11T23:20:00.743721+0000","last_peered":"2024-09-11T23:20:00.743721+0000","last_clean":"2024-09-11T23:20:00.743721+0000","last_became_active":"2024-09-11T23:19:59.740681+0000","last_became_peered":"2024-09-11T23:19:59.740681+0000","last_unstale":"2024-09-11T23:20:00.743721+0000","last_undegraded":"2024-09-11T23:20:00.743721+0000","last_fullsized":"2024-09-11T23:20:00.743721+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:36.948147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:09.726955+0000","last_change":"2024-09-11T23:19:57.612730+0000","last_active":"2024-09-11T23:20:09.726955+0000","last_peered":"2024-09-11T23:20:09.726955+0000","last_clean":"2024-09-11T23:20:09.726955+0000","last_became_active":"2024-09-11T23:19:57.612369+0000","last_became_peered":"2024-09-11T23:19:57.612369+0000","last_unstale":"2024-09-11T23:20:09.726955+0000","last_undegraded":"2024-09-11T23:20:09.726955+0000","last_fullsized":"2024-09-11T23:20:09.726955+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:17.823616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:09.727988+0000","last_change":"2024-09-11T23:19:57.611470+0000","last_active":"2024-09-11T23:20:09.727988+0000","last_peered":"2024-09-11T23:20:09.727988+0000","last_clean":"2024-09-11T23:20:09.727988+0000","last_became_active":"2024-09-11T23:19:57.611141+0000","last_became_peered":"2024-09-11T23:19:57.611141+0000","last_unstale":"2024-09-11T23:20:09.727988+0000","last_undegraded":"2024-09-11T23:20:09.727988+0000","last_fullsized":"2024-09-11T23:20:09.727988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:43.173955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.743939+0000","last_change":"2024-09-11T23:19:59.732375+0000","last_active":"2024-09-11T23:20:00.743939+0000","last_peered":"2024-09-11T23:20:00.743939+0000","last_clean":"2024-09-11T23:20:00.743939+0000","last_became_active":"2024-09-11T23:19:59.731793+0000","last_became_peered":"2024-09-11T23:19:59.731793+0000","last_unstale":"2024-09-11T23:20:00.743939+0000","last_undegraded":"2024-09-11T23:20:00.743939+0000","last_fullsized":"2024-09-11T23:20:00.743939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:02:47.278646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.743426+0000","last_change":"2024-09-11T23:19:57.614062+0000","last_active":"2024-09-11T23:20:00.743426+0000","last_peered":"2024-09-11T23:20:00.743426+0000","last_clean":"2024-09-11T23:20:00.743426+0000","last_became_active":"2024-09-11T23:19:57.613749+0000","last_became_peered":"2024-09-11T23:19:57.613749+0000","last_unstale":"2024-09-11T23:20:00.743426+0000","last_undegraded":"2024-09-11T23:20:00.743426+0000","last_fullsized":"2024-09-11T23:20:00.743426+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:45:22.299379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.742304+0000","last_change":"2024-09-11T23:19:59.730758+0000","last_active":"2024-09-11T23:20:00.742304+0000","last_peered":"2024-09-11T23:20:00.742304+0000","last_clean":"2024-09-11T23:20:00.742304+0000","last_became_active":"2024-09-11T23:19:59.730423+0000","last_became_peered":"2024-09-11T23:19:59.730423+0000","last_unstale":"2024-09-11T23:20:00.742304+0000","last_undegraded":"2024-09-11T23:20:00.742304+0000","last_fullsized":"2024-09-11T23:20:00.742304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:30.679451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:19.744821+0000","last_change":"2024-09-11T23:19:57.615574+0000","last_active":"2024-09-11T23:20:19.744821+0000","last_peered":"2024-09-11T23:20:19.744821+0000","last_clean":"2024-09-11T23:20:19.744821+0000","last_became_active":"2024-09-11T23:19:57.615192+0000","last_became_peered":"2024-09-11T23:19:57.615192+0000","last_unstale":"2024-09-11T23:20:19.744821+0000","last_undegraded":"2024-09-11T23:20:19.744821+0000","last_fullsized":"2024-09-11T23:20:19.744821+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:29.186899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746162+0000","last_change":"2024-09-11T23:19:59.738433+0000","last_active":"2024-09-11T23:20:00.746162+0000","last_peered":"2024-09-11T23:20:00.746162+0000","last_clean":"2024-09-11T23:20:00.746162+0000","last_became_active":"2024-09-11T23:19:59.738081+0000","last_became_peered":"2024-09-11T23:19:59.738081+0000","last_unstale":"2024-09-11T23:20:00.746162+0000","last_undegraded":"2024-09-11T23:20:00.746162+0000","last_fullsized":"2024-09-11T23:20:00.746162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:10:46.786454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:08.723576+0000","last_change":"2024-09-11T23:19:57.613410+0000","last_active":"2024-09-11T23:20:08.723576+0000","last_peered":"2024-09-11T23:20:08.723576+0000","last_clean":"2024-09-11T23:20:08.723576+0000","last_became_active":"2024-09-11T23:19:57.613070+0000","last_became_peered":"2024-09-11T23:19:57.613070+0000","last_unstale":"2024-09-11T23:20:08.723576+0000","last_undegraded":"2024-09-11T23:20:08.723576+0000","last_fullsized":"2024-09-11T23:20:08.723576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:07:53.971089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746895+0000","last_change":"2024-09-11T23:19:59.741081+0000","last_active":"2024-09-11T23:20:00.746895+0000","last_peered":"2024-09-11T23:20:00.746895+0000","last_clean":"2024-09-11T23:20:00.746895+0000","last_became_active":"2024-09-11T23:19:59.739984+0000","last_became_peered":"2024-09-11T23:19:59.739984+0000","last_unstale":"2024-09-11T23:20:00.746895+0000","last_undegraded":"2024-09-11T23:20:00.746895+0000","last_fullsized":"2024-09-11T23:20:00.746895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:03.513121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:08.723433+0000","last_change":"2024-09-11T23:19:57.823272+0000","last_active":"2024-09-11T23:20:08.723433+0000","last_peered":"2024-09-11T23:20:08.723433+0000","last_clean":"2024-09-11T23:20:08.723433+0000","last_became_active":"2024-09-11T23:19:57.822957+0000","last_became_peered":"2024-09-11T23:19:57.822957+0000","last_unstale":"2024-09-11T23:20:08.723433+0000","last_undegraded":"2024-09-11T23:20:08.723433+0000","last_fullsized":"2024-09-11T23:20:08.723433+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:06.779385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.745482+0000","last_change":"2024-09-11T23:19:59.738398+0000","last_active":"2024-09-11T23:20:00.745482+0000","last_peered":"2024-09-11T23:20:00.745482+0000","last_clean":"2024-09-11T23:20:00.745482+0000","last_became_active":"2024-09-11T23:19:59.738042+0000","last_became_peered":"2024-09-11T23:19:59.738042+0000","last_unstale":"2024-09-11T23:20:00.745482+0000","last_undegraded":"2024-09-11T23:20:00.745482+0000","last_fullsized":"2024-09-11T23:20:00.745482+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:05.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'21","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:20.747384+0000","last_change":"2024-09-11T23:19:57.615036+0000","last_active":"2024-09-11T23:20:20.747384+0000","last_peered":"2024-09-11T23:20:20.747384+0000","last_clean":"2024-09-11T23:20:20.747384+0000","last_became_active":"2024-09-11T23:19:57.614641+0000","last_became_peered":"2024-09-11T23:19:57.614641+0000","last_unstale":"2024-09-11T23:20:20.747384+0000","last_undegraded":"2024-09-11T23:20:20.747384+0000","last_fullsized":"2024-09-11T23:20:20.747384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:05.430508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42203,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":27,"num_read_kb":54,"num_write":22,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.745280+0000","last_change":"2024-09-11T23:19:59.741597+0000","last_active":"2024-09-11T23:20:00.745280+0000","last_peered":"2024-09-11T23:20:00.745280+0000","last_clean":"2024-09-11T23:20:00.745280+0000","last_became_active":"2024-09-11T23:19:59.741296+0000","last_became_peered":"2024-09-11T23:19:59.741296+0000","last_unstale":"2024-09-11T23:20:00.745280+0000","last_undegraded":"2024-09-11T23:20:00.745280+0000","last_fullsized":"2024-09-11T23:20:00.745280+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:06:20.742916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:09.727457+0000","last_change":"2024-09-11T23:19:57.617550+0000","last_active":"2024-09-11T23:20:09.727457+0000","last_peered":"2024-09-11T23:20:09.727457+0000","last_clean":"2024-09-11T23:20:09.727457+0000","last_became_active":"2024-09-11T23:19:57.617211+0000","last_became_peered":"2024-09-11T23:19:57.617211+0000","last_unstale":"2024-09-11T23:20:09.727457+0000","last_undegraded":"2024-09-11T23:20:09.727457+0000","last_fullsized":"2024-09-11T23:20:09.727457+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:19.707742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.728298+0000","last_change":"2024-09-11T23:19:59.742070+0000","last_active":"2024-09-11T23:20:00.728298+0000","last_peered":"2024-09-11T23:20:00.728298+0000","last_clean":"2024-09-11T23:20:00.728298+0000","last_became_active":"2024-09-11T23:19:59.739367+0000","last_became_peered":"2024-09-11T23:19:59.739367+0000","last_unstale":"2024-09-11T23:20:00.728298+0000","last_undegraded":"2024-09-11T23:20:00.728298+0000","last_fullsized":"2024-09-11T23:20:00.728298+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:27.850136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:08.722144+0000","last_change":"2024-09-11T23:19:57.618054+0000","last_active":"2024-09-11T23:20:08.722144+0000","last_peered":"2024-09-11T23:20:08.722144+0000","last_clean":"2024-09-11T23:20:08.722144+0000","last_became_active":"2024-09-11T23:19:57.617702+0000","last_became_peered":"2024-09-11T23:19:57.617702+0000","last_unstale":"2024-09-11T23:20:08.722144+0000","last_undegraded":"2024-09-11T23:20:08.722144+0000","last_fullsized":"2024-09-11T23:20:08.722144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:40.171851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.747476+0000","last_change":"2024-09-11T23:19:59.744000+0000","last_active":"2024-09-11T23:20:00.747476+0000","last_peered":"2024-09-11T23:20:00.747476+0000","last_clean":"2024-09-11T23:20:00.747476+0000","last_became_active":"2024-09-11T23:19:59.743629+0000","last_became_peered":"2024-09-11T23:19:59.743629+0000","last_unstale":"2024-09-11T23:20:00.747476+0000","last_undegraded":"2024-09-11T23:20:00.747476+0000","last_fullsized":"2024-09-11T23:20:00.747476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:20:04.133258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:21.748521+0000","last_change":"2024-09-11T23:19:57.615117+0000","last_active":"2024-09-11T23:20:21.748521+0000","last_peered":"2024-09-11T23:20:21.748521+0000","last_clean":"2024-09-11T23:20:21.748521+0000","last_became_active":"2024-09-11T23:19:57.614758+0000","last_became_peered":"2024-09-11T23:19:57.614758+0000","last_unstale":"2024-09-11T23:20:21.748521+0000","last_undegraded":"2024-09-11T23:20:21.748521+0000","last_fullsized":"2024-09-11T23:20:21.748521+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:58.113986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.747353+0000","last_change":"2024-09-11T23:19:59.739403+0000","last_active":"2024-09-11T23:20:00.747353+0000","last_peered":"2024-09-11T23:20:00.747353+0000","last_clean":"2024-09-11T23:20:00.747353+0000","last_became_active":"2024-09-11T23:19:59.739078+0000","last_became_peered":"2024-09-11T23:19:59.739078+0000","last_unstale":"2024-09-11T23:20:00.747353+0000","last_undegraded":"2024-09-11T23:20:00.747353+0000","last_fullsized":"2024-09-11T23:20:00.747353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:46.168720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:09.729626+0000","last_change":"2024-09-11T23:19:57.823323+0000","last_active":"2024-09-11T23:20:09.729626+0000","last_peered":"2024-09-11T23:20:09.729626+0000","last_clean":"2024-09-11T23:20:09.729626+0000","last_became_active":"2024-09-11T23:19:57.822947+0000","last_became_peered":"2024-09-11T23:19:57.822947+0000","last_unstale":"2024-09-11T23:20:09.729626+0000","last_undegraded":"2024-09-11T23:20:09.729626+0000","last_fullsized":"2024-09-11T23:20:09.729626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:12.542911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.742898+0000","last_change":"2024-09-11T23:19:59.741163+0000","last_active":"2024-09-11T23:20:00.742898+0000","last_peered":"2024-09-11T23:20:00.742898+0000","last_clean":"2024-09-11T23:20:00.742898+0000","last_became_active":"2024-09-11T23:19:59.740633+0000","last_became_peered":"2024-09-11T23:19:59.740633+0000","last_unstale":"2024-09-11T23:20:00.742898+0000","last_undegraded":"2024-09-11T23:20:00.742898+0000","last_fullsized":"2024-09-11T23:20:00.742898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:26.301775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:10.724091+0000","last_change":"2024-09-11T23:19:57.615804+0000","last_active":"2024-09-11T23:20:10.724091+0000","last_peered":"2024-09-11T23:20:10.724091+0000","last_clean":"2024-09-11T23:20:10.724091+0000","last_became_active":"2024-09-11T23:19:57.615443+0000","last_became_peered":"2024-09-11T23:19:57.615443+0000","last_unstale":"2024-09-11T23:20:10.724091+0000","last_undegraded":"2024-09-11T23:20:10.724091+0000","last_fullsized":"2024-09-11T23:20:10.724091+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:39.296873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.731472+0000","last_change":"2024-09-11T23:19:59.735411+0000","last_active":"2024-09-11T23:20:00.731472+0000","last_peered":"2024-09-11T23:20:00.731472+0000","last_clean":"2024-09-11T23:20:00.731472+0000","last_became_active":"2024-09-11T23:19:59.735089+0000","last_became_peered":"2024-09-11T23:19:59.735089+0000","last_unstale":"2024-09-11T23:20:00.731472+0000","last_undegraded":"2024-09-11T23:20:00.731472+0000","last_fullsized":"2024-09-11T23:20:00.731472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:06.691739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.743223+0000","last_change":"2024-09-11T23:19:57.612873+0000","last_active":"2024-09-11T23:20:00.743223+0000","last_peered":"2024-09-11T23:20:00.743223+0000","last_clean":"2024-09-11T23:20:00.743223+0000","last_became_active":"2024-09-11T23:19:57.612566+0000","last_became_peered":"2024-09-11T23:19:57.612566+0000","last_unstale":"2024-09-11T23:20:00.743223+0000","last_undegraded":"2024-09-11T23:20:00.743223+0000","last_fullsized":"2024-09-11T23:20:00.743223+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:46.218212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.747753+0000","last_change":"2024-09-11T23:19:59.736352+0000","last_active":"2024-09-11T23:20:00.747753+0000","last_peered":"2024-09-11T23:20:00.747753+0000","last_clean":"2024-09-11T23:20:00.747753+0000","last_became_active":"2024-09-11T23:19:59.735402+0000","last_became_peered":"2024-09-11T23:19:59.735402+0000","last_unstale":"2024-09-11T23:20:00.747753+0000","last_undegraded":"2024-09-11T23:20:00.747753+0000","last_fullsized":"2024-09-11T23:20:00.747753+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:41.854526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.752591+0000","last_change":"2024-09-11T23:19:57.612439+0000","last_active":"2024-09-11T23:20:00.752591+0000","last_peered":"2024-09-11T23:20:00.752591+0000","last_clean":"2024-09-11T23:20:00.752591+0000","last_became_active":"2024-09-11T23:19:57.612030+0000","last_became_peered":"2024-09-11T23:19:57.612030+0000","last_unstale":"2024-09-11T23:20:00.752591+0000","last_undegraded":"2024-09-11T23:20:00.752591+0000","last_fullsized":"2024-09-11T23:20:00.752591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:36:24.579246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.747209+0000","last_change":"2024-09-11T23:19:59.741241+0000","last_active":"2024-09-11T23:20:00.747209+0000","last_peered":"2024-09-11T23:20:00.747209+0000","last_clean":"2024-09-11T23:20:00.747209+0000","last_became_active":"2024-09-11T23:19:59.740265+0000","last_became_peered":"2024-09-11T23:19:59.740265+0000","last_unstale":"2024-09-11T23:20:00.747209+0000","last_undegraded":"2024-09-11T23:20:00.747209+0000","last_fullsized":"2024-09-11T23:20:00.747209+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:00.676762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:09.728669+0000","last_change":"2024-09-11T23:19:57.825238+0000","last_active":"2024-09-11T23:20:09.728669+0000","last_peered":"2024-09-11T23:20:09.728669+0000","last_clean":"2024-09-11T23:20:09.728669+0000","last_became_active":"2024-09-11T23:19:57.824732+0000","last_became_peered":"2024-09-11T23:19:57.824732+0000","last_unstale":"2024-09-11T23:20:09.728669+0000","last_undegraded":"2024-09-11T23:20:09.728669+0000","last_fullsized":"2024-09-11T23:20:09.728669+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:24.465863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.747591+0000","last_change":"2024-09-11T23:19:59.741338+0000","last_active":"2024-09-11T23:20:00.747591+0000","last_peered":"2024-09-11T23:20:00.747591+0000","last_clean":"2024-09-11T23:20:00.747591+0000","last_became_active":"2024-09-11T23:19:59.741019+0000","last_became_peered":"2024-09-11T23:19:59.741019+0000","last_unstale":"2024-09-11T23:20:00.747591+0000","last_undegraded":"2024-09-11T23:20:00.747591+0000","last_fullsized":"2024-09-11T23:20:00.747591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:36:03.480006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":344,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.747469+0000","last_change":"2024-09-11T23:18:45.606646+0000","last_active":"2024-09-11T23:20:00.747469+0000","last_peered":"2024-09-11T23:20:00.747469+0000","last_clean":"2024-09-11T23:20:00.747469+0000","last_became_active":"2024-09-11T23:18:45.606065+0000","last_became_peered":"2024-09-11T23:18:45.606065+0000","last_unstale":"2024-09-11T23:20:00.747469+0000","last_undegraded":"2024-09-11T23:20:00.747469+0000","last_fullsized":"2024-09-11T23:20:00.747469+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:15:17.148562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:15:17.148562+0000","last_clean_scrub_stamp":"2024-09-11T23:15:17.148562+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:07.992971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:10.729494+0000","last_change":"2024-09-11T23:19:57.612584+0000","last_active":"2024-09-11T23:20:10.729494+0000","last_peered":"2024-09-11T23:20:10.729494+0000","last_clean":"2024-09-11T23:20:10.729494+0000","last_became_active":"2024-09-11T23:19:57.612211+0000","last_became_peered":"2024-09-11T23:19:57.612211+0000","last_unstale":"2024-09-11T23:20:10.729494+0000","last_undegraded":"2024-09-11T23:20:10.729494+0000","last_fullsized":"2024-09-11T23:20:10.729494+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:04.026233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744949+0000","last_change":"2024-09-11T23:19:59.734502+0000","last_active":"2024-09-11T23:20:00.744949+0000","last_peered":"2024-09-11T23:20:00.744949+0000","last_clean":"2024-09-11T23:20:00.744949+0000","last_became_active":"2024-09-11T23:19:59.734160+0000","last_became_peered":"2024-09-11T23:19:59.734160+0000","last_unstale":"2024-09-11T23:20:00.744949+0000","last_undegraded":"2024-09-11T23:20:00.744949+0000","last_fullsized":"2024-09-11T23:20:00.744949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:58.335773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:09.727469+0000","last_change":"2024-09-11T23:19:57.618283+0000","last_active":"2024-09-11T23:20:09.727469+0000","last_peered":"2024-09-11T23:20:09.727469+0000","last_clean":"2024-09-11T23:20:09.727469+0000","last_became_active":"2024-09-11T23:19:57.617942+0000","last_became_peered":"2024-09-11T23:19:57.617942+0000","last_unstale":"2024-09-11T23:20:09.727469+0000","last_undegraded":"2024-09-11T23:20:09.727469+0000","last_fullsized":"2024-09-11T23:20:09.727469+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:30.832602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.728041+0000","last_change":"2024-09-11T23:19:59.740730+0000","last_active":"2024-09-11T23:20:00.728041+0000","last_peered":"2024-09-11T23:20:00.728041+0000","last_clean":"2024-09-11T23:20:00.728041+0000","last_became_active":"2024-09-11T23:19:59.740369+0000","last_became_peered":"2024-09-11T23:19:59.740369+0000","last_unstale":"2024-09-11T23:20:00.728041+0000","last_undegraded":"2024-09-11T23:20:00.728041+0000","last_fullsized":"2024-09-11T23:20:00.728041+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:50.264644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:21.747869+0000","last_change":"2024-09-11T23:19:57.823253+0000","last_active":"2024-09-11T23:20:21.747869+0000","last_peered":"2024-09-11T23:20:21.747869+0000","last_clean":"2024-09-11T23:20:21.747869+0000","last_became_active":"2024-09-11T23:19:57.822895+0000","last_became_peered":"2024-09-11T23:19:57.822895+0000","last_unstale":"2024-09-11T23:20:21.747869+0000","last_undegraded":"2024-09-11T23:20:21.747869+0000","last_fullsized":"2024-09-11T23:20:21.747869+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:22.998371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.743210+0000","last_change":"2024-09-11T23:19:59.742248+0000","last_active":"2024-09-11T23:20:00.743210+0000","last_peered":"2024-09-11T23:20:00.743210+0000","last_clean":"2024-09-11T23:20:00.743210+0000","last_became_active":"2024-09-11T23:19:59.741810+0000","last_became_peered":"2024-09-11T23:19:59.741810+0000","last_unstale":"2024-09-11T23:20:00.743210+0000","last_undegraded":"2024-09-11T23:20:00.743210+0000","last_fullsized":"2024-09-11T23:20:00.743210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:19:52.720286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.747924+0000","last_change":"2024-09-11T23:19:57.616244+0000","last_active":"2024-09-11T23:20:00.747924+0000","last_peered":"2024-09-11T23:20:00.747924+0000","last_clean":"2024-09-11T23:20:00.747924+0000","last_became_active":"2024-09-11T23:19:57.615868+0000","last_became_peered":"2024-09-11T23:19:57.615868+0000","last_unstale":"2024-09-11T23:20:00.747924+0000","last_undegraded":"2024-09-11T23:20:00.747924+0000","last_fullsized":"2024-09-11T23:20:00.747924+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:42.510896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.742387+0000","last_change":"2024-09-11T23:19:59.731524+0000","last_active":"2024-09-11T23:20:00.742387+0000","last_peered":"2024-09-11T23:20:00.742387+0000","last_clean":"2024-09-11T23:20:00.742387+0000","last_became_active":"2024-09-11T23:19:59.731185+0000","last_became_peered":"2024-09-11T23:19:59.731185+0000","last_unstale":"2024-09-11T23:20:00.742387+0000","last_undegraded":"2024-09-11T23:20:00.742387+0000","last_fullsized":"2024-09-11T23:20:00.742387+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:56.443226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.742378+0000","last_change":"2024-09-11T23:19:57.824334+0000","last_active":"2024-09-11T23:20:00.742378+0000","last_peered":"2024-09-11T23:20:00.742378+0000","last_clean":"2024-09-11T23:20:00.742378+0000","last_became_active":"2024-09-11T23:19:57.823972+0000","last_became_peered":"2024-09-11T23:19:57.823972+0000","last_unstale":"2024-09-11T23:20:00.742378+0000","last_undegraded":"2024-09-11T23:20:00.742378+0000","last_fullsized":"2024-09-11T23:20:00.742378+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:39.183281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:01.696043+0000","last_change":"2024-09-11T23:19:59.728064+0000","last_active":"2024-09-11T23:20:01.696043+0000","last_peered":"2024-09-11T23:20:01.696043+0000","last_clean":"2024-09-11T23:20:01.696043+0000","last_became_active":"2024-09-11T23:19:59.727732+0000","last_became_peered":"2024-09-11T23:19:59.727732+0000","last_unstale":"2024-09-11T23:20:01.696043+0000","last_undegraded":"2024-09-11T23:20:01.696043+0000","last_fullsized":"2024-09-11T23:20:01.696043+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:15.383974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:08.721231+0000","last_change":"2024-09-11T23:19:57.824362+0000","last_active":"2024-09-11T23:20:08.721231+0000","last_peered":"2024-09-11T23:20:08.721231+0000","last_clean":"2024-09-11T23:20:08.721231+0000","last_became_active":"2024-09-11T23:19:57.823753+0000","last_became_peered":"2024-09-11T23:19:57.823753+0000","last_unstale":"2024-09-11T23:20:08.721231+0000","last_undegraded":"2024-09-11T23:20:08.721231+0000","last_fullsized":"2024-09-11T23:20:08.721231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:56.357268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.728474+0000","last_change":"2024-09-11T23:19:59.730623+0000","last_active":"2024-09-11T23:20:00.728474+0000","last_peered":"2024-09-11T23:20:00.728474+0000","last_clean":"2024-09-11T23:20:00.728474+0000","last_became_active":"2024-09-11T23:19:59.730307+0000","last_became_peered":"2024-09-11T23:19:59.730307+0000","last_unstale":"2024-09-11T23:20:00.728474+0000","last_undegraded":"2024-09-11T23:20:00.728474+0000","last_fullsized":"2024-09-11T23:20:00.728474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:52.874884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'10","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:24.743300+0000","last_change":"2024-09-11T23:19:57.611730+0000","last_active":"2024-09-11T23:20:24.743300+0000","last_peered":"2024-09-11T23:20:24.743300+0000","last_clean":"2024-09-11T23:20:24.743300+0000","last_became_active":"2024-09-11T23:19:57.611394+0000","last_became_peered":"2024-09-11T23:19:57.611394+0000","last_unstale":"2024-09-11T23:20:24.743300+0000","last_undegraded":"2024-09-11T23:20:24.743300+0000","last_fullsized":"2024-09-11T23:20:24.743300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:15.518573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744483+0000","last_change":"2024-09-11T23:19:59.741396+0000","last_active":"2024-09-11T23:20:00.744483+0000","last_peered":"2024-09-11T23:20:00.744483+0000","last_clean":"2024-09-11T23:20:00.744483+0000","last_became_active":"2024-09-11T23:19:59.741102+0000","last_became_peered":"2024-09-11T23:19:59.741102+0000","last_unstale":"2024-09-11T23:20:00.744483+0000","last_undegraded":"2024-09-11T23:20:00.744483+0000","last_fullsized":"2024-09-11T23:20:00.744483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:08.759512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:08.722269+0000","last_change":"2024-09-11T23:19:57.824812+0000","last_active":"2024-09-11T23:20:08.722269+0000","last_peered":"2024-09-11T23:20:08.722269+0000","last_clean":"2024-09-11T23:20:08.722269+0000","last_became_active":"2024-09-11T23:19:57.824454+0000","last_became_peered":"2024-09-11T23:19:57.824454+0000","last_unstale":"2024-09-11T23:20:08.722269+0000","last_undegraded":"2024-09-11T23:20:08.722269+0000","last_fullsized":"2024-09-11T23:20:08.722269+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:25.275573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746707+0000","last_change":"2024-09-11T23:19:59.735857+0000","last_active":"2024-09-11T23:20:00.746707+0000","last_peered":"2024-09-11T23:20:00.746707+0000","last_clean":"2024-09-11T23:20:00.746707+0000","last_became_active":"2024-09-11T23:19:59.735541+0000","last_became_peered":"2024-09-11T23:19:59.735541+0000","last_unstale":"2024-09-11T23:20:00.746707+0000","last_undegraded":"2024-09-11T23:20:00.746707+0000","last_fullsized":"2024-09-11T23:20:00.746707+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:21.600490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:08.722858+0000","last_change":"2024-09-11T23:19:57.823468+0000","last_active":"2024-09-11T23:20:08.722858+0000","last_peered":"2024-09-11T23:20:08.722858+0000","last_clean":"2024-09-11T23:20:08.722858+0000","last_became_active":"2024-09-11T23:19:57.823154+0000","last_became_peered":"2024-09-11T23:19:57.823154+0000","last_unstale":"2024-09-11T23:20:08.722858+0000","last_undegraded":"2024-09-11T23:20:08.722858+0000","last_fullsized":"2024-09-11T23:20:08.722858+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:28.770627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.742722+0000","last_change":"2024-09-11T23:19:59.741263+0000","last_active":"2024-09-11T23:20:00.742722+0000","last_peered":"2024-09-11T23:20:00.742722+0000","last_clean":"2024-09-11T23:20:00.742722+0000","last_became_active":"2024-09-11T23:19:59.740938+0000","last_became_peered":"2024-09-11T23:19:59.740938+0000","last_unstale":"2024-09-11T23:20:00.742722+0000","last_undegraded":"2024-09-11T23:20:00.742722+0000","last_fullsized":"2024-09-11T23:20:00.742722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:54.768652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:08.723486+0000","last_change":"2024-09-11T23:19:57.612517+0000","last_active":"2024-09-11T23:20:08.723486+0000","last_peered":"2024-09-11T23:20:08.723486+0000","last_clean":"2024-09-11T23:20:08.723486+0000","last_became_active":"2024-09-11T23:19:57.612188+0000","last_became_peered":"2024-09-11T23:19:57.612188+0000","last_unstale":"2024-09-11T23:20:08.723486+0000","last_undegraded":"2024-09-11T23:20:08.723486+0000","last_fullsized":"2024-09-11T23:20:08.723486+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:52.896413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.731401+0000","last_change":"2024-09-11T23:19:59.732520+0000","last_active":"2024-09-11T23:20:00.731401+0000","last_peered":"2024-09-11T23:20:00.731401+0000","last_clean":"2024-09-11T23:20:00.731401+0000","last_became_active":"2024-09-11T23:19:59.732173+0000","last_became_peered":"2024-09-11T23:19:59.732173+0000","last_unstale":"2024-09-11T23:20:00.731401+0000","last_undegraded":"2024-09-11T23:20:00.731401+0000","last_fullsized":"2024-09-11T23:20:00.731401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:04.626312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:01.159050+0000","last_change":"2024-09-11T23:19:57.613962+0000","last_active":"2024-09-11T23:20:01.159050+0000","last_peered":"2024-09-11T23:20:01.159050+0000","last_clean":"2024-09-11T23:20:01.159050+0000","last_became_active":"2024-09-11T23:19:57.613593+0000","last_became_peered":"2024-09-11T23:19:57.613593+0000","last_unstale":"2024-09-11T23:20:01.159050+0000","last_undegraded":"2024-09-11T23:20:01.159050+0000","last_fullsized":"2024-09-11T23:20:01.159050+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:12.400377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744905+0000","last_change":"2024-09-11T23:19:59.734667+0000","last_active":"2024-09-11T23:20:00.744905+0000","last_peered":"2024-09-11T23:20:00.744905+0000","last_clean":"2024-09-11T23:20:00.744905+0000","last_became_active":"2024-09-11T23:19:59.733754+0000","last_became_peered":"2024-09-11T23:19:59.733754+0000","last_unstale":"2024-09-11T23:20:00.744905+0000","last_undegraded":"2024-09-11T23:20:00.744905+0000","last_fullsized":"2024-09-11T23:20:00.744905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:57.392272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:08.722751+0000","last_change":"2024-09-11T23:19:57.614579+0000","last_active":"2024-09-11T23:20:08.722751+0000","last_peered":"2024-09-11T23:20:08.722751+0000","last_clean":"2024-09-11T23:20:08.722751+0000","last_became_active":"2024-09-11T23:19:57.614227+0000","last_became_peered":"2024-09-11T23:19:57.614227+0000","last_unstale":"2024-09-11T23:20:08.722751+0000","last_undegraded":"2024-09-11T23:20:08.722751+0000","last_fullsized":"2024-09-11T23:20:08.722751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:47.952920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.747070+0000","last_change":"2024-09-11T23:19:59.744219+0000","last_active":"2024-09-11T23:20:00.747070+0000","last_peered":"2024-09-11T23:20:00.747070+0000","last_clean":"2024-09-11T23:20:00.747070+0000","last_became_active":"2024-09-11T23:19:59.743914+0000","last_became_peered":"2024-09-11T23:19:59.743914+0000","last_unstale":"2024-09-11T23:20:00.747070+0000","last_undegraded":"2024-09-11T23:20:00.747070+0000","last_fullsized":"2024-09-11T23:20:00.747070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:52.149588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744576+0000","last_change":"2024-09-11T23:19:59.740502+0000","last_active":"2024-09-11T23:20:00.744576+0000","last_peered":"2024-09-11T23:20:00.744576+0000","last_clean":"2024-09-11T23:20:00.744576+0000","last_became_active":"2024-09-11T23:19:59.740122+0000","last_became_peered":"2024-09-11T23:19:59.740122+0000","last_unstale":"2024-09-11T23:20:00.744576+0000","last_undegraded":"2024-09-11T23:20:00.744576+0000","last_fullsized":"2024-09-11T23:20:00.744576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:36.446241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.752059+0000","last_change":"2024-09-11T23:19:57.610389+0000","last_active":"2024-09-11T23:20:00.752059+0000","last_peered":"2024-09-11T23:20:00.752059+0000","last_clean":"2024-09-11T23:20:00.752059+0000","last_became_active":"2024-09-11T23:19:57.609911+0000","last_became_peered":"2024-09-11T23:19:57.609911+0000","last_unstale":"2024-09-11T23:20:00.752059+0000","last_undegraded":"2024-09-11T23:20:00.752059+0000","last_fullsized":"2024-09-11T23:20:00.752059+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:23.949929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.745349+0000","last_change":"2024-09-11T23:19:59.736867+0000","last_active":"2024-09-11T23:20:00.745349+0000","last_peered":"2024-09-11T23:20:00.745349+0000","last_clean":"2024-09-11T23:20:00.745349+0000","last_became_active":"2024-09-11T23:19:59.736341+0000","last_became_peered":"2024-09-11T23:19:59.736341+0000","last_unstale":"2024-09-11T23:20:00.745349+0000","last_undegraded":"2024-09-11T23:20:00.745349+0000","last_fullsized":"2024-09-11T23:20:00.745349+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:57:51.015235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:09.727223+0000","last_change":"2024-09-11T23:19:57.615855+0000","last_active":"2024-09-11T23:20:09.727223+0000","last_peered":"2024-09-11T23:20:09.727223+0000","last_clean":"2024-09-11T23:20:09.727223+0000","last_became_active":"2024-09-11T23:19:57.615308+0000","last_became_peered":"2024-09-11T23:19:57.615308+0000","last_unstale":"2024-09-11T23:20:09.727223+0000","last_undegraded":"2024-09-11T23:20:09.727223+0000","last_fullsized":"2024-09-11T23:20:09.727223+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:29:16.143982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.746858+0000","last_change":"2024-09-11T23:19:59.742049+0000","last_active":"2024-09-11T23:20:00.746858+0000","last_peered":"2024-09-11T23:20:00.746858+0000","last_clean":"2024-09-11T23:20:00.746858+0000","last_became_active":"2024-09-11T23:19:59.741763+0000","last_became_peered":"2024-09-11T23:19:59.741763+0000","last_unstale":"2024-09-11T23:20:00.746858+0000","last_undegraded":"2024-09-11T23:20:00.746858+0000","last_fullsized":"2024-09-11T23:20:00.746858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:48.698896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:01.232954+0000","last_change":"2024-09-11T23:19:57.822949+0000","last_active":"2024-09-11T23:20:01.232954+0000","last_peered":"2024-09-11T23:20:01.232954+0000","last_clean":"2024-09-11T23:20:01.232954+0000","last_became_active":"2024-09-11T23:19:57.822564+0000","last_became_peered":"2024-09-11T23:19:57.822564+0000","last_unstale":"2024-09-11T23:20:01.232954+0000","last_undegraded":"2024-09-11T23:20:01.232954+0000","last_fullsized":"2024-09-11T23:20:01.232954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:29.532109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:00.744563+0000","last_change":"2024-09-11T23:19:59.735275+0000","last_active":"2024-09-11T23:20:00.744563+0000","last_peered":"2024-09-11T23:20:00.744563+0000","last_clean":"2024-09-11T23:20:00.744563+0000","last_became_active":"2024-09-11T23:19:59.734954+0000","last_became_peered":"2024-09-11T23:19:59.734954+0000","last_unstale":"2024-09-11T23:20:00.744563+0000","last_undegraded":"2024-09-11T23:20:00.744563+0000","last_fullsized":"2024-09-11T23:20:00.744563+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:55:43.511578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T23:20:09.729237+0000","last_change":"2024-09-11T23:19:57.613323+0000","last_active":"2024-09-11T23:20:09.729237+0000","last_peered":"2024-09-11T23:20:09.729237+0000","last_clean":"2024-09-11T23:20:09.729237+0000","last_became_active":"2024-09-11T23:19:57.612985+0000","last_became_peered":"2024-09-11T23:19:57.612985+0000","last_unstale":"2024-09-11T23:20:09.729237+0000","last_undegraded":"2024-09-11T23:20:09.729237+0000","last_fullsized":"2024-09-11T23:20:09.729237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:13:09.179547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":46206,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":96,"num_read_kb":97,"num_write":153,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":164120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285620,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514513,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":424539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":1058616,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906973,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":424775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168609,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":424695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":1016627,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561067,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":424164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790000,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1015918,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":424994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149755,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":905216,"data_stored":424475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":480261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":946176,"data_stored":478911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":425435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34680,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34680,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34680,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":44174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:20:29.272 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T23:20:29.273 INFO:tasks.cephadm:Setting up client nodes... 2024-09-11T23:20:29.273 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T23:20:29.274 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:28 smithi090 ceph-mon[35130]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:20:29.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:28 smithi097 ceph-mon[35562]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:20:29.531 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:29.934 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:29 smithi043 ceph-mon[30950]: pgmap v296: 97 pgs: 97 active+clean; 634 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-09-11T23:20:30.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.025+0000 7f9541b92640 1 -- 172.21.15.43:0/22237103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f953c102e00 msgr2=0x7f953c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:30.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.025+0000 7f9541b92640 1 --2- 172.21.15.43:0/22237103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f953c102e00 0x7f953c103280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9530009900 tx=0x7f953002a410 comp rx=0 tx=0).stop 2024-09-11T23:20:30.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.025+0000 7f9541b92640 1 -- 172.21.15.43:0/22237103 shutdown_connections 2024-09-11T23:20:30.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.025+0000 7f9541b92640 1 --2- 172.21.15.43:0/22237103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953c1037c0 0x7f953c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.025+0000 7f9541b92640 1 --2- 172.21.15.43:0/22237103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f953c102e00 0x7f953c103280 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.025+0000 7f9541b92640 1 --2- 172.21.15.43:0/22237103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f953c101c00 0x7f953c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.025+0000 7f9541b92640 1 -- 172.21.15.43:0/22237103 >> 172.21.15.43:0/22237103 conn(0x7f953c0fd390 msgr2=0x7f953c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:30.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.025+0000 7f9541b92640 1 -- 172.21.15.43:0/22237103 shutdown_connections 2024-09-11T23:20:30.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.025+0000 7f9541b92640 1 -- 172.21.15.43:0/22237103 wait complete. 2024-09-11T23:20:30.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.026+0000 7f9541b92640 1 Processor -- start 2024-09-11T23:20:30.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.026+0000 7f9541b92640 1 -- start start 2024-09-11T23:20:30.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.026+0000 7f9541b92640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953c101c00 0x7f953c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.026+0000 7f9541b92640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f953c102e00 0x7f953c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.026+0000 7f9541b92640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f953c1037c0 0x7f953c19f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.026+0000 7f9541b92640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f953c10d860 con 0x7f953c102e00 2024-09-11T23:20:30.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.026+0000 7f9541b92640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f953c10d6e0 con 0x7f953c1037c0 2024-09-11T23:20:30.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.026+0000 7f9541b92640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f953c10d9e0 con 0x7f953c101c00 2024-09-11T23:20:30.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.026+0000 7f953affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f953c102e00 0x7f953c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.026+0000 7f953affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f953c102e00 0x7f953c1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40010/0 (socket says 172.21.15.43:40010) 2024-09-11T23:20:30.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.026+0000 7f953affd640 1 -- 172.21.15.43:0/3385618022 learned_addr learned my addr 172.21.15.43:0/3385618022 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:30.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.027+0000 7f953bfff640 1 --2- 172.21.15.43:0/3385618022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f953c1037c0 0x7f953c19f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.027+0000 7f953b7fe640 1 --2- 172.21.15.43:0/3385618022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953c101c00 0x7f953c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.027+0000 7f953affd640 1 -- 172.21.15.43:0/3385618022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953c101c00 msgr2=0x7f953c197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:30.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.027+0000 7f953affd640 1 --2- 172.21.15.43:0/3385618022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953c101c00 0x7f953c197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.027+0000 7f953affd640 1 -- 172.21.15.43:0/3385618022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f953c1037c0 msgr2=0x7f953c19f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:30.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.027+0000 7f953affd640 1 --2- 172.21.15.43:0/3385618022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f953c1037c0 0x7f953c19f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.027+0000 7f953affd640 1 -- 172.21.15.43:0/3385618022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953c19fad0 con 0x7f953c102e00 2024-09-11T23:20:30.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.027+0000 7f953affd640 1 --2- 172.21.15.43:0/3385618022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f953c102e00 0x7f953c1983e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9530009230 tx=0x7f9530004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:30.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.027+0000 7f9538ff9640 1 -- 172.21.15.43:0/3385618022 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953003f030 con 0x7f953c102e00 2024-09-11T23:20:30.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.027+0000 7f9541b92640 1 -- 172.21.15.43:0/3385618022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f953c19fd60 con 0x7f953c102e00 2024-09-11T23:20:30.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.027+0000 7f9538ff9640 1 -- 172.21.15.43:0/3385618022 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9530003ce0 con 0x7f953c102e00 2024-09-11T23:20:30.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.027+0000 7f9538ff9640 1 -- 172.21.15.43:0/3385618022 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530035070 con 0x7f953c102e00 2024-09-11T23:20:30.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.027+0000 7f9541b92640 1 -- 172.21.15.43:0/3385618022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f953c1a01e0 con 0x7f953c102e00 2024-09-11T23:20:30.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.028+0000 7f9541b92640 1 -- 172.21.15.43:0/3385618022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f953c108fa0 con 0x7f953c102e00 2024-09-11T23:20:30.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.030+0000 7f9538ff9640 1 -- 172.21.15.43:0/3385618022 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9530042030 con 0x7f953c102e00 2024-09-11T23:20:30.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.031+0000 7f9538ff9640 1 --2- 172.21.15.43:0/3385618022 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f95240a2c40 0x7f95240a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:30.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.031+0000 7f953b7fe640 1 --2- 172.21.15.43:0/3385618022 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f95240a2c40 0x7f95240a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:30.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.031+0000 7f9538ff9640 1 -- 172.21.15.43:0/3385618022 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f95300efa90 con 0x7f953c102e00 2024-09-11T23:20:30.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.031+0000 7f953b7fe640 1 --2- 172.21.15.43:0/3385618022 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f95240a2c40 0x7f95240a5100 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f952c005db0 tx=0x7f952c005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:30.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.034+0000 7f9538ff9640 1 -- 172.21.15.43:0/3385618022 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f95300bca40 con 0x7f953c102e00 2024-09-11T23:20:30.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:29 smithi090 ceph-mon[35130]: pgmap v296: 97 pgs: 97 active+clean; 634 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-09-11T23:20:30.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.276+0000 7f9541b92640 1 -- 172.21.15.43:0/3385618022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f953c02d7c0 con 0x7f953c102e00 2024-09-11T23:20:30.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.278+0000 7f9538ff9640 1 -- 172.21.15.43:0/3385618022 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f95300bc340 con 0x7f953c102e00 2024-09-11T23:20:30.278 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-09-11T23:20:30.279 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQC+JeJmp5qJEBAAxZ6zNFGrPlf/51+PuBUbOA== 2024-09-11T23:20:30.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.280+0000 7f9541b92640 1 -- 172.21.15.43:0/3385618022 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f95240a2c40 msgr2=0x7f95240a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:30.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.280+0000 7f9541b92640 1 --2- 172.21.15.43:0/3385618022 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f95240a2c40 0x7f95240a5100 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f952c005db0 tx=0x7f952c005d40 comp rx=0 tx=0).stop 2024-09-11T23:20:30.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.280+0000 7f9541b92640 1 -- 172.21.15.43:0/3385618022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f953c102e00 msgr2=0x7f953c1983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:30.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.280+0000 7f9541b92640 1 --2- 172.21.15.43:0/3385618022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f953c102e00 0x7f953c1983e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9530009230 tx=0x7f9530004360 comp rx=0 tx=0).stop 2024-09-11T23:20:30.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.281+0000 7f9541b92640 1 -- 172.21.15.43:0/3385618022 shutdown_connections 2024-09-11T23:20:30.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.281+0000 7f9541b92640 1 --2- 172.21.15.43:0/3385618022 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f95240a2c40 0x7f95240a5100 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.281+0000 7f9541b92640 1 --2- 172.21.15.43:0/3385618022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f953c1037c0 0x7f953c19f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.281+0000 7f9541b92640 1 --2- 172.21.15.43:0/3385618022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f953c102e00 0x7f953c1983e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.281+0000 7f9541b92640 1 --2- 172.21.15.43:0/3385618022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953c101c00 0x7f953c197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:30.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.281+0000 7f9541b92640 1 -- 172.21.15.43:0/3385618022 >> 172.21.15.43:0/3385618022 conn(0x7f953c0fd390 msgr2=0x7f953c106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:30.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.281+0000 7f9541b92640 1 -- 172.21.15.43:0/3385618022 shutdown_connections 2024-09-11T23:20:30.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:30.281+0000 7f9541b92640 1 -- 172.21.15.43:0/3385618022 wait complete. 2024-09-11T23:20:30.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:29 smithi097 ceph-mon[35562]: pgmap v296: 97 pgs: 97 active+clean; 634 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-09-11T23:20:30.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:30 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3385618022' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T23:20:30.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:30 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3385618022' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T23:20:30.987 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T23:20:30.987 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-11T23:20:30.987 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T23:20:31.027 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T23:20:31.027 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-11T23:20:31.027 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-11T23:20:31.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:30 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3385618022' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T23:20:31.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:30 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3385618022' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T23:20:31.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:30 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3385618022' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T23:20:31.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:30 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3385618022' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T23:20:31.324 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:31.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.798+0000 7f0d25086640 1 -- 172.21.15.43:0/3328629775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d20101de0 msgr2=0x7f0d201021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:31.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.798+0000 7f0d25086640 1 --2- 172.21.15.43:0/3328629775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d20101de0 0x7f0d201021c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0d10009900 tx=0x7f0d1002a410 comp rx=0 tx=0).stop 2024-09-11T23:20:31.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.798+0000 7f0d25086640 1 -- 172.21.15.43:0/3328629775 shutdown_connections 2024-09-11T23:20:31.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.798+0000 7f0d25086640 1 --2- 172.21.15.43:0/3328629775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d20106f10 0x7f0d2010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:31.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.798+0000 7f0d25086640 1 --2- 172.21.15.43:0/3328629775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d20102790 0x7f0d201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:31.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.798+0000 7f0d25086640 1 --2- 172.21.15.43:0/3328629775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d20101de0 0x7f0d201021c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:31.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.798+0000 7f0d25086640 1 -- 172.21.15.43:0/3328629775 >> 172.21.15.43:0/3328629775 conn(0x7f0d200fd8c0 msgr2=0x7f0d200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:31.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.798+0000 7f0d25086640 1 -- 172.21.15.43:0/3328629775 shutdown_connections 2024-09-11T23:20:31.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.798+0000 7f0d25086640 1 -- 172.21.15.43:0/3328629775 wait complete. 2024-09-11T23:20:31.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.799+0000 7f0d25086640 1 Processor -- start 2024-09-11T23:20:31.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.800+0000 7f0d25086640 1 -- start start 2024-09-11T23:20:31.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.800+0000 7f0d25086640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d20101de0 0x7f0d201985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:31.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.800+0000 7f0d25086640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d20102790 0x7f0d20198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:31.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.800+0000 7f0d25086640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d20106f10 0x7f0d2019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:31.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.800+0000 7f0d25086640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20112060 con 0x7f0d20106f10 2024-09-11T23:20:31.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.800+0000 7f0d25086640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20111ee0 con 0x7f0d20102790 2024-09-11T23:20:31.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.800+0000 7f0d25086640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0d201121e0 con 0x7f0d20101de0 2024-09-11T23:20:31.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.800+0000 7f0d1f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d20106f10 0x7f0d2019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:31.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.800+0000 7f0d1f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d20106f10 0x7f0d2019ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43990/0 (socket says 172.21.15.43:43990) 2024-09-11T23:20:31.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.800+0000 7f0d1f59e640 1 -- 172.21.15.43:0/2877713969 learned_addr learned my addr 172.21.15.43:0/2877713969 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:31.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.800+0000 7f0d1ed9d640 1 --2- 172.21.15.43:0/2877713969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d20101de0 0x7f0d201985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:31.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.800+0000 7f0d1e59c640 1 --2- 172.21.15.43:0/2877713969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d20102790 0x7f0d20198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:31.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.800+0000 7f0d1f59e640 1 -- 172.21.15.43:0/2877713969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d20101de0 msgr2=0x7f0d201985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:31.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.800+0000 7f0d1f59e640 1 --2- 172.21.15.43:0/2877713969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d20101de0 0x7f0d201985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:31.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.800+0000 7f0d1f59e640 1 -- 172.21.15.43:0/2877713969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d20102790 msgr2=0x7f0d20198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:31.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.800+0000 7f0d1f59e640 1 --2- 172.21.15.43:0/2877713969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d20102790 0x7f0d20198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:31.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.800+0000 7f0d1f59e640 1 -- 172.21.15.43:0/2877713969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2019d570 con 0x7f0d20106f10 2024-09-11T23:20:31.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.800+0000 7f0d1ed9d640 1 --2- 172.21.15.43:0/2877713969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d20101de0 0x7f0d201985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:20:31.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.801+0000 7f0d1f59e640 1 --2- 172.21.15.43:0/2877713969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d20106f10 0x7f0d2019ce90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800b970 tx=0x7f0d1800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:31.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.801+0000 7f0d07fff640 1 -- 172.21.15.43:0/2877713969 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18019070 con 0x7f0d20106f10 2024-09-11T23:20:31.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.801+0000 7f0d07fff640 1 -- 172.21.15.43:0/2877713969 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d1800cb40 con 0x7f0d20106f10 2024-09-11T23:20:31.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.801+0000 7f0d07fff640 1 -- 172.21.15.43:0/2877713969 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18007650 con 0x7f0d20106f10 2024-09-11T23:20:31.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.801+0000 7f0d25086640 1 -- 172.21.15.43:0/2877713969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d2019d860 con 0x7f0d20106f10 2024-09-11T23:20:31.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.801+0000 7f0d25086640 1 -- 172.21.15.43:0/2877713969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d201a5140 con 0x7f0d20106f10 2024-09-11T23:20:31.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.802+0000 7f0d25086640 1 -- 172.21.15.43:0/2877713969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d20102240 con 0x7f0d20106f10 2024-09-11T23:20:31.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.803+0000 7f0d07fff640 1 -- 172.21.15.43:0/2877713969 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0d1801c070 con 0x7f0d20106f10 2024-09-11T23:20:31.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.803+0000 7f0d07fff640 1 --2- 172.21.15.43:0/2877713969 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0d080a2c10 0x7f0d080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:31.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.804+0000 7f0d1ed9d640 1 --2- 172.21.15.43:0/2877713969 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0d080a2c10 0x7f0d080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:31.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.804+0000 7f0d07fff640 1 -- 172.21.15.43:0/2877713969 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f0d180ce5b0 con 0x7f0d20106f10 2024-09-11T23:20:31.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.804+0000 7f0d1ed9d640 1 --2- 172.21.15.43:0/2877713969 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0d080a2c10 0x7f0d080a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0d10005ab0 tx=0x7f0d10000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:31.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:31.808+0000 7f0d07fff640 1 -- 172.21.15.43:0/2877713969 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d1809b670 con 0x7f0d20106f10 2024-09-11T23:20:32.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:32.033+0000 7f0d25086640 1 -- 172.21.15.43:0/2877713969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0d20109b60 con 0x7f0d20106f10 2024-09-11T23:20:32.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:32.037+0000 7f0d07fff640 1 -- 172.21.15.43:0/2877713969 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263200 (secure 0 0 0) 0x7f0d1809af70 con 0x7f0d20106f10 2024-09-11T23:20:32.041 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:20:32.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:32.042+0000 7f0d25086640 1 -- 172.21.15.43:0/2877713969 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0d080a2c10 msgr2=0x7f0d080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:32.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:32.042+0000 7f0d25086640 1 --2- 172.21.15.43:0/2877713969 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0d080a2c10 0x7f0d080a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0d10005ab0 tx=0x7f0d10000bc0 comp rx=0 tx=0).stop 2024-09-11T23:20:32.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:32.042+0000 7f0d25086640 1 -- 172.21.15.43:0/2877713969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d20106f10 msgr2=0x7f0d2019ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:32.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:32.042+0000 7f0d25086640 1 --2- 172.21.15.43:0/2877713969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d20106f10 0x7f0d2019ce90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800b970 tx=0x7f0d1800be30 comp rx=0 tx=0).stop 2024-09-11T23:20:32.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:32.042+0000 7f0d25086640 1 -- 172.21.15.43:0/2877713969 shutdown_connections 2024-09-11T23:20:32.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:32.042+0000 7f0d25086640 1 --2- 172.21.15.43:0/2877713969 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f0d080a2c10 0x7f0d080a50d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:32.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:32.042+0000 7f0d25086640 1 --2- 172.21.15.43:0/2877713969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d20106f10 0x7f0d2019ce90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:32.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:32.042+0000 7f0d25086640 1 --2- 172.21.15.43:0/2877713969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d20102790 0x7f0d20198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:32.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:32.042+0000 7f0d25086640 1 --2- 172.21.15.43:0/2877713969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d20101de0 0x7f0d201985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:32.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:32.042+0000 7f0d25086640 1 -- 172.21.15.43:0/2877713969 >> 172.21.15.43:0/2877713969 conn(0x7f0d200fd8c0 msgr2=0x7f0d200fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:32.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:32.042+0000 7f0d25086640 1 -- 172.21.15.43:0/2877713969 shutdown_connections 2024-09-11T23:20:32.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:32.042+0000 7f0d25086640 1 -- 172.21.15.43:0/2877713969 wait complete. 2024-09-11T23:20:32.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:32 smithi043 ceph-mon[30950]: pgmap v297: 97 pgs: 97 active+clean; 634 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-09-11T23:20:32.342 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:32 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2877713969' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T23:20:32.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:32 smithi097 ceph-mon[35562]: pgmap v297: 97 pgs: 97 active+clean; 634 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-09-11T23:20:32.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:32 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2877713969' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T23:20:32.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:32 smithi090 ceph-mon[35130]: pgmap v297: 97 pgs: 97 active+clean; 634 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-09-11T23:20:32.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:32 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2877713969' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T23:20:32.848 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2370495020},{"type":"v1","addr":"172.21.15.43:6801","nonce":2370495020}]},"active_addr":"172.21.15.43:6801/2370495020","active_change":"2024-09-11T23:13:00.548496+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.43:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":481605249}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":2398981254}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":1579053924}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":2262294782}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":2122974177}]}]} 2024-09-11T23:20:32.853 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-11T23:20:32.853 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T23:20:32.854 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:20:33.109 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:33.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.662+0000 7f106fb70640 1 -- 172.21.15.43:0/2643182326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1068101420 msgr2=0x7f106810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:33.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.662+0000 7f106fb70640 1 --2- 172.21.15.43:0/2643182326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1068101420 0x7f106810d950 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1060009cd0 tx=0x7f106002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:20:33.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.662+0000 7f106fb70640 1 -- 172.21.15.43:0/2643182326 shutdown_connections 2024-09-11T23:20:33.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.662+0000 7f106fb70640 1 --2- 172.21.15.43:0/2643182326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1068101420 0x7f106810d950 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:33.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.662+0000 7f106fb70640 1 --2- 172.21.15.43:0/2643182326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1068100a80 0x7f1068100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:33.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.662+0000 7f106fb70640 1 --2- 172.21.15.43:0/2643182326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1068106a80 0x7f1068106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:33.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.662+0000 7f106fb70640 1 -- 172.21.15.43:0/2643182326 >> 172.21.15.43:0/2643182326 conn(0x7f10680fc7a0 msgr2=0x7f10680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:33.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.662+0000 7f106fb70640 1 -- 172.21.15.43:0/2643182326 shutdown_connections 2024-09-11T23:20:33.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.662+0000 7f106fb70640 1 -- 172.21.15.43:0/2643182326 wait complete. 2024-09-11T23:20:33.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.662+0000 7f106fb70640 1 Processor -- start 2024-09-11T23:20:33.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.663+0000 7f106fb70640 1 -- start start 2024-09-11T23:20:33.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.663+0000 7f106fb70640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1068100a80 0x7f106810e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:33.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.663+0000 7f106fb70640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1068101420 0x7f106810e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:33.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.663+0000 7f106fb70640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1068106a80 0x7f106810ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:33.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.663+0000 7f106fb70640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1068111fa0 con 0x7f1068106a80 2024-09-11T23:20:33.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.663+0000 7f106fb70640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1068111e20 con 0x7f1068101420 2024-09-11T23:20:33.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.663+0000 7f106fb70640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1068112120 con 0x7f1068100a80 2024-09-11T23:20:33.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.663+0000 7f106e10d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1068106a80 0x7f106810ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:33.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.663+0000 7f106e10d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1068106a80 0x7f106810ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44004/0 (socket says 172.21.15.43:44004) 2024-09-11T23:20:33.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.663+0000 7f106e10d640 1 -- 172.21.15.43:0/2579965279 learned_addr learned my addr 172.21.15.43:0/2579965279 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:33.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.663+0000 7f106d90c640 1 --2- 172.21.15.43:0/2579965279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1068100a80 0x7f106810e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:33.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.663+0000 7f106d10b640 1 --2- 172.21.15.43:0/2579965279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1068101420 0x7f106810e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:33.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.663+0000 7f106e10d640 1 -- 172.21.15.43:0/2579965279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1068100a80 msgr2=0x7f106810e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:33.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.663+0000 7f106e10d640 1 --2- 172.21.15.43:0/2579965279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1068100a80 0x7f106810e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:33.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.663+0000 7f106e10d640 1 -- 172.21.15.43:0/2579965279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1068101420 msgr2=0x7f106810e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:33.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.663+0000 7f106e10d640 1 --2- 172.21.15.43:0/2579965279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1068101420 0x7f106810e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:33.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.663+0000 7f106e10d640 1 -- 172.21.15.43:0/2579965279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10681a68c0 con 0x7f1068106a80 2024-09-11T23:20:33.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.663+0000 7f106d90c640 1 --2- 172.21.15.43:0/2579965279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1068100a80 0x7f106810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:20:33.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.663+0000 7f106d10b640 1 --2- 172.21.15.43:0/2579965279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1068101420 0x7f106810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:20:33.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.663+0000 7f106e10d640 1 --2- 172.21.15.43:0/2579965279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1068106a80 0x7f106810ee60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1060006010 tx=0x7f1060003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:33.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.664+0000 7f105effd640 1 -- 172.21.15.43:0/2579965279 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060004450 con 0x7f1068106a80 2024-09-11T23:20:33.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.664+0000 7f106fb70640 1 -- 172.21.15.43:0/2579965279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10681a6b50 con 0x7f1068106a80 2024-09-11T23:20:33.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.664+0000 7f105effd640 1 -- 172.21.15.43:0/2579965279 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1060003e10 con 0x7f1068106a80 2024-09-11T23:20:33.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.664+0000 7f105effd640 1 -- 172.21.15.43:0/2579965279 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10600466e0 con 0x7f1068106a80 2024-09-11T23:20:33.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.664+0000 7f106fb70640 1 -- 172.21.15.43:0/2579965279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10681a7090 con 0x7f1068106a80 2024-09-11T23:20:33.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.665+0000 7f106fb70640 1 -- 172.21.15.43:0/2579965279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1068100ee0 con 0x7f1068106a80 2024-09-11T23:20:33.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.666+0000 7f105effd640 1 -- 172.21.15.43:0/2579965279 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f10600305a0 con 0x7f1068106a80 2024-09-11T23:20:33.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.666+0000 7f105effd640 1 --2- 172.21.15.43:0/2579965279 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f10540a2c10 0x7f10540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:33.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.667+0000 7f106d90c640 1 --2- 172.21.15.43:0/2579965279 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f10540a2c10 0x7f10540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:33.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.667+0000 7f105effd640 1 -- 172.21.15.43:0/2579965279 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f106002e030 con 0x7f1068106a80 2024-09-11T23:20:33.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.667+0000 7f106d90c640 1 --2- 172.21.15.43:0/2579965279 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f10540a2c10 0x7f10540a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f106400baa0 tx=0x7f1064005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:33.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.669+0000 7f105effd640 1 -- 172.21.15.43:0/2579965279 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f106004b060 con 0x7f1068106a80 2024-09-11T23:20:33.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.846+0000 7f106fb70640 1 -- 172.21.15.43:0/2579965279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10681a7350 con 0x7f1068106a80 2024-09-11T23:20:33.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.847+0000 7f105effd640 1 -- 172.21.15.43:0/2579965279 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22145 (secure 0 0 0) 0x7f1060039380 con 0x7f1068106a80 2024-09-11T23:20:33.847 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:20:33.848 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":82,"fsid":"25c0cd84-7093-11ef-bcea-c7b262605968","created":"2024-09-11T23:12:03.504406+0000","modified":"2024-09-11T23:20:00.465617+0000","last_up_change":"2024-09-11T23:19:07.797873+0000","last_in_change":"2024-09-11T23:18:44.595176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:15:17.058713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:19:56.221406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:19:58.705875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24076596-3909-4c87-85f1-cbf168070dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6803","nonce":43839997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6805","nonce":43839997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6809","nonce":43839997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6807","nonce":43839997}]},"public_addr":"172.21.15.43:6803/43839997","cluster_addr":"172.21.15.43:6805/43839997","heartbeat_back_addr":"172.21.15.43:6809/43839997","heartbeat_front_addr":"172.21.15.43:6807/43839997","state":["exists","up"]},{"osd":1,"uuid":"a0472a57-243c-489d-a247-972ee10a71f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6801","nonce":1825149246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6803","nonce":1825149246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6807","nonce":1825149246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6805","nonce":1825149246}]},"public_addr":"172.21.15.90:6801/1825149246","cluster_addr":"172.21.15.90:6803/1825149246","heartbeat_back_addr":"172.21.15.90:6807/1825149246","heartbeat_front_addr":"172.21.15.90:6805/1825149246","state":["exists","up"]},{"osd":2,"uuid":"40366536-0fcf-45a0-b8e3-49178401dc34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6801","nonce":1456846443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6803","nonce":1456846443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6807","nonce":1456846443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6805","nonce":1456846443}]},"public_addr":"172.21.15.97:6801/1456846443","cluster_addr":"172.21.15.97:6803/1456846443","heartbeat_back_addr":"172.21.15.97:6807/1456846443","heartbeat_front_addr":"172.21.15.97:6805/1456846443","state":["exists","up"]},{"osd":3,"uuid":"3062133e-000b-4421-9d53-ac434b9ed9b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6811","nonce":1962837804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6813","nonce":1962837804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6817","nonce":1962837804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6815","nonce":1962837804}]},"public_addr":"172.21.15.43:6811/1962837804","cluster_addr":"172.21.15.43:6813/1962837804","heartbeat_back_addr":"172.21.15.43:6817/1962837804","heartbeat_front_addr":"172.21.15.43:6815/1962837804","state":["exists","up"]},{"osd":4,"uuid":"2e73fa4b-f22c-4319-828d-87b91b9f19f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6809","nonce":181614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6811","nonce":181614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6815","nonce":181614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6813","nonce":181614173}]},"public_addr":"172.21.15.90:6809/181614173","cluster_addr":"172.21.15.90:6811/181614173","heartbeat_back_addr":"172.21.15.90:6815/181614173","heartbeat_front_addr":"172.21.15.90:6813/181614173","state":["exists","up"]},{"osd":5,"uuid":"1e1ad4f9-e724-440f-b515-3ec1ad97d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6809","nonce":1540039543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6811","nonce":1540039543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6815","nonce":1540039543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6813","nonce":1540039543}]},"public_addr":"172.21.15.97:6809/1540039543","cluster_addr":"172.21.15.97:6811/1540039543","heartbeat_back_addr":"172.21.15.97:6815/1540039543","heartbeat_front_addr":"172.21.15.97:6813/1540039543","state":["exists","up"]},{"osd":6,"uuid":"d34df8df-f40d-4159-8b29-5401b1b0e431","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6819","nonce":3785677637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6821","nonce":3785677637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6825","nonce":3785677637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6823","nonce":3785677637}]},"public_addr":"172.21.15.43:6819/3785677637","cluster_addr":"172.21.15.43:6821/3785677637","heartbeat_back_addr":"172.21.15.43:6825/3785677637","heartbeat_front_addr":"172.21.15.43:6823/3785677637","state":["exists","up"]},{"osd":7,"uuid":"b2e01078-528b-49fd-b000-9f584f784b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6817","nonce":236874962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6819","nonce":236874962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6823","nonce":236874962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6821","nonce":236874962}]},"public_addr":"172.21.15.90:6817/236874962","cluster_addr":"172.21.15.90:6819/236874962","heartbeat_back_addr":"172.21.15.90:6823/236874962","heartbeat_front_addr":"172.21.15.90:6821/236874962","state":["exists","up"]},{"osd":8,"uuid":"abef8f27-bf9b-4d63-bfbe-f967fc3d8b17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6817","nonce":1707327301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6819","nonce":1707327301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6823","nonce":1707327301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6821","nonce":1707327301}]},"public_addr":"172.21.15.97:6817/1707327301","cluster_addr":"172.21.15.97:6819/1707327301","heartbeat_back_addr":"172.21.15.97:6823/1707327301","heartbeat_front_addr":"172.21.15.97:6821/1707327301","state":["exists","up"]},{"osd":9,"uuid":"190ab8b2-a835-47e6-9b39-16a58331c58a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6827","nonce":3170306011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6829","nonce":3170306011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6833","nonce":3170306011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6831","nonce":3170306011}]},"public_addr":"172.21.15.43:6827/3170306011","cluster_addr":"172.21.15.43:6829/3170306011","heartbeat_back_addr":"172.21.15.43:6833/3170306011","heartbeat_front_addr":"172.21.15.43:6831/3170306011","state":["exists","up"]},{"osd":10,"uuid":"b32ab5fe-466f-4c8e-8696-cd9f820a3263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6825","nonce":59487041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6827","nonce":59487041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6831","nonce":59487041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6829","nonce":59487041}]},"public_addr":"172.21.15.90:6825/59487041","cluster_addr":"172.21.15.90:6827/59487041","heartbeat_back_addr":"172.21.15.90:6831/59487041","heartbeat_front_addr":"172.21.15.90:6829/59487041","state":["exists","up"]},{"osd":11,"uuid":"d0faa589-21c8-4a9f-8a73-813aafe6d33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6825","nonce":194699315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6827","nonce":194699315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6831","nonce":194699315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6829","nonce":194699315}]},"public_addr":"172.21.15.97:6825/194699315","cluster_addr":"172.21.15.97:6827/194699315","heartbeat_back_addr":"172.21.15.97:6831/194699315","heartbeat_front_addr":"172.21.15.97:6829/194699315","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:23.381247+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:48.843190+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:13.538979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:40.136983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:05.513790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:31.247090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:56.635221+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:22.667327+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:47.910422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:14.353914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:40.584503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:19:06.476766+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/76575371":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/4006124758":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/120495444":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6800/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:6800/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/206044413":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1512201905":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1375379108":"2024-09-12T23:12:35.756598+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:20:33.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.850+0000 7f106fb70640 1 -- 172.21.15.43:0/2579965279 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f10540a2c10 msgr2=0x7f10540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:33.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.850+0000 7f106fb70640 1 --2- 172.21.15.43:0/2579965279 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f10540a2c10 0x7f10540a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f106400baa0 tx=0x7f1064005d90 comp rx=0 tx=0).stop 2024-09-11T23:20:33.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.850+0000 7f106fb70640 1 -- 172.21.15.43:0/2579965279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1068106a80 msgr2=0x7f106810ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:33.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.850+0000 7f106fb70640 1 --2- 172.21.15.43:0/2579965279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1068106a80 0x7f106810ee60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1060006010 tx=0x7f1060003c20 comp rx=0 tx=0).stop 2024-09-11T23:20:33.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.850+0000 7f106fb70640 1 -- 172.21.15.43:0/2579965279 shutdown_connections 2024-09-11T23:20:33.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.850+0000 7f106fb70640 1 --2- 172.21.15.43:0/2579965279 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f10540a2c10 0x7f10540a50d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:33.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.850+0000 7f106fb70640 1 --2- 172.21.15.43:0/2579965279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1068106a80 0x7f106810ee60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:33.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.850+0000 7f106fb70640 1 --2- 172.21.15.43:0/2579965279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1068101420 0x7f106810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:33.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.850+0000 7f106fb70640 1 --2- 172.21.15.43:0/2579965279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1068100a80 0x7f106810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:33.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.850+0000 7f106fb70640 1 -- 172.21.15.43:0/2579965279 >> 172.21.15.43:0/2579965279 conn(0x7f10680fc7a0 msgr2=0x7f10680fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:33.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.850+0000 7f106fb70640 1 -- 172.21.15.43:0/2579965279 shutdown_connections 2024-09-11T23:20:33.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:33.850+0000 7f106fb70640 1 -- 172.21.15.43:0/2579965279 wait complete. 2024-09-11T23:20:34.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:34 smithi097 ceph-mon[35562]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-09-11T23:20:34.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:34 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2579965279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:20:34.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:34 smithi043 ceph-mon[30950]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-09-11T23:20:34.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:34 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2579965279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:20:34.692 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T23:20:34.692 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:20:34.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:34 smithi090 ceph-mon[35130]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-09-11T23:20:34.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:34 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2579965279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:20:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:35.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.463+0000 7f86fe7b8640 1 -- 172.21.15.43:0/3159350285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f810e500 msgr2=0x7f86f8110910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:35.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.463+0000 7f86fe7b8640 1 --2- 172.21.15.43:0/3159350285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f810e500 0x7f86f8110910 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009ca0 tx=0x7f86f002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:20:35.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.463+0000 7f86fe7b8640 1 -- 172.21.15.43:0/3159350285 shutdown_connections 2024-09-11T23:20:35.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.463+0000 7f86fe7b8640 1 --2- 172.21.15.43:0/3159350285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f810e500 0x7f86f8110910 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:35.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.463+0000 7f86fe7b8640 1 --2- 172.21.15.43:0/3159350285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86f8101440 0x7f86f810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:35.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.463+0000 7f86fe7b8640 1 --2- 172.21.15.43:0/3159350285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86f8100b20 0x7f86f8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:35.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.463+0000 7f86fe7b8640 1 -- 172.21.15.43:0/3159350285 >> 172.21.15.43:0/3159350285 conn(0x7f86f80fc7d0 msgr2=0x7f86f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:35.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.463+0000 7f86fe7b8640 1 -- 172.21.15.43:0/3159350285 shutdown_connections 2024-09-11T23:20:35.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.463+0000 7f86fe7b8640 1 -- 172.21.15.43:0/3159350285 wait complete. 2024-09-11T23:20:35.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.464+0000 7f86fe7b8640 1 Processor -- start 2024-09-11T23:20:35.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.464+0000 7f86fe7b8640 1 -- start start 2024-09-11T23:20:35.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.464+0000 7f86fe7b8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f8100b20 0x7f86f819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:35.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.464+0000 7f86fe7b8640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86f8101440 0x7f86f819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:35.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.464+0000 7f86fe7b8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86f810e500 0x7f86f819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:35.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.464+0000 7f86fe7b8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f86f8114250 con 0x7f86f8100b20 2024-09-11T23:20:35.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.464+0000 7f86fe7b8640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81140d0 con 0x7f86f8101440 2024-09-11T23:20:35.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.464+0000 7f86fe7b8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f86f81143d0 con 0x7f86f810e500 2024-09-11T23:20:35.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.464+0000 7f86f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f8100b20 0x7f86f819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:35.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.464+0000 7f86f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f8100b20 0x7f86f819a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44030/0 (socket says 172.21.15.43:44030) 2024-09-11T23:20:35.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.464+0000 7f86f7fff640 1 -- 172.21.15.43:0/1578739356 learned_addr learned my addr 172.21.15.43:0/1578739356 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:35.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.464+0000 7f86fcd55640 1 --2- 172.21.15.43:0/1578739356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86f810e500 0x7f86f819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:35.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.464+0000 7f86f7fff640 1 -- 172.21.15.43:0/1578739356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86f810e500 msgr2=0x7f86f819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:35.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.464+0000 7f86effff640 1 --2- 172.21.15.43:0/1578739356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86f8101440 0x7f86f819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:35.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.464+0000 7f86f7fff640 1 --2- 172.21.15.43:0/1578739356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86f810e500 0x7f86f819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:35.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.464+0000 7f86f7fff640 1 -- 172.21.15.43:0/1578739356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86f8101440 msgr2=0x7f86f819ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:35.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.464+0000 7f86f7fff640 1 --2- 172.21.15.43:0/1578739356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86f8101440 0x7f86f819ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:35.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.464+0000 7f86f7fff640 1 -- 172.21.15.43:0/1578739356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f819f6c0 con 0x7f86f8100b20 2024-09-11T23:20:35.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.464+0000 7f86fcd55640 1 --2- 172.21.15.43:0/1578739356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86f810e500 0x7f86f819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:20:35.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.464+0000 7f86f7fff640 1 --2- 172.21.15.43:0/1578739356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f8100b20 0x7f86f819a710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f86e800d950 tx=0x7f86e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:35.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.465+0000 7f86f5ffb640 1 -- 172.21.15.43:0/1578739356 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8017070 con 0x7f86f8100b20 2024-09-11T23:20:35.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.465+0000 7f86fe7b8640 1 -- 172.21.15.43:0/1578739356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f819f9b0 con 0x7f86f8100b20 2024-09-11T23:20:35.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.465+0000 7f86f5ffb640 1 -- 172.21.15.43:0/1578739356 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e8009c30 con 0x7f86f8100b20 2024-09-11T23:20:35.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.465+0000 7f86f5ffb640 1 -- 172.21.15.43:0/1578739356 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e800f6b0 con 0x7f86f8100b20 2024-09-11T23:20:35.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.465+0000 7f86fe7b8640 1 -- 172.21.15.43:0/1578739356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f81a7290 con 0x7f86f8100b20 2024-09-11T23:20:35.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.466+0000 7f86fe7b8640 1 -- 172.21.15.43:0/1578739356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f8100f00 con 0x7f86f8100b20 2024-09-11T23:20:35.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.467+0000 7f86f5ffb640 1 -- 172.21.15.43:0/1578739356 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f86e80109d0 con 0x7f86f8100b20 2024-09-11T23:20:35.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.468+0000 7f86f5ffb640 1 --2- 172.21.15.43:0/1578739356 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f86e40a2c90 0x7f86e40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:35.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.468+0000 7f86effff640 1 --2- 172.21.15.43:0/1578739356 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f86e40a2c90 0x7f86e40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:35.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.468+0000 7f86f5ffb640 1 -- 172.21.15.43:0/1578739356 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f86e80cedc0 con 0x7f86f8100b20 2024-09-11T23:20:35.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.468+0000 7f86effff640 1 --2- 172.21.15.43:0/1578739356 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f86e40a2c90 0x7f86e40a5150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f86f810ee00 tx=0x7f86e0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:35.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.472+0000 7f86f5ffb640 1 -- 172.21.15.43:0/1578739356 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f86e8009de0 con 0x7f86f8100b20 2024-09-11T23:20:35.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.650+0000 7f86fe7b8640 1 -- 172.21.15.43:0/1578739356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86f8109600 con 0x7f86f8100b20 2024-09-11T23:20:35.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.652+0000 7f86f5ffb640 1 -- 172.21.15.43:0/1578739356 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22145 (secure 0 0 0) 0x7f86e80bb070 con 0x7f86f8100b20 2024-09-11T23:20:35.653 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:20:35.653 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":82,"fsid":"25c0cd84-7093-11ef-bcea-c7b262605968","created":"2024-09-11T23:12:03.504406+0000","modified":"2024-09-11T23:20:00.465617+0000","last_up_change":"2024-09-11T23:19:07.797873+0000","last_in_change":"2024-09-11T23:18:44.595176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:15:17.058713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:19:56.221406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:19:58.705875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24076596-3909-4c87-85f1-cbf168070dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6803","nonce":43839997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6805","nonce":43839997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6809","nonce":43839997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6807","nonce":43839997}]},"public_addr":"172.21.15.43:6803/43839997","cluster_addr":"172.21.15.43:6805/43839997","heartbeat_back_addr":"172.21.15.43:6809/43839997","heartbeat_front_addr":"172.21.15.43:6807/43839997","state":["exists","up"]},{"osd":1,"uuid":"a0472a57-243c-489d-a247-972ee10a71f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6801","nonce":1825149246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6803","nonce":1825149246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6807","nonce":1825149246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6805","nonce":1825149246}]},"public_addr":"172.21.15.90:6801/1825149246","cluster_addr":"172.21.15.90:6803/1825149246","heartbeat_back_addr":"172.21.15.90:6807/1825149246","heartbeat_front_addr":"172.21.15.90:6805/1825149246","state":["exists","up"]},{"osd":2,"uuid":"40366536-0fcf-45a0-b8e3-49178401dc34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6801","nonce":1456846443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6803","nonce":1456846443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6807","nonce":1456846443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6805","nonce":1456846443}]},"public_addr":"172.21.15.97:6801/1456846443","cluster_addr":"172.21.15.97:6803/1456846443","heartbeat_back_addr":"172.21.15.97:6807/1456846443","heartbeat_front_addr":"172.21.15.97:6805/1456846443","state":["exists","up"]},{"osd":3,"uuid":"3062133e-000b-4421-9d53-ac434b9ed9b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6811","nonce":1962837804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6813","nonce":1962837804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6817","nonce":1962837804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6815","nonce":1962837804}]},"public_addr":"172.21.15.43:6811/1962837804","cluster_addr":"172.21.15.43:6813/1962837804","heartbeat_back_addr":"172.21.15.43:6817/1962837804","heartbeat_front_addr":"172.21.15.43:6815/1962837804","state":["exists","up"]},{"osd":4,"uuid":"2e73fa4b-f22c-4319-828d-87b91b9f19f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6809","nonce":181614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6811","nonce":181614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6815","nonce":181614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6813","nonce":181614173}]},"public_addr":"172.21.15.90:6809/181614173","cluster_addr":"172.21.15.90:6811/181614173","heartbeat_back_addr":"172.21.15.90:6815/181614173","heartbeat_front_addr":"172.21.15.90:6813/181614173","state":["exists","up"]},{"osd":5,"uuid":"1e1ad4f9-e724-440f-b515-3ec1ad97d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6809","nonce":1540039543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6811","nonce":1540039543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6815","nonce":1540039543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6813","nonce":1540039543}]},"public_addr":"172.21.15.97:6809/1540039543","cluster_addr":"172.21.15.97:6811/1540039543","heartbeat_back_addr":"172.21.15.97:6815/1540039543","heartbeat_front_addr":"172.21.15.97:6813/1540039543","state":["exists","up"]},{"osd":6,"uuid":"d34df8df-f40d-4159-8b29-5401b1b0e431","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6819","nonce":3785677637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6821","nonce":3785677637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6825","nonce":3785677637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6823","nonce":3785677637}]},"public_addr":"172.21.15.43:6819/3785677637","cluster_addr":"172.21.15.43:6821/3785677637","heartbeat_back_addr":"172.21.15.43:6825/3785677637","heartbeat_front_addr":"172.21.15.43:6823/3785677637","state":["exists","up"]},{"osd":7,"uuid":"b2e01078-528b-49fd-b000-9f584f784b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6817","nonce":236874962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6819","nonce":236874962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6823","nonce":236874962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6821","nonce":236874962}]},"public_addr":"172.21.15.90:6817/236874962","cluster_addr":"172.21.15.90:6819/236874962","heartbeat_back_addr":"172.21.15.90:6823/236874962","heartbeat_front_addr":"172.21.15.90:6821/236874962","state":["exists","up"]},{"osd":8,"uuid":"abef8f27-bf9b-4d63-bfbe-f967fc3d8b17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6817","nonce":1707327301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6819","nonce":1707327301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6823","nonce":1707327301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6821","nonce":1707327301}]},"public_addr":"172.21.15.97:6817/1707327301","cluster_addr":"172.21.15.97:6819/1707327301","heartbeat_back_addr":"172.21.15.97:6823/1707327301","heartbeat_front_addr":"172.21.15.97:6821/1707327301","state":["exists","up"]},{"osd":9,"uuid":"190ab8b2-a835-47e6-9b39-16a58331c58a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6827","nonce":3170306011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6829","nonce":3170306011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6833","nonce":3170306011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6831","nonce":3170306011}]},"public_addr":"172.21.15.43:6827/3170306011","cluster_addr":"172.21.15.43:6829/3170306011","heartbeat_back_addr":"172.21.15.43:6833/3170306011","heartbeat_front_addr":"172.21.15.43:6831/3170306011","state":["exists","up"]},{"osd":10,"uuid":"b32ab5fe-466f-4c8e-8696-cd9f820a3263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6825","nonce":59487041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6827","nonce":59487041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6831","nonce":59487041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6829","nonce":59487041}]},"public_addr":"172.21.15.90:6825/59487041","cluster_addr":"172.21.15.90:6827/59487041","heartbeat_back_addr":"172.21.15.90:6831/59487041","heartbeat_front_addr":"172.21.15.90:6829/59487041","state":["exists","up"]},{"osd":11,"uuid":"d0faa589-21c8-4a9f-8a73-813aafe6d33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6825","nonce":194699315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6827","nonce":194699315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6831","nonce":194699315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6829","nonce":194699315}]},"public_addr":"172.21.15.97:6825/194699315","cluster_addr":"172.21.15.97:6827/194699315","heartbeat_back_addr":"172.21.15.97:6831/194699315","heartbeat_front_addr":"172.21.15.97:6829/194699315","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:23.381247+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:48.843190+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:13.538979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:40.136983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:05.513790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:31.247090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:56.635221+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:22.667327+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:47.910422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:14.353914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:40.584503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:19:06.476766+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/76575371":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/4006124758":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/120495444":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6800/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:6800/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/206044413":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1512201905":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1375379108":"2024-09-12T23:12:35.756598+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:20:35.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.655+0000 7f86fe7b8640 1 -- 172.21.15.43:0/1578739356 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f86e40a2c90 msgr2=0x7f86e40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:35.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.655+0000 7f86fe7b8640 1 --2- 172.21.15.43:0/1578739356 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f86e40a2c90 0x7f86e40a5150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f86f810ee00 tx=0x7f86e0005f90 comp rx=0 tx=0).stop 2024-09-11T23:20:35.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.655+0000 7f86fe7b8640 1 -- 172.21.15.43:0/1578739356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f8100b20 msgr2=0x7f86f819a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:35.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.655+0000 7f86fe7b8640 1 --2- 172.21.15.43:0/1578739356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f8100b20 0x7f86f819a710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f86e800d950 tx=0x7f86e800de10 comp rx=0 tx=0).stop 2024-09-11T23:20:35.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.655+0000 7f86fe7b8640 1 -- 172.21.15.43:0/1578739356 shutdown_connections 2024-09-11T23:20:35.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.655+0000 7f86fe7b8640 1 --2- 172.21.15.43:0/1578739356 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f86e40a2c90 0x7f86e40a5150 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:35.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.655+0000 7f86fe7b8640 1 --2- 172.21.15.43:0/1578739356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86f810e500 0x7f86f819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:35.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.655+0000 7f86fe7b8640 1 --2- 172.21.15.43:0/1578739356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86f8101440 0x7f86f819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:35.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.655+0000 7f86fe7b8640 1 --2- 172.21.15.43:0/1578739356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f8100b20 0x7f86f819a710 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:35.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.655+0000 7f86fe7b8640 1 -- 172.21.15.43:0/1578739356 >> 172.21.15.43:0/1578739356 conn(0x7f86f80fc7d0 msgr2=0x7f86f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:35.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.655+0000 7f86fe7b8640 1 -- 172.21.15.43:0/1578739356 shutdown_connections 2024-09-11T23:20:35.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:35.655+0000 7f86fe7b8640 1 -- 172.21.15.43:0/1578739356 wait complete. 2024-09-11T23:20:36.469 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T23:20:36.469 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T23:20:36.470 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T23:20:36.470 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T23:20:36.470 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T23:20:36.471 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T23:20:36.471 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T23:20:36.472 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T23:20:36.472 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-11T23:20:36.473 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-11T23:20:36.473 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-11T23:20:36.474 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-11T23:20:36.537 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:36 smithi043 ceph-mon[30950]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-11T23:20:36.537 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:36 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1578739356' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:20:36.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:36 smithi097 ceph-mon[35562]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-11T23:20:36.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:36 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1578739356' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:20:36.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:36 smithi090 ceph-mon[35130]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-11T23:20:36.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:36 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1578739356' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:20:37.145 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:37.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.728+0000 7f42089dc640 1 -- 172.21.15.43:0/2072858437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4204100a80 msgr2=0x7f4204100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:37.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.728+0000 7f42089dc640 1 --2- 172.21.15.43:0/2072858437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4204100a80 0x7f4204100ee0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009930 tx=0x7f41ec02a420 comp rx=0 tx=0).stop 2024-09-11T23:20:37.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.728+0000 7f42089dc640 1 -- 172.21.15.43:0/2072858437 shutdown_connections 2024-09-11T23:20:37.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.728+0000 7f42089dc640 1 --2- 172.21.15.43:0/2072858437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4204101420 0x7f420410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.728+0000 7f42089dc640 1 --2- 172.21.15.43:0/2072858437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4204100a80 0x7f4204100ee0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.728+0000 7f42089dc640 1 --2- 172.21.15.43:0/2072858437 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4204106a80 0x7f4204106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.728+0000 7f42089dc640 1 -- 172.21.15.43:0/2072858437 >> 172.21.15.43:0/2072858437 conn(0x7f42040fc7a0 msgr2=0x7f42040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:37.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.728+0000 7f42089dc640 1 -- 172.21.15.43:0/2072858437 shutdown_connections 2024-09-11T23:20:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.728+0000 7f42089dc640 1 -- 172.21.15.43:0/2072858437 wait complete. 2024-09-11T23:20:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.729+0000 7f42089dc640 1 Processor -- start 2024-09-11T23:20:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.729+0000 7f42089dc640 1 -- start start 2024-09-11T23:20:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.729+0000 7f42089dc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4204100a80 0x7f4204199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.729+0000 7f42089dc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4204101420 0x7f42041998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.729+0000 7f42089dc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4204106a80 0x7f42041a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.729+0000 7f42089dc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4204110ea0 con 0x7f4204100a80 2024-09-11T23:20:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.729+0000 7f42089dc640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4204110d20 con 0x7f4204106a80 2024-09-11T23:20:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.729+0000 7f42089dc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4204111020 con 0x7f4204101420 2024-09-11T23:20:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.729+0000 7f420259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4204100a80 0x7f4204199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.729+0000 7f420259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4204100a80 0x7f4204199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44050/0 (socket says 172.21.15.43:44050) 2024-09-11T23:20:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.729+0000 7f420259c640 1 -- 172.21.15.43:0/3479881407 learned_addr learned my addr 172.21.15.43:0/3479881407 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.729+0000 7f4202d9d640 1 --2- 172.21.15.43:0/3479881407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4204106a80 0x7f42041a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.729+0000 7f41f9d9b640 1 --2- 172.21.15.43:0/3479881407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4204101420 0x7f42041998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.730+0000 7f420259c640 1 -- 172.21.15.43:0/3479881407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4204101420 msgr2=0x7f42041998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.730+0000 7f420259c640 1 --2- 172.21.15.43:0/3479881407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4204101420 0x7f42041998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.730+0000 7f420259c640 1 -- 172.21.15.43:0/3479881407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4204106a80 msgr2=0x7f42041a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.730+0000 7f420259c640 1 --2- 172.21.15.43:0/3479881407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4204106a80 0x7f42041a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.730+0000 7f420259c640 1 -- 172.21.15.43:0/3479881407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42041a4950 con 0x7f4204100a80 2024-09-11T23:20:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.730+0000 7f41f9d9b640 1 --2- 172.21.15.43:0/3479881407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4204101420 0x7f42041998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.730+0000 7f420259c640 1 --2- 172.21.15.43:0/3479881407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4204100a80 0x7f4204199390 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f41f400d950 tx=0x7f41f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.730+0000 7f41fbfff640 1 -- 172.21.15.43:0/3479881407 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4017070 con 0x7f4204100a80 2024-09-11T23:20:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.730+0000 7f41fbfff640 1 -- 172.21.15.43:0/3479881407 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f4009c30 con 0x7f4204100a80 2024-09-11T23:20:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.730+0000 7f42089dc640 1 -- 172.21.15.43:0/3479881407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42041a4b20 con 0x7f4204100a80 2024-09-11T23:20:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.730+0000 7f41fbfff640 1 -- 172.21.15.43:0/3479881407 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f400f6b0 con 0x7f4204100a80 2024-09-11T23:20:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.730+0000 7f42089dc640 1 -- 172.21.15.43:0/3479881407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42041a4fe0 con 0x7f4204100a80 2024-09-11T23:20:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.731+0000 7f42089dc640 1 -- 172.21.15.43:0/3479881407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4204062900 con 0x7f4204100a80 2024-09-11T23:20:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.732+0000 7f41fbfff640 1 -- 172.21.15.43:0/3479881407 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f41f40109d0 con 0x7f4204100a80 2024-09-11T23:20:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.733+0000 7f41fbfff640 1 --2- 172.21.15.43:0/3479881407 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f41f00a2c10 0x7f41f00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.733+0000 7f41f9d9b640 1 --2- 172.21.15.43:0/3479881407 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f41f00a2c10 0x7f41f00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.733+0000 7f41fbfff640 1 -- 172.21.15.43:0/3479881407 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f41f40cecb0 con 0x7f4204100a80 2024-09-11T23:20:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.733+0000 7f41fbfff640 1 --2- 172.21.15.43:0/3479881407 >> [v2:172.21.15.90:6808/181614173,v1:172.21.15.90:6809/181614173] conn(0x7f41f00ac460 0x7f41f00ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.733+0000 7f41fbfff640 1 -- 172.21.15.43:0/3479881407 --> [v2:172.21.15.90:6808/181614173,v1:172.21.15.90:6809/181614173] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f41f4004470 con 0x7f41f00ac460 2024-09-11T23:20:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.733+0000 7f41fbfff640 1 -- 172.21.15.43:0/3479881407 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f41f40d1400 con 0x7f4204100a80 2024-09-11T23:20:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.733+0000 7f41f9d9b640 1 --2- 172.21.15.43:0/3479881407 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f41f00a2c10 0x7f41f00a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f41ec000c00 tx=0x7f41ec005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.734+0000 7f4202d9d640 1 --2- 172.21.15.43:0/3479881407 >> [v2:172.21.15.90:6808/181614173,v1:172.21.15.90:6809/181614173] conn(0x7f41f00ac460 0x7f41f00ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.734+0000 7f4202d9d640 1 --2- 172.21.15.43:0/3479881407 >> [v2:172.21.15.90:6808/181614173,v1:172.21.15.90:6809/181614173] conn(0x7f41f00ac460 0x7f41f00ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.735+0000 7f41fbfff640 1 -- 172.21.15.43:0/3479881407 <== osd.4 v2:172.21.15.90:6808/181614173 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f41f4004470 con 0x7f41f00ac460 2024-09-11T23:20:37.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.749+0000 7f42089dc640 1 -- 172.21.15.43:0/3479881407 --> [v2:172.21.15.90:6808/181614173,v1:172.21.15.90:6809/181614173] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4204062d80 con 0x7f41f00ac460 2024-09-11T23:20:37.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.749+0000 7f41fbfff640 1 -- 172.21.15.43:0/3479881407 <== osd.4 v2:172.21.15.90:6808/181614173 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4204062d80 con 0x7f41f00ac460 2024-09-11T23:20:37.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.750+0000 7f42089dc640 1 -- 172.21.15.43:0/3479881407 >> [v2:172.21.15.90:6808/181614173,v1:172.21.15.90:6809/181614173] conn(0x7f41f00ac460 msgr2=0x7f41f00ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:37.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.750+0000 7f42089dc640 1 --2- 172.21.15.43:0/3479881407 >> [v2:172.21.15.90:6808/181614173,v1:172.21.15.90:6809/181614173] conn(0x7f41f00ac460 0x7f41f00ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:37.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.750+0000 7f42089dc640 1 -- 172.21.15.43:0/3479881407 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f41f00a2c10 msgr2=0x7f41f00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:37.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.750+0000 7f42089dc640 1 --2- 172.21.15.43:0/3479881407 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f41f00a2c10 0x7f41f00a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f41ec000c00 tx=0x7f41ec005a20 comp rx=0 tx=0).stop 2024-09-11T23:20:37.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.750+0000 7f42089dc640 1 -- 172.21.15.43:0/3479881407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4204100a80 msgr2=0x7f4204199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:37.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.750+0000 7f42089dc640 1 --2- 172.21.15.43:0/3479881407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4204100a80 0x7f4204199390 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f41f400d950 tx=0x7f41f400de10 comp rx=0 tx=0).stop 2024-09-11T23:20:37.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.750+0000 7f4202d9d640 1 -- 172.21.15.43:0/3479881407 reap_dead start 2024-09-11T23:20:37.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.750+0000 7f42089dc640 1 -- 172.21.15.43:0/3479881407 shutdown_connections 2024-09-11T23:20:37.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.750+0000 7f42089dc640 1 -- 172.21.15.43:0/3479881407 >> 172.21.15.43:0/3479881407 conn(0x7f42040fc7a0 msgr2=0x7f42040fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:37.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.750+0000 7f42089dc640 1 -- 172.21.15.43:0/3479881407 shutdown_connections 2024-09-11T23:20:37.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:37.750+0000 7f42089dc640 1 -- 172.21.15.43:0/3479881407 wait complete. 2024-09-11T23:20:37.999 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:37.999 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:38.000 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:38.003 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:38.008 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:38.008 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:38.008 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:38.009 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:38.009 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:38.009 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:38.013 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:38.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:38 smithi097 ceph-mon[35562]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-09-11T23:20:38.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:38 smithi043 ceph-mon[30950]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-09-11T23:20:38.605 INFO:teuthology.orchestra.run.smithi043.stdout:141733920825 2024-09-11T23:20:38.606 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T23:20:38.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:38 smithi090 ceph-mon[35130]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-09-11T23:20:39.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.146+0000 7f44718a5640 1 -- 172.21.15.43:0/107213805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f446c101de0 msgr2=0x7f446c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:39.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.146+0000 7f44718a5640 1 --2- 172.21.15.43:0/107213805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f446c101de0 0x7f446c1021c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f445c009900 tx=0x7f445c02a410 comp rx=0 tx=0).stop 2024-09-11T23:20:39.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.146+0000 7f44718a5640 1 -- 172.21.15.43:0/107213805 shutdown_connections 2024-09-11T23:20:39.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.146+0000 7f44718a5640 1 --2- 172.21.15.43:0/107213805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f446c106f10 0x7f446c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:39.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.146+0000 7f44718a5640 1 --2- 172.21.15.43:0/107213805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f446c102790 0x7f446c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:39.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.146+0000 7f44718a5640 1 --2- 172.21.15.43:0/107213805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f446c101de0 0x7f446c1021c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:39.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.146+0000 7f44718a5640 1 -- 172.21.15.43:0/107213805 >> 172.21.15.43:0/107213805 conn(0x7f446c0fd8c0 msgr2=0x7f446c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:39.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.146+0000 7f44718a5640 1 -- 172.21.15.43:0/107213805 shutdown_connections 2024-09-11T23:20:39.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.146+0000 7f44718a5640 1 -- 172.21.15.43:0/107213805 wait complete. 2024-09-11T23:20:39.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.147+0000 7f44718a5640 1 Processor -- start 2024-09-11T23:20:39.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.147+0000 7f44718a5640 1 -- start start 2024-09-11T23:20:39.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.147+0000 7f44718a5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f446c101de0 0x7f446c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:39.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.147+0000 7f44718a5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f446c102790 0x7f446c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:39.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.147+0000 7f44718a5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f446c106f10 0x7f446c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:39.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.147+0000 7f44718a5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f446c112010 con 0x7f446c101de0 2024-09-11T23:20:39.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.147+0000 7f44718a5640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f446c111e90 con 0x7f446c102790 2024-09-11T23:20:39.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.147+0000 7f44718a5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f446c112190 con 0x7f446c106f10 2024-09-11T23:20:39.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.147+0000 7f446affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f446c101de0 0x7f446c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:39.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.147+0000 7f446affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f446c101de0 0x7f446c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44062/0 (socket says 172.21.15.43:44062) 2024-09-11T23:20:39.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.147+0000 7f446affd640 1 -- 172.21.15.43:0/523031875 learned_addr learned my addr 172.21.15.43:0/523031875 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:39.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.147+0000 7f446b7fe640 1 --2- 172.21.15.43:0/523031875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f446c106f10 0x7f446c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:39.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.147+0000 7f446a7fc640 1 --2- 172.21.15.43:0/523031875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f446c102790 0x7f446c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:39.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.147+0000 7f446affd640 1 -- 172.21.15.43:0/523031875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f446c106f10 msgr2=0x7f446c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:39.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.147+0000 7f446affd640 1 --2- 172.21.15.43:0/523031875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f446c106f10 0x7f446c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:39.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.147+0000 7f446affd640 1 -- 172.21.15.43:0/523031875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f446c102790 msgr2=0x7f446c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:39.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.147+0000 7f446affd640 1 --2- 172.21.15.43:0/523031875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f446c102790 0x7f446c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:39.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.147+0000 7f446affd640 1 -- 172.21.15.43:0/523031875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446c1a6a70 con 0x7f446c101de0 2024-09-11T23:20:39.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.148+0000 7f446affd640 1 --2- 172.21.15.43:0/523031875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f446c101de0 0x7f446c0744f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f445c005ab0 tx=0x7f445c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:39.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.148+0000 7f44708a3640 1 -- 172.21.15.43:0/523031875 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c03f030 con 0x7f446c101de0 2024-09-11T23:20:39.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.148+0000 7f44708a3640 1 -- 172.21.15.43:0/523031875 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f445c003ce0 con 0x7f446c101de0 2024-09-11T23:20:39.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.148+0000 7f44718a5640 1 -- 172.21.15.43:0/523031875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f446c1a6d00 con 0x7f446c101de0 2024-09-11T23:20:39.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.148+0000 7f44708a3640 1 -- 172.21.15.43:0/523031875 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c035070 con 0x7f446c101de0 2024-09-11T23:20:39.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.148+0000 7f44718a5640 1 -- 172.21.15.43:0/523031875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f446c1a7190 con 0x7f446c101de0 2024-09-11T23:20:39.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.149+0000 7f44718a5640 1 -- 172.21.15.43:0/523031875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f446c0628c0 con 0x7f446c101de0 2024-09-11T23:20:39.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.150+0000 7f44708a3640 1 -- 172.21.15.43:0/523031875 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f445c042030 con 0x7f446c101de0 2024-09-11T23:20:39.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.151+0000 7f44708a3640 1 --2- 172.21.15.43:0/523031875 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f44540a2c10 0x7f44540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:39.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.151+0000 7f446a7fc640 1 --2- 172.21.15.43:0/523031875 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f44540a2c10 0x7f44540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:39.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.151+0000 7f44708a3640 1 -- 172.21.15.43:0/523031875 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f445c0efc00 con 0x7f446c101de0 2024-09-11T23:20:39.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.151+0000 7f44708a3640 1 --2- 172.21.15.43:0/523031875 >> [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011] conn(0x7f44540ac460 0x7f44540ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:39.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.151+0000 7f44708a3640 1 -- 172.21.15.43:0/523031875 --> [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f445c001d30 con 0x7f44540ac460 2024-09-11T23:20:39.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.151+0000 7f44708a3640 1 -- 172.21.15.43:0/523031875 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f445c0f00e0 con 0x7f446c101de0 2024-09-11T23:20:39.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.151+0000 7f446b7fe640 1 --2- 172.21.15.43:0/523031875 >> [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011] conn(0x7f44540ac460 0x7f44540ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:39.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.151+0000 7f446a7fc640 1 --2- 172.21.15.43:0/523031875 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f44540a2c10 0x7f44540a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f446c075f80 tx=0x7f4460008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:39.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.152+0000 7f446b7fe640 1 --2- 172.21.15.43:0/523031875 >> [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011] conn(0x7f44540ac460 0x7f44540ae880 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:39.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.152+0000 7f44708a3640 1 -- 172.21.15.43:0/523031875 <== osd.9 v2:172.21.15.43:6826/3170306011 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f445c001d30 con 0x7f44540ac460 2024-09-11T23:20:39.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.166+0000 7f44718a5640 1 -- 172.21.15.43:0/523031875 --> [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f446c062d40 con 0x7f44540ac460 2024-09-11T23:20:39.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.166+0000 7f44708a3640 1 -- 172.21.15.43:0/523031875 <== osd.9 v2:172.21.15.43:6826/3170306011 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f446c062d40 con 0x7f44540ac460 2024-09-11T23:20:39.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.166+0000 7f44718a5640 1 -- 172.21.15.43:0/523031875 >> [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011] conn(0x7f44540ac460 msgr2=0x7f44540ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:39.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.166+0000 7f44718a5640 1 --2- 172.21.15.43:0/523031875 >> [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011] conn(0x7f44540ac460 0x7f44540ae880 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:39.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.167+0000 7f44718a5640 1 -- 172.21.15.43:0/523031875 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f44540a2c10 msgr2=0x7f44540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:39.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.167+0000 7f44718a5640 1 --2- 172.21.15.43:0/523031875 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f44540a2c10 0x7f44540a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f446c075f80 tx=0x7f4460008040 comp rx=0 tx=0).stop 2024-09-11T23:20:39.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.167+0000 7f44718a5640 1 -- 172.21.15.43:0/523031875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f446c101de0 msgr2=0x7f446c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:39.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.167+0000 7f44718a5640 1 --2- 172.21.15.43:0/523031875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f446c101de0 0x7f446c0744f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f445c005ab0 tx=0x7f445c0042f0 comp rx=0 tx=0).stop 2024-09-11T23:20:39.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.167+0000 7f446b7fe640 1 -- 172.21.15.43:0/523031875 reap_dead start 2024-09-11T23:20:39.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.167+0000 7f44718a5640 1 -- 172.21.15.43:0/523031875 shutdown_connections 2024-09-11T23:20:39.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.167+0000 7f44718a5640 1 -- 172.21.15.43:0/523031875 >> 172.21.15.43:0/523031875 conn(0x7f446c0fd8c0 msgr2=0x7f446c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:39.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.167+0000 7f44718a5640 1 -- 172.21.15.43:0/523031875 shutdown_connections 2024-09-11T23:20:39.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:39.167+0000 7f44718a5640 1 -- 172.21.15.43:0/523031875 wait complete. 2024-09-11T23:20:39.966 INFO:teuthology.orchestra.run.smithi043.stdout:274877906976 2024-09-11T23:20:39.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-11T23:20:40.522 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:40 smithi043 ceph-mon[30950]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-11T23:20:40.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:40 smithi097 ceph-mon[35562]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-11T23:20:40.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:40 smithi090 ceph-mon[35130]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-11T23:20:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.863+0000 7f9002390640 1 -- 172.21.15.43:0/2607373900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ffc103780 msgr2=0x7f8ffc107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.863+0000 7f9002390640 1 --2- 172.21.15.43:0/2607373900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ffc103780 0x7f8ffc107bc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0009930 tx=0x7f8ff002a420 comp rx=0 tx=0).stop 2024-09-11T23:20:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.863+0000 7f9002390640 1 -- 172.21.15.43:0/2607373900 shutdown_connections 2024-09-11T23:20:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.863+0000 7f9002390640 1 --2- 172.21.15.43:0/2607373900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ffc108300 0x7f8ffc10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.863+0000 7f9002390640 1 --2- 172.21.15.43:0/2607373900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ffc103780 0x7f8ffc107bc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.863+0000 7f9002390640 1 --2- 172.21.15.43:0/2607373900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ffc102dd0 0x7f8ffc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.863+0000 7f9002390640 1 -- 172.21.15.43:0/2607373900 >> 172.21.15.43:0/2607373900 conn(0x7f8ffc0fc7d0 msgr2=0x7f8ffc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.863+0000 7f9002390640 1 -- 172.21.15.43:0/2607373900 shutdown_connections 2024-09-11T23:20:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.863+0000 7f9002390640 1 -- 172.21.15.43:0/2607373900 wait complete. 2024-09-11T23:20:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.863+0000 7f9002390640 1 Processor -- start 2024-09-11T23:20:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.863+0000 7f9002390640 1 -- start start 2024-09-11T23:20:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.864+0000 7f9002390640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ffc102dd0 0x7f8ffc198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.864+0000 7f9002390640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ffc103780 0x7f8ffc198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.864+0000 7f9002390640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ffc108300 0x7f8ffc19d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.864+0000 7f9002390640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc112150 con 0x7f8ffc108300 2024-09-11T23:20:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.864+0000 7f9002390640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc111fd0 con 0x7f8ffc102dd0 2024-09-11T23:20:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.864+0000 7f9002390640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc1122d0 con 0x7f8ffc103780 2024-09-11T23:20:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.864+0000 7f900092d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ffc108300 0x7f8ffc19d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.864+0000 7f900092d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ffc108300 0x7f8ffc19d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45822/0 (socket says 172.21.15.43:45822) 2024-09-11T23:20:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.864+0000 7f900092d640 1 -- 172.21.15.43:0/3276210465 learned_addr learned my addr 172.21.15.43:0/3276210465 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.864+0000 7f8ffbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ffc102dd0 0x7f8ffc198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.864+0000 7f8ffb7fe640 1 --2- 172.21.15.43:0/3276210465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ffc103780 0x7f8ffc198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.864+0000 7f900092d640 1 -- 172.21.15.43:0/3276210465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ffc103780 msgr2=0x7f8ffc198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.864+0000 7f900092d640 1 --2- 172.21.15.43:0/3276210465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ffc103780 0x7f8ffc198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.864+0000 7f900092d640 1 -- 172.21.15.43:0/3276210465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ffc102dd0 msgr2=0x7f8ffc198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.864+0000 7f900092d640 1 --2- 172.21.15.43:0/3276210465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ffc102dd0 0x7f8ffc198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.864+0000 7f900092d640 1 -- 172.21.15.43:0/3276210465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ffc19d700 con 0x7f8ffc108300 2024-09-11T23:20:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.864+0000 7f8ffb7fe640 1 --2- 172.21.15.43:0/3276210465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ffc103780 0x7f8ffc198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.865+0000 7f900092d640 1 --2- 172.21.15.43:0/3276210465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ffc108300 0x7f8ffc19d020 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8ff400b970 tx=0x7f8ff400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.865+0000 7f8ff97fa640 1 -- 172.21.15.43:0/3276210465 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4019070 con 0x7f8ffc108300 2024-09-11T23:20:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.865+0000 7f9002390640 1 -- 172.21.15.43:0/3276210465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ffc19d9f0 con 0x7f8ffc108300 2024-09-11T23:20:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.865+0000 7f8ff97fa640 1 -- 172.21.15.43:0/3276210465 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ff4014440 con 0x7f8ffc108300 2024-09-11T23:20:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.865+0000 7f8ff97fa640 1 -- 172.21.15.43:0/3276210465 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff40075d0 con 0x7f8ffc108300 2024-09-11T23:20:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.865+0000 7f9002390640 1 -- 172.21.15.43:0/3276210465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ffc1a52d0 con 0x7f8ffc108300 2024-09-11T23:20:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.866+0000 7f9002390640 1 -- 172.21.15.43:0/3276210465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8ffc062940 con 0x7f8ffc108300 2024-09-11T23:20:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.867+0000 7f8ff97fa640 1 -- 172.21.15.43:0/3276210465 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8ff401c070 con 0x7f8ffc108300 2024-09-11T23:20:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.867+0000 7f8ff97fa640 1 --2- 172.21.15.43:0/3276210465 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f8fe40a2c10 0x7f8fe40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.868+0000 7f8ff97fa640 1 -- 172.21.15.43:0/3276210465 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f8ff40ced30 con 0x7f8ffc108300 2024-09-11T23:20:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.868+0000 7f8ffbfff640 1 --2- 172.21.15.43:0/3276210465 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f8fe40a2c10 0x7f8fe40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.868+0000 7f8ff97fa640 1 --2- 172.21.15.43:0/3276210465 >> [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804] conn(0x7f8fe40ac460 0x7f8fe40ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.868+0000 7f8ff97fa640 1 -- 172.21.15.43:0/3276210465 --> [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8fe40aef30 con 0x7f8fe40ac460 2024-09-11T23:20:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.868+0000 7f8ff97fa640 1 -- 172.21.15.43:0/3276210465 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f8ff40cf210 con 0x7f8ffc108300 2024-09-11T23:20:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.868+0000 7f8ffb7fe640 1 --2- 172.21.15.43:0/3276210465 >> [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804] conn(0x7f8fe40ac460 0x7f8fe40ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.868+0000 7f8ffbfff640 1 --2- 172.21.15.43:0/3276210465 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f8fe40a2c10 0x7f8fe40a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009990 tx=0x7f8fec009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.868+0000 7f8ffb7fe640 1 --2- 172.21.15.43:0/3276210465 >> [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804] conn(0x7f8fe40ac460 0x7f8fe40ae880 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.869+0000 7f8ff97fa640 1 -- 172.21.15.43:0/3276210465 <== osd.3 v2:172.21.15.43:6810/1962837804 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8fe40aef30 con 0x7f8fe40ac460 2024-09-11T23:20:40.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.883+0000 7f9002390640 1 -- 172.21.15.43:0/3276210465 --> [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8ffc062dc0 con 0x7f8fe40ac460 2024-09-11T23:20:40.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.883+0000 7f8ff97fa640 1 -- 172.21.15.43:0/3276210465 <== osd.3 v2:172.21.15.43:6810/1962837804 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8ffc062dc0 con 0x7f8fe40ac460 2024-09-11T23:20:40.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.883+0000 7f9002390640 1 -- 172.21.15.43:0/3276210465 >> [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804] conn(0x7f8fe40ac460 msgr2=0x7f8fe40ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:40.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.883+0000 7f9002390640 1 --2- 172.21.15.43:0/3276210465 >> [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804] conn(0x7f8fe40ac460 0x7f8fe40ae880 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:40.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.883+0000 7f9002390640 1 -- 172.21.15.43:0/3276210465 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f8fe40a2c10 msgr2=0x7f8fe40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:40.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.883+0000 7f9002390640 1 --2- 172.21.15.43:0/3276210465 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f8fe40a2c10 0x7f8fe40a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009990 tx=0x7f8fec009110 comp rx=0 tx=0).stop 2024-09-11T23:20:40.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.883+0000 7f9002390640 1 -- 172.21.15.43:0/3276210465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ffc108300 msgr2=0x7f8ffc19d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:40.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.883+0000 7f9002390640 1 --2- 172.21.15.43:0/3276210465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ffc108300 0x7f8ffc19d020 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8ff400b970 tx=0x7f8ff400bd70 comp rx=0 tx=0).stop 2024-09-11T23:20:40.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.883+0000 7f900092d640 1 -- 172.21.15.43:0/3276210465 reap_dead start 2024-09-11T23:20:40.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.883+0000 7f9002390640 1 -- 172.21.15.43:0/3276210465 shutdown_connections 2024-09-11T23:20:40.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.883+0000 7f9002390640 1 -- 172.21.15.43:0/3276210465 >> 172.21.15.43:0/3276210465 conn(0x7f8ffc0fc7d0 msgr2=0x7f8ffc105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:40.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.883+0000 7f9002390640 1 -- 172.21.15.43:0/3276210465 shutdown_connections 2024-09-11T23:20:40.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:40.884+0000 7f9002390640 1 -- 172.21.15.43:0/3276210465 wait complete. 2024-09-11T23:20:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.058+0000 7f62714a5640 1 -- 172.21.15.43:0/1617486579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f626c108300 msgr2=0x7f626c10ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.058+0000 7f62714a5640 1 --2- 172.21.15.43:0/1617486579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f626c108300 0x7f626c10ec70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f625c009cd0 tx=0x7f625c02a8d0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.059+0000 7f62714a5640 1 -- 172.21.15.43:0/1617486579 shutdown_connections 2024-09-11T23:20:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.059+0000 7f62714a5640 1 --2- 172.21.15.43:0/1617486579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f626c108300 0x7f626c10ec70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.059+0000 7f62714a5640 1 --2- 172.21.15.43:0/1617486579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626c103780 0x7f626c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.059+0000 7f62714a5640 1 --2- 172.21.15.43:0/1617486579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f626c102dd0 0x7f626c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.059+0000 7f62714a5640 1 -- 172.21.15.43:0/1617486579 >> 172.21.15.43:0/1617486579 conn(0x7f626c0fc7d0 msgr2=0x7f626c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.059+0000 7f62714a5640 1 -- 172.21.15.43:0/1617486579 shutdown_connections 2024-09-11T23:20:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.059+0000 7f62714a5640 1 -- 172.21.15.43:0/1617486579 wait complete. 2024-09-11T23:20:42.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.059+0000 7f62714a5640 1 Processor -- start 2024-09-11T23:20:42.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.060+0000 7f62714a5640 1 -- start start 2024-09-11T23:20:42.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.060+0000 7f62714a5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f626c102dd0 0x7f626c100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.060+0000 7f62714a5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f626c103780 0x7f626c100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.060+0000 7f62714a5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626c108300 0x7f626c1014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.060+0000 7f62714a5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f626c1120d0 con 0x7f626c102dd0 2024-09-11T23:20:42.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.060+0000 7f62714a5640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f626c111f50 con 0x7f626c103780 2024-09-11T23:20:42.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.060+0000 7f62714a5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f626c112250 con 0x7f626c108300 2024-09-11T23:20:42.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.060+0000 7f626affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f626c102dd0 0x7f626c100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.060+0000 7f626affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f626c102dd0 0x7f626c100a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45872/0 (socket says 172.21.15.43:45872) 2024-09-11T23:20:42.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.060+0000 7f626affd640 1 -- 172.21.15.43:0/344083629 learned_addr learned my addr 172.21.15.43:0/344083629 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:42.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.060+0000 7f626a7fc640 1 --2- 172.21.15.43:0/344083629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f626c103780 0x7f626c100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.060+0000 7f626b7fe640 1 --2- 172.21.15.43:0/344083629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626c108300 0x7f626c1014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.060+0000 7f626affd640 1 -- 172.21.15.43:0/344083629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626c108300 msgr2=0x7f626c1014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:42.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.060+0000 7f626affd640 1 --2- 172.21.15.43:0/344083629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f626c108300 0x7f626c1014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.060+0000 7f626affd640 1 -- 172.21.15.43:0/344083629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f626c103780 msgr2=0x7f626c100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:42.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.060+0000 7f626affd640 1 --2- 172.21.15.43:0/344083629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f626c103780 0x7f626c100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.060+0000 7f626affd640 1 -- 172.21.15.43:0/344083629 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626c1a69d0 con 0x7f626c102dd0 2024-09-11T23:20:42.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.061+0000 7f626affd640 1 --2- 172.21.15.43:0/344083629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f626c102dd0 0x7f626c100a30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f626400c920 tx=0x7f626400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:42.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.061+0000 7f6263fff640 1 -- 172.21.15.43:0/344083629 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264007c70 con 0x7f626c102dd0 2024-09-11T23:20:42.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.061+0000 7f6263fff640 1 -- 172.21.15.43:0/344083629 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6264007e10 con 0x7f626c102dd0 2024-09-11T23:20:42.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.061+0000 7f62714a5640 1 -- 172.21.15.43:0/344083629 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626c1a6cc0 con 0x7f626c102dd0 2024-09-11T23:20:42.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.061+0000 7f6263fff640 1 -- 172.21.15.43:0/344083629 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264018890 con 0x7f626c102dd0 2024-09-11T23:20:42.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.061+0000 7f62714a5640 1 -- 172.21.15.43:0/344083629 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f626c1a71b0 con 0x7f626c102dd0 2024-09-11T23:20:42.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.062+0000 7f62714a5640 1 -- 172.21.15.43:0/344083629 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f626c107720 con 0x7f626c102dd0 2024-09-11T23:20:42.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.063+0000 7f6263fff640 1 -- 172.21.15.43:0/344083629 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6264018a70 con 0x7f626c102dd0 2024-09-11T23:20:42.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.064+0000 7f6263fff640 1 --2- 172.21.15.43:0/344083629 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f62580a2c10 0x7f62580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.064+0000 7f626a7fc640 1 --2- 172.21.15.43:0/344083629 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f62580a2c10 0x7f62580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.064+0000 7f6263fff640 1 -- 172.21.15.43:0/344083629 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f62640cf170 con 0x7f626c102dd0 2024-09-11T23:20:42.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.064+0000 7f6263fff640 1 --2- 172.21.15.43:0/344083629 >> [v2:172.21.15.90:6816/236874962,v1:172.21.15.90:6817/236874962] conn(0x7f62580ac460 0x7f62580ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.064+0000 7f6263fff640 1 -- 172.21.15.43:0/344083629 --> [v2:172.21.15.90:6816/236874962,v1:172.21.15.90:6817/236874962] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6264012ec0 con 0x7f62580ac460 2024-09-11T23:20:42.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.064+0000 7f6263fff640 1 -- 172.21.15.43:0/344083629 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f62640d6050 con 0x7f626c102dd0 2024-09-11T23:20:42.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.064+0000 7f626a7fc640 1 --2- 172.21.15.43:0/344083629 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f62580a2c10 0x7f62580a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f626c1024c0 tx=0x7f625400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:42.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.064+0000 7f626b7fe640 1 --2- 172.21.15.43:0/344083629 >> [v2:172.21.15.90:6816/236874962,v1:172.21.15.90:6817/236874962] conn(0x7f62580ac460 0x7f62580ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.065+0000 7f626b7fe640 1 --2- 172.21.15.43:0/344083629 >> [v2:172.21.15.90:6816/236874962,v1:172.21.15.90:6817/236874962] conn(0x7f62580ac460 0x7f62580ae880 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:42.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.066+0000 7f6263fff640 1 -- 172.21.15.43:0/344083629 <== osd.7 v2:172.21.15.90:6816/236874962 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6264012ec0 con 0x7f62580ac460 2024-09-11T23:20:42.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.079+0000 7f62714a5640 1 -- 172.21.15.43:0/344083629 --> [v2:172.21.15.90:6816/236874962,v1:172.21.15.90:6817/236874962] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f626c06a7b0 con 0x7f62580ac460 2024-09-11T23:20:42.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.080+0000 7f6263fff640 1 -- 172.21.15.43:0/344083629 <== osd.7 v2:172.21.15.90:6816/236874962 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f626c06a7b0 con 0x7f62580ac460 2024-09-11T23:20:42.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.080+0000 7f62714a5640 1 -- 172.21.15.43:0/344083629 >> [v2:172.21.15.90:6816/236874962,v1:172.21.15.90:6817/236874962] conn(0x7f62580ac460 msgr2=0x7f62580ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:42.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.080+0000 7f62714a5640 1 --2- 172.21.15.43:0/344083629 >> [v2:172.21.15.90:6816/236874962,v1:172.21.15.90:6817/236874962] conn(0x7f62580ac460 0x7f62580ae880 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.080+0000 7f62714a5640 1 -- 172.21.15.43:0/344083629 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f62580a2c10 msgr2=0x7f62580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:42.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.080+0000 7f62714a5640 1 --2- 172.21.15.43:0/344083629 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f62580a2c10 0x7f62580a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f626c1024c0 tx=0x7f625400b390 comp rx=0 tx=0).stop 2024-09-11T23:20:42.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.080+0000 7f62714a5640 1 -- 172.21.15.43:0/344083629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f626c102dd0 msgr2=0x7f626c100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.080+0000 7f62714a5640 1 --2- 172.21.15.43:0/344083629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f626c102dd0 0x7f626c100a30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f626400c920 tx=0x7f626400cde0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.080+0000 7f626b7fe640 1 -- 172.21.15.43:0/344083629 reap_dead start 2024-09-11T23:20:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.081+0000 7f62714a5640 1 -- 172.21.15.43:0/344083629 shutdown_connections 2024-09-11T23:20:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.081+0000 7f62714a5640 1 -- 172.21.15.43:0/344083629 >> 172.21.15.43:0/344083629 conn(0x7f626c0fc7d0 msgr2=0x7f626c105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.081+0000 7f62714a5640 1 -- 172.21.15.43:0/344083629 shutdown_connections 2024-09-11T23:20:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:42.081+0000 7f62714a5640 1 -- 172.21.15.43:0/344083629 wait complete. 2024-09-11T23:20:42.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:42 smithi043 ceph-mon[30950]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-11T23:20:42.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.964+0000 7f6c693a9640 1 -- 172.21.15.43:0/1990620690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c64100d20 msgr2=0x7f6c64106860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.964+0000 7f6c693a9640 1 --2- 172.21.15.43:0/1990620690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c64100d20 0x7f6c64106860 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009930 tx=0x7f6c5802a420 comp rx=0 tx=0).stop 2024-09-11T23:20:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.964+0000 7f6c693a9640 1 -- 172.21.15.43:0/1990620690 shutdown_connections 2024-09-11T23:20:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.964+0000 7f6c693a9640 1 --2- 172.21.15.43:0/1990620690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c64106fc0 0x7f6c6410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.964+0000 7f6c693a9640 1 --2- 172.21.15.43:0/1990620690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c64100d20 0x7f6c64106860 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.964+0000 7f6c693a9640 1 --2- 172.21.15.43:0/1990620690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c64100370 0x7f6c64100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.964+0000 7f6c693a9640 1 -- 172.21.15.43:0/1990620690 >> 172.21.15.43:0/1990620690 conn(0x7f6c64077210 msgr2=0x7f6c640fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.964+0000 7f6c693a9640 1 -- 172.21.15.43:0/1990620690 shutdown_connections 2024-09-11T23:20:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.964+0000 7f6c693a9640 1 -- 172.21.15.43:0/1990620690 wait complete. 2024-09-11T23:20:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.965+0000 7f6c693a9640 1 Processor -- start 2024-09-11T23:20:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.965+0000 7f6c693a9640 1 -- start start 2024-09-11T23:20:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.965+0000 7f6c693a9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c64100370 0x7f6c641a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.965+0000 7f6c693a9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c64100d20 0x7f6c641a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.965+0000 7f6c693a9640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c64106fc0 0x7f6c6419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.965+0000 7f6c693a9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6c64112040 con 0x7f6c64100d20 2024-09-11T23:20:42.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.965+0000 7f6c693a9640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6c64111ec0 con 0x7f6c64106fc0 2024-09-11T23:20:42.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.965+0000 7f6c693a9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6c641121c0 con 0x7f6c64100370 2024-09-11T23:20:42.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.965+0000 7f6c627fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c64100d20 0x7f6c641a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.965+0000 7f6c627fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c64100d20 0x7f6c641a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45840/0 (socket says 172.21.15.43:45840) 2024-09-11T23:20:42.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.965+0000 7f6c627fc640 1 -- 172.21.15.43:0/3224381253 learned_addr learned my addr 172.21.15.43:0/3224381253 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:42.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.965+0000 7f6c637fe640 1 --2- 172.21.15.43:0/3224381253 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c64106fc0 0x7f6c6419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.966+0000 7f6c62ffd640 1 --2- 172.21.15.43:0/3224381253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c64100370 0x7f6c641a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.966+0000 7f6c627fc640 1 -- 172.21.15.43:0/3224381253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c64100370 msgr2=0x7f6c641a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:42.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.966+0000 7f6c627fc640 1 --2- 172.21.15.43:0/3224381253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c64100370 0x7f6c641a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.966+0000 7f6c627fc640 1 -- 172.21.15.43:0/3224381253 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c64106fc0 msgr2=0x7f6c6419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.966+0000 7f6c627fc640 1 --2- 172.21.15.43:0/3224381253 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c64106fc0 0x7f6c6419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.966+0000 7f6c627fc640 1 -- 172.21.15.43:0/3224381253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c6419ad20 con 0x7f6c64100d20 2024-09-11T23:20:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.966+0000 7f6c627fc640 1 --2- 172.21.15.43:0/3224381253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c64100d20 0x7f6c641a0970 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009230 tx=0x7f6c580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.966+0000 7f6c4bfff640 1 -- 172.21.15.43:0/3224381253 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5803f030 con 0x7f6c64100d20 2024-09-11T23:20:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.966+0000 7f6c693a9640 1 -- 172.21.15.43:0/3224381253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c6419afb0 con 0x7f6c64100d20 2024-09-11T23:20:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.966+0000 7f6c4bfff640 1 -- 172.21.15.43:0/3224381253 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c58003ce0 con 0x7f6c64100d20 2024-09-11T23:20:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.966+0000 7f6c4bfff640 1 -- 172.21.15.43:0/3224381253 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58035070 con 0x7f6c64100d20 2024-09-11T23:20:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.966+0000 7f6c693a9640 1 -- 172.21.15.43:0/3224381253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c641a7300 con 0x7f6c64100d20 2024-09-11T23:20:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.967+0000 7f6c693a9640 1 -- 172.21.15.43:0/3224381253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6c6410e150 con 0x7f6c64100d20 2024-09-11T23:20:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.968+0000 7f6c4bfff640 1 -- 172.21.15.43:0/3224381253 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6c58042030 con 0x7f6c64100d20 2024-09-11T23:20:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.969+0000 7f6c4bfff640 1 --2- 172.21.15.43:0/3224381253 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f6c4c0a2c10 0x7f6c4c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.969+0000 7f6c62ffd640 1 --2- 172.21.15.43:0/3224381253 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f6c4c0a2c10 0x7f6c4c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.969+0000 7f6c4bfff640 1 -- 172.21.15.43:0/3224381253 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f6c580efa80 con 0x7f6c64100d20 2024-09-11T23:20:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.969+0000 7f6c4bfff640 1 --2- 172.21.15.43:0/3224381253 >> [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997] conn(0x7f6c4c0ac450 0x7f6c4c0ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.969+0000 7f6c4bfff640 1 -- 172.21.15.43:0/3224381253 --> [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6c5800b0b0 con 0x7f6c4c0ac450 2024-09-11T23:20:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.969+0000 7f6c4bfff640 1 -- 172.21.15.43:0/3224381253 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f6c580eff60 con 0x7f6c64100d20 2024-09-11T23:20:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.969+0000 7f6c637fe640 1 --2- 172.21.15.43:0/3224381253 >> [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997] conn(0x7f6c4c0ac450 0x7f6c4c0ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.969+0000 7f6c62ffd640 1 --2- 172.21.15.43:0/3224381253 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f6c4c0a2c10 0x7f6c4c0a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009890 tx=0x7f6c54008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.970+0000 7f6c637fe640 1 --2- 172.21.15.43:0/3224381253 >> [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997] conn(0x7f6c4c0ac450 0x7f6c4c0ae870 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.970+0000 7f6c4bfff640 1 -- 172.21.15.43:0/3224381253 <== osd.0 v2:172.21.15.43:6802/43839997 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6c5800b0b0 con 0x7f6c4c0ac450 2024-09-11T23:20:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.984+0000 7f6c693a9640 1 -- 172.21.15.43:0/3224381253 --> [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6c64103430 con 0x7f6c4c0ac450 2024-09-11T23:20:42.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.984+0000 7f6c4bfff640 1 -- 172.21.15.43:0/3224381253 <== osd.0 v2:172.21.15.43:6802/43839997 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6c64103430 con 0x7f6c4c0ac450 2024-09-11T23:20:42.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.985+0000 7f6c693a9640 1 -- 172.21.15.43:0/3224381253 >> [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997] conn(0x7f6c4c0ac450 msgr2=0x7f6c4c0ae870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:42.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.985+0000 7f6c693a9640 1 --2- 172.21.15.43:0/3224381253 >> [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997] conn(0x7f6c4c0ac450 0x7f6c4c0ae870 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.985+0000 7f6c693a9640 1 -- 172.21.15.43:0/3224381253 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f6c4c0a2c10 msgr2=0x7f6c4c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:42.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.985+0000 7f6c693a9640 1 --2- 172.21.15.43:0/3224381253 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f6c4c0a2c10 0x7f6c4c0a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009890 tx=0x7f6c54008040 comp rx=0 tx=0).stop 2024-09-11T23:20:42.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.985+0000 7f6c693a9640 1 -- 172.21.15.43:0/3224381253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c64100d20 msgr2=0x7f6c641a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:42.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.985+0000 7f6c693a9640 1 --2- 172.21.15.43:0/3224381253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c64100d20 0x7f6c641a0970 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009230 tx=0x7f6c580042f0 comp rx=0 tx=0).stop 2024-09-11T23:20:42.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.985+0000 7f6c637fe640 1 -- 172.21.15.43:0/3224381253 reap_dead start 2024-09-11T23:20:42.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.985+0000 7f6c693a9640 1 -- 172.21.15.43:0/3224381253 shutdown_connections 2024-09-11T23:20:42.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.985+0000 7f6c693a9640 1 -- 172.21.15.43:0/3224381253 >> 172.21.15.43:0/3224381253 conn(0x7f6c64077210 msgr2=0x7f6c64102850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:42.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.985+0000 7f6c693a9640 1 -- 172.21.15.43:0/3224381253 shutdown_connections 2024-09-11T23:20:42.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:41.985+0000 7f6c693a9640 1 -- 172.21.15.43:0/3224381253 wait complete. 2024-09-11T23:20:42.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:42 smithi097 ceph-mon[35562]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-11T23:20:42.572 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:42 smithi090 ceph-mon[35130]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-11T23:20:43.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.019+0000 7f67f951e640 1 -- 172.21.15.43:0/1426658544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67f4100e20 msgr2=0x7f67f4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:43.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.019+0000 7f67f951e640 1 --2- 172.21.15.43:0/1426658544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67f4100e20 0x7f67f4101200 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009900 tx=0x7f67e402a480 comp rx=0 tx=0).stop 2024-09-11T23:20:43.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.019+0000 7f67f951e640 1 -- 172.21.15.43:0/1426658544 shutdown_connections 2024-09-11T23:20:43.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.019+0000 7f67f951e640 1 --2- 172.21.15.43:0/1426658544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67f4102ef0 0x7f67f410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:43.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.019+0000 7f67f951e640 1 --2- 172.21.15.43:0/1426658544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67f4102570 0x7f67f41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:43.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.019+0000 7f67f951e640 1 --2- 172.21.15.43:0/1426658544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67f4100e20 0x7f67f4101200 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:43.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.019+0000 7f67f951e640 1 -- 172.21.15.43:0/1426658544 >> 172.21.15.43:0/1426658544 conn(0x7f67f40fc7d0 msgr2=0x7f67f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:43.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.019+0000 7f67f951e640 1 -- 172.21.15.43:0/1426658544 shutdown_connections 2024-09-11T23:20:43.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.019+0000 7f67f951e640 1 -- 172.21.15.43:0/1426658544 wait complete. 2024-09-11T23:20:43.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.019+0000 7f67f951e640 1 Processor -- start 2024-09-11T23:20:43.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.020+0000 7f67f951e640 1 -- start start 2024-09-11T23:20:43.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.020+0000 7f67f951e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67f4100e20 0x7f67f4198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:43.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.020+0000 7f67f951e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67f4102570 0x7f67f4198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:43.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.020+0000 7f67f951e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67f4102ef0 0x7f67f419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:43.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.020+0000 7f67f951e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f67f4110ff0 con 0x7f67f4102570 2024-09-11T23:20:43.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.020+0000 7f67f951e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f67f4110e70 con 0x7f67f4102ef0 2024-09-11T23:20:43.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.020+0000 7f67f951e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f67f4111170 con 0x7f67f4100e20 2024-09-11T23:20:43.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.020+0000 7f67f27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67f4102570 0x7f67f4198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:43.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.020+0000 7f67f27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67f4102570 0x7f67f4198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45898/0 (socket says 172.21.15.43:45898) 2024-09-11T23:20:43.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.020+0000 7f67f27fc640 1 -- 172.21.15.43:0/2770697034 learned_addr learned my addr 172.21.15.43:0/2770697034 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:43.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.020+0000 7f67f37fe640 1 --2- 172.21.15.43:0/2770697034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67f4102ef0 0x7f67f419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:43.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.020+0000 7f67f2ffd640 1 --2- 172.21.15.43:0/2770697034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67f4100e20 0x7f67f4198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:43.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.020+0000 7f67f27fc640 1 -- 172.21.15.43:0/2770697034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67f4100e20 msgr2=0x7f67f4198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:43.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.020+0000 7f67f27fc640 1 --2- 172.21.15.43:0/2770697034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67f4100e20 0x7f67f4198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:43.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.020+0000 7f67f27fc640 1 -- 172.21.15.43:0/2770697034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67f4102ef0 msgr2=0x7f67f419ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.020+0000 7f67f27fc640 1 --2- 172.21.15.43:0/2770697034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f67f4102ef0 0x7f67f419ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.020+0000 7f67f27fc640 1 -- 172.21.15.43:0/2770697034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f419d4f0 con 0x7f67f4102570 2024-09-11T23:20:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.020+0000 7f67f2ffd640 1 --2- 172.21.15.43:0/2770697034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67f4100e20 0x7f67f4198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:20:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.020+0000 7f67f27fc640 1 --2- 172.21.15.43:0/2770697034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67f4102570 0x7f67f4198a80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f67e800c920 tx=0x7f67e800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.021+0000 7f67dbfff640 1 -- 172.21.15.43:0/2770697034 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8007c70 con 0x7f67f4102570 2024-09-11T23:20:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.021+0000 7f67f951e640 1 -- 172.21.15.43:0/2770697034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f419d7e0 con 0x7f67f4102570 2024-09-11T23:20:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.021+0000 7f67dbfff640 1 -- 172.21.15.43:0/2770697034 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e8007e10 con 0x7f67f4102570 2024-09-11T23:20:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.021+0000 7f67dbfff640 1 -- 172.21.15.43:0/2770697034 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8018890 con 0x7f67f4102570 2024-09-11T23:20:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.021+0000 7f67f951e640 1 -- 172.21.15.43:0/2770697034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f41a50c0 con 0x7f67f4102570 2024-09-11T23:20:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.022+0000 7f67f951e640 1 -- 172.21.15.43:0/2770697034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f67f4062980 con 0x7f67f4102570 2024-09-11T23:20:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.023+0000 7f67dbfff640 1 -- 172.21.15.43:0/2770697034 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f67e8018a30 con 0x7f67f4102570 2024-09-11T23:20:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.023+0000 7f67dbfff640 1 --2- 172.21.15.43:0/2770697034 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f67dc0a2c10 0x7f67dc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.024+0000 7f67f2ffd640 1 --2- 172.21.15.43:0/2770697034 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f67dc0a2c10 0x7f67dc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:43.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.024+0000 7f67dbfff640 1 -- 172.21.15.43:0/2770697034 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f67e80cf660 con 0x7f67f4102570 2024-09-11T23:20:43.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.024+0000 7f67dbfff640 1 --2- 172.21.15.43:0/2770697034 >> [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637] conn(0x7f67dc0ac460 0x7f67dc0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:43.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.024+0000 7f67dbfff640 1 -- 172.21.15.43:0/2770697034 --> [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f67dc0aef30 con 0x7f67dc0ac460 2024-09-11T23:20:43.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.024+0000 7f67dbfff640 1 -- 172.21.15.43:0/2770697034 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f67e80cfb40 con 0x7f67f4102570 2024-09-11T23:20:43.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.024+0000 7f67f37fe640 1 --2- 172.21.15.43:0/2770697034 >> [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637] conn(0x7f67dc0ac460 0x7f67dc0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.024+0000 7f67f2ffd640 1 --2- 172.21.15.43:0/2770697034 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f67dc0a2c10 0x7f67dc0a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f67e4005ab0 tx=0x7f67e4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.024+0000 7f67f37fe640 1 --2- 172.21.15.43:0/2770697034 >> [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637] conn(0x7f67dc0ac460 0x7f67dc0ae880 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.025+0000 7f67dbfff640 1 -- 172.21.15.43:0/2770697034 <== osd.6 v2:172.21.15.43:6818/3785677637 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f67dc0aef30 con 0x7f67dc0ac460 2024-09-11T23:20:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.039+0000 7f67f951e640 1 -- 172.21.15.43:0/2770697034 --> [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f67f4062d80 con 0x7f67dc0ac460 2024-09-11T23:20:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.039+0000 7f67dbfff640 1 -- 172.21.15.43:0/2770697034 <== osd.6 v2:172.21.15.43:6818/3785677637 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f67f4062d80 con 0x7f67dc0ac460 2024-09-11T23:20:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.039+0000 7f67f951e640 1 -- 172.21.15.43:0/2770697034 >> [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637] conn(0x7f67dc0ac460 msgr2=0x7f67dc0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.039+0000 7f67f951e640 1 --2- 172.21.15.43:0/2770697034 >> [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637] conn(0x7f67dc0ac460 0x7f67dc0ae880 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.039+0000 7f67f951e640 1 -- 172.21.15.43:0/2770697034 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f67dc0a2c10 msgr2=0x7f67dc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.039+0000 7f67f951e640 1 --2- 172.21.15.43:0/2770697034 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f67dc0a2c10 0x7f67dc0a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f67e4005ab0 tx=0x7f67e4000bc0 comp rx=0 tx=0).stop 2024-09-11T23:20:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.039+0000 7f67f951e640 1 -- 172.21.15.43:0/2770697034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67f4102570 msgr2=0x7f67f4198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.039+0000 7f67f951e640 1 --2- 172.21.15.43:0/2770697034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67f4102570 0x7f67f4198a80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f67e800c920 tx=0x7f67e800cde0 comp rx=0 tx=0).stop 2024-09-11T23:20:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.039+0000 7f67f37fe640 1 -- 172.21.15.43:0/2770697034 reap_dead start 2024-09-11T23:20:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.039+0000 7f67f951e640 1 -- 172.21.15.43:0/2770697034 shutdown_connections 2024-09-11T23:20:43.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.039+0000 7f67f951e640 1 -- 172.21.15.43:0/2770697034 >> 172.21.15.43:0/2770697034 conn(0x7f67f40fc7d0 msgr2=0x7f67f406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:43.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.040+0000 7f67f951e640 1 -- 172.21.15.43:0/2770697034 shutdown_connections 2024-09-11T23:20:43.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:43.040+0000 7f67f951e640 1 -- 172.21.15.43:0/2770697034 wait complete. 2024-09-11T23:20:43.921 INFO:teuthology.orchestra.run.smithi043.stdout:188978561072 2024-09-11T23:20:43.921 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T23:20:44.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.078+0000 7f9f8f44a640 1 -- 172.21.15.43:0/3323271878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f881082a0 msgr2=0x7f9f8810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:44.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.078+0000 7f9f8f44a640 1 --2- 172.21.15.43:0/3323271878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f881082a0 0x7f9f8810ec40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009cd0 tx=0x7f9f8002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.078+0000 7f9f8f44a640 1 -- 172.21.15.43:0/3323271878 shutdown_connections 2024-09-11T23:20:44.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.078+0000 7f9f8f44a640 1 --2- 172.21.15.43:0/3323271878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f881082a0 0x7f9f8810ec40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.078+0000 7f9f8f44a640 1 --2- 172.21.15.43:0/3323271878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f881036e0 0x7f9f88107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.078+0000 7f9f8f44a640 1 --2- 172.21.15.43:0/3323271878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f88102d30 0x7f9f88103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.078+0000 7f9f8f44a640 1 -- 172.21.15.43:0/3323271878 >> 172.21.15.43:0/3323271878 conn(0x7f9f880fc7d0 msgr2=0x7f9f880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:44.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.079+0000 7f9f8f44a640 1 -- 172.21.15.43:0/3323271878 shutdown_connections 2024-09-11T23:20:44.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.079+0000 7f9f8f44a640 1 -- 172.21.15.43:0/3323271878 wait complete. 2024-09-11T23:20:44.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.079+0000 7f9f8f44a640 1 Processor -- start 2024-09-11T23:20:44.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.079+0000 7f9f8f44a640 1 -- start start 2024-09-11T23:20:44.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.080+0000 7f9f8f44a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f88102d30 0x7f9f88100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.080+0000 7f9f8f44a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f881036e0 0x7f9f88100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.080+0000 7f9f8f44a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f881082a0 0x7f9f88101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.080+0000 7f9f8f44a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881120a0 con 0x7f9f88102d30 2024-09-11T23:20:44.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.080+0000 7f9f8f44a640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9f88111f20 con 0x7f9f881082a0 2024-09-11T23:20:44.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.080+0000 7f9f8f44a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9f88112220 con 0x7f9f881036e0 2024-09-11T23:20:44.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.080+0000 7f9f8d1e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f88102d30 0x7f9f88100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.080+0000 7f9f8d1e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f88102d30 0x7f9f88100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45928/0 (socket says 172.21.15.43:45928) 2024-09-11T23:20:44.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.080+0000 7f9f8d1e6640 1 -- 172.21.15.43:0/1331660862 learned_addr learned my addr 172.21.15.43:0/1331660862 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:44.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.080+0000 7f9f8c9e5640 1 --2- 172.21.15.43:0/1331660862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f881036e0 0x7f9f88100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.080+0000 7f9f8d9e7640 1 --2- 172.21.15.43:0/1331660862 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f881082a0 0x7f9f88101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.080+0000 7f9f8d1e6640 1 -- 172.21.15.43:0/1331660862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f881036e0 msgr2=0x7f9f88100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:44.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.080+0000 7f9f8d1e6640 1 --2- 172.21.15.43:0/1331660862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f881036e0 0x7f9f88100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.080+0000 7f9f8d1e6640 1 -- 172.21.15.43:0/1331660862 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f881082a0 msgr2=0x7f9f88101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:44.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.080+0000 7f9f8d1e6640 1 --2- 172.21.15.43:0/1331660862 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f881082a0 0x7f9f88101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.080+0000 7f9f8d1e6640 1 -- 172.21.15.43:0/1331660862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f881a6a00 con 0x7f9f88102d30 2024-09-11T23:20:44.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.080+0000 7f9f8c9e5640 1 --2- 172.21.15.43:0/1331660862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f881036e0 0x7f9f88100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:44.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.080+0000 7f9f8d9e7640 1 --2- 172.21.15.43:0/1331660862 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f881082a0 0x7f9f88101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:44.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.081+0000 7f9f8d1e6640 1 --2- 172.21.15.43:0/1331660862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f88102d30 0x7f9f88100990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400c890 tx=0x7f9f8400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:44.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.081+0000 7f9f7e7fc640 1 -- 172.21.15.43:0/1331660862 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84004220 con 0x7f9f88102d30 2024-09-11T23:20:44.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.081+0000 7f9f8f44a640 1 -- 172.21.15.43:0/1331660862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f881a6cf0 con 0x7f9f88102d30 2024-09-11T23:20:44.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.081+0000 7f9f7e7fc640 1 -- 172.21.15.43:0/1331660862 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f840043c0 con 0x7f9f88102d30 2024-09-11T23:20:44.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.081+0000 7f9f7e7fc640 1 -- 172.21.15.43:0/1331660862 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84004a00 con 0x7f9f88102d30 2024-09-11T23:20:44.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.081+0000 7f9f8f44a640 1 -- 172.21.15.43:0/1331660862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f881a7220 con 0x7f9f88102d30 2024-09-11T23:20:44.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.082+0000 7f9f8f44a640 1 -- 172.21.15.43:0/1331660862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9f881076b0 con 0x7f9f88102d30 2024-09-11T23:20:44.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.083+0000 7f9f7e7fc640 1 -- 172.21.15.43:0/1331660862 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9f840079b0 con 0x7f9f88102d30 2024-09-11T23:20:44.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.084+0000 7f9f7e7fc640 1 --2- 172.21.15.43:0/1331660862 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9f740a2c90 0x7f9f740a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.085+0000 7f9f8c9e5640 1 --2- 172.21.15.43:0/1331660862 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9f740a2c90 0x7f9f740a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.085+0000 7f9f7e7fc640 1 -- 172.21.15.43:0/1331660862 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f9f840cf2a0 con 0x7f9f88102d30 2024-09-11T23:20:44.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.085+0000 7f9f7e7fc640 1 --2- 172.21.15.43:0/1331660862 >> [v2:172.21.15.90:6824/59487041,v1:172.21.15.90:6825/59487041] conn(0x7f9f740ac4e0 0x7f9f740ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:44.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.085+0000 7f9f7e7fc640 1 -- 172.21.15.43:0/1331660862 --> [v2:172.21.15.90:6824/59487041,v1:172.21.15.90:6825/59487041] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9f740aefb0 con 0x7f9f740ac4e0 2024-09-11T23:20:44.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.085+0000 7f9f7e7fc640 1 -- 172.21.15.43:0/1331660862 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f9f840cf650 con 0x7f9f88102d30 2024-09-11T23:20:44.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.085+0000 7f9f8c9e5640 1 --2- 172.21.15.43:0/1331660862 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9f740a2c90 0x7f9f740a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9f88102420 tx=0x7f9f78005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:44.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.085+0000 7f9f8d9e7640 1 --2- 172.21.15.43:0/1331660862 >> [v2:172.21.15.90:6824/59487041,v1:172.21.15.90:6825/59487041] conn(0x7f9f740ac4e0 0x7f9f740ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:44.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.086+0000 7f9f8d9e7640 1 --2- 172.21.15.43:0/1331660862 >> [v2:172.21.15.90:6824/59487041,v1:172.21.15.90:6825/59487041] conn(0x7f9f740ac4e0 0x7f9f740ae900 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:44.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.087+0000 7f9f7e7fc640 1 -- 172.21.15.43:0/1331660862 <== osd.10 v2:172.21.15.90:6824/59487041 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f740aefb0 con 0x7f9f740ac4e0 2024-09-11T23:20:44.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.115+0000 7f9f8f44a640 1 -- 172.21.15.43:0/1331660862 --> [v2:172.21.15.90:6824/59487041,v1:172.21.15.90:6825/59487041] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9f8806a850 con 0x7f9f740ac4e0 2024-09-11T23:20:44.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.115+0000 7f9f7e7fc640 1 -- 172.21.15.43:0/1331660862 <== osd.10 v2:172.21.15.90:6824/59487041 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9f8806a850 con 0x7f9f740ac4e0 2024-09-11T23:20:44.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.116+0000 7f9f8f44a640 1 -- 172.21.15.43:0/1331660862 >> [v2:172.21.15.90:6824/59487041,v1:172.21.15.90:6825/59487041] conn(0x7f9f740ac4e0 msgr2=0x7f9f740ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:44.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.116+0000 7f9f8f44a640 1 --2- 172.21.15.43:0/1331660862 >> [v2:172.21.15.90:6824/59487041,v1:172.21.15.90:6825/59487041] conn(0x7f9f740ac4e0 0x7f9f740ae900 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:44.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.116+0000 7f9f8f44a640 1 -- 172.21.15.43:0/1331660862 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9f740a2c90 msgr2=0x7f9f740a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:44.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.116+0000 7f9f8f44a640 1 --2- 172.21.15.43:0/1331660862 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9f740a2c90 0x7f9f740a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9f88102420 tx=0x7f9f78005f90 comp rx=0 tx=0).stop 2024-09-11T23:20:44.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.117+0000 7f9f8f44a640 1 -- 172.21.15.43:0/1331660862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f88102d30 msgr2=0x7f9f88100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:44.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.117+0000 7f9f8f44a640 1 --2- 172.21.15.43:0/1331660862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f88102d30 0x7f9f88100990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400c890 tx=0x7f9f8400cd50 comp rx=0 tx=0).stop 2024-09-11T23:20:44.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.117+0000 7f9f8d9e7640 1 -- 172.21.15.43:0/1331660862 reap_dead start 2024-09-11T23:20:44.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.117+0000 7f9f8f44a640 1 -- 172.21.15.43:0/1331660862 shutdown_connections 2024-09-11T23:20:44.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.117+0000 7f9f8f44a640 1 -- 172.21.15.43:0/1331660862 >> 172.21.15.43:0/1331660862 conn(0x7f9f880fc7d0 msgr2=0x7f9f88105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:44.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.117+0000 7f9f8f44a640 1 -- 172.21.15.43:0/1331660862 shutdown_connections 2024-09-11T23:20:44.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.117+0000 7f9f8f44a640 1 -- 172.21.15.43:0/1331660862 wait complete. 2024-09-11T23:20:44.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:44 smithi043 ceph-mon[30950]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-11T23:20:44.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:44 smithi090 ceph-mon[35130]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-11T23:20:44.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:44 smithi097 ceph-mon[35562]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-11T23:20:45.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.647+0000 7fc3bed6e640 1 -- 172.21.15.43:0/680728498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b8101de0 msgr2=0x7fc3b81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:45.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.647+0000 7fc3bed6e640 1 --2- 172.21.15.43:0/680728498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b8101de0 0x7fc3b81021c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009900 tx=0x7fc3ac02a410 comp rx=0 tx=0).stop 2024-09-11T23:20:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.647+0000 7fc3bed6e640 1 -- 172.21.15.43:0/680728498 shutdown_connections 2024-09-11T23:20:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.647+0000 7fc3bed6e640 1 --2- 172.21.15.43:0/680728498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3b8106f10 0x7fc3b810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.647+0000 7fc3bed6e640 1 --2- 172.21.15.43:0/680728498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3b8102790 0x7fc3b81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.647+0000 7fc3bed6e640 1 --2- 172.21.15.43:0/680728498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b8101de0 0x7fc3b81021c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.647+0000 7fc3bed6e640 1 -- 172.21.15.43:0/680728498 >> 172.21.15.43:0/680728498 conn(0x7fc3b80fd8c0 msgr2=0x7fc3b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.647+0000 7fc3bed6e640 1 -- 172.21.15.43:0/680728498 shutdown_connections 2024-09-11T23:20:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.647+0000 7fc3bed6e640 1 -- 172.21.15.43:0/680728498 wait complete. 2024-09-11T23:20:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.648+0000 7fc3bed6e640 1 Processor -- start 2024-09-11T23:20:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.648+0000 7fc3bed6e640 1 -- start start 2024-09-11T23:20:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3bed6e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3b8101de0 0x7fc3b8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3bed6e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3b8102790 0x7fc3b8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3bed6e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b8106f10 0x7fc3b819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3bed6e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8111f70 con 0x7fc3b8106f10 2024-09-11T23:20:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3bed6e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8111df0 con 0x7fc3b8101de0 2024-09-11T23:20:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3bed6e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81120f0 con 0x7fc3b8102790 2024-09-11T23:20:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3bd30b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b8106f10 0x7fc3b819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3bd30b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b8106f10 0x7fc3b819cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45962/0 (socket says 172.21.15.43:45962) 2024-09-11T23:20:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3bd30b640 1 -- 172.21.15.43:0/3178611931 learned_addr learned my addr 172.21.15.43:0/3178611931 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3bcb0a640 1 --2- 172.21.15.43:0/3178611931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3b8101de0 0x7fc3b8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3b7fff640 1 --2- 172.21.15.43:0/3178611931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3b8102790 0x7fc3b8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3bd30b640 1 -- 172.21.15.43:0/3178611931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3b8102790 msgr2=0x7fc3b8198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3bd30b640 1 --2- 172.21.15.43:0/3178611931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3b8102790 0x7fc3b8198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3bd30b640 1 -- 172.21.15.43:0/3178611931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3b8101de0 msgr2=0x7fc3b8198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3bd30b640 1 --2- 172.21.15.43:0/3178611931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3b8101de0 0x7fc3b8198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:45.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3bd30b640 1 -- 172.21.15.43:0/3178611931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b819d3f0 con 0x7fc3b8106f10 2024-09-11T23:20:45.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3bcb0a640 1 --2- 172.21.15.43:0/3178611931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3b8101de0 0x7fc3b8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:45.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3bd30b640 1 --2- 172.21.15.43:0/3178611931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b8106f10 0x7fc3b819cd10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc3b000eaa0 tx=0x7fc3b000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:45.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3b5ffb640 1 -- 172.21.15.43:0/3178611931 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b000cdb0 con 0x7fc3b8106f10 2024-09-11T23:20:45.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3b5ffb640 1 -- 172.21.15.43:0/3178611931 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3b0005020 con 0x7fc3b8106f10 2024-09-11T23:20:45.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3bed6e640 1 -- 172.21.15.43:0/3178611931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b819d6e0 con 0x7fc3b8106f10 2024-09-11T23:20:45.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.649+0000 7fc3b5ffb640 1 -- 172.21.15.43:0/3178611931 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b00116b0 con 0x7fc3b8106f10 2024-09-11T23:20:45.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.650+0000 7fc3bed6e640 1 -- 172.21.15.43:0/3178611931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b81a4fc0 con 0x7fc3b8106f10 2024-09-11T23:20:45.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.650+0000 7fc3bed6e640 1 -- 172.21.15.43:0/3178611931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc3b8062980 con 0x7fc3b8106f10 2024-09-11T23:20:45.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.652+0000 7fc3b5ffb640 1 -- 172.21.15.43:0/3178611931 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc3b0005560 con 0x7fc3b8106f10 2024-09-11T23:20:45.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.652+0000 7fc3b5ffb640 1 --2- 172.21.15.43:0/3178611931 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fc3a80a2c10 0x7fc3a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:45.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.652+0000 7fc3bcb0a640 1 --2- 172.21.15.43:0/3178611931 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fc3a80a2c10 0x7fc3a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:45.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.652+0000 7fc3b5ffb640 1 -- 172.21.15.43:0/3178611931 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fc3b00cf6e0 con 0x7fc3b8106f10 2024-09-11T23:20:45.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.653+0000 7fc3b5ffb640 1 --2- 172.21.15.43:0/3178611931 >> [v2:172.21.15.90:6800/1825149246,v1:172.21.15.90:6801/1825149246] conn(0x7fc3a80ac460 0x7fc3a80ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:45.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.653+0000 7fc3b5ffb640 1 -- 172.21.15.43:0/3178611931 --> [v2:172.21.15.90:6800/1825149246,v1:172.21.15.90:6801/1825149246] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc3b0014ec0 con 0x7fc3a80ac460 2024-09-11T23:20:45.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.653+0000 7fc3b5ffb640 1 -- 172.21.15.43:0/3178611931 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fc3b00cfbc0 con 0x7fc3b8106f10 2024-09-11T23:20:45.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.653+0000 7fc3bcb0a640 1 --2- 172.21.15.43:0/3178611931 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fc3a80a2c10 0x7fc3a80a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac005ab0 tx=0x7fc3ac005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:45.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.653+0000 7fc3b7fff640 1 --2- 172.21.15.43:0/3178611931 >> [v2:172.21.15.90:6800/1825149246,v1:172.21.15.90:6801/1825149246] conn(0x7fc3a80ac460 0x7fc3a80ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:45.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.654+0000 7fc3b7fff640 1 --2- 172.21.15.43:0/3178611931 >> [v2:172.21.15.90:6800/1825149246,v1:172.21.15.90:6801/1825149246] conn(0x7fc3a80ac460 0x7fc3a80ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.654+0000 7fc3b5ffb640 1 -- 172.21.15.43:0/3178611931 <== osd.1 v2:172.21.15.90:6800/1825149246 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3b0014ec0 con 0x7fc3a80ac460 2024-09-11T23:20:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.671+0000 7fc3bed6e640 1 -- 172.21.15.43:0/3178611931 --> [v2:172.21.15.90:6800/1825149246,v1:172.21.15.90:6801/1825149246] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc3b8062e00 con 0x7fc3a80ac460 2024-09-11T23:20:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.672+0000 7fc3b5ffb640 1 -- 172.21.15.43:0/3178611931 <== osd.1 v2:172.21.15.90:6800/1825149246 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc3b8062e00 con 0x7fc3a80ac460 2024-09-11T23:20:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.672+0000 7fc3bed6e640 1 -- 172.21.15.43:0/3178611931 >> [v2:172.21.15.90:6800/1825149246,v1:172.21.15.90:6801/1825149246] conn(0x7fc3a80ac460 msgr2=0x7fc3a80ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.672+0000 7fc3bed6e640 1 --2- 172.21.15.43:0/3178611931 >> [v2:172.21.15.90:6800/1825149246,v1:172.21.15.90:6801/1825149246] conn(0x7fc3a80ac460 0x7fc3a80ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.672+0000 7fc3bed6e640 1 -- 172.21.15.43:0/3178611931 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fc3a80a2c10 msgr2=0x7fc3a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.672+0000 7fc3bed6e640 1 --2- 172.21.15.43:0/3178611931 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fc3a80a2c10 0x7fc3a80a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac005ab0 tx=0x7fc3ac005a20 comp rx=0 tx=0).stop 2024-09-11T23:20:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.672+0000 7fc3bed6e640 1 -- 172.21.15.43:0/3178611931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b8106f10 msgr2=0x7fc3b819cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.672+0000 7fc3bed6e640 1 --2- 172.21.15.43:0/3178611931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b8106f10 0x7fc3b819cd10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc3b000eaa0 tx=0x7fc3b000ef60 comp rx=0 tx=0).stop 2024-09-11T23:20:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.672+0000 7fc3bd30b640 1 -- 172.21.15.43:0/3178611931 reap_dead start 2024-09-11T23:20:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.672+0000 7fc3bed6e640 1 -- 172.21.15.43:0/3178611931 shutdown_connections 2024-09-11T23:20:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.672+0000 7fc3bed6e640 1 -- 172.21.15.43:0/3178611931 >> 172.21.15.43:0/3178611931 conn(0x7fc3b80fd8c0 msgr2=0x7fc3b80fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:45.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.672+0000 7fc3bed6e640 1 -- 172.21.15.43:0/3178611931 shutdown_connections 2024-09-11T23:20:45.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.672+0000 7fc3bed6e640 1 -- 172.21.15.43:0/3178611931 wait complete. 2024-09-11T23:20:45.966 INFO:teuthology.orchestra.run.smithi043.stdout:34359738447 2024-09-11T23:20:45.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T23:20:46.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.136+0000 7f8d9c8cb640 1 -- 172.21.15.43:0/1765799758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d9810e3b0 msgr2=0x7f8d98110800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:46.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.136+0000 7f8d9c8cb640 1 --2- 172.21.15.43:0/1765799758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d9810e3b0 0x7f8d98110800 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009ca0 tx=0x7f8d9002a950 comp rx=0 tx=0).stop 2024-09-11T23:20:46.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.136+0000 7f8d9c8cb640 1 -- 172.21.15.43:0/1765799758 shutdown_connections 2024-09-11T23:20:46.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.136+0000 7f8d9c8cb640 1 --2- 172.21.15.43:0/1765799758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d9810e3b0 0x7f8d98110800 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.136+0000 7f8d9c8cb640 1 --2- 172.21.15.43:0/1765799758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d98074f40 0x7f8d9810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.136+0000 7f8d9c8cb640 1 --2- 172.21.15.43:0/1765799758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d98074620 0x7f8d98074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.136+0000 7f8d9c8cb640 1 -- 172.21.15.43:0/1765799758 >> 172.21.15.43:0/1765799758 conn(0x7f8d980fd8b0 msgr2=0x7f8d980ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:46.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.136+0000 7f8d9c8cb640 1 -- 172.21.15.43:0/1765799758 shutdown_connections 2024-09-11T23:20:46.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.136+0000 7f8d9c8cb640 1 -- 172.21.15.43:0/1765799758 wait complete. 2024-09-11T23:20:46.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.137+0000 7f8d9c8cb640 1 Processor -- start 2024-09-11T23:20:46.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.137+0000 7f8d9c8cb640 1 -- start start 2024-09-11T23:20:46.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.137+0000 7f8d9c8cb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d98074620 0x7f8d9819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.137+0000 7f8d9c8cb640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d98074f40 0x7f8d9819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.137+0000 7f8d9c8cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d9810e3b0 0x7f8d9819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.137+0000 7f8d9c8cb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8d981141a0 con 0x7f8d9810e3b0 2024-09-11T23:20:46.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.137+0000 7f8d9c8cb640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8d98114020 con 0x7f8d98074f40 2024-09-11T23:20:46.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.137+0000 7f8d9c8cb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8d98114320 con 0x7f8d98074620 2024-09-11T23:20:46.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.138+0000 7f8d96d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d9810e3b0 0x7f8d9819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.138+0000 7f8d96d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d9810e3b0 0x7f8d9819eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45972/0 (socket says 172.21.15.43:45972) 2024-09-11T23:20:46.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.138+0000 7f8d96d9d640 1 -- 172.21.15.43:0/3884123041 learned_addr learned my addr 172.21.15.43:0/3884123041 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:46.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.138+0000 7f8d95d9b640 1 --2- 172.21.15.43:0/3884123041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d98074f40 0x7f8d9819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.138+0000 7f8d9659c640 1 --2- 172.21.15.43:0/3884123041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d98074620 0x7f8d9819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.138+0000 7f8d96d9d640 1 -- 172.21.15.43:0/3884123041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d98074620 msgr2=0x7f8d9819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:46.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.138+0000 7f8d96d9d640 1 --2- 172.21.15.43:0/3884123041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d98074620 0x7f8d9819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.138+0000 7f8d96d9d640 1 -- 172.21.15.43:0/3884123041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d98074f40 msgr2=0x7f8d9819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:46.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.138+0000 7f8d96d9d640 1 --2- 172.21.15.43:0/3884123041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d98074f40 0x7f8d9819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.138+0000 7f8d96d9d640 1 -- 172.21.15.43:0/3884123041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d9819f5c0 con 0x7f8d9810e3b0 2024-09-11T23:20:46.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.138+0000 7f8d96d9d640 1 --2- 172.21.15.43:0/3884123041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d9810e3b0 0x7f8d9819eee0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8d90006010 tx=0x7f8d9002f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:46.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.138+0000 7f8d877fe640 1 -- 172.21.15.43:0/3884123041 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d90034040 con 0x7f8d9810e3b0 2024-09-11T23:20:46.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.138+0000 7f8d877fe640 1 -- 172.21.15.43:0/3884123041 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d90003e00 con 0x7f8d9810e3b0 2024-09-11T23:20:46.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.138+0000 7f8d9c8cb640 1 -- 172.21.15.43:0/3884123041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d9819f850 con 0x7f8d9810e3b0 2024-09-11T23:20:46.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.138+0000 7f8d877fe640 1 -- 172.21.15.43:0/3884123041 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d90045480 con 0x7f8d9810e3b0 2024-09-11T23:20:46.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.138+0000 7f8d9c8cb640 1 -- 172.21.15.43:0/3884123041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d981a7190 con 0x7f8d9810e3b0 2024-09-11T23:20:46.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.139+0000 7f8d9c8cb640 1 -- 172.21.15.43:0/3884123041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8d980ff990 con 0x7f8d9810e3b0 2024-09-11T23:20:46.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.141+0000 7f8d877fe640 1 -- 172.21.15.43:0/3884123041 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8d90037020 con 0x7f8d9810e3b0 2024-09-11T23:20:46.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.141+0000 7f8d877fe640 1 --2- 172.21.15.43:0/3884123041 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f8d800a2c10 0x7f8d800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.141+0000 7f8d9659c640 1 --2- 172.21.15.43:0/3884123041 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f8d800a2c10 0x7f8d800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.141+0000 7f8d877fe640 1 -- 172.21.15.43:0/3884123041 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f8d900f0150 con 0x7f8d9810e3b0 2024-09-11T23:20:46.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.142+0000 7f8d877fe640 1 --2- 172.21.15.43:0/3884123041 >> [v2:172.21.15.97:6816/1707327301,v1:172.21.15.97:6817/1707327301] conn(0x7f8d800ac460 0x7f8d800ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:46.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.142+0000 7f8d877fe640 1 -- 172.21.15.43:0/3884123041 --> [v2:172.21.15.97:6816/1707327301,v1:172.21.15.97:6817/1707327301] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8d9003dec0 con 0x7f8d800ac460 2024-09-11T23:20:46.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.142+0000 7f8d877fe640 1 -- 172.21.15.43:0/3884123041 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f8d900f0630 con 0x7f8d9810e3b0 2024-09-11T23:20:46.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.142+0000 7f8d9659c640 1 --2- 172.21.15.43:0/3884123041 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f8d800a2c10 0x7f8d800a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8d88005e10 tx=0x7f8d88005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:46.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.142+0000 7f8d95d9b640 1 --2- 172.21.15.43:0/3884123041 >> [v2:172.21.15.97:6816/1707327301,v1:172.21.15.97:6817/1707327301] conn(0x7f8d800ac460 0x7f8d800ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:46.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.142+0000 7f8d95d9b640 1 --2- 172.21.15.43:0/3884123041 >> [v2:172.21.15.97:6816/1707327301,v1:172.21.15.97:6817/1707327301] conn(0x7f8d800ac460 0x7f8d800ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:46.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.143+0000 7f8d877fe640 1 -- 172.21.15.43:0/3884123041 <== osd.8 v2:172.21.15.97:6816/1707327301 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d9003dec0 con 0x7f8d800ac460 2024-09-11T23:20:46.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.157+0000 7f8d9c8cb640 1 -- 172.21.15.43:0/3884123041 --> [v2:172.21.15.97:6816/1707327301,v1:172.21.15.97:6817/1707327301] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8d98075cd0 con 0x7f8d800ac460 2024-09-11T23:20:46.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.158+0000 7f8d877fe640 1 -- 172.21.15.43:0/3884123041 <== osd.8 v2:172.21.15.97:6816/1707327301 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8d98075cd0 con 0x7f8d800ac460 2024-09-11T23:20:46.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.158+0000 7f8d9c8cb640 1 -- 172.21.15.43:0/3884123041 >> [v2:172.21.15.97:6816/1707327301,v1:172.21.15.97:6817/1707327301] conn(0x7f8d800ac460 msgr2=0x7f8d800ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:46.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.158+0000 7f8d9c8cb640 1 --2- 172.21.15.43:0/3884123041 >> [v2:172.21.15.97:6816/1707327301,v1:172.21.15.97:6817/1707327301] conn(0x7f8d800ac460 0x7f8d800ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.158+0000 7f8d9c8cb640 1 -- 172.21.15.43:0/3884123041 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f8d800a2c10 msgr2=0x7f8d800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:46.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.158+0000 7f8d9c8cb640 1 --2- 172.21.15.43:0/3884123041 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f8d800a2c10 0x7f8d800a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8d88005e10 tx=0x7f8d88005da0 comp rx=0 tx=0).stop 2024-09-11T23:20:46.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.158+0000 7f8d9c8cb640 1 -- 172.21.15.43:0/3884123041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d9810e3b0 msgr2=0x7f8d9819eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:46.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.158+0000 7f8d9c8cb640 1 --2- 172.21.15.43:0/3884123041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d9810e3b0 0x7f8d9819eee0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8d90006010 tx=0x7f8d9002f710 comp rx=0 tx=0).stop 2024-09-11T23:20:46.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.158+0000 7f8d96d9d640 1 -- 172.21.15.43:0/3884123041 reap_dead start 2024-09-11T23:20:46.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.158+0000 7f8d9c8cb640 1 -- 172.21.15.43:0/3884123041 shutdown_connections 2024-09-11T23:20:46.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.158+0000 7f8d9c8cb640 1 -- 172.21.15.43:0/3884123041 >> 172.21.15.43:0/3884123041 conn(0x7f8d980fd8b0 msgr2=0x7f8d980fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:46.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.158+0000 7f8d9c8cb640 1 -- 172.21.15.43:0/3884123041 shutdown_connections 2024-09-11T23:20:46.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:46.159+0000 7f8d9c8cb640 1 -- 172.21.15.43:0/3884123041 wait complete. 2024-09-11T23:20:46.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:46 smithi043 ceph-mon[30950]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-09-11T23:20:46.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:46 smithi090 ceph-mon[35130]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-09-11T23:20:46.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:46 smithi097 ceph-mon[35562]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-09-11T23:20:46.878 INFO:teuthology.orchestra.run.smithi043.stdout:240518168614 2024-09-11T23:20:46.878 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-11T23:20:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.063+0000 7f3853b52640 1 -- 172.21.15.43:0/2774068278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f384c100100 msgr2=0x7f384c100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.063+0000 7f3853b52640 1 --2- 172.21.15.43:0/2774068278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f384c100100 0x7f384c100560 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f383c009900 tx=0x7f383c02a410 comp rx=0 tx=0).stop 2024-09-11T23:20:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.063+0000 7f3853b52640 1 -- 172.21.15.43:0/2774068278 shutdown_connections 2024-09-11T23:20:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.063+0000 7f3853b52640 1 --2- 172.21.15.43:0/2774068278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f384c100aa0 0x7f384c10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.063+0000 7f3853b52640 1 --2- 172.21.15.43:0/2774068278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f384c100100 0x7f384c100560 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.063+0000 7f3853b52640 1 --2- 172.21.15.43:0/2774068278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f384c106b80 0x7f384c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.063+0000 7f3853b52640 1 -- 172.21.15.43:0/2774068278 >> 172.21.15.43:0/2774068278 conn(0x7f384c077230 msgr2=0x7f384c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.063+0000 7f3853b52640 1 -- 172.21.15.43:0/2774068278 shutdown_connections 2024-09-11T23:20:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.063+0000 7f3853b52640 1 -- 172.21.15.43:0/2774068278 wait complete. 2024-09-11T23:20:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.064+0000 7f3853b52640 1 Processor -- start 2024-09-11T23:20:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.064+0000 7f3853b52640 1 -- start start 2024-09-11T23:20:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.064+0000 7f3853b52640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f384c100100 0x7f384c1a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.064+0000 7f3853b52640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f384c100aa0 0x7f384c1a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.064+0000 7f3853b52640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f384c106b80 0x7f384c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.064+0000 7f3853b52640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f384c112090 con 0x7f384c100aa0 2024-09-11T23:20:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.064+0000 7f3853b52640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f384c111f10 con 0x7f384c106b80 2024-09-11T23:20:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.064+0000 7f3853b52640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f384c112210 con 0x7f384c100100 2024-09-11T23:20:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.064+0000 7f38510ed640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f384c100aa0 0x7f384c1a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.064+0000 7f38510ed640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f384c100aa0 0x7f384c1a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45998/0 (socket says 172.21.15.43:45998) 2024-09-11T23:20:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.064+0000 7f38510ed640 1 -- 172.21.15.43:0/1406883493 learned_addr learned my addr 172.21.15.43:0/1406883493 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.064+0000 7f38518ee640 1 --2- 172.21.15.43:0/1406883493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f384c100100 0x7f384c1a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.064+0000 7f38520ef640 1 --2- 172.21.15.43:0/1406883493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f384c106b80 0x7f384c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:47.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.065+0000 7f38510ed640 1 -- 172.21.15.43:0/1406883493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f384c100100 msgr2=0x7f384c1a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:47.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.065+0000 7f38510ed640 1 --2- 172.21.15.43:0/1406883493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f384c100100 0x7f384c1a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:47.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.065+0000 7f38510ed640 1 -- 172.21.15.43:0/1406883493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f384c106b80 msgr2=0x7f384c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:47.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.065+0000 7f38510ed640 1 --2- 172.21.15.43:0/1406883493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f384c106b80 0x7f384c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:47.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.065+0000 7f38510ed640 1 -- 172.21.15.43:0/1406883493 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384c19acc0 con 0x7f384c100aa0 2024-09-11T23:20:47.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.065+0000 7f38520ef640 1 --2- 172.21.15.43:0/1406883493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f384c106b80 0x7f384c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:20:47.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.065+0000 7f38510ed640 1 --2- 172.21.15.43:0/1406883493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f384c100aa0 0x7f384c1a0910 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f383c006010 tx=0x7f383c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:47.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.065+0000 7f3842ffd640 1 -- 172.21.15.43:0/1406883493 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c03f030 con 0x7f384c100aa0 2024-09-11T23:20:47.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.065+0000 7f3853b52640 1 -- 172.21.15.43:0/1406883493 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f384c19af50 con 0x7f384c100aa0 2024-09-11T23:20:47.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.065+0000 7f3842ffd640 1 -- 172.21.15.43:0/1406883493 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383c003ce0 con 0x7f384c100aa0 2024-09-11T23:20:47.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.065+0000 7f3842ffd640 1 -- 172.21.15.43:0/1406883493 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c035070 con 0x7f384c100aa0 2024-09-11T23:20:47.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.065+0000 7f3853b52640 1 -- 172.21.15.43:0/1406883493 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f384c1a72a0 con 0x7f384c100aa0 2024-09-11T23:20:47.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.066+0000 7f3853b52640 1 -- 172.21.15.43:0/1406883493 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f384c1024f0 con 0x7f384c100aa0 2024-09-11T23:20:47.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.068+0000 7f3842ffd640 1 -- 172.21.15.43:0/1406883493 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f383c042030 con 0x7f384c100aa0 2024-09-11T23:20:47.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.068+0000 7f3842ffd640 1 --2- 172.21.15.43:0/1406883493 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f38380a2c10 0x7f38380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:47.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.068+0000 7f38518ee640 1 --2- 172.21.15.43:0/1406883493 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f38380a2c10 0x7f38380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:47.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.068+0000 7f3842ffd640 1 -- 172.21.15.43:0/1406883493 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f383c0efd00 con 0x7f384c100aa0 2024-09-11T23:20:47.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.069+0000 7f3842ffd640 1 --2- 172.21.15.43:0/1406883493 >> [v2:172.21.15.97:6808/1540039543,v1:172.21.15.97:6809/1540039543] conn(0x7f38380ac460 0x7f38380ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:47.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.069+0000 7f3842ffd640 1 -- 172.21.15.43:0/1406883493 --> [v2:172.21.15.97:6808/1540039543,v1:172.21.15.97:6809/1540039543] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f383c03be50 con 0x7f38380ac460 2024-09-11T23:20:47.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.069+0000 7f3842ffd640 1 -- 172.21.15.43:0/1406883493 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f383c0f01e0 con 0x7f384c100aa0 2024-09-11T23:20:47.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.069+0000 7f38518ee640 1 --2- 172.21.15.43:0/1406883493 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f38380a2c10 0x7f38380a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3848009890 tx=0x7f3848008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:47.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.069+0000 7f38520ef640 1 --2- 172.21.15.43:0/1406883493 >> [v2:172.21.15.97:6808/1540039543,v1:172.21.15.97:6809/1540039543] conn(0x7f38380ac460 0x7f38380ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:47.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.069+0000 7f38520ef640 1 --2- 172.21.15.43:0/1406883493 >> [v2:172.21.15.97:6808/1540039543,v1:172.21.15.97:6809/1540039543] conn(0x7f38380ac460 0x7f38380ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:47.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.070+0000 7f3842ffd640 1 -- 172.21.15.43:0/1406883493 <== osd.5 v2:172.21.15.97:6808/1540039543 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f383c03be50 con 0x7f38380ac460 2024-09-11T23:20:47.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.084+0000 7f3853b52640 1 -- 172.21.15.43:0/1406883493 --> [v2:172.21.15.97:6808/1540039543,v1:172.21.15.97:6809/1540039543] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f384c103420 con 0x7f38380ac460 2024-09-11T23:20:47.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.085+0000 7f3842ffd640 1 -- 172.21.15.43:0/1406883493 <== osd.5 v2:172.21.15.97:6808/1540039543 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f384c103420 con 0x7f38380ac460 2024-09-11T23:20:47.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.085+0000 7f3853b52640 1 -- 172.21.15.43:0/1406883493 >> [v2:172.21.15.97:6808/1540039543,v1:172.21.15.97:6809/1540039543] conn(0x7f38380ac460 msgr2=0x7f38380ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:47.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.085+0000 7f3853b52640 1 --2- 172.21.15.43:0/1406883493 >> [v2:172.21.15.97:6808/1540039543,v1:172.21.15.97:6809/1540039543] conn(0x7f38380ac460 0x7f38380ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:47.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.085+0000 7f3853b52640 1 -- 172.21.15.43:0/1406883493 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f38380a2c10 msgr2=0x7f38380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:47.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.085+0000 7f3853b52640 1 --2- 172.21.15.43:0/1406883493 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f38380a2c10 0x7f38380a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3848009890 tx=0x7f3848008040 comp rx=0 tx=0).stop 2024-09-11T23:20:47.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.085+0000 7f3853b52640 1 -- 172.21.15.43:0/1406883493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f384c100aa0 msgr2=0x7f384c1a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:47.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.085+0000 7f3853b52640 1 --2- 172.21.15.43:0/1406883493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f384c100aa0 0x7f384c1a0910 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f383c006010 tx=0x7f383c0042f0 comp rx=0 tx=0).stop 2024-09-11T23:20:47.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.085+0000 7f38520ef640 1 -- 172.21.15.43:0/1406883493 reap_dead start 2024-09-11T23:20:47.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.085+0000 7f3853b52640 1 -- 172.21.15.43:0/1406883493 shutdown_connections 2024-09-11T23:20:47.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.085+0000 7f3853b52640 1 -- 172.21.15.43:0/1406883493 >> 172.21.15.43:0/1406883493 conn(0x7f384c077230 msgr2=0x7f384c104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:47.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.085+0000 7f3853b52640 1 -- 172.21.15.43:0/1406883493 shutdown_connections 2024-09-11T23:20:47.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.085+0000 7f3853b52640 1 -- 172.21.15.43:0/1406883493 wait complete. 2024-09-11T23:20:47.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.178+0000 7f35a2513640 1 -- 172.21.15.43:0/1314221831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f359c1036e0 msgr2=0x7f359c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:47.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.178+0000 7f35a2513640 1 --2- 172.21.15.43:0/1314221831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f359c1036e0 0x7f359c107b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3590009930 tx=0x7f359002a420 comp rx=0 tx=0).stop 2024-09-11T23:20:47.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.178+0000 7f35a2513640 1 -- 172.21.15.43:0/1314221831 shutdown_connections 2024-09-11T23:20:47.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.178+0000 7f35a2513640 1 --2- 172.21.15.43:0/1314221831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f359c1082a0 0x7f359c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:47.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.178+0000 7f35a2513640 1 --2- 172.21.15.43:0/1314221831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f359c1036e0 0x7f359c107b90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:47.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.178+0000 7f35a2513640 1 --2- 172.21.15.43:0/1314221831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f359c102d30 0x7f359c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:47.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.178+0000 7f35a2513640 1 -- 172.21.15.43:0/1314221831 >> 172.21.15.43:0/1314221831 conn(0x7f359c0fc7d0 msgr2=0x7f359c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:47.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.179+0000 7f35a2513640 1 -- 172.21.15.43:0/1314221831 shutdown_connections 2024-09-11T23:20:47.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.179+0000 7f35a2513640 1 -- 172.21.15.43:0/1314221831 wait complete. 2024-09-11T23:20:47.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.179+0000 7f35a2513640 1 Processor -- start 2024-09-11T23:20:47.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.179+0000 7f35a2513640 1 -- start start 2024-09-11T23:20:47.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.180+0000 7f35a2513640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f359c102d30 0x7f359c073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:47.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.180+0000 7f35a2513640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f359c1036e0 0x7f359c06cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:47.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.180+0000 7f35a2513640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f359c1082a0 0x7f359c06d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:47.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.180+0000 7f35a2513640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1120a0 con 0x7f359c1082a0 2024-09-11T23:20:47.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.180+0000 7f35a2513640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f359c111f20 con 0x7f359c1036e0 2024-09-11T23:20:47.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.180+0000 7f35a2513640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f359c112220 con 0x7f359c102d30 2024-09-11T23:20:47.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.180+0000 7f35a0ab0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f359c1082a0 0x7f359c06d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:47.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.180+0000 7f35a0ab0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f359c1082a0 0x7f359c06d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45940/0 (socket says 172.21.15.43:45940) 2024-09-11T23:20:47.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.180+0000 7f35a0ab0640 1 -- 172.21.15.43:0/2259215743 learned_addr learned my addr 172.21.15.43:0/2259215743 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:47.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.180+0000 7f359b7fe640 1 --2- 172.21.15.43:0/2259215743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f359c1036e0 0x7f359c06cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:47.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.180+0000 7f359bfff640 1 --2- 172.21.15.43:0/2259215743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f359c102d30 0x7f359c073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.180+0000 7f35a0ab0640 1 -- 172.21.15.43:0/2259215743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f359c102d30 msgr2=0x7f359c073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.180+0000 7f35a0ab0640 1 --2- 172.21.15.43:0/2259215743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f359c102d30 0x7f359c073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.180+0000 7f35a0ab0640 1 -- 172.21.15.43:0/2259215743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f359c1036e0 msgr2=0x7f359c06cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.180+0000 7f35a0ab0640 1 --2- 172.21.15.43:0/2259215743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f359c1036e0 0x7f359c06cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.180+0000 7f35a0ab0640 1 -- 172.21.15.43:0/2259215743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c06d6a0 con 0x7f359c1082a0 2024-09-11T23:20:47.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.180+0000 7f359bfff640 1 --2- 172.21.15.43:0/2259215743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f359c102d30 0x7f359c073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:47.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.180+0000 7f35a0ab0640 1 --2- 172.21.15.43:0/2259215743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f359c1082a0 0x7f359c06d060 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f359400cad0 tx=0x7f359400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:47.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.180+0000 7f35997fa640 1 -- 172.21.15.43:0/2259215743 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359400ae20 con 0x7f359c1082a0 2024-09-11T23:20:47.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.180+0000 7f35a2513640 1 -- 172.21.15.43:0/2259215743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359c071570 con 0x7f359c1082a0 2024-09-11T23:20:47.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.181+0000 7f35997fa640 1 -- 172.21.15.43:0/2259215743 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3594005020 con 0x7f359c1082a0 2024-09-11T23:20:47.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.181+0000 7f35997fa640 1 -- 172.21.15.43:0/2259215743 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359400f6f0 con 0x7f359c1082a0 2024-09-11T23:20:47.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.181+0000 7f35a2513640 1 -- 172.21.15.43:0/2259215743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359c071a30 con 0x7f359c1082a0 2024-09-11T23:20:47.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.182+0000 7f35a2513640 1 -- 172.21.15.43:0/2259215743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f359c071f50 con 0x7f359c1082a0 2024-09-11T23:20:47.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.183+0000 7f35997fa640 1 -- 172.21.15.43:0/2259215743 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f35940051d0 con 0x7f359c1082a0 2024-09-11T23:20:47.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.184+0000 7f35997fa640 1 --2- 172.21.15.43:0/2259215743 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f35840a2c10 0x7f35840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:47.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.184+0000 7f359bfff640 1 --2- 172.21.15.43:0/2259215743 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f35840a2c10 0x7f35840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:47.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.184+0000 7f35997fa640 1 -- 172.21.15.43:0/2259215743 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f35940cf510 con 0x7f359c1082a0 2024-09-11T23:20:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.184+0000 7f35997fa640 1 --2- 172.21.15.43:0/2259215743 >> [v2:172.21.15.97:6824/194699315,v1:172.21.15.97:6825/194699315] conn(0x7f35840ac460 0x7f35840ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.184+0000 7f35997fa640 1 -- 172.21.15.43:0/2259215743 --> [v2:172.21.15.97:6824/194699315,v1:172.21.15.97:6825/194699315] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f35840aef30 con 0x7f35840ac460 2024-09-11T23:20:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.184+0000 7f35997fa640 1 -- 172.21.15.43:0/2259215743 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f35940cf8c0 con 0x7f359c1082a0 2024-09-11T23:20:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.184+0000 7f359bfff640 1 --2- 172.21.15.43:0/2259215743 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f35840a2c10 0x7f35840a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f358c0058c0 tx=0x7f358c005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.184+0000 7f359b7fe640 1 --2- 172.21.15.43:0/2259215743 >> [v2:172.21.15.97:6824/194699315,v1:172.21.15.97:6825/194699315] conn(0x7f35840ac460 0x7f35840ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.185+0000 7f359b7fe640 1 --2- 172.21.15.43:0/2259215743 >> [v2:172.21.15.97:6824/194699315,v1:172.21.15.97:6825/194699315] conn(0x7f35840ac460 0x7f35840ae880 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.185+0000 7f35997fa640 1 -- 172.21.15.43:0/2259215743 <== osd.11 v2:172.21.15.97:6824/194699315 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f35840aef30 con 0x7f35840ac460 2024-09-11T23:20:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.200+0000 7f35a2513640 1 -- 172.21.15.43:0/2259215743 --> [v2:172.21.15.97:6824/194699315,v1:172.21.15.97:6825/194699315] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f359c0723d0 con 0x7f35840ac460 2024-09-11T23:20:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.200+0000 7f35997fa640 1 -- 172.21.15.43:0/2259215743 <== osd.11 v2:172.21.15.97:6824/194699315 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f359c0723d0 con 0x7f35840ac460 2024-09-11T23:20:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.200+0000 7f35a2513640 1 -- 172.21.15.43:0/2259215743 >> [v2:172.21.15.97:6824/194699315,v1:172.21.15.97:6825/194699315] conn(0x7f35840ac460 msgr2=0x7f35840ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.201+0000 7f35a2513640 1 --2- 172.21.15.43:0/2259215743 >> [v2:172.21.15.97:6824/194699315,v1:172.21.15.97:6825/194699315] conn(0x7f35840ac460 0x7f35840ae880 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.201+0000 7f35a2513640 1 -- 172.21.15.43:0/2259215743 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f35840a2c10 msgr2=0x7f35840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.201+0000 7f35a2513640 1 --2- 172.21.15.43:0/2259215743 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f35840a2c10 0x7f35840a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f358c0058c0 tx=0x7f358c005ec0 comp rx=0 tx=0).stop 2024-09-11T23:20:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.201+0000 7f35a2513640 1 -- 172.21.15.43:0/2259215743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f359c1082a0 msgr2=0x7f359c06d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.201+0000 7f35a2513640 1 --2- 172.21.15.43:0/2259215743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f359c1082a0 0x7f359c06d060 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f359400cad0 tx=0x7f359400cf90 comp rx=0 tx=0).stop 2024-09-11T23:20:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.201+0000 7f35a0ab0640 1 -- 172.21.15.43:0/2259215743 reap_dead start 2024-09-11T23:20:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.201+0000 7f35a2513640 1 -- 172.21.15.43:0/2259215743 shutdown_connections 2024-09-11T23:20:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.201+0000 7f35a2513640 1 -- 172.21.15.43:0/2259215743 >> 172.21.15.43:0/2259215743 conn(0x7f359c0fc7d0 msgr2=0x7f359c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.201+0000 7f35a2513640 1 -- 172.21.15.43:0/2259215743 shutdown_connections 2024-09-11T23:20:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:44.201+0000 7f35a2513640 1 -- 172.21.15.43:0/2259215743 wait complete. 2024-09-11T23:20:47.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.919+0000 7f07672c5640 1 -- 172.21.15.43:0/1482689890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f076010c1a0 msgr2=0x7f076010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:47.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.919+0000 7f07672c5640 1 --2- 172.21.15.43:0/1482689890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f076010c1a0 0x7f076010e610 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0758009ca0 tx=0x7f075802a950 comp rx=0 tx=0).stop 2024-09-11T23:20:47.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.920+0000 7f07672c5640 1 -- 172.21.15.43:0/1482689890 shutdown_connections 2024-09-11T23:20:47.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.920+0000 7f07672c5640 1 --2- 172.21.15.43:0/1482689890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f076010c1a0 0x7f076010e610 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:47.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.920+0000 7f07672c5640 1 --2- 172.21.15.43:0/1482689890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07601036e0 0x7f076010bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:47.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.920+0000 7f07672c5640 1 --2- 172.21.15.43:0/1482689890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0760102dc0 0x7f07601031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:47.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.920+0000 7f07672c5640 1 -- 172.21.15.43:0/1482689890 >> 172.21.15.43:0/1482689890 conn(0x7f07600fc7a0 msgr2=0x7f07600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:47.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.920+0000 7f07672c5640 1 -- 172.21.15.43:0/1482689890 shutdown_connections 2024-09-11T23:20:47.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.920+0000 7f07672c5640 1 -- 172.21.15.43:0/1482689890 wait complete. 2024-09-11T23:20:47.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.920+0000 7f07672c5640 1 Processor -- start 2024-09-11T23:20:47.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.920+0000 7f07672c5640 1 -- start start 2024-09-11T23:20:47.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f07672c5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0760102dc0 0x7f0760100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:47.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f07672c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07601036e0 0x7f0760100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:47.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f07672c5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f076010c1a0 0x7f07601014a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:47.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f07672c5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0760111f30 con 0x7f07601036e0 2024-09-11T23:20:47.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f07672c5640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0760111db0 con 0x7f076010c1a0 2024-09-11T23:20:47.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f07672c5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f07601120b0 con 0x7f0760102dc0 2024-09-11T23:20:47.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f0764860640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07601036e0 0x7f0760100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:47.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f0764860640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07601036e0 0x7f0760100f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46018/0 (socket says 172.21.15.43:46018) 2024-09-11T23:20:47.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f0764860640 1 -- 172.21.15.43:0/2365422808 learned_addr learned my addr 172.21.15.43:0/2365422808 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:47.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f0765862640 1 --2- 172.21.15.43:0/2365422808 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f076010c1a0 0x7f07601014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:47.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f0765061640 1 --2- 172.21.15.43:0/2365422808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0760102dc0 0x7f0760100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:47.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f0764860640 1 -- 172.21.15.43:0/2365422808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0760102dc0 msgr2=0x7f0760100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f0764860640 1 --2- 172.21.15.43:0/2365422808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0760102dc0 0x7f0760100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f0764860640 1 -- 172.21.15.43:0/2365422808 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f076010c1a0 msgr2=0x7f07601014a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f0764860640 1 --2- 172.21.15.43:0/2365422808 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f076010c1a0 0x7f07601014a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f0764860640 1 -- 172.21.15.43:0/2365422808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07601a6880 con 0x7f07601036e0 2024-09-11T23:20:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f0765862640 1 --2- 172.21.15.43:0/2365422808 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f076010c1a0 0x7f07601014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f0764860640 1 --2- 172.21.15.43:0/2365422808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07601036e0 0x7f0760100f60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f075c00d950 tx=0x7f075c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f07567fc640 1 -- 172.21.15.43:0/2365422808 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c017030 con 0x7f07601036e0 2024-09-11T23:20:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f07567fc640 1 -- 172.21.15.43:0/2365422808 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f075c004020 con 0x7f07601036e0 2024-09-11T23:20:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f07672c5640 1 -- 172.21.15.43:0/2365422808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07601a6b70 con 0x7f07601036e0 2024-09-11T23:20:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f07567fc640 1 -- 172.21.15.43:0/2365422808 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c0047a0 con 0x7f07601036e0 2024-09-11T23:20:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.921+0000 7f07672c5640 1 -- 172.21.15.43:0/2365422808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07601a7100 con 0x7f07601036e0 2024-09-11T23:20:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.922+0000 7f07672c5640 1 -- 172.21.15.43:0/2365422808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f07600628c0 con 0x7f07601036e0 2024-09-11T23:20:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.924+0000 7f07567fc640 1 -- 172.21.15.43:0/2365422808 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f075c0041d0 con 0x7f07601036e0 2024-09-11T23:20:47.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.924+0000 7f07567fc640 1 --2- 172.21.15.43:0/2365422808 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f074c0a2c90 0x7f074c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.925+0000 7f0765061640 1 --2- 172.21.15.43:0/2365422808 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f074c0a2c90 0x7f074c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.925+0000 7f07567fc640 1 -- 172.21.15.43:0/2365422808 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f075c0cf070 con 0x7f07601036e0 2024-09-11T23:20:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.925+0000 7f07567fc640 1 --2- 172.21.15.43:0/2365422808 >> [v2:172.21.15.97:6800/1456846443,v1:172.21.15.97:6801/1456846443] conn(0x7f074c0ac4e0 0x7f074c0ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.925+0000 7f07567fc640 1 -- 172.21.15.43:0/2365422808 --> [v2:172.21.15.97:6800/1456846443,v1:172.21.15.97:6801/1456846443] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f074c0aefb0 con 0x7f074c0ac4e0 2024-09-11T23:20:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.925+0000 7f07567fc640 1 -- 172.21.15.43:0/2365422808 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f075c0cf550 con 0x7f07601036e0 2024-09-11T23:20:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.925+0000 7f0765061640 1 --2- 172.21.15.43:0/2365422808 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f074c0a2c90 0x7f074c0a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0750005fd0 tx=0x7f075000b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.925+0000 7f0765862640 1 --2- 172.21.15.43:0/2365422808 >> [v2:172.21.15.97:6800/1456846443,v1:172.21.15.97:6801/1456846443] conn(0x7f074c0ac4e0 0x7f074c0ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.925+0000 7f0765862640 1 --2- 172.21.15.43:0/2365422808 >> [v2:172.21.15.97:6800/1456846443,v1:172.21.15.97:6801/1456846443] conn(0x7f074c0ac4e0 0x7f074c0ae900 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.926+0000 7f07567fc640 1 -- 172.21.15.43:0/2365422808 <== osd.2 v2:172.21.15.97:6800/1456846443 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f074c0aefb0 con 0x7f074c0ac4e0 2024-09-11T23:20:47.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.940+0000 7f07672c5640 1 -- 172.21.15.43:0/2365422808 --> [v2:172.21.15.97:6800/1456846443,v1:172.21.15.97:6801/1456846443] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0760062cf0 con 0x7f074c0ac4e0 2024-09-11T23:20:47.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.941+0000 7f07567fc640 1 -- 172.21.15.43:0/2365422808 <== osd.2 v2:172.21.15.97:6800/1456846443 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0760062cf0 con 0x7f074c0ac4e0 2024-09-11T23:20:47.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.941+0000 7f07672c5640 1 -- 172.21.15.43:0/2365422808 >> [v2:172.21.15.97:6800/1456846443,v1:172.21.15.97:6801/1456846443] conn(0x7f074c0ac4e0 msgr2=0x7f074c0ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:47.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.941+0000 7f07672c5640 1 --2- 172.21.15.43:0/2365422808 >> [v2:172.21.15.97:6800/1456846443,v1:172.21.15.97:6801/1456846443] conn(0x7f074c0ac4e0 0x7f074c0ae900 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:47.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.941+0000 7f07672c5640 1 -- 172.21.15.43:0/2365422808 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f074c0a2c90 msgr2=0x7f074c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:47.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.941+0000 7f07672c5640 1 --2- 172.21.15.43:0/2365422808 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f074c0a2c90 0x7f074c0a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0750005fd0 tx=0x7f075000b590 comp rx=0 tx=0).stop 2024-09-11T23:20:47.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.941+0000 7f07672c5640 1 -- 172.21.15.43:0/2365422808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07601036e0 msgr2=0x7f0760100f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:47.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.941+0000 7f07672c5640 1 --2- 172.21.15.43:0/2365422808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07601036e0 0x7f0760100f60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f075c00d950 tx=0x7f075c00de10 comp rx=0 tx=0).stop 2024-09-11T23:20:47.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.941+0000 7f0765862640 1 -- 172.21.15.43:0/2365422808 reap_dead start 2024-09-11T23:20:47.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.941+0000 7f07672c5640 1 -- 172.21.15.43:0/2365422808 shutdown_connections 2024-09-11T23:20:47.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.941+0000 7f07672c5640 1 -- 172.21.15.43:0/2365422808 >> 172.21.15.43:0/2365422808 conn(0x7f07600fc7a0 msgr2=0x7f07600fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:47.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.941+0000 7f07672c5640 1 -- 172.21.15.43:0/2365422808 shutdown_connections 2024-09-11T23:20:47.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:47.941+0000 7f07672c5640 1 -- 172.21.15.43:0/2365422808 wait complete. 2024-09-11T23:20:48.073 INFO:teuthology.orchestra.run.smithi043.stdout:55834574921 2024-09-11T23:20:48.074 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T23:20:48.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:48 smithi043 ceph-mon[30950]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T23:20:48.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:48 smithi090 ceph-mon[35130]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T23:20:48.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:48 smithi097 ceph-mon[35562]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T23:20:48.805 INFO:teuthology.orchestra.run.smithi043.stdout:326417514518 2024-09-11T23:20:48.805 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-11T23:20:49.410 INFO:teuthology.orchestra.run.smithi043.stdout:77309411398 2024-09-11T23:20:49.410 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T23:20:49.930 INFO:teuthology.orchestra.run.smithi043.stdout:111669149758 2024-09-11T23:20:49.930 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T23:20:50.388 INFO:teuthology.orchestra.run.smithi043.stdout:296352743450 2024-09-11T23:20:50.389 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-11T23:20:50.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:50 smithi043 ceph-mon[30950]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T23:20:50.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:50 smithi090 ceph-mon[35130]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T23:20:50.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:50 smithi097 ceph-mon[35562]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T23:20:50.900 INFO:teuthology.orchestra.run.smithi043.stdout:210453397546 2024-09-11T23:20:50.901 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T23:20:51.368 INFO:teuthology.orchestra.run.smithi043.stdout:158913790005 2024-09-11T23:20:51.368 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T23:20:51.385 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:51.409 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:51.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.947+0000 7f09e50aa640 1 -- 172.21.15.43:0/4233382186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09e0100a80 msgr2=0x7f09e0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:51.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.947+0000 7f09e50aa640 1 --2- 172.21.15.43:0/4233382186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09e0100a80 0x7f09e0100ee0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009930 tx=0x7f09c802a420 comp rx=0 tx=0).stop 2024-09-11T23:20:51.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.947+0000 7f09e50aa640 1 -- 172.21.15.43:0/4233382186 shutdown_connections 2024-09-11T23:20:51.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.947+0000 7f09e50aa640 1 --2- 172.21.15.43:0/4233382186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09e0101420 0x7f09e010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.947+0000 7f09e50aa640 1 --2- 172.21.15.43:0/4233382186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09e0100a80 0x7f09e0100ee0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.947+0000 7f09e50aa640 1 --2- 172.21.15.43:0/4233382186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09e0106a80 0x7f09e0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.947+0000 7f09e50aa640 1 -- 172.21.15.43:0/4233382186 >> 172.21.15.43:0/4233382186 conn(0x7f09e00fc7a0 msgr2=0x7f09e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:51.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.947+0000 7f09e50aa640 1 -- 172.21.15.43:0/4233382186 shutdown_connections 2024-09-11T23:20:51.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.947+0000 7f09e50aa640 1 -- 172.21.15.43:0/4233382186 wait complete. 2024-09-11T23:20:51.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.948+0000 7f09e50aa640 1 Processor -- start 2024-09-11T23:20:51.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.948+0000 7f09e50aa640 1 -- start start 2024-09-11T23:20:51.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.948+0000 7f09e50aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09e0100a80 0x7f09e01985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.948+0000 7f09e50aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09e0101420 0x7f09e0198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.948+0000 7f09e50aa640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09e0106a80 0x7f09e019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.948+0000 7f09e50aa640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f09e0110f20 con 0x7f09e0100a80 2024-09-11T23:20:51.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.948+0000 7f09e50aa640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f09e0110da0 con 0x7f09e0106a80 2024-09-11T23:20:51.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.948+0000 7f09e50aa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01110a0 con 0x7f09e0101420 2024-09-11T23:20:51.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.948+0000 7f09ded9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09e0100a80 0x7f09e01985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:51.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.948+0000 7f09ded9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09e0100a80 0x7f09e01985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38400/0 (socket says 172.21.15.43:38400) 2024-09-11T23:20:51.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.948+0000 7f09ded9d640 1 -- 172.21.15.43:0/1202548362 learned_addr learned my addr 172.21.15.43:0/1202548362 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:51.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.948+0000 7f09df59e640 1 --2- 172.21.15.43:0/1202548362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09e0106a80 0x7f09e019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:51.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.948+0000 7f09d659c640 1 --2- 172.21.15.43:0/1202548362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09e0101420 0x7f09e0198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:51.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.949+0000 7f09ded9d640 1 -- 172.21.15.43:0/1202548362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09e0101420 msgr2=0x7f09e0198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:51.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.949+0000 7f09ded9d640 1 --2- 172.21.15.43:0/1202548362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09e0101420 0x7f09e0198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.949+0000 7f09ded9d640 1 -- 172.21.15.43:0/1202548362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09e0106a80 msgr2=0x7f09e019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:51.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.949+0000 7f09ded9d640 1 --2- 172.21.15.43:0/1202548362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09e0106a80 0x7f09e019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:51.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.949+0000 7f09ded9d640 1 -- 172.21.15.43:0/1202548362 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e019d570 con 0x7f09e0100a80 2024-09-11T23:20:51.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.949+0000 7f09ded9d640 1 --2- 172.21.15.43:0/1202548362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09e0100a80 0x7f09e01985c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f09d000b4a0 tx=0x7f09d000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:51.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.949+0000 7f09dcd99640 1 -- 172.21.15.43:0/1202548362 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d0014070 con 0x7f09e0100a80 2024-09-11T23:20:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.949+0000 7f09dcd99640 1 -- 172.21.15.43:0/1202548362 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09d0010af0 con 0x7f09e0100a80 2024-09-11T23:20:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.949+0000 7f09e50aa640 1 -- 172.21.15.43:0/1202548362 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e019d860 con 0x7f09e0100a80 2024-09-11T23:20:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.949+0000 7f09dcd99640 1 -- 172.21.15.43:0/1202548362 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d0017d10 con 0x7f09e0100a80 2024-09-11T23:20:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.949+0000 7f09e50aa640 1 -- 172.21.15.43:0/1202548362 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e0102580 con 0x7f09e0100a80 2024-09-11T23:20:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.950+0000 7f09e50aa640 1 -- 172.21.15.43:0/1202548362 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e0100f60 con 0x7f09e0100a80 2024-09-11T23:20:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.951+0000 7f09dcd99640 1 -- 172.21.15.43:0/1202548362 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f09d0017070 con 0x7f09e0100a80 2024-09-11T23:20:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.952+0000 7f09dcd99640 1 --2- 172.21.15.43:0/1202548362 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f09cc0a2c10 0x7f09cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.952+0000 7f09d659c640 1 --2- 172.21.15.43:0/1202548362 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f09cc0a2c10 0x7f09cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.952+0000 7f09dcd99640 1 -- 172.21.15.43:0/1202548362 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f09d00ce390 con 0x7f09e0100a80 2024-09-11T23:20:51.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.953+0000 7f09d659c640 1 --2- 172.21.15.43:0/1202548362 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f09cc0a2c10 0x7f09cc0a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f09c80096c0 tx=0x7f09c8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:51.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:51.957+0000 7f09dcd99640 1 -- 172.21.15.43:0/1202548362 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09d009b450 con 0x7f09e0100a80 2024-09-11T23:20:52.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.136+0000 7f09e50aa640 1 -- 172.21.15.43:0/1202548362 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f09e0102840 con 0x7f09e0100a80 2024-09-11T23:20:52.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.137+0000 7f09dcd99640 1 -- 172.21.15.43:0/1202548362 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f09d009ad50 con 0x7f09e0100a80 2024-09-11T23:20:52.137 INFO:teuthology.orchestra.run.smithi043.stdout:274877906978 2024-09-11T23:20:52.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.139+0000 7f09e50aa640 1 -- 172.21.15.43:0/1202548362 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f09cc0a2c10 msgr2=0x7f09cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:52.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.139+0000 7f09e50aa640 1 --2- 172.21.15.43:0/1202548362 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f09cc0a2c10 0x7f09cc0a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f09c80096c0 tx=0x7f09c8000bc0 comp rx=0 tx=0).stop 2024-09-11T23:20:52.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.139+0000 7f09e50aa640 1 -- 172.21.15.43:0/1202548362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09e0100a80 msgr2=0x7f09e01985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:52.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.139+0000 7f09e50aa640 1 --2- 172.21.15.43:0/1202548362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09e0100a80 0x7f09e01985c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f09d000b4a0 tx=0x7f09d000b960 comp rx=0 tx=0).stop 2024-09-11T23:20:52.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.139+0000 7f09e50aa640 1 -- 172.21.15.43:0/1202548362 shutdown_connections 2024-09-11T23:20:52.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.139+0000 7f09e50aa640 1 --2- 172.21.15.43:0/1202548362 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f09cc0a2c10 0x7f09cc0a50d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:52.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.139+0000 7f09e50aa640 1 --2- 172.21.15.43:0/1202548362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09e0106a80 0x7f09e019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:52.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.139+0000 7f09e50aa640 1 --2- 172.21.15.43:0/1202548362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09e0101420 0x7f09e0198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:52.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.139+0000 7f09e50aa640 1 --2- 172.21.15.43:0/1202548362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09e0100a80 0x7f09e01985c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:52.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.139+0000 7f09e50aa640 1 -- 172.21.15.43:0/1202548362 >> 172.21.15.43:0/1202548362 conn(0x7f09e00fc7a0 msgr2=0x7f09e0104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:52.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.139+0000 7f09e50aa640 1 -- 172.21.15.43:0/1202548362 shutdown_connections 2024-09-11T23:20:52.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.139+0000 7f09e50aa640 1 -- 172.21.15.43:0/1202548362 wait complete. 2024-09-11T23:20:52.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:52 smithi043 ceph-mon[30950]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:20:52.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:52 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1202548362' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T23:20:52.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.627+0000 7fdb8e3b3640 1 -- 172.21.15.43:0/4036000101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb88101e10 msgr2=0x7fdb881021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.627+0000 7fdb8e3b3640 1 --2- 172.21.15.43:0/4036000101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb88101e10 0x7fdb881021f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009900 tx=0x7fdb7802a410 comp rx=0 tx=0).stop 2024-09-11T23:20:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.627+0000 7fdb8e3b3640 1 -- 172.21.15.43:0/4036000101 shutdown_connections 2024-09-11T23:20:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.627+0000 7fdb8e3b3640 1 --2- 172.21.15.43:0/4036000101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb88106f40 0x7fdb8810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.627+0000 7fdb8e3b3640 1 --2- 172.21.15.43:0/4036000101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb881027c0 0x7fdb88106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.627+0000 7fdb8e3b3640 1 --2- 172.21.15.43:0/4036000101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb88101e10 0x7fdb881021f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.627+0000 7fdb8e3b3640 1 -- 172.21.15.43:0/4036000101 >> 172.21.15.43:0/4036000101 conn(0x7fdb880fd8d0 msgr2=0x7fdb880ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.627+0000 7fdb8e3b3640 1 -- 172.21.15.43:0/4036000101 shutdown_connections 2024-09-11T23:20:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.627+0000 7fdb8e3b3640 1 -- 172.21.15.43:0/4036000101 wait complete. 2024-09-11T23:20:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.628+0000 7fdb8e3b3640 1 Processor -- start 2024-09-11T23:20:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.628+0000 7fdb8e3b3640 1 -- start start 2024-09-11T23:20:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.628+0000 7fdb8e3b3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb88101e10 0x7fdb881983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.628+0000 7fdb8e3b3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb881027c0 0x7fdb88198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.628+0000 7fdb8e3b3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb88106f40 0x7fdb8819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.628+0000 7fdb8e3b3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdb88111fa0 con 0x7fdb88101e10 2024-09-11T23:20:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.628+0000 7fdb8e3b3640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdb88111e20 con 0x7fdb881027c0 2024-09-11T23:20:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.628+0000 7fdb8e3b3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdb88112120 con 0x7fdb88106f40 2024-09-11T23:20:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.628+0000 7fdb87fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb88101e10 0x7fdb881983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.628+0000 7fdb87fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb88101e10 0x7fdb881983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38414/0 (socket says 172.21.15.43:38414) 2024-09-11T23:20:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.628+0000 7fdb87fff640 1 -- 172.21.15.43:0/2923180125 learned_addr learned my addr 172.21.15.43:0/2923180125 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.628+0000 7fdb877fe640 1 --2- 172.21.15.43:0/2923180125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb881027c0 0x7fdb88198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.629+0000 7fdb8c950640 1 --2- 172.21.15.43:0/2923180125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb88106f40 0x7fdb8819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.629+0000 7fdb87fff640 1 -- 172.21.15.43:0/2923180125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb88106f40 msgr2=0x7fdb8819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.629+0000 7fdb87fff640 1 --2- 172.21.15.43:0/2923180125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb88106f40 0x7fdb8819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.629+0000 7fdb87fff640 1 -- 172.21.15.43:0/2923180125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb881027c0 msgr2=0x7fdb88198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.629+0000 7fdb87fff640 1 --2- 172.21.15.43:0/2923180125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb881027c0 0x7fdb88198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.629+0000 7fdb87fff640 1 -- 172.21.15.43:0/2923180125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8819d3a0 con 0x7fdb88101e10 2024-09-11T23:20:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.629+0000 7fdb8c950640 1 --2- 172.21.15.43:0/2923180125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb88106f40 0x7fdb8819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:20:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.629+0000 7fdb87fff640 1 --2- 172.21.15.43:0/2923180125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb88101e10 0x7fdb881983f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdb78006010 tx=0x7fdb780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.629+0000 7fdb857fa640 1 -- 172.21.15.43:0/2923180125 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7803f030 con 0x7fdb88101e10 2024-09-11T23:20:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.629+0000 7fdb857fa640 1 -- 172.21.15.43:0/2923180125 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb78003ce0 con 0x7fdb88101e10 2024-09-11T23:20:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.629+0000 7fdb8e3b3640 1 -- 172.21.15.43:0/2923180125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb8819d630 con 0x7fdb88101e10 2024-09-11T23:20:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.629+0000 7fdb857fa640 1 -- 172.21.15.43:0/2923180125 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb78035070 con 0x7fdb88101e10 2024-09-11T23:20:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.629+0000 7fdb8e3b3640 1 -- 172.21.15.43:0/2923180125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb881a4f70 con 0x7fdb88101e10 2024-09-11T23:20:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.630+0000 7fdb8e3b3640 1 -- 172.21.15.43:0/2923180125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb881032f0 con 0x7fdb88101e10 2024-09-11T23:20:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.631+0000 7fdb857fa640 1 -- 172.21.15.43:0/2923180125 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdb78042030 con 0x7fdb88101e10 2024-09-11T23:20:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.632+0000 7fdb857fa640 1 --2- 172.21.15.43:0/2923180125 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fdb700a2c10 0x7fdb700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.633+0000 7fdb877fe640 1 --2- 172.21.15.43:0/2923180125 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fdb700a2c10 0x7fdb700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.633+0000 7fdb857fa640 1 -- 172.21.15.43:0/2923180125 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fdb780efa80 con 0x7fdb88101e10 2024-09-11T23:20:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.633+0000 7fdb877fe640 1 --2- 172.21.15.43:0/2923180125 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fdb700a2c10 0x7fdb700a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdb880ffe30 tx=0x7fdb7c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.636+0000 7fdb857fa640 1 -- 172.21.15.43:0/2923180125 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdb780bc9f0 con 0x7fdb88101e10 2024-09-11T23:20:52.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:52 smithi090 ceph-mon[35130]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:20:52.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:52 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1202548362' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T23:20:52.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:52 smithi097 ceph-mon[35562]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:20:52.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:52 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1202548362' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T23:20:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.820+0000 7fdb8e3b3640 1 -- 172.21.15.43:0/2923180125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fdb8810a120 con 0x7fdb88101e10 2024-09-11T23:20:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.821+0000 7fdb857fa640 1 -- 172.21.15.43:0/2923180125 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdb780bc2f0 con 0x7fdb88101e10 2024-09-11T23:20:52.821 INFO:teuthology.orchestra.run.smithi043.stdout:141733920828 2024-09-11T23:20:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.823+0000 7fdb8e3b3640 1 -- 172.21.15.43:0/2923180125 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fdb700a2c10 msgr2=0x7fdb700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.823+0000 7fdb8e3b3640 1 --2- 172.21.15.43:0/2923180125 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fdb700a2c10 0x7fdb700a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdb880ffe30 tx=0x7fdb7c00b390 comp rx=0 tx=0).stop 2024-09-11T23:20:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.823+0000 7fdb8e3b3640 1 -- 172.21.15.43:0/2923180125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb88101e10 msgr2=0x7fdb881983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.823+0000 7fdb8e3b3640 1 --2- 172.21.15.43:0/2923180125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb88101e10 0x7fdb881983f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdb78006010 tx=0x7fdb780042f0 comp rx=0 tx=0).stop 2024-09-11T23:20:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.823+0000 7fdb8e3b3640 1 -- 172.21.15.43:0/2923180125 shutdown_connections 2024-09-11T23:20:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.823+0000 7fdb8e3b3640 1 --2- 172.21.15.43:0/2923180125 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fdb700a2c10 0x7fdb700a50d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.823+0000 7fdb8e3b3640 1 --2- 172.21.15.43:0/2923180125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb88106f40 0x7fdb8819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.823+0000 7fdb8e3b3640 1 --2- 172.21.15.43:0/2923180125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb881027c0 0x7fdb88198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.823+0000 7fdb8e3b3640 1 --2- 172.21.15.43:0/2923180125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb88101e10 0x7fdb881983f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.823+0000 7fdb8e3b3640 1 -- 172.21.15.43:0/2923180125 >> 172.21.15.43:0/2923180125 conn(0x7fdb880fd8d0 msgr2=0x7fdb880fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.823+0000 7fdb8e3b3640 1 -- 172.21.15.43:0/2923180125 shutdown_connections 2024-09-11T23:20:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:52.823+0000 7fdb8e3b3640 1 -- 172.21.15.43:0/2923180125 wait complete. 2024-09-11T23:20:53.128 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906976 got 274877906978 for osd.9 2024-09-11T23:20:53.128 DEBUG:teuthology.parallel:result is None 2024-09-11T23:20:53.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:53 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2923180125' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T23:20:53.410 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:53.410 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:53.415 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:53.415 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:53.418 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:53.425 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:53.430 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:53.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:53 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2923180125' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T23:20:53.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:53 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2923180125' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T23:20:54.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.230+0000 7f7186ef4640 1 -- 172.21.15.43:0/2893984795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7180101de0 msgr2=0x7f71801021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:54.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.230+0000 7f7186ef4640 1 --2- 172.21.15.43:0/2893984795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7180101de0 0x7f71801021c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7174009900 tx=0x7f717402a410 comp rx=0 tx=0).stop 2024-09-11T23:20:54.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.230+0000 7f7186ef4640 1 -- 172.21.15.43:0/2893984795 shutdown_connections 2024-09-11T23:20:54.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.230+0000 7f7186ef4640 1 --2- 172.21.15.43:0/2893984795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7180106f10 0x7f718010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.230+0000 7f7186ef4640 1 --2- 172.21.15.43:0/2893984795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7180102790 0x7f71801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.230+0000 7f7186ef4640 1 --2- 172.21.15.43:0/2893984795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7180101de0 0x7f71801021c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.230+0000 7f7186ef4640 1 -- 172.21.15.43:0/2893984795 >> 172.21.15.43:0/2893984795 conn(0x7f71800fd8e0 msgr2=0x7f71800ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:54.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.231+0000 7f7186ef4640 1 -- 172.21.15.43:0/2893984795 shutdown_connections 2024-09-11T23:20:54.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.231+0000 7f7186ef4640 1 -- 172.21.15.43:0/2893984795 wait complete. 2024-09-11T23:20:54.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.231+0000 7f7186ef4640 1 Processor -- start 2024-09-11T23:20:54.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.231+0000 7f7186ef4640 1 -- start start 2024-09-11T23:20:54.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.231+0000 7f7186ef4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7180101de0 0x7f718019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.231+0000 7f7186ef4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7180102790 0x7f718019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.231+0000 7f7186ef4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7180106f10 0x7f718019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.231+0000 7f7186ef4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7180111f30 con 0x7f7180101de0 2024-09-11T23:20:54.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.231+0000 7f7186ef4640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7180111db0 con 0x7f7180106f10 2024-09-11T23:20:54.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.231+0000 7f7186ef4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f71801120b0 con 0x7f7180102790 2024-09-11T23:20:54.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.231+0000 7f7184c90640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7180101de0 0x7f718019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:54.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.231+0000 7f7184c90640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7180101de0 0x7f718019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38428/0 (socket says 172.21.15.43:38428) 2024-09-11T23:20:54.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.231+0000 7f7184c90640 1 -- 172.21.15.43:0/925739183 learned_addr learned my addr 172.21.15.43:0/925739183 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:54.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.232+0000 7f7185491640 1 --2- 172.21.15.43:0/925739183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7180106f10 0x7f718019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:54.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.232+0000 7f7184c90640 1 -- 172.21.15.43:0/925739183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7180102790 msgr2=0x7f718019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:54.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.232+0000 7f7184c90640 1 --2- 172.21.15.43:0/925739183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7180102790 0x7f718019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.232+0000 7f7184c90640 1 -- 172.21.15.43:0/925739183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7180106f10 msgr2=0x7f718019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:54.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.232+0000 7f7184c90640 1 --2- 172.21.15.43:0/925739183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7180106f10 0x7f718019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.232+0000 7f7184c90640 1 -- 172.21.15.43:0/925739183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718019f580 con 0x7f7180101de0 2024-09-11T23:20:54.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.232+0000 7f7184c90640 1 --2- 172.21.15.43:0/925739183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7180101de0 0x7f718019a5d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7174006010 tx=0x7f71740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:54.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.232+0000 7f717dffb640 1 -- 172.21.15.43:0/925739183 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717403f030 con 0x7f7180101de0 2024-09-11T23:20:54.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.232+0000 7f717dffb640 1 -- 172.21.15.43:0/925739183 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7174003ce0 con 0x7f7180101de0 2024-09-11T23:20:54.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.232+0000 7f7186ef4640 1 -- 172.21.15.43:0/925739183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f718019f810 con 0x7f7180101de0 2024-09-11T23:20:54.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.232+0000 7f717dffb640 1 -- 172.21.15.43:0/925739183 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174035070 con 0x7f7180101de0 2024-09-11T23:20:54.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.232+0000 7f7186ef4640 1 -- 172.21.15.43:0/925739183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71801a7150 con 0x7f7180101de0 2024-09-11T23:20:54.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.233+0000 7f7186ef4640 1 -- 172.21.15.43:0/925739183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71801021c0 con 0x7f7180101de0 2024-09-11T23:20:54.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.235+0000 7f717dffb640 1 -- 172.21.15.43:0/925739183 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7174042030 con 0x7f7180101de0 2024-09-11T23:20:54.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.235+0000 7f717dffb640 1 --2- 172.21.15.43:0/925739183 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f71700a2c90 0x7f71700a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:54.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.235+0000 7f717dffb640 1 -- 172.21.15.43:0/925739183 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f71740ef790 con 0x7f7180101de0 2024-09-11T23:20:54.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.235+0000 7f717ffff640 1 --2- 172.21.15.43:0/925739183 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f71700a2c90 0x7f71700a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:54.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.236+0000 7f717ffff640 1 --2- 172.21.15.43:0/925739183 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f71700a2c90 0x7f71700a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f71800ff710 tx=0x7f716c005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:54.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.239+0000 7f717dffb640 1 -- 172.21.15.43:0/925739183 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f71740bc800 con 0x7f7180101de0 2024-09-11T23:20:54.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:54 smithi043 ceph-mon[30950]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T23:20:54.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.418+0000 7f7186ef4640 1 -- 172.21.15.43:0/925739183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f71801a7320 con 0x7f7180101de0 2024-09-11T23:20:54.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.419+0000 7f717dffb640 1 -- 172.21.15.43:0/925739183 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f71740bc100 con 0x7f7180101de0 2024-09-11T23:20:54.419 INFO:teuthology.orchestra.run.smithi043.stdout:188978561073 2024-09-11T23:20:54.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.421+0000 7f7186ef4640 1 -- 172.21.15.43:0/925739183 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f71700a2c90 msgr2=0x7f71700a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:54.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.421+0000 7f7186ef4640 1 --2- 172.21.15.43:0/925739183 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f71700a2c90 0x7f71700a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f71800ff710 tx=0x7f716c005c70 comp rx=0 tx=0).stop 2024-09-11T23:20:54.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.421+0000 7f7186ef4640 1 -- 172.21.15.43:0/925739183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7180101de0 msgr2=0x7f718019a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:54.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.421+0000 7f7186ef4640 1 --2- 172.21.15.43:0/925739183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7180101de0 0x7f718019a5d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7174006010 tx=0x7f71740042f0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.421+0000 7f7186ef4640 1 -- 172.21.15.43:0/925739183 shutdown_connections 2024-09-11T23:20:54.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.421+0000 7f7186ef4640 1 --2- 172.21.15.43:0/925739183 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f71700a2c90 0x7f71700a5150 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.421+0000 7f7186ef4640 1 --2- 172.21.15.43:0/925739183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7180106f10 0x7f718019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.421+0000 7f7186ef4640 1 --2- 172.21.15.43:0/925739183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7180102790 0x7f718019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.421+0000 7f7186ef4640 1 --2- 172.21.15.43:0/925739183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7180101de0 0x7f718019a5d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:54.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.421+0000 7f7186ef4640 1 -- 172.21.15.43:0/925739183 >> 172.21.15.43:0/925739183 conn(0x7f71800fd8e0 msgr2=0x7f71800feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:54.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.421+0000 7f7186ef4640 1 -- 172.21.15.43:0/925739183 shutdown_connections 2024-09-11T23:20:54.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:54.421+0000 7f7186ef4640 1 -- 172.21.15.43:0/925739183 wait complete. 2024-09-11T23:20:54.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:54 smithi090 ceph-mon[35130]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T23:20:54.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:54 smithi097 ceph-mon[35562]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T23:20:55.176 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561072 got 188978561073 for osd.6 2024-09-11T23:20:55.177 DEBUG:teuthology.parallel:result is None 2024-09-11T23:20:55.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:55 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/925739183' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T23:20:55.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.394+0000 7f3ecd0b2640 1 -- 172.21.15.43:0/3744714577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ec8106f10 msgr2=0x7f3ec810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:55.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.394+0000 7f3ecd0b2640 1 --2- 172.21.15.43:0/3744714577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ec8106f10 0x7f3ec810ea90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009ca0 tx=0x7f3ec002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:20:55.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.394+0000 7f3ecd0b2640 1 -- 172.21.15.43:0/3744714577 shutdown_connections 2024-09-11T23:20:55.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.394+0000 7f3ecd0b2640 1 --2- 172.21.15.43:0/3744714577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ec8106f10 0x7f3ec810ea90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:55.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.394+0000 7f3ecd0b2640 1 --2- 172.21.15.43:0/3744714577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ec8102790 0x7f3ec81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:55.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.394+0000 7f3ecd0b2640 1 --2- 172.21.15.43:0/3744714577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ec8101de0 0x7f3ec81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:55.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.394+0000 7f3ecd0b2640 1 -- 172.21.15.43:0/3744714577 >> 172.21.15.43:0/3744714577 conn(0x7f3ec80fd8c0 msgr2=0x7f3ec80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:55.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.394+0000 7f3ecd0b2640 1 -- 172.21.15.43:0/3744714577 shutdown_connections 2024-09-11T23:20:55.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.394+0000 7f3ecd0b2640 1 -- 172.21.15.43:0/3744714577 wait complete. 2024-09-11T23:20:55.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.395+0000 7f3ecd0b2640 1 Processor -- start 2024-09-11T23:20:55.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.395+0000 7f3ecd0b2640 1 -- start start 2024-09-11T23:20:55.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.395+0000 7f3ecd0b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ec8101de0 0x7f3ec80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:55.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.395+0000 7f3ecd0b2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ec8102790 0x7f3ec8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:55.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.395+0000 7f3ecd0b2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ec8106f10 0x7f3ec8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:55.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.395+0000 7f3ecd0b2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec8111f70 con 0x7f3ec8101de0 2024-09-11T23:20:55.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.395+0000 7f3ecd0b2640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec8111df0 con 0x7f3ec8102790 2024-09-11T23:20:55.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.395+0000 7f3ecd0b2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec81120f0 con 0x7f3ec8106f10 2024-09-11T23:20:55.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.395+0000 7f3ec6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ec8101de0 0x7f3ec80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:55.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.395+0000 7f3ec6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ec8101de0 0x7f3ec80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38450/0 (socket says 172.21.15.43:38450) 2024-09-11T23:20:55.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.395+0000 7f3ec6d9d640 1 -- 172.21.15.43:0/415077043 learned_addr learned my addr 172.21.15.43:0/415077043 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:55.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.396+0000 7f3ec759e640 1 --2- 172.21.15.43:0/415077043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ec8106f10 0x7f3ec8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:55.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.396+0000 7f3ec659c640 1 --2- 172.21.15.43:0/415077043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ec8102790 0x7f3ec8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:55.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.396+0000 7f3ec6d9d640 1 -- 172.21.15.43:0/415077043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ec8106f10 msgr2=0x7f3ec8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:55.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.396+0000 7f3ec6d9d640 1 --2- 172.21.15.43:0/415077043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ec8106f10 0x7f3ec8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:55.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.396+0000 7f3ec6d9d640 1 -- 172.21.15.43:0/415077043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ec8102790 msgr2=0x7f3ec8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:55.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.396+0000 7f3ec6d9d640 1 --2- 172.21.15.43:0/415077043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ec8102790 0x7f3ec8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:55.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.396+0000 7f3ec6d9d640 1 -- 172.21.15.43:0/415077043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec81a6890 con 0x7f3ec8101de0 2024-09-11T23:20:55.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.396+0000 7f3ec6d9d640 1 --2- 172.21.15.43:0/415077043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ec8101de0 0x7f3ec80744f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3eb800d770 tx=0x7f3eb800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:55.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.396+0000 7f3eaffff640 1 -- 172.21.15.43:0/415077043 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8017070 con 0x7f3ec8101de0 2024-09-11T23:20:55.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.396+0000 7f3ecd0b2640 1 -- 172.21.15.43:0/415077043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec81a6b80 con 0x7f3ec8101de0 2024-09-11T23:20:55.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.396+0000 7f3eaffff640 1 -- 172.21.15.43:0/415077043 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3eb8005500 con 0x7f3ec8101de0 2024-09-11T23:20:55.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.396+0000 7f3eaffff640 1 -- 172.21.15.43:0/415077043 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8004e00 con 0x7f3ec8101de0 2024-09-11T23:20:55.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.396+0000 7f3ecd0b2640 1 -- 172.21.15.43:0/415077043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec81a70c0 con 0x7f3ec8101de0 2024-09-11T23:20:55.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.397+0000 7f3ecd0b2640 1 -- 172.21.15.43:0/415077043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec81021c0 con 0x7f3ec8101de0 2024-09-11T23:20:55.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.398+0000 7f3eaffff640 1 -- 172.21.15.43:0/415077043 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3eb8004fa0 con 0x7f3ec8101de0 2024-09-11T23:20:55.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.399+0000 7f3eaffff640 1 --2- 172.21.15.43:0/415077043 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3eb00a2c10 0x7f3eb00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:55.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.399+0000 7f3ec659c640 1 --2- 172.21.15.43:0/415077043 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3eb00a2c10 0x7f3eb00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:55.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.399+0000 7f3eaffff640 1 -- 172.21.15.43:0/415077043 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f3eb80ceb00 con 0x7f3ec8101de0 2024-09-11T23:20:55.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.400+0000 7f3ec659c640 1 --2- 172.21.15.43:0/415077043 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3eb00a2c10 0x7f3eb00a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc005e00 tx=0x7f3ebc005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:55.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.403+0000 7f3eaffff640 1 -- 172.21.15.43:0/415077043 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3eb809bab0 con 0x7f3ec8101de0 2024-09-11T23:20:55.423 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:55.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.590+0000 7f3ecd0b2640 1 -- 172.21.15.43:0/415077043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f3ec802d7c0 con 0x7f3ec8101de0 2024-09-11T23:20:55.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.590+0000 7f3eaffff640 1 -- 172.21.15.43:0/415077043 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3eb809b3b0 con 0x7f3ec8101de0 2024-09-11T23:20:55.591 INFO:teuthology.orchestra.run.smithi043.stdout:240518168616 2024-09-11T23:20:55.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.592+0000 7f3ecd0b2640 1 -- 172.21.15.43:0/415077043 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3eb00a2c10 msgr2=0x7f3eb00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:55.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.592+0000 7f3ecd0b2640 1 --2- 172.21.15.43:0/415077043 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3eb00a2c10 0x7f3eb00a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc005e00 tx=0x7f3ebc005c50 comp rx=0 tx=0).stop 2024-09-11T23:20:55.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.592+0000 7f3ecd0b2640 1 -- 172.21.15.43:0/415077043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ec8101de0 msgr2=0x7f3ec80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:55.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.592+0000 7f3ecd0b2640 1 --2- 172.21.15.43:0/415077043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ec8101de0 0x7f3ec80744f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3eb800d770 tx=0x7f3eb800dc30 comp rx=0 tx=0).stop 2024-09-11T23:20:55.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.592+0000 7f3ecd0b2640 1 -- 172.21.15.43:0/415077043 shutdown_connections 2024-09-11T23:20:55.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.592+0000 7f3ecd0b2640 1 --2- 172.21.15.43:0/415077043 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3eb00a2c10 0x7f3eb00a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:55.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.592+0000 7f3ecd0b2640 1 --2- 172.21.15.43:0/415077043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ec8106f10 0x7f3ec8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:55.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.592+0000 7f3ecd0b2640 1 --2- 172.21.15.43:0/415077043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ec8102790 0x7f3ec8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:55.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.592+0000 7f3ecd0b2640 1 --2- 172.21.15.43:0/415077043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ec8101de0 0x7f3ec80744f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:55.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.592+0000 7f3ecd0b2640 1 -- 172.21.15.43:0/415077043 >> 172.21.15.43:0/415077043 conn(0x7f3ec80fd8c0 msgr2=0x7f3ec80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:55.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.593+0000 7f3ecd0b2640 1 -- 172.21.15.43:0/415077043 shutdown_connections 2024-09-11T23:20:55.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:55.593+0000 7f3ecd0b2640 1 -- 172.21.15.43:0/415077043 wait complete. 2024-09-11T23:20:55.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:55 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/925739183' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T23:20:55.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:55 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/925739183' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T23:20:55.823 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920825 got 141733920828 for osd.4 2024-09-11T23:20:55.823 DEBUG:teuthology.parallel:result is None 2024-09-11T23:20:56.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.456+0000 7f932a957640 1 -- 172.21.15.43:0/1056152301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9324102db0 msgr2=0x7f9324103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:56.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.456+0000 7f932a957640 1 --2- 172.21.15.43:0/1056152301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9324102db0 0x7f9324103190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9314009900 tx=0x7f931402a480 comp rx=0 tx=0).stop 2024-09-11T23:20:56.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.456+0000 7f932a957640 1 -- 172.21.15.43:0/1056152301 shutdown_connections 2024-09-11T23:20:56.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.456+0000 7f932a957640 1 --2- 172.21.15.43:0/1056152301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f932410c190 0x7f932410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:56.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.456+0000 7f932a957640 1 --2- 172.21.15.43:0/1056152301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93241036d0 0x7f932410bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:56.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.456+0000 7f932a957640 1 --2- 172.21.15.43:0/1056152301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9324102db0 0x7f9324103190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:56.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.456+0000 7f932a957640 1 -- 172.21.15.43:0/1056152301 >> 172.21.15.43:0/1056152301 conn(0x7f93240fc790 msgr2=0x7f93240febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:56.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.456+0000 7f932a957640 1 -- 172.21.15.43:0/1056152301 shutdown_connections 2024-09-11T23:20:56.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.456+0000 7f932a957640 1 -- 172.21.15.43:0/1056152301 wait complete. 2024-09-11T23:20:56.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.457+0000 7f932a957640 1 Processor -- start 2024-09-11T23:20:56.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.457+0000 7f932a957640 1 -- start start 2024-09-11T23:20:56.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.458+0000 7f932a957640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9324102db0 0x7f9324198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:56.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.458+0000 7f932a957640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93241036d0 0x7f9324198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:56.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.458+0000 7f932a957640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f932410c190 0x7f932419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:56.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.458+0000 7f932a957640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9324111f60 con 0x7f9324102db0 2024-09-11T23:20:56.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.458+0000 7f932a957640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9324111de0 con 0x7f93241036d0 2024-09-11T23:20:56.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.458+0000 7f932a957640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f93241120e0 con 0x7f932410c190 2024-09-11T23:20:56.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.458+0000 7f9323fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9324102db0 0x7f9324198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:56.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.458+0000 7f9323fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9324102db0 0x7f9324198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38488/0 (socket says 172.21.15.43:38488) 2024-09-11T23:20:56.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.458+0000 7f9323fff640 1 -- 172.21.15.43:0/2644700171 learned_addr learned my addr 172.21.15.43:0/2644700171 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:56.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.458+0000 7f9328ef4640 1 --2- 172.21.15.43:0/2644700171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f932410c190 0x7f932419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:56.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.458+0000 7f93237fe640 1 --2- 172.21.15.43:0/2644700171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93241036d0 0x7f9324198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:56.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.458+0000 7f9323fff640 1 -- 172.21.15.43:0/2644700171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f932410c190 msgr2=0x7f932419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:56.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.458+0000 7f9323fff640 1 --2- 172.21.15.43:0/2644700171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f932410c190 0x7f932419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:56.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.458+0000 7f9323fff640 1 -- 172.21.15.43:0/2644700171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93241036d0 msgr2=0x7f9324198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:56.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.458+0000 7f9323fff640 1 --2- 172.21.15.43:0/2644700171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93241036d0 0x7f9324198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:56.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.458+0000 7f9323fff640 1 -- 172.21.15.43:0/2644700171 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932419d400 con 0x7f9324102db0 2024-09-11T23:20:56.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.458+0000 7f93237fe640 1 --2- 172.21.15.43:0/2644700171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93241036d0 0x7f9324198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:56.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.459+0000 7f9323fff640 1 --2- 172.21.15.43:0/2644700171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9324102db0 0x7f9324198450 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9314006010 tx=0x7f9314004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:56.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.459+0000 7f93217fa640 1 -- 172.21.15.43:0/2644700171 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931403f030 con 0x7f9324102db0 2024-09-11T23:20:56.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.459+0000 7f932a957640 1 -- 172.21.15.43:0/2644700171 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932419d690 con 0x7f9324102db0 2024-09-11T23:20:56.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.459+0000 7f93217fa640 1 -- 172.21.15.43:0/2644700171 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9314004440 con 0x7f9324102db0 2024-09-11T23:20:56.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.459+0000 7f93217fa640 1 -- 172.21.15.43:0/2644700171 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9314035070 con 0x7f9324102db0 2024-09-11T23:20:56.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.459+0000 7f932a957640 1 -- 172.21.15.43:0/2644700171 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9324100940 con 0x7f9324102db0 2024-09-11T23:20:56.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.460+0000 7f932a957640 1 -- 172.21.15.43:0/2644700171 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f932406aff0 con 0x7f9324102db0 2024-09-11T23:20:56.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.462+0000 7f93217fa640 1 -- 172.21.15.43:0/2644700171 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9314042030 con 0x7f9324102db0 2024-09-11T23:20:56.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.462+0000 7f93217fa640 1 --2- 172.21.15.43:0/2644700171 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f930c0a2c10 0x7f930c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:56.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.462+0000 7f93217fa640 1 -- 172.21.15.43:0/2644700171 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f93140ef4b0 con 0x7f9324102db0 2024-09-11T23:20:56.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.462+0000 7f93237fe640 1 --2- 172.21.15.43:0/2644700171 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f930c0a2c10 0x7f930c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:56.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.463+0000 7f93237fe640 1 --2- 172.21.15.43:0/2644700171 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f930c0a2c10 0x7f930c0a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f93240fee00 tx=0x7f9318012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:56.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.468+0000 7f93217fa640 1 -- 172.21.15.43:0/2644700171 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f93140bc460 con 0x7f9324102db0 2024-09-11T23:20:56.598 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:56 smithi043 ceph-mon[30950]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-11T23:20:56.598 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:56 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/415077043' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T23:20:56.696 INFO:teuthology.orchestra.run.smithi043.stdout:296352743452 2024-09-11T23:20:56.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.695+0000 7f932a957640 1 -- 172.21.15.43:0/2644700171 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f9324107230 con 0x7f9324102db0 2024-09-11T23:20:56.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.696+0000 7f93217fa640 1 -- 172.21.15.43:0/2644700171 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f93140bbd60 con 0x7f9324102db0 2024-09-11T23:20:56.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.698+0000 7f932a957640 1 -- 172.21.15.43:0/2644700171 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f930c0a2c10 msgr2=0x7f930c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:56.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.698+0000 7f932a957640 1 --2- 172.21.15.43:0/2644700171 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f930c0a2c10 0x7f930c0a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f93240fee00 tx=0x7f9318012040 comp rx=0 tx=0).stop 2024-09-11T23:20:56.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.698+0000 7f932a957640 1 -- 172.21.15.43:0/2644700171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9324102db0 msgr2=0x7f9324198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:56.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.698+0000 7f932a957640 1 --2- 172.21.15.43:0/2644700171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9324102db0 0x7f9324198450 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9314006010 tx=0x7f9314004040 comp rx=0 tx=0).stop 2024-09-11T23:20:56.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.698+0000 7f932a957640 1 -- 172.21.15.43:0/2644700171 shutdown_connections 2024-09-11T23:20:56.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.698+0000 7f932a957640 1 --2- 172.21.15.43:0/2644700171 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f930c0a2c10 0x7f930c0a50d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:56.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.698+0000 7f932a957640 1 --2- 172.21.15.43:0/2644700171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f932410c190 0x7f932419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:56.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.698+0000 7f932a957640 1 --2- 172.21.15.43:0/2644700171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93241036d0 0x7f9324198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:56.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.698+0000 7f932a957640 1 --2- 172.21.15.43:0/2644700171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9324102db0 0x7f9324198450 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:56.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.698+0000 7f932a957640 1 -- 172.21.15.43:0/2644700171 >> 172.21.15.43:0/2644700171 conn(0x7f93240fc790 msgr2=0x7f93240fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:56.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.698+0000 7f932a957640 1 -- 172.21.15.43:0/2644700171 shutdown_connections 2024-09-11T23:20:56.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.698+0000 7f932a957640 1 -- 172.21.15.43:0/2644700171 wait complete. 2024-09-11T23:20:56.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:56 smithi090 ceph-mon[35130]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-11T23:20:56.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:56 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/415077043' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T23:20:56.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:56 smithi097 ceph-mon[35562]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-11T23:20:56.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:56 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/415077043' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T23:20:57.352 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168614 got 240518168616 for osd.8 2024-09-11T23:20:57.353 DEBUG:teuthology.parallel:result is None 2024-09-11T23:20:57.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:57 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:20:57.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:57 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2176934075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T23:20:57.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:57 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2644700171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T23:20:57.556 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:20:57.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:57 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:20:57.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:57 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2176934075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T23:20:57.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:57 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2644700171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T23:20:57.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:57 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:20:57.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:57 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2176934075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T23:20:57.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:57 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2644700171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T23:20:57.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.334+0000 7f461f97b640 1 -- 172.21.15.43:0/2815476265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4618101dd0 msgr2=0x7f46181021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:57.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.334+0000 7f461f97b640 1 --2- 172.21.15.43:0/2815476265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4618101dd0 0x7f46181021b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4614009900 tx=0x7f461402a410 comp rx=0 tx=0).stop 2024-09-11T23:20:57.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.334+0000 7f461f97b640 1 -- 172.21.15.43:0/2815476265 shutdown_connections 2024-09-11T23:20:57.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.334+0000 7f461f97b640 1 --2- 172.21.15.43:0/2815476265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4618106f00 0x7f461810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:57.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.334+0000 7f461f97b640 1 --2- 172.21.15.43:0/2815476265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4618102780 0x7f46181067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:57.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.334+0000 7f461f97b640 1 --2- 172.21.15.43:0/2815476265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4618101dd0 0x7f46181021b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:57.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.334+0000 7f461f97b640 1 -- 172.21.15.43:0/2815476265 >> 172.21.15.43:0/2815476265 conn(0x7f46180fd8b0 msgr2=0x7f46180ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:57.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.334+0000 7f461f97b640 1 -- 172.21.15.43:0/2815476265 shutdown_connections 2024-09-11T23:20:57.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.334+0000 7f461f97b640 1 -- 172.21.15.43:0/2815476265 wait complete. 2024-09-11T23:20:57.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.335+0000 7f461f97b640 1 Processor -- start 2024-09-11T23:20:57.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.335+0000 7f461f97b640 1 -- start start 2024-09-11T23:20:57.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.335+0000 7f461f97b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4618101dd0 0x7f46180744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:57.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.335+0000 7f461f97b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4618102780 0x7f4618074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:57.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.335+0000 7f461f97b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4618106f00 0x7f4618074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:57.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.335+0000 7f461f97b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4618111f60 con 0x7f4618102780 2024-09-11T23:20:57.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.335+0000 7f461f97b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4618111de0 con 0x7f4618101dd0 2024-09-11T23:20:57.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.335+0000 7f461f97b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f46181120e0 con 0x7f4618106f00 2024-09-11T23:20:57.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.335+0000 7f461cf16640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4618102780 0x7f4618074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:57.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.335+0000 7f461cf16640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4618102780 0x7f4618074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38470/0 (socket says 172.21.15.43:38470) 2024-09-11T23:20:57.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.335+0000 7f461cf16640 1 -- 172.21.15.43:0/2176934075 learned_addr learned my addr 172.21.15.43:0/2176934075 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:57.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.336+0000 7f461d717640 1 --2- 172.21.15.43:0/2176934075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4618101dd0 0x7f46180744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:57.897 INFO:teuthology.orchestra.run.smithi043.stdout:34359738449 2024-09-11T23:20:57.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.336+0000 7f461df18640 1 --2- 172.21.15.43:0/2176934075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4618106f00 0x7f4618074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:57.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.336+0000 7f461cf16640 1 -- 172.21.15.43:0/2176934075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4618106f00 msgr2=0x7f4618074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:57.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.336+0000 7f461cf16640 1 --2- 172.21.15.43:0/2176934075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4618106f00 0x7f4618074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:57.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.336+0000 7f461cf16640 1 -- 172.21.15.43:0/2176934075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4618101dd0 msgr2=0x7f46180744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.336+0000 7f461cf16640 1 --2- 172.21.15.43:0/2176934075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4618101dd0 0x7f46180744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.336+0000 7f461cf16640 1 -- 172.21.15.43:0/2176934075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46181a6990 con 0x7f4618102780 2024-09-11T23:20:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.336+0000 7f461cf16640 1 --2- 172.21.15.43:0/2176934075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4618102780 0x7f4618074a30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f460800d950 tx=0x7f460800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.336+0000 7f460e7fc640 1 -- 172.21.15.43:0/2176934075 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4608017030 con 0x7f4618102780 2024-09-11T23:20:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.336+0000 7f461f97b640 1 -- 172.21.15.43:0/2176934075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46181a6c80 con 0x7f4618102780 2024-09-11T23:20:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.336+0000 7f460e7fc640 1 -- 172.21.15.43:0/2176934075 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4608004020 con 0x7f4618102780 2024-09-11T23:20:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.336+0000 7f460e7fc640 1 -- 172.21.15.43:0/2176934075 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46080047a0 con 0x7f4618102780 2024-09-11T23:20:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.336+0000 7f461f97b640 1 -- 172.21.15.43:0/2176934075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46181a71c0 con 0x7f4618102780 2024-09-11T23:20:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.337+0000 7f461f97b640 1 -- 172.21.15.43:0/2176934075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46181021b0 con 0x7f4618102780 2024-09-11T23:20:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.339+0000 7f460e7fc640 1 -- 172.21.15.43:0/2176934075 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f46080041d0 con 0x7f4618102780 2024-09-11T23:20:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.339+0000 7f460e7fc640 1 --2- 172.21.15.43:0/2176934075 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f46040a2c10 0x7f46040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.339+0000 7f461d717640 1 --2- 172.21.15.43:0/2176934075 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f46040a2c10 0x7f46040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.339+0000 7f460e7fc640 1 -- 172.21.15.43:0/2176934075 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f46080ceff0 con 0x7f4618102780 2024-09-11T23:20:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.340+0000 7f461d717640 1 --2- 172.21.15.43:0/2176934075 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f46040a2c10 0x7f46040a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4614009230 tx=0x7f46140058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.347+0000 7f460e7fc640 1 -- 172.21.15.43:0/2176934075 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f460809bfa0 con 0x7f4618102780 2024-09-11T23:20:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.542+0000 7f461f97b640 1 -- 172.21.15.43:0/2176934075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f461810a0e0 con 0x7f4618102780 2024-09-11T23:20:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.543+0000 7f460e7fc640 1 -- 172.21.15.43:0/2176934075 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f460809b8a0 con 0x7f4618102780 2024-09-11T23:20:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.545+0000 7f461f97b640 1 -- 172.21.15.43:0/2176934075 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f46040a2c10 msgr2=0x7f46040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.545+0000 7f461f97b640 1 --2- 172.21.15.43:0/2176934075 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f46040a2c10 0x7f46040a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4614009230 tx=0x7f46140058d0 comp rx=0 tx=0).stop 2024-09-11T23:20:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.545+0000 7f461f97b640 1 -- 172.21.15.43:0/2176934075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4618102780 msgr2=0x7f4618074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:57.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.545+0000 7f461f97b640 1 --2- 172.21.15.43:0/2176934075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4618102780 0x7f4618074a30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f460800d950 tx=0x7f460800de10 comp rx=0 tx=0).stop 2024-09-11T23:20:57.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.545+0000 7f461f97b640 1 -- 172.21.15.43:0/2176934075 shutdown_connections 2024-09-11T23:20:57.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.545+0000 7f461f97b640 1 --2- 172.21.15.43:0/2176934075 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f46040a2c10 0x7f46040a50d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:57.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.545+0000 7f461f97b640 1 --2- 172.21.15.43:0/2176934075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4618106f00 0x7f4618074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:57.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.545+0000 7f461f97b640 1 --2- 172.21.15.43:0/2176934075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4618102780 0x7f4618074a30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:57.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.545+0000 7f461f97b640 1 --2- 172.21.15.43:0/2176934075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4618101dd0 0x7f46180744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:57.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.545+0000 7f461f97b640 1 -- 172.21.15.43:0/2176934075 >> 172.21.15.43:0/2176934075 conn(0x7f46180fd8b0 msgr2=0x7f46180feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:57.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.545+0000 7f461f97b640 1 -- 172.21.15.43:0/2176934075 shutdown_connections 2024-09-11T23:20:57.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:56.545+0000 7f461f97b640 1 -- 172.21.15.43:0/2176934075 wait complete. 2024-09-11T23:20:58.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:58 smithi043 ceph-mon[30950]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2024-09-11T23:20:58.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:58 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:20:58.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:58 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:20:58.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:58 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:20:58.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:58 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:20:58.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.676+0000 7f3977fb9640 1 -- 172.21.15.43:0/4133704863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3970100e20 msgr2=0x7f3970101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.676+0000 7f3977fb9640 1 --2- 172.21.15.43:0/4133704863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3970100e20 0x7f3970101200 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f396c009900 tx=0x7f396c02a480 comp rx=0 tx=0).stop 2024-09-11T23:20:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.676+0000 7f3977fb9640 1 -- 172.21.15.43:0/4133704863 shutdown_connections 2024-09-11T23:20:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.676+0000 7f3977fb9640 1 --2- 172.21.15.43:0/4133704863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3970102ef0 0x7f397010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.676+0000 7f3977fb9640 1 --2- 172.21.15.43:0/4133704863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3970102570 0x7f39701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.676+0000 7f3977fb9640 1 --2- 172.21.15.43:0/4133704863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3970100e20 0x7f3970101200 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.676+0000 7f3977fb9640 1 -- 172.21.15.43:0/4133704863 >> 172.21.15.43:0/4133704863 conn(0x7f39700fc7d0 msgr2=0x7f39700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.676+0000 7f3977fb9640 1 -- 172.21.15.43:0/4133704863 shutdown_connections 2024-09-11T23:20:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.676+0000 7f3977fb9640 1 -- 172.21.15.43:0/4133704863 wait complete. 2024-09-11T23:20:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.676+0000 7f3977fb9640 1 Processor -- start 2024-09-11T23:20:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.677+0000 7f3977fb9640 1 -- start start 2024-09-11T23:20:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.677+0000 7f3977fb9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3970100e20 0x7f3970199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.677+0000 7f3977fb9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3970102570 0x7f3970199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.677+0000 7f3977fb9640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3970102ef0 0x7f39701a44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.677+0000 7f3977fb9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3970110fd0 con 0x7f3970100e20 2024-09-11T23:20:58.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.677+0000 7f3977fb9640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3970110e50 con 0x7f3970102ef0 2024-09-11T23:20:58.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.677+0000 7f3977fb9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3970111150 con 0x7f3970102570 2024-09-11T23:20:58.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.677+0000 7f3975d55640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3970100e20 0x7f3970199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.677+0000 7f3975d55640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3970100e20 0x7f3970199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38536/0 (socket says 172.21.15.43:38536) 2024-09-11T23:20:58.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.677+0000 7f3975d55640 1 -- 172.21.15.43:0/1667747684 learned_addr learned my addr 172.21.15.43:0/1667747684 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:58.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.677+0000 7f3975554640 1 --2- 172.21.15.43:0/1667747684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3970102570 0x7f3970199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.677+0000 7f3976556640 1 --2- 172.21.15.43:0/1667747684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3970102ef0 0x7f39701a44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.677+0000 7f3975d55640 1 -- 172.21.15.43:0/1667747684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3970102570 msgr2=0x7f3970199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:58.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.677+0000 7f3975d55640 1 --2- 172.21.15.43:0/1667747684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3970102570 0x7f3970199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.677+0000 7f3975d55640 1 -- 172.21.15.43:0/1667747684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3970102ef0 msgr2=0x7f39701a44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:58.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.677+0000 7f3975d55640 1 --2- 172.21.15.43:0/1667747684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3970102ef0 0x7f39701a44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.677+0000 7f3975d55640 1 -- 172.21.15.43:0/1667747684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39701a49e0 con 0x7f3970100e20 2024-09-11T23:20:58.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.677+0000 7f3975554640 1 --2- 172.21.15.43:0/1667747684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3970102570 0x7f3970199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:20:58.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.677+0000 7f3976556640 1 --2- 172.21.15.43:0/1667747684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3970102ef0 0x7f39701a44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:20:58.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.677+0000 7f3975d55640 1 --2- 172.21.15.43:0/1667747684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3970100e20 0x7f3970199420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f396c02aa70 tx=0x7f396c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:58.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.678+0000 7f3966ffd640 1 -- 172.21.15.43:0/1667747684 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c03f070 con 0x7f3970100e20 2024-09-11T23:20:58.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.678+0000 7f3966ffd640 1 -- 172.21.15.43:0/1667747684 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f396c031c30 con 0x7f3970100e20 2024-09-11T23:20:58.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.678+0000 7f3977fb9640 1 -- 172.21.15.43:0/1667747684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39701a4bb0 con 0x7f3970100e20 2024-09-11T23:20:58.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.678+0000 7f3966ffd640 1 -- 172.21.15.43:0/1667747684 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c035030 con 0x7f3970100e20 2024-09-11T23:20:58.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.678+0000 7f3977fb9640 1 -- 172.21.15.43:0/1667747684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39701a5000 con 0x7f3970100e20 2024-09-11T23:20:58.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.679+0000 7f3977fb9640 1 -- 172.21.15.43:0/1667747684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3970101200 con 0x7f3970100e20 2024-09-11T23:20:58.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.680+0000 7f3966ffd640 1 -- 172.21.15.43:0/1667747684 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f396c042070 con 0x7f3970100e20 2024-09-11T23:20:58.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.681+0000 7f3966ffd640 1 --2- 172.21.15.43:0/1667747684 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f395c0a2c10 0x7f395c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:58.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.681+0000 7f3975554640 1 --2- 172.21.15.43:0/1667747684 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f395c0a2c10 0x7f395c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.681+0000 7f3966ffd640 1 -- 172.21.15.43:0/1667747684 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f396c0ef850 con 0x7f3970100e20 2024-09-11T23:20:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.681+0000 7f3975554640 1 --2- 172.21.15.43:0/1667747684 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f395c0a2c10 0x7f395c0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f397006aac0 tx=0x7f396000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.685+0000 7f3966ffd640 1 -- 172.21.15.43:0/1667747684 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f396c031de0 con 0x7f3970100e20 2024-09-11T23:20:58.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:58 smithi090 ceph-mon[35130]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2024-09-11T23:20:58.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:58 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:20:58.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:58 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:20:58.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:58 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:20:58.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:58 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:20:58.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:58 smithi097 ceph-mon[35562]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s 2024-09-11T23:20:58.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:58 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:20:58.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:58 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:20:58.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:58 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:20:58.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:58 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:20:58.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.893+0000 7f3977fb9640 1 -- 172.21.15.43:0/1667747684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3970109090 con 0x7f3970100e20 2024-09-11T23:20:58.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.893+0000 7f3966ffd640 1 -- 172.21.15.43:0/1667747684 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f396c0bc700 con 0x7f3970100e20 2024-09-11T23:20:58.893 INFO:teuthology.orchestra.run.smithi043.stdout:111669149762 2024-09-11T23:20:58.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.895+0000 7f3977fb9640 1 -- 172.21.15.43:0/1667747684 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f395c0a2c10 msgr2=0x7f395c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:58.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.895+0000 7f3977fb9640 1 --2- 172.21.15.43:0/1667747684 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f395c0a2c10 0x7f395c0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f397006aac0 tx=0x7f396000b390 comp rx=0 tx=0).stop 2024-09-11T23:20:58.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.896+0000 7f3977fb9640 1 -- 172.21.15.43:0/1667747684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3970100e20 msgr2=0x7f3970199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:58.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.896+0000 7f3977fb9640 1 --2- 172.21.15.43:0/1667747684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3970100e20 0x7f3970199420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f396c02aa70 tx=0x7f396c003c10 comp rx=0 tx=0).stop 2024-09-11T23:20:58.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.896+0000 7f3977fb9640 1 -- 172.21.15.43:0/1667747684 shutdown_connections 2024-09-11T23:20:58.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.896+0000 7f3977fb9640 1 --2- 172.21.15.43:0/1667747684 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f395c0a2c10 0x7f395c0a50d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.896+0000 7f3977fb9640 1 --2- 172.21.15.43:0/1667747684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3970102ef0 0x7f39701a44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.896+0000 7f3977fb9640 1 --2- 172.21.15.43:0/1667747684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3970102570 0x7f3970199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.896+0000 7f3977fb9640 1 --2- 172.21.15.43:0/1667747684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3970100e20 0x7f3970199420 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:58.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.896+0000 7f3977fb9640 1 -- 172.21.15.43:0/1667747684 >> 172.21.15.43:0/1667747684 conn(0x7f39700fc7d0 msgr2=0x7f39700fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:58.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.896+0000 7f3977fb9640 1 -- 172.21.15.43:0/1667747684 shutdown_connections 2024-09-11T23:20:58.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.896+0000 7f3977fb9640 1 -- 172.21.15.43:0/1667747684 wait complete. 2024-09-11T23:20:59.525 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:59 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/859525523' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T23:20:59.525 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:20:59 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1667747684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T23:20:59.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.609+0000 7fcf2d2aa640 1 -- 172.21.15.43:0/2909057949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf28100a80 msgr2=0x7fcf28100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:59.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.609+0000 7fcf2d2aa640 1 --2- 172.21.15.43:0/2909057949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf28100a80 0x7fcf28100ee0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009930 tx=0x7fcf1c02a420 comp rx=0 tx=0).stop 2024-09-11T23:20:59.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.609+0000 7fcf2d2aa640 1 -- 172.21.15.43:0/2909057949 shutdown_connections 2024-09-11T23:20:59.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.609+0000 7fcf2d2aa640 1 --2- 172.21.15.43:0/2909057949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf28101420 0x7fcf2810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.609+0000 7fcf2d2aa640 1 --2- 172.21.15.43:0/2909057949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf28100a80 0x7fcf28100ee0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.609+0000 7fcf2d2aa640 1 --2- 172.21.15.43:0/2909057949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf28106a80 0x7fcf28106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.609+0000 7fcf2d2aa640 1 -- 172.21.15.43:0/2909057949 >> 172.21.15.43:0/2909057949 conn(0x7fcf280fc7a0 msgr2=0x7fcf280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:59.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.609+0000 7fcf2d2aa640 1 -- 172.21.15.43:0/2909057949 shutdown_connections 2024-09-11T23:20:59.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.609+0000 7fcf2d2aa640 1 -- 172.21.15.43:0/2909057949 wait complete. 2024-09-11T23:20:59.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.610+0000 7fcf2d2aa640 1 Processor -- start 2024-09-11T23:20:59.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.610+0000 7fcf2d2aa640 1 -- start start 2024-09-11T23:20:59.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.610+0000 7fcf2d2aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf28100a80 0x7fcf28198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:59.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.610+0000 7fcf2d2aa640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf28101420 0x7fcf281989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:59.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.610+0000 7fcf2d2aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf28106a80 0x7fcf2819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:59.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.610+0000 7fcf2d2aa640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcf28110e40 con 0x7fcf28106a80 2024-09-11T23:20:59.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.610+0000 7fcf2d2aa640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcf28110cc0 con 0x7fcf28101420 2024-09-11T23:20:59.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.610+0000 7fcf2d2aa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcf28110fc0 con 0x7fcf28100a80 2024-09-11T23:20:59.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.610+0000 7fcf277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf28106a80 0x7fcf2819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:59.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.610+0000 7fcf277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf28106a80 0x7fcf2819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38550/0 (socket says 172.21.15.43:38550) 2024-09-11T23:20:59.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.611+0000 7fcf277fe640 1 -- 172.21.15.43:0/1607353687 learned_addr learned my addr 172.21.15.43:0/1607353687 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:59.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.611+0000 7fcf26ffd640 1 --2- 172.21.15.43:0/1607353687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf28100a80 0x7fcf28198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:59.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.611+0000 7fcf267fc640 1 --2- 172.21.15.43:0/1607353687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf28101420 0x7fcf281989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:59.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.611+0000 7fcf277fe640 1 -- 172.21.15.43:0/1607353687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf28100a80 msgr2=0x7fcf28198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:59.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.611+0000 7fcf277fe640 1 --2- 172.21.15.43:0/1607353687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf28100a80 0x7fcf28198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.611+0000 7fcf277fe640 1 -- 172.21.15.43:0/1607353687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf28101420 msgr2=0x7fcf281989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.611+0000 7fcf277fe640 1 --2- 172.21.15.43:0/1607353687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf28101420 0x7fcf281989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.611+0000 7fcf277fe640 1 -- 172.21.15.43:0/1607353687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2819d410 con 0x7fcf28106a80 2024-09-11T23:20:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.611+0000 7fcf277fe640 1 --2- 172.21.15.43:0/1607353687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf28106a80 0x7fcf2819cd30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000eaa0 tx=0x7fcf2000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.611+0000 7fcf0ffff640 1 -- 172.21.15.43:0/1607353687 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2000cdb0 con 0x7fcf28106a80 2024-09-11T23:20:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.611+0000 7fcf0ffff640 1 -- 172.21.15.43:0/1607353687 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf20005020 con 0x7fcf28106a80 2024-09-11T23:20:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.611+0000 7fcf2d2aa640 1 -- 172.21.15.43:0/1607353687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf2819d700 con 0x7fcf28106a80 2024-09-11T23:20:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.611+0000 7fcf0ffff640 1 -- 172.21.15.43:0/1607353687 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf200116b0 con 0x7fcf28106a80 2024-09-11T23:20:59.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.611+0000 7fcf2d2aa640 1 -- 172.21.15.43:0/1607353687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf281a4fe0 con 0x7fcf28106a80 2024-09-11T23:20:59.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.612+0000 7fcf2d2aa640 1 -- 172.21.15.43:0/1607353687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf28100ee0 con 0x7fcf28106a80 2024-09-11T23:20:59.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.614+0000 7fcf0ffff640 1 -- 172.21.15.43:0/1607353687 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcf20005560 con 0x7fcf28106a80 2024-09-11T23:20:59.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.614+0000 7fcf0ffff640 1 --2- 172.21.15.43:0/1607353687 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fcf100a2c10 0x7fcf100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:59.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.614+0000 7fcf26ffd640 1 --2- 172.21.15.43:0/1607353687 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fcf100a2c10 0x7fcf100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:59.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.614+0000 7fcf0ffff640 1 -- 172.21.15.43:0/1607353687 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fcf200cf5b0 con 0x7fcf28106a80 2024-09-11T23:20:59.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.615+0000 7fcf26ffd640 1 --2- 172.21.15.43:0/1607353687 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fcf100a2c10 0x7fcf100a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800bb60 tx=0x7fcf1800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:59.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.618+0000 7fcf0ffff640 1 -- 172.21.15.43:0/1607353687 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf20016030 con 0x7fcf28106a80 2024-09-11T23:20:59.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:59 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/859525523' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T23:20:59.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:20:59 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1667747684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T23:20:59.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.800+0000 7fcf2d2aa640 1 -- 172.21.15.43:0/1607353687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fcf281997c0 con 0x7fcf28106a80 2024-09-11T23:20:59.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.800+0000 7fcf0ffff640 1 -- 172.21.15.43:0/1607353687 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fcf281997c0 con 0x7fcf28106a80 2024-09-11T23:20:59.801 INFO:teuthology.orchestra.run.smithi043.stdout:326417514520 2024-09-11T23:20:59.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:59 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/859525523' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T23:20:59.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:20:59 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1667747684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T23:20:59.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.802+0000 7fcf2d2aa640 1 -- 172.21.15.43:0/1607353687 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fcf100a2c10 msgr2=0x7fcf100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:59.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.802+0000 7fcf2d2aa640 1 --2- 172.21.15.43:0/1607353687 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fcf100a2c10 0x7fcf100a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800bb60 tx=0x7fcf1800b390 comp rx=0 tx=0).stop 2024-09-11T23:20:59.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.802+0000 7fcf2d2aa640 1 -- 172.21.15.43:0/1607353687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf28106a80 msgr2=0x7fcf2819cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.802+0000 7fcf2d2aa640 1 --2- 172.21.15.43:0/1607353687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf28106a80 0x7fcf2819cd30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000eaa0 tx=0x7fcf2000ef60 comp rx=0 tx=0).stop 2024-09-11T23:20:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.802+0000 7fcf2d2aa640 1 -- 172.21.15.43:0/1607353687 shutdown_connections 2024-09-11T23:20:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.802+0000 7fcf2d2aa640 1 --2- 172.21.15.43:0/1607353687 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fcf100a2c10 0x7fcf100a50d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.802+0000 7fcf2d2aa640 1 --2- 172.21.15.43:0/1607353687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf28106a80 0x7fcf2819cd30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.802+0000 7fcf2d2aa640 1 --2- 172.21.15.43:0/1607353687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf28101420 0x7fcf281989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.802+0000 7fcf2d2aa640 1 --2- 172.21.15.43:0/1607353687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf28100a80 0x7fcf28198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.802+0000 7fcf2d2aa640 1 -- 172.21.15.43:0/1607353687 >> 172.21.15.43:0/1607353687 conn(0x7fcf280fc7a0 msgr2=0x7fcf2806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.803+0000 7fcf2d2aa640 1 -- 172.21.15.43:0/1607353687 shutdown_connections 2024-09-11T23:20:59.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.803+0000 7fcf2d2aa640 1 -- 172.21.15.43:0/1607353687 wait complete. 2024-09-11T23:20:59.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.837+0000 7f34406d8640 1 -- 172.21.15.43:0/3449750672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3438102d30 msgr2=0x7f3438103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:59.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.837+0000 7f34406d8640 1 --2- 172.21.15.43:0/3449750672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3438102d30 0x7f3438103110 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3428009930 tx=0x7f342802a420 comp rx=0 tx=0).stop 2024-09-11T23:20:59.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.838+0000 7f34406d8640 1 -- 172.21.15.43:0/3449750672 shutdown_connections 2024-09-11T23:20:59.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.838+0000 7f34406d8640 1 --2- 172.21.15.43:0/3449750672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34381082a0 0x7f343810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.838+0000 7f34406d8640 1 --2- 172.21.15.43:0/3449750672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34381036e0 0x7f3438107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.838+0000 7f34406d8640 1 --2- 172.21.15.43:0/3449750672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3438102d30 0x7f3438103110 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.838+0000 7f34406d8640 1 -- 172.21.15.43:0/3449750672 >> 172.21.15.43:0/3449750672 conn(0x7f34380fc7d0 msgr2=0x7f34380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:20:59.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.838+0000 7f34406d8640 1 -- 172.21.15.43:0/3449750672 shutdown_connections 2024-09-11T23:20:59.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.838+0000 7f34406d8640 1 -- 172.21.15.43:0/3449750672 wait complete. 2024-09-11T23:20:59.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.838+0000 7f34406d8640 1 Processor -- start 2024-09-11T23:20:59.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.838+0000 7f34406d8640 1 -- start start 2024-09-11T23:20:59.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.838+0000 7f34406d8640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3438102d30 0x7f34381a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:59.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.838+0000 7f34406d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34381036e0 0x7f34381a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:59.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.838+0000 7f34406d8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34381082a0 0x7f343819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:59.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.838+0000 7f34406d8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f34381120a0 con 0x7f34381036e0 2024-09-11T23:20:59.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.838+0000 7f34406d8640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3438111f20 con 0x7f3438102d30 2024-09-11T23:20:59.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.838+0000 7f34406d8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3438112220 con 0x7f34381082a0 2024-09-11T23:20:59.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.839+0000 7f343dc73640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34381036e0 0x7f34381a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:59.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.839+0000 7f343dc73640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34381036e0 0x7f34381a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38568/0 (socket says 172.21.15.43:38568) 2024-09-11T23:20:59.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.839+0000 7f343dc73640 1 -- 172.21.15.43:0/1497727782 learned_addr learned my addr 172.21.15.43:0/1497727782 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:20:59.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.839+0000 7f343ec75640 1 --2- 172.21.15.43:0/1497727782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34381082a0 0x7f343819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:59.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.839+0000 7f343e474640 1 --2- 172.21.15.43:0/1497727782 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3438102d30 0x7f34381a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:59.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.839+0000 7f343dc73640 1 -- 172.21.15.43:0/1497727782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34381082a0 msgr2=0x7f343819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:59.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.839+0000 7f343dc73640 1 --2- 172.21.15.43:0/1497727782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34381082a0 0x7f343819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.839+0000 7f343dc73640 1 -- 172.21.15.43:0/1497727782 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3438102d30 msgr2=0x7f34381a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:20:59.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.839+0000 7f343dc73640 1 --2- 172.21.15.43:0/1497727782 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3438102d30 0x7f34381a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:20:59.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.839+0000 7f343dc73640 1 -- 172.21.15.43:0/1497727782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343819ad00 con 0x7f34381036e0 2024-09-11T23:20:59.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.839+0000 7f343dc73640 1 --2- 172.21.15.43:0/1497727782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34381036e0 0x7f34381a0950 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f342c00d740 tx=0x7f342c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:59.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.839+0000 7f34377fe640 1 -- 172.21.15.43:0/1497727782 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c017070 con 0x7f34381036e0 2024-09-11T23:20:59.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.839+0000 7f34377fe640 1 -- 172.21.15.43:0/1497727782 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f342c00de50 con 0x7f34381036e0 2024-09-11T23:20:59.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.839+0000 7f34406d8640 1 -- 172.21.15.43:0/1497727782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f343819aff0 con 0x7f34381036e0 2024-09-11T23:20:59.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.839+0000 7f34377fe640 1 -- 172.21.15.43:0/1497727782 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c00f6b0 con 0x7f34381036e0 2024-09-11T23:20:59.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.839+0000 7f34406d8640 1 -- 172.21.15.43:0/1497727782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34381a72e0 con 0x7f34381036e0 2024-09-11T23:20:59.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.840+0000 7f34406d8640 1 -- 172.21.15.43:0/1497727782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f343806a850 con 0x7f34381036e0 2024-09-11T23:20:59.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.842+0000 7f34377fe640 1 -- 172.21.15.43:0/1497727782 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f342c01a070 con 0x7f34381036e0 2024-09-11T23:20:59.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.842+0000 7f34377fe640 1 --2- 172.21.15.43:0/1497727782 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f34200a2c90 0x7f34200a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:20:59.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.843+0000 7f343e474640 1 --2- 172.21.15.43:0/1497727782 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f34200a2c90 0x7f34200a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:20:59.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.843+0000 7f34377fe640 1 -- 172.21.15.43:0/1497727782 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f342c0ce9f0 con 0x7f34381036e0 2024-09-11T23:20:59.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.843+0000 7f343e474640 1 --2- 172.21.15.43:0/1497727782 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f34200a2c90 0x7f34200a5150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3428000c00 tx=0x7f3428005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:20:59.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:59.847+0000 7f34377fe640 1 -- 172.21.15.43:0/1497727782 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f342c09b9a0 con 0x7f34381036e0 2024-09-11T23:21:00.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.024+0000 7f34406d8640 1 -- 172.21.15.43:0/1497727782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f343810c2c0 con 0x7f34381036e0 2024-09-11T23:21:00.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.024+0000 7f34377fe640 1 -- 172.21.15.43:0/1497727782 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f342c09b2a0 con 0x7f34381036e0 2024-09-11T23:21:00.024 INFO:teuthology.orchestra.run.smithi043.stdout:55834574924 2024-09-11T23:21:00.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.026+0000 7f34406d8640 1 -- 172.21.15.43:0/1497727782 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f34200a2c90 msgr2=0x7f34200a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:00.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.026+0000 7f34406d8640 1 --2- 172.21.15.43:0/1497727782 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f34200a2c90 0x7f34200a5150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3428000c00 tx=0x7f3428005a20 comp rx=0 tx=0).stop 2024-09-11T23:21:00.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.026+0000 7f34406d8640 1 -- 172.21.15.43:0/1497727782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34381036e0 msgr2=0x7f34381a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:00.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.026+0000 7f34406d8640 1 --2- 172.21.15.43:0/1497727782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34381036e0 0x7f34381a0950 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f342c00d740 tx=0x7f342c00dc00 comp rx=0 tx=0).stop 2024-09-11T23:21:00.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.026+0000 7f34406d8640 1 -- 172.21.15.43:0/1497727782 shutdown_connections 2024-09-11T23:21:00.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.026+0000 7f34406d8640 1 --2- 172.21.15.43:0/1497727782 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f34200a2c90 0x7f34200a5150 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:00.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.026+0000 7f34406d8640 1 --2- 172.21.15.43:0/1497727782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34381082a0 0x7f343819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:00.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.026+0000 7f34406d8640 1 --2- 172.21.15.43:0/1497727782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34381036e0 0x7f34381a0950 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:00.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.026+0000 7f34406d8640 1 --2- 172.21.15.43:0/1497727782 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3438102d30 0x7f34381a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:00.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.026+0000 7f34406d8640 1 -- 172.21.15.43:0/1497727782 >> 172.21.15.43:0/1497727782 conn(0x7f34380fc7d0 msgr2=0x7f3438100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:00.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.026+0000 7f34406d8640 1 -- 172.21.15.43:0/1497727782 shutdown_connections 2024-09-11T23:21:00.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.026+0000 7f34406d8640 1 -- 172.21.15.43:0/1497727782 wait complete. 2024-09-11T23:21:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.063+0000 7efc632bb640 1 -- 172.21.15.43:0/3104859440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c102d30 msgr2=0x7efc5c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.063+0000 7efc632bb640 1 --2- 172.21.15.43:0/3104859440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c102d30 0x7efc5c103110 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efc58009cd0 tx=0x7efc5802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:21:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.063+0000 7efc632bb640 1 -- 172.21.15.43:0/3104859440 shutdown_connections 2024-09-11T23:21:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.063+0000 7efc632bb640 1 --2- 172.21.15.43:0/3104859440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc5c1082a0 0x7efc5c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.063+0000 7efc632bb640 1 --2- 172.21.15.43:0/3104859440 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc5c1036e0 0x7efc5c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.063+0000 7efc632bb640 1 --2- 172.21.15.43:0/3104859440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c102d30 0x7efc5c103110 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:00.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.063+0000 7efc632bb640 1 -- 172.21.15.43:0/3104859440 >> 172.21.15.43:0/3104859440 conn(0x7efc5c0fc7d0 msgr2=0x7efc5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:00.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.063+0000 7efc632bb640 1 -- 172.21.15.43:0/3104859440 shutdown_connections 2024-09-11T23:21:00.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.063+0000 7efc632bb640 1 -- 172.21.15.43:0/3104859440 wait complete. 2024-09-11T23:21:00.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.064+0000 7efc632bb640 1 Processor -- start 2024-09-11T23:21:00.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.064+0000 7efc632bb640 1 -- start start 2024-09-11T23:21:00.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.064+0000 7efc632bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c102d30 0x7efc5c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:00.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.064+0000 7efc632bb640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc5c1036e0 0x7efc5c198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:00.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.064+0000 7efc632bb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc5c1082a0 0x7efc5c19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:00.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.064+0000 7efc632bb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c1120a0 con 0x7efc5c102d30 2024-09-11T23:21:00.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.064+0000 7efc632bb640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c111f20 con 0x7efc5c1036e0 2024-09-11T23:21:00.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.064+0000 7efc632bb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c112220 con 0x7efc5c1082a0 2024-09-11T23:21:00.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.064+0000 7efc61057640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c102d30 0x7efc5c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:00.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.064+0000 7efc61057640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c102d30 0x7efc5c1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38582/0 (socket says 172.21.15.43:38582) 2024-09-11T23:21:00.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.064+0000 7efc61057640 1 -- 172.21.15.43:0/3240701328 learned_addr learned my addr 172.21.15.43:0/3240701328 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:00.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.064+0000 7efc61858640 1 --2- 172.21.15.43:0/3240701328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc5c1082a0 0x7efc5c19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:00.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.065+0000 7efc60856640 1 --2- 172.21.15.43:0/3240701328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc5c1036e0 0x7efc5c198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:00.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.065+0000 7efc61057640 1 -- 172.21.15.43:0/3240701328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc5c1082a0 msgr2=0x7efc5c19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:00.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.065+0000 7efc61057640 1 --2- 172.21.15.43:0/3240701328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc5c1082a0 0x7efc5c19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:00.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.065+0000 7efc61057640 1 -- 172.21.15.43:0/3240701328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc5c1036e0 msgr2=0x7efc5c198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:00.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.065+0000 7efc61057640 1 --2- 172.21.15.43:0/3240701328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc5c1036e0 0x7efc5c198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:00.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.065+0000 7efc61057640 1 -- 172.21.15.43:0/3240701328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5c19d5a0 con 0x7efc5c102d30 2024-09-11T23:21:00.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.065+0000 7efc61858640 1 --2- 172.21.15.43:0/3240701328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc5c1082a0 0x7efc5c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:00.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.065+0000 7efc60856640 1 --2- 172.21.15.43:0/3240701328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc5c1036e0 0x7efc5c198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:21:00.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.065+0000 7efc61057640 1 --2- 172.21.15.43:0/3240701328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c102d30 0x7efc5c1985f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efc5802ae00 tx=0x7efc58003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:00.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.065+0000 7efc527fc640 1 -- 172.21.15.43:0/3240701328 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58004040 con 0x7efc5c102d30 2024-09-11T23:21:00.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.065+0000 7efc632bb640 1 -- 172.21.15.43:0/3240701328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5c19d830 con 0x7efc5c102d30 2024-09-11T23:21:00.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.065+0000 7efc527fc640 1 -- 172.21.15.43:0/3240701328 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc580046c0 con 0x7efc5c102d30 2024-09-11T23:21:00.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.065+0000 7efc527fc640 1 -- 172.21.15.43:0/3240701328 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58046480 con 0x7efc5c102d30 2024-09-11T23:21:00.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.065+0000 7efc632bb640 1 -- 172.21.15.43:0/3240701328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5c1a5170 con 0x7efc5c102d30 2024-09-11T23:21:00.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.066+0000 7efc632bb640 1 -- 172.21.15.43:0/3240701328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc5c103110 con 0x7efc5c102d30 2024-09-11T23:21:00.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.067+0000 7efc527fc640 1 -- 172.21.15.43:0/3240701328 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efc580305a0 con 0x7efc5c102d30 2024-09-11T23:21:00.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.068+0000 7efc527fc640 1 --2- 172.21.15.43:0/3240701328 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7efc480a2c90 0x7efc480a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:00.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.068+0000 7efc527fc640 1 -- 172.21.15.43:0/3240701328 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7efc5802e030 con 0x7efc5c102d30 2024-09-11T23:21:00.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.068+0000 7efc60856640 1 --2- 172.21.15.43:0/3240701328 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7efc480a2c90 0x7efc480a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:00.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.069+0000 7efc60856640 1 --2- 172.21.15.43:0/3240701328 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7efc480a2c90 0x7efc480a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efc5c0fee20 tx=0x7efc4c005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:00.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.072+0000 7efc527fc640 1 -- 172.21.15.43:0/3240701328 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc5804b060 con 0x7efc5c102d30 2024-09-11T23:21:00.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.250+0000 7efc632bb640 1 -- 172.21.15.43:0/3240701328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7efc5c10a2b0 con 0x7efc5c102d30 2024-09-11T23:21:00.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.250+0000 7efc527fc640 1 -- 172.21.15.43:0/3240701328 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efc58030890 con 0x7efc5c102d30 2024-09-11T23:21:00.250 INFO:teuthology.orchestra.run.smithi043.stdout:158913790008 2024-09-11T23:21:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.253+0000 7efc632bb640 1 -- 172.21.15.43:0/3240701328 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7efc480a2c90 msgr2=0x7efc480a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.253+0000 7efc632bb640 1 --2- 172.21.15.43:0/3240701328 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7efc480a2c90 0x7efc480a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efc5c0fee20 tx=0x7efc4c005e50 comp rx=0 tx=0).stop 2024-09-11T23:21:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.253+0000 7efc632bb640 1 -- 172.21.15.43:0/3240701328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c102d30 msgr2=0x7efc5c1985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.253+0000 7efc632bb640 1 --2- 172.21.15.43:0/3240701328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c102d30 0x7efc5c1985f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efc5802ae00 tx=0x7efc58003970 comp rx=0 tx=0).stop 2024-09-11T23:21:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.253+0000 7efc632bb640 1 -- 172.21.15.43:0/3240701328 shutdown_connections 2024-09-11T23:21:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.253+0000 7efc632bb640 1 --2- 172.21.15.43:0/3240701328 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7efc480a2c90 0x7efc480a5150 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.253+0000 7efc632bb640 1 --2- 172.21.15.43:0/3240701328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc5c1082a0 0x7efc5c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.253+0000 7efc632bb640 1 --2- 172.21.15.43:0/3240701328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc5c1036e0 0x7efc5c198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.253+0000 7efc632bb640 1 --2- 172.21.15.43:0/3240701328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c102d30 0x7efc5c1985f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.253+0000 7efc632bb640 1 -- 172.21.15.43:0/3240701328 >> 172.21.15.43:0/3240701328 conn(0x7efc5c0fc7d0 msgr2=0x7efc5c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:00.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.253+0000 7efc632bb640 1 -- 172.21.15.43:0/3240701328 shutdown_connections 2024-09-11T23:21:00.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:00.253+0000 7efc632bb640 1 -- 172.21.15.43:0/3240701328 wait complete. 2024-09-11T23:21:00.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:00 smithi043 ceph-mon[30950]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-11T23:21:00.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:00 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1607353687' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T23:21:00.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:00 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1497727782' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T23:21:00.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:00 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3240701328' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T23:21:00.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:00 smithi090 ceph-mon[35130]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-11T23:21:00.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:00 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1607353687' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T23:21:00.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:00 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1497727782' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T23:21:00.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:00 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3240701328' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T23:21:00.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:00 smithi097 ceph-mon[35562]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-11T23:21:00.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:00 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1607353687' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T23:21:00.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:00 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1497727782' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T23:21:00.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:00 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3240701328' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T23:21:01.456 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:01 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-11T23:21:01.456 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:01 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-11T23:21:01.456 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:01 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-09-11T23:21:01.456 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:01 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 7]}]: dispatch 2024-09-11T23:21:01.456 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:01 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-11T23:21:01.456 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:01 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-09-11T23:21:01.456 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:01 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-09-11T23:21:01.457 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:01 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-09-11T23:21:01.457 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:01 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-09-11T23:21:01.457 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:01 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-11T23:21:01.490 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514518 got 326417514520 for osd.11 2024-09-11T23:21:01.490 DEBUG:teuthology.parallel:result is None 2024-09-11T23:21:01.612 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790005 got 158913790008 for osd.5 2024-09-11T23:21:01.612 DEBUG:teuthology.parallel:result is None 2024-09-11T23:21:01.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:01 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-11T23:21:01.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:01 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-11T23:21:01.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:01 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-09-11T23:21:01.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:01 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 7]}]: dispatch 2024-09-11T23:21:01.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:01 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-11T23:21:01.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:01 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-09-11T23:21:01.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:01 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-09-11T23:21:01.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:01 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-09-11T23:21:01.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:01 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-09-11T23:21:01.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:01 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-11T23:21:01.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.526+0000 7f367155c640 1 -- 172.21.15.43:0/4292514534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f366c103660 msgr2=0x7f366c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:01.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.526+0000 7f367155c640 1 --2- 172.21.15.43:0/4292514534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f366c103660 0x7f366c10bb50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3660009900 tx=0x7f366002a480 comp rx=0 tx=0).stop 2024-09-11T23:21:01.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.526+0000 7f367155c640 1 -- 172.21.15.43:0/4292514534 shutdown_connections 2024-09-11T23:21:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.526+0000 7f367155c640 1 --2- 172.21.15.43:0/4292514534 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f366c10c170 0x7f366c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.526+0000 7f367155c640 1 --2- 172.21.15.43:0/4292514534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f366c103660 0x7f366c10bb50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.526+0000 7f367155c640 1 --2- 172.21.15.43:0/4292514534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c102d40 0x7f366c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.526+0000 7f367155c640 1 -- 172.21.15.43:0/4292514534 >> 172.21.15.43:0/4292514534 conn(0x7f366c0fc7a0 msgr2=0x7f366c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.526+0000 7f367155c640 1 -- 172.21.15.43:0/4292514534 shutdown_connections 2024-09-11T23:21:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.526+0000 7f367155c640 1 -- 172.21.15.43:0/4292514534 wait complete. 2024-09-11T23:21:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.527+0000 7f367155c640 1 Processor -- start 2024-09-11T23:21:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.528+0000 7f367155c640 1 -- start start 2024-09-11T23:21:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.528+0000 7f367155c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c102d40 0x7f366c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.528+0000 7f367155c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f366c103660 0x7f366c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.528+0000 7f367155c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f366c10c170 0x7f366c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:01.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.528+0000 7f367155c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f366c111f70 con 0x7f366c103660 2024-09-11T23:21:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.528+0000 7f367155c640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f366c111df0 con 0x7f366c10c170 2024-09-11T23:21:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.528+0000 7f367155c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f366c1120f0 con 0x7f366c102d40 2024-09-11T23:21:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.528+0000 7f366a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f366c103660 0x7f366c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.528+0000 7f366affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c102d40 0x7f366c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.528+0000 7f366a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f366c103660 0x7f366c100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38512/0 (socket says 172.21.15.43:38512) 2024-09-11T23:21:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.528+0000 7f366affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c102d40 0x7f366c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:60284/0 (socket says 172.21.15.43:60284) 2024-09-11T23:21:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.528+0000 7f366a7fc640 1 -- 172.21.15.43:0/859525523 learned_addr learned my addr 172.21.15.43:0/859525523 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.528+0000 7f366b7fe640 1 --2- 172.21.15.43:0/859525523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f366c10c170 0x7f366c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.528+0000 7f366affd640 1 -- 172.21.15.43:0/859525523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f366c10c170 msgr2=0x7f366c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.528+0000 7f366affd640 1 --2- 172.21.15.43:0/859525523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f366c10c170 0x7f366c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.528+0000 7f366affd640 1 -- 172.21.15.43:0/859525523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f366c103660 msgr2=0x7f366c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.528+0000 7f366affd640 1 --2- 172.21.15.43:0/859525523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f366c103660 0x7f366c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.528+0000 7f366affd640 1 -- 172.21.15.43:0/859525523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c1a69f0 con 0x7f366c102d40 2024-09-11T23:21:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.529+0000 7f366affd640 1 --2- 172.21.15.43:0/859525523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c102d40 0x7f366c1009a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f365c00e9f0 tx=0x7f365c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.529+0000 7f3653fff640 1 -- 172.21.15.43:0/859525523 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c00ccc0 con 0x7f366c102d40 2024-09-11T23:21:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.529+0000 7f367155c640 1 -- 172.21.15.43:0/859525523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366c1a6ce0 con 0x7f366c102d40 2024-09-11T23:21:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.529+0000 7f367155c640 1 -- 172.21.15.43:0/859525523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f366c1a71c0 con 0x7f366c102d40 2024-09-11T23:21:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.529+0000 7f3653fff640 1 -- 172.21.15.43:0/859525523 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f365c005020 con 0x7f366c102d40 2024-09-11T23:21:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.529+0000 7f3653fff640 1 -- 172.21.15.43:0/859525523 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c01c070 con 0x7f366c102d40 2024-09-11T23:21:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.530+0000 7f367155c640 1 -- 172.21.15.43:0/859525523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f366c103120 con 0x7f366c102d40 2024-09-11T23:21:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.531+0000 7f3653fff640 1 -- 172.21.15.43:0/859525523 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f365c005540 con 0x7f366c102d40 2024-09-11T23:21:01.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.531+0000 7f3653fff640 1 --2- 172.21.15.43:0/859525523 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f36540a2c10 0x7f36540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:01.794 INFO:teuthology.orchestra.run.smithi043.stdout:77309411400 2024-09-11T23:21:01.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.532+0000 7f366a7fc640 1 --2- 172.21.15.43:0/859525523 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f36540a2c10 0x7f36540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:01.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.532+0000 7f3653fff640 1 -- 172.21.15.43:0/859525523 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f365c0cf0d0 con 0x7f366c102d40 2024-09-11T23:21:01.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.532+0000 7f366a7fc640 1 --2- 172.21.15.43:0/859525523 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f36540a2c10 0x7f36540a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3660031550 tx=0x7f3660005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:01.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.536+0000 7f3653fff640 1 -- 172.21.15.43:0/859525523 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f365c09c000 con 0x7f366c102d40 2024-09-11T23:21:01.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.740+0000 7f367155c640 1 -- 172.21.15.43:0/859525523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f366c1071e0 con 0x7f366c102d40 2024-09-11T23:21:01.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.740+0000 7f3653fff640 1 -- 172.21.15.43:0/859525523 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f365c09b900 con 0x7f366c102d40 2024-09-11T23:21:01.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.743+0000 7f367155c640 1 -- 172.21.15.43:0/859525523 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f36540a2c10 msgr2=0x7f36540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:01.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.743+0000 7f367155c640 1 --2- 172.21.15.43:0/859525523 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f36540a2c10 0x7f36540a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3660031550 tx=0x7f3660005a20 comp rx=0 tx=0).stop 2024-09-11T23:21:01.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.743+0000 7f367155c640 1 -- 172.21.15.43:0/859525523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c102d40 msgr2=0x7f366c1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:01.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.743+0000 7f367155c640 1 --2- 172.21.15.43:0/859525523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c102d40 0x7f366c1009a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f365c00e9f0 tx=0x7f365c00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:21:01.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.743+0000 7f367155c640 1 -- 172.21.15.43:0/859525523 shutdown_connections 2024-09-11T23:21:01.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.743+0000 7f367155c640 1 --2- 172.21.15.43:0/859525523 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f36540a2c10 0x7f36540a50d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:01.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.743+0000 7f367155c640 1 --2- 172.21.15.43:0/859525523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f366c10c170 0x7f366c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:01.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.743+0000 7f367155c640 1 --2- 172.21.15.43:0/859525523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f366c103660 0x7f366c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:01.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.743+0000 7f367155c640 1 --2- 172.21.15.43:0/859525523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f366c102d40 0x7f366c1009a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:01.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.743+0000 7f367155c640 1 -- 172.21.15.43:0/859525523 >> 172.21.15.43:0/859525523 conn(0x7f366c0fc7a0 msgr2=0x7f366c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:01.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.743+0000 7f367155c640 1 -- 172.21.15.43:0/859525523 shutdown_connections 2024-09-11T23:21:01.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:20:58.743+0000 7f367155c640 1 -- 172.21.15.43:0/859525523 wait complete. 2024-09-11T23:21:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:01 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-11T23:21:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:01 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-11T23:21:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:01 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-09-11T23:21:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:01 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 7]}]: dispatch 2024-09-11T23:21:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:01 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-11T23:21:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:01 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-09-11T23:21:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:01 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-09-11T23:21:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:01 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-09-11T23:21:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:01 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-09-11T23:21:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:01 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-11T23:21:02.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:02 smithi043 ceph-mon[30950]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:21:02.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-11T23:21:02.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-11T23:21:02.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-11T23:21:02.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 7]}]': finished 2024-09-11T23:21:02.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-11T23:21:02.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-09-11T23:21:02.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-11T23:21:02.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-11T23:21:02.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-09-11T23:21:02.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:02 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-11T23:21:02.397 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:02 smithi043 ceph-mon[30950]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T23:21:02.434 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411398 got 77309411400 for osd.2 2024-09-11T23:21:02.435 DEBUG:teuthology.parallel:result is None 2024-09-11T23:21:02.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:02 smithi090 ceph-mon[35130]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:21:02.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:02 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-11T23:21:02.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:02 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-11T23:21:02.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:02 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-11T23:21:02.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:02 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 7]}]': finished 2024-09-11T23:21:02.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:02 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-11T23:21:02.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:02 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-09-11T23:21:02.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:02 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-11T23:21:02.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:02 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-11T23:21:02.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:02 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-09-11T23:21:02.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:02 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-11T23:21:02.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:02 smithi090 ceph-mon[35130]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T23:21:02.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:02 smithi097 ceph-mon[35562]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:21:02.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:02 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-11T23:21:02.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:02 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-11T23:21:02.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:02 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-11T23:21:02.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:02 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 7]}]': finished 2024-09-11T23:21:02.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:02 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-11T23:21:02.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:02 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-09-11T23:21:02.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:02 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-11T23:21:02.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:02 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-11T23:21:02.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:02 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-09-11T23:21:02.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:02 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-11T23:21:02.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:02 smithi097 ceph-mon[35562]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T23:21:02.841 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149758 got 111669149762 for osd.3 2024-09-11T23:21:02.841 DEBUG:teuthology.parallel:result is None 2024-09-11T23:21:03.313 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743450 got 296352743452 for osd.10 2024-09-11T23:21:03.313 DEBUG:teuthology.parallel:result is None 2024-09-11T23:21:03.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:03 smithi043 ceph-mon[30950]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T23:21:03.707 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574921 got 55834574924 for osd.1 2024-09-11T23:21:03.707 DEBUG:teuthology.parallel:result is None 2024-09-11T23:21:03.723 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:03.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:03 smithi090 ceph-mon[35130]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T23:21:03.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:03 smithi097 ceph-mon[35562]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T23:21:04.216 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738447 got 34359738449 for osd.0 2024-09-11T23:21:04.217 DEBUG:teuthology.parallel:result is None 2024-09-11T23:21:04.678 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:04 smithi043 ceph-mon[30950]: pgmap v315: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-11T23:21:04.678 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:04 smithi043 ceph-mon[30950]: Health check failed: Reduced data availability: 2 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-11T23:21:04.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:04 smithi090 ceph-mon[35130]: pgmap v315: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-11T23:21:04.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:04 smithi090 ceph-mon[35130]: Health check failed: Reduced data availability: 2 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-11T23:21:04.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.761+0000 7f5c51c9a640 1 -- 172.21.15.43:0/679433213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c4c102570 msgr2=0x7f5c4c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:04.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.761+0000 7f5c51c9a640 1 --2- 172.21.15.43:0/679433213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c4c102570 0x7f5c4c1029b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5c40009900 tx=0x7f5c4002a480 comp rx=0 tx=0).stop 2024-09-11T23:21:04.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.761+0000 7f5c51c9a640 1 -- 172.21.15.43:0/679433213 shutdown_connections 2024-09-11T23:21:04.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.761+0000 7f5c51c9a640 1 --2- 172.21.15.43:0/679433213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c4c102ef0 0x7f5c4c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:04.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.761+0000 7f5c51c9a640 1 --2- 172.21.15.43:0/679433213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c4c102570 0x7f5c4c1029b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:04.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.761+0000 7f5c51c9a640 1 --2- 172.21.15.43:0/679433213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c4c100e20 0x7f5c4c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:04.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.761+0000 7f5c51c9a640 1 -- 172.21.15.43:0/679433213 >> 172.21.15.43:0/679433213 conn(0x7f5c4c0fc7d0 msgr2=0x7f5c4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:04.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.761+0000 7f5c51c9a640 1 -- 172.21.15.43:0/679433213 shutdown_connections 2024-09-11T23:21:04.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.761+0000 7f5c51c9a640 1 -- 172.21.15.43:0/679433213 wait complete. 2024-09-11T23:21:04.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.762+0000 7f5c51c9a640 1 Processor -- start 2024-09-11T23:21:04.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.762+0000 7f5c51c9a640 1 -- start start 2024-09-11T23:21:04.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.762+0000 7f5c51c9a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c4c100e20 0x7f5c4c1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:04.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.762+0000 7f5c51c9a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c4c102570 0x7f5c4c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:04.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.763+0000 7f5c51c9a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c4c102ef0 0x7f5c4c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:04.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.763+0000 7f5c51c9a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c110f80 con 0x7f5c4c102570 2024-09-11T23:21:04.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.763+0000 7f5c51c9a640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c110e00 con 0x7f5c4c100e20 2024-09-11T23:21:04.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.763+0000 7f5c51c9a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c111100 con 0x7f5c4c102ef0 2024-09-11T23:21:04.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.763+0000 7f5c4affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c4c102570 0x7f5c4c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:04.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.763+0000 7f5c4affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c4c102570 0x7f5c4c198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59406/0 (socket says 172.21.15.43:59406) 2024-09-11T23:21:04.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.763+0000 7f5c4affd640 1 -- 172.21.15.43:0/2415190897 learned_addr learned my addr 172.21.15.43:0/2415190897 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:04.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.763+0000 7f5c4bfff640 1 --2- 172.21.15.43:0/2415190897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c4c102ef0 0x7f5c4c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:04.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.763+0000 7f5c4b7fe640 1 --2- 172.21.15.43:0/2415190897 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c4c100e20 0x7f5c4c1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:04.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.763+0000 7f5c4affd640 1 -- 172.21.15.43:0/2415190897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c4c102ef0 msgr2=0x7f5c4c19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:04.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.763+0000 7f5c4affd640 1 --2- 172.21.15.43:0/2415190897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c4c102ef0 0x7f5c4c19cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:04.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.763+0000 7f5c4affd640 1 -- 172.21.15.43:0/2415190897 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c4c100e20 msgr2=0x7f5c4c1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:04.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.763+0000 7f5c4affd640 1 --2- 172.21.15.43:0/2415190897 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c4c100e20 0x7f5c4c1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:04.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.763+0000 7f5c4affd640 1 -- 172.21.15.43:0/2415190897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4c19d4a0 con 0x7f5c4c102570 2024-09-11T23:21:04.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.763+0000 7f5c4affd640 1 --2- 172.21.15.43:0/2415190897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c4c102570 0x7f5c4c198a30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5c40006010 tx=0x7f5c40003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:04.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.763+0000 7f5c48ff9640 1 -- 172.21.15.43:0/2415190897 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4003f070 con 0x7f5c4c102570 2024-09-11T23:21:04.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.763+0000 7f5c48ff9640 1 -- 172.21.15.43:0/2415190897 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c40031c30 con 0x7f5c4c102570 2024-09-11T23:21:04.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.763+0000 7f5c51c9a640 1 -- 172.21.15.43:0/2415190897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c4c19d730 con 0x7f5c4c102570 2024-09-11T23:21:04.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.763+0000 7f5c48ff9640 1 -- 172.21.15.43:0/2415190897 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c40035030 con 0x7f5c4c102570 2024-09-11T23:21:04.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.763+0000 7f5c51c9a640 1 -- 172.21.15.43:0/2415190897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c4c1a5070 con 0x7f5c4c102570 2024-09-11T23:21:04.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.764+0000 7f5c51c9a640 1 -- 172.21.15.43:0/2415190897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c4c1021b0 con 0x7f5c4c102570 2024-09-11T23:21:04.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.766+0000 7f5c48ff9640 1 -- 172.21.15.43:0/2415190897 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5c40042070 con 0x7f5c4c102570 2024-09-11T23:21:04.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.766+0000 7f5c48ff9640 1 --2- 172.21.15.43:0/2415190897 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f5c340a2c10 0x7f5c340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:04.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.766+0000 7f5c4b7fe640 1 --2- 172.21.15.43:0/2415190897 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f5c340a2c10 0x7f5c340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:04.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.766+0000 7f5c48ff9640 1 -- 172.21.15.43:0/2415190897 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f5c400efd20 con 0x7f5c4c102570 2024-09-11T23:21:04.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.767+0000 7f5c4b7fe640 1 --2- 172.21.15.43:0/2415190897 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f5c340a2c10 0x7f5c340a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c005fd0 tx=0x7f5c3c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:04.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.770+0000 7f5c48ff9640 1 -- 172.21.15.43:0/2415190897 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c400bc4b0 con 0x7f5c4c102570 2024-09-11T23:21:04.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:04 smithi097 ceph-mon[35562]: pgmap v315: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-11T23:21:04.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:04 smithi097 ceph-mon[35562]: Health check failed: Reduced data availability: 2 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-11T23:21:04.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.951+0000 7f5c51c9a640 1 -- 172.21.15.43:0/2415190897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f5c4c1a5240 con 0x7f5c4c102570 2024-09-11T23:21:04.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.951+0000 7f5c48ff9640 1 -- 172.21.15.43:0/2415190897 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5c400c07b0 con 0x7f5c4c102570 2024-09-11T23:21:04.951 INFO:teuthology.orchestra.run.smithi043.stdout:210453397550 2024-09-11T23:21:04.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.953+0000 7f5c51c9a640 1 -- 172.21.15.43:0/2415190897 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f5c340a2c10 msgr2=0x7f5c340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:04.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.953+0000 7f5c51c9a640 1 --2- 172.21.15.43:0/2415190897 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f5c340a2c10 0x7f5c340a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c005fd0 tx=0x7f5c3c00b390 comp rx=0 tx=0).stop 2024-09-11T23:21:04.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.953+0000 7f5c51c9a640 1 -- 172.21.15.43:0/2415190897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c4c102570 msgr2=0x7f5c4c198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.953+0000 7f5c51c9a640 1 --2- 172.21.15.43:0/2415190897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c4c102570 0x7f5c4c198a30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5c40006010 tx=0x7f5c40003c10 comp rx=0 tx=0).stop 2024-09-11T23:21:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.953+0000 7f5c51c9a640 1 -- 172.21.15.43:0/2415190897 shutdown_connections 2024-09-11T23:21:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.953+0000 7f5c51c9a640 1 --2- 172.21.15.43:0/2415190897 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f5c340a2c10 0x7f5c340a50d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.953+0000 7f5c51c9a640 1 --2- 172.21.15.43:0/2415190897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c4c102ef0 0x7f5c4c19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.953+0000 7f5c51c9a640 1 --2- 172.21.15.43:0/2415190897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c4c102570 0x7f5c4c198a30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.953+0000 7f5c51c9a640 1 --2- 172.21.15.43:0/2415190897 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c4c100e20 0x7f5c4c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.953+0000 7f5c51c9a640 1 -- 172.21.15.43:0/2415190897 >> 172.21.15.43:0/2415190897 conn(0x7f5c4c0fc7d0 msgr2=0x7f5c4c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.953+0000 7f5c51c9a640 1 -- 172.21.15.43:0/2415190897 shutdown_connections 2024-09-11T23:21:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:04.953+0000 7f5c51c9a640 1 -- 172.21.15.43:0/2415190897 wait complete. 2024-09-11T23:21:05.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:05 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2415190897' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T23:21:05.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:05 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2415190897' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T23:21:05.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:05 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2415190897' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T23:21:06.066 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397546 got 210453397550 for osd.7 2024-09-11T23:21:06.067 DEBUG:teuthology.parallel:result is None 2024-09-11T23:21:06.067 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T23:21:06.067 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:21:06.352 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:06.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:06 smithi043 ceph-mon[30950]: pgmap v316: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-11T23:21:06.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:06 smithi090 ceph-mon[35130]: pgmap v316: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-11T23:21:06.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:06 smithi097 ceph-mon[35562]: pgmap v316: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-11T23:21:06.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.853+0000 7fce8e5d9640 1 -- 172.21.15.43:0/1842465295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce881036e0 msgr2=0x7fce88107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:06.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.853+0000 7fce8e5d9640 1 --2- 172.21.15.43:0/1842465295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce881036e0 0x7fce88107b90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fce7c009930 tx=0x7fce7c02a420 comp rx=0 tx=0).stop 2024-09-11T23:21:06.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.853+0000 7fce8e5d9640 1 -- 172.21.15.43:0/1842465295 shutdown_connections 2024-09-11T23:21:06.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.853+0000 7fce8e5d9640 1 --2- 172.21.15.43:0/1842465295 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce881082a0 0x7fce8810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.853+0000 7fce8e5d9640 1 --2- 172.21.15.43:0/1842465295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce881036e0 0x7fce88107b90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.853+0000 7fce8e5d9640 1 --2- 172.21.15.43:0/1842465295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce88102d30 0x7fce88103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.853+0000 7fce8e5d9640 1 -- 172.21.15.43:0/1842465295 >> 172.21.15.43:0/1842465295 conn(0x7fce880fc7d0 msgr2=0x7fce880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:06.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.853+0000 7fce8e5d9640 1 -- 172.21.15.43:0/1842465295 shutdown_connections 2024-09-11T23:21:06.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.853+0000 7fce8e5d9640 1 -- 172.21.15.43:0/1842465295 wait complete. 2024-09-11T23:21:06.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.854+0000 7fce8e5d9640 1 Processor -- start 2024-09-11T23:21:06.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.854+0000 7fce8e5d9640 1 -- start start 2024-09-11T23:21:06.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.854+0000 7fce8e5d9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce88102d30 0x7fce88198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.854+0000 7fce8e5d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce881036e0 0x7fce88198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.854+0000 7fce8e5d9640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce881082a0 0x7fce8819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.854+0000 7fce8e5d9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fce881120c0 con 0x7fce88102d30 2024-09-11T23:21:06.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.854+0000 7fce8e5d9640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fce88111f40 con 0x7fce881082a0 2024-09-11T23:21:06.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.854+0000 7fce8e5d9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fce88112240 con 0x7fce881036e0 2024-09-11T23:21:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.854+0000 7fce87fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce88102d30 0x7fce88198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.854+0000 7fce87fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce88102d30 0x7fce88198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59434/0 (socket says 172.21.15.43:59434) 2024-09-11T23:21:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.854+0000 7fce87fff640 1 -- 172.21.15.43:0/3376713356 learned_addr learned my addr 172.21.15.43:0/3376713356 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.854+0000 7fce877fe640 1 --2- 172.21.15.43:0/3376713356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce881036e0 0x7fce88198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.855+0000 7fce8cb76640 1 --2- 172.21.15.43:0/3376713356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce881082a0 0x7fce8819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.855+0000 7fce87fff640 1 -- 172.21.15.43:0/3376713356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce881036e0 msgr2=0x7fce88198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.855+0000 7fce87fff640 1 --2- 172.21.15.43:0/3376713356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce881036e0 0x7fce88198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.855+0000 7fce87fff640 1 -- 172.21.15.43:0/3376713356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce881082a0 msgr2=0x7fce8819cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.855+0000 7fce87fff640 1 --2- 172.21.15.43:0/3376713356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce881082a0 0x7fce8819cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.855+0000 7fce87fff640 1 -- 172.21.15.43:0/3376713356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8819d5d0 con 0x7fce88102d30 2024-09-11T23:21:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.855+0000 7fce8cb76640 1 --2- 172.21.15.43:0/3376713356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce881082a0 0x7fce8819cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.855+0000 7fce87fff640 1 --2- 172.21.15.43:0/3376713356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce88102d30 0x7fce88198620 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fce7800c890 tx=0x7fce7800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:06.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.855+0000 7fce857fa640 1 -- 172.21.15.43:0/3376713356 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78004220 con 0x7fce88102d30 2024-09-11T23:21:06.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.855+0000 7fce857fa640 1 -- 172.21.15.43:0/3376713356 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce780043c0 con 0x7fce88102d30 2024-09-11T23:21:06.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.855+0000 7fce8e5d9640 1 -- 172.21.15.43:0/3376713356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce8819d8c0 con 0x7fce88102d30 2024-09-11T23:21:06.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.855+0000 7fce857fa640 1 -- 172.21.15.43:0/3376713356 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780049e0 con 0x7fce88102d30 2024-09-11T23:21:06.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.855+0000 7fce8e5d9640 1 -- 172.21.15.43:0/3376713356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce881a51a0 con 0x7fce88102d30 2024-09-11T23:21:06.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.857+0000 7fce8e5d9640 1 -- 172.21.15.43:0/3376713356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce88103110 con 0x7fce88102d30 2024-09-11T23:21:06.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.858+0000 7fce857fa640 1 -- 172.21.15.43:0/3376713356 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fce780079b0 con 0x7fce88102d30 2024-09-11T23:21:06.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.858+0000 7fce857fa640 1 --2- 172.21.15.43:0/3376713356 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fce700a2c90 0x7fce700a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:06.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.858+0000 7fce877fe640 1 --2- 172.21.15.43:0/3376713356 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fce700a2c90 0x7fce700a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:06.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.858+0000 7fce857fa640 1 -- 172.21.15.43:0/3376713356 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7fce780cf270 con 0x7fce88102d30 2024-09-11T23:21:06.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.859+0000 7fce877fe640 1 --2- 172.21.15.43:0/3376713356 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fce700a2c90 0x7fce700a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fce7c005940 tx=0x7fce7c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:06.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:06.862+0000 7fce857fa640 1 -- 172.21.15.43:0/3376713356 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce78014030 con 0x7fce88102d30 2024-09-11T23:21:07.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:07.068+0000 7fce8e5d9640 1 -- 172.21.15.43:0/3376713356 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fce88062630 con 0x7fce700a2c90 2024-09-11T23:21:07.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:07.081+0000 7fce857fa640 1 -- 172.21.15.43:0/3376713356 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258927 (secure 0 0 0) 0x7fce88062630 con 0x7fce700a2c90 2024-09-11T23:21:07.082 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:21:07.085 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-11T23:21:07.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:07.084+0000 7fce8e5d9640 1 -- 172.21.15.43:0/3376713356 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fce700a2c90 msgr2=0x7fce700a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:07.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:07.084+0000 7fce8e5d9640 1 --2- 172.21.15.43:0/3376713356 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fce700a2c90 0x7fce700a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fce7c005940 tx=0x7fce7c0058d0 comp rx=0 tx=0).stop 2024-09-11T23:21:07.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:07.084+0000 7fce8e5d9640 1 -- 172.21.15.43:0/3376713356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce88102d30 msgr2=0x7fce88198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:07.084+0000 7fce8e5d9640 1 --2- 172.21.15.43:0/3376713356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce88102d30 0x7fce88198620 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fce7800c890 tx=0x7fce7800cd50 comp rx=0 tx=0).stop 2024-09-11T23:21:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:07.084+0000 7fce8e5d9640 1 -- 172.21.15.43:0/3376713356 shutdown_connections 2024-09-11T23:21:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:07.084+0000 7fce8e5d9640 1 --2- 172.21.15.43:0/3376713356 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fce700a2c90 0x7fce700a5150 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:07.084+0000 7fce8e5d9640 1 --2- 172.21.15.43:0/3376713356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce881082a0 0x7fce8819cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:07.084+0000 7fce8e5d9640 1 --2- 172.21.15.43:0/3376713356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce881036e0 0x7fce88198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:07.084+0000 7fce8e5d9640 1 --2- 172.21.15.43:0/3376713356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce88102d30 0x7fce88198620 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:07.084+0000 7fce8e5d9640 1 -- 172.21.15.43:0/3376713356 >> 172.21.15.43:0/3376713356 conn(0x7fce880fc7d0 msgr2=0x7fce88105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:07.084+0000 7fce8e5d9640 1 -- 172.21.15.43:0/3376713356 shutdown_connections 2024-09-11T23:21:07.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:07.084+0000 7fce8e5d9640 1 -- 172.21.15.43:0/3376713356 wait complete. 2024-09-11T23:21:07.792 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-11T23:21:06.648164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654954,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":474,"num_read_kb":355,"num_write":343,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":294,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368792,"kb_used_data":11148,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573032,"statfs":{"total":1151940427776,"available":1151562784768,"internally_reserved":0,"allocated":11415552,"data_stored":7233865,"data_compressed":169740,"data_compressed_allocated":221184,"data_compressed_original":1941504,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001905"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355548+0000","last_change":"2024-09-11T23:19:59.734378+0000","last_active":"2024-09-11T23:21:02.355548+0000","last_peered":"2024-09-11T23:21:02.355548+0000","last_clean":"2024-09-11T23:21:02.355548+0000","last_became_active":"2024-09-11T23:19:59.734067+0000","last_became_peered":"2024-09-11T23:19:59.734067+0000","last_unstale":"2024-09-11T23:21:02.355548+0000","last_undegraded":"2024-09-11T23:21:02.355548+0000","last_fullsized":"2024-09-11T23:21:02.355548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:28:18.384453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.433031+0000","last_change":"2024-09-11T23:19:59.733892+0000","last_active":"2024-09-11T23:21:02.433031+0000","last_peered":"2024-09-11T23:21:02.433031+0000","last_clean":"2024-09-11T23:21:02.433031+0000","last_became_active":"2024-09-11T23:19:59.733540+0000","last_became_peered":"2024-09-11T23:19:59.733540+0000","last_unstale":"2024-09-11T23:21:02.433031+0000","last_undegraded":"2024-09-11T23:21:02.433031+0000","last_fullsized":"2024-09-11T23:21:02.433031+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:42.186458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.433126+0000","last_change":"2024-09-11T23:19:59.739443+0000","last_active":"2024-09-11T23:21:02.433126+0000","last_peered":"2024-09-11T23:21:02.433126+0000","last_clean":"2024-09-11T23:21:02.433126+0000","last_became_active":"2024-09-11T23:19:59.739025+0000","last_became_peered":"2024-09-11T23:19:59.739025+0000","last_unstale":"2024-09-11T23:21:02.433126+0000","last_undegraded":"2024-09-11T23:21:02.433126+0000","last_fullsized":"2024-09-11T23:21:02.433126+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:40.819321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.363511+0000","last_change":"2024-09-11T23:21:02.363511+0000","last_active":"2024-09-11T23:21:02.363511+0000","last_peered":"2024-09-11T23:21:02.363511+0000","last_clean":"2024-09-11T23:21:02.363511+0000","last_became_active":"2024-09-11T23:21:02.363156+0000","last_became_peered":"2024-09-11T23:21:02.363156+0000","last_unstale":"2024-09-11T23:21:02.363511+0000","last_undegraded":"2024-09-11T23:21:02.363511+0000","last_fullsized":"2024-09-11T23:21:02.363511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:17.059509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.358522+0000","last_change":"2024-09-11T23:19:59.738662+0000","last_active":"2024-09-11T23:21:02.358522+0000","last_peered":"2024-09-11T23:21:02.358522+0000","last_clean":"2024-09-11T23:21:02.358522+0000","last_became_active":"2024-09-11T23:19:59.738320+0000","last_became_peered":"2024-09-11T23:19:59.738320+0000","last_unstale":"2024-09-11T23:21:02.358522+0000","last_undegraded":"2024-09-11T23:21:02.358522+0000","last_fullsized":"2024-09-11T23:21:02.358522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:15.645179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:21:01.359159+0000","last_change":"2024-09-11T23:19:59.731442+0000","last_active":"2024-09-11T23:21:01.359159+0000","last_peered":"2024-09-11T23:21:01.359159+0000","last_clean":"2024-09-11T23:21:01.359159+0000","last_became_active":"2024-09-11T23:19:59.731079+0000","last_became_peered":"2024-09-11T23:19:59.731079+0000","last_unstale":"2024-09-11T23:21:01.359159+0000","last_undegraded":"2024-09-11T23:21:01.359159+0000","last_fullsized":"2024-09-11T23:21:01.359159+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:32.764991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.359992+0000","last_change":"2024-09-11T23:21:02.359992+0000","last_active":"2024-09-11T23:21:02.359992+0000","last_peered":"2024-09-11T23:21:02.359992+0000","last_clean":"2024-09-11T23:21:02.359992+0000","last_became_active":"2024-09-11T23:21:02.359564+0000","last_became_peered":"2024-09-11T23:21:02.359564+0000","last_unstale":"2024-09-11T23:21:02.359992+0000","last_undegraded":"2024-09-11T23:21:02.359992+0000","last_fullsized":"2024-09-11T23:21:02.359992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:53.633219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.354781+0000","last_change":"2024-09-11T23:19:59.740733+0000","last_active":"2024-09-11T23:21:02.354781+0000","last_peered":"2024-09-11T23:21:02.354781+0000","last_clean":"2024-09-11T23:21:02.354781+0000","last_became_active":"2024-09-11T23:19:59.740438+0000","last_became_peered":"2024-09-11T23:19:59.740438+0000","last_unstale":"2024-09-11T23:21:02.354781+0000","last_undegraded":"2024-09-11T23:21:02.354781+0000","last_fullsized":"2024-09-11T23:21:02.354781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:27.063305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.358498+0000","last_change":"2024-09-11T23:19:59.742439+0000","last_active":"2024-09-11T23:21:02.358498+0000","last_peered":"2024-09-11T23:21:02.358498+0000","last_clean":"2024-09-11T23:21:02.358498+0000","last_became_active":"2024-09-11T23:19:59.741983+0000","last_became_peered":"2024-09-11T23:19:59.741983+0000","last_unstale":"2024-09-11T23:21:02.358498+0000","last_undegraded":"2024-09-11T23:21:02.358498+0000","last_fullsized":"2024-09-11T23:21:02.358498+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:39.477167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:21:01.359519+0000","last_change":"2024-09-11T23:19:59.732544+0000","last_active":"2024-09-11T23:21:01.359519+0000","last_peered":"2024-09-11T23:21:01.359519+0000","last_clean":"2024-09-11T23:21:01.359519+0000","last_became_active":"2024-09-11T23:19:59.732205+0000","last_became_peered":"2024-09-11T23:19:59.732205+0000","last_unstale":"2024-09-11T23:21:01.359519+0000","last_undegraded":"2024-09-11T23:21:01.359519+0000","last_fullsized":"2024-09-11T23:21:01.359519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:22.494344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355822+0000","last_change":"2024-09-11T23:19:59.741857+0000","last_active":"2024-09-11T23:21:02.355822+0000","last_peered":"2024-09-11T23:21:02.355822+0000","last_clean":"2024-09-11T23:21:02.355822+0000","last_became_active":"2024-09-11T23:19:59.741484+0000","last_became_peered":"2024-09-11T23:19:59.741484+0000","last_unstale":"2024-09-11T23:21:02.355822+0000","last_undegraded":"2024-09-11T23:21:02.355822+0000","last_fullsized":"2024-09-11T23:21:02.355822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:52.998258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.433280+0000","last_change":"2024-09-11T23:19:59.736042+0000","last_active":"2024-09-11T23:21:02.433280+0000","last_peered":"2024-09-11T23:21:02.433280+0000","last_clean":"2024-09-11T23:21:02.433280+0000","last_became_active":"2024-09-11T23:19:59.735731+0000","last_became_peered":"2024-09-11T23:19:59.735731+0000","last_unstale":"2024-09-11T23:21:02.433280+0000","last_undegraded":"2024-09-11T23:21:02.433280+0000","last_fullsized":"2024-09-11T23:21:02.433280+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:19.085804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352541+0000","last_change":"2024-09-11T23:19:59.742049+0000","last_active":"2024-09-11T23:21:02.352541+0000","last_peered":"2024-09-11T23:21:02.352541+0000","last_clean":"2024-09-11T23:21:02.352541+0000","last_became_active":"2024-09-11T23:19:59.740974+0000","last_became_peered":"2024-09-11T23:19:59.740974+0000","last_unstale":"2024-09-11T23:21:02.352541+0000","last_undegraded":"2024-09-11T23:21:02.352541+0000","last_fullsized":"2024-09-11T23:21:02.352541+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:10.443113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.432709+0000","last_change":"2024-09-11T23:19:59.734270+0000","last_active":"2024-09-11T23:21:02.432709+0000","last_peered":"2024-09-11T23:21:02.432709+0000","last_clean":"2024-09-11T23:21:02.432709+0000","last_became_active":"2024-09-11T23:19:59.733967+0000","last_became_peered":"2024-09-11T23:19:59.733967+0000","last_unstale":"2024-09-11T23:21:02.432709+0000","last_undegraded":"2024-09-11T23:21:02.432709+0000","last_fullsized":"2024-09-11T23:21:02.432709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:02.185757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.432580+0000","last_change":"2024-09-11T23:19:59.734152+0000","last_active":"2024-09-11T23:21:02.432580+0000","last_peered":"2024-09-11T23:21:02.432580+0000","last_clean":"2024-09-11T23:21:02.432580+0000","last_became_active":"2024-09-11T23:19:59.733163+0000","last_became_peered":"2024-09-11T23:19:59.733163+0000","last_unstale":"2024-09-11T23:21:02.432580+0000","last_undegraded":"2024-09-11T23:21:02.432580+0000","last_fullsized":"2024-09-11T23:21:02.432580+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:22.370591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.357557+0000","last_change":"2024-09-11T23:19:59.735517+0000","last_active":"2024-09-11T23:21:02.357557+0000","last_peered":"2024-09-11T23:21:02.357557+0000","last_clean":"2024-09-11T23:21:02.357557+0000","last_became_active":"2024-09-11T23:19:59.735182+0000","last_became_peered":"2024-09-11T23:19:59.735182+0000","last_unstale":"2024-09-11T23:21:02.357557+0000","last_undegraded":"2024-09-11T23:21:02.357557+0000","last_fullsized":"2024-09-11T23:21:02.357557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:25.989416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355566+0000","last_change":"2024-09-11T23:19:59.733579+0000","last_active":"2024-09-11T23:21:02.355566+0000","last_peered":"2024-09-11T23:21:02.355566+0000","last_clean":"2024-09-11T23:21:02.355566+0000","last_became_active":"2024-09-11T23:19:59.733245+0000","last_became_peered":"2024-09-11T23:19:59.733245+0000","last_unstale":"2024-09-11T23:21:02.355566+0000","last_undegraded":"2024-09-11T23:21:02.355566+0000","last_fullsized":"2024-09-11T23:21:02.355566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:22.361158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.356089+0000","last_change":"2024-09-11T23:19:59.734354+0000","last_active":"2024-09-11T23:21:02.356089+0000","last_peered":"2024-09-11T23:21:02.356089+0000","last_clean":"2024-09-11T23:21:02.356089+0000","last_became_active":"2024-09-11T23:19:59.733988+0000","last_became_peered":"2024-09-11T23:19:59.733988+0000","last_unstale":"2024-09-11T23:21:02.356089+0000","last_undegraded":"2024-09-11T23:21:02.356089+0000","last_fullsized":"2024-09-11T23:21:02.356089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:41.401496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.360420+0000","last_change":"2024-09-11T23:19:59.730318+0000","last_active":"2024-09-11T23:21:02.360420+0000","last_peered":"2024-09-11T23:21:02.360420+0000","last_clean":"2024-09-11T23:21:02.360420+0000","last_became_active":"2024-09-11T23:19:59.730009+0000","last_became_peered":"2024-09-11T23:19:59.730009+0000","last_unstale":"2024-09-11T23:21:02.360420+0000","last_undegraded":"2024-09-11T23:21:02.360420+0000","last_fullsized":"2024-09-11T23:21:02.360420+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:01:38.531967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355062+0000","last_change":"2024-09-11T23:19:59.737254+0000","last_active":"2024-09-11T23:21:02.355062+0000","last_peered":"2024-09-11T23:21:02.355062+0000","last_clean":"2024-09-11T23:21:02.355062+0000","last_became_active":"2024-09-11T23:19:59.736936+0000","last_became_peered":"2024-09-11T23:19:59.736936+0000","last_unstale":"2024-09-11T23:21:02.355062+0000","last_undegraded":"2024-09-11T23:21:02.355062+0000","last_fullsized":"2024-09-11T23:21:02.355062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:00.361468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352632+0000","last_change":"2024-09-11T23:19:59.741617+0000","last_active":"2024-09-11T23:21:02.352632+0000","last_peered":"2024-09-11T23:21:02.352632+0000","last_clean":"2024-09-11T23:21:02.352632+0000","last_became_active":"2024-09-11T23:19:59.740358+0000","last_became_peered":"2024-09-11T23:19:59.740358+0000","last_unstale":"2024-09-11T23:21:02.352632+0000","last_undegraded":"2024-09-11T23:21:02.352632+0000","last_fullsized":"2024-09-11T23:21:02.352632+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:18.541105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.358132+0000","last_change":"2024-09-11T23:19:59.736437+0000","last_active":"2024-09-11T23:21:02.358132+0000","last_peered":"2024-09-11T23:21:02.358132+0000","last_clean":"2024-09-11T23:21:02.358132+0000","last_became_active":"2024-09-11T23:19:59.736029+0000","last_became_peered":"2024-09-11T23:19:59.736029+0000","last_unstale":"2024-09-11T23:21:02.358132+0000","last_undegraded":"2024-09-11T23:21:02.358132+0000","last_fullsized":"2024-09-11T23:21:02.358132+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:29.226939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355141+0000","last_change":"2024-09-11T23:19:59.736104+0000","last_active":"2024-09-11T23:21:02.355141+0000","last_peered":"2024-09-11T23:21:02.355141+0000","last_clean":"2024-09-11T23:21:02.355141+0000","last_became_active":"2024-09-11T23:19:59.735760+0000","last_became_peered":"2024-09-11T23:19:59.735760+0000","last_unstale":"2024-09-11T23:21:02.355141+0000","last_undegraded":"2024-09-11T23:21:02.355141+0000","last_fullsized":"2024-09-11T23:21:02.355141+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:56.554313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352652+0000","last_change":"2024-09-11T23:19:59.735455+0000","last_active":"2024-09-11T23:21:02.352652+0000","last_peered":"2024-09-11T23:21:02.352652+0000","last_clean":"2024-09-11T23:21:02.352652+0000","last_became_active":"2024-09-11T23:19:59.735078+0000","last_became_peered":"2024-09-11T23:19:59.735078+0000","last_unstale":"2024-09-11T23:21:02.352652+0000","last_undegraded":"2024-09-11T23:21:02.352652+0000","last_fullsized":"2024-09-11T23:21:02.352652+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:21:16.336006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355992+0000","last_change":"2024-09-11T23:19:59.745997+0000","last_active":"2024-09-11T23:21:02.355992+0000","last_peered":"2024-09-11T23:21:02.355992+0000","last_clean":"2024-09-11T23:21:02.355992+0000","last_became_active":"2024-09-11T23:19:59.745715+0000","last_became_peered":"2024-09-11T23:19:59.745715+0000","last_unstale":"2024-09-11T23:21:02.355992+0000","last_undegraded":"2024-09-11T23:21:02.355992+0000","last_fullsized":"2024-09-11T23:21:02.355992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:34.776229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.361426+0000","last_change":"2024-09-11T23:19:59.734181+0000","last_active":"2024-09-11T23:21:02.361426+0000","last_peered":"2024-09-11T23:21:02.361426+0000","last_clean":"2024-09-11T23:21:02.361426+0000","last_became_active":"2024-09-11T23:19:59.733793+0000","last_became_peered":"2024-09-11T23:19:59.733793+0000","last_unstale":"2024-09-11T23:21:02.361426+0000","last_undegraded":"2024-09-11T23:21:02.361426+0000","last_fullsized":"2024-09-11T23:21:02.361426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:36.906308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355383+0000","last_change":"2024-09-11T23:19:59.738713+0000","last_active":"2024-09-11T23:21:02.355383+0000","last_peered":"2024-09-11T23:21:02.355383+0000","last_clean":"2024-09-11T23:21:02.355383+0000","last_became_active":"2024-09-11T23:19:59.737579+0000","last_became_peered":"2024-09-11T23:19:59.737579+0000","last_unstale":"2024-09-11T23:21:02.355383+0000","last_undegraded":"2024-09-11T23:21:02.355383+0000","last_fullsized":"2024-09-11T23:21:02.355383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:47.135775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352455+0000","last_change":"2024-09-11T23:19:59.741381+0000","last_active":"2024-09-11T23:21:02.352455+0000","last_peered":"2024-09-11T23:21:02.352455+0000","last_clean":"2024-09-11T23:21:02.352455+0000","last_became_active":"2024-09-11T23:19:59.740914+0000","last_became_peered":"2024-09-11T23:19:59.740914+0000","last_unstale":"2024-09-11T23:21:02.352455+0000","last_undegraded":"2024-09-11T23:21:02.352455+0000","last_fullsized":"2024-09-11T23:21:02.352455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:24:04.365704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.433019+0000","last_change":"2024-09-11T23:19:59.736840+0000","last_active":"2024-09-11T23:21:02.433019+0000","last_peered":"2024-09-11T23:21:02.433019+0000","last_clean":"2024-09-11T23:21:02.433019+0000","last_became_active":"2024-09-11T23:19:59.736554+0000","last_became_peered":"2024-09-11T23:19:59.736554+0000","last_unstale":"2024-09-11T23:21:02.433019+0000","last_undegraded":"2024-09-11T23:21:02.433019+0000","last_fullsized":"2024-09-11T23:21:02.433019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:48.508369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.359012+0000","last_change":"2024-09-11T23:19:59.735719+0000","last_active":"2024-09-11T23:21:02.359012+0000","last_peered":"2024-09-11T23:21:02.359012+0000","last_clean":"2024-09-11T23:21:02.359012+0000","last_became_active":"2024-09-11T23:19:59.735397+0000","last_became_peered":"2024-09-11T23:19:59.735397+0000","last_unstale":"2024-09-11T23:21:02.359012+0000","last_undegraded":"2024-09-11T23:21:02.359012+0000","last_fullsized":"2024-09-11T23:21:02.359012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:26:19.392730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.361161+0000","last_change":"2024-09-11T23:19:59.737704+0000","last_active":"2024-09-11T23:21:02.361161+0000","last_peered":"2024-09-11T23:21:02.361161+0000","last_clean":"2024-09-11T23:21:02.361161+0000","last_became_active":"2024-09-11T23:19:59.737176+0000","last_became_peered":"2024-09-11T23:19:59.737176+0000","last_unstale":"2024-09-11T23:21:02.361161+0000","last_undegraded":"2024-09-11T23:21:02.361161+0000","last_fullsized":"2024-09-11T23:21:02.361161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:22.938171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.354986+0000","last_change":"2024-09-11T23:19:59.735499+0000","last_active":"2024-09-11T23:21:02.354986+0000","last_peered":"2024-09-11T23:21:02.354986+0000","last_clean":"2024-09-11T23:21:02.354986+0000","last_became_active":"2024-09-11T23:19:59.735088+0000","last_became_peered":"2024-09-11T23:19:59.735088+0000","last_unstale":"2024-09-11T23:21:02.354986+0000","last_undegraded":"2024-09-11T23:21:02.354986+0000","last_fullsized":"2024-09-11T23:21:02.354986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:12.633104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.358475+0000","last_change":"2024-09-11T23:19:59.743972+0000","last_active":"2024-09-11T23:21:02.358475+0000","last_peered":"2024-09-11T23:21:02.358475+0000","last_clean":"2024-09-11T23:21:02.358475+0000","last_became_active":"2024-09-11T23:19:59.743675+0000","last_became_peered":"2024-09-11T23:19:59.743675+0000","last_unstale":"2024-09-11T23:21:02.358475+0000","last_undegraded":"2024-09-11T23:21:02.358475+0000","last_fullsized":"2024-09-11T23:21:02.358475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:46.645696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":82,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.353055+0000","last_change":"2024-09-11T23:19:57.615472+0000","last_active":"2024-09-11T23:21:02.353055+0000","last_peered":"2024-09-11T23:21:02.353055+0000","last_clean":"2024-09-11T23:21:02.353055+0000","last_became_active":"2024-09-11T23:19:57.615104+0000","last_became_peered":"2024-09-11T23:19:57.615104+0000","last_unstale":"2024-09-11T23:21:02.353055+0000","last_undegraded":"2024-09-11T23:21:02.353055+0000","last_fullsized":"2024-09-11T23:21:02.353055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:03.601363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352907+0000","last_change":"2024-09-11T23:19:59.741027+0000","last_active":"2024-09-11T23:21:02.352907+0000","last_peered":"2024-09-11T23:21:02.352907+0000","last_clean":"2024-09-11T23:21:02.352907+0000","last_became_active":"2024-09-11T23:19:59.740681+0000","last_became_peered":"2024-09-11T23:19:59.740681+0000","last_unstale":"2024-09-11T23:21:02.352907+0000","last_undegraded":"2024-09-11T23:21:02.352907+0000","last_fullsized":"2024-09-11T23:21:02.352907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:36.948147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:21:01.359327+0000","last_change":"2024-09-11T23:19:57.612730+0000","last_active":"2024-09-11T23:21:01.359327+0000","last_peered":"2024-09-11T23:21:01.359327+0000","last_clean":"2024-09-11T23:21:01.359327+0000","last_became_active":"2024-09-11T23:19:57.612369+0000","last_became_peered":"2024-09-11T23:19:57.612369+0000","last_unstale":"2024-09-11T23:21:01.359327+0000","last_undegraded":"2024-09-11T23:21:01.359327+0000","last_fullsized":"2024-09-11T23:21:01.359327+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:17.823616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352175+0000","last_change":"2024-09-11T23:19:57.611470+0000","last_active":"2024-09-11T23:21:02.352175+0000","last_peered":"2024-09-11T23:21:02.352175+0000","last_clean":"2024-09-11T23:21:02.352175+0000","last_became_active":"2024-09-11T23:19:57.611141+0000","last_became_peered":"2024-09-11T23:19:57.611141+0000","last_unstale":"2024-09-11T23:21:02.352175+0000","last_undegraded":"2024-09-11T23:21:02.352175+0000","last_fullsized":"2024-09-11T23:21:02.352175+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:43.173955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.354859+0000","last_change":"2024-09-11T23:19:59.732375+0000","last_active":"2024-09-11T23:21:02.354859+0000","last_peered":"2024-09-11T23:21:02.354859+0000","last_clean":"2024-09-11T23:21:02.354859+0000","last_became_active":"2024-09-11T23:19:59.731793+0000","last_became_peered":"2024-09-11T23:19:59.731793+0000","last_unstale":"2024-09-11T23:21:02.354859+0000","last_undegraded":"2024-09-11T23:21:02.354859+0000","last_fullsized":"2024-09-11T23:21:02.354859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:02:47.278646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.353064+0000","last_change":"2024-09-11T23:19:57.614062+0000","last_active":"2024-09-11T23:21:02.353064+0000","last_peered":"2024-09-11T23:21:02.353064+0000","last_clean":"2024-09-11T23:21:02.353064+0000","last_became_active":"2024-09-11T23:19:57.613749+0000","last_became_peered":"2024-09-11T23:19:57.613749+0000","last_unstale":"2024-09-11T23:21:02.353064+0000","last_undegraded":"2024-09-11T23:21:02.353064+0000","last_fullsized":"2024-09-11T23:21:02.353064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:45:22.299379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.356130+0000","last_change":"2024-09-11T23:19:59.730758+0000","last_active":"2024-09-11T23:21:02.356130+0000","last_peered":"2024-09-11T23:21:02.356130+0000","last_clean":"2024-09-11T23:21:02.356130+0000","last_became_active":"2024-09-11T23:19:59.730423+0000","last_became_peered":"2024-09-11T23:19:59.730423+0000","last_unstale":"2024-09-11T23:21:02.356130+0000","last_undegraded":"2024-09-11T23:21:02.356130+0000","last_fullsized":"2024-09-11T23:21:02.356130+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:30.679451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":74,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:21:01.359507+0000","last_change":"2024-09-11T23:19:57.615574+0000","last_active":"2024-09-11T23:21:01.359507+0000","last_peered":"2024-09-11T23:21:01.359507+0000","last_clean":"2024-09-11T23:21:01.359507+0000","last_became_active":"2024-09-11T23:19:57.615192+0000","last_became_peered":"2024-09-11T23:19:57.615192+0000","last_unstale":"2024-09-11T23:21:01.359507+0000","last_undegraded":"2024-09-11T23:21:01.359507+0000","last_fullsized":"2024-09-11T23:21:01.359507+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:29.186899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.358363+0000","last_change":"2024-09-11T23:19:59.738433+0000","last_active":"2024-09-11T23:21:02.358363+0000","last_peered":"2024-09-11T23:21:02.358363+0000","last_clean":"2024-09-11T23:21:02.358363+0000","last_became_active":"2024-09-11T23:19:59.738081+0000","last_became_peered":"2024-09-11T23:19:59.738081+0000","last_unstale":"2024-09-11T23:21:02.358363+0000","last_undegraded":"2024-09-11T23:21:02.358363+0000","last_fullsized":"2024-09-11T23:21:02.358363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:10:46.786454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.437498+0000","last_change":"2024-09-11T23:21:02.437498+0000","last_active":"2024-09-11T23:21:02.437498+0000","last_peered":"2024-09-11T23:21:02.437498+0000","last_clean":"2024-09-11T23:21:02.437498+0000","last_became_active":"2024-09-11T23:21:02.434895+0000","last_became_peered":"2024-09-11T23:21:02.434895+0000","last_unstale":"2024-09-11T23:21:02.437498+0000","last_undegraded":"2024-09-11T23:21:02.437498+0000","last_fullsized":"2024-09-11T23:21:02.437498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:33.142041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.433161+0000","last_change":"2024-09-11T23:19:59.741081+0000","last_active":"2024-09-11T23:21:02.433161+0000","last_peered":"2024-09-11T23:21:02.433161+0000","last_clean":"2024-09-11T23:21:02.433161+0000","last_became_active":"2024-09-11T23:19:59.739984+0000","last_became_peered":"2024-09-11T23:19:59.739984+0000","last_unstale":"2024-09-11T23:21:02.433161+0000","last_undegraded":"2024-09-11T23:21:02.433161+0000","last_fullsized":"2024-09-11T23:21:02.433161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:03.513121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.353300+0000","last_change":"2024-09-11T23:19:57.823272+0000","last_active":"2024-09-11T23:21:02.353300+0000","last_peered":"2024-09-11T23:21:02.353300+0000","last_clean":"2024-09-11T23:21:02.353300+0000","last_became_active":"2024-09-11T23:19:57.822957+0000","last_became_peered":"2024-09-11T23:19:57.822957+0000","last_unstale":"2024-09-11T23:21:02.353300+0000","last_undegraded":"2024-09-11T23:21:02.353300+0000","last_fullsized":"2024-09-11T23:21:02.353300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:06.779385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.357966+0000","last_change":"2024-09-11T23:19:59.738398+0000","last_active":"2024-09-11T23:21:02.357966+0000","last_peered":"2024-09-11T23:21:02.357966+0000","last_clean":"2024-09-11T23:21:02.357966+0000","last_became_active":"2024-09-11T23:19:59.738042+0000","last_became_peered":"2024-09-11T23:19:59.738042+0000","last_unstale":"2024-09-11T23:21:02.357966+0000","last_undegraded":"2024-09-11T23:21:02.357966+0000","last_fullsized":"2024-09-11T23:21:02.357966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:05.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'26","reported_seq":155,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:21:01.359641+0000","last_change":"2024-09-11T23:19:57.615036+0000","last_active":"2024-09-11T23:21:01.359641+0000","last_peered":"2024-09-11T23:21:01.359641+0000","last_clean":"2024-09-11T23:21:01.359641+0000","last_became_active":"2024-09-11T23:19:57.614641+0000","last_became_peered":"2024-09-11T23:19:57.614641+0000","last_unstale":"2024-09-11T23:21:01.359641+0000","last_undegraded":"2024-09-11T23:21:01.359641+0000","last_fullsized":"2024-09-11T23:21:01.359641+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:05.430508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60349,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":109,"num_read_kb":113,"num_write":27,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.359507+0000","last_change":"2024-09-11T23:19:59.741597+0000","last_active":"2024-09-11T23:21:02.359507+0000","last_peered":"2024-09-11T23:21:02.359507+0000","last_clean":"2024-09-11T23:21:02.359507+0000","last_became_active":"2024-09-11T23:19:59.741296+0000","last_became_peered":"2024-09-11T23:19:59.741296+0000","last_unstale":"2024-09-11T23:21:02.359507+0000","last_undegraded":"2024-09-11T23:21:02.359507+0000","last_fullsized":"2024-09-11T23:21:02.359507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:06:20.742916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.361428+0000","last_change":"2024-09-11T23:19:57.617550+0000","last_active":"2024-09-11T23:21:02.361428+0000","last_peered":"2024-09-11T23:21:02.361428+0000","last_clean":"2024-09-11T23:21:02.361428+0000","last_became_active":"2024-09-11T23:19:57.617211+0000","last_became_peered":"2024-09-11T23:19:57.617211+0000","last_unstale":"2024-09-11T23:21:02.361428+0000","last_undegraded":"2024-09-11T23:21:02.361428+0000","last_fullsized":"2024-09-11T23:21:02.361428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:19.707742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352215+0000","last_change":"2024-09-11T23:19:59.742070+0000","last_active":"2024-09-11T23:21:02.352215+0000","last_peered":"2024-09-11T23:21:02.352215+0000","last_clean":"2024-09-11T23:21:02.352215+0000","last_became_active":"2024-09-11T23:19:59.739367+0000","last_became_peered":"2024-09-11T23:19:59.739367+0000","last_unstale":"2024-09-11T23:21:02.352215+0000","last_undegraded":"2024-09-11T23:21:02.352215+0000","last_fullsized":"2024-09-11T23:21:02.352215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:27.850136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352872+0000","last_change":"2024-09-11T23:19:57.618054+0000","last_active":"2024-09-11T23:21:02.352872+0000","last_peered":"2024-09-11T23:21:02.352872+0000","last_clean":"2024-09-11T23:21:02.352872+0000","last_became_active":"2024-09-11T23:19:57.617702+0000","last_became_peered":"2024-09-11T23:19:57.617702+0000","last_unstale":"2024-09-11T23:21:02.352872+0000","last_undegraded":"2024-09-11T23:21:02.352872+0000","last_fullsized":"2024-09-11T23:21:02.352872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:40.171851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355453+0000","last_change":"2024-09-11T23:19:59.744000+0000","last_active":"2024-09-11T23:21:02.355453+0000","last_peered":"2024-09-11T23:21:02.355453+0000","last_clean":"2024-09-11T23:21:02.355453+0000","last_became_active":"2024-09-11T23:19:59.743629+0000","last_became_peered":"2024-09-11T23:19:59.743629+0000","last_unstale":"2024-09-11T23:21:02.355453+0000","last_undegraded":"2024-09-11T23:21:02.355453+0000","last_fullsized":"2024-09-11T23:21:02.355453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:20:04.133258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":82,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.433487+0000","last_change":"2024-09-11T23:19:57.615117+0000","last_active":"2024-09-11T23:21:02.433487+0000","last_peered":"2024-09-11T23:21:02.433487+0000","last_clean":"2024-09-11T23:21:02.433487+0000","last_became_active":"2024-09-11T23:19:57.614758+0000","last_became_peered":"2024-09-11T23:19:57.614758+0000","last_unstale":"2024-09-11T23:21:02.433487+0000","last_undegraded":"2024-09-11T23:21:02.433487+0000","last_fullsized":"2024-09-11T23:21:02.433487+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:58.113986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.354969+0000","last_change":"2024-09-11T23:19:59.739403+0000","last_active":"2024-09-11T23:21:02.354969+0000","last_peered":"2024-09-11T23:21:02.354969+0000","last_clean":"2024-09-11T23:21:02.354969+0000","last_became_active":"2024-09-11T23:19:59.739078+0000","last_became_peered":"2024-09-11T23:19:59.739078+0000","last_unstale":"2024-09-11T23:21:02.354969+0000","last_undegraded":"2024-09-11T23:21:02.354969+0000","last_fullsized":"2024-09-11T23:21:02.354969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:46.168720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.358347+0000","last_change":"2024-09-11T23:19:57.823323+0000","last_active":"2024-09-11T23:21:02.358347+0000","last_peered":"2024-09-11T23:21:02.358347+0000","last_clean":"2024-09-11T23:21:02.358347+0000","last_became_active":"2024-09-11T23:19:57.822947+0000","last_became_peered":"2024-09-11T23:19:57.822947+0000","last_unstale":"2024-09-11T23:21:02.358347+0000","last_undegraded":"2024-09-11T23:21:02.358347+0000","last_fullsized":"2024-09-11T23:21:02.358347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:12.542911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352730+0000","last_change":"2024-09-11T23:19:59.741163+0000","last_active":"2024-09-11T23:21:02.352730+0000","last_peered":"2024-09-11T23:21:02.352730+0000","last_clean":"2024-09-11T23:21:02.352730+0000","last_became_active":"2024-09-11T23:19:59.740633+0000","last_became_peered":"2024-09-11T23:19:59.740633+0000","last_unstale":"2024-09-11T23:21:02.352730+0000","last_undegraded":"2024-09-11T23:21:02.352730+0000","last_fullsized":"2024-09-11T23:21:02.352730+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:26.301775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.359358+0000","last_change":"2024-09-11T23:19:57.615804+0000","last_active":"2024-09-11T23:21:02.359358+0000","last_peered":"2024-09-11T23:21:02.359358+0000","last_clean":"2024-09-11T23:21:02.359358+0000","last_became_active":"2024-09-11T23:19:57.615443+0000","last_became_peered":"2024-09-11T23:19:57.615443+0000","last_unstale":"2024-09-11T23:21:02.359358+0000","last_undegraded":"2024-09-11T23:21:02.359358+0000","last_fullsized":"2024-09-11T23:21:02.359358+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:39.296873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.360875+0000","last_change":"2024-09-11T23:21:02.360875+0000","last_active":"2024-09-11T23:21:02.360875+0000","last_peered":"2024-09-11T23:21:02.360875+0000","last_clean":"2024-09-11T23:21:02.360875+0000","last_became_active":"2024-09-11T23:21:02.360480+0000","last_became_peered":"2024-09-11T23:21:02.360480+0000","last_unstale":"2024-09-11T23:21:02.360875+0000","last_undegraded":"2024-09-11T23:21:02.360875+0000","last_fullsized":"2024-09-11T23:21:02.360875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:04.141587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352532+0000","last_change":"2024-09-11T23:19:57.612873+0000","last_active":"2024-09-11T23:21:02.352532+0000","last_peered":"2024-09-11T23:21:02.352532+0000","last_clean":"2024-09-11T23:21:02.352532+0000","last_became_active":"2024-09-11T23:19:57.612566+0000","last_became_peered":"2024-09-11T23:19:57.612566+0000","last_unstale":"2024-09-11T23:21:02.352532+0000","last_undegraded":"2024-09-11T23:21:02.352532+0000","last_fullsized":"2024-09-11T23:21:02.352532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:46.218212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355460+0000","last_change":"2024-09-11T23:19:59.736352+0000","last_active":"2024-09-11T23:21:02.355460+0000","last_peered":"2024-09-11T23:21:02.355460+0000","last_clean":"2024-09-11T23:21:02.355460+0000","last_became_active":"2024-09-11T23:19:59.735402+0000","last_became_peered":"2024-09-11T23:19:59.735402+0000","last_unstale":"2024-09-11T23:21:02.355460+0000","last_undegraded":"2024-09-11T23:21:02.355460+0000","last_fullsized":"2024-09-11T23:21:02.355460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:41.854526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.353478+0000","last_change":"2024-09-11T23:19:57.612439+0000","last_active":"2024-09-11T23:21:02.353478+0000","last_peered":"2024-09-11T23:21:02.353478+0000","last_clean":"2024-09-11T23:21:02.353478+0000","last_became_active":"2024-09-11T23:19:57.612030+0000","last_became_peered":"2024-09-11T23:19:57.612030+0000","last_unstale":"2024-09-11T23:21:02.353478+0000","last_undegraded":"2024-09-11T23:21:02.353478+0000","last_fullsized":"2024-09-11T23:21:02.353478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:36:24.579246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355577+0000","last_change":"2024-09-11T23:19:59.741241+0000","last_active":"2024-09-11T23:21:02.355577+0000","last_peered":"2024-09-11T23:21:02.355577+0000","last_clean":"2024-09-11T23:21:02.355577+0000","last_became_active":"2024-09-11T23:19:59.740265+0000","last_became_peered":"2024-09-11T23:19:59.740265+0000","last_unstale":"2024-09-11T23:21:02.355577+0000","last_undegraded":"2024-09-11T23:21:02.355577+0000","last_fullsized":"2024-09-11T23:21:02.355577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:00.676762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.361869+0000","last_change":"2024-09-11T23:19:57.825238+0000","last_active":"2024-09-11T23:21:02.361869+0000","last_peered":"2024-09-11T23:21:02.361869+0000","last_clean":"2024-09-11T23:21:02.361869+0000","last_became_active":"2024-09-11T23:19:57.824732+0000","last_became_peered":"2024-09-11T23:19:57.824732+0000","last_unstale":"2024-09-11T23:21:02.361869+0000","last_undegraded":"2024-09-11T23:21:02.361869+0000","last_fullsized":"2024-09-11T23:21:02.361869+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:24.465863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355828+0000","last_change":"2024-09-11T23:19:59.741338+0000","last_active":"2024-09-11T23:21:02.355828+0000","last_peered":"2024-09-11T23:21:02.355828+0000","last_clean":"2024-09-11T23:21:02.355828+0000","last_became_active":"2024-09-11T23:19:59.741019+0000","last_became_peered":"2024-09-11T23:19:59.741019+0000","last_unstale":"2024-09-11T23:21:02.355828+0000","last_undegraded":"2024-09-11T23:21:02.355828+0000","last_fullsized":"2024-09-11T23:21:02.355828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:36:03.480006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":348,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355126+0000","last_change":"2024-09-11T23:18:45.606646+0000","last_active":"2024-09-11T23:21:02.355126+0000","last_peered":"2024-09-11T23:21:02.355126+0000","last_clean":"2024-09-11T23:21:02.355126+0000","last_became_active":"2024-09-11T23:18:45.606065+0000","last_became_peered":"2024-09-11T23:18:45.606065+0000","last_unstale":"2024-09-11T23:21:02.355126+0000","last_undegraded":"2024-09-11T23:21:02.355126+0000","last_fullsized":"2024-09-11T23:21:02.355126+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:15:17.148562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:15:17.148562+0000","last_clean_scrub_stamp":"2024-09-11T23:15:17.148562+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:07.992971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.433439+0000","last_change":"2024-09-11T23:19:57.612584+0000","last_active":"2024-09-11T23:21:02.433439+0000","last_peered":"2024-09-11T23:21:02.433439+0000","last_clean":"2024-09-11T23:21:02.433439+0000","last_became_active":"2024-09-11T23:19:57.612211+0000","last_became_peered":"2024-09-11T23:19:57.612211+0000","last_unstale":"2024-09-11T23:21:02.433439+0000","last_undegraded":"2024-09-11T23:21:02.433439+0000","last_fullsized":"2024-09-11T23:21:02.433439+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:04.026233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.359249+0000","last_change":"2024-09-11T23:21:02.359249+0000","last_active":"2024-09-11T23:21:02.359249+0000","last_peered":"2024-09-11T23:21:02.359249+0000","last_clean":"2024-09-11T23:21:02.359249+0000","last_became_active":"2024-09-11T23:21:02.358906+0000","last_became_peered":"2024-09-11T23:21:02.358906+0000","last_unstale":"2024-09-11T23:21:02.359249+0000","last_undegraded":"2024-09-11T23:21:02.359249+0000","last_fullsized":"2024-09-11T23:21:02.359249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:05.973581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355093+0000","last_change":"2024-09-11T23:19:57.618283+0000","last_active":"2024-09-11T23:21:02.355093+0000","last_peered":"2024-09-11T23:21:02.355093+0000","last_clean":"2024-09-11T23:21:02.355093+0000","last_became_active":"2024-09-11T23:19:57.617942+0000","last_became_peered":"2024-09-11T23:19:57.617942+0000","last_unstale":"2024-09-11T23:21:02.355093+0000","last_undegraded":"2024-09-11T23:21:02.355093+0000","last_fullsized":"2024-09-11T23:21:02.355093+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:30.832602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352168+0000","last_change":"2024-09-11T23:19:59.740730+0000","last_active":"2024-09-11T23:21:02.352168+0000","last_peered":"2024-09-11T23:21:02.352168+0000","last_clean":"2024-09-11T23:21:02.352168+0000","last_became_active":"2024-09-11T23:19:59.740369+0000","last_became_peered":"2024-09-11T23:19:59.740369+0000","last_unstale":"2024-09-11T23:21:02.352168+0000","last_undegraded":"2024-09-11T23:21:02.352168+0000","last_fullsized":"2024-09-11T23:21:02.352168+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:50.264644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":79,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352367+0000","last_change":"2024-09-11T23:19:57.823253+0000","last_active":"2024-09-11T23:21:02.352367+0000","last_peered":"2024-09-11T23:21:02.352367+0000","last_clean":"2024-09-11T23:21:02.352367+0000","last_became_active":"2024-09-11T23:19:57.822895+0000","last_became_peered":"2024-09-11T23:19:57.822895+0000","last_unstale":"2024-09-11T23:21:02.352367+0000","last_undegraded":"2024-09-11T23:21:02.352367+0000","last_fullsized":"2024-09-11T23:21:02.352367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:22.998371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352732+0000","last_change":"2024-09-11T23:19:59.742248+0000","last_active":"2024-09-11T23:21:02.352732+0000","last_peered":"2024-09-11T23:21:02.352732+0000","last_clean":"2024-09-11T23:21:02.352732+0000","last_became_active":"2024-09-11T23:19:59.741810+0000","last_became_peered":"2024-09-11T23:19:59.741810+0000","last_unstale":"2024-09-11T23:21:02.352732+0000","last_undegraded":"2024-09-11T23:21:02.352732+0000","last_fullsized":"2024-09-11T23:21:02.352732+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:19:52.720286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.356343+0000","last_change":"2024-09-11T23:19:57.616244+0000","last_active":"2024-09-11T23:21:02.356343+0000","last_peered":"2024-09-11T23:21:02.356343+0000","last_clean":"2024-09-11T23:21:02.356343+0000","last_became_active":"2024-09-11T23:19:57.615868+0000","last_became_peered":"2024-09-11T23:19:57.615868+0000","last_unstale":"2024-09-11T23:21:02.356343+0000","last_undegraded":"2024-09-11T23:21:02.356343+0000","last_fullsized":"2024-09-11T23:21:02.356343+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:42.510896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"peering","last_fresh":"2024-09-11T23:20:01.696031+0000","last_change":"2024-09-11T23:21:01.359591+0000","last_active":"2024-09-11T23:21:01.359591+0000","last_peered":"2024-09-11T23:20:01.696031+0000","last_clean":"2024-09-11T23:20:01.696031+0000","last_became_active":"2024-09-11T23:19:59.731185+0000","last_became_peered":"2024-09-11T23:19:59.731185+0000","last_unstale":"2024-09-11T23:20:01.696031+0000","last_undegraded":"2024-09-11T23:20:01.696031+0000","last_fullsized":"2024-09-11T23:20:01.696031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4,9],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.357334+0000","last_change":"2024-09-11T23:19:57.824334+0000","last_active":"2024-09-11T23:21:02.357334+0000","last_peered":"2024-09-11T23:21:02.357334+0000","last_clean":"2024-09-11T23:21:02.357334+0000","last_became_active":"2024-09-11T23:19:57.823972+0000","last_became_peered":"2024-09-11T23:19:57.823972+0000","last_unstale":"2024-09-11T23:21:02.357334+0000","last_undegraded":"2024-09-11T23:21:02.357334+0000","last_fullsized":"2024-09-11T23:21:02.357334+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:39.183281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T23:21:01.359313+0000","last_change":"2024-09-11T23:19:59.728064+0000","last_active":"2024-09-11T23:21:01.359313+0000","last_peered":"2024-09-11T23:21:01.359313+0000","last_clean":"2024-09-11T23:21:01.359313+0000","last_became_active":"2024-09-11T23:19:59.727732+0000","last_became_peered":"2024-09-11T23:19:59.727732+0000","last_unstale":"2024-09-11T23:21:01.359313+0000","last_undegraded":"2024-09-11T23:21:01.359313+0000","last_fullsized":"2024-09-11T23:21:01.359313+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:15.383974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.359667+0000","last_change":"2024-09-11T23:19:57.824362+0000","last_active":"2024-09-11T23:21:02.359667+0000","last_peered":"2024-09-11T23:21:02.359667+0000","last_clean":"2024-09-11T23:21:02.359667+0000","last_became_active":"2024-09-11T23:19:57.823753+0000","last_became_peered":"2024-09-11T23:19:57.823753+0000","last_unstale":"2024-09-11T23:21:02.359667+0000","last_undegraded":"2024-09-11T23:21:02.359667+0000","last_fullsized":"2024-09-11T23:21:02.359667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:56.357268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352560+0000","last_change":"2024-09-11T23:19:59.730623+0000","last_active":"2024-09-11T23:21:02.352560+0000","last_peered":"2024-09-11T23:21:02.352560+0000","last_clean":"2024-09-11T23:21:02.352560+0000","last_became_active":"2024-09-11T23:19:59.730307+0000","last_became_peered":"2024-09-11T23:19:59.730307+0000","last_unstale":"2024-09-11T23:21:02.352560+0000","last_undegraded":"2024-09-11T23:21:02.352560+0000","last_fullsized":"2024-09-11T23:21:02.352560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:52.874884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'11","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.356311+0000","last_change":"2024-09-11T23:19:57.611730+0000","last_active":"2024-09-11T23:21:02.356311+0000","last_peered":"2024-09-11T23:21:02.356311+0000","last_clean":"2024-09-11T23:21:02.356311+0000","last_became_active":"2024-09-11T23:19:57.611394+0000","last_became_peered":"2024-09-11T23:19:57.611394+0000","last_unstale":"2024-09-11T23:21:02.356311+0000","last_undegraded":"2024-09-11T23:21:02.356311+0000","last_fullsized":"2024-09-11T23:21:02.356311+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:15.518573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.356512+0000","last_change":"2024-09-11T23:19:59.741396+0000","last_active":"2024-09-11T23:21:02.356512+0000","last_peered":"2024-09-11T23:21:02.356512+0000","last_clean":"2024-09-11T23:21:02.356512+0000","last_became_active":"2024-09-11T23:19:59.741102+0000","last_became_peered":"2024-09-11T23:19:59.741102+0000","last_unstale":"2024-09-11T23:21:02.356512+0000","last_undegraded":"2024-09-11T23:21:02.356512+0000","last_fullsized":"2024-09-11T23:21:02.356512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:08.759512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.433510+0000","last_change":"2024-09-11T23:19:57.824812+0000","last_active":"2024-09-11T23:21:02.433510+0000","last_peered":"2024-09-11T23:21:02.433510+0000","last_clean":"2024-09-11T23:21:02.433510+0000","last_became_active":"2024-09-11T23:19:57.824454+0000","last_became_peered":"2024-09-11T23:19:57.824454+0000","last_unstale":"2024-09-11T23:21:02.433510+0000","last_undegraded":"2024-09-11T23:21:02.433510+0000","last_fullsized":"2024-09-11T23:21:02.433510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:25.275573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.433239+0000","last_change":"2024-09-11T23:19:59.735857+0000","last_active":"2024-09-11T23:21:02.433239+0000","last_peered":"2024-09-11T23:21:02.433239+0000","last_clean":"2024-09-11T23:21:02.433239+0000","last_became_active":"2024-09-11T23:19:59.735541+0000","last_became_peered":"2024-09-11T23:19:59.735541+0000","last_unstale":"2024-09-11T23:21:02.433239+0000","last_undegraded":"2024-09-11T23:21:02.433239+0000","last_fullsized":"2024-09-11T23:21:02.433239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:21.600490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.357702+0000","last_change":"2024-09-11T23:19:57.823468+0000","last_active":"2024-09-11T23:21:02.357702+0000","last_peered":"2024-09-11T23:21:02.357702+0000","last_clean":"2024-09-11T23:21:02.357702+0000","last_became_active":"2024-09-11T23:19:57.823154+0000","last_became_peered":"2024-09-11T23:19:57.823154+0000","last_unstale":"2024-09-11T23:21:02.357702+0000","last_undegraded":"2024-09-11T23:21:02.357702+0000","last_fullsized":"2024-09-11T23:21:02.357702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:28.770627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.358342+0000","last_change":"2024-09-11T23:19:59.741263+0000","last_active":"2024-09-11T23:21:02.358342+0000","last_peered":"2024-09-11T23:21:02.358342+0000","last_clean":"2024-09-11T23:21:02.358342+0000","last_became_active":"2024-09-11T23:19:59.740938+0000","last_became_peered":"2024-09-11T23:19:59.740938+0000","last_unstale":"2024-09-11T23:21:02.358342+0000","last_undegraded":"2024-09-11T23:21:02.358342+0000","last_fullsized":"2024-09-11T23:21:02.358342+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:54.768652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355447+0000","last_change":"2024-09-11T23:19:57.612517+0000","last_active":"2024-09-11T23:21:02.355447+0000","last_peered":"2024-09-11T23:21:02.355447+0000","last_clean":"2024-09-11T23:21:02.355447+0000","last_became_active":"2024-09-11T23:19:57.612188+0000","last_became_peered":"2024-09-11T23:19:57.612188+0000","last_unstale":"2024-09-11T23:21:02.355447+0000","last_undegraded":"2024-09-11T23:21:02.355447+0000","last_fullsized":"2024-09-11T23:21:02.355447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:52.896413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355054+0000","last_change":"2024-09-11T23:19:59.732520+0000","last_active":"2024-09-11T23:21:02.355054+0000","last_peered":"2024-09-11T23:21:02.355054+0000","last_clean":"2024-09-11T23:21:02.355054+0000","last_became_active":"2024-09-11T23:19:59.732173+0000","last_became_peered":"2024-09-11T23:19:59.732173+0000","last_unstale":"2024-09-11T23:21:02.355054+0000","last_undegraded":"2024-09-11T23:21:02.355054+0000","last_fullsized":"2024-09-11T23:21:02.355054+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:04.626312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.366039+0000","last_change":"2024-09-11T23:21:02.365533+0000","last_active":"2024-09-11T23:21:02.366039+0000","last_peered":"2024-09-11T23:21:02.366039+0000","last_clean":"2024-09-11T23:21:02.366039+0000","last_became_active":"2024-09-11T23:21:02.359345+0000","last_became_peered":"2024-09-11T23:21:02.359345+0000","last_unstale":"2024-09-11T23:21:02.366039+0000","last_undegraded":"2024-09-11T23:21:02.366039+0000","last_fullsized":"2024-09-11T23:21:02.366039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:27.195666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355115+0000","last_change":"2024-09-11T23:19:59.734667+0000","last_active":"2024-09-11T23:21:02.355115+0000","last_peered":"2024-09-11T23:21:02.355115+0000","last_clean":"2024-09-11T23:21:02.355115+0000","last_became_active":"2024-09-11T23:19:59.733754+0000","last_became_peered":"2024-09-11T23:19:59.733754+0000","last_unstale":"2024-09-11T23:21:02.355115+0000","last_undegraded":"2024-09-11T23:21:02.355115+0000","last_fullsized":"2024-09-11T23:21:02.355115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:57.392272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.358672+0000","last_change":"2024-09-11T23:19:57.614579+0000","last_active":"2024-09-11T23:21:02.358672+0000","last_peered":"2024-09-11T23:21:02.358672+0000","last_clean":"2024-09-11T23:21:02.358672+0000","last_became_active":"2024-09-11T23:19:57.614227+0000","last_became_peered":"2024-09-11T23:19:57.614227+0000","last_unstale":"2024-09-11T23:21:02.358672+0000","last_undegraded":"2024-09-11T23:21:02.358672+0000","last_fullsized":"2024-09-11T23:21:02.358672+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:47.952920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.356232+0000","last_change":"2024-09-11T23:19:59.744219+0000","last_active":"2024-09-11T23:21:02.356232+0000","last_peered":"2024-09-11T23:21:02.356232+0000","last_clean":"2024-09-11T23:21:02.356232+0000","last_became_active":"2024-09-11T23:19:59.743914+0000","last_became_peered":"2024-09-11T23:19:59.743914+0000","last_unstale":"2024-09-11T23:21:02.356232+0000","last_undegraded":"2024-09-11T23:21:02.356232+0000","last_fullsized":"2024-09-11T23:21:02.356232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:52.149588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.361647+0000","last_change":"2024-09-11T23:21:02.361647+0000","last_active":"2024-09-11T23:21:02.361647+0000","last_peered":"2024-09-11T23:21:02.361647+0000","last_clean":"2024-09-11T23:21:02.361647+0000","last_became_active":"2024-09-11T23:21:02.361319+0000","last_became_peered":"2024-09-11T23:21:02.361319+0000","last_unstale":"2024-09-11T23:21:02.361647+0000","last_undegraded":"2024-09-11T23:21:02.361647+0000","last_fullsized":"2024-09-11T23:21:02.361647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:45.125525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.358921+0000","last_change":"2024-09-11T23:19:57.610389+0000","last_active":"2024-09-11T23:21:02.358921+0000","last_peered":"2024-09-11T23:21:02.358921+0000","last_clean":"2024-09-11T23:21:02.358921+0000","last_became_active":"2024-09-11T23:19:57.609911+0000","last_became_peered":"2024-09-11T23:19:57.609911+0000","last_unstale":"2024-09-11T23:21:02.358921+0000","last_undegraded":"2024-09-11T23:21:02.358921+0000","last_fullsized":"2024-09-11T23:21:02.358921+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:23.949929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.357813+0000","last_change":"2024-09-11T23:19:59.736867+0000","last_active":"2024-09-11T23:21:02.357813+0000","last_peered":"2024-09-11T23:21:02.357813+0000","last_clean":"2024-09-11T23:21:02.357813+0000","last_became_active":"2024-09-11T23:19:59.736341+0000","last_became_peered":"2024-09-11T23:19:59.736341+0000","last_unstale":"2024-09-11T23:21:02.357813+0000","last_undegraded":"2024-09-11T23:21:02.357813+0000","last_fullsized":"2024-09-11T23:21:02.357813+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:57:51.015235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.369377+0000","last_change":"2024-09-11T23:21:02.368959+0000","last_active":"2024-09-11T23:21:02.369377+0000","last_peered":"2024-09-11T23:21:02.369377+0000","last_clean":"2024-09-11T23:21:02.369377+0000","last_became_active":"2024-09-11T23:21:02.363263+0000","last_became_peered":"2024-09-11T23:21:02.363263+0000","last_unstale":"2024-09-11T23:21:02.369377+0000","last_undegraded":"2024-09-11T23:21:02.369377+0000","last_fullsized":"2024-09-11T23:21:02.369377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:29.827330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.433021+0000","last_change":"2024-09-11T23:19:59.742049+0000","last_active":"2024-09-11T23:21:02.433021+0000","last_peered":"2024-09-11T23:21:02.433021+0000","last_clean":"2024-09-11T23:21:02.433021+0000","last_became_active":"2024-09-11T23:19:59.741763+0000","last_became_peered":"2024-09-11T23:19:59.741763+0000","last_unstale":"2024-09-11T23:21:02.433021+0000","last_undegraded":"2024-09-11T23:21:02.433021+0000","last_fullsized":"2024-09-11T23:21:02.433021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:48.698896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.356390+0000","last_change":"2024-09-11T23:19:57.822949+0000","last_active":"2024-09-11T23:21:02.356390+0000","last_peered":"2024-09-11T23:21:02.356390+0000","last_clean":"2024-09-11T23:21:02.356390+0000","last_became_active":"2024-09-11T23:19:57.822564+0000","last_became_peered":"2024-09-11T23:19:57.822564+0000","last_unstale":"2024-09-11T23:21:02.356390+0000","last_undegraded":"2024-09-11T23:21:02.356390+0000","last_fullsized":"2024-09-11T23:21:02.356390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:29.532109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.363639+0000","last_change":"2024-09-11T23:21:02.363639+0000","last_active":"2024-09-11T23:21:02.363639+0000","last_peered":"2024-09-11T23:21:02.363639+0000","last_clean":"2024-09-11T23:21:02.363639+0000","last_became_active":"2024-09-11T23:21:02.363247+0000","last_became_peered":"2024-09-11T23:21:02.363247+0000","last_unstale":"2024-09-11T23:21:02.363639+0000","last_undegraded":"2024-09-11T23:21:02.363639+0000","last_fullsized":"2024-09-11T23:21:02.363639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:18:58.906939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.356266+0000","last_change":"2024-09-11T23:19:57.613323+0000","last_active":"2024-09-11T23:21:02.356266+0000","last_peered":"2024-09-11T23:21:02.356266+0000","last_clean":"2024-09-11T23:21:02.356266+0000","last_became_active":"2024-09-11T23:19:57.612985+0000","last_became_peered":"2024-09-11T23:19:57.612985+0000","last_unstale":"2024-09-11T23:21:02.356266+0000","last_undegraded":"2024-09-11T23:21:02.356266+0000","last_fullsized":"2024-09-11T23:21:02.356266+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:13:09.179547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64352,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":338,"num_read_kb":236,"num_write":160,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285620,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514522,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":909312,"data_stored":441815,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743454,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":1076762,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906982,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":925696,"data_stored":442051,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168618,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":925696,"data_stored":441971,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397551,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":974848,"data_stored":1033903,"data_compressed":42106,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561076,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":901120,"data_stored":432923,"data_compressed":2892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790009,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":987136,"data_stored":1033194,"data_compressed":42106,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920831,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":917504,"data_stored":442270,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149764,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":929792,"data_stored":441751,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":502932,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":974848,"data_stored":501582,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":933888,"data_stored":442711,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34680,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34680,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34680,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:21:07.796 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:21:08.074 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:08.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:07 smithi043 ceph-mon[30950]: pgmap v317: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-11T23:21:08.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:21:08.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:07 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:21:08.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:07 smithi043 ceph-mon[30950]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:21:08.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:07 smithi090 ceph-mon[35130]: pgmap v317: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-11T23:21:08.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:21:08.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:07 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:21:08.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:07 smithi090 ceph-mon[35130]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:21:08.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:07 smithi097 ceph-mon[35562]: pgmap v317: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-11T23:21:08.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:21:08.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:07 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:21:08.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:07 smithi097 ceph-mon[35562]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:21:08.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.552+0000 7f3ef7435640 1 -- 172.21.15.43:0/4152568122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef0103660 msgr2=0x7f3ef010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:08.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.552+0000 7f3ef7435640 1 --2- 172.21.15.43:0/4152568122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef0103660 0x7f3ef010bb50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009900 tx=0x7f3ee002a480 comp rx=0 tx=0).stop 2024-09-11T23:21:08.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.552+0000 7f3ef7435640 1 -- 172.21.15.43:0/4152568122 shutdown_connections 2024-09-11T23:21:08.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.552+0000 7f3ef7435640 1 --2- 172.21.15.43:0/4152568122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef010c170 0x7f3ef010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:08.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.552+0000 7f3ef7435640 1 --2- 172.21.15.43:0/4152568122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef0103660 0x7f3ef010bb50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:08.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.552+0000 7f3ef7435640 1 --2- 172.21.15.43:0/4152568122 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ef0102d40 0x7f3ef0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:08.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.552+0000 7f3ef7435640 1 -- 172.21.15.43:0/4152568122 >> 172.21.15.43:0/4152568122 conn(0x7f3ef00fc7a0 msgr2=0x7f3ef00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:08.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.552+0000 7f3ef7435640 1 -- 172.21.15.43:0/4152568122 shutdown_connections 2024-09-11T23:21:08.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.552+0000 7f3ef7435640 1 -- 172.21.15.43:0/4152568122 wait complete. 2024-09-11T23:21:08.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.553+0000 7f3ef7435640 1 Processor -- start 2024-09-11T23:21:08.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.553+0000 7f3ef7435640 1 -- start start 2024-09-11T23:21:08.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.553+0000 7f3ef7435640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ef0102d40 0x7f3ef0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:08.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.553+0000 7f3ef7435640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef0103660 0x7f3ef01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:08.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.553+0000 7f3ef7435640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef010c170 0x7f3ef019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:08.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.553+0000 7f3ef7435640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef0111fb0 con 0x7f3ef010c170 2024-09-11T23:21:08.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.553+0000 7f3ef7435640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef0111e30 con 0x7f3ef0102d40 2024-09-11T23:21:08.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.553+0000 7f3ef7435640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef0112130 con 0x7f3ef0103660 2024-09-11T23:21:08.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.553+0000 7f3ef59d2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef010c170 0x7f3ef019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:08.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.553+0000 7f3ef59d2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef010c170 0x7f3ef019cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59438/0 (socket says 172.21.15.43:59438) 2024-09-11T23:21:08.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.553+0000 7f3ef59d2640 1 -- 172.21.15.43:0/3684634134 learned_addr learned my addr 172.21.15.43:0/3684634134 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:08.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.553+0000 7f3ef49d0640 1 --2- 172.21.15.43:0/3684634134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef0103660 0x7f3ef01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:08.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.554+0000 7f3ef59d2640 1 -- 172.21.15.43:0/3684634134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef0103660 msgr2=0x7f3ef01989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:08.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.554+0000 7f3ef59d2640 1 --2- 172.21.15.43:0/3684634134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef0103660 0x7f3ef01989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:08.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.554+0000 7f3ef59d2640 1 -- 172.21.15.43:0/3684634134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ef0102d40 msgr2=0x7f3ef0198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:08.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.554+0000 7f3ef51d1640 1 --2- 172.21.15.43:0/3684634134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ef0102d40 0x7f3ef0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:08.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.554+0000 7f3ef59d2640 1 --2- 172.21.15.43:0/3684634134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ef0102d40 0x7f3ef0198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:08.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.554+0000 7f3ef59d2640 1 -- 172.21.15.43:0/3684634134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef019d410 con 0x7f3ef010c170 2024-09-11T23:21:08.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.554+0000 7f3ef59d2640 1 --2- 172.21.15.43:0/3684634134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef010c170 0x7f3ef019cd30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800cce0 tx=0x7f3ee800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:08.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.554+0000 7f3ee67fc640 1 -- 172.21.15.43:0/3684634134 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee800f450 con 0x7f3ef010c170 2024-09-11T23:21:08.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.554+0000 7f3ef7435640 1 -- 172.21.15.43:0/3684634134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef019d700 con 0x7f3ef010c170 2024-09-11T23:21:08.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.554+0000 7f3ee67fc640 1 -- 172.21.15.43:0/3684634134 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ee800fad0 con 0x7f3ef010c170 2024-09-11T23:21:08.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.554+0000 7f3ee67fc640 1 -- 172.21.15.43:0/3684634134 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8018c10 con 0x7f3ef010c170 2024-09-11T23:21:08.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.554+0000 7f3ef7435640 1 -- 172.21.15.43:0/3684634134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef01a4fe0 con 0x7f3ef010c170 2024-09-11T23:21:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.555+0000 7f3ef7435640 1 -- 172.21.15.43:0/3684634134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef0103120 con 0x7f3ef010c170 2024-09-11T23:21:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.556+0000 7f3ee67fc640 1 -- 172.21.15.43:0/3684634134 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3ee800f5f0 con 0x7f3ef010c170 2024-09-11T23:21:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.557+0000 7f3ee67fc640 1 --2- 172.21.15.43:0/3684634134 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3edc0a2c10 0x7f3edc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.557+0000 7f3ef51d1640 1 --2- 172.21.15.43:0/3684634134 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3edc0a2c10 0x7f3edc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.557+0000 7f3ee67fc640 1 -- 172.21.15.43:0/3684634134 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f3ee80cf9c0 con 0x7f3ef010c170 2024-09-11T23:21:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.558+0000 7f3ef51d1640 1 --2- 172.21.15.43:0/3684634134 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3edc0a2c10 0x7f3edc0a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00a810 tx=0x7f3eec008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.561+0000 7f3ee67fc640 1 -- 172.21.15.43:0/3684634134 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ee8014030 con 0x7f3ef010c170 2024-09-11T23:21:08.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.734+0000 7f3ef7435640 1 -- 172.21.15.43:0/3684634134 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3ef0062630 con 0x7f3edc0a2c10 2024-09-11T23:21:08.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.741+0000 7f3ee67fc640 1 -- 172.21.15.43:0/3684634134 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258965 (secure 0 0 0) 0x7f3ef0062630 con 0x7f3edc0a2c10 2024-09-11T23:21:08.744 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:21:08.745 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-11T23:21:08.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.744+0000 7f3ef7435640 1 -- 172.21.15.43:0/3684634134 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3edc0a2c10 msgr2=0x7f3edc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:08.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.744+0000 7f3ef7435640 1 --2- 172.21.15.43:0/3684634134 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3edc0a2c10 0x7f3edc0a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00a810 tx=0x7f3eec008040 comp rx=0 tx=0).stop 2024-09-11T23:21:08.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.744+0000 7f3ef7435640 1 -- 172.21.15.43:0/3684634134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef010c170 msgr2=0x7f3ef019cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:08.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.744+0000 7f3ef7435640 1 --2- 172.21.15.43:0/3684634134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef010c170 0x7f3ef019cd30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800cce0 tx=0x7f3ee800a620 comp rx=0 tx=0).stop 2024-09-11T23:21:08.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.744+0000 7f3ef7435640 1 -- 172.21.15.43:0/3684634134 shutdown_connections 2024-09-11T23:21:08.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.744+0000 7f3ef7435640 1 --2- 172.21.15.43:0/3684634134 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3edc0a2c10 0x7f3edc0a50d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:08.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.744+0000 7f3ef7435640 1 --2- 172.21.15.43:0/3684634134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ef010c170 0x7f3ef019cd30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:08.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.744+0000 7f3ef7435640 1 --2- 172.21.15.43:0/3684634134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef0103660 0x7f3ef01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:08.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.744+0000 7f3ef7435640 1 --2- 172.21.15.43:0/3684634134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ef0102d40 0x7f3ef0198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:08.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.744+0000 7f3ef7435640 1 -- 172.21.15.43:0/3684634134 >> 172.21.15.43:0/3684634134 conn(0x7f3ef00fc7a0 msgr2=0x7f3ef00fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:08.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.744+0000 7f3ef7435640 1 -- 172.21.15.43:0/3684634134 shutdown_connections 2024-09-11T23:21:08.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:08.745+0000 7f3ef7435640 1 -- 172.21.15.43:0/3684634134 wait complete. 2024-09-11T23:21:09.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:08 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:21:09.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:08 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:21:09.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:08 smithi043 ceph-mon[30950]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:21:09.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:08 smithi043 ceph-mon[30950]: Health check update: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T23:21:09.272 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:08 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:21:09.272 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:08 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:21:09.273 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:08 smithi090 ceph-mon[35130]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:21:09.273 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:08 smithi090 ceph-mon[35130]: Health check update: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T23:21:09.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:08 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:21:09.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:08 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:21:09.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:08 smithi097 ceph-mon[35562]: from='mgr.14150 172.21.15.43:0/2041994966' entity='mgr.x' 2024-09-11T23:21:09.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:08 smithi097 ceph-mon[35562]: Health check update: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T23:21:09.507 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-11T23:21:08.648613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654954,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":504,"num_read_kb":370,"num_write":343,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":292,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368836,"kb_used_data":11184,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572988,"statfs":{"total":1151940427776,"available":1151562739712,"internally_reserved":0,"allocated":11452416,"data_stored":7259658,"data_compressed":178428,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001886"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355548+0000","last_change":"2024-09-11T23:19:59.734378+0000","last_active":"2024-09-11T23:21:02.355548+0000","last_peered":"2024-09-11T23:21:02.355548+0000","last_clean":"2024-09-11T23:21:02.355548+0000","last_became_active":"2024-09-11T23:19:59.734067+0000","last_became_peered":"2024-09-11T23:19:59.734067+0000","last_unstale":"2024-09-11T23:21:02.355548+0000","last_undegraded":"2024-09-11T23:21:02.355548+0000","last_fullsized":"2024-09-11T23:21:02.355548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:28:18.384453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.433031+0000","last_change":"2024-09-11T23:19:59.733892+0000","last_active":"2024-09-11T23:21:02.433031+0000","last_peered":"2024-09-11T23:21:02.433031+0000","last_clean":"2024-09-11T23:21:02.433031+0000","last_became_active":"2024-09-11T23:19:59.733540+0000","last_became_peered":"2024-09-11T23:19:59.733540+0000","last_unstale":"2024-09-11T23:21:02.433031+0000","last_undegraded":"2024-09-11T23:21:02.433031+0000","last_fullsized":"2024-09-11T23:21:02.433031+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:42.186458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.433126+0000","last_change":"2024-09-11T23:19:59.739443+0000","last_active":"2024-09-11T23:21:02.433126+0000","last_peered":"2024-09-11T23:21:02.433126+0000","last_clean":"2024-09-11T23:21:02.433126+0000","last_became_active":"2024-09-11T23:19:59.739025+0000","last_became_peered":"2024-09-11T23:19:59.739025+0000","last_unstale":"2024-09-11T23:21:02.433126+0000","last_undegraded":"2024-09-11T23:21:02.433126+0000","last_fullsized":"2024-09-11T23:21:02.433126+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:40.819321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.363511+0000","last_change":"2024-09-11T23:21:02.363511+0000","last_active":"2024-09-11T23:21:02.363511+0000","last_peered":"2024-09-11T23:21:02.363511+0000","last_clean":"2024-09-11T23:21:02.363511+0000","last_became_active":"2024-09-11T23:21:02.363156+0000","last_became_peered":"2024-09-11T23:21:02.363156+0000","last_unstale":"2024-09-11T23:21:02.363511+0000","last_undegraded":"2024-09-11T23:21:02.363511+0000","last_fullsized":"2024-09-11T23:21:02.363511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:17.059509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.358522+0000","last_change":"2024-09-11T23:19:59.738662+0000","last_active":"2024-09-11T23:21:02.358522+0000","last_peered":"2024-09-11T23:21:02.358522+0000","last_clean":"2024-09-11T23:21:02.358522+0000","last_became_active":"2024-09-11T23:19:59.738320+0000","last_became_peered":"2024-09-11T23:19:59.738320+0000","last_unstale":"2024-09-11T23:21:02.358522+0000","last_undegraded":"2024-09-11T23:21:02.358522+0000","last_fullsized":"2024-09-11T23:21:02.358522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:15.645179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.353078+0000","last_change":"2024-09-11T23:19:59.731442+0000","last_active":"2024-09-11T23:21:02.353078+0000","last_peered":"2024-09-11T23:21:02.353078+0000","last_clean":"2024-09-11T23:21:02.353078+0000","last_became_active":"2024-09-11T23:19:59.731079+0000","last_became_peered":"2024-09-11T23:19:59.731079+0000","last_unstale":"2024-09-11T23:21:02.353078+0000","last_undegraded":"2024-09-11T23:21:02.353078+0000","last_fullsized":"2024-09-11T23:21:02.353078+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:32.764991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.359992+0000","last_change":"2024-09-11T23:21:02.359992+0000","last_active":"2024-09-11T23:21:02.359992+0000","last_peered":"2024-09-11T23:21:02.359992+0000","last_clean":"2024-09-11T23:21:02.359992+0000","last_became_active":"2024-09-11T23:21:02.359564+0000","last_became_peered":"2024-09-11T23:21:02.359564+0000","last_unstale":"2024-09-11T23:21:02.359992+0000","last_undegraded":"2024-09-11T23:21:02.359992+0000","last_fullsized":"2024-09-11T23:21:02.359992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:53.633219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.354781+0000","last_change":"2024-09-11T23:19:59.740733+0000","last_active":"2024-09-11T23:21:02.354781+0000","last_peered":"2024-09-11T23:21:02.354781+0000","last_clean":"2024-09-11T23:21:02.354781+0000","last_became_active":"2024-09-11T23:19:59.740438+0000","last_became_peered":"2024-09-11T23:19:59.740438+0000","last_unstale":"2024-09-11T23:21:02.354781+0000","last_undegraded":"2024-09-11T23:21:02.354781+0000","last_fullsized":"2024-09-11T23:21:02.354781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:27.063305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.358498+0000","last_change":"2024-09-11T23:19:59.742439+0000","last_active":"2024-09-11T23:21:02.358498+0000","last_peered":"2024-09-11T23:21:02.358498+0000","last_clean":"2024-09-11T23:21:02.358498+0000","last_became_active":"2024-09-11T23:19:59.741983+0000","last_became_peered":"2024-09-11T23:19:59.741983+0000","last_unstale":"2024-09-11T23:21:02.358498+0000","last_undegraded":"2024-09-11T23:21:02.358498+0000","last_fullsized":"2024-09-11T23:21:02.358498+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:39.477167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.353624+0000","last_change":"2024-09-11T23:19:59.732544+0000","last_active":"2024-09-11T23:21:02.353624+0000","last_peered":"2024-09-11T23:21:02.353624+0000","last_clean":"2024-09-11T23:21:02.353624+0000","last_became_active":"2024-09-11T23:19:59.732205+0000","last_became_peered":"2024-09-11T23:19:59.732205+0000","last_unstale":"2024-09-11T23:21:02.353624+0000","last_undegraded":"2024-09-11T23:21:02.353624+0000","last_fullsized":"2024-09-11T23:21:02.353624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:22.494344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355822+0000","last_change":"2024-09-11T23:19:59.741857+0000","last_active":"2024-09-11T23:21:02.355822+0000","last_peered":"2024-09-11T23:21:02.355822+0000","last_clean":"2024-09-11T23:21:02.355822+0000","last_became_active":"2024-09-11T23:19:59.741484+0000","last_became_peered":"2024-09-11T23:19:59.741484+0000","last_unstale":"2024-09-11T23:21:02.355822+0000","last_undegraded":"2024-09-11T23:21:02.355822+0000","last_fullsized":"2024-09-11T23:21:02.355822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:52.998258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.433280+0000","last_change":"2024-09-11T23:19:59.736042+0000","last_active":"2024-09-11T23:21:02.433280+0000","last_peered":"2024-09-11T23:21:02.433280+0000","last_clean":"2024-09-11T23:21:02.433280+0000","last_became_active":"2024-09-11T23:19:59.735731+0000","last_became_peered":"2024-09-11T23:19:59.735731+0000","last_unstale":"2024-09-11T23:21:02.433280+0000","last_undegraded":"2024-09-11T23:21:02.433280+0000","last_fullsized":"2024-09-11T23:21:02.433280+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:19.085804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352541+0000","last_change":"2024-09-11T23:19:59.742049+0000","last_active":"2024-09-11T23:21:02.352541+0000","last_peered":"2024-09-11T23:21:02.352541+0000","last_clean":"2024-09-11T23:21:02.352541+0000","last_became_active":"2024-09-11T23:19:59.740974+0000","last_became_peered":"2024-09-11T23:19:59.740974+0000","last_unstale":"2024-09-11T23:21:02.352541+0000","last_undegraded":"2024-09-11T23:21:02.352541+0000","last_fullsized":"2024-09-11T23:21:02.352541+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:10.443113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.432709+0000","last_change":"2024-09-11T23:19:59.734270+0000","last_active":"2024-09-11T23:21:02.432709+0000","last_peered":"2024-09-11T23:21:02.432709+0000","last_clean":"2024-09-11T23:21:02.432709+0000","last_became_active":"2024-09-11T23:19:59.733967+0000","last_became_peered":"2024-09-11T23:19:59.733967+0000","last_unstale":"2024-09-11T23:21:02.432709+0000","last_undegraded":"2024-09-11T23:21:02.432709+0000","last_fullsized":"2024-09-11T23:21:02.432709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:02.185757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.432580+0000","last_change":"2024-09-11T23:19:59.734152+0000","last_active":"2024-09-11T23:21:02.432580+0000","last_peered":"2024-09-11T23:21:02.432580+0000","last_clean":"2024-09-11T23:21:02.432580+0000","last_became_active":"2024-09-11T23:19:59.733163+0000","last_became_peered":"2024-09-11T23:19:59.733163+0000","last_unstale":"2024-09-11T23:21:02.432580+0000","last_undegraded":"2024-09-11T23:21:02.432580+0000","last_fullsized":"2024-09-11T23:21:02.432580+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:22.370591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.357557+0000","last_change":"2024-09-11T23:19:59.735517+0000","last_active":"2024-09-11T23:21:02.357557+0000","last_peered":"2024-09-11T23:21:02.357557+0000","last_clean":"2024-09-11T23:21:02.357557+0000","last_became_active":"2024-09-11T23:19:59.735182+0000","last_became_peered":"2024-09-11T23:19:59.735182+0000","last_unstale":"2024-09-11T23:21:02.357557+0000","last_undegraded":"2024-09-11T23:21:02.357557+0000","last_fullsized":"2024-09-11T23:21:02.357557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:25.989416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355566+0000","last_change":"2024-09-11T23:19:59.733579+0000","last_active":"2024-09-11T23:21:02.355566+0000","last_peered":"2024-09-11T23:21:02.355566+0000","last_clean":"2024-09-11T23:21:02.355566+0000","last_became_active":"2024-09-11T23:19:59.733245+0000","last_became_peered":"2024-09-11T23:19:59.733245+0000","last_unstale":"2024-09-11T23:21:02.355566+0000","last_undegraded":"2024-09-11T23:21:02.355566+0000","last_fullsized":"2024-09-11T23:21:02.355566+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:22.361158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.356089+0000","last_change":"2024-09-11T23:19:59.734354+0000","last_active":"2024-09-11T23:21:02.356089+0000","last_peered":"2024-09-11T23:21:02.356089+0000","last_clean":"2024-09-11T23:21:02.356089+0000","last_became_active":"2024-09-11T23:19:59.733988+0000","last_became_peered":"2024-09-11T23:19:59.733988+0000","last_unstale":"2024-09-11T23:21:02.356089+0000","last_undegraded":"2024-09-11T23:21:02.356089+0000","last_fullsized":"2024-09-11T23:21:02.356089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:41.401496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.360420+0000","last_change":"2024-09-11T23:19:59.730318+0000","last_active":"2024-09-11T23:21:02.360420+0000","last_peered":"2024-09-11T23:21:02.360420+0000","last_clean":"2024-09-11T23:21:02.360420+0000","last_became_active":"2024-09-11T23:19:59.730009+0000","last_became_peered":"2024-09-11T23:19:59.730009+0000","last_unstale":"2024-09-11T23:21:02.360420+0000","last_undegraded":"2024-09-11T23:21:02.360420+0000","last_fullsized":"2024-09-11T23:21:02.360420+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:01:38.531967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355062+0000","last_change":"2024-09-11T23:19:59.737254+0000","last_active":"2024-09-11T23:21:02.355062+0000","last_peered":"2024-09-11T23:21:02.355062+0000","last_clean":"2024-09-11T23:21:02.355062+0000","last_became_active":"2024-09-11T23:19:59.736936+0000","last_became_peered":"2024-09-11T23:19:59.736936+0000","last_unstale":"2024-09-11T23:21:02.355062+0000","last_undegraded":"2024-09-11T23:21:02.355062+0000","last_fullsized":"2024-09-11T23:21:02.355062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:00.361468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352632+0000","last_change":"2024-09-11T23:19:59.741617+0000","last_active":"2024-09-11T23:21:02.352632+0000","last_peered":"2024-09-11T23:21:02.352632+0000","last_clean":"2024-09-11T23:21:02.352632+0000","last_became_active":"2024-09-11T23:19:59.740358+0000","last_became_peered":"2024-09-11T23:19:59.740358+0000","last_unstale":"2024-09-11T23:21:02.352632+0000","last_undegraded":"2024-09-11T23:21:02.352632+0000","last_fullsized":"2024-09-11T23:21:02.352632+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:18.541105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.358132+0000","last_change":"2024-09-11T23:19:59.736437+0000","last_active":"2024-09-11T23:21:02.358132+0000","last_peered":"2024-09-11T23:21:02.358132+0000","last_clean":"2024-09-11T23:21:02.358132+0000","last_became_active":"2024-09-11T23:19:59.736029+0000","last_became_peered":"2024-09-11T23:19:59.736029+0000","last_unstale":"2024-09-11T23:21:02.358132+0000","last_undegraded":"2024-09-11T23:21:02.358132+0000","last_fullsized":"2024-09-11T23:21:02.358132+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:29.226939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355141+0000","last_change":"2024-09-11T23:19:59.736104+0000","last_active":"2024-09-11T23:21:02.355141+0000","last_peered":"2024-09-11T23:21:02.355141+0000","last_clean":"2024-09-11T23:21:02.355141+0000","last_became_active":"2024-09-11T23:19:59.735760+0000","last_became_peered":"2024-09-11T23:19:59.735760+0000","last_unstale":"2024-09-11T23:21:02.355141+0000","last_undegraded":"2024-09-11T23:21:02.355141+0000","last_fullsized":"2024-09-11T23:21:02.355141+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:56.554313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352652+0000","last_change":"2024-09-11T23:19:59.735455+0000","last_active":"2024-09-11T23:21:02.352652+0000","last_peered":"2024-09-11T23:21:02.352652+0000","last_clean":"2024-09-11T23:21:02.352652+0000","last_became_active":"2024-09-11T23:19:59.735078+0000","last_became_peered":"2024-09-11T23:19:59.735078+0000","last_unstale":"2024-09-11T23:21:02.352652+0000","last_undegraded":"2024-09-11T23:21:02.352652+0000","last_fullsized":"2024-09-11T23:21:02.352652+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:21:16.336006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355992+0000","last_change":"2024-09-11T23:19:59.745997+0000","last_active":"2024-09-11T23:21:02.355992+0000","last_peered":"2024-09-11T23:21:02.355992+0000","last_clean":"2024-09-11T23:21:02.355992+0000","last_became_active":"2024-09-11T23:19:59.745715+0000","last_became_peered":"2024-09-11T23:19:59.745715+0000","last_unstale":"2024-09-11T23:21:02.355992+0000","last_undegraded":"2024-09-11T23:21:02.355992+0000","last_fullsized":"2024-09-11T23:21:02.355992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:34.776229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.361426+0000","last_change":"2024-09-11T23:19:59.734181+0000","last_active":"2024-09-11T23:21:02.361426+0000","last_peered":"2024-09-11T23:21:02.361426+0000","last_clean":"2024-09-11T23:21:02.361426+0000","last_became_active":"2024-09-11T23:19:59.733793+0000","last_became_peered":"2024-09-11T23:19:59.733793+0000","last_unstale":"2024-09-11T23:21:02.361426+0000","last_undegraded":"2024-09-11T23:21:02.361426+0000","last_fullsized":"2024-09-11T23:21:02.361426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:36.906308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355383+0000","last_change":"2024-09-11T23:19:59.738713+0000","last_active":"2024-09-11T23:21:02.355383+0000","last_peered":"2024-09-11T23:21:02.355383+0000","last_clean":"2024-09-11T23:21:02.355383+0000","last_became_active":"2024-09-11T23:19:59.737579+0000","last_became_peered":"2024-09-11T23:19:59.737579+0000","last_unstale":"2024-09-11T23:21:02.355383+0000","last_undegraded":"2024-09-11T23:21:02.355383+0000","last_fullsized":"2024-09-11T23:21:02.355383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:47.135775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352455+0000","last_change":"2024-09-11T23:19:59.741381+0000","last_active":"2024-09-11T23:21:02.352455+0000","last_peered":"2024-09-11T23:21:02.352455+0000","last_clean":"2024-09-11T23:21:02.352455+0000","last_became_active":"2024-09-11T23:19:59.740914+0000","last_became_peered":"2024-09-11T23:19:59.740914+0000","last_unstale":"2024-09-11T23:21:02.352455+0000","last_undegraded":"2024-09-11T23:21:02.352455+0000","last_fullsized":"2024-09-11T23:21:02.352455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:24:04.365704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.433019+0000","last_change":"2024-09-11T23:19:59.736840+0000","last_active":"2024-09-11T23:21:02.433019+0000","last_peered":"2024-09-11T23:21:02.433019+0000","last_clean":"2024-09-11T23:21:02.433019+0000","last_became_active":"2024-09-11T23:19:59.736554+0000","last_became_peered":"2024-09-11T23:19:59.736554+0000","last_unstale":"2024-09-11T23:21:02.433019+0000","last_undegraded":"2024-09-11T23:21:02.433019+0000","last_fullsized":"2024-09-11T23:21:02.433019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:48.508369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.359012+0000","last_change":"2024-09-11T23:19:59.735719+0000","last_active":"2024-09-11T23:21:02.359012+0000","last_peered":"2024-09-11T23:21:02.359012+0000","last_clean":"2024-09-11T23:21:02.359012+0000","last_became_active":"2024-09-11T23:19:59.735397+0000","last_became_peered":"2024-09-11T23:19:59.735397+0000","last_unstale":"2024-09-11T23:21:02.359012+0000","last_undegraded":"2024-09-11T23:21:02.359012+0000","last_fullsized":"2024-09-11T23:21:02.359012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:26:19.392730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.361161+0000","last_change":"2024-09-11T23:19:59.737704+0000","last_active":"2024-09-11T23:21:02.361161+0000","last_peered":"2024-09-11T23:21:02.361161+0000","last_clean":"2024-09-11T23:21:02.361161+0000","last_became_active":"2024-09-11T23:19:59.737176+0000","last_became_peered":"2024-09-11T23:19:59.737176+0000","last_unstale":"2024-09-11T23:21:02.361161+0000","last_undegraded":"2024-09-11T23:21:02.361161+0000","last_fullsized":"2024-09-11T23:21:02.361161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:22.938171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.354986+0000","last_change":"2024-09-11T23:19:59.735499+0000","last_active":"2024-09-11T23:21:02.354986+0000","last_peered":"2024-09-11T23:21:02.354986+0000","last_clean":"2024-09-11T23:21:02.354986+0000","last_became_active":"2024-09-11T23:19:59.735088+0000","last_became_peered":"2024-09-11T23:19:59.735088+0000","last_unstale":"2024-09-11T23:21:02.354986+0000","last_undegraded":"2024-09-11T23:21:02.354986+0000","last_fullsized":"2024-09-11T23:21:02.354986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:12.633104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.358475+0000","last_change":"2024-09-11T23:19:59.743972+0000","last_active":"2024-09-11T23:21:02.358475+0000","last_peered":"2024-09-11T23:21:02.358475+0000","last_clean":"2024-09-11T23:21:02.358475+0000","last_became_active":"2024-09-11T23:19:59.743675+0000","last_became_peered":"2024-09-11T23:19:59.743675+0000","last_unstale":"2024-09-11T23:21:02.358475+0000","last_undegraded":"2024-09-11T23:21:02.358475+0000","last_fullsized":"2024-09-11T23:21:02.358475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:46.645696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":87,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:06.808134+0000","last_change":"2024-09-11T23:19:57.615472+0000","last_active":"2024-09-11T23:21:06.808134+0000","last_peered":"2024-09-11T23:21:06.808134+0000","last_clean":"2024-09-11T23:21:06.808134+0000","last_became_active":"2024-09-11T23:19:57.615104+0000","last_became_peered":"2024-09-11T23:19:57.615104+0000","last_unstale":"2024-09-11T23:21:06.808134+0000","last_undegraded":"2024-09-11T23:21:06.808134+0000","last_fullsized":"2024-09-11T23:21:06.808134+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:03.601363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":60,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352907+0000","last_change":"2024-09-11T23:19:59.741027+0000","last_active":"2024-09-11T23:21:02.352907+0000","last_peered":"2024-09-11T23:21:02.352907+0000","last_clean":"2024-09-11T23:21:02.352907+0000","last_became_active":"2024-09-11T23:19:59.740681+0000","last_became_peered":"2024-09-11T23:19:59.740681+0000","last_unstale":"2024-09-11T23:21:02.352907+0000","last_undegraded":"2024-09-11T23:21:02.352907+0000","last_fullsized":"2024-09-11T23:21:02.352907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:36.948147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.353180+0000","last_change":"2024-09-11T23:19:57.612730+0000","last_active":"2024-09-11T23:21:02.353180+0000","last_peered":"2024-09-11T23:21:02.353180+0000","last_clean":"2024-09-11T23:21:02.353180+0000","last_became_active":"2024-09-11T23:19:57.612369+0000","last_became_peered":"2024-09-11T23:19:57.612369+0000","last_unstale":"2024-09-11T23:21:02.353180+0000","last_undegraded":"2024-09-11T23:21:02.353180+0000","last_fullsized":"2024-09-11T23:21:02.353180+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:17.823616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352175+0000","last_change":"2024-09-11T23:19:57.611470+0000","last_active":"2024-09-11T23:21:02.352175+0000","last_peered":"2024-09-11T23:21:02.352175+0000","last_clean":"2024-09-11T23:21:02.352175+0000","last_became_active":"2024-09-11T23:19:57.611141+0000","last_became_peered":"2024-09-11T23:19:57.611141+0000","last_unstale":"2024-09-11T23:21:02.352175+0000","last_undegraded":"2024-09-11T23:21:02.352175+0000","last_fullsized":"2024-09-11T23:21:02.352175+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:43.173955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.354859+0000","last_change":"2024-09-11T23:19:59.732375+0000","last_active":"2024-09-11T23:21:02.354859+0000","last_peered":"2024-09-11T23:21:02.354859+0000","last_clean":"2024-09-11T23:21:02.354859+0000","last_became_active":"2024-09-11T23:19:59.731793+0000","last_became_peered":"2024-09-11T23:19:59.731793+0000","last_unstale":"2024-09-11T23:21:02.354859+0000","last_undegraded":"2024-09-11T23:21:02.354859+0000","last_fullsized":"2024-09-11T23:21:02.354859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:02:47.278646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.353064+0000","last_change":"2024-09-11T23:19:57.614062+0000","last_active":"2024-09-11T23:21:02.353064+0000","last_peered":"2024-09-11T23:21:02.353064+0000","last_clean":"2024-09-11T23:21:02.353064+0000","last_became_active":"2024-09-11T23:19:57.613749+0000","last_became_peered":"2024-09-11T23:19:57.613749+0000","last_unstale":"2024-09-11T23:21:02.353064+0000","last_undegraded":"2024-09-11T23:21:02.353064+0000","last_fullsized":"2024-09-11T23:21:02.353064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:45:22.299379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.356130+0000","last_change":"2024-09-11T23:19:59.730758+0000","last_active":"2024-09-11T23:21:02.356130+0000","last_peered":"2024-09-11T23:21:02.356130+0000","last_clean":"2024-09-11T23:21:02.356130+0000","last_became_active":"2024-09-11T23:19:59.730423+0000","last_became_peered":"2024-09-11T23:19:59.730423+0000","last_unstale":"2024-09-11T23:21:02.356130+0000","last_undegraded":"2024-09-11T23:21:02.356130+0000","last_fullsized":"2024-09-11T23:21:02.356130+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:30.679451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":81,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:04.805313+0000","last_change":"2024-09-11T23:19:57.615574+0000","last_active":"2024-09-11T23:21:04.805313+0000","last_peered":"2024-09-11T23:21:04.805313+0000","last_clean":"2024-09-11T23:21:04.805313+0000","last_became_active":"2024-09-11T23:19:57.615192+0000","last_became_peered":"2024-09-11T23:19:57.615192+0000","last_unstale":"2024-09-11T23:21:04.805313+0000","last_undegraded":"2024-09-11T23:21:04.805313+0000","last_fullsized":"2024-09-11T23:21:04.805313+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:29.186899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.358363+0000","last_change":"2024-09-11T23:19:59.738433+0000","last_active":"2024-09-11T23:21:02.358363+0000","last_peered":"2024-09-11T23:21:02.358363+0000","last_clean":"2024-09-11T23:21:02.358363+0000","last_became_active":"2024-09-11T23:19:59.738081+0000","last_became_peered":"2024-09-11T23:19:59.738081+0000","last_unstale":"2024-09-11T23:21:02.358363+0000","last_undegraded":"2024-09-11T23:21:02.358363+0000","last_fullsized":"2024-09-11T23:21:02.358363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:10:46.786454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.437498+0000","last_change":"2024-09-11T23:21:02.437498+0000","last_active":"2024-09-11T23:21:02.437498+0000","last_peered":"2024-09-11T23:21:02.437498+0000","last_clean":"2024-09-11T23:21:02.437498+0000","last_became_active":"2024-09-11T23:21:02.434895+0000","last_became_peered":"2024-09-11T23:21:02.434895+0000","last_unstale":"2024-09-11T23:21:02.437498+0000","last_undegraded":"2024-09-11T23:21:02.437498+0000","last_fullsized":"2024-09-11T23:21:02.437498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:33.142041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.433161+0000","last_change":"2024-09-11T23:19:59.741081+0000","last_active":"2024-09-11T23:21:02.433161+0000","last_peered":"2024-09-11T23:21:02.433161+0000","last_clean":"2024-09-11T23:21:02.433161+0000","last_became_active":"2024-09-11T23:19:59.739984+0000","last_became_peered":"2024-09-11T23:19:59.739984+0000","last_unstale":"2024-09-11T23:21:02.433161+0000","last_undegraded":"2024-09-11T23:21:02.433161+0000","last_fullsized":"2024-09-11T23:21:02.433161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:03.513121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.353300+0000","last_change":"2024-09-11T23:19:57.823272+0000","last_active":"2024-09-11T23:21:02.353300+0000","last_peered":"2024-09-11T23:21:02.353300+0000","last_clean":"2024-09-11T23:21:02.353300+0000","last_became_active":"2024-09-11T23:19:57.822957+0000","last_became_peered":"2024-09-11T23:19:57.822957+0000","last_unstale":"2024-09-11T23:21:02.353300+0000","last_undegraded":"2024-09-11T23:21:02.353300+0000","last_fullsized":"2024-09-11T23:21:02.353300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:06.779385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.357966+0000","last_change":"2024-09-11T23:19:59.738398+0000","last_active":"2024-09-11T23:21:02.357966+0000","last_peered":"2024-09-11T23:21:02.357966+0000","last_clean":"2024-09-11T23:21:02.357966+0000","last_became_active":"2024-09-11T23:19:59.738042+0000","last_became_peered":"2024-09-11T23:19:59.738042+0000","last_unstale":"2024-09-11T23:21:02.357966+0000","last_undegraded":"2024-09-11T23:21:02.357966+0000","last_fullsized":"2024-09-11T23:21:02.357966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:05.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'26","reported_seq":167,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:05.806368+0000","last_change":"2024-09-11T23:19:57.615036+0000","last_active":"2024-09-11T23:21:05.806368+0000","last_peered":"2024-09-11T23:21:05.806368+0000","last_clean":"2024-09-11T23:21:05.806368+0000","last_became_active":"2024-09-11T23:19:57.614641+0000","last_became_peered":"2024-09-11T23:19:57.614641+0000","last_unstale":"2024-09-11T23:21:05.806368+0000","last_undegraded":"2024-09-11T23:21:05.806368+0000","last_fullsized":"2024-09-11T23:21:05.806368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:05.430508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60349,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":119,"num_read_kb":118,"num_write":27,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.359507+0000","last_change":"2024-09-11T23:19:59.741597+0000","last_active":"2024-09-11T23:21:02.359507+0000","last_peered":"2024-09-11T23:21:02.359507+0000","last_clean":"2024-09-11T23:21:02.359507+0000","last_became_active":"2024-09-11T23:19:59.741296+0000","last_became_peered":"2024-09-11T23:19:59.741296+0000","last_unstale":"2024-09-11T23:21:02.359507+0000","last_undegraded":"2024-09-11T23:21:02.359507+0000","last_fullsized":"2024-09-11T23:21:02.359507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:06:20.742916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.361428+0000","last_change":"2024-09-11T23:19:57.617550+0000","last_active":"2024-09-11T23:21:02.361428+0000","last_peered":"2024-09-11T23:21:02.361428+0000","last_clean":"2024-09-11T23:21:02.361428+0000","last_became_active":"2024-09-11T23:19:57.617211+0000","last_became_peered":"2024-09-11T23:19:57.617211+0000","last_unstale":"2024-09-11T23:21:02.361428+0000","last_undegraded":"2024-09-11T23:21:02.361428+0000","last_fullsized":"2024-09-11T23:21:02.361428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:19.707742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352215+0000","last_change":"2024-09-11T23:19:59.742070+0000","last_active":"2024-09-11T23:21:02.352215+0000","last_peered":"2024-09-11T23:21:02.352215+0000","last_clean":"2024-09-11T23:21:02.352215+0000","last_became_active":"2024-09-11T23:19:59.739367+0000","last_became_peered":"2024-09-11T23:19:59.739367+0000","last_unstale":"2024-09-11T23:21:02.352215+0000","last_undegraded":"2024-09-11T23:21:02.352215+0000","last_fullsized":"2024-09-11T23:21:02.352215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:27.850136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352872+0000","last_change":"2024-09-11T23:19:57.618054+0000","last_active":"2024-09-11T23:21:02.352872+0000","last_peered":"2024-09-11T23:21:02.352872+0000","last_clean":"2024-09-11T23:21:02.352872+0000","last_became_active":"2024-09-11T23:19:57.617702+0000","last_became_peered":"2024-09-11T23:19:57.617702+0000","last_unstale":"2024-09-11T23:21:02.352872+0000","last_undegraded":"2024-09-11T23:21:02.352872+0000","last_fullsized":"2024-09-11T23:21:02.352872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:40.171851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355453+0000","last_change":"2024-09-11T23:19:59.744000+0000","last_active":"2024-09-11T23:21:02.355453+0000","last_peered":"2024-09-11T23:21:02.355453+0000","last_clean":"2024-09-11T23:21:02.355453+0000","last_became_active":"2024-09-11T23:19:59.743629+0000","last_became_peered":"2024-09-11T23:19:59.743629+0000","last_unstale":"2024-09-11T23:21:02.355453+0000","last_undegraded":"2024-09-11T23:21:02.355453+0000","last_fullsized":"2024-09-11T23:21:02.355453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:20:04.133258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":87,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:06.808465+0000","last_change":"2024-09-11T23:19:57.615117+0000","last_active":"2024-09-11T23:21:06.808465+0000","last_peered":"2024-09-11T23:21:06.808465+0000","last_clean":"2024-09-11T23:21:06.808465+0000","last_became_active":"2024-09-11T23:19:57.614758+0000","last_became_peered":"2024-09-11T23:19:57.614758+0000","last_unstale":"2024-09-11T23:21:06.808465+0000","last_undegraded":"2024-09-11T23:21:06.808465+0000","last_fullsized":"2024-09-11T23:21:06.808465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:58.113986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":62,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.354969+0000","last_change":"2024-09-11T23:19:59.739403+0000","last_active":"2024-09-11T23:21:02.354969+0000","last_peered":"2024-09-11T23:21:02.354969+0000","last_clean":"2024-09-11T23:21:02.354969+0000","last_became_active":"2024-09-11T23:19:59.739078+0000","last_became_peered":"2024-09-11T23:19:59.739078+0000","last_unstale":"2024-09-11T23:21:02.354969+0000","last_undegraded":"2024-09-11T23:21:02.354969+0000","last_fullsized":"2024-09-11T23:21:02.354969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:46.168720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.358347+0000","last_change":"2024-09-11T23:19:57.823323+0000","last_active":"2024-09-11T23:21:02.358347+0000","last_peered":"2024-09-11T23:21:02.358347+0000","last_clean":"2024-09-11T23:21:02.358347+0000","last_became_active":"2024-09-11T23:19:57.822947+0000","last_became_peered":"2024-09-11T23:19:57.822947+0000","last_unstale":"2024-09-11T23:21:02.358347+0000","last_undegraded":"2024-09-11T23:21:02.358347+0000","last_fullsized":"2024-09-11T23:21:02.358347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:12.542911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352730+0000","last_change":"2024-09-11T23:19:59.741163+0000","last_active":"2024-09-11T23:21:02.352730+0000","last_peered":"2024-09-11T23:21:02.352730+0000","last_clean":"2024-09-11T23:21:02.352730+0000","last_became_active":"2024-09-11T23:19:59.740633+0000","last_became_peered":"2024-09-11T23:19:59.740633+0000","last_unstale":"2024-09-11T23:21:02.352730+0000","last_undegraded":"2024-09-11T23:21:02.352730+0000","last_fullsized":"2024-09-11T23:21:02.352730+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:26.301775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.359358+0000","last_change":"2024-09-11T23:19:57.615804+0000","last_active":"2024-09-11T23:21:02.359358+0000","last_peered":"2024-09-11T23:21:02.359358+0000","last_clean":"2024-09-11T23:21:02.359358+0000","last_became_active":"2024-09-11T23:19:57.615443+0000","last_became_peered":"2024-09-11T23:19:57.615443+0000","last_unstale":"2024-09-11T23:21:02.359358+0000","last_undegraded":"2024-09-11T23:21:02.359358+0000","last_fullsized":"2024-09-11T23:21:02.359358+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:39.296873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.360875+0000","last_change":"2024-09-11T23:21:02.360875+0000","last_active":"2024-09-11T23:21:02.360875+0000","last_peered":"2024-09-11T23:21:02.360875+0000","last_clean":"2024-09-11T23:21:02.360875+0000","last_became_active":"2024-09-11T23:21:02.360480+0000","last_became_peered":"2024-09-11T23:21:02.360480+0000","last_unstale":"2024-09-11T23:21:02.360875+0000","last_undegraded":"2024-09-11T23:21:02.360875+0000","last_fullsized":"2024-09-11T23:21:02.360875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:04.141587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352532+0000","last_change":"2024-09-11T23:19:57.612873+0000","last_active":"2024-09-11T23:21:02.352532+0000","last_peered":"2024-09-11T23:21:02.352532+0000","last_clean":"2024-09-11T23:21:02.352532+0000","last_became_active":"2024-09-11T23:19:57.612566+0000","last_became_peered":"2024-09-11T23:19:57.612566+0000","last_unstale":"2024-09-11T23:21:02.352532+0000","last_undegraded":"2024-09-11T23:21:02.352532+0000","last_fullsized":"2024-09-11T23:21:02.352532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:46.218212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355460+0000","last_change":"2024-09-11T23:19:59.736352+0000","last_active":"2024-09-11T23:21:02.355460+0000","last_peered":"2024-09-11T23:21:02.355460+0000","last_clean":"2024-09-11T23:21:02.355460+0000","last_became_active":"2024-09-11T23:19:59.735402+0000","last_became_peered":"2024-09-11T23:19:59.735402+0000","last_unstale":"2024-09-11T23:21:02.355460+0000","last_undegraded":"2024-09-11T23:21:02.355460+0000","last_fullsized":"2024-09-11T23:21:02.355460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:41.854526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.353478+0000","last_change":"2024-09-11T23:19:57.612439+0000","last_active":"2024-09-11T23:21:02.353478+0000","last_peered":"2024-09-11T23:21:02.353478+0000","last_clean":"2024-09-11T23:21:02.353478+0000","last_became_active":"2024-09-11T23:19:57.612030+0000","last_became_peered":"2024-09-11T23:19:57.612030+0000","last_unstale":"2024-09-11T23:21:02.353478+0000","last_undegraded":"2024-09-11T23:21:02.353478+0000","last_fullsized":"2024-09-11T23:21:02.353478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:36:24.579246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355577+0000","last_change":"2024-09-11T23:19:59.741241+0000","last_active":"2024-09-11T23:21:02.355577+0000","last_peered":"2024-09-11T23:21:02.355577+0000","last_clean":"2024-09-11T23:21:02.355577+0000","last_became_active":"2024-09-11T23:19:59.740265+0000","last_became_peered":"2024-09-11T23:19:59.740265+0000","last_unstale":"2024-09-11T23:21:02.355577+0000","last_undegraded":"2024-09-11T23:21:02.355577+0000","last_fullsized":"2024-09-11T23:21:02.355577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:00.676762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.361869+0000","last_change":"2024-09-11T23:19:57.825238+0000","last_active":"2024-09-11T23:21:02.361869+0000","last_peered":"2024-09-11T23:21:02.361869+0000","last_clean":"2024-09-11T23:21:02.361869+0000","last_became_active":"2024-09-11T23:19:57.824732+0000","last_became_peered":"2024-09-11T23:19:57.824732+0000","last_unstale":"2024-09-11T23:21:02.361869+0000","last_undegraded":"2024-09-11T23:21:02.361869+0000","last_fullsized":"2024-09-11T23:21:02.361869+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:24.465863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355828+0000","last_change":"2024-09-11T23:19:59.741338+0000","last_active":"2024-09-11T23:21:02.355828+0000","last_peered":"2024-09-11T23:21:02.355828+0000","last_clean":"2024-09-11T23:21:02.355828+0000","last_became_active":"2024-09-11T23:19:59.741019+0000","last_became_peered":"2024-09-11T23:19:59.741019+0000","last_unstale":"2024-09-11T23:21:02.355828+0000","last_undegraded":"2024-09-11T23:21:02.355828+0000","last_fullsized":"2024-09-11T23:21:02.355828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:36:03.480006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":348,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355126+0000","last_change":"2024-09-11T23:18:45.606646+0000","last_active":"2024-09-11T23:21:02.355126+0000","last_peered":"2024-09-11T23:21:02.355126+0000","last_clean":"2024-09-11T23:21:02.355126+0000","last_became_active":"2024-09-11T23:18:45.606065+0000","last_became_peered":"2024-09-11T23:18:45.606065+0000","last_unstale":"2024-09-11T23:21:02.355126+0000","last_undegraded":"2024-09-11T23:21:02.355126+0000","last_fullsized":"2024-09-11T23:21:02.355126+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:15:17.148562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:15:17.148562+0000","last_clean_scrub_stamp":"2024-09-11T23:15:17.148562+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:07.992971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.433439+0000","last_change":"2024-09-11T23:19:57.612584+0000","last_active":"2024-09-11T23:21:02.433439+0000","last_peered":"2024-09-11T23:21:02.433439+0000","last_clean":"2024-09-11T23:21:02.433439+0000","last_became_active":"2024-09-11T23:19:57.612211+0000","last_became_peered":"2024-09-11T23:19:57.612211+0000","last_unstale":"2024-09-11T23:21:02.433439+0000","last_undegraded":"2024-09-11T23:21:02.433439+0000","last_fullsized":"2024-09-11T23:21:02.433439+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:04.026233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.359249+0000","last_change":"2024-09-11T23:21:02.359249+0000","last_active":"2024-09-11T23:21:02.359249+0000","last_peered":"2024-09-11T23:21:02.359249+0000","last_clean":"2024-09-11T23:21:02.359249+0000","last_became_active":"2024-09-11T23:21:02.358906+0000","last_became_peered":"2024-09-11T23:21:02.358906+0000","last_unstale":"2024-09-11T23:21:02.359249+0000","last_undegraded":"2024-09-11T23:21:02.359249+0000","last_fullsized":"2024-09-11T23:21:02.359249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:05.973581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355093+0000","last_change":"2024-09-11T23:19:57.618283+0000","last_active":"2024-09-11T23:21:02.355093+0000","last_peered":"2024-09-11T23:21:02.355093+0000","last_clean":"2024-09-11T23:21:02.355093+0000","last_became_active":"2024-09-11T23:19:57.617942+0000","last_became_peered":"2024-09-11T23:19:57.617942+0000","last_unstale":"2024-09-11T23:21:02.355093+0000","last_undegraded":"2024-09-11T23:21:02.355093+0000","last_fullsized":"2024-09-11T23:21:02.355093+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:30.832602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352168+0000","last_change":"2024-09-11T23:19:59.740730+0000","last_active":"2024-09-11T23:21:02.352168+0000","last_peered":"2024-09-11T23:21:02.352168+0000","last_clean":"2024-09-11T23:21:02.352168+0000","last_became_active":"2024-09-11T23:19:59.740369+0000","last_became_peered":"2024-09-11T23:19:59.740369+0000","last_unstale":"2024-09-11T23:21:02.352168+0000","last_undegraded":"2024-09-11T23:21:02.352168+0000","last_fullsized":"2024-09-11T23:21:02.352168+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:50.264644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":84,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:06.807958+0000","last_change":"2024-09-11T23:19:57.823253+0000","last_active":"2024-09-11T23:21:06.807958+0000","last_peered":"2024-09-11T23:21:06.807958+0000","last_clean":"2024-09-11T23:21:06.807958+0000","last_became_active":"2024-09-11T23:19:57.822895+0000","last_became_peered":"2024-09-11T23:19:57.822895+0000","last_unstale":"2024-09-11T23:21:06.807958+0000","last_undegraded":"2024-09-11T23:21:06.807958+0000","last_fullsized":"2024-09-11T23:21:06.807958+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:22.998371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352732+0000","last_change":"2024-09-11T23:19:59.742248+0000","last_active":"2024-09-11T23:21:02.352732+0000","last_peered":"2024-09-11T23:21:02.352732+0000","last_clean":"2024-09-11T23:21:02.352732+0000","last_became_active":"2024-09-11T23:19:59.741810+0000","last_became_peered":"2024-09-11T23:19:59.741810+0000","last_unstale":"2024-09-11T23:21:02.352732+0000","last_undegraded":"2024-09-11T23:21:02.352732+0000","last_fullsized":"2024-09-11T23:21:02.352732+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:19:52.720286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.356343+0000","last_change":"2024-09-11T23:19:57.616244+0000","last_active":"2024-09-11T23:21:02.356343+0000","last_peered":"2024-09-11T23:21:02.356343+0000","last_clean":"2024-09-11T23:21:02.356343+0000","last_became_active":"2024-09-11T23:19:57.615868+0000","last_became_peered":"2024-09-11T23:19:57.615868+0000","last_unstale":"2024-09-11T23:21:02.356343+0000","last_undegraded":"2024-09-11T23:21:02.356343+0000","last_fullsized":"2024-09-11T23:21:02.356343+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:42.510896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.434265+0000","last_change":"2024-09-11T23:21:02.434265+0000","last_active":"2024-09-11T23:21:02.434265+0000","last_peered":"2024-09-11T23:21:02.434265+0000","last_clean":"2024-09-11T23:21:02.434265+0000","last_became_active":"2024-09-11T23:21:02.433840+0000","last_became_peered":"2024-09-11T23:21:02.433840+0000","last_unstale":"2024-09-11T23:21:02.434265+0000","last_undegraded":"2024-09-11T23:21:02.434265+0000","last_fullsized":"2024-09-11T23:21:02.434265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:04.619695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.357334+0000","last_change":"2024-09-11T23:19:57.824334+0000","last_active":"2024-09-11T23:21:02.357334+0000","last_peered":"2024-09-11T23:21:02.357334+0000","last_clean":"2024-09-11T23:21:02.357334+0000","last_became_active":"2024-09-11T23:19:57.823972+0000","last_became_peered":"2024-09-11T23:19:57.823972+0000","last_unstale":"2024-09-11T23:21:02.357334+0000","last_undegraded":"2024-09-11T23:21:02.357334+0000","last_fullsized":"2024-09-11T23:21:02.357334+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:39.183281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.353316+0000","last_change":"2024-09-11T23:19:59.728064+0000","last_active":"2024-09-11T23:21:02.353316+0000","last_peered":"2024-09-11T23:21:02.353316+0000","last_clean":"2024-09-11T23:21:02.353316+0000","last_became_active":"2024-09-11T23:19:59.727732+0000","last_became_peered":"2024-09-11T23:19:59.727732+0000","last_unstale":"2024-09-11T23:21:02.353316+0000","last_undegraded":"2024-09-11T23:21:02.353316+0000","last_fullsized":"2024-09-11T23:21:02.353316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:15.383974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.359667+0000","last_change":"2024-09-11T23:19:57.824362+0000","last_active":"2024-09-11T23:21:02.359667+0000","last_peered":"2024-09-11T23:21:02.359667+0000","last_clean":"2024-09-11T23:21:02.359667+0000","last_became_active":"2024-09-11T23:19:57.823753+0000","last_became_peered":"2024-09-11T23:19:57.823753+0000","last_unstale":"2024-09-11T23:21:02.359667+0000","last_undegraded":"2024-09-11T23:21:02.359667+0000","last_fullsized":"2024-09-11T23:21:02.359667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:56.357268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.352560+0000","last_change":"2024-09-11T23:19:59.730623+0000","last_active":"2024-09-11T23:21:02.352560+0000","last_peered":"2024-09-11T23:21:02.352560+0000","last_clean":"2024-09-11T23:21:02.352560+0000","last_became_active":"2024-09-11T23:19:59.730307+0000","last_became_peered":"2024-09-11T23:19:59.730307+0000","last_unstale":"2024-09-11T23:21:02.352560+0000","last_undegraded":"2024-09-11T23:21:02.352560+0000","last_fullsized":"2024-09-11T23:21:02.352560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:52.874884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'11","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.356311+0000","last_change":"2024-09-11T23:19:57.611730+0000","last_active":"2024-09-11T23:21:02.356311+0000","last_peered":"2024-09-11T23:21:02.356311+0000","last_clean":"2024-09-11T23:21:02.356311+0000","last_became_active":"2024-09-11T23:19:57.611394+0000","last_became_peered":"2024-09-11T23:19:57.611394+0000","last_unstale":"2024-09-11T23:21:02.356311+0000","last_undegraded":"2024-09-11T23:21:02.356311+0000","last_fullsized":"2024-09-11T23:21:02.356311+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:15.518573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.356512+0000","last_change":"2024-09-11T23:19:59.741396+0000","last_active":"2024-09-11T23:21:02.356512+0000","last_peered":"2024-09-11T23:21:02.356512+0000","last_clean":"2024-09-11T23:21:02.356512+0000","last_became_active":"2024-09-11T23:19:59.741102+0000","last_became_peered":"2024-09-11T23:19:59.741102+0000","last_unstale":"2024-09-11T23:21:02.356512+0000","last_undegraded":"2024-09-11T23:21:02.356512+0000","last_fullsized":"2024-09-11T23:21:02.356512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:08.759512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.433510+0000","last_change":"2024-09-11T23:19:57.824812+0000","last_active":"2024-09-11T23:21:02.433510+0000","last_peered":"2024-09-11T23:21:02.433510+0000","last_clean":"2024-09-11T23:21:02.433510+0000","last_became_active":"2024-09-11T23:19:57.824454+0000","last_became_peered":"2024-09-11T23:19:57.824454+0000","last_unstale":"2024-09-11T23:21:02.433510+0000","last_undegraded":"2024-09-11T23:21:02.433510+0000","last_fullsized":"2024-09-11T23:21:02.433510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:25.275573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.433239+0000","last_change":"2024-09-11T23:19:59.735857+0000","last_active":"2024-09-11T23:21:02.433239+0000","last_peered":"2024-09-11T23:21:02.433239+0000","last_clean":"2024-09-11T23:21:02.433239+0000","last_became_active":"2024-09-11T23:19:59.735541+0000","last_became_peered":"2024-09-11T23:19:59.735541+0000","last_unstale":"2024-09-11T23:21:02.433239+0000","last_undegraded":"2024-09-11T23:21:02.433239+0000","last_fullsized":"2024-09-11T23:21:02.433239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:21.600490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.357702+0000","last_change":"2024-09-11T23:19:57.823468+0000","last_active":"2024-09-11T23:21:02.357702+0000","last_peered":"2024-09-11T23:21:02.357702+0000","last_clean":"2024-09-11T23:21:02.357702+0000","last_became_active":"2024-09-11T23:19:57.823154+0000","last_became_peered":"2024-09-11T23:19:57.823154+0000","last_unstale":"2024-09-11T23:21:02.357702+0000","last_undegraded":"2024-09-11T23:21:02.357702+0000","last_fullsized":"2024-09-11T23:21:02.357702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:28.770627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.358342+0000","last_change":"2024-09-11T23:19:59.741263+0000","last_active":"2024-09-11T23:21:02.358342+0000","last_peered":"2024-09-11T23:21:02.358342+0000","last_clean":"2024-09-11T23:21:02.358342+0000","last_became_active":"2024-09-11T23:19:59.740938+0000","last_became_peered":"2024-09-11T23:19:59.740938+0000","last_unstale":"2024-09-11T23:21:02.358342+0000","last_undegraded":"2024-09-11T23:21:02.358342+0000","last_fullsized":"2024-09-11T23:21:02.358342+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:54.768652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355447+0000","last_change":"2024-09-11T23:19:57.612517+0000","last_active":"2024-09-11T23:21:02.355447+0000","last_peered":"2024-09-11T23:21:02.355447+0000","last_clean":"2024-09-11T23:21:02.355447+0000","last_became_active":"2024-09-11T23:19:57.612188+0000","last_became_peered":"2024-09-11T23:19:57.612188+0000","last_unstale":"2024-09-11T23:21:02.355447+0000","last_undegraded":"2024-09-11T23:21:02.355447+0000","last_fullsized":"2024-09-11T23:21:02.355447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:52.896413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355054+0000","last_change":"2024-09-11T23:19:59.732520+0000","last_active":"2024-09-11T23:21:02.355054+0000","last_peered":"2024-09-11T23:21:02.355054+0000","last_clean":"2024-09-11T23:21:02.355054+0000","last_became_active":"2024-09-11T23:19:59.732173+0000","last_became_peered":"2024-09-11T23:19:59.732173+0000","last_unstale":"2024-09-11T23:21:02.355054+0000","last_undegraded":"2024-09-11T23:21:02.355054+0000","last_fullsized":"2024-09-11T23:21:02.355054+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:04.626312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.366039+0000","last_change":"2024-09-11T23:21:02.365533+0000","last_active":"2024-09-11T23:21:02.366039+0000","last_peered":"2024-09-11T23:21:02.366039+0000","last_clean":"2024-09-11T23:21:02.366039+0000","last_became_active":"2024-09-11T23:21:02.359345+0000","last_became_peered":"2024-09-11T23:21:02.359345+0000","last_unstale":"2024-09-11T23:21:02.366039+0000","last_undegraded":"2024-09-11T23:21:02.366039+0000","last_fullsized":"2024-09-11T23:21:02.366039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:27.195666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.355115+0000","last_change":"2024-09-11T23:19:59.734667+0000","last_active":"2024-09-11T23:21:02.355115+0000","last_peered":"2024-09-11T23:21:02.355115+0000","last_clean":"2024-09-11T23:21:02.355115+0000","last_became_active":"2024-09-11T23:19:59.733754+0000","last_became_peered":"2024-09-11T23:19:59.733754+0000","last_unstale":"2024-09-11T23:21:02.355115+0000","last_undegraded":"2024-09-11T23:21:02.355115+0000","last_fullsized":"2024-09-11T23:21:02.355115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:57.392272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.358672+0000","last_change":"2024-09-11T23:19:57.614579+0000","last_active":"2024-09-11T23:21:02.358672+0000","last_peered":"2024-09-11T23:21:02.358672+0000","last_clean":"2024-09-11T23:21:02.358672+0000","last_became_active":"2024-09-11T23:19:57.614227+0000","last_became_peered":"2024-09-11T23:19:57.614227+0000","last_unstale":"2024-09-11T23:21:02.358672+0000","last_undegraded":"2024-09-11T23:21:02.358672+0000","last_fullsized":"2024-09-11T23:21:02.358672+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:47.952920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.356232+0000","last_change":"2024-09-11T23:19:59.744219+0000","last_active":"2024-09-11T23:21:02.356232+0000","last_peered":"2024-09-11T23:21:02.356232+0000","last_clean":"2024-09-11T23:21:02.356232+0000","last_became_active":"2024-09-11T23:19:59.743914+0000","last_became_peered":"2024-09-11T23:19:59.743914+0000","last_unstale":"2024-09-11T23:21:02.356232+0000","last_undegraded":"2024-09-11T23:21:02.356232+0000","last_fullsized":"2024-09-11T23:21:02.356232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:52.149588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.361647+0000","last_change":"2024-09-11T23:21:02.361647+0000","last_active":"2024-09-11T23:21:02.361647+0000","last_peered":"2024-09-11T23:21:02.361647+0000","last_clean":"2024-09-11T23:21:02.361647+0000","last_became_active":"2024-09-11T23:21:02.361319+0000","last_became_peered":"2024-09-11T23:21:02.361319+0000","last_unstale":"2024-09-11T23:21:02.361647+0000","last_undegraded":"2024-09-11T23:21:02.361647+0000","last_fullsized":"2024-09-11T23:21:02.361647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:45.125525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.358921+0000","last_change":"2024-09-11T23:19:57.610389+0000","last_active":"2024-09-11T23:21:02.358921+0000","last_peered":"2024-09-11T23:21:02.358921+0000","last_clean":"2024-09-11T23:21:02.358921+0000","last_became_active":"2024-09-11T23:19:57.609911+0000","last_became_peered":"2024-09-11T23:19:57.609911+0000","last_unstale":"2024-09-11T23:21:02.358921+0000","last_undegraded":"2024-09-11T23:21:02.358921+0000","last_fullsized":"2024-09-11T23:21:02.358921+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:23.949929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.357813+0000","last_change":"2024-09-11T23:19:59.736867+0000","last_active":"2024-09-11T23:21:02.357813+0000","last_peered":"2024-09-11T23:21:02.357813+0000","last_clean":"2024-09-11T23:21:02.357813+0000","last_became_active":"2024-09-11T23:19:59.736341+0000","last_became_peered":"2024-09-11T23:19:59.736341+0000","last_unstale":"2024-09-11T23:21:02.357813+0000","last_undegraded":"2024-09-11T23:21:02.357813+0000","last_fullsized":"2024-09-11T23:21:02.357813+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:57:51.015235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.369377+0000","last_change":"2024-09-11T23:21:02.368959+0000","last_active":"2024-09-11T23:21:02.369377+0000","last_peered":"2024-09-11T23:21:02.369377+0000","last_clean":"2024-09-11T23:21:02.369377+0000","last_became_active":"2024-09-11T23:21:02.363263+0000","last_became_peered":"2024-09-11T23:21:02.363263+0000","last_unstale":"2024-09-11T23:21:02.369377+0000","last_undegraded":"2024-09-11T23:21:02.369377+0000","last_fullsized":"2024-09-11T23:21:02.369377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:29.827330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.433021+0000","last_change":"2024-09-11T23:19:59.742049+0000","last_active":"2024-09-11T23:21:02.433021+0000","last_peered":"2024-09-11T23:21:02.433021+0000","last_clean":"2024-09-11T23:21:02.433021+0000","last_became_active":"2024-09-11T23:19:59.741763+0000","last_became_peered":"2024-09-11T23:19:59.741763+0000","last_unstale":"2024-09-11T23:21:02.433021+0000","last_undegraded":"2024-09-11T23:21:02.433021+0000","last_fullsized":"2024-09-11T23:21:02.433021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:48.698896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.356390+0000","last_change":"2024-09-11T23:19:57.822949+0000","last_active":"2024-09-11T23:21:02.356390+0000","last_peered":"2024-09-11T23:21:02.356390+0000","last_clean":"2024-09-11T23:21:02.356390+0000","last_became_active":"2024-09-11T23:19:57.822564+0000","last_became_peered":"2024-09-11T23:19:57.822564+0000","last_unstale":"2024-09-11T23:21:02.356390+0000","last_undegraded":"2024-09-11T23:21:02.356390+0000","last_fullsized":"2024-09-11T23:21:02.356390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:29.532109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.363639+0000","last_change":"2024-09-11T23:21:02.363639+0000","last_active":"2024-09-11T23:21:02.363639+0000","last_peered":"2024-09-11T23:21:02.363639+0000","last_clean":"2024-09-11T23:21:02.363639+0000","last_became_active":"2024-09-11T23:21:02.363247+0000","last_became_peered":"2024-09-11T23:21:02.363247+0000","last_unstale":"2024-09-11T23:21:02.363639+0000","last_undegraded":"2024-09-11T23:21:02.363639+0000","last_fullsized":"2024-09-11T23:21:02.363639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:18:58.906939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T23:21:02.356266+0000","last_change":"2024-09-11T23:19:57.613323+0000","last_active":"2024-09-11T23:21:02.356266+0000","last_peered":"2024-09-11T23:21:02.356266+0000","last_clean":"2024-09-11T23:21:02.356266+0000","last_became_active":"2024-09-11T23:19:57.612985+0000","last_became_peered":"2024-09-11T23:19:57.612985+0000","last_unstale":"2024-09-11T23:21:02.356266+0000","last_undegraded":"2024-09-11T23:21:02.356266+0000","last_fullsized":"2024-09-11T23:21:02.356266+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:13:09.179547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64352,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":368,"num_read_kb":251,"num_write":160,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285620,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514522,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":909312,"data_stored":441815,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743455,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1060864,"data_stored":1094038,"data_compressed":42106,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906982,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":925696,"data_stored":442051,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168618,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":925696,"data_stored":441971,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397551,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":974848,"data_stored":1033903,"data_compressed":42106,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561077,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":913408,"data_stored":441440,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790010,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":987136,"data_stored":1033194,"data_compressed":42106,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920832,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":917504,"data_stored":442270,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149764,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":929792,"data_stored":441751,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":502932,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":974848,"data_stored":501582,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":933888,"data_stored":442711,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34680,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34680,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34680,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:21:09.512 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T23:21:09.512 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T23:21:09.512 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-11T23:21:09.512 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-11T23:21:09.776 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:10.186 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:09 smithi043 ceph-mon[30950]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-11T23:21:10.186 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:09 smithi043 ceph-mon[30950]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:21:10.186 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:09 smithi043 ceph-mon[30950]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T23:21:10.186 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:09 smithi043 ceph-mon[30950]: Cluster is now healthy 2024-09-11T23:21:10.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:09 smithi090 ceph-mon[35130]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-11T23:21:10.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:09 smithi090 ceph-mon[35130]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:21:10.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:09 smithi090 ceph-mon[35130]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T23:21:10.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:09 smithi090 ceph-mon[35130]: Cluster is now healthy 2024-09-11T23:21:10.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:09 smithi097 ceph-mon[35562]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-11T23:21:10.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:09 smithi097 ceph-mon[35562]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:21:10.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:09 smithi097 ceph-mon[35562]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T23:21:10.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:09 smithi097 ceph-mon[35562]: Cluster is now healthy 2024-09-11T23:21:10.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.267+0000 7f64938b3640 1 -- 172.21.15.43:0/14201383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f648c1008d0 msgr2=0x7f648c100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:10.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.267+0000 7f64938b3640 1 --2- 172.21.15.43:0/14201383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f648c1008d0 0x7f648c100cb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6488009900 tx=0x7f648802a410 comp rx=0 tx=0).stop 2024-09-11T23:21:10.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.267+0000 7f64938b3640 1 -- 172.21.15.43:0/14201383 shutdown_connections 2024-09-11T23:21:10.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.267+0000 7f64938b3640 1 --2- 172.21.15.43:0/14201383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f648c10e480 0x7f648c110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:10.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.267+0000 7f64938b3640 1 --2- 172.21.15.43:0/14201383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f648c101280 0x7f648c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:10.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.267+0000 7f64938b3640 1 --2- 172.21.15.43:0/14201383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f648c1008d0 0x7f648c100cb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:10.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.267+0000 7f64938b3640 1 -- 172.21.15.43:0/14201383 >> 172.21.15.43:0/14201383 conn(0x7f648c0fc7a0 msgr2=0x7f648c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:10.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.267+0000 7f64938b3640 1 -- 172.21.15.43:0/14201383 shutdown_connections 2024-09-11T23:21:10.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.267+0000 7f64938b3640 1 -- 172.21.15.43:0/14201383 wait complete. 2024-09-11T23:21:10.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.268+0000 7f64938b3640 1 Processor -- start 2024-09-11T23:21:10.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.268+0000 7f64938b3640 1 -- start start 2024-09-11T23:21:10.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.268+0000 7f64938b3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f648c1008d0 0x7f648c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:10.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.268+0000 7f64938b3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f648c101280 0x7f648c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.268+0000 7f64938b3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f648c10e480 0x7f648c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.268+0000 7f64938b3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f648c114250 con 0x7f648c1008d0 2024-09-11T23:21:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.268+0000 7f64938b3640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f648c1140d0 con 0x7f648c10e480 2024-09-11T23:21:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.268+0000 7f64938b3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f648c1143d0 con 0x7f648c101280 2024-09-11T23:21:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.268+0000 7f649164f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f648c1008d0 0x7f648c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.268+0000 7f649164f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f648c1008d0 0x7f648c19a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38520/0 (socket says 172.21.15.43:38520) 2024-09-11T23:21:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.268+0000 7f649164f640 1 -- 172.21.15.43:0/3767676054 learned_addr learned my addr 172.21.15.43:0/3767676054 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.268+0000 7f6490e4e640 1 --2- 172.21.15.43:0/3767676054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f648c101280 0x7f648c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.269+0000 7f649164f640 1 -- 172.21.15.43:0/3767676054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f648c101280 msgr2=0x7f648c19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.269+0000 7f6491e50640 1 --2- 172.21.15.43:0/3767676054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f648c10e480 0x7f648c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.269+0000 7f649164f640 1 --2- 172.21.15.43:0/3767676054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f648c101280 0x7f648c19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.269+0000 7f649164f640 1 -- 172.21.15.43:0/3767676054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f648c10e480 msgr2=0x7f648c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:10.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.269+0000 7f649164f640 1 --2- 172.21.15.43:0/3767676054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f648c10e480 0x7f648c19efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:10.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.269+0000 7f649164f640 1 -- 172.21.15.43:0/3767676054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648c19f6c0 con 0x7f648c1008d0 2024-09-11T23:21:10.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.269+0000 7f6490e4e640 1 --2- 172.21.15.43:0/3767676054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f648c101280 0x7f648c19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:10.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.269+0000 7f649164f640 1 --2- 172.21.15.43:0/3767676054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f648c1008d0 0x7f648c19a710 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6488006010 tx=0x7f6488003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:10.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.269+0000 7f64827fc640 1 -- 172.21.15.43:0/3767676054 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648803f070 con 0x7f648c1008d0 2024-09-11T23:21:10.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.269+0000 7f64938b3640 1 -- 172.21.15.43:0/3767676054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f648c19f950 con 0x7f648c1008d0 2024-09-11T23:21:10.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.269+0000 7f64827fc640 1 -- 172.21.15.43:0/3767676054 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6488031c90 con 0x7f648c1008d0 2024-09-11T23:21:10.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.269+0000 7f64827fc640 1 -- 172.21.15.43:0/3767676054 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488035030 con 0x7f648c1008d0 2024-09-11T23:21:10.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.269+0000 7f64938b3640 1 -- 172.21.15.43:0/3767676054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f648c1a7290 con 0x7f648c1008d0 2024-09-11T23:21:10.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.270+0000 7f64938b3640 1 -- 172.21.15.43:0/3767676054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f648c100cb0 con 0x7f648c1008d0 2024-09-11T23:21:10.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.271+0000 7f64827fc640 1 -- 172.21.15.43:0/3767676054 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6488042070 con 0x7f648c1008d0 2024-09-11T23:21:10.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.272+0000 7f64827fc640 1 --2- 172.21.15.43:0/3767676054 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f64780a2c10 0x7f64780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:10.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.272+0000 7f64827fc640 1 -- 172.21.15.43:0/3767676054 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f64880f00d0 con 0x7f648c1008d0 2024-09-11T23:21:10.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.272+0000 7f6490e4e640 1 --2- 172.21.15.43:0/3767676054 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f64780a2c10 0x7f64780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:10.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.272+0000 7f6490e4e640 1 --2- 172.21.15.43:0/3767676054 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f64780a2c10 0x7f64780a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f648c1022e0 tx=0x7f647c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:10.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.275+0000 7f64827fc640 1 -- 172.21.15.43:0/3767676054 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f64880bceb0 con 0x7f648c1008d0 2024-09-11T23:21:10.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.493+0000 7f64938b3640 1 -- 172.21.15.43:0/3767676054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f648c109610 con 0x7f648c1008d0 2024-09-11T23:21:10.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.493+0000 7f64827fc640 1 -- 172.21.15.43:0/3767676054 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f64880bc7b0 con 0x7f648c1008d0 2024-09-11T23:21:10.493 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:21:10.493 INFO:teuthology.orchestra.run.smithi043.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T23:21:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.495+0000 7f64938b3640 1 -- 172.21.15.43:0/3767676054 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f64780a2c10 msgr2=0x7f64780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.495+0000 7f64938b3640 1 --2- 172.21.15.43:0/3767676054 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f64780a2c10 0x7f64780a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f648c1022e0 tx=0x7f647c008040 comp rx=0 tx=0).stop 2024-09-11T23:21:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.495+0000 7f64938b3640 1 -- 172.21.15.43:0/3767676054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f648c1008d0 msgr2=0x7f648c19a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.495+0000 7f64938b3640 1 --2- 172.21.15.43:0/3767676054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f648c1008d0 0x7f648c19a710 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6488006010 tx=0x7f6488003ba0 comp rx=0 tx=0).stop 2024-09-11T23:21:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.495+0000 7f64938b3640 1 -- 172.21.15.43:0/3767676054 shutdown_connections 2024-09-11T23:21:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.495+0000 7f64938b3640 1 --2- 172.21.15.43:0/3767676054 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f64780a2c10 0x7f64780a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.495+0000 7f64938b3640 1 --2- 172.21.15.43:0/3767676054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f648c10e480 0x7f648c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.495+0000 7f64938b3640 1 --2- 172.21.15.43:0/3767676054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f648c101280 0x7f648c19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:10.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.495+0000 7f64938b3640 1 --2- 172.21.15.43:0/3767676054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f648c1008d0 0x7f648c19a710 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:10.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.495+0000 7f64938b3640 1 -- 172.21.15.43:0/3767676054 >> 172.21.15.43:0/3767676054 conn(0x7f648c0fc7a0 msgr2=0x7f648c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:10.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.495+0000 7f64938b3640 1 -- 172.21.15.43:0/3767676054 shutdown_connections 2024-09-11T23:21:10.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:10.495+0000 7f64938b3640 1 -- 172.21.15.43:0/3767676054 wait complete. 2024-09-11T23:21:11.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:10 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3767676054' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T23:21:11.171 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-11T23:21:11.172 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:21:11.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:10 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3767676054' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T23:21:11.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:10 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3767676054' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T23:21:11.430 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.901+0000 7f191b74d640 1 -- 172.21.15.43:0/214069888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1914106a80 msgr2=0x7f1914106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:11.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.901+0000 7f191b74d640 1 --2- 172.21.15.43:0/214069888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1914106a80 0x7f1914106e60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1910009930 tx=0x7f191002a420 comp rx=0 tx=0).stop 2024-09-11T23:21:11.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.901+0000 7f191b74d640 1 -- 172.21.15.43:0/214069888 shutdown_connections 2024-09-11T23:21:11.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.901+0000 7f191b74d640 1 --2- 172.21.15.43:0/214069888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1914101420 0x7f191410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:11.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.901+0000 7f191b74d640 1 --2- 172.21.15.43:0/214069888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1914100a80 0x7f1914100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:11.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.901+0000 7f191b74d640 1 --2- 172.21.15.43:0/214069888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1914106a80 0x7f1914106e60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:11.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.901+0000 7f191b74d640 1 -- 172.21.15.43:0/214069888 >> 172.21.15.43:0/214069888 conn(0x7f19140fc7a0 msgr2=0x7f19140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:11.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.901+0000 7f191b74d640 1 -- 172.21.15.43:0/214069888 shutdown_connections 2024-09-11T23:21:11.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.901+0000 7f191b74d640 1 -- 172.21.15.43:0/214069888 wait complete. 2024-09-11T23:21:11.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.902+0000 7f191b74d640 1 Processor -- start 2024-09-11T23:21:11.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.902+0000 7f191b74d640 1 -- start start 2024-09-11T23:21:11.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.902+0000 7f191b74d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1914100a80 0x7f1914197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:11.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.902+0000 7f191b74d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1914101420 0x7f1914197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:11.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.902+0000 7f191b74d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1914106a80 0x7f191419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:11.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.902+0000 7f191b74d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1914110e40 con 0x7f1914101420 2024-09-11T23:21:11.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.902+0000 7f191b74d640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1914110cc0 con 0x7f1914100a80 2024-09-11T23:21:11.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.902+0000 7f191b74d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1914110fc0 con 0x7f1914106a80 2024-09-11T23:21:11.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.902+0000 7f1918ce8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1914101420 0x7f1914197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:11.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.902+0000 7f1918ce8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1914101420 0x7f1914197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38538/0 (socket says 172.21.15.43:38538) 2024-09-11T23:21:11.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.902+0000 7f1918ce8640 1 -- 172.21.15.43:0/4019285134 learned_addr learned my addr 172.21.15.43:0/4019285134 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:11.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.902+0000 7f19194e9640 1 --2- 172.21.15.43:0/4019285134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1914100a80 0x7f1914197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:11.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.902+0000 7f1919cea640 1 --2- 172.21.15.43:0/4019285134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1914106a80 0x7f191419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:11.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.902+0000 7f1918ce8640 1 -- 172.21.15.43:0/4019285134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1914106a80 msgr2=0x7f191419bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:11.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.902+0000 7f1918ce8640 1 --2- 172.21.15.43:0/4019285134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1914106a80 0x7f191419bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:11.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.902+0000 7f1918ce8640 1 -- 172.21.15.43:0/4019285134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1914100a80 msgr2=0x7f1914197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:11.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.902+0000 7f1918ce8640 1 --2- 172.21.15.43:0/4019285134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1914100a80 0x7f1914197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:11.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.902+0000 7f1918ce8640 1 -- 172.21.15.43:0/4019285134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191419c2b0 con 0x7f1914101420 2024-09-11T23:21:11.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.903+0000 7f1918ce8640 1 --2- 172.21.15.43:0/4019285134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1914101420 0x7f1914197840 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f190400c920 tx=0x7f190400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:11.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.903+0000 7f190a7fc640 1 -- 172.21.15.43:0/4019285134 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1904007c70 con 0x7f1914101420 2024-09-11T23:21:11.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.903+0000 7f190a7fc640 1 -- 172.21.15.43:0/4019285134 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1904007e10 con 0x7f1914101420 2024-09-11T23:21:11.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.903+0000 7f191b74d640 1 -- 172.21.15.43:0/4019285134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f191419c5a0 con 0x7f1914101420 2024-09-11T23:21:11.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.903+0000 7f190a7fc640 1 -- 172.21.15.43:0/4019285134 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1904018890 con 0x7f1914101420 2024-09-11T23:21:11.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.903+0000 7f191b74d640 1 -- 172.21.15.43:0/4019285134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19141a3e80 con 0x7f1914101420 2024-09-11T23:21:11.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.904+0000 7f191b74d640 1 -- 172.21.15.43:0/4019285134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1914100ee0 con 0x7f1914101420 2024-09-11T23:21:11.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.905+0000 7f190a7fc640 1 -- 172.21.15.43:0/4019285134 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1904018a50 con 0x7f1914101420 2024-09-11T23:21:11.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.906+0000 7f190a7fc640 1 --2- 172.21.15.43:0/4019285134 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f19000a2c10 0x7f19000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:11.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.906+0000 7f19194e9640 1 --2- 172.21.15.43:0/4019285134 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f19000a2c10 0x7f19000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:11.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.906+0000 7f190a7fc640 1 -- 172.21.15.43:0/4019285134 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f19040cf2b0 con 0x7f1914101420 2024-09-11T23:21:11.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.907+0000 7f19194e9640 1 --2- 172.21.15.43:0/4019285134 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f19000a2c10 0x7f19000a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1910009900 tx=0x7f19100058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:11.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:11.910+0000 7f190a7fc640 1 -- 172.21.15.43:0/4019285134 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1904014030 con 0x7f1914101420 2024-09-11T23:21:12.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:11 smithi043 ceph-mon[30950]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-11T23:21:12.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:12.123+0000 7f191b74d640 1 -- 172.21.15.43:0/4019285134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f191402d7c0 con 0x7f1914101420 2024-09-11T23:21:12.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:12.124+0000 7f190a7fc640 1 -- 172.21.15.43:0/4019285134 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11896 (secure 0 0 0) 0x7f190409c090 con 0x7f1914101420 2024-09-11T23:21:12.125 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:21:12.125 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":23,"btime":"2024-09-11T23:20:12:774166+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:20:12.440327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24457,24470,24497]},"id":1}]} 2024-09-11T23:21:12.125 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 23 2024-09-11T23:21:12.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:12.126+0000 7f191b74d640 1 -- 172.21.15.43:0/4019285134 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f19000a2c10 msgr2=0x7f19000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:12.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:12.126+0000 7f191b74d640 1 --2- 172.21.15.43:0/4019285134 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f19000a2c10 0x7f19000a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1910009900 tx=0x7f19100058d0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:12.126+0000 7f191b74d640 1 -- 172.21.15.43:0/4019285134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1914101420 msgr2=0x7f1914197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:12.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:12.126+0000 7f191b74d640 1 --2- 172.21.15.43:0/4019285134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1914101420 0x7f1914197840 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f190400c920 tx=0x7f190400cde0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:12.126+0000 7f191b74d640 1 -- 172.21.15.43:0/4019285134 shutdown_connections 2024-09-11T23:21:12.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:12.126+0000 7f191b74d640 1 --2- 172.21.15.43:0/4019285134 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f19000a2c10 0x7f19000a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:12.126+0000 7f191b74d640 1 --2- 172.21.15.43:0/4019285134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1914106a80 0x7f191419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:12.126+0000 7f191b74d640 1 --2- 172.21.15.43:0/4019285134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1914101420 0x7f1914197840 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:12.126+0000 7f191b74d640 1 --2- 172.21.15.43:0/4019285134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1914100a80 0x7f1914197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:12.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:12.126+0000 7f191b74d640 1 -- 172.21.15.43:0/4019285134 >> 172.21.15.43:0/4019285134 conn(0x7f19140fc7a0 msgr2=0x7f19140fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:12.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:12.127+0000 7f191b74d640 1 -- 172.21.15.43:0/4019285134 shutdown_connections 2024-09-11T23:21:12.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:12.127+0000 7f191b74d640 1 -- 172.21.15.43:0/4019285134 wait complete. 2024-09-11T23:21:12.203 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:11 smithi097 ceph-mon[35562]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-11T23:21:12.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:11 smithi090 ceph-mon[35130]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-09-11T23:21:12.847 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:21:13.109 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:13.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:12 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/4019285134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:13.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:12 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/4019285134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:13.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:12 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/4019285134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:13.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.608+0000 7f55feb8a640 1 -- 172.21.15.43:0/1872551541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f8101de0 msgr2=0x7f55f81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:13.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.608+0000 7f55feb8a640 1 --2- 172.21.15.43:0/1872551541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f8101de0 0x7f55f81021c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009900 tx=0x7f55ec02a410 comp rx=0 tx=0).stop 2024-09-11T23:21:13.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.608+0000 7f55feb8a640 1 -- 172.21.15.43:0/1872551541 shutdown_connections 2024-09-11T23:21:13.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.608+0000 7f55feb8a640 1 --2- 172.21.15.43:0/1872551541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55f8106f10 0x7f55f810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:13.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.608+0000 7f55feb8a640 1 --2- 172.21.15.43:0/1872551541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55f8102790 0x7f55f81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:13.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.608+0000 7f55feb8a640 1 --2- 172.21.15.43:0/1872551541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f8101de0 0x7f55f81021c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:13.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.608+0000 7f55feb8a640 1 -- 172.21.15.43:0/1872551541 >> 172.21.15.43:0/1872551541 conn(0x7f55f80fd8c0 msgr2=0x7f55f80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:13.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.608+0000 7f55feb8a640 1 -- 172.21.15.43:0/1872551541 shutdown_connections 2024-09-11T23:21:13.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.608+0000 7f55feb8a640 1 -- 172.21.15.43:0/1872551541 wait complete. 2024-09-11T23:21:13.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.609+0000 7f55feb8a640 1 Processor -- start 2024-09-11T23:21:13.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.609+0000 7f55feb8a640 1 -- start start 2024-09-11T23:21:13.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.609+0000 7f55feb8a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55f8101de0 0x7f55f8198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:13.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.609+0000 7f55feb8a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55f8102790 0x7f55f8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:13.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.609+0000 7f55feb8a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f8106f10 0x7f55f819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:13.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.609+0000 7f55feb8a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8111fc0 con 0x7f55f8106f10 2024-09-11T23:21:13.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.609+0000 7f55feb8a640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8111e40 con 0x7f55f8101de0 2024-09-11T23:21:13.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.609+0000 7f55feb8a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8112140 con 0x7f55f8102790 2024-09-11T23:21:13.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.610+0000 7f55fd127640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f8106f10 0x7f55f819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:13.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.610+0000 7f55fd127640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f8106f10 0x7f55f819ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38552/0 (socket says 172.21.15.43:38552) 2024-09-11T23:21:13.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.610+0000 7f55fd127640 1 -- 172.21.15.43:0/1901889825 learned_addr learned my addr 172.21.15.43:0/1901889825 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:13.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.610+0000 7f55fc926640 1 --2- 172.21.15.43:0/1901889825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55f8101de0 0x7f55f8198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:13.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.610+0000 7f55fd127640 1 -- 172.21.15.43:0/1901889825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55f8102790 msgr2=0x7f55f8198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:13.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.610+0000 7f55f7fff640 1 --2- 172.21.15.43:0/1901889825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55f8102790 0x7f55f8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:13.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.610+0000 7f55fd127640 1 --2- 172.21.15.43:0/1901889825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55f8102790 0x7f55f8198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:13.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.610+0000 7f55fd127640 1 -- 172.21.15.43:0/1901889825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55f8101de0 msgr2=0x7f55f8198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:13.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.610+0000 7f55fd127640 1 --2- 172.21.15.43:0/1901889825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55f8101de0 0x7f55f8198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:13.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.610+0000 7f55fd127640 1 -- 172.21.15.43:0/1901889825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f819d3d0 con 0x7f55f8106f10 2024-09-11T23:21:13.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.610+0000 7f55fc926640 1 --2- 172.21.15.43:0/1901889825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55f8101de0 0x7f55f8198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:13.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.610+0000 7f55fd127640 1 --2- 172.21.15.43:0/1901889825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f8106f10 0x7f55f819ccf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f55f000eaa0 tx=0x7f55f000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.610+0000 7f55f5ffb640 1 -- 172.21.15.43:0/1901889825 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f000cdb0 con 0x7f55f8106f10 2024-09-11T23:21:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.610+0000 7f55f5ffb640 1 -- 172.21.15.43:0/1901889825 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f0005020 con 0x7f55f8106f10 2024-09-11T23:21:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.610+0000 7f55feb8a640 1 -- 172.21.15.43:0/1901889825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f819d6c0 con 0x7f55f8106f10 2024-09-11T23:21:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.610+0000 7f55f5ffb640 1 -- 172.21.15.43:0/1901889825 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f00116b0 con 0x7f55f8106f10 2024-09-11T23:21:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.610+0000 7f55feb8a640 1 -- 172.21.15.43:0/1901889825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f81a4fa0 con 0x7f55f8106f10 2024-09-11T23:21:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.611+0000 7f55feb8a640 1 -- 172.21.15.43:0/1901889825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f81021c0 con 0x7f55f8106f10 2024-09-11T23:21:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.612+0000 7f55f5ffb640 1 -- 172.21.15.43:0/1901889825 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f55f0005560 con 0x7f55f8106f10 2024-09-11T23:21:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.613+0000 7f55f5ffb640 1 --2- 172.21.15.43:0/1901889825 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f55e80a2c90 0x7f55e80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.613+0000 7f55fc926640 1 --2- 172.21.15.43:0/1901889825 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f55e80a2c90 0x7f55e80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.613+0000 7f55f5ffb640 1 -- 172.21.15.43:0/1901889825 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f55f00cf440 con 0x7f55f8106f10 2024-09-11T23:21:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.614+0000 7f55fc926640 1 --2- 172.21.15.43:0/1901889825 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f55e80a2c90 0x7f55e80a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f55ec000c00 tx=0x7f55ec005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:13.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.617+0000 7f55f5ffb640 1 -- 172.21.15.43:0/1901889825 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55f0016030 con 0x7f55f8106f10 2024-09-11T23:21:13.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.831+0000 7f55feb8a640 1 -- 172.21.15.43:0/1901889825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55f802d7c0 con 0x7f55f8106f10 2024-09-11T23:21:13.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.832+0000 7f55f5ffb640 1 -- 172.21.15.43:0/1901889825 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11896 (secure 0 0 0) 0x7f55f009c1c0 con 0x7f55f8106f10 2024-09-11T23:21:13.832 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:21:13.832 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":23,"btime":"2024-09-11T23:20:12:774166+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:20:12.440327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24457,24470,24497]},"id":1}]} 2024-09-11T23:21:13.832 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 23 2024-09-11T23:21:13.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.834+0000 7f55feb8a640 1 -- 172.21.15.43:0/1901889825 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f55e80a2c90 msgr2=0x7f55e80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:13.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.834+0000 7f55feb8a640 1 --2- 172.21.15.43:0/1901889825 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f55e80a2c90 0x7f55e80a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f55ec000c00 tx=0x7f55ec005a40 comp rx=0 tx=0).stop 2024-09-11T23:21:13.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.834+0000 7f55feb8a640 1 -- 172.21.15.43:0/1901889825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f8106f10 msgr2=0x7f55f819ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:13.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.834+0000 7f55feb8a640 1 --2- 172.21.15.43:0/1901889825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f8106f10 0x7f55f819ccf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f55f000eaa0 tx=0x7f55f000ef60 comp rx=0 tx=0).stop 2024-09-11T23:21:13.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.834+0000 7f55feb8a640 1 -- 172.21.15.43:0/1901889825 shutdown_connections 2024-09-11T23:21:13.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.834+0000 7f55feb8a640 1 --2- 172.21.15.43:0/1901889825 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f55e80a2c90 0x7f55e80a5150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:13.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.834+0000 7f55feb8a640 1 --2- 172.21.15.43:0/1901889825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f8106f10 0x7f55f819ccf0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:13.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.834+0000 7f55feb8a640 1 --2- 172.21.15.43:0/1901889825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55f8102790 0x7f55f8198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:13.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.834+0000 7f55feb8a640 1 --2- 172.21.15.43:0/1901889825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55f8101de0 0x7f55f8198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:13.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.834+0000 7f55feb8a640 1 -- 172.21.15.43:0/1901889825 >> 172.21.15.43:0/1901889825 conn(0x7f55f80fd8c0 msgr2=0x7f55f80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:13.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.834+0000 7f55feb8a640 1 -- 172.21.15.43:0/1901889825 shutdown_connections 2024-09-11T23:21:13.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:13.834+0000 7f55feb8a640 1 -- 172.21.15.43:0/1901889825 wait complete. 2024-09-11T23:21:14.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:13 smithi043 ceph-mon[30950]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T23:21:14.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:13 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1901889825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:14.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:13 smithi090 ceph-mon[35130]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T23:21:14.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:13 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1901889825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:14.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:13 smithi097 ceph-mon[35562]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T23:21:14.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:13 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1901889825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:14.537 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:21:14.799 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:15.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.260+0000 7f7e1dcdd640 1 -- 172.21.15.43:0/713516809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e181082a0 msgr2=0x7f7e1810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:15.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.260+0000 7f7e1dcdd640 1 --2- 172.21.15.43:0/713516809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e181082a0 0x7f7e1810ec40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009cd0 tx=0x7f7e1002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:21:15.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.260+0000 7f7e1dcdd640 1 -- 172.21.15.43:0/713516809 shutdown_connections 2024-09-11T23:21:15.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.260+0000 7f7e1dcdd640 1 --2- 172.21.15.43:0/713516809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e181082a0 0x7f7e1810ec40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:15.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.260+0000 7f7e1dcdd640 1 --2- 172.21.15.43:0/713516809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e181036e0 0x7f7e18107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:15.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.260+0000 7f7e1dcdd640 1 --2- 172.21.15.43:0/713516809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e18102d30 0x7f7e18103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:15.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.260+0000 7f7e1dcdd640 1 -- 172.21.15.43:0/713516809 >> 172.21.15.43:0/713516809 conn(0x7f7e180fc7d0 msgr2=0x7f7e180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:15.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.260+0000 7f7e1dcdd640 1 -- 172.21.15.43:0/713516809 shutdown_connections 2024-09-11T23:21:15.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.260+0000 7f7e1dcdd640 1 -- 172.21.15.43:0/713516809 wait complete. 2024-09-11T23:21:15.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.261+0000 7f7e1dcdd640 1 Processor -- start 2024-09-11T23:21:15.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.261+0000 7f7e1dcdd640 1 -- start start 2024-09-11T23:21:15.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.261+0000 7f7e1dcdd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e18102d30 0x7f7e18198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:15.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.261+0000 7f7e1dcdd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e181036e0 0x7f7e18198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:15.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.261+0000 7f7e1dcdd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e181082a0 0x7f7e1819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:15.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.261+0000 7f7e1dcdd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7e18112100 con 0x7f7e181082a0 2024-09-11T23:21:15.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.262+0000 7f7e1dcdd640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7e18111f80 con 0x7f7e18102d30 2024-09-11T23:21:15.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.262+0000 7f7e1dcdd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7e18112280 con 0x7f7e181036e0 2024-09-11T23:21:15.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.262+0000 7f7e17fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e181082a0 0x7f7e1819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:15.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.262+0000 7f7e17fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e181082a0 0x7f7e1819ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38562/0 (socket says 172.21.15.43:38562) 2024-09-11T23:21:15.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.262+0000 7f7e17fff640 1 -- 172.21.15.43:0/1669759748 learned_addr learned my addr 172.21.15.43:0/1669759748 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:15.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.262+0000 7f7e177fe640 1 --2- 172.21.15.43:0/1669759748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e18102d30 0x7f7e18198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:15.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.262+0000 7f7e16ffd640 1 --2- 172.21.15.43:0/1669759748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e181036e0 0x7f7e18198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:15.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.262+0000 7f7e17fff640 1 -- 172.21.15.43:0/1669759748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e181036e0 msgr2=0x7f7e18198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:15.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.262+0000 7f7e17fff640 1 --2- 172.21.15.43:0/1669759748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e181036e0 0x7f7e18198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:15.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.262+0000 7f7e17fff640 1 -- 172.21.15.43:0/1669759748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e18102d30 msgr2=0x7f7e18198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:15.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.262+0000 7f7e17fff640 1 --2- 172.21.15.43:0/1669759748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e18102d30 0x7f7e18198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:15.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.262+0000 7f7e17fff640 1 -- 172.21.15.43:0/1669759748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1819d500 con 0x7f7e181082a0 2024-09-11T23:21:15.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.263+0000 7f7e17fff640 1 --2- 172.21.15.43:0/1669759748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e181082a0 0x7f7e1819ce20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7e10006010 tx=0x7f7e10031df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:15.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.263+0000 7f7e14ff9640 1 -- 172.21.15.43:0/1669759748 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1003fa10 con 0x7f7e181082a0 2024-09-11T23:21:15.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.263+0000 7f7e14ff9640 1 -- 172.21.15.43:0/1669759748 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e1003fbb0 con 0x7f7e181082a0 2024-09-11T23:21:15.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.263+0000 7f7e1dcdd640 1 -- 172.21.15.43:0/1669759748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e1819d790 con 0x7f7e181082a0 2024-09-11T23:21:15.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.263+0000 7f7e14ff9640 1 -- 172.21.15.43:0/1669759748 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10030900 con 0x7f7e181082a0 2024-09-11T23:21:15.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.263+0000 7f7e1dcdd640 1 -- 172.21.15.43:0/1669759748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e181008c0 con 0x7f7e181082a0 2024-09-11T23:21:15.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.264+0000 7f7e1dcdd640 1 -- 172.21.15.43:0/1669759748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e1806a850 con 0x7f7e181082a0 2024-09-11T23:21:15.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.266+0000 7f7e14ff9640 1 -- 172.21.15.43:0/1669759748 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7e10030aa0 con 0x7f7e181082a0 2024-09-11T23:21:15.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.266+0000 7f7e14ff9640 1 --2- 172.21.15.43:0/1669759748 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f7e040a2c10 0x7f7e040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:15.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.266+0000 7f7e14ff9640 1 -- 172.21.15.43:0/1669759748 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f7e1002e030 con 0x7f7e181082a0 2024-09-11T23:21:15.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.266+0000 7f7e177fe640 1 --2- 172.21.15.43:0/1669759748 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f7e040a2c10 0x7f7e040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:15.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.267+0000 7f7e177fe640 1 --2- 172.21.15.43:0/1669759748 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f7e040a2c10 0x7f7e040a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009890 tx=0x7f7e00008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:15.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.269+0000 7f7e14ff9640 1 -- 172.21.15.43:0/1669759748 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7e1004f060 con 0x7f7e181082a0 2024-09-11T23:21:15.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.447+0000 7f7e1dcdd640 1 -- 172.21.15.43:0/1669759748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e181998b0 con 0x7f7e181082a0 2024-09-11T23:21:15.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.449+0000 7f7e14ff9640 1 -- 172.21.15.43:0/1669759748 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22671 (secure 0 0 0) 0x7f7e100c1fe0 con 0x7f7e181082a0 2024-09-11T23:21:15.449 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:21:15.449 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":84,"fsid":"25c0cd84-7093-11ef-bcea-c7b262605968","created":"2024-09-11T23:12:03.504406+0000","modified":"2024-09-11T23:21:02.348513+0000","last_up_change":"2024-09-11T23:19:07.797873+0000","last_in_change":"2024-09-11T23:18:44.595176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:15:17.058713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:19:56.221406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:19:58.705875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24076596-3909-4c87-85f1-cbf168070dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6803","nonce":43839997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6805","nonce":43839997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6809","nonce":43839997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6807","nonce":43839997}]},"public_addr":"172.21.15.43:6803/43839997","cluster_addr":"172.21.15.43:6805/43839997","heartbeat_back_addr":"172.21.15.43:6809/43839997","heartbeat_front_addr":"172.21.15.43:6807/43839997","state":["exists","up"]},{"osd":1,"uuid":"a0472a57-243c-489d-a247-972ee10a71f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6801","nonce":1825149246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6803","nonce":1825149246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6807","nonce":1825149246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6805","nonce":1825149246}]},"public_addr":"172.21.15.90:6801/1825149246","cluster_addr":"172.21.15.90:6803/1825149246","heartbeat_back_addr":"172.21.15.90:6807/1825149246","heartbeat_front_addr":"172.21.15.90:6805/1825149246","state":["exists","up"]},{"osd":2,"uuid":"40366536-0fcf-45a0-b8e3-49178401dc34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6801","nonce":1456846443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6803","nonce":1456846443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6807","nonce":1456846443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6805","nonce":1456846443}]},"public_addr":"172.21.15.97:6801/1456846443","cluster_addr":"172.21.15.97:6803/1456846443","heartbeat_back_addr":"172.21.15.97:6807/1456846443","heartbeat_front_addr":"172.21.15.97:6805/1456846443","state":["exists","up"]},{"osd":3,"uuid":"3062133e-000b-4421-9d53-ac434b9ed9b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6811","nonce":1962837804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6813","nonce":1962837804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6817","nonce":1962837804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6815","nonce":1962837804}]},"public_addr":"172.21.15.43:6811/1962837804","cluster_addr":"172.21.15.43:6813/1962837804","heartbeat_back_addr":"172.21.15.43:6817/1962837804","heartbeat_front_addr":"172.21.15.43:6815/1962837804","state":["exists","up"]},{"osd":4,"uuid":"2e73fa4b-f22c-4319-828d-87b91b9f19f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6809","nonce":181614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6811","nonce":181614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6815","nonce":181614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6813","nonce":181614173}]},"public_addr":"172.21.15.90:6809/181614173","cluster_addr":"172.21.15.90:6811/181614173","heartbeat_back_addr":"172.21.15.90:6815/181614173","heartbeat_front_addr":"172.21.15.90:6813/181614173","state":["exists","up"]},{"osd":5,"uuid":"1e1ad4f9-e724-440f-b515-3ec1ad97d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6809","nonce":1540039543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6811","nonce":1540039543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6815","nonce":1540039543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6813","nonce":1540039543}]},"public_addr":"172.21.15.97:6809/1540039543","cluster_addr":"172.21.15.97:6811/1540039543","heartbeat_back_addr":"172.21.15.97:6815/1540039543","heartbeat_front_addr":"172.21.15.97:6813/1540039543","state":["exists","up"]},{"osd":6,"uuid":"d34df8df-f40d-4159-8b29-5401b1b0e431","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6819","nonce":3785677637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6821","nonce":3785677637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6825","nonce":3785677637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6823","nonce":3785677637}]},"public_addr":"172.21.15.43:6819/3785677637","cluster_addr":"172.21.15.43:6821/3785677637","heartbeat_back_addr":"172.21.15.43:6825/3785677637","heartbeat_front_addr":"172.21.15.43:6823/3785677637","state":["exists","up"]},{"osd":7,"uuid":"b2e01078-528b-49fd-b000-9f584f784b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6817","nonce":236874962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6819","nonce":236874962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6823","nonce":236874962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6821","nonce":236874962}]},"public_addr":"172.21.15.90:6817/236874962","cluster_addr":"172.21.15.90:6819/236874962","heartbeat_back_addr":"172.21.15.90:6823/236874962","heartbeat_front_addr":"172.21.15.90:6821/236874962","state":["exists","up"]},{"osd":8,"uuid":"abef8f27-bf9b-4d63-bfbe-f967fc3d8b17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6817","nonce":1707327301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6819","nonce":1707327301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6823","nonce":1707327301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6821","nonce":1707327301}]},"public_addr":"172.21.15.97:6817/1707327301","cluster_addr":"172.21.15.97:6819/1707327301","heartbeat_back_addr":"172.21.15.97:6823/1707327301","heartbeat_front_addr":"172.21.15.97:6821/1707327301","state":["exists","up"]},{"osd":9,"uuid":"190ab8b2-a835-47e6-9b39-16a58331c58a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6827","nonce":3170306011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6829","nonce":3170306011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6833","nonce":3170306011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6831","nonce":3170306011}]},"public_addr":"172.21.15.43:6827/3170306011","cluster_addr":"172.21.15.43:6829/3170306011","heartbeat_back_addr":"172.21.15.43:6833/3170306011","heartbeat_front_addr":"172.21.15.43:6831/3170306011","state":["exists","up"]},{"osd":10,"uuid":"b32ab5fe-466f-4c8e-8696-cd9f820a3263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6825","nonce":59487041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6827","nonce":59487041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6831","nonce":59487041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6829","nonce":59487041}]},"public_addr":"172.21.15.90:6825/59487041","cluster_addr":"172.21.15.90:6827/59487041","heartbeat_back_addr":"172.21.15.90:6831/59487041","heartbeat_front_addr":"172.21.15.90:6829/59487041","state":["exists","up"]},{"osd":11,"uuid":"d0faa589-21c8-4a9f-8a73-813aafe6d33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6825","nonce":194699315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6827","nonce":194699315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6831","nonce":194699315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6829","nonce":194699315}]},"public_addr":"172.21.15.97:6825/194699315","cluster_addr":"172.21.15.97:6827/194699315","heartbeat_back_addr":"172.21.15.97:6831/194699315","heartbeat_front_addr":"172.21.15.97:6829/194699315","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:23.381247+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:48.843190+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:13.538979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:40.136983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:05.513790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:31.247090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:56.635221+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:22.667327+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:47.910422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:14.353914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:40.584503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:19:06.476766+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/76575371":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/4006124758":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/120495444":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6800/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:6800/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/206044413":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1512201905":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1375379108":"2024-09-12T23:12:35.756598+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:21:15.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.452+0000 7f7e1dcdd640 1 -- 172.21.15.43:0/1669759748 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f7e040a2c10 msgr2=0x7f7e040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:15.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.452+0000 7f7e1dcdd640 1 --2- 172.21.15.43:0/1669759748 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f7e040a2c10 0x7f7e040a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009890 tx=0x7f7e00008040 comp rx=0 tx=0).stop 2024-09-11T23:21:15.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.452+0000 7f7e1dcdd640 1 -- 172.21.15.43:0/1669759748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e181082a0 msgr2=0x7f7e1819ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:15.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.452+0000 7f7e1dcdd640 1 --2- 172.21.15.43:0/1669759748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e181082a0 0x7f7e1819ce20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7e10006010 tx=0x7f7e10031df0 comp rx=0 tx=0).stop 2024-09-11T23:21:15.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.452+0000 7f7e1dcdd640 1 -- 172.21.15.43:0/1669759748 shutdown_connections 2024-09-11T23:21:15.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.452+0000 7f7e1dcdd640 1 --2- 172.21.15.43:0/1669759748 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f7e040a2c10 0x7f7e040a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:15.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.452+0000 7f7e1dcdd640 1 --2- 172.21.15.43:0/1669759748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e181082a0 0x7f7e1819ce20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:15.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.452+0000 7f7e1dcdd640 1 --2- 172.21.15.43:0/1669759748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e181036e0 0x7f7e18198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:15.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.452+0000 7f7e1dcdd640 1 --2- 172.21.15.43:0/1669759748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e18102d30 0x7f7e18198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:15.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.452+0000 7f7e1dcdd640 1 -- 172.21.15.43:0/1669759748 >> 172.21.15.43:0/1669759748 conn(0x7f7e180fc7d0 msgr2=0x7f7e18106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:15.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.452+0000 7f7e1dcdd640 1 -- 172.21.15.43:0/1669759748 shutdown_connections 2024-09-11T23:21:15.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:15.452+0000 7f7e1dcdd640 1 -- 172.21.15.43:0/1669759748 wait complete. 2024-09-11T23:21:16.128 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:21:16.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:15 smithi090 ceph-mon[35130]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T23:21:16.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:15 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1669759748' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:21:16.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:15 smithi097 ceph-mon[35562]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T23:21:16.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:15 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1669759748' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:21:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:15 smithi043 ceph-mon[30950]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T23:21:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:15 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1669759748' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:21:16.387 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:16.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.819+0000 7fb50b3c2640 1 -- 172.21.15.43:0/4019289487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb504101420 msgr2=0x7fb50410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:16.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.819+0000 7fb50b3c2640 1 --2- 172.21.15.43:0/4019289487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb504101420 0x7fb50410d950 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009cd0 tx=0x7fb4fc02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:21:16.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.819+0000 7fb50b3c2640 1 -- 172.21.15.43:0/4019289487 shutdown_connections 2024-09-11T23:21:16.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.819+0000 7fb50b3c2640 1 --2- 172.21.15.43:0/4019289487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb504101420 0x7fb50410d950 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.819+0000 7fb50b3c2640 1 --2- 172.21.15.43:0/4019289487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb504100a80 0x7fb504100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.819+0000 7fb50b3c2640 1 --2- 172.21.15.43:0/4019289487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb504106a80 0x7fb504106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.819+0000 7fb50b3c2640 1 -- 172.21.15.43:0/4019289487 >> 172.21.15.43:0/4019289487 conn(0x7fb5040fc7a0 msgr2=0x7fb5040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.819+0000 7fb50b3c2640 1 -- 172.21.15.43:0/4019289487 shutdown_connections 2024-09-11T23:21:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.819+0000 7fb50b3c2640 1 -- 172.21.15.43:0/4019289487 wait complete. 2024-09-11T23:21:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.820+0000 7fb50b3c2640 1 Processor -- start 2024-09-11T23:21:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.820+0000 7fb50b3c2640 1 -- start start 2024-09-11T23:21:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.820+0000 7fb50b3c2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb504100a80 0x7fb5040fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.820+0000 7fb50b3c2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb504101420 0x7fb5040fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.820+0000 7fb50b3c2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb504106a80 0x7fb5040fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.820+0000 7fb50b3c2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb504110e80 con 0x7fb504100a80 2024-09-11T23:21:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.820+0000 7fb50b3c2640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb504110d00 con 0x7fb504101420 2024-09-11T23:21:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.820+0000 7fb50b3c2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb504111000 con 0x7fb504106a80 2024-09-11T23:21:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.820+0000 7fb50915e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb504100a80 0x7fb5040fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.820+0000 7fb50915e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb504100a80 0x7fb5040fcde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38576/0 (socket says 172.21.15.43:38576) 2024-09-11T23:21:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.820+0000 7fb50915e640 1 -- 172.21.15.43:0/1139256315 learned_addr learned my addr 172.21.15.43:0/1139256315 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.820+0000 7fb50995f640 1 --2- 172.21.15.43:0/1139256315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb504106a80 0x7fb5040fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.820+0000 7fb50895d640 1 --2- 172.21.15.43:0/1139256315 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb504101420 0x7fb5040fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.820+0000 7fb50915e640 1 -- 172.21.15.43:0/1139256315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb504106a80 msgr2=0x7fb5040fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.820+0000 7fb50915e640 1 --2- 172.21.15.43:0/1139256315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb504106a80 0x7fb5040fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.820+0000 7fb50915e640 1 -- 172.21.15.43:0/1139256315 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb504101420 msgr2=0x7fb5040fd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.820+0000 7fb50915e640 1 --2- 172.21.15.43:0/1139256315 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb504101420 0x7fb5040fd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.820+0000 7fb50915e640 1 -- 172.21.15.43:0/1139256315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5041a68e0 con 0x7fb504100a80 2024-09-11T23:21:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.820+0000 7fb50895d640 1 --2- 172.21.15.43:0/1139256315 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb504101420 0x7fb5040fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.820+0000 7fb50995f640 1 --2- 172.21.15.43:0/1139256315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb504106a80 0x7fb5040fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.821+0000 7fb50915e640 1 --2- 172.21.15.43:0/1139256315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb504100a80 0x7fb5040fcde0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb50000d950 tx=0x7fb50000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.821+0000 7fb4fa7fc640 1 -- 172.21.15.43:0/1139256315 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500017070 con 0x7fb504100a80 2024-09-11T23:21:16.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.821+0000 7fb4fa7fc640 1 -- 172.21.15.43:0/1139256315 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb500009c30 con 0x7fb504100a80 2024-09-11T23:21:16.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.821+0000 7fb50b3c2640 1 -- 172.21.15.43:0/1139256315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5041a6bd0 con 0x7fb504100a80 2024-09-11T23:21:16.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.821+0000 7fb4fa7fc640 1 -- 172.21.15.43:0/1139256315 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50000f6b0 con 0x7fb504100a80 2024-09-11T23:21:16.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.821+0000 7fb50b3c2640 1 -- 172.21.15.43:0/1139256315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5041a7160 con 0x7fb504100a80 2024-09-11T23:21:16.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.822+0000 7fb50b3c2640 1 -- 172.21.15.43:0/1139256315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb504100ee0 con 0x7fb504100a80 2024-09-11T23:21:16.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.823+0000 7fb4fa7fc640 1 -- 172.21.15.43:0/1139256315 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb5000109d0 con 0x7fb504100a80 2024-09-11T23:21:16.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.824+0000 7fb4fa7fc640 1 --2- 172.21.15.43:0/1139256315 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb4f00a2c90 0x7fb4f00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:16.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.824+0000 7fb50895d640 1 --2- 172.21.15.43:0/1139256315 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb4f00a2c90 0x7fb4f00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:16.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.824+0000 7fb4fa7fc640 1 -- 172.21.15.43:0/1139256315 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7fb5000cee20 con 0x7fb504100a80 2024-09-11T23:21:16.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.824+0000 7fb50895d640 1 --2- 172.21.15.43:0/1139256315 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb4f00a2c90 0x7fb4f00a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40058c0 tx=0x7fb4f4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:16.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:16.828+0000 7fb4fa7fc640 1 -- 172.21.15.43:0/1139256315 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb500009de0 con 0x7fb504100a80 2024-09-11T23:21:17.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:17.040+0000 7fb50b3c2640 1 -- 172.21.15.43:0/1139256315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb50410a000 con 0x7fb504100a80 2024-09-11T23:21:17.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:17.041+0000 7fb4fa7fc640 1 -- 172.21.15.43:0/1139256315 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11896 (secure 0 0 0) 0x7fb50009bba0 con 0x7fb504100a80 2024-09-11T23:21:17.041 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:21:17.042 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":23,"btime":"2024-09-11T23:20:12:774166+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:20:12.440327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24457,24470,24497]},"id":1}]} 2024-09-11T23:21:17.042 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 23 2024-09-11T23:21:17.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:17.043+0000 7fb50b3c2640 1 -- 172.21.15.43:0/1139256315 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb4f00a2c90 msgr2=0x7fb4f00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:17.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:17.043+0000 7fb50b3c2640 1 --2- 172.21.15.43:0/1139256315 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb4f00a2c90 0x7fb4f00a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40058c0 tx=0x7fb4f4005850 comp rx=0 tx=0).stop 2024-09-11T23:21:17.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:17.043+0000 7fb50b3c2640 1 -- 172.21.15.43:0/1139256315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb504100a80 msgr2=0x7fb5040fcde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:17.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:17.043+0000 7fb50b3c2640 1 --2- 172.21.15.43:0/1139256315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb504100a80 0x7fb5040fcde0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb50000d950 tx=0x7fb50000de10 comp rx=0 tx=0).stop 2024-09-11T23:21:17.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:17.043+0000 7fb50b3c2640 1 -- 172.21.15.43:0/1139256315 shutdown_connections 2024-09-11T23:21:17.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:17.043+0000 7fb50b3c2640 1 --2- 172.21.15.43:0/1139256315 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb4f00a2c90 0x7fb4f00a5150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:17.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:17.043+0000 7fb50b3c2640 1 --2- 172.21.15.43:0/1139256315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb504106a80 0x7fb5040fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:17.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:17.043+0000 7fb50b3c2640 1 --2- 172.21.15.43:0/1139256315 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb504101420 0x7fb5040fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:17.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:17.043+0000 7fb50b3c2640 1 --2- 172.21.15.43:0/1139256315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb504100a80 0x7fb5040fcde0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:17.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:17.043+0000 7fb50b3c2640 1 -- 172.21.15.43:0/1139256315 >> 172.21.15.43:0/1139256315 conn(0x7fb5040fc7a0 msgr2=0x7fb50406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:17.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:17.043+0000 7fb50b3c2640 1 -- 172.21.15.43:0/1139256315 shutdown_connections 2024-09-11T23:21:17.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:17.043+0000 7fb50b3c2640 1 -- 172.21.15.43:0/1139256315 wait complete. 2024-09-11T23:21:17.745 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T23:20:00.465830+0000', 'modified': '2024-09-11T23:20:12.440327+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24497, 'mds_1': 24470, 'mds_2': 24457}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24436': {'gid': 24436, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6835/1640511441', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 1640511441}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 1640511441}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6837/191447096', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 191447096}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 191447096}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.90:6835/4233140081', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6834', 'nonce': 4233140081}, {'type': 'v1', 'addr': '172.21.15.90:6835', 'nonce': 4233140081}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.97:6839/1424670024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 1424670024}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 1424670024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.90:6837/2946725567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6836', 'nonce': 2946725567}, {'type': 'v1', 'addr': '172.21.15.90:6837', 'nonce': 2946725567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.90:6839/3234895964', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6838', 'nonce': 3234895964}, {'type': 'v1', 'addr': '172.21.15.90:6839', 'nonce': 3234895964}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24497, 'qdb_cluster': [24457, 24470, 24497]} 2024-09-11T23:21:17.745 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-11T23:21:17.745 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T23:21:18.010 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:18.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:17 smithi043 ceph-mon[30950]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:21:18.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:17 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1139256315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:18.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:17 smithi090 ceph-mon[35130]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:21:18.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:17 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1139256315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:18.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:17 smithi097 ceph-mon[35562]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:21:18.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:17 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1139256315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:18.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.497+0000 7f306c391640 1 -- 172.21.15.43:0/2759517714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f306410c170 msgr2=0x7f306410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:18.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.497+0000 7f306c391640 1 --2- 172.21.15.43:0/2759517714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f306410c170 0x7f306410e610 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f305c009ca0 tx=0x7f305c02a950 comp rx=0 tx=0).stop 2024-09-11T23:21:18.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.497+0000 7f306c391640 1 -- 172.21.15.43:0/2759517714 shutdown_connections 2024-09-11T23:21:18.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.497+0000 7f306c391640 1 --2- 172.21.15.43:0/2759517714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f306410c170 0x7f306410e610 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.497+0000 7f306c391640 1 --2- 172.21.15.43:0/2759517714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3064103660 0x7f306410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.497+0000 7f306c391640 1 --2- 172.21.15.43:0/2759517714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3064102d40 0x7f3064103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.497+0000 7f306c391640 1 -- 172.21.15.43:0/2759517714 >> 172.21.15.43:0/2759517714 conn(0x7f30640fc7a0 msgr2=0x7f30640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:18.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.497+0000 7f306c391640 1 -- 172.21.15.43:0/2759517714 shutdown_connections 2024-09-11T23:21:18.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.497+0000 7f306c391640 1 -- 172.21.15.43:0/2759517714 wait complete. 2024-09-11T23:21:18.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.498+0000 7f306c391640 1 Processor -- start 2024-09-11T23:21:18.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.498+0000 7f306c391640 1 -- start start 2024-09-11T23:21:18.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.498+0000 7f306c391640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3064102d40 0x7f30641009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:18.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.498+0000 7f306c391640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3064103660 0x7f3064100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:18.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.498+0000 7f306c391640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f306410c170 0x7f3064101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:18.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.498+0000 7f306c391640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3064111f30 con 0x7f3064102d40 2024-09-11T23:21:18.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.498+0000 7f306c391640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3064111db0 con 0x7f3064103660 2024-09-11T23:21:18.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.498+0000 7f306c391640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f30641120b0 con 0x7f306410c170 2024-09-11T23:21:18.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.498+0000 7f306a12d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3064102d40 0x7f30641009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:18.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.498+0000 7f306a12d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3064102d40 0x7f30641009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38592/0 (socket says 172.21.15.43:38592) 2024-09-11T23:21:18.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.498+0000 7f306a12d640 1 -- 172.21.15.43:0/167961168 learned_addr learned my addr 172.21.15.43:0/167961168 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:18.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.498+0000 7f306a12d640 1 -- 172.21.15.43:0/167961168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f306410c170 msgr2=0x7f3064101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:18.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.498+0000 7f306a12d640 1 --2- 172.21.15.43:0/167961168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f306410c170 0x7f3064101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.498+0000 7f306a12d640 1 -- 172.21.15.43:0/167961168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3064103660 msgr2=0x7f3064100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:18.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.498+0000 7f306992c640 1 --2- 172.21.15.43:0/167961168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3064103660 0x7f3064100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:18.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.499+0000 7f306a12d640 1 --2- 172.21.15.43:0/167961168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3064103660 0x7f3064100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.499+0000 7f306a12d640 1 -- 172.21.15.43:0/167961168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30641a6820 con 0x7f3064102d40 2024-09-11T23:21:18.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.499+0000 7f306a12d640 1 --2- 172.21.15.43:0/167961168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3064102d40 0x7f30641009a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f306000c920 tx=0x7f306000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:18.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.499+0000 7f305b7fe640 1 -- 172.21.15.43:0/167961168 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060007c70 con 0x7f3064102d40 2024-09-11T23:21:18.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.499+0000 7f306c391640 1 -- 172.21.15.43:0/167961168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30641a6b10 con 0x7f3064102d40 2024-09-11T23:21:18.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.499+0000 7f305b7fe640 1 -- 172.21.15.43:0/167961168 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3060007e10 con 0x7f3064102d40 2024-09-11T23:21:18.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.499+0000 7f305b7fe640 1 -- 172.21.15.43:0/167961168 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060018890 con 0x7f3064102d40 2024-09-11T23:21:18.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.499+0000 7f306c391640 1 -- 172.21.15.43:0/167961168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30641a70a0 con 0x7f3064102d40 2024-09-11T23:21:18.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.501+0000 7f305b7fe640 1 -- 172.21.15.43:0/167961168 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3060018a70 con 0x7f3064102d40 2024-09-11T23:21:18.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.501+0000 7f306e76a740 1 Processor -- start 2024-09-11T23:21:18.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.502+0000 7f306e76a740 1 -- start start 2024-09-11T23:21:18.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.502+0000 7f306e76a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565e92ba9d0 0x5565e92badb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:18.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.502+0000 7f306e76a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565e92bb2f0 0x5565e92c5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:18.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.502+0000 7f306e76a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92c5bb0 0x5565e92c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:18.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.502+0000 7f306e76a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5565e9221090 con 0x5565e92c5bb0 2024-09-11T23:21:18.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.502+0000 7f306e76a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5565e91c8a70 con 0x5565e92bb2f0 2024-09-11T23:21:18.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.502+0000 7f306e76a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5565e921d050 con 0x5565e92ba9d0 2024-09-11T23:21:18.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.502+0000 7f306992c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92c5bb0 0x5565e92c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:18.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.502+0000 7f305b7fe640 1 --2- 172.21.15.43:0/167961168 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f30500a2c90 0x7f30500a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:18.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.502+0000 7f306992c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92c5bb0 0x5565e92c7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38608/0 (socket says 172.21.15.43:38608) 2024-09-11T23:21:18.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.502+0000 7f306992c640 1 -- 172.21.15.43:0/831104597 learned_addr learned my addr 172.21.15.43:0/831104597 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:18.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.502+0000 7f306a92e640 1 --2- 172.21.15.43:0/831104597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565e92ba9d0 0x5565e92badb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:18.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.502+0000 7f306a92e640 1 --2- 172.21.15.43:0/167961168 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f30500a2c90 0x7f30500a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:18.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.502+0000 7f305b7fe640 1 -- 172.21.15.43:0/167961168 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f30600cf350 con 0x7f3064102d40 2024-09-11T23:21:18.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.502+0000 7f306a12d640 1 --2- 172.21.15.43:0/831104597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565e92bb2f0 0x5565e92c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:18.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.502+0000 7f306992c640 1 -- 172.21.15.43:0/831104597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565e92ba9d0 msgr2=0x5565e92badb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:18.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.502+0000 7f306992c640 1 --2- 172.21.15.43:0/831104597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565e92ba9d0 0x5565e92badb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.502+0000 7f306992c640 1 -- 172.21.15.43:0/831104597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565e92bb2f0 msgr2=0x5565e92c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:18.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.502+0000 7f306992c640 1 --2- 172.21.15.43:0/831104597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565e92bb2f0 0x5565e92c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.502+0000 7f306992c640 1 -- 172.21.15.43:0/831104597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565e91d5130 con 0x5565e92c5bb0 2024-09-11T23:21:18.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.503+0000 7f306992c640 1 --2- 172.21.15.43:0/831104597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92c5bb0 0x5565e92c7fa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3054009b50 tx=0x7f305400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d63e4659cfe649 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:18.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.503+0000 7f306a92e640 1 --2- 172.21.15.43:0/167961168 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f30500a2c90 0x7f30500a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f305c004af0 tx=0x7f305c0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:18.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.503+0000 7f3058dea640 1 -- 172.21.15.43:0/831104597 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054016070 con 0x5565e92c5bb0 2024-09-11T23:21:18.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.503+0000 7f3058dea640 1 -- 172.21.15.43:0/831104597 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3054009e40 con 0x5565e92c5bb0 2024-09-11T23:21:18.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.503+0000 7f3058dea640 1 -- 172.21.15.43:0/831104597 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054011470 con 0x5565e92c5bb0 2024-09-11T23:21:18.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.503+0000 7f306e76a740 1 -- 172.21.15.43:0/831104597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92c5bb0 msgr2=0x5565e92c7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:18.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.503+0000 7f306e76a740 1 --2- 172.21.15.43:0/831104597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92c5bb0 0x5565e92c7fa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3054009b50 tx=0x7f305400e6c0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.504+0000 7f306e76a740 1 -- 172.21.15.43:0/831104597 shutdown_connections 2024-09-11T23:21:18.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.504+0000 7f306e76a740 1 --2- 172.21.15.43:0/831104597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92c5bb0 0x5565e92c7fa0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.504+0000 7f306e76a740 1 --2- 172.21.15.43:0/831104597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565e92bb2f0 0x5565e92c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.504+0000 7f306e76a740 1 --2- 172.21.15.43:0/831104597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565e92ba9d0 0x5565e92badb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.504+0000 7f306e76a740 1 -- 172.21.15.43:0/831104597 >> 172.21.15.43:0/831104597 conn(0x5565e92b6f60 msgr2=0x5565e92b7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:18.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.504+0000 7f306e76a740 1 -- 172.21.15.43:0/831104597 shutdown_connections 2024-09-11T23:21:18.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.504+0000 7f306e76a740 1 -- 172.21.15.43:0/831104597 wait complete. 2024-09-11T23:21:18.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.504+0000 7f306e76a740 1 -- start start 2024-09-11T23:21:18.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.504+0000 7f306e76a740 1 Processor -- start 2024-09-11T23:21:18.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.504+0000 7f306e76a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565e92ba9d0 0x5565e92c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:18.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.504+0000 7f306e76a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565e92bb2f0 0x5565e92c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:18.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.504+0000 7f306e76a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92c5bb0 0x5565e92c2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:18.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.504+0000 7f306e76a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5565e92c8780 con 0x5565e92c5bb0 2024-09-11T23:21:18.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.504+0000 7f306e76a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5565e91dbf20 con 0x5565e92bb2f0 2024-09-11T23:21:18.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.504+0000 7f306e76a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5565e92c8900 con 0x5565e92ba9d0 2024-09-11T23:21:18.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.504+0000 7f306e76a740 10 client.0 _ll_register_callbacks cb 0x5565e91a5030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:18.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.504+0000 7f306a92e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92c5bb0 0x5565e92c2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:18.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.504+0000 7f306a92e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92c5bb0 0x5565e92c2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38616/0 (socket says 172.21.15.43:38616) 2024-09-11T23:21:18.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.504+0000 7f306a92e640 1 -- 172.21.15.43:0/3183214376 learned_addr learned my addr 172.21.15.43:0/3183214376 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:18.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.505+0000 7f306992c640 1 --2- 172.21.15.43:0/3183214376 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565e92bb2f0 0x5565e92c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:18.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.505+0000 7f306a12d640 1 --2- 172.21.15.43:0/3183214376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565e92ba9d0 0x5565e92c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:18.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.505+0000 7f306a92e640 1 -- 172.21.15.43:0/3183214376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565e92ba9d0 msgr2=0x5565e92c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:18.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.505+0000 7f306a92e640 1 --2- 172.21.15.43:0/3183214376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565e92ba9d0 0x5565e92c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.505+0000 7f306a92e640 1 -- 172.21.15.43:0/3183214376 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565e92bb2f0 msgr2=0x5565e92c26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:18.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.505+0000 7f306a92e640 1 --2- 172.21.15.43:0/3183214376 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565e92bb2f0 0x5565e92c26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.505+0000 7f306a92e640 1 -- 172.21.15.43:0/3183214376 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565e8f922b0 con 0x5565e92c5bb0 2024-09-11T23:21:18.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.505+0000 7f306a92e640 1 --2- 172.21.15.43:0/3183214376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92c5bb0 0x5565e92c2c20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f305c035000 tx=0x7f305c0093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c12e63dc96a3e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:18.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.505+0000 7f304dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:21:18.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.505+0000 7f304dffb640 1 -- 172.21.15.43:0/3183214376 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c041070 con 0x5565e92c5bb0 2024-09-11T23:21:18.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.505+0000 7f304dffb640 1 -- 172.21.15.43:0/3183214376 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f305c004230 con 0x5565e92c5bb0 2024-09-11T23:21:18.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.505+0000 7f306e76a740 1 -- 172.21.15.43:0/3183214376 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5565e922e600 con 0x5565e92c5bb0 2024-09-11T23:21:18.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.505+0000 7f304dffb640 1 -- 172.21.15.43:0/3183214376 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f305c057070 con 0x5565e92c5bb0 2024-09-11T23:21:18.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.505+0000 7f306e76a740 10 client.14946 fetch_fsmap learned FSMap version 23 2024-09-11T23:21:18.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.506+0000 7f306e76a740 1 -- 172.21.15.43:0/3183214376 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5565e92d4000 con 0x5565e92c5bb0 2024-09-11T23:21:18.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.506+0000 7f304dffb640 1 -- 172.21.15.43:0/3183214376 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5565e922e600 con 0x5565e92c5bb0 2024-09-11T23:21:18.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.506+0000 7f304dffb640 1 -- 172.21.15.43:0/3183214376 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f305c030c60 con 0x5565e92c5bb0 2024-09-11T23:21:18.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.506+0000 7f306e76a740 10 client.14946 fetch_fsmap finished waiting for FSMap version 23 2024-09-11T23:21:18.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.506+0000 7f306e76a740 10 client.14946 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:21:18.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.506+0000 7f306e76a740 20 client.14946 populate_metadata read hostname 'smithi043' 2024-09-11T23:21:18.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.506+0000 7f306e76a740 1 --2- 172.21.15.43:0/3183214376 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5565e92d66a0 0x5565e92d7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:18.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.506+0000 7f306e76a740 4 client.14946 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:21:18.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.506+0000 7f306e76a740 1 -- 172.21.15.43:0/3183214376 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565e9228b10 con 0x5565e92d66a0 2024-09-11T23:21:18.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.506+0000 7f306a12d640 1 --2- 172.21.15.43:0/3183214376 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5565e92d66a0 0x5565e92d7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:18.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.507+0000 7f304dffb640 10 client.14946 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:21:18.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.507+0000 7f306a12d640 1 --2- 172.21.15.43:0/3183214376 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5565e92d66a0 0x5565e92d7b10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52fc5f2d21a8e5ba server_cookie=27f9b0a25390a121 in_seq=0 out_seq=0 2024-09-11T23:21:18.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.507+0000 7f304dffb640 1 -- 172.21.15.43:0/3183214376 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565e9228b10 con 0x5565e92d66a0 2024-09-11T23:21:18.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.507+0000 7f304dffb640 10 client.14946 handle_command_reply: tid=0 2024-09-11T23:21:18.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.508+0000 7f306e76a740 1 client.14946 shutdown 2024-09-11T23:21:18.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.508+0000 7f306e76a740 1 -- 172.21.15.43:0/3183214376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92c5bb0 msgr2=0x5565e92c2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.508+0000 7f306e76a740 1 --2- 172.21.15.43:0/3183214376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92c5bb0 0x5565e92c2c20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f305c035000 tx=0x7f305c0093c0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.508+0000 7f306e76a740 1 -- 172.21.15.43:0/3183214376 shutdown_connections 2024-09-11T23:21:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.508+0000 7f306e76a740 1 --2- 172.21.15.43:0/3183214376 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5565e92d66a0 0x5565e92d7b10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.508+0000 7f306e76a740 1 --2- 172.21.15.43:0/3183214376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92c5bb0 0x5565e92c2c20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.508+0000 7f306e76a740 1 --2- 172.21.15.43:0/3183214376 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565e92bb2f0 0x5565e92c26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.508+0000 7f306e76a740 1 --2- 172.21.15.43:0/3183214376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565e92ba9d0 0x5565e92c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.508+0000 7f304dffb640 0 client.14946 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:21:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.508+0000 7f306e76a740 1 -- 172.21.15.43:0/3183214376 >> 172.21.15.43:0/3183214376 conn(0x5565e92b6f60 msgr2=0x5565e92bcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.508+0000 7f306e76a740 1 -- 172.21.15.43:0/3183214376 shutdown_connections 2024-09-11T23:21:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.508+0000 7f306e76a740 1 -- 172.21.15.43:0/3183214376 wait complete. 2024-09-11T23:21:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.508+0000 7f306e76a740 20 client.14946 trim_cache size 0 max 16384 2024-09-11T23:21:18.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.526+0000 7f306e76a740 1 Processor -- start 2024-09-11T23:21:18.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.526+0000 7f306e76a740 1 -- start start 2024-09-11T23:21:18.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.526+0000 7f306e76a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565e92ba9d0 0x5565e92c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:18.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.526+0000 7f306e76a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92bb2f0 0x5565e92c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:18.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.526+0000 7f306e76a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565e92c5bb0 0x5565e92dfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:18.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.526+0000 7f306e76a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5565e92d5e50 con 0x5565e92bb2f0 2024-09-11T23:21:18.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.526+0000 7f306e76a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5565e92d6520 con 0x5565e92ba9d0 2024-09-11T23:21:18.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.526+0000 7f306e76a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5565e92d6bf0 con 0x5565e92c5bb0 2024-09-11T23:21:18.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.526+0000 7f306992c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92bb2f0 0x5565e92c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.526+0000 7f306992c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92bb2f0 0x5565e92c2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38632/0 (socket says 172.21.15.43:38632) 2024-09-11T23:21:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.526+0000 7f306992c640 1 -- 172.21.15.43:0/993965904 learned_addr learned my addr 172.21.15.43:0/993965904 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.526+0000 7f306a12d640 1 --2- 172.21.15.43:0/993965904 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565e92ba9d0 0x5565e92c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.526+0000 7f306a92e640 1 --2- 172.21.15.43:0/993965904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565e92c5bb0 0x5565e92dfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.526+0000 7f306992c640 1 -- 172.21.15.43:0/993965904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565e92c5bb0 msgr2=0x5565e92dfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.526+0000 7f306992c640 1 --2- 172.21.15.43:0/993965904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565e92c5bb0 0x5565e92dfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.526+0000 7f306992c640 1 -- 172.21.15.43:0/993965904 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565e92ba9d0 msgr2=0x5565e92c26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.526+0000 7f306992c640 1 --2- 172.21.15.43:0/993965904 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565e92ba9d0 0x5565e92c26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.526+0000 7f306992c640 1 -- 172.21.15.43:0/993965904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565e92b6d90 con 0x5565e92bb2f0 2024-09-11T23:21:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.527+0000 7f306992c640 1 --2- 172.21.15.43:0/993965904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92bb2f0 0x5565e92c2be0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3054009b50 tx=0x7f30540092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5cdcb797cb0878 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.527+0000 7f304ffff640 1 -- 172.21.15.43:0/993965904 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054016040 con 0x5565e92bb2f0 2024-09-11T23:21:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.527+0000 7f304ffff640 1 -- 172.21.15.43:0/993965904 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3054029070 con 0x5565e92bb2f0 2024-09-11T23:21:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.527+0000 7f304ffff640 1 -- 172.21.15.43:0/993965904 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054024660 con 0x5565e92bb2f0 2024-09-11T23:21:18.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.527+0000 7f306e76a740 1 -- 172.21.15.43:0/993965904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92bb2f0 msgr2=0x5565e92c2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:18.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.527+0000 7f306e76a740 1 --2- 172.21.15.43:0/993965904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92bb2f0 0x5565e92c2be0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3054009b50 tx=0x7f30540092a0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.527+0000 7f306e76a740 1 -- 172.21.15.43:0/993965904 shutdown_connections 2024-09-11T23:21:18.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.527+0000 7f306e76a740 1 --2- 172.21.15.43:0/993965904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565e92c5bb0 0x5565e92dfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.527+0000 7f306e76a740 1 --2- 172.21.15.43:0/993965904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92bb2f0 0x5565e92c2be0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.527+0000 7f306e76a740 1 --2- 172.21.15.43:0/993965904 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565e92ba9d0 0x5565e92c26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.527+0000 7f306e76a740 1 -- 172.21.15.43:0/993965904 >> 172.21.15.43:0/993965904 conn(0x5565e92b6f60 msgr2=0x5565e92b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:18.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.528+0000 7f306e76a740 1 -- 172.21.15.43:0/993965904 shutdown_connections 2024-09-11T23:21:18.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.528+0000 7f306e76a740 1 -- 172.21.15.43:0/993965904 wait complete. 2024-09-11T23:21:18.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.528+0000 7f306e76a740 1 -- start start 2024-09-11T23:21:18.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.528+0000 7f306e76a740 1 Processor -- start 2024-09-11T23:21:18.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.528+0000 7f306e76a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565e92ba9d0 0x5565e92beda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:18.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.528+0000 7f306e76a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92bb2f0 0x5565e92bf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:18.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.528+0000 7f306e76a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565e92c5bb0 0x5565e92ded60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:18.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.528+0000 7f306e76a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5565e92d7990 con 0x5565e92bb2f0 2024-09-11T23:21:18.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.528+0000 7f306e76a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5565e92d72c0 con 0x5565e92c5bb0 2024-09-11T23:21:18.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.528+0000 7f306e76a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5565e92d8060 con 0x5565e92ba9d0 2024-09-11T23:21:18.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.528+0000 7f306e76a740 10 client.0 _ll_register_callbacks cb 0x7f305c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:18.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.528+0000 7f306992c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92bb2f0 0x5565e92bf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:18.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.528+0000 7f306992c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92bb2f0 0x5565e92bf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38636/0 (socket says 172.21.15.43:38636) 2024-09-11T23:21:18.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.528+0000 7f306992c640 1 -- 172.21.15.43:0/2882673757 learned_addr learned my addr 172.21.15.43:0/2882673757 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:18.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.528+0000 7f306a12d640 1 --2- 172.21.15.43:0/2882673757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565e92ba9d0 0x5565e92beda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:18.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.528+0000 7f306a92e640 1 --2- 172.21.15.43:0/2882673757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565e92c5bb0 0x5565e92ded60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:18.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.529+0000 7f306992c640 1 -- 172.21.15.43:0/2882673757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565e92ba9d0 msgr2=0x5565e92beda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:18.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.529+0000 7f306992c640 1 --2- 172.21.15.43:0/2882673757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565e92ba9d0 0x5565e92beda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.529+0000 7f306992c640 1 -- 172.21.15.43:0/2882673757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565e92c5bb0 msgr2=0x5565e92ded60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:18.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.529+0000 7f306992c640 1 --2- 172.21.15.43:0/2882673757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565e92c5bb0 0x5565e92ded60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.529+0000 7f306992c640 1 -- 172.21.15.43:0/2882673757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565e92df2a0 con 0x5565e92bb2f0 2024-09-11T23:21:18.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.529+0000 7f306992c640 1 --2- 172.21.15.43:0/2882673757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92bb2f0 0x5565e92bf2e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3054004b30 tx=0x7f30540116a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4393cf122bb15d1e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:18.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.529+0000 7f304dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:21:18.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.529+0000 7f304dffb640 1 -- 172.21.15.43:0/2882673757 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054016070 con 0x5565e92bb2f0 2024-09-11T23:21:18.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.529+0000 7f304dffb640 1 -- 172.21.15.43:0/2882673757 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f3054024400 con 0x5565e92bb2f0 2024-09-11T23:21:18.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.529+0000 7f306e76a740 1 -- 172.21.15.43:0/2882673757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3054029070 con 0x5565e92bb2f0 2024-09-11T23:21:18.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.529+0000 7f304dffb640 1 -- 172.21.15.43:0/2882673757 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3054009750 con 0x5565e92bb2f0 2024-09-11T23:21:18.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.529+0000 7f306e76a740 10 client.14952 fetch_fsmap learned FSMap version 23 2024-09-11T23:21:18.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.529+0000 7f306e76a740 1 -- 172.21.15.43:0/2882673757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5565e92e05a0 con 0x5565e92bb2f0 2024-09-11T23:21:18.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.529+0000 7f304dffb640 1 -- 172.21.15.43:0/2882673757 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3054029070 con 0x5565e92bb2f0 2024-09-11T23:21:18.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.530+0000 7f304dffb640 1 -- 172.21.15.43:0/2882673757 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f305400f040 con 0x5565e92bb2f0 2024-09-11T23:21:18.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.530+0000 7f306e76a740 10 client.14952 fetch_fsmap finished waiting for FSMap version 23 2024-09-11T23:21:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.530+0000 7f306e76a740 10 client.14952 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:21:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.530+0000 7f306e76a740 20 client.14952 populate_metadata read hostname 'smithi043' 2024-09-11T23:21:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.530+0000 7f306e76a740 1 --2- 172.21.15.43:0/2882673757 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5565e9344990 0x5565e93436a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.530+0000 7f306e76a740 4 client.14952 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:21:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.530+0000 7f306e76a740 1 -- 172.21.15.43:0/2882673757 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "status"}) -- 0x5565e91a50e0 con 0x5565e9344990 2024-09-11T23:21:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.530+0000 7f306a12d640 1 --2- 172.21.15.43:0/2882673757 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5565e9344990 0x5565e93436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.530+0000 7f304dffb640 10 client.14952 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:21:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.530+0000 7f306a12d640 1 --2- 172.21.15.43:0/2882673757 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5565e9344990 0x5565e93436a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a982c3fc6a19978e server_cookie=680ddf982d6f1cea in_seq=0 out_seq=0 2024-09-11T23:21:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.531+0000 7f304dffb640 1 -- 172.21.15.43:0/2882673757 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5565e91a50e0 con 0x5565e9344990 2024-09-11T23:21:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.531+0000 7f304dffb640 10 client.14952 handle_command_reply: tid=0 2024-09-11T23:21:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.531+0000 7f306e76a740 1 client.14952 shutdown 2024-09-11T23:21:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.531+0000 7f306e76a740 1 -- 172.21.15.43:0/2882673757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92bb2f0 msgr2=0x5565e92bf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.531+0000 7f306e76a740 1 --2- 172.21.15.43:0/2882673757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92bb2f0 0x5565e92bf2e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3054004b30 tx=0x7f30540116a0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.531+0000 7f306e76a740 1 -- 172.21.15.43:0/2882673757 shutdown_connections 2024-09-11T23:21:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.531+0000 7f306e76a740 1 --2- 172.21.15.43:0/2882673757 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5565e9344990 0x5565e93436a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.531+0000 7f306e76a740 1 --2- 172.21.15.43:0/2882673757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565e92c5bb0 0x5565e92ded60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.531+0000 7f306e76a740 1 --2- 172.21.15.43:0/2882673757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565e92bb2f0 0x5565e92bf2e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.531+0000 7f304dffb640 0 client.14952 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:21:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.531+0000 7f306e76a740 1 --2- 172.21.15.43:0/2882673757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565e92ba9d0 0x5565e92beda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.531+0000 7f306e76a740 1 -- 172.21.15.43:0/2882673757 >> 172.21.15.43:0/2882673757 conn(0x5565e92b6f60 msgr2=0x5565e92b7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.531+0000 7f306e76a740 1 -- 172.21.15.43:0/2882673757 shutdown_connections 2024-09-11T23:21:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.531+0000 7f306e76a740 1 -- 172.21.15.43:0/2882673757 wait complete. 2024-09-11T23:21:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.531+0000 7f306e76a740 20 client.14952 trim_cache size 0 max 16384 2024-09-11T23:21:18.544 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:21:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "25c0cd84-7093-11ef-bcea-c7b262605968", 2024-09-11T23:21:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 2, 2024-09-11T23:21:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24457, 2024-09-11T23:21:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-11T23:21:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-11T23:21:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-11T23:21:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 68.811954213000007, 2024-09-11T23:21:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 23, 2024-09-11T23:21:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 84, 2024-09-11T23:21:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 82, 2024-09-11T23:21:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 98.056442594000004 2024-09-11T23:21:18.545 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:21:18.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.531+0000 7f306c391640 1 -- 172.21.15.43:0/167961168 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f30500a2c90 msgr2=0x7f30500a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:18.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.531+0000 7f306c391640 1 --2- 172.21.15.43:0/167961168 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f30500a2c90 0x7f30500a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f305c004af0 tx=0x7f305c0095c0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.531+0000 7f306c391640 1 -- 172.21.15.43:0/167961168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3064102d40 msgr2=0x7f30641009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:18.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.531+0000 7f306c391640 1 --2- 172.21.15.43:0/167961168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3064102d40 0x7f30641009a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f306000c920 tx=0x7f306000cde0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.532+0000 7f306c391640 1 -- 172.21.15.43:0/167961168 shutdown_connections 2024-09-11T23:21:18.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.532+0000 7f306c391640 1 --2- 172.21.15.43:0/167961168 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f30500a2c90 0x7f30500a5150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.532+0000 7f306c391640 1 --2- 172.21.15.43:0/167961168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f306410c170 0x7f3064101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.532+0000 7f306c391640 1 --2- 172.21.15.43:0/167961168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3064103660 0x7f3064100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.532+0000 7f306c391640 1 --2- 172.21.15.43:0/167961168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3064102d40 0x7f30641009a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:18.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.532+0000 7f306c391640 1 -- 172.21.15.43:0/167961168 >> 172.21.15.43:0/167961168 conn(0x7f30640fc7a0 msgr2=0x7f30640fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:18.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.532+0000 7f306c391640 1 -- 172.21.15.43:0/167961168 shutdown_connections 2024-09-11T23:21:18.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:18.532+0000 7f306c391640 1 -- 172.21.15.43:0/167961168 wait complete. 2024-09-11T23:21:19.275 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T23:21:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:20.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.001+0000 7f095f6bb640 1 -- 172.21.15.43:0/1456298720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0958100480 msgr2=0x7f0958100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:20.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.001+0000 7f095f6bb640 1 --2- 172.21.15.43:0/1456298720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0958100480 0x7f0958100860 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0954009900 tx=0x7f095402a410 comp rx=0 tx=0).stop 2024-09-11T23:21:20.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.001+0000 7f095f6bb640 1 -- 172.21.15.43:0/1456298720 shutdown_connections 2024-09-11T23:21:20.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.001+0000 7f095f6bb640 1 --2- 172.21.15.43:0/1456298720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09581055b0 0x7f095810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.001+0000 7f095f6bb640 1 --2- 172.21.15.43:0/1456298720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0958100e30 0x7f0958104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.001+0000 7f095f6bb640 1 --2- 172.21.15.43:0/1456298720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0958100480 0x7f0958100860 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.001+0000 7f095f6bb640 1 -- 172.21.15.43:0/1456298720 >> 172.21.15.43:0/1456298720 conn(0x7f0958077210 msgr2=0x7f09580fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:20.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.001+0000 7f095f6bb640 1 -- 172.21.15.43:0/1456298720 shutdown_connections 2024-09-11T23:21:20.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.001+0000 7f095f6bb640 1 -- 172.21.15.43:0/1456298720 wait complete. 2024-09-11T23:21:20.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.002+0000 7f095f6bb640 1 Processor -- start 2024-09-11T23:21:20.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.002+0000 7f095f6bb640 1 -- start start 2024-09-11T23:21:20.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.002+0000 7f095f6bb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0958100480 0x7f0958199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.002+0000 7f095f6bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0958100e30 0x7f0958199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.002+0000 7f095f6bb640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09581055b0 0x7f09581a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.002+0000 7f095f6bb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0958110ed0 con 0x7f0958100e30 2024-09-11T23:21:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.002+0000 7f095f6bb640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0958110d50 con 0x7f09581055b0 2024-09-11T23:21:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.002+0000 7f095f6bb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0958111050 con 0x7f0958100480 2024-09-11T23:21:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.002+0000 7f095cc56640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0958100e30 0x7f0958199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.002+0000 7f095cc56640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0958100e30 0x7f0958199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38646/0 (socket says 172.21.15.43:38646) 2024-09-11T23:21:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.002+0000 7f095cc56640 1 -- 172.21.15.43:0/57131302 learned_addr learned my addr 172.21.15.43:0/57131302 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.002+0000 7f095d457640 1 --2- 172.21.15.43:0/57131302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0958100480 0x7f0958199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.002+0000 7f095dc58640 1 --2- 172.21.15.43:0/57131302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09581055b0 0x7f09581a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.002+0000 7f095cc56640 1 -- 172.21.15.43:0/57131302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0958100480 msgr2=0x7f0958199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:20.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.002+0000 7f095cc56640 1 --2- 172.21.15.43:0/57131302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0958100480 0x7f0958199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.002+0000 7f095cc56640 1 -- 172.21.15.43:0/57131302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09581055b0 msgr2=0x7f09581a43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:20.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.002+0000 7f095cc56640 1 --2- 172.21.15.43:0/57131302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09581055b0 0x7f09581a43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.002+0000 7f095cc56640 1 -- 172.21.15.43:0/57131302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09581a4900 con 0x7f0958100e30 2024-09-11T23:21:20.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.002+0000 7f095d457640 1 --2- 172.21.15.43:0/57131302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0958100480 0x7f0958199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:20.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.003+0000 7f095cc56640 1 --2- 172.21.15.43:0/57131302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0958100e30 0x7f0958199880 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f094800c970 tx=0x7f094800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:20.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.003+0000 7f094e7fc640 1 -- 172.21.15.43:0/57131302 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948007bb0 con 0x7f0958100e30 2024-09-11T23:21:20.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.003+0000 7f094e7fc640 1 -- 172.21.15.43:0/57131302 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0948007d50 con 0x7f0958100e30 2024-09-11T23:21:20.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.003+0000 7f095f6bb640 1 -- 172.21.15.43:0/57131302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09581a4ad0 con 0x7f0958100e30 2024-09-11T23:21:20.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.003+0000 7f094e7fc640 1 -- 172.21.15.43:0/57131302 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094800f450 con 0x7f0958100e30 2024-09-11T23:21:20.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.003+0000 7f095f6bb640 1 -- 172.21.15.43:0/57131302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09581a4fe0 con 0x7f0958100e30 2024-09-11T23:21:20.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.005+0000 7f094e7fc640 1 -- 172.21.15.43:0/57131302 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0948005020 con 0x7f0958100e30 2024-09-11T23:21:20.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.006+0000 7f0961a94740 1 Processor -- start 2024-09-11T23:21:20.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.006+0000 7f0961a94740 1 -- start start 2024-09-11T23:21:20.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.006+0000 7f0961a94740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661ea09d0 0x55e661ea0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:20.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.006+0000 7f0961a94740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661ea12f0 0x55e661eab670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:20.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.006+0000 7f0961a94740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e661eabbb0 0x55e661eadfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:20.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.006+0000 7f0961a94740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e661e07090 con 0x55e661ea09d0 2024-09-11T23:21:20.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.006+0000 7f0961a94740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e661daea70 con 0x55e661ea12f0 2024-09-11T23:21:20.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.006+0000 7f0961a94740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e661e03050 con 0x55e661eabbb0 2024-09-11T23:21:20.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.006+0000 7f095dc58640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661ea09d0 0x55e661ea0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:20.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.006+0000 7f094e7fc640 1 --2- 172.21.15.43:0/57131302 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f09440a2c90 0x7f09440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:20.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.006+0000 7f095dc58640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661ea09d0 0x55e661ea0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38662/0 (socket says 172.21.15.43:38662) 2024-09-11T23:21:20.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.006+0000 7f095dc58640 1 -- 172.21.15.43:0/3181296962 learned_addr learned my addr 172.21.15.43:0/3181296962 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:20.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.007+0000 7f095dc58640 1 --2- 172.21.15.43:0/57131302 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f09440a2c90 0x7f09440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:20.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.007+0000 7f095d457640 1 --2- 172.21.15.43:0/3181296962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661ea12f0 0x55e661eab670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:20.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.007+0000 7f094e7fc640 1 -- 172.21.15.43:0/57131302 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f09480cf060 con 0x7f0958100e30 2024-09-11T23:21:20.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.007+0000 7f095dc58640 1 -- 172.21.15.43:0/3181296962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e661eabbb0 msgr2=0x55e661eadfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:20.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.007+0000 7f095dc58640 1 --2- 172.21.15.43:0/3181296962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e661eabbb0 0x55e661eadfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.007+0000 7f095dc58640 1 -- 172.21.15.43:0/3181296962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661ea12f0 msgr2=0x55e661eab670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:20.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.007+0000 7f095dc58640 1 --2- 172.21.15.43:0/3181296962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661ea12f0 0x55e661eab670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.007+0000 7f095dc58640 1 -- 172.21.15.43:0/3181296962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e661dbb130 con 0x55e661ea09d0 2024-09-11T23:21:20.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.007+0000 7f095dc58640 1 --2- 172.21.15.43:0/3181296962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661ea09d0 0x55e661ea0db0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f095000cb20 tx=0x7f09500072b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a58da2d95873b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:20.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.007+0000 7f09437fe640 1 -- 172.21.15.43:0/3181296962 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950015070 con 0x55e661ea09d0 2024-09-11T23:21:20.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.007+0000 7f095dc58640 1 --2- 172.21.15.43:0/57131302 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f09440a2c90 0x7f09440a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f095001f000 tx=0x7f095001b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:20.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.007+0000 7f09437fe640 1 -- 172.21.15.43:0/3181296962 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0950016040 con 0x55e661ea09d0 2024-09-11T23:21:20.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.007+0000 7f09437fe640 1 -- 172.21.15.43:0/3181296962 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095001e020 con 0x55e661ea09d0 2024-09-11T23:21:20.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.008+0000 7f0961a94740 1 -- 172.21.15.43:0/3181296962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661ea09d0 msgr2=0x55e661ea0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:20.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.008+0000 7f0961a94740 1 --2- 172.21.15.43:0/3181296962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661ea09d0 0x55e661ea0db0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f095000cb20 tx=0x7f09500072b0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.008+0000 7f0961a94740 1 -- 172.21.15.43:0/3181296962 shutdown_connections 2024-09-11T23:21:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.008+0000 7f0961a94740 1 --2- 172.21.15.43:0/3181296962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e661eabbb0 0x55e661eadfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.008+0000 7f0961a94740 1 --2- 172.21.15.43:0/3181296962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661ea12f0 0x55e661eab670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.008+0000 7f0961a94740 1 --2- 172.21.15.43:0/3181296962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661ea09d0 0x55e661ea0db0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.008+0000 7f0961a94740 1 -- 172.21.15.43:0/3181296962 >> 172.21.15.43:0/3181296962 conn(0x55e661e9cf60 msgr2=0x55e661e9d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.008+0000 7f0961a94740 1 -- 172.21.15.43:0/3181296962 shutdown_connections 2024-09-11T23:21:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.008+0000 7f0961a94740 1 -- 172.21.15.43:0/3181296962 wait complete. 2024-09-11T23:21:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.008+0000 7f0961a94740 1 -- start start 2024-09-11T23:21:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.008+0000 7f0961a94740 1 Processor -- start 2024-09-11T23:21:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.009+0000 7f0961a94740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661ea09d0 0x55e661eab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.009+0000 7f0961a94740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e661ea12f0 0x55e661ea8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.009+0000 7f0961a94740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661eabbb0 0x55e661ea8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.009+0000 7f0961a94740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e661eae780 con 0x55e661eabbb0 2024-09-11T23:21:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.009+0000 7f0961a94740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e661dc1f20 con 0x55e661ea09d0 2024-09-11T23:21:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.009+0000 7f0961a94740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e661eae900 con 0x55e661ea12f0 2024-09-11T23:21:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.009+0000 7f0961a94740 10 client.0 _ll_register_callbacks cb 0x55e661d8b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.009+0000 7f095dc58640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661eabbb0 0x55e661ea8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.009+0000 7f095dc58640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661eabbb0 0x55e661ea8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38676/0 (socket says 172.21.15.43:38676) 2024-09-11T23:21:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.009+0000 7f095dc58640 1 -- 172.21.15.43:0/3132920375 learned_addr learned my addr 172.21.15.43:0/3132920375 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.009+0000 7f095cc56640 1 --2- 172.21.15.43:0/3132920375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e661ea12f0 0x55e661ea8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.009+0000 7f095d457640 1 --2- 172.21.15.43:0/3132920375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661ea09d0 0x55e661eab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.009+0000 7f095dc58640 1 -- 172.21.15.43:0/3132920375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e661ea12f0 msgr2=0x55e661ea8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.009+0000 7f095dc58640 1 --2- 172.21.15.43:0/3132920375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e661ea12f0 0x55e661ea8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.009+0000 7f095dc58640 1 -- 172.21.15.43:0/3132920375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661ea09d0 msgr2=0x55e661eab4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.009+0000 7f095dc58640 1 --2- 172.21.15.43:0/3132920375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661ea09d0 0x55e661eab4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.009+0000 7f095dc58640 1 -- 172.21.15.43:0/3132920375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e661b782b0 con 0x55e661eabbb0 2024-09-11T23:21:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.009+0000 7f095d457640 1 --2- 172.21.15.43:0/3132920375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661ea09d0 0x55e661eab4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:21:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.009+0000 7f095dc58640 1 --2- 172.21.15.43:0/3132920375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661eabbb0 0x55e661ea8d50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0950004a80 tx=0x7f095000a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3fd478882ecdaa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.009+0000 7f0940ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:21:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.009+0000 7f0940ff9640 1 -- 172.21.15.43:0/3132920375 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950015040 con 0x55e661eabbb0 2024-09-11T23:21:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.010+0000 7f0940ff9640 1 -- 172.21.15.43:0/3132920375 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f095000a790 con 0x55e661eabbb0 2024-09-11T23:21:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.010+0000 7f0961a94740 1 -- 172.21.15.43:0/3132920375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0950016040 con 0x55e661eabbb0 2024-09-11T23:21:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.010+0000 7f0940ff9640 1 -- 172.21.15.43:0/3132920375 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f095001aab0 con 0x55e661eabbb0 2024-09-11T23:21:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.010+0000 7f0961a94740 10 client.14964 fetch_fsmap learned FSMap version 23 2024-09-11T23:21:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.010+0000 7f0961a94740 1 -- 172.21.15.43:0/3132920375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55e661eba160 con 0x55e661eabbb0 2024-09-11T23:21:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.010+0000 7f0940ff9640 1 -- 172.21.15.43:0/3132920375 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0950016040 con 0x55e661eabbb0 2024-09-11T23:21:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.010+0000 7f0940ff9640 1 -- 172.21.15.43:0/3132920375 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f0950037760 con 0x55e661eabbb0 2024-09-11T23:21:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.010+0000 7f0961a94740 10 client.14964 fetch_fsmap finished waiting for FSMap version 23 2024-09-11T23:21:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.010+0000 7f0961a94740 10 client.14964 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T23:21:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.010+0000 7f0961a94740 20 client.14964 populate_metadata read hostname 'smithi043' 2024-09-11T23:21:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.010+0000 7f0961a94740 1 --2- 172.21.15.43:0/3132920375 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55e661ebc3f0 0x55e661ebd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.010+0000 7f0961a94740 4 client.14964 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:21:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.010+0000 7f0961a94740 1 -- 172.21.15.43:0/3132920375 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e661e14600 con 0x55e661ebc3f0 2024-09-11T23:21:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.010+0000 7f095d457640 1 --2- 172.21.15.43:0/3132920375 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55e661ebc3f0 0x55e661ebd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.011+0000 7f095d457640 1 --2- 172.21.15.43:0/3132920375 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55e661ebc3f0 0x55e661ebd860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ae56519290ca9b5 server_cookie=3026305e75cb0b81 in_seq=0 out_seq=0 2024-09-11T23:21:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.011+0000 7f0940ff9640 10 client.14964 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:21:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.012+0000 7f0940ff9640 1 -- 172.21.15.43:0/3132920375 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e661e14600 con 0x55e661ebc3f0 2024-09-11T23:21:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.012+0000 7f0940ff9640 10 client.14964 handle_command_reply: tid=0 2024-09-11T23:21:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.012+0000 7f0961a94740 1 client.14964 shutdown 2024-09-11T23:21:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.012+0000 7f0961a94740 1 -- 172.21.15.43:0/3132920375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661eabbb0 msgr2=0x55e661ea8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:20.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.012+0000 7f0961a94740 1 --2- 172.21.15.43:0/3132920375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661eabbb0 0x55e661ea8d50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0950004a80 tx=0x7f095000a400 comp rx=0 tx=0).stop 2024-09-11T23:21:20.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.012+0000 7f0961a94740 1 -- 172.21.15.43:0/3132920375 shutdown_connections 2024-09-11T23:21:20.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.012+0000 7f0961a94740 1 --2- 172.21.15.43:0/3132920375 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55e661ebc3f0 0x55e661ebd860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.012+0000 7f0961a94740 1 --2- 172.21.15.43:0/3132920375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661eabbb0 0x55e661ea8d50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.012+0000 7f0961a94740 1 --2- 172.21.15.43:0/3132920375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e661ea12f0 0x55e661ea8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.012+0000 7f0940ff9640 0 client.14964 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:21:20.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.012+0000 7f0961a94740 1 --2- 172.21.15.43:0/3132920375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661ea09d0 0x55e661eab4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.012+0000 7f0961a94740 1 -- 172.21.15.43:0/3132920375 >> 172.21.15.43:0/3132920375 conn(0x55e661e9cf60 msgr2=0x55e661ea2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:20.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.012+0000 7f0961a94740 1 -- 172.21.15.43:0/3132920375 shutdown_connections 2024-09-11T23:21:20.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.012+0000 7f0961a94740 1 -- 172.21.15.43:0/3132920375 wait complete. 2024-09-11T23:21:20.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.012+0000 7f0961a94740 20 client.14964 trim_cache size 0 max 16384 2024-09-11T23:21:20.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.030+0000 7f0961a94740 1 Processor -- start 2024-09-11T23:21:20.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.030+0000 7f0961a94740 1 -- start start 2024-09-11T23:21:20.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.030+0000 7f0961a94740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661ea09d0 0x55e661ebd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:20.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.030+0000 7f0961a94740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e661ea12f0 0x55e661ea2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:20.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.030+0000 7f0961a94740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661eabbb0 0x55e661ec5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:20.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.030+0000 7f0961a94740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e661ebbba0 con 0x55e661eabbb0 2024-09-11T23:21:20.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.030+0000 7f0961a94740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e661ebc270 con 0x55e661ea09d0 2024-09-11T23:21:20.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.030+0000 7f0961a94740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e661ebc940 con 0x55e661ea12f0 2024-09-11T23:21:20.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.030+0000 7f095dc58640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661eabbb0 0x55e661ec5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:20.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.031+0000 7f095dc58640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661eabbb0 0x55e661ec5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38682/0 (socket says 172.21.15.43:38682) 2024-09-11T23:21:20.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.031+0000 7f095dc58640 1 -- 172.21.15.43:0/2409040923 learned_addr learned my addr 172.21.15.43:0/2409040923 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:20.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.031+0000 7f095d457640 1 --2- 172.21.15.43:0/2409040923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661ea09d0 0x55e661ebd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:20.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.031+0000 7f095dc58640 1 -- 172.21.15.43:0/2409040923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e661ea12f0 msgr2=0x55e661ea2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:20.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.031+0000 7f095dc58640 1 --2- 172.21.15.43:0/2409040923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e661ea12f0 0x55e661ea2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.031+0000 7f095dc58640 1 -- 172.21.15.43:0/2409040923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661ea09d0 msgr2=0x55e661ebd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:20.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.031+0000 7f095dc58640 1 --2- 172.21.15.43:0/2409040923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661ea09d0 0x55e661ebd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.031+0000 7f095dc58640 1 -- 172.21.15.43:0/2409040923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e661ea6d60 con 0x55e661eabbb0 2024-09-11T23:21:20.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.031+0000 7f095dc58640 1 --2- 172.21.15.43:0/2409040923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661eabbb0 0x55e661ec5f30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0950007830 tx=0x7f09500072b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d49bfbe74a228f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:20.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.031+0000 7f0942ffd640 1 -- 172.21.15.43:0/2409040923 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950015070 con 0x55e661eabbb0 2024-09-11T23:21:20.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.031+0000 7f0942ffd640 1 -- 172.21.15.43:0/2409040923 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f095003c020 con 0x55e661eabbb0 2024-09-11T23:21:20.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.031+0000 7f0942ffd640 1 -- 172.21.15.43:0/2409040923 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095000a790 con 0x55e661eabbb0 2024-09-11T23:21:20.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.032+0000 7f0961a94740 1 -- 172.21.15.43:0/2409040923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661eabbb0 msgr2=0x55e661ec5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:20.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.032+0000 7f0961a94740 1 --2- 172.21.15.43:0/2409040923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661eabbb0 0x55e661ec5f30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0950007830 tx=0x7f09500072b0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.032+0000 7f0961a94740 1 -- 172.21.15.43:0/2409040923 shutdown_connections 2024-09-11T23:21:20.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.032+0000 7f0961a94740 1 --2- 172.21.15.43:0/2409040923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661eabbb0 0x55e661ec5f30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.032+0000 7f0961a94740 1 --2- 172.21.15.43:0/2409040923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e661ea12f0 0x55e661ea2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.032+0000 7f0961a94740 1 --2- 172.21.15.43:0/2409040923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661ea09d0 0x55e661ebd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.032+0000 7f0961a94740 1 -- 172.21.15.43:0/2409040923 >> 172.21.15.43:0/2409040923 conn(0x55e661e9cf60 msgr2=0x55e661ea2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:20.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.032+0000 7f0961a94740 1 -- 172.21.15.43:0/2409040923 shutdown_connections 2024-09-11T23:21:20.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.032+0000 7f0961a94740 1 -- 172.21.15.43:0/2409040923 wait complete. 2024-09-11T23:21:20.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.032+0000 7f0961a94740 1 -- start start 2024-09-11T23:21:20.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.032+0000 7f0961a94740 1 Processor -- start 2024-09-11T23:21:20.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.033+0000 7f0961a94740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e661ea09d0 0x55e661ea3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:20.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.033+0000 7f0961a94740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661ea12f0 0x55e661ebf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:20.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.033+0000 7f0961a94740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661eabbb0 0x55e661ebfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:20.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.033+0000 7f0961a94740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e661ebd6e0 con 0x55e661ea12f0 2024-09-11T23:21:20.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.033+0000 7f0961a94740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e661ebd010 con 0x55e661eabbb0 2024-09-11T23:21:20.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.033+0000 7f0961a94740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e661ebddb0 con 0x55e661ea09d0 2024-09-11T23:21:20.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.033+0000 7f0961a94740 10 client.0 _ll_register_callbacks cb 0x7f095000c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:20.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.033+0000 7f095cc56640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661ea12f0 0x55e661ebf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:20.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.033+0000 7f095cc56640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661ea12f0 0x55e661ebf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38690/0 (socket says 172.21.15.43:38690) 2024-09-11T23:21:20.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.033+0000 7f095cc56640 1 -- 172.21.15.43:0/2729029316 learned_addr learned my addr 172.21.15.43:0/2729029316 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:20.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.033+0000 7f095dc58640 1 --2- 172.21.15.43:0/2729029316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661eabbb0 0x55e661ebfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:20.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.033+0000 7f095d457640 1 --2- 172.21.15.43:0/2729029316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e661ea09d0 0x55e661ea3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:20.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.033+0000 7f095cc56640 1 -- 172.21.15.43:0/2729029316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e661ea09d0 msgr2=0x55e661ea3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:20.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.033+0000 7f095cc56640 1 --2- 172.21.15.43:0/2729029316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e661ea09d0 0x55e661ea3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.033+0000 7f095cc56640 1 -- 172.21.15.43:0/2729029316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661eabbb0 msgr2=0x55e661ebfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:20.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.033+0000 7f095cc56640 1 --2- 172.21.15.43:0/2729029316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661eabbb0 0x55e661ebfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.033+0000 7f095cc56640 1 -- 172.21.15.43:0/2729029316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e661ec03c0 con 0x55e661ea12f0 2024-09-11T23:21:20.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.033+0000 7f095dc58640 1 --2- 172.21.15.43:0/2729029316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661eabbb0 0x55e661ebfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:20.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.033+0000 7f095cc56640 1 --2- 172.21.15.43:0/2729029316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661ea12f0 0x55e661ebf790 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f094809b3e0 tx=0x7f0948096130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c9762361b411ae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:20.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.033+0000 7f0940ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:21:20.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.034+0000 7f0940ff9640 1 -- 172.21.15.43:0/2729029316 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094801d040 con 0x55e661ea12f0 2024-09-11T23:21:20.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.034+0000 7f0940ff9640 1 -- 172.21.15.43:0/2729029316 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f0948005310 con 0x55e661ea12f0 2024-09-11T23:21:20.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.034+0000 7f0961a94740 1 -- 172.21.15.43:0/2729029316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f095003c020 con 0x55e661ea12f0 2024-09-11T23:21:20.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.034+0000 7f0940ff9640 1 -- 172.21.15.43:0/2729029316 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f094809bde0 con 0x55e661ea12f0 2024-09-11T23:21:20.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.034+0000 7f0961a94740 10 client.14970 fetch_fsmap learned FSMap version 23 2024-09-11T23:21:20.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.034+0000 7f0961a94740 1 -- 172.21.15.43:0/2729029316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55e661f2aa70 con 0x55e661ea12f0 2024-09-11T23:21:20.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.034+0000 7f0940ff9640 1 -- 172.21.15.43:0/2729029316 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f095003c020 con 0x55e661ea12f0 2024-09-11T23:21:20.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.034+0000 7f0940ff9640 1 -- 172.21.15.43:0/2729029316 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f094809ab30 con 0x55e661ea12f0 2024-09-11T23:21:20.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.034+0000 7f0961a94740 10 client.14970 fetch_fsmap finished waiting for FSMap version 23 2024-09-11T23:21:20.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.034+0000 7f0961a94740 10 client.14970 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T23:21:20.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.034+0000 7f0961a94740 20 client.14970 populate_metadata read hostname 'smithi043' 2024-09-11T23:21:20.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.034+0000 7f0961a94740 1 --2- 172.21.15.43:0/2729029316 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55e661ebc3f0 0x55e661f2e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:20.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.034+0000 7f0961a94740 4 client.14970 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:21:20.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.034+0000 7f0961a94740 1 -- 172.21.15.43:0/2729029316 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "status"}) -- 0x55e661e0eb10 con 0x55e661ebc3f0 2024-09-11T23:21:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.035+0000 7f095d457640 1 --2- 172.21.15.43:0/2729029316 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55e661ebc3f0 0x55e661f2e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.035+0000 7f0940ff9640 10 client.14970 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:21:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.035+0000 7f095d457640 1 --2- 172.21.15.43:0/2729029316 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55e661ebc3f0 0x55e661f2e340 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c768e34050b6e88 server_cookie=c18566d3343108fc in_seq=0 out_seq=0 2024-09-11T23:21:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.035+0000 7f0940ff9640 1 -- 172.21.15.43:0/2729029316 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e661e0eb10 con 0x55e661ebc3f0 2024-09-11T23:21:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.035+0000 7f0940ff9640 10 client.14970 handle_command_reply: tid=0 2024-09-11T23:21:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.035+0000 7f0961a94740 1 client.14970 shutdown 2024-09-11T23:21:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.035+0000 7f0961a94740 1 -- 172.21.15.43:0/2729029316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661ea12f0 msgr2=0x55e661ebf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.035+0000 7f0961a94740 1 --2- 172.21.15.43:0/2729029316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661ea12f0 0x55e661ebf790 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f094809b3e0 tx=0x7f0948096130 comp rx=0 tx=0).stop 2024-09-11T23:21:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.035+0000 7f0961a94740 1 -- 172.21.15.43:0/2729029316 shutdown_connections 2024-09-11T23:21:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.035+0000 7f0961a94740 1 --2- 172.21.15.43:0/2729029316 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55e661ebc3f0 0x55e661f2e340 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.035+0000 7f0961a94740 1 --2- 172.21.15.43:0/2729029316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e661eabbb0 0x55e661ebfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.035+0000 7f0961a94740 1 --2- 172.21.15.43:0/2729029316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e661ea12f0 0x55e661ebf790 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.035+0000 7f0961a94740 1 --2- 172.21.15.43:0/2729029316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e661ea09d0 0x55e661ea3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.035+0000 7f0961a94740 1 -- 172.21.15.43:0/2729029316 >> 172.21.15.43:0/2729029316 conn(0x55e661e9cf60 msgr2=0x55e661eaa4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:20.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.035+0000 7f0940ff9640 0 client.14970 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:21:20.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.035+0000 7f0961a94740 1 -- 172.21.15.43:0/2729029316 shutdown_connections 2024-09-11T23:21:20.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.035+0000 7f0961a94740 1 -- 172.21.15.43:0/2729029316 wait complete. 2024-09-11T23:21:20.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.036+0000 7f0961a94740 20 client.14970 trim_cache size 0 max 16384 2024-09-11T23:21:20.050 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:21:20.050 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "25c0cd84-7093-11ef-bcea-c7b262605968", 2024-09-11T23:21:20.050 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 1, 2024-09-11T23:21:20.050 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24470, 2024-09-11T23:21:20.050 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-11T23:21:20.051 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-11T23:21:20.051 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-11T23:21:20.051 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 71.322728362999996, 2024-09-11T23:21:20.051 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 23, 2024-09-11T23:21:20.051 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 84, 2024-09-11T23:21:20.051 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 82, 2024-09-11T23:21:20.051 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 102.185261709 2024-09-11T23:21:20.051 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:21:20.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.036+0000 7f095f6bb640 1 -- 172.21.15.43:0/57131302 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f09440a2c90 msgr2=0x7f09440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:20.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.036+0000 7f095f6bb640 1 --2- 172.21.15.43:0/57131302 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f09440a2c90 0x7f09440a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f095001f000 tx=0x7f095001b000 comp rx=0 tx=0).stop 2024-09-11T23:21:20.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.036+0000 7f095f6bb640 1 -- 172.21.15.43:0/57131302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0958100e30 msgr2=0x7f0958199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:20.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.036+0000 7f095f6bb640 1 --2- 172.21.15.43:0/57131302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0958100e30 0x7f0958199880 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f094800c970 tx=0x7f094800ce30 comp rx=0 tx=0).stop 2024-09-11T23:21:20.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.036+0000 7f095f6bb640 1 -- 172.21.15.43:0/57131302 shutdown_connections 2024-09-11T23:21:20.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.036+0000 7f095f6bb640 1 --2- 172.21.15.43:0/57131302 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f09440a2c90 0x7f09440a5150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.036+0000 7f095f6bb640 1 --2- 172.21.15.43:0/57131302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09581055b0 0x7f09581a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.036+0000 7f095f6bb640 1 --2- 172.21.15.43:0/57131302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0958100e30 0x7f0958199880 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.036+0000 7f095f6bb640 1 --2- 172.21.15.43:0/57131302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0958100480 0x7f0958199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:20.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.036+0000 7f095f6bb640 1 -- 172.21.15.43:0/57131302 >> 172.21.15.43:0/57131302 conn(0x7f0958077210 msgr2=0x7f0958103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:20.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.036+0000 7f095f6bb640 1 -- 172.21.15.43:0/57131302 shutdown_connections 2024-09-11T23:21:20.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:20.036+0000 7f095f6bb640 1 -- 172.21.15.43:0/57131302 wait complete. 2024-09-11T23:21:20.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:19 smithi090 ceph-mon[35130]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T23:21:20.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:19 smithi097 ceph-mon[35562]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T23:21:20.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:19 smithi043 ceph-mon[30950]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T23:21:20.806 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T23:21:21.061 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:21.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.554+0000 7f9ec694d640 1 -- 172.21.15.43:0/2854267281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ec0103700 msgr2=0x7f9ec010bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.554+0000 7f9ec694d640 1 --2- 172.21.15.43:0/2854267281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ec0103700 0x7f9ec010bbf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009900 tx=0x7f9ea802a480 comp rx=0 tx=0).stop 2024-09-11T23:21:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.554+0000 7f9ec694d640 1 -- 172.21.15.43:0/2854267281 shutdown_connections 2024-09-11T23:21:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.554+0000 7f9ec694d640 1 --2- 172.21.15.43:0/2854267281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ec010c180 0x7f9ec010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.554+0000 7f9ec694d640 1 --2- 172.21.15.43:0/2854267281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ec0103700 0x7f9ec010bbf0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.554+0000 7f9ec694d640 1 --2- 172.21.15.43:0/2854267281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ec0102de0 0x7f9ec01031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.554+0000 7f9ec694d640 1 -- 172.21.15.43:0/2854267281 >> 172.21.15.43:0/2854267281 conn(0x7f9ec00fc7a0 msgr2=0x7f9ec00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.554+0000 7f9ec694d640 1 -- 172.21.15.43:0/2854267281 shutdown_connections 2024-09-11T23:21:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.554+0000 7f9ec694d640 1 -- 172.21.15.43:0/2854267281 wait complete. 2024-09-11T23:21:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.554+0000 7f9ec694d640 1 Processor -- start 2024-09-11T23:21:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.555+0000 7f9ec694d640 1 -- start start 2024-09-11T23:21:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.555+0000 7f9ec694d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ec0102de0 0x7f9ec0198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.555+0000 7f9ec694d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ec0103700 0x7f9ec0198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.555+0000 7f9ec694d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ec010c180 0x7f9ec019cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.555+0000 7f9ec694d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec0112030 con 0x7f9ec010c180 2024-09-11T23:21:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.555+0000 7f9ec694d640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec0111eb0 con 0x7f9ec0103700 2024-09-11T23:21:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.555+0000 7f9ec694d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec01121b0 con 0x7f9ec0102de0 2024-09-11T23:21:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.555+0000 7f9ec4eea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ec010c180 0x7f9ec019cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.555+0000 7f9ec4eea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ec010c180 0x7f9ec019cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59008/0 (socket says 172.21.15.43:59008) 2024-09-11T23:21:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.555+0000 7f9ec4eea640 1 -- 172.21.15.43:0/4154722057 learned_addr learned my addr 172.21.15.43:0/4154722057 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.555+0000 7f9eb7fff640 1 --2- 172.21.15.43:0/4154722057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ec0103700 0x7f9ec0198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.555+0000 7f9ebffff640 1 --2- 172.21.15.43:0/4154722057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ec0102de0 0x7f9ec0198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.555+0000 7f9ec4eea640 1 -- 172.21.15.43:0/4154722057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ec0102de0 msgr2=0x7f9ec0198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.555+0000 7f9ec4eea640 1 --2- 172.21.15.43:0/4154722057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ec0102de0 0x7f9ec0198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.555+0000 7f9ec4eea640 1 -- 172.21.15.43:0/4154722057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ec0103700 msgr2=0x7f9ec0198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.555+0000 7f9ec4eea640 1 --2- 172.21.15.43:0/4154722057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ec0103700 0x7f9ec0198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.555+0000 7f9ec4eea640 1 -- 172.21.15.43:0/4154722057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec019d5c0 con 0x7f9ec010c180 2024-09-11T23:21:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.555+0000 7f9eb7fff640 1 --2- 172.21.15.43:0/4154722057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ec0103700 0x7f9ec0198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.555+0000 7f9ebffff640 1 --2- 172.21.15.43:0/4154722057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ec0102de0 0x7f9ec0198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:21:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.555+0000 7f9ec4eea640 1 --2- 172.21.15.43:0/4154722057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ec010c180 0x7f9ec019cee0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800b970 tx=0x7f9eb800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.556+0000 7f9ebdffb640 1 -- 172.21.15.43:0/4154722057 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8019070 con 0x7f9ec010c180 2024-09-11T23:21:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.556+0000 7f9ec694d640 1 -- 172.21.15.43:0/4154722057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec019d8b0 con 0x7f9ec010c180 2024-09-11T23:21:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.556+0000 7f9ebdffb640 1 -- 172.21.15.43:0/4154722057 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eb800cb40 con 0x7f9ec010c180 2024-09-11T23:21:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.556+0000 7f9ebdffb640 1 -- 172.21.15.43:0/4154722057 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8007650 con 0x7f9ec010c180 2024-09-11T23:21:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.556+0000 7f9ec694d640 1 -- 172.21.15.43:0/4154722057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec01a5190 con 0x7f9ec010c180 2024-09-11T23:21:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.559+0000 7f9ebdffb640 1 -- 172.21.15.43:0/4154722057 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9eb801c070 con 0x7f9ec010c180 2024-09-11T23:21:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.559+0000 7f9ec8d26740 1 Processor -- start 2024-09-11T23:21:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.559+0000 7f9ec8d26740 1 -- start start 2024-09-11T23:21:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.559+0000 7f9ec8d26740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa3f9d0 0x55687aa3fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.559+0000 7f9ec8d26740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55687aa402f0 0x55687aa4a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.559+0000 7f9ec8d26740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55687aa4abb0 0x55687aa4cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.559+0000 7f9ec8d26740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55687a9a6090 con 0x55687aa3f9d0 2024-09-11T23:21:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.559+0000 7f9ec8d26740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55687a94da70 con 0x55687aa4abb0 2024-09-11T23:21:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.559+0000 7f9ec8d26740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55687a9a2050 con 0x55687aa402f0 2024-09-11T23:21:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.559+0000 7f9eb7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa3f9d0 0x55687aa3fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.559+0000 7f9eb7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa3f9d0 0x55687aa3fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59016/0 (socket says 172.21.15.43:59016) 2024-09-11T23:21:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.559+0000 7f9eb7fff640 1 -- 172.21.15.43:0/2623050329 learned_addr learned my addr 172.21.15.43:0/2623050329 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.559+0000 7f9ebffff640 1 --2- 172.21.15.43:0/2623050329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55687aa402f0 0x55687aa4a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.559+0000 7f9eb7fff640 1 --2- 172.21.15.43:0/2623050329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55687aa4abb0 0x55687aa4cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.560+0000 7f9eb7fff640 1 -- 172.21.15.43:0/2623050329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55687aa402f0 msgr2=0x55687aa4a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:21.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.560+0000 7f9eb7fff640 1 --2- 172.21.15.43:0/2623050329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55687aa402f0 0x55687aa4a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.560+0000 7f9eb7fff640 1 -- 172.21.15.43:0/2623050329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55687aa4abb0 msgr2=0x55687aa4cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:21.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.560+0000 7f9eb7fff640 1 --2- 172.21.15.43:0/2623050329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55687aa4abb0 0x55687aa4cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.560+0000 7f9eb7fff640 1 -- 172.21.15.43:0/2623050329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55687a95a130 con 0x55687aa3f9d0 2024-09-11T23:21:21.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.560+0000 7f9ebdffb640 1 --2- 172.21.15.43:0/4154722057 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9eac0a2c10 0x7f9eac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:21.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.560+0000 7f9eb7fff640 1 --2- 172.21.15.43:0/2623050329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa3f9d0 0x55687aa3fdb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8005ec0 tx=0x7f9ea8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3a2d326bf4a907 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:21.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.560+0000 7f9ebffff640 1 --2- 172.21.15.43:0/4154722057 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9eac0a2c10 0x7f9eac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:21.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.560+0000 7f9ebdffb640 1 -- 172.21.15.43:0/4154722057 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f9eb80ce6f0 con 0x7f9ec010c180 2024-09-11T23:21:21.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.560+0000 7f9eb67fc640 1 -- 172.21.15.43:0/2623050329 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8043030 con 0x55687aa3f9d0 2024-09-11T23:21:21.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.560+0000 7f9eb67fc640 1 -- 172.21.15.43:0/2623050329 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea8005020 con 0x55687aa3f9d0 2024-09-11T23:21:21.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.560+0000 7f9eb67fc640 1 -- 172.21.15.43:0/2623050329 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8035050 con 0x55687aa3f9d0 2024-09-11T23:21:21.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.561+0000 7f9ec8d26740 1 -- 172.21.15.43:0/2623050329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa3f9d0 msgr2=0x55687aa3fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.561+0000 7f9ebffff640 1 --2- 172.21.15.43:0/4154722057 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9eac0a2c10 0x7f9eac0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55687aa3cdc0 tx=0x7f9eb0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.561+0000 7f9ec8d26740 1 --2- 172.21.15.43:0/2623050329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa3f9d0 0x55687aa3fdb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8005ec0 tx=0x7f9ea8031400 comp rx=0 tx=0).stop 2024-09-11T23:21:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.561+0000 7f9ec8d26740 1 -- 172.21.15.43:0/2623050329 shutdown_connections 2024-09-11T23:21:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.561+0000 7f9ec8d26740 1 --2- 172.21.15.43:0/2623050329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55687aa4abb0 0x55687aa4cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.561+0000 7f9ec8d26740 1 --2- 172.21.15.43:0/2623050329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55687aa402f0 0x55687aa4a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.561+0000 7f9ec8d26740 1 --2- 172.21.15.43:0/2623050329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa3f9d0 0x55687aa3fdb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.561+0000 7f9ec8d26740 1 -- 172.21.15.43:0/2623050329 >> 172.21.15.43:0/2623050329 conn(0x55687aa3bf60 msgr2=0x55687aa3c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.561+0000 7f9ec8d26740 1 -- 172.21.15.43:0/2623050329 shutdown_connections 2024-09-11T23:21:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.561+0000 7f9ec8d26740 1 -- 172.21.15.43:0/2623050329 wait complete. 2024-09-11T23:21:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.561+0000 7f9ec8d26740 1 -- start start 2024-09-11T23:21:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.561+0000 7f9ec8d26740 1 Processor -- start 2024-09-11T23:21:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.562+0000 7f9ec8d26740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa3f9d0 0x55687aa4a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.562+0000 7f9ec8d26740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55687aa402f0 0x55687aa477c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:21.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.562+0000 7f9ec8d26740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55687aa4abb0 0x55687aa47d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:21.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.562+0000 7f9ec8d26740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55687aa4d780 con 0x55687aa3f9d0 2024-09-11T23:21:21.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.562+0000 7f9ec8d26740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55687a960f20 con 0x55687aa4abb0 2024-09-11T23:21:21.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.562+0000 7f9ec8d26740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55687aa4d900 con 0x55687aa402f0 2024-09-11T23:21:21.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.562+0000 7f9ec8d26740 10 client.0 _ll_register_callbacks cb 0x55687a92a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:21.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.562+0000 7f9ebffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa3f9d0 0x55687aa4a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:21.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.562+0000 7f9ebffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa3f9d0 0x55687aa4a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59032/0 (socket says 172.21.15.43:59032) 2024-09-11T23:21:21.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.562+0000 7f9ebffff640 1 -- 172.21.15.43:0/3101397989 learned_addr learned my addr 172.21.15.43:0/3101397989 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:21.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.562+0000 7f9eb7fff640 1 --2- 172.21.15.43:0/3101397989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55687aa402f0 0x55687aa477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:21.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.562+0000 7f9ec4eea640 1 --2- 172.21.15.43:0/3101397989 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55687aa4abb0 0x55687aa47d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:21.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.562+0000 7f9ebffff640 1 -- 172.21.15.43:0/3101397989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55687aa402f0 msgr2=0x55687aa477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:21.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.562+0000 7f9ebffff640 1 --2- 172.21.15.43:0/3101397989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55687aa402f0 0x55687aa477c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.562+0000 7f9ebffff640 1 -- 172.21.15.43:0/3101397989 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55687aa4abb0 msgr2=0x55687aa47d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:21.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.562+0000 7f9ebffff640 1 --2- 172.21.15.43:0/3101397989 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55687aa4abb0 0x55687aa47d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.562+0000 7f9ebffff640 1 -- 172.21.15.43:0/3101397989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55687a7172b0 con 0x55687aa3f9d0 2024-09-11T23:21:21.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.562+0000 7f9ec4eea640 1 --2- 172.21.15.43:0/3101397989 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55687aa4abb0 0x55687aa47d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:21:21.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.562+0000 7f9ebffff640 1 --2- 172.21.15.43:0/3101397989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa3f9d0 0x55687aa4a470 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00157a0 tx=0x7f9eb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56569f27c87cd8b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:21.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.562+0000 7f9e9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:21:21.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.563+0000 7f9e9f7fe640 1 -- 172.21.15.43:0/3101397989 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0023070 con 0x55687aa3f9d0 2024-09-11T23:21:21.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.563+0000 7f9e9f7fe640 1 -- 172.21.15.43:0/3101397989 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f9eb0013b20 con 0x55687aa3f9d0 2024-09-11T23:21:21.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.563+0000 7f9ec8d26740 1 -- 172.21.15.43:0/3101397989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea8005020 con 0x55687aa3f9d0 2024-09-11T23:21:21.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.563+0000 7f9e9f7fe640 1 -- 172.21.15.43:0/3101397989 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9eb002f050 con 0x55687aa3f9d0 2024-09-11T23:21:21.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.563+0000 7f9ec8d26740 10 client.14982 fetch_fsmap learned FSMap version 23 2024-09-11T23:21:21.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.563+0000 7f9ec8d26740 1 -- 172.21.15.43:0/3101397989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55687aa59180 con 0x55687aa3f9d0 2024-09-11T23:21:21.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.563+0000 7f9e9f7fe640 1 -- 172.21.15.43:0/3101397989 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea8005020 con 0x55687aa3f9d0 2024-09-11T23:21:21.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.563+0000 7f9e9f7fe640 1 -- 172.21.15.43:0/3101397989 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f9eb001e070 con 0x55687aa3f9d0 2024-09-11T23:21:21.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.563+0000 7f9ec8d26740 10 client.14982 fetch_fsmap finished waiting for FSMap version 23 2024-09-11T23:21:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.563+0000 7f9ec8d26740 10 client.14982 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:21:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.563+0000 7f9ec8d26740 20 client.14982 populate_metadata read hostname 'smithi043' 2024-09-11T23:21:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.563+0000 7f9ec8d26740 1 --2- 172.21.15.43:0/3101397989 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55687aa5b7e0 0x55687aa5cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.563+0000 7f9ec8d26740 4 client.14982 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:21:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.563+0000 7f9ec8d26740 1 -- 172.21.15.43:0/3101397989 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55687a9b3600 con 0x55687aa5b7e0 2024-09-11T23:21:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.564+0000 7f9eb7fff640 1 --2- 172.21.15.43:0/3101397989 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55687aa5b7e0 0x55687aa5cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.564+0000 7f9e9f7fe640 10 client.14982 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:21:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.564+0000 7f9eb7fff640 1 --2- 172.21.15.43:0/3101397989 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55687aa5b7e0 0x55687aa5cc50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b98d7b0308f0f13 server_cookie=97133aab82971e95 in_seq=0 out_seq=0 2024-09-11T23:21:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.565+0000 7f9e9f7fe640 1 -- 172.21.15.43:0/3101397989 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55687a9b3600 con 0x55687aa5b7e0 2024-09-11T23:21:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.565+0000 7f9e9f7fe640 10 client.14982 handle_command_reply: tid=0 2024-09-11T23:21:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.565+0000 7f9ec8d26740 1 client.14982 shutdown 2024-09-11T23:21:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.565+0000 7f9ec8d26740 1 -- 172.21.15.43:0/3101397989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa3f9d0 msgr2=0x55687aa4a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.565+0000 7f9ec8d26740 1 --2- 172.21.15.43:0/3101397989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa3f9d0 0x55687aa4a470 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00157a0 tx=0x7f9eb00046c0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.565+0000 7f9ec8d26740 1 -- 172.21.15.43:0/3101397989 shutdown_connections 2024-09-11T23:21:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.565+0000 7f9ec8d26740 1 --2- 172.21.15.43:0/3101397989 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55687aa5b7e0 0x55687aa5cc50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.565+0000 7f9ec8d26740 1 --2- 172.21.15.43:0/3101397989 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55687aa4abb0 0x55687aa47d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.565+0000 7f9ec8d26740 1 --2- 172.21.15.43:0/3101397989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55687aa402f0 0x55687aa477c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.565+0000 7f9ec8d26740 1 --2- 172.21.15.43:0/3101397989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa3f9d0 0x55687aa4a470 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.565+0000 7f9ec8d26740 1 -- 172.21.15.43:0/3101397989 >> 172.21.15.43:0/3101397989 conn(0x55687aa3bf60 msgr2=0x55687aa41c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.565+0000 7f9e9f7fe640 0 client.14982 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:21:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.565+0000 7f9ec8d26740 1 -- 172.21.15.43:0/3101397989 shutdown_connections 2024-09-11T23:21:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.565+0000 7f9ec8d26740 1 -- 172.21.15.43:0/3101397989 wait complete. 2024-09-11T23:21:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.565+0000 7f9ec8d26740 20 client.14982 trim_cache size 0 max 16384 2024-09-11T23:21:21.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.584+0000 7f9ec8d26740 1 Processor -- start 2024-09-11T23:21:21.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.584+0000 7f9ec8d26740 1 -- start start 2024-09-11T23:21:21.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.584+0000 7f9ec8d26740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55687aa3f9d0 0x55687aa65320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:21.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.584+0000 7f9ec8d26740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55687aa402f0 0x55687aa44a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:21.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.584+0000 7f9ec8d26740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa4abb0 0x55687aa44f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:21.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.584+0000 7f9ec8d26740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55687aa5af90 con 0x55687aa4abb0 2024-09-11T23:21:21.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.584+0000 7f9ec8d26740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55687aa5b660 con 0x55687aa402f0 2024-09-11T23:21:21.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.584+0000 7f9ec8d26740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55687aa5bd30 con 0x55687aa3f9d0 2024-09-11T23:21:21.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.584+0000 7f9ec4eea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa4abb0 0x55687aa44f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:21.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.584+0000 7f9ec4eea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa4abb0 0x55687aa44f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59040/0 (socket says 172.21.15.43:59040) 2024-09-11T23:21:21.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.584+0000 7f9ec4eea640 1 -- 172.21.15.43:0/905625150 learned_addr learned my addr 172.21.15.43:0/905625150 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:21.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.584+0000 7f9ebffff640 1 --2- 172.21.15.43:0/905625150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55687aa3f9d0 0x55687aa65320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:21.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.584+0000 7f9ec4eea640 1 -- 172.21.15.43:0/905625150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55687aa3f9d0 msgr2=0x55687aa65320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:21.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.584+0000 7f9ec4eea640 1 --2- 172.21.15.43:0/905625150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55687aa3f9d0 0x55687aa65320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.584+0000 7f9eb7fff640 1 --2- 172.21.15.43:0/905625150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55687aa402f0 0x55687aa44a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:21.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.584+0000 7f9ec4eea640 1 -- 172.21.15.43:0/905625150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55687aa402f0 msgr2=0x55687aa44a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:21.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.584+0000 7f9ec4eea640 1 --2- 172.21.15.43:0/905625150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55687aa402f0 0x55687aa44a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.584+0000 7f9ec4eea640 1 -- 172.21.15.43:0/905625150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55687aa3ba00 con 0x55687aa4abb0 2024-09-11T23:21:21.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.585+0000 7f9ec4eea640 1 --2- 172.21.15.43:0/905625150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa4abb0 0x55687aa44f60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9eb80cc1c0 tx=0x7f9eb809a900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17716ded520aa8da server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:21.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.585+0000 7f9eb5ffb640 1 -- 172.21.15.43:0/905625150 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb809f290 con 0x55687aa4abb0 2024-09-11T23:21:21.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.585+0000 7f9eb5ffb640 1 -- 172.21.15.43:0/905625150 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eb80a3b50 con 0x55687aa4abb0 2024-09-11T23:21:21.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.585+0000 7f9eb5ffb640 1 -- 172.21.15.43:0/905625150 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb80a3920 con 0x55687aa4abb0 2024-09-11T23:21:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.585+0000 7f9ec8d26740 1 -- 172.21.15.43:0/905625150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa4abb0 msgr2=0x55687aa44f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.585+0000 7f9ec8d26740 1 --2- 172.21.15.43:0/905625150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa4abb0 0x55687aa44f60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9eb80cc1c0 tx=0x7f9eb809a900 comp rx=0 tx=0).stop 2024-09-11T23:21:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.585+0000 7f9ec8d26740 1 -- 172.21.15.43:0/905625150 shutdown_connections 2024-09-11T23:21:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.585+0000 7f9ec8d26740 1 --2- 172.21.15.43:0/905625150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa4abb0 0x55687aa44f60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.585+0000 7f9ec8d26740 1 --2- 172.21.15.43:0/905625150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55687aa402f0 0x55687aa44a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.585+0000 7f9ec8d26740 1 --2- 172.21.15.43:0/905625150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55687aa3f9d0 0x55687aa65320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.585+0000 7f9ec8d26740 1 -- 172.21.15.43:0/905625150 >> 172.21.15.43:0/905625150 conn(0x55687aa3bf60 msgr2=0x55687aa46090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.586+0000 7f9ec8d26740 1 -- 172.21.15.43:0/905625150 shutdown_connections 2024-09-11T23:21:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.586+0000 7f9ec8d26740 1 -- 172.21.15.43:0/905625150 wait complete. 2024-09-11T23:21:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.586+0000 7f9ec8d26740 1 -- start start 2024-09-11T23:21:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.586+0000 7f9ec8d26740 1 Processor -- start 2024-09-11T23:21:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.586+0000 7f9ec8d26740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa3f9d0 0x55687aac2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:21.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.586+0000 7f9ec8d26740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55687aa402f0 0x55687aac2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:21.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.586+0000 7f9ec8d26740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55687aa4abb0 0x55687aa5eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:21.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.586+0000 7f9ec8d26740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55687aa5cad0 con 0x55687aa3f9d0 2024-09-11T23:21:21.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.586+0000 7f9ec8d26740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55687aa5c400 con 0x55687aa4abb0 2024-09-11T23:21:21.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.586+0000 7f9ec8d26740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55687aa5d1a0 con 0x55687aa402f0 2024-09-11T23:21:21.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.586+0000 7f9ec8d26740 10 client.0 _ll_register_callbacks cb 0x7f9eb0015450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:21.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.586+0000 7f9ebffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa3f9d0 0x55687aac2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:21.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.587+0000 7f9ebffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa3f9d0 0x55687aac2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59054/0 (socket says 172.21.15.43:59054) 2024-09-11T23:21:21.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.587+0000 7f9ebffff640 1 -- 172.21.15.43:0/3741914351 learned_addr learned my addr 172.21.15.43:0/3741914351 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:21.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.587+0000 7f9eb7fff640 1 --2- 172.21.15.43:0/3741914351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55687aa402f0 0x55687aac2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:21.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.587+0000 7f9ec4eea640 1 --2- 172.21.15.43:0/3741914351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55687aa4abb0 0x55687aa5eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:21.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.587+0000 7f9ebffff640 1 -- 172.21.15.43:0/3741914351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55687aa402f0 msgr2=0x55687aac2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:21.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.587+0000 7f9ebffff640 1 --2- 172.21.15.43:0/3741914351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55687aa402f0 0x55687aac2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.587+0000 7f9ebffff640 1 -- 172.21.15.43:0/3741914351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55687aa4abb0 msgr2=0x55687aa5eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:21.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.587+0000 7f9ebffff640 1 --2- 172.21.15.43:0/3741914351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55687aa4abb0 0x55687aa5eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.587+0000 7f9ebffff640 1 -- 172.21.15.43:0/3741914351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55687aa5f330 con 0x55687aa3f9d0 2024-09-11T23:21:21.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.587+0000 7f9ebffff640 1 --2- 172.21.15.43:0/3741914351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa3f9d0 0x55687aac2680 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9eb000a270 tx=0x7f9eb00127e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d19af400e1c103e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:21.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.587+0000 7f9e9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:21:21.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.587+0000 7f9e9f7fe640 1 -- 172.21.15.43:0/3741914351 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0037070 con 0x55687aa3f9d0 2024-09-11T23:21:21.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.587+0000 7f9ec8d26740 1 -- 172.21.15.43:0/3741914351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9eb80a3b50 con 0x55687aa3f9d0 2024-09-11T23:21:21.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.587+0000 7f9e9f7fe640 1 -- 172.21.15.43:0/3741914351 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f9eb00137f0 con 0x55687aa3f9d0 2024-09-11T23:21:21.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.587+0000 7f9e9f7fe640 1 -- 172.21.15.43:0/3741914351 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9eb0023040 con 0x55687aa3f9d0 2024-09-11T23:21:21.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.587+0000 7f9ec8d26740 10 client.14988 fetch_fsmap learned FSMap version 23 2024-09-11T23:21:21.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.587+0000 7f9ec8d26740 1 -- 172.21.15.43:0/3741914351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55687aac9e60 con 0x55687aa3f9d0 2024-09-11T23:21:21.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.588+0000 7f9e9f7fe640 1 -- 172.21.15.43:0/3741914351 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eb80a3b50 con 0x55687aa3f9d0 2024-09-11T23:21:21.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.588+0000 7f9e9f7fe640 1 -- 172.21.15.43:0/3741914351 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f9eb003a030 con 0x55687aa3f9d0 2024-09-11T23:21:21.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.588+0000 7f9ec8d26740 10 client.14988 fetch_fsmap finished waiting for FSMap version 23 2024-09-11T23:21:21.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.588+0000 7f9ec8d26740 10 client.14988 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:21:21.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.588+0000 7f9ec8d26740 20 client.14988 populate_metadata read hostname 'smithi043' 2024-09-11T23:21:21.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.588+0000 7f9ec8d26740 1 --2- 172.21.15.43:0/3741914351 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55687aacea20 0x55687aacd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:21.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.588+0000 7f9ec8d26740 4 client.14988 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:21:21.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.588+0000 7f9ec8d26740 1 -- 172.21.15.43:0/3741914351 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "status"}) -- 0x55687a9adb10 con 0x55687aacea20 2024-09-11T23:21:21.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.588+0000 7f9eb7fff640 1 --2- 172.21.15.43:0/3741914351 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55687aacea20 0x55687aacd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:21.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.588+0000 7f9eb7fff640 1 --2- 172.21.15.43:0/3741914351 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55687aacea20 0x55687aacd730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df53db55f77dde6d server_cookie=fffa87d331f05658 in_seq=0 out_seq=0 2024-09-11T23:21:21.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.588+0000 7f9e9f7fe640 10 client.14988 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:21:21.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.589+0000 7f9e9f7fe640 1 -- 172.21.15.43:0/3741914351 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55687a9adb10 con 0x55687aacea20 2024-09-11T23:21:21.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.589+0000 7f9e9f7fe640 10 client.14988 handle_command_reply: tid=0 2024-09-11T23:21:21.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.589+0000 7f9ec8d26740 1 client.14988 shutdown 2024-09-11T23:21:21.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.589+0000 7f9ec8d26740 1 -- 172.21.15.43:0/3741914351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa3f9d0 msgr2=0x55687aac2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:21.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.589+0000 7f9ec8d26740 1 --2- 172.21.15.43:0/3741914351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa3f9d0 0x55687aac2680 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9eb000a270 tx=0x7f9eb00127e0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.589+0000 7f9ec8d26740 1 -- 172.21.15.43:0/3741914351 shutdown_connections 2024-09-11T23:21:21.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.589+0000 7f9ec8d26740 1 --2- 172.21.15.43:0/3741914351 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55687aacea20 0x55687aacd730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.589+0000 7f9ec8d26740 1 --2- 172.21.15.43:0/3741914351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55687aa4abb0 0x55687aa5eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.589+0000 7f9ec8d26740 1 --2- 172.21.15.43:0/3741914351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55687aa402f0 0x55687aac2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.589+0000 7f9e9f7fe640 0 client.14988 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:21:21.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.589+0000 7f9ec8d26740 1 --2- 172.21.15.43:0/3741914351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55687aa3f9d0 0x55687aac2680 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.589+0000 7f9ec8d26740 1 -- 172.21.15.43:0/3741914351 >> 172.21.15.43:0/3741914351 conn(0x55687aa3bf60 msgr2=0x55687aa48e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:21.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.589+0000 7f9ec8d26740 1 -- 172.21.15.43:0/3741914351 shutdown_connections 2024-09-11T23:21:21.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.589+0000 7f9ec8d26740 1 -- 172.21.15.43:0/3741914351 wait complete. 2024-09-11T23:21:21.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.589+0000 7f9ec8d26740 20 client.14988 trim_cache size 0 max 16384 2024-09-11T23:21:21.602 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:21:21.602 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "25c0cd84-7093-11ef-bcea-c7b262605968", 2024-09-11T23:21:21.602 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-11T23:21:21.602 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24497, 2024-09-11T23:21:21.602 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-11T23:21:21.602 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-11T23:21:21.602 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-11T23:21:21.602 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 80.853578962, 2024-09-11T23:21:21.602 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 23, 2024-09-11T23:21:21.602 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 84, 2024-09-11T23:21:21.602 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 82, 2024-09-11T23:21:21.602 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 91.874703362999995 2024-09-11T23:21:21.603 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:21:21.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.590+0000 7f9ec694d640 1 -- 172.21.15.43:0/4154722057 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9eac0a2c10 msgr2=0x7f9eac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:21.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.590+0000 7f9ec694d640 1 --2- 172.21.15.43:0/4154722057 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9eac0a2c10 0x7f9eac0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55687aa3cdc0 tx=0x7f9eb0008040 comp rx=0 tx=0).stop 2024-09-11T23:21:21.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.590+0000 7f9ec694d640 1 -- 172.21.15.43:0/4154722057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ec010c180 msgr2=0x7f9ec019cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:21.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.590+0000 7f9ec694d640 1 --2- 172.21.15.43:0/4154722057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ec010c180 0x7f9ec019cee0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800b970 tx=0x7f9eb800be30 comp rx=0 tx=0).stop 2024-09-11T23:21:21.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.590+0000 7f9ec694d640 1 -- 172.21.15.43:0/4154722057 shutdown_connections 2024-09-11T23:21:21.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.590+0000 7f9ec694d640 1 --2- 172.21.15.43:0/4154722057 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f9eac0a2c10 0x7f9eac0a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.590+0000 7f9ec694d640 1 --2- 172.21.15.43:0/4154722057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ec010c180 0x7f9ec019cee0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.590+0000 7f9ec694d640 1 --2- 172.21.15.43:0/4154722057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ec0103700 0x7f9ec0198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.590+0000 7f9ec694d640 1 --2- 172.21.15.43:0/4154722057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ec0102de0 0x7f9ec0198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:21.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.590+0000 7f9ec694d640 1 -- 172.21.15.43:0/4154722057 >> 172.21.15.43:0/4154722057 conn(0x7f9ec00fc7a0 msgr2=0x7f9ec00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:21.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.590+0000 7f9ec694d640 1 -- 172.21.15.43:0/4154722057 shutdown_connections 2024-09-11T23:21:21.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:21.590+0000 7f9ec694d640 1 -- 172.21.15.43:0/4154722057 wait complete. 2024-09-11T23:21:22.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:21 smithi090 ceph-mon[35130]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:21:22.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:21 smithi097 ceph-mon[35562]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:21:22.307 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:21 smithi043 ceph-mon[30950]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:21:22.321 INFO:tasks.cephadm:Setup complete, yielding 2024-09-11T23:21:22.322 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T23:21:22.332 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-09-11T23:21:22.332 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-11T23:21:22.585 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:23.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.055+0000 7ff6e97b6640 1 -- 172.21.15.43:0/1283077437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e4102690 msgr2=0x7ff6e4108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:23.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.055+0000 7ff6e97b6640 1 --2- 172.21.15.43:0/1283077437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e4102690 0x7ff6e4108f20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009ca0 tx=0x7ff6dc02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:21:23.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.055+0000 7ff6e97b6640 1 -- 172.21.15.43:0/1283077437 shutdown_connections 2024-09-11T23:21:23.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.055+0000 7ff6e97b6640 1 --2- 172.21.15.43:0/1283077437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e4102690 0x7ff6e4108f20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:23.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.055+0000 7ff6e97b6640 1 --2- 172.21.15.43:0/1283077437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6e4101cd0 0x7ff6e4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.055+0000 7ff6e97b6640 1 --2- 172.21.15.43:0/1283077437 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6e4100ad0 0x7ff6e4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.055+0000 7ff6e97b6640 1 -- 172.21.15.43:0/1283077437 >> 172.21.15.43:0/1283077437 conn(0x7ff6e40fc280 msgr2=0x7ff6e40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.055+0000 7ff6e97b6640 1 -- 172.21.15.43:0/1283077437 shutdown_connections 2024-09-11T23:21:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.055+0000 7ff6e97b6640 1 -- 172.21.15.43:0/1283077437 wait complete. 2024-09-11T23:21:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.055+0000 7ff6e97b6640 1 Processor -- start 2024-09-11T23:21:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.055+0000 7ff6e97b6640 1 -- start start 2024-09-11T23:21:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.056+0000 7ff6e97b6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6e4100ad0 0x7ff6e4197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.056+0000 7ff6e97b6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e4101cd0 0x7ff6e4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.056+0000 7ff6e97b6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6e4102690 0x7ff6e419f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.056+0000 7ff6e97b6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e410d900 con 0x7ff6e4101cd0 2024-09-11T23:21:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.056+0000 7ff6e97b6640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e410d780 con 0x7ff6e4102690 2024-09-11T23:21:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.056+0000 7ff6e97b6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e410da80 con 0x7ff6e4100ad0 2024-09-11T23:21:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.056+0000 7ff6dbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e4101cd0 0x7ff6e4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:23.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.056+0000 7ff6dbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e4101cd0 0x7ff6e4198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59072/0 (socket says 172.21.15.43:59072) 2024-09-11T23:21:23.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.056+0000 7ff6dbfff640 1 -- 172.21.15.43:0/1924843706 learned_addr learned my addr 172.21.15.43:0/1924843706 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:23.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.056+0000 7ff6e2ffd640 1 --2- 172.21.15.43:0/1924843706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6e4100ad0 0x7ff6e4197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:23.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.056+0000 7ff6e37fe640 1 --2- 172.21.15.43:0/1924843706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6e4102690 0x7ff6e419f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:23.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.056+0000 7ff6dbfff640 1 -- 172.21.15.43:0/1924843706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6e4100ad0 msgr2=0x7ff6e4197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:23.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.056+0000 7ff6dbfff640 1 --2- 172.21.15.43:0/1924843706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6e4100ad0 0x7ff6e4197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:23.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.056+0000 7ff6dbfff640 1 -- 172.21.15.43:0/1924843706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6e4102690 msgr2=0x7ff6e419f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:23.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.056+0000 7ff6dbfff640 1 --2- 172.21.15.43:0/1924843706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6e4102690 0x7ff6e419f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:23.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.056+0000 7ff6dbfff640 1 -- 172.21.15.43:0/1924843706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e419fb00 con 0x7ff6e4101cd0 2024-09-11T23:21:23.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.056+0000 7ff6e37fe640 1 --2- 172.21.15.43:0/1924843706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6e4102690 0x7ff6e419f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.056+0000 7ff6dbfff640 1 --2- 172.21.15.43:0/1924843706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e4101cd0 0x7ff6e4198410 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00b4a0 tx=0x7ff6cc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.057+0000 7ff6e0ff9640 1 -- 172.21.15.43:0/1924843706 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc014070 con 0x7ff6e4101cd0 2024-09-11T23:21:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.057+0000 7ff6e97b6640 1 -- 172.21.15.43:0/1924843706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e419fdf0 con 0x7ff6e4101cd0 2024-09-11T23:21:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.057+0000 7ff6e0ff9640 1 -- 172.21.15.43:0/1924843706 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6cc010af0 con 0x7ff6e4101cd0 2024-09-11T23:21:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.057+0000 7ff6e0ff9640 1 -- 172.21.15.43:0/1924843706 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc017d10 con 0x7ff6e4101cd0 2024-09-11T23:21:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.057+0000 7ff6e97b6640 1 -- 172.21.15.43:0/1924843706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e41a02d0 con 0x7ff6e4101cd0 2024-09-11T23:21:23.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.058+0000 7ff6e97b6640 1 -- 172.21.15.43:0/1924843706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6e4107e80 con 0x7ff6e4101cd0 2024-09-11T23:21:23.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.059+0000 7ff6e0ff9640 1 -- 172.21.15.43:0/1924843706 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff6cc017070 con 0x7ff6e4101cd0 2024-09-11T23:21:23.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.059+0000 7ff6e0ff9640 1 --2- 172.21.15.43:0/1924843706 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff6d00a2c40 0x7ff6d00a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:23.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.060+0000 7ff6e0ff9640 1 -- 172.21.15.43:0/1924843706 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7ff6cc0ce470 con 0x7ff6e4101cd0 2024-09-11T23:21:23.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.060+0000 7ff6e2ffd640 1 --2- 172.21.15.43:0/1924843706 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff6d00a2c40 0x7ff6d00a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:23.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.060+0000 7ff6e2ffd640 1 --2- 172.21.15.43:0/1924843706 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff6d00a2c40 0x7ff6d00a5100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4101b30 tx=0x7ff6d400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:23.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.064+0000 7ff6e0ff9640 1 -- 172.21.15.43:0/1924843706 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff6cc09b360 con 0x7ff6e4101cd0 2024-09-11T23:21:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.257+0000 7ff6e97b6640 1 -- 172.21.15.43:0/1924843706 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7ff6e4062630 con 0x7ff6d00a2c40 2024-09-11T23:21:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.258+0000 7ff6e0ff9640 1 -- 172.21.15.43:0/1924843706 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7ff6e4062630 con 0x7ff6d00a2c40 2024-09-11T23:21:23.258 INFO:teuthology.orchestra.run.smithi043.stdout:Backend: cephadm 2024-09-11T23:21:23.258 INFO:teuthology.orchestra.run.smithi043.stdout:Available: Yes 2024-09-11T23:21:23.258 INFO:teuthology.orchestra.run.smithi043.stdout:Paused: No 2024-09-11T23:21:23.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.260+0000 7ff6e97b6640 1 -- 172.21.15.43:0/1924843706 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff6d00a2c40 msgr2=0x7ff6d00a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:23.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.260+0000 7ff6e97b6640 1 --2- 172.21.15.43:0/1924843706 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff6d00a2c40 0x7ff6d00a5100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4101b30 tx=0x7ff6d400a2d0 comp rx=0 tx=0).stop 2024-09-11T23:21:23.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.260+0000 7ff6e97b6640 1 -- 172.21.15.43:0/1924843706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e4101cd0 msgr2=0x7ff6e4198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.260+0000 7ff6e97b6640 1 --2- 172.21.15.43:0/1924843706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e4101cd0 0x7ff6e4198410 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00b4a0 tx=0x7ff6cc00b960 comp rx=0 tx=0).stop 2024-09-11T23:21:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.260+0000 7ff6e97b6640 1 -- 172.21.15.43:0/1924843706 shutdown_connections 2024-09-11T23:21:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.260+0000 7ff6e97b6640 1 --2- 172.21.15.43:0/1924843706 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ff6d00a2c40 0x7ff6d00a5100 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.260+0000 7ff6e97b6640 1 --2- 172.21.15.43:0/1924843706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6e4102690 0x7ff6e419f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.260+0000 7ff6e97b6640 1 --2- 172.21.15.43:0/1924843706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e4101cd0 0x7ff6e4198410 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.260+0000 7ff6e97b6640 1 --2- 172.21.15.43:0/1924843706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6e4100ad0 0x7ff6e4197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.260+0000 7ff6e97b6640 1 -- 172.21.15.43:0/1924843706 >> 172.21.15.43:0/1924843706 conn(0x7ff6e40fc280 msgr2=0x7ff6e4105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.260+0000 7ff6e97b6640 1 -- 172.21.15.43:0/1924843706 shutdown_connections 2024-09-11T23:21:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:23.260+0000 7ff6e97b6640 1 -- 172.21.15.43:0/1924843706 wait complete. 2024-09-11T23:21:23.927 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-11T23:21:24.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:23 smithi043 ceph-mon[30950]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T23:21:24.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:23 smithi043 ceph-mon[30950]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:21:24.196 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:24.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:23 smithi090 ceph-mon[35130]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T23:21:24.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:23 smithi090 ceph-mon[35130]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:21:24.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:23 smithi097 ceph-mon[35562]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T23:21:24.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:23 smithi097 ceph-mon[35562]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:21:24.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.714+0000 7fd0009b6640 1 -- 172.21.15.43:0/1611402634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcffc102e00 msgr2=0x7fcffc103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:24.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.714+0000 7fd0009b6640 1 --2- 172.21.15.43:0/1611402634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcffc102e00 0x7fcffc103280 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009900 tx=0x7fcfe402a410 comp rx=0 tx=0).stop 2024-09-11T23:21:24.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.714+0000 7fd0009b6640 1 -- 172.21.15.43:0/1611402634 shutdown_connections 2024-09-11T23:21:24.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.714+0000 7fd0009b6640 1 --2- 172.21.15.43:0/1611402634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcffc1037c0 0x7fcffc10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:24.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.714+0000 7fd0009b6640 1 --2- 172.21.15.43:0/1611402634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcffc102e00 0x7fcffc103280 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:24.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.714+0000 7fd0009b6640 1 --2- 172.21.15.43:0/1611402634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcffc101c00 0x7fcffc102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:24.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.714+0000 7fd0009b6640 1 -- 172.21.15.43:0/1611402634 >> 172.21.15.43:0/1611402634 conn(0x7fcffc0fd390 msgr2=0x7fcffc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:24.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.714+0000 7fd0009b6640 1 -- 172.21.15.43:0/1611402634 shutdown_connections 2024-09-11T23:21:24.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.714+0000 7fd0009b6640 1 -- 172.21.15.43:0/1611402634 wait complete. 2024-09-11T23:21:24.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.715+0000 7fd0009b6640 1 Processor -- start 2024-09-11T23:21:24.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.715+0000 7fd0009b6640 1 -- start start 2024-09-11T23:21:24.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.715+0000 7fd0009b6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcffc101c00 0x7fcffc198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:24.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.715+0000 7fd0009b6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcffc102e00 0x7fcffc1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:24.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.715+0000 7fd0009b6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcffc1037c0 0x7fcffc19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:24.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.715+0000 7fd0009b6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc10d9d0 con 0x7fcffc101c00 2024-09-11T23:21:24.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.715+0000 7fd0009b6640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc10d850 con 0x7fcffc102e00 2024-09-11T23:21:24.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.715+0000 7fd0009b6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc10db50 con 0x7fcffc1037c0 2024-09-11T23:21:24.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.715+0000 7fcffa59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcffc101c00 0x7fcffc198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:24.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.715+0000 7fcffa59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcffc101c00 0x7fcffc198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59096/0 (socket says 172.21.15.43:59096) 2024-09-11T23:21:24.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.715+0000 7fcffa59c640 1 -- 172.21.15.43:0/2014950757 learned_addr learned my addr 172.21.15.43:0/2014950757 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:24.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.715+0000 7fcff3fff640 1 --2- 172.21.15.43:0/2014950757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcffc102e00 0x7fcffc1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:24.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.715+0000 7fcffad9d640 1 --2- 172.21.15.43:0/2014950757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcffc1037c0 0x7fcffc19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:24.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.716+0000 7fcffa59c640 1 -- 172.21.15.43:0/2014950757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcffc1037c0 msgr2=0x7fcffc19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:24.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.716+0000 7fcffa59c640 1 --2- 172.21.15.43:0/2014950757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcffc1037c0 0x7fcffc19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:24.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.716+0000 7fcffa59c640 1 -- 172.21.15.43:0/2014950757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcffc102e00 msgr2=0x7fcffc1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:24.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.716+0000 7fcffa59c640 1 --2- 172.21.15.43:0/2014950757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcffc102e00 0x7fcffc1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:24.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.716+0000 7fcffa59c640 1 -- 172.21.15.43:0/2014950757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcffc19fcb0 con 0x7fcffc101c00 2024-09-11T23:21:24.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.716+0000 7fcffad9d640 1 --2- 172.21.15.43:0/2014950757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcffc1037c0 0x7fcffc19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:21:24.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.716+0000 7fcffa59c640 1 --2- 172.21.15.43:0/2014950757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcffc101c00 0x7fcffc198080 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00b450 tx=0x7fcfec00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:24.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.716+0000 7fcff37fe640 1 -- 172.21.15.43:0/2014950757 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec015020 con 0x7fcffc101c00 2024-09-11T23:21:24.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.716+0000 7fcff37fe640 1 -- 172.21.15.43:0/2014950757 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfec011d70 con 0x7fcffc101c00 2024-09-11T23:21:24.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.716+0000 7fd0009b6640 1 -- 172.21.15.43:0/2014950757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcffc19ffa0 con 0x7fcffc101c00 2024-09-11T23:21:24.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.716+0000 7fcff37fe640 1 -- 172.21.15.43:0/2014950757 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec00f340 con 0x7fcffc101c00 2024-09-11T23:21:24.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.716+0000 7fd0009b6640 1 -- 172.21.15.43:0/2014950757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcffc1a0470 con 0x7fcffc101c00 2024-09-11T23:21:24.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.717+0000 7fd0009b6640 1 -- 172.21.15.43:0/2014950757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcffc109050 con 0x7fcffc101c00 2024-09-11T23:21:24.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.718+0000 7fcff37fe640 1 -- 172.21.15.43:0/2014950757 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcfec018020 con 0x7fcffc101c00 2024-09-11T23:21:24.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.719+0000 7fcff37fe640 1 --2- 172.21.15.43:0/2014950757 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fcfe80a2c40 0x7fcfe80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:24.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.719+0000 7fcff3fff640 1 --2- 172.21.15.43:0/2014950757 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fcfe80a2c40 0x7fcfe80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:24.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.719+0000 7fcff37fe640 1 -- 172.21.15.43:0/2014950757 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7fcfec0ceaf0 con 0x7fcffc101c00 2024-09-11T23:21:24.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.720+0000 7fcff3fff640 1 --2- 172.21.15.43:0/2014950757 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fcfe80a2c40 0x7fcfe80a5100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4031440 tx=0x7fcfe40315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:24.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.724+0000 7fcff37fe640 1 -- 172.21.15.43:0/2014950757 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcfec01f020 con 0x7fcffc101c00 2024-09-11T23:21:24.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.908+0000 7fd0009b6640 1 -- 172.21.15.43:0/2014950757 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fcffc062630 con 0x7fcfe80a2c40 2024-09-11T23:21:24.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.922+0000 7fcff37fe640 1 -- 172.21.15.43:0/2014950757 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7fcffc062630 con 0x7fcfe80a2c40 2024-09-11T23:21:24.922 INFO:teuthology.orchestra.run.smithi043.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-11T23:21:24.922 INFO:teuthology.orchestra.run.smithi043.stdout:mds.a smithi043 running (98s) 18s ago 97s 16.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 0697bba84f7e 2024-09-11T23:21:24.922 INFO:teuthology.orchestra.run.smithi043.stdout:mds.b smithi090 running (95s) 27s ago 95s 26.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 de004e4ae010 2024-09-11T23:21:24.922 INFO:teuthology.orchestra.run.smithi043.stdout:mds.c smithi097 running (110s) 27s ago 109s 16.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 9eaae0bf1504 2024-09-11T23:21:24.922 INFO:teuthology.orchestra.run.smithi043.stdout:mds.d smithi043 running (118s) 18s ago 117s 13.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 03d0b56e736e 2024-09-11T23:21:24.923 INFO:teuthology.orchestra.run.smithi043.stdout:mds.e smithi090 running (115s) 27s ago 114s 17.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 53810d43153a 2024-09-11T23:21:24.923 INFO:teuthology.orchestra.run.smithi043.stdout:mds.f smithi097 running (112s) 27s ago 112s 17.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1c28dd39f10a 2024-09-11T23:21:24.923 INFO:teuthology.orchestra.run.smithi043.stdout:mds.g smithi043 running (102s) 18s ago 101s 15.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 e7f1f9ed7f3c 2024-09-11T23:21:24.923 INFO:teuthology.orchestra.run.smithi043.stdout:mds.h smithi090 running (107s) 27s ago 106s 18.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 a55012e7c451 2024-09-11T23:21:24.923 INFO:teuthology.orchestra.run.smithi043.stdout:mds.i smithi097 running (104s) 27s ago 104s 20.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 47fd2598a74d 2024-09-11T23:21:24.923 INFO:teuthology.orchestra.run.smithi043.stdout:mds.j smithi043 running (2m) 18s ago 2m 16.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 fa72d4c80cdc 2024-09-11T23:21:24.923 INFO:teuthology.orchestra.run.smithi043.stdout:mds.k smithi090 running (2m) 27s ago 2m 15.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 c753dd3ba9cf 2024-09-11T23:21:24.923 INFO:teuthology.orchestra.run.smithi043.stdout:mds.l smithi097 running (2m) 27s ago 2m 14.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 a21dd9902d79 2024-09-11T23:21:24.923 INFO:teuthology.orchestra.run.smithi043.stdout:mgr.x smithi043 *:9283,8765 running (9m) 18s ago 9m 508M - 19.1.1-298-gb53b6d3e 54467abc55f8 9dc9cd17074c 2024-09-11T23:21:24.923 INFO:teuthology.orchestra.run.smithi043.stdout:mgr.y smithi090 *:8443,8765 running (7m) 27s ago 7m 443M - 19.1.1-298-gb53b6d3e 54467abc55f8 98d6ff6839de 2024-09-11T23:21:24.923 INFO:teuthology.orchestra.run.smithi043.stdout:mgr.z smithi097 *:8443,8765 running (7m) 27s ago 7m 442M - 19.1.1-298-gb53b6d3e 54467abc55f8 907fd4394ca1 2024-09-11T23:21:24.923 INFO:teuthology.orchestra.run.smithi043.stdout:mon.a smithi043 running (9m) 18s ago 9m 50.6M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 176c0fba3df5 2024-09-11T23:21:24.923 INFO:teuthology.orchestra.run.smithi043.stdout:mon.b smithi090 running (7m) 27s ago 7m 39.7M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 172cb0b311f0 2024-09-11T23:21:24.924 INFO:teuthology.orchestra.run.smithi043.stdout:mon.c smithi097 running (7m) 27s ago 7m 40.2M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 fba50890eb51 2024-09-11T23:21:24.924 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0 smithi043 running (7m) 18s ago 7m 44.0M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 c2fca8036d73 2024-09-11T23:21:24.924 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1 smithi090 running (6m) 27s ago 6m 54.9M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 b9da261655af 2024-09-11T23:21:24.924 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2 smithi097 running (6m) 27s ago 6m 47.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 af41eb00c809 2024-09-11T23:21:24.924 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3 smithi043 running (5m) 18s ago 5m 57.5M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 26543efd1f9d 2024-09-11T23:21:24.924 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4 smithi090 running (5m) 27s ago 5m 52.3M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 b794ba6904af 2024-09-11T23:21:24.924 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5 smithi097 running (5m) 27s ago 5m 46.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 aea2317b7054 2024-09-11T23:21:24.924 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6 smithi043 running (4m) 18s ago 4m 41.6M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 4055caad1f2a 2024-09-11T23:21:24.924 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7 smithi090 running (4m) 27s ago 4m 54.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 30578d87addf 2024-09-11T23:21:24.924 INFO:teuthology.orchestra.run.smithi043.stdout:osd.8 smithi097 running (3m) 27s ago 3m 44.8M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 ebb786151e18 2024-09-11T23:21:24.924 INFO:teuthology.orchestra.run.smithi043.stdout:osd.9 smithi043 running (3m) 18s ago 3m 52.1M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 7dc9095d7a81 2024-09-11T23:21:24.924 INFO:teuthology.orchestra.run.smithi043.stdout:osd.10 smithi090 running (2m) 27s ago 2m 42.8M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 2d6ef181a656 2024-09-11T23:21:24.925 INFO:teuthology.orchestra.run.smithi043.stdout:osd.11 smithi097 running (2m) 27s ago 2m 38.8M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 cfd0a54a2a00 2024-09-11T23:21:24.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.924+0000 7fd0009b6640 1 -- 172.21.15.43:0/2014950757 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fcfe80a2c40 msgr2=0x7fcfe80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:24.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.924+0000 7fd0009b6640 1 --2- 172.21.15.43:0/2014950757 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fcfe80a2c40 0x7fcfe80a5100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4031440 tx=0x7fcfe40315b0 comp rx=0 tx=0).stop 2024-09-11T23:21:24.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.924+0000 7fd0009b6640 1 -- 172.21.15.43:0/2014950757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcffc101c00 msgr2=0x7fcffc198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:24.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.924+0000 7fd0009b6640 1 --2- 172.21.15.43:0/2014950757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcffc101c00 0x7fcffc198080 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00b450 tx=0x7fcfec00b910 comp rx=0 tx=0).stop 2024-09-11T23:21:24.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.924+0000 7fd0009b6640 1 -- 172.21.15.43:0/2014950757 shutdown_connections 2024-09-11T23:21:24.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.924+0000 7fd0009b6640 1 --2- 172.21.15.43:0/2014950757 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fcfe80a2c40 0x7fcfe80a5100 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:24.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.924+0000 7fd0009b6640 1 --2- 172.21.15.43:0/2014950757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcffc1037c0 0x7fcffc19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:24.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.924+0000 7fd0009b6640 1 --2- 172.21.15.43:0/2014950757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcffc102e00 0x7fcffc1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:24.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.924+0000 7fd0009b6640 1 --2- 172.21.15.43:0/2014950757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcffc101c00 0x7fcffc198080 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:24.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.924+0000 7fd0009b6640 1 -- 172.21.15.43:0/2014950757 >> 172.21.15.43:0/2014950757 conn(0x7fcffc0fd390 msgr2=0x7fcffc106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:24.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.924+0000 7fd0009b6640 1 -- 172.21.15.43:0/2014950757 shutdown_connections 2024-09-11T23:21:24.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:24.924+0000 7fd0009b6640 1 -- 172.21.15.43:0/2014950757 wait complete. 2024-09-11T23:21:25.590 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-11T23:21:25.855 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:26.207 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:25 smithi043 ceph-mon[30950]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-11T23:21:26.207 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:25 smithi043 ceph-mon[30950]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:21:26.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:25 smithi090 ceph-mon[35130]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-11T23:21:26.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:25 smithi090 ceph-mon[35130]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:21:26.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.291+0000 7f5d4a0b0640 1 -- 172.21.15.43:0/3252234924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d44107940 msgr2=0x7f5d44109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:26.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.291+0000 7f5d4a0b0640 1 --2- 172.21.15.43:0/3252234924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d44107940 0x7f5d44109d30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009ca0 tx=0x7f5d3c02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:21:26.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.292+0000 7f5d4a0b0640 1 -- 172.21.15.43:0/3252234924 shutdown_connections 2024-09-11T23:21:26.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.292+0000 7f5d4a0b0640 1 --2- 172.21.15.43:0/3252234924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d44107940 0x7f5d44109d30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:26.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.292+0000 7f5d4a0b0640 1 --2- 172.21.15.43:0/3252234924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d44105010 0x7f5d44107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:26.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.292+0000 7f5d4a0b0640 1 --2- 172.21.15.43:0/3252234924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d441026e0 0x7f5d44104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:26.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.292+0000 7f5d4a0b0640 1 -- 172.21.15.43:0/3252234924 >> 172.21.15.43:0/3252234924 conn(0x7f5d440fc280 msgr2=0x7f5d440fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:26.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.292+0000 7f5d4a0b0640 1 -- 172.21.15.43:0/3252234924 shutdown_connections 2024-09-11T23:21:26.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.292+0000 7f5d4a0b0640 1 -- 172.21.15.43:0/3252234924 wait complete. 2024-09-11T23:21:26.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.292+0000 7f5d4a0b0640 1 Processor -- start 2024-09-11T23:21:26.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.293+0000 7f5d4a0b0640 1 -- start start 2024-09-11T23:21:26.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.293+0000 7f5d4a0b0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d441026e0 0x7f5d44102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:26.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.293+0000 7f5d4a0b0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d44105010 0x7f5d44100690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:26.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.293+0000 7f5d4a0b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d44107940 0x7f5d44100bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:26.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.293+0000 7f5d4a0b0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4410d880 con 0x7f5d44107940 2024-09-11T23:21:26.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.293+0000 7f5d4a0b0640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4410d700 con 0x7f5d441026e0 2024-09-11T23:21:26.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.293+0000 7f5d4a0b0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4410da00 con 0x7f5d44105010 2024-09-11T23:21:26.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.293+0000 7f5d43fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d44107940 0x7f5d44100bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:26.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.293+0000 7f5d43fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d44107940 0x7f5d44100bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59112/0 (socket says 172.21.15.43:59112) 2024-09-11T23:21:26.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.293+0000 7f5d43fff640 1 -- 172.21.15.43:0/387199529 learned_addr learned my addr 172.21.15.43:0/387199529 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:26.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.293+0000 7f5d42ffd640 1 --2- 172.21.15.43:0/387199529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d44105010 0x7f5d44100690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:26.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.293+0000 7f5d437fe640 1 --2- 172.21.15.43:0/387199529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d441026e0 0x7f5d44102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:26.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.293+0000 7f5d43fff640 1 -- 172.21.15.43:0/387199529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d44105010 msgr2=0x7f5d44100690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:26.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.293+0000 7f5d43fff640 1 --2- 172.21.15.43:0/387199529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d44105010 0x7f5d44100690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:26.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.293+0000 7f5d43fff640 1 -- 172.21.15.43:0/387199529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d441026e0 msgr2=0x7f5d44102040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:26.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.293+0000 7f5d43fff640 1 --2- 172.21.15.43:0/387199529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d441026e0 0x7f5d44102040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:26.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.293+0000 7f5d43fff640 1 -- 172.21.15.43:0/387199529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d44101490 con 0x7f5d44107940 2024-09-11T23:21:26.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.293+0000 7f5d437fe640 1 --2- 172.21.15.43:0/387199529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d441026e0 0x7f5d44102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:21:26.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.294+0000 7f5d43fff640 1 --2- 172.21.15.43:0/387199529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d44107940 0x7f5d44100bd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c005e30 tx=0x7f5d3c031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:26.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.294+0000 7f5d40ff9640 1 -- 172.21.15.43:0/387199529 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c003c30 con 0x7f5d44107940 2024-09-11T23:21:26.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.294+0000 7f5d4a0b0640 1 -- 172.21.15.43:0/387199529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d441a2000 con 0x7f5d44107940 2024-09-11T23:21:26.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.294+0000 7f5d40ff9640 1 -- 172.21.15.43:0/387199529 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d3c003dd0 con 0x7f5d44107940 2024-09-11T23:21:26.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.294+0000 7f5d40ff9640 1 -- 172.21.15.43:0/387199529 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c02fde0 con 0x7f5d44107940 2024-09-11T23:21:26.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.294+0000 7f5d4a0b0640 1 -- 172.21.15.43:0/387199529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d441a2560 con 0x7f5d44107940 2024-09-11T23:21:26.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.295+0000 7f5d4a0b0640 1 -- 172.21.15.43:0/387199529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d441053f0 con 0x7f5d44107940 2024-09-11T23:21:26.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.296+0000 7f5d40ff9640 1 -- 172.21.15.43:0/387199529 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5d3c004050 con 0x7f5d44107940 2024-09-11T23:21:26.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.297+0000 7f5d40ff9640 1 --2- 172.21.15.43:0/387199529 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f5d300a2c10 0x7f5d300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:26.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.298+0000 7f5d437fe640 1 --2- 172.21.15.43:0/387199529 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f5d300a2c10 0x7f5d300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:26.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.298+0000 7f5d40ff9640 1 -- 172.21.15.43:0/387199529 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f5d3c0f04f0 con 0x7f5d44107940 2024-09-11T23:21:26.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.298+0000 7f5d437fe640 1 --2- 172.21.15.43:0/387199529 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f5d300a2c10 0x7f5d300a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5d34005d00 tx=0x7f5d34005ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:26.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:25 smithi097 ceph-mon[35562]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-11T23:21:26.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:25 smithi097 ceph-mon[35562]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:21:26.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.301+0000 7f5d40ff9640 1 -- 172.21.15.43:0/387199529 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5d3c0bd380 con 0x7f5d44107940 2024-09-11T23:21:26.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.511+0000 7f5d4a0b0640 1 -- 172.21.15.43:0/387199529 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f5d44062630 con 0x7f5d300a2c10 2024-09-11T23:21:26.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.517+0000 7f5d40ff9640 1 -- 172.21.15.43:0/387199529 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f5d44062630 con 0x7f5d300a2c10 2024-09-11T23:21:26.517 INFO:teuthology.orchestra.run.smithi043.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-11T23:21:26.517 INFO:teuthology.orchestra.run.smithi043.stdout:mds.all 12/12 28s ago 2m smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi090=b;smithi090=e;smithi090=h;smithi090=k;smithi097=c;smithi097=f;smithi097=i;smithi097=l;count:12 2024-09-11T23:21:26.517 INFO:teuthology.orchestra.run.smithi043.stdout:mgr 3/3 28s ago 7m smithi043=x;smithi090=y;smithi097=z;count:3 2024-09-11T23:21:26.517 INFO:teuthology.orchestra.run.smithi043.stdout:mon 3/3 28s ago 7m smithi043:172.21.15.43=a;smithi090:172.21.15.90=b;smithi097:172.21.15.97=c;count:3 2024-09-11T23:21:26.517 INFO:teuthology.orchestra.run.smithi043.stdout:osd 12 28s ago - 2024-09-11T23:21:26.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.519+0000 7f5d4a0b0640 1 -- 172.21.15.43:0/387199529 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f5d300a2c10 msgr2=0x7f5d300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:26.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.519+0000 7f5d4a0b0640 1 --2- 172.21.15.43:0/387199529 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f5d300a2c10 0x7f5d300a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5d34005d00 tx=0x7f5d34005ed0 comp rx=0 tx=0).stop 2024-09-11T23:21:26.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.519+0000 7f5d4a0b0640 1 -- 172.21.15.43:0/387199529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d44107940 msgr2=0x7f5d44100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:26.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.519+0000 7f5d4a0b0640 1 --2- 172.21.15.43:0/387199529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d44107940 0x7f5d44100bd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c005e30 tx=0x7f5d3c031ee0 comp rx=0 tx=0).stop 2024-09-11T23:21:26.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.519+0000 7f5d4a0b0640 1 -- 172.21.15.43:0/387199529 shutdown_connections 2024-09-11T23:21:26.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.519+0000 7f5d4a0b0640 1 --2- 172.21.15.43:0/387199529 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f5d300a2c10 0x7f5d300a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:26.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.519+0000 7f5d4a0b0640 1 --2- 172.21.15.43:0/387199529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d44107940 0x7f5d44100bd0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:26.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.519+0000 7f5d4a0b0640 1 --2- 172.21.15.43:0/387199529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d44105010 0x7f5d44100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:26.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.519+0000 7f5d4a0b0640 1 --2- 172.21.15.43:0/387199529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d441026e0 0x7f5d44102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:26.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.519+0000 7f5d4a0b0640 1 -- 172.21.15.43:0/387199529 >> 172.21.15.43:0/387199529 conn(0x7f5d440fc280 msgr2=0x7f5d441030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:26.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.520+0000 7f5d4a0b0640 1 -- 172.21.15.43:0/387199529 shutdown_connections 2024-09-11T23:21:26.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:26.520+0000 7f5d4a0b0640 1 -- 172.21.15.43:0/387199529 wait complete. 2024-09-11T23:21:27.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:26 smithi043 ceph-mon[30950]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:21:27.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-11T23:21:27.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:26 smithi090 ceph-mon[35130]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:21:27.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:26 smithi097 ceph-mon[35562]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:21:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.868+0000 7ffbd26c1640 1 -- 172.21.15.43:0/1895733081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbcc101cd0 msgr2=0x7ffbcc102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.868+0000 7ffbd26c1640 1 --2- 172.21.15.43:0/1895733081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbcc101cd0 0x7ffbcc102150 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009900 tx=0x7ffbc002a410 comp rx=0 tx=0).stop 2024-09-11T23:21:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.868+0000 7ffbd26c1640 1 -- 172.21.15.43:0/1895733081 shutdown_connections 2024-09-11T23:21:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.868+0000 7ffbd26c1640 1 --2- 172.21.15.43:0/1895733081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbcc102690 0x7ffbcc108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.868+0000 7ffbd26c1640 1 --2- 172.21.15.43:0/1895733081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbcc101cd0 0x7ffbcc102150 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.868+0000 7ffbd26c1640 1 --2- 172.21.15.43:0/1895733081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbcc100ad0 0x7ffbcc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.868+0000 7ffbd26c1640 1 -- 172.21.15.43:0/1895733081 >> 172.21.15.43:0/1895733081 conn(0x7ffbcc0fc280 msgr2=0x7ffbcc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.868+0000 7ffbd26c1640 1 -- 172.21.15.43:0/1895733081 shutdown_connections 2024-09-11T23:21:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.868+0000 7ffbd26c1640 1 -- 172.21.15.43:0/1895733081 wait complete. 2024-09-11T23:21:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.869+0000 7ffbd26c1640 1 Processor -- start 2024-09-11T23:21:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.869+0000 7ffbd26c1640 1 -- start start 2024-09-11T23:21:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.869+0000 7ffbd26c1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbcc100ad0 0x7ffbcc197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.869+0000 7ffbd26c1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbcc101cd0 0x7ffbcc1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.869+0000 7ffbd26c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbcc102690 0x7ffbcc19f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.869+0000 7ffbd26c1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc10d880 con 0x7ffbcc102690 2024-09-11T23:21:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.869+0000 7ffbd26c1640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc10d700 con 0x7ffbcc100ad0 2024-09-11T23:21:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.869+0000 7ffbd26c1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc10da00 con 0x7ffbcc101cd0 2024-09-11T23:21:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.869+0000 7ffbd0c5e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbcc102690 0x7ffbcc19f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.869+0000 7ffbd0c5e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbcc102690 0x7ffbcc19f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59142/0 (socket says 172.21.15.43:59142) 2024-09-11T23:21:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.869+0000 7ffbd0c5e640 1 -- 172.21.15.43:0/1951493130 learned_addr learned my addr 172.21.15.43:0/1951493130 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.869+0000 7ffbcbfff640 1 --2- 172.21.15.43:0/1951493130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbcc100ad0 0x7ffbcc197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.869+0000 7ffbcb7fe640 1 --2- 172.21.15.43:0/1951493130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbcc101cd0 0x7ffbcc1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.869+0000 7ffbd0c5e640 1 -- 172.21.15.43:0/1951493130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbcc101cd0 msgr2=0x7ffbcc1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.869+0000 7ffbd0c5e640 1 --2- 172.21.15.43:0/1951493130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbcc101cd0 0x7ffbcc1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.869+0000 7ffbd0c5e640 1 -- 172.21.15.43:0/1951493130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbcc100ad0 msgr2=0x7ffbcc197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.869+0000 7ffbd0c5e640 1 --2- 172.21.15.43:0/1951493130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbcc100ad0 0x7ffbcc197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.869+0000 7ffbd0c5e640 1 -- 172.21.15.43:0/1951493130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbcc19fab0 con 0x7ffbcc102690 2024-09-11T23:21:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.869+0000 7ffbcb7fe640 1 --2- 172.21.15.43:0/1951493130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbcc101cd0 0x7ffbcc1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:21:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.870+0000 7ffbd0c5e640 1 --2- 172.21.15.43:0/1951493130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbcc102690 0x7ffbcc19f440 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ffbc400cce0 tx=0x7ffbc400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.870+0000 7ffbc97fa640 1 -- 172.21.15.43:0/1951493130 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc400f450 con 0x7ffbcc102690 2024-09-11T23:21:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.870+0000 7ffbd26c1640 1 -- 172.21.15.43:0/1951493130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbcc19fda0 con 0x7ffbcc102690 2024-09-11T23:21:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.870+0000 7ffbc97fa640 1 -- 172.21.15.43:0/1951493130 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc400fad0 con 0x7ffbcc102690 2024-09-11T23:21:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.870+0000 7ffbc97fa640 1 -- 172.21.15.43:0/1951493130 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4018c10 con 0x7ffbcc102690 2024-09-11T23:21:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.870+0000 7ffbd26c1640 1 -- 172.21.15.43:0/1951493130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbcc1a0200 con 0x7ffbcc102690 2024-09-11T23:21:27.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.871+0000 7ffbd26c1640 1 -- 172.21.15.43:0/1951493130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbcc107e70 con 0x7ffbcc102690 2024-09-11T23:21:27.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.872+0000 7ffbc97fa640 1 -- 172.21.15.43:0/1951493130 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ffbc400f5f0 con 0x7ffbcc102690 2024-09-11T23:21:27.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.873+0000 7ffbc97fa640 1 --2- 172.21.15.43:0/1951493130 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ffbb40a2c40 0x7ffbb40a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:27.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.873+0000 7ffbcbfff640 1 --2- 172.21.15.43:0/1951493130 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ffbb40a2c40 0x7ffbb40a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:27.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.873+0000 7ffbc97fa640 1 -- 172.21.15.43:0/1951493130 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7ffbc40cf8f0 con 0x7ffbcc102690 2024-09-11T23:21:27.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.873+0000 7ffbcbfff640 1 --2- 172.21.15.43:0/1951493130 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ffbb40a2c40 0x7ffbb40a5100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc101b30 tx=0x7ffbbc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:27.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:27.877+0000 7ffbc97fa640 1 -- 172.21.15.43:0/1951493130 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffbc4014030 con 0x7ffbcc102690 2024-09-11T23:21:28.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:28.061+0000 7ffbd26c1640 1 -- 172.21.15.43:0/1951493130 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7ffbcc062630 con 0x7ffbb40a2c40 2024-09-11T23:21:28.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:28.062+0000 7ffbc97fa640 1 -- 172.21.15.43:0/1951493130 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7ffbcc062630 con 0x7ffbb40a2c40 2024-09-11T23:21:28.062 INFO:teuthology.orchestra.run.smithi043.stdout:HOST ADDR LABELS STATUS 2024-09-11T23:21:28.062 INFO:teuthology.orchestra.run.smithi043.stdout:smithi043 172.21.15.43 2024-09-11T23:21:28.062 INFO:teuthology.orchestra.run.smithi043.stdout:smithi090 172.21.15.90 2024-09-11T23:21:28.062 INFO:teuthology.orchestra.run.smithi043.stdout:smithi097 172.21.15.97 2024-09-11T23:21:28.062 INFO:teuthology.orchestra.run.smithi043.stdout:3 hosts in cluster 2024-09-11T23:21:28.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:28.064+0000 7ffbd26c1640 1 -- 172.21.15.43:0/1951493130 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ffbb40a2c40 msgr2=0x7ffbb40a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:28.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:28.064+0000 7ffbd26c1640 1 --2- 172.21.15.43:0/1951493130 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ffbb40a2c40 0x7ffbb40a5100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc101b30 tx=0x7ffbbc00b390 comp rx=0 tx=0).stop 2024-09-11T23:21:28.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:28.064+0000 7ffbd26c1640 1 -- 172.21.15.43:0/1951493130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbcc102690 msgr2=0x7ffbcc19f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:28.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:28.064+0000 7ffbd26c1640 1 --2- 172.21.15.43:0/1951493130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbcc102690 0x7ffbcc19f440 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ffbc400cce0 tx=0x7ffbc400a620 comp rx=0 tx=0).stop 2024-09-11T23:21:28.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:28.064+0000 7ffbd26c1640 1 -- 172.21.15.43:0/1951493130 shutdown_connections 2024-09-11T23:21:28.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:28.064+0000 7ffbd26c1640 1 --2- 172.21.15.43:0/1951493130 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7ffbb40a2c40 0x7ffbb40a5100 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:28.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:28.064+0000 7ffbd26c1640 1 --2- 172.21.15.43:0/1951493130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbcc102690 0x7ffbcc19f440 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:28.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:28.064+0000 7ffbd26c1640 1 --2- 172.21.15.43:0/1951493130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbcc101cd0 0x7ffbcc1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:28.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:28.064+0000 7ffbd26c1640 1 --2- 172.21.15.43:0/1951493130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbcc100ad0 0x7ffbcc197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:28.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:28.064+0000 7ffbd26c1640 1 -- 172.21.15.43:0/1951493130 >> 172.21.15.43:0/1951493130 conn(0x7ffbcc0fc280 msgr2=0x7ffbcc105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:28.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:28.064+0000 7ffbd26c1640 1 -- 172.21.15.43:0/1951493130 shutdown_connections 2024-09-11T23:21:28.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:28.064+0000 7ffbd26c1640 1 -- 172.21.15.43:0/1951493130 wait complete. 2024-09-11T23:21:28.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:27 smithi090 ceph-mon[35130]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:21:28.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:27 smithi097 ceph-mon[35562]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:21:28.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:27 smithi043 ceph-mon[30950]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:21:28.790 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-11T23:21:29.050 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:29.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:28 smithi043 ceph-mon[30950]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:21:29.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:28 smithi090 ceph-mon[35130]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:21:29.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:28 smithi097 ceph-mon[35562]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:21:29.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.529+0000 7fb7e081d640 1 -- 172.21.15.43:0/1480268060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7dc105090 msgr2=0x7fb7dc107480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:29.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.529+0000 7fb7e081d640 1 --2- 172.21.15.43:0/1480268060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7dc105090 0x7fb7dc107480 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4009900 tx=0x7fb7c402a410 comp rx=0 tx=0).stop 2024-09-11T23:21:29.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.529+0000 7fb7e081d640 1 -- 172.21.15.43:0/1480268060 shutdown_connections 2024-09-11T23:21:29.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.529+0000 7fb7e081d640 1 --2- 172.21.15.43:0/1480268060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7dc1079c0 0x7fb7dc109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:29.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.529+0000 7fb7e081d640 1 --2- 172.21.15.43:0/1480268060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7dc105090 0x7fb7dc107480 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:29.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.529+0000 7fb7e081d640 1 --2- 172.21.15.43:0/1480268060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7dc102760 0x7fb7dc104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:29.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.529+0000 7fb7e081d640 1 -- 172.21.15.43:0/1480268060 >> 172.21.15.43:0/1480268060 conn(0x7fb7dc0fc2a0 msgr2=0x7fb7dc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:29.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.529+0000 7fb7e081d640 1 -- 172.21.15.43:0/1480268060 shutdown_connections 2024-09-11T23:21:29.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.529+0000 7fb7e081d640 1 -- 172.21.15.43:0/1480268060 wait complete. 2024-09-11T23:21:29.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.530+0000 7fb7e081d640 1 Processor -- start 2024-09-11T23:21:29.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.530+0000 7fb7e081d640 1 -- start start 2024-09-11T23:21:29.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7e081d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7dc102760 0x7fb7dc197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:29.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7e081d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7dc105090 0x7fb7dc198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:29.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7e081d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7dc1079c0 0x7fb7dc19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:29.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7e081d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc10d8a0 con 0x7fb7dc105090 2024-09-11T23:21:29.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7e081d640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc10d720 con 0x7fb7dc102760 2024-09-11T23:21:29.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7e081d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc10da20 con 0x7fb7dc1079c0 2024-09-11T23:21:29.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7d9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7dc105090 0x7fb7dc198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:29.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7d9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7dc105090 0x7fb7dc198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59156/0 (socket says 172.21.15.43:59156) 2024-09-11T23:21:29.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7d9d9b640 1 -- 172.21.15.43:0/4218007057 learned_addr learned my addr 172.21.15.43:0/4218007057 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:29.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7dad9d640 1 --2- 172.21.15.43:0/4218007057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7dc1079c0 0x7fb7dc19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:29.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7da59c640 1 --2- 172.21.15.43:0/4218007057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7dc102760 0x7fb7dc197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:29.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7d9d9b640 1 -- 172.21.15.43:0/4218007057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7dc1079c0 msgr2=0x7fb7dc19f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:29.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7d9d9b640 1 --2- 172.21.15.43:0/4218007057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7dc1079c0 0x7fb7dc19f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:29.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7d9d9b640 1 -- 172.21.15.43:0/4218007057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7dc102760 msgr2=0x7fb7dc197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:29.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7d9d9b640 1 --2- 172.21.15.43:0/4218007057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7dc102760 0x7fb7dc197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:29.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7d9d9b640 1 -- 172.21.15.43:0/4218007057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7dc19fb50 con 0x7fb7dc105090 2024-09-11T23:21:29.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7d9d9b640 1 --2- 172.21.15.43:0/4218007057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7dc105090 0x7fb7dc198460 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4031610 tx=0x7fb7c40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:29.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7d37fe640 1 -- 172.21.15.43:0/4218007057 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c403f030 con 0x7fb7dc105090 2024-09-11T23:21:29.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7e081d640 1 -- 172.21.15.43:0/4218007057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7dc19fde0 con 0x7fb7dc105090 2024-09-11T23:21:29.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7d37fe640 1 -- 172.21.15.43:0/4218007057 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7c4003ce0 con 0x7fb7dc105090 2024-09-11T23:21:29.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7d37fe640 1 -- 172.21.15.43:0/4218007057 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4035070 con 0x7fb7dc105090 2024-09-11T23:21:29.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.531+0000 7fb7e081d640 1 -- 172.21.15.43:0/4218007057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7dc1a0240 con 0x7fb7dc105090 2024-09-11T23:21:29.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.533+0000 7fb7e081d640 1 -- 172.21.15.43:0/4218007057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7dc0fdd90 con 0x7fb7dc105090 2024-09-11T23:21:29.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.534+0000 7fb7d37fe640 1 -- 172.21.15.43:0/4218007057 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb7c4042030 con 0x7fb7dc105090 2024-09-11T23:21:29.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.535+0000 7fb7d37fe640 1 --2- 172.21.15.43:0/4218007057 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb7c80a2c40 0x7fb7c80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:29.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.535+0000 7fb7da59c640 1 --2- 172.21.15.43:0/4218007057 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb7c80a2c40 0x7fb7c80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:29.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.535+0000 7fb7d37fe640 1 -- 172.21.15.43:0/4218007057 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7fb7c40efa00 con 0x7fb7dc105090 2024-09-11T23:21:29.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.535+0000 7fb7da59c640 1 --2- 172.21.15.43:0/4218007057 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb7c80a2c40 0x7fb7c80a5100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc005fd0 tx=0x7fb7cc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:29.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.538+0000 7fb7d37fe640 1 -- 172.21.15.43:0/4218007057 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb7c40bc7e0 con 0x7fb7dc105090 2024-09-11T23:21:29.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.721+0000 7fb7e081d640 1 -- 172.21.15.43:0/4218007057 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fb7dc062630 con 0x7fb7c80a2c40 2024-09-11T23:21:29.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.723+0000 7fb7d37fe640 1 -- 172.21.15.43:0/4218007057 <== mgr.14150 v2:172.21.15.43:6800/2370495020 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fb7dc062630 con 0x7fb7c80a2c40 2024-09-11T23:21:29.723 INFO:teuthology.orchestra.run.smithi043.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-11T23:21:29.723 INFO:teuthology.orchestra.run.smithi043.stdout:smithi043 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300GH480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-11T23:21:29.723 INFO:teuthology.orchestra.run.smithi043.stdout:smithi090 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UK400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-11T23:21:29.723 INFO:teuthology.orchestra.run.smithi043.stdout:smithi097 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013T400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-11T23:21:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.726+0000 7fb7e081d640 1 -- 172.21.15.43:0/4218007057 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb7c80a2c40 msgr2=0x7fb7c80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.726+0000 7fb7e081d640 1 --2- 172.21.15.43:0/4218007057 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb7c80a2c40 0x7fb7c80a5100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc005fd0 tx=0x7fb7cc005850 comp rx=0 tx=0).stop 2024-09-11T23:21:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.726+0000 7fb7e081d640 1 -- 172.21.15.43:0/4218007057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7dc105090 msgr2=0x7fb7dc198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.726+0000 7fb7e081d640 1 --2- 172.21.15.43:0/4218007057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7dc105090 0x7fb7dc198460 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4031610 tx=0x7fb7c40043b0 comp rx=0 tx=0).stop 2024-09-11T23:21:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.726+0000 7fb7e081d640 1 -- 172.21.15.43:0/4218007057 shutdown_connections 2024-09-11T23:21:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.726+0000 7fb7e081d640 1 --2- 172.21.15.43:0/4218007057 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb7c80a2c40 0x7fb7c80a5100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.726+0000 7fb7e081d640 1 --2- 172.21.15.43:0/4218007057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7dc1079c0 0x7fb7dc19f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.726+0000 7fb7e081d640 1 --2- 172.21.15.43:0/4218007057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7dc105090 0x7fb7dc198460 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.726+0000 7fb7e081d640 1 --2- 172.21.15.43:0/4218007057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7dc102760 0x7fb7dc197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.726+0000 7fb7e081d640 1 -- 172.21.15.43:0/4218007057 >> 172.21.15.43:0/4218007057 conn(0x7fb7dc0fc2a0 msgr2=0x7fb7dc1030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.726+0000 7fb7e081d640 1 -- 172.21.15.43:0/4218007057 shutdown_connections 2024-09-11T23:21:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:29.726+0000 7fb7e081d640 1 -- 172.21.15.43:0/4218007057 wait complete. 2024-09-11T23:21:30.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:29 smithi043 ceph-mon[30950]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T23:21:30.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:29 smithi090 ceph-mon[35130]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T23:21:30.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:29 smithi097 ceph-mon[35562]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T23:21:30.581 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T23:21:30.590 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-09-11T23:21:30.590 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-11T23:21:30.871 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:31.229 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:30 smithi043 ceph-mon[30950]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:21:31.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:30 smithi090 ceph-mon[35130]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:21:31.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:30 smithi097 ceph-mon[35562]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:21:31.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.310+0000 7fd1233e0640 1 -- 172.21.15.43:0/3287168961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c102690 msgr2=0x7fd11c108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:31.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.310+0000 7fd1233e0640 1 --2- 172.21.15.43:0/3287168961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c102690 0x7fd11c108f20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd114009ca0 tx=0x7fd11402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.310+0000 7fd1233e0640 1 -- 172.21.15.43:0/3287168961 shutdown_connections 2024-09-11T23:21:31.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.310+0000 7fd1233e0640 1 --2- 172.21.15.43:0/3287168961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c102690 0x7fd11c108f20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.310+0000 7fd1233e0640 1 --2- 172.21.15.43:0/3287168961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd11c101cd0 0x7fd11c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.310+0000 7fd1233e0640 1 --2- 172.21.15.43:0/3287168961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd11c100ad0 0x7fd11c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.310+0000 7fd1233e0640 1 -- 172.21.15.43:0/3287168961 >> 172.21.15.43:0/3287168961 conn(0x7fd11c0fc280 msgr2=0x7fd11c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:31.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.310+0000 7fd1233e0640 1 -- 172.21.15.43:0/3287168961 shutdown_connections 2024-09-11T23:21:31.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.310+0000 7fd1233e0640 1 -- 172.21.15.43:0/3287168961 wait complete. 2024-09-11T23:21:31.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.311+0000 7fd1233e0640 1 Processor -- start 2024-09-11T23:21:31.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.311+0000 7fd1233e0640 1 -- start start 2024-09-11T23:21:31.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.311+0000 7fd1233e0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd11c100ad0 0x7fd11c104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.311+0000 7fd1233e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c101cd0 0x7fd11c1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.311+0000 7fd1233e0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd11c102690 0x7fd11c1035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.311+0000 7fd1233e0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c10c750 con 0x7fd11c101cd0 2024-09-11T23:21:31.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.311+0000 7fd1233e0640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c10c5d0 con 0x7fd11c100ad0 2024-09-11T23:21:31.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.311+0000 7fd1233e0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c10c8d0 con 0x7fd11c102690 2024-09-11T23:21:31.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.311+0000 7fd12097b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c101cd0 0x7fd11c1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:31.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.311+0000 7fd12097b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c101cd0 0x7fd11c1030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46418/0 (socket says 172.21.15.43:46418) 2024-09-11T23:21:31.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.311+0000 7fd12097b640 1 -- 172.21.15.43:0/804677101 learned_addr learned my addr 172.21.15.43:0/804677101 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:31.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.311+0000 7fd12197d640 1 --2- 172.21.15.43:0/804677101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd11c102690 0x7fd11c1035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:31.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.311+0000 7fd12117c640 1 --2- 172.21.15.43:0/804677101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd11c100ad0 0x7fd11c104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:31.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.311+0000 7fd12097b640 1 -- 172.21.15.43:0/804677101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd11c102690 msgr2=0x7fd11c1035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:31.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.311+0000 7fd12097b640 1 --2- 172.21.15.43:0/804677101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd11c102690 0x7fd11c1035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.311+0000 7fd12097b640 1 -- 172.21.15.43:0/804677101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd11c100ad0 msgr2=0x7fd11c104a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:31.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.311+0000 7fd12097b640 1 --2- 172.21.15.43:0/804677101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd11c100ad0 0x7fd11c104a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.311+0000 7fd12097b640 1 -- 172.21.15.43:0/804677101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd11c103eb0 con 0x7fd11c101cd0 2024-09-11T23:21:31.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.311+0000 7fd12117c640 1 --2- 172.21.15.43:0/804677101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd11c100ad0 0x7fd11c104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:31.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.311+0000 7fd12097b640 1 --2- 172.21.15.43:0/804677101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c101cd0 0x7fd11c1030b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd11800d950 tx=0x7fd11800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:31.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.312+0000 7fd1127fc640 1 -- 172.21.15.43:0/804677101 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118017030 con 0x7fd11c101cd0 2024-09-11T23:21:31.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.312+0000 7fd1127fc640 1 -- 172.21.15.43:0/804677101 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd118004020 con 0x7fd11c101cd0 2024-09-11T23:21:31.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.312+0000 7fd1233e0640 1 -- 172.21.15.43:0/804677101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd11c1a1ff0 con 0x7fd11c101cd0 2024-09-11T23:21:31.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.312+0000 7fd1127fc640 1 -- 172.21.15.43:0/804677101 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1180047a0 con 0x7fd11c101cd0 2024-09-11T23:21:31.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.312+0000 7fd1233e0640 1 -- 172.21.15.43:0/804677101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd11c1a24f0 con 0x7fd11c101cd0 2024-09-11T23:21:31.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.313+0000 7fd1233e0640 1 -- 172.21.15.43:0/804677101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd11c100ed0 con 0x7fd11c101cd0 2024-09-11T23:21:31.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.314+0000 7fd1127fc640 1 -- 172.21.15.43:0/804677101 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd1180041d0 con 0x7fd11c101cd0 2024-09-11T23:21:31.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.314+0000 7fd1127fc640 1 --2- 172.21.15.43:0/804677101 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd1080a2c00 0x7fd1080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:31.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.315+0000 7fd12117c640 1 --2- 172.21.15.43:0/804677101 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd1080a2c00 0x7fd1080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:31.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.315+0000 7fd1127fc640 1 -- 172.21.15.43:0/804677101 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7fd1180ce9a0 con 0x7fd11c101cd0 2024-09-11T23:21:31.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.315+0000 7fd12117c640 1 --2- 172.21.15.43:0/804677101 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd1080a2c00 0x7fd1080a50c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd10c0058c0 tx=0x7fd10c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:31.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.319+0000 7fd1127fc640 1 -- 172.21.15.43:0/804677101 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd11809b8e0 con 0x7fd11c101cd0 2024-09-11T23:21:31.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.532+0000 7fd1233e0640 1 -- 172.21.15.43:0/804677101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fd11c1a2f10 con 0x7fd11c101cd0 2024-09-11T23:21:31.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.532+0000 7fd1127fc640 1 -- 172.21.15.43:0/804677101 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) ==== 76+0+3076 (secure 0 0 0) 0x7fd1180b9e20 con 0x7fd11c101cd0 2024-09-11T23:21:31.534 INFO:teuthology.orchestra.run.smithi043.stdout:e23 2024-09-11T23:21:31.534 INFO:teuthology.orchestra.run.smithi043.stdout:btime 2024-09-11T23:20:12:774166+0000 2024-09-11T23:21:31.534 INFO:teuthology.orchestra.run.smithi043.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T23:21:31.534 INFO:teuthology.orchestra.run.smithi043.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T23:21:31.534 INFO:teuthology.orchestra.run.smithi043.stdout:legacy client fscid: 1 2024-09-11T23:21:31.534 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:21:31.534 INFO:teuthology.orchestra.run.smithi043.stdout:Filesystem 'cephfs' (1) 2024-09-11T23:21:31.534 INFO:teuthology.orchestra.run.smithi043.stdout:fs_name cephfs 2024-09-11T23:21:31.534 INFO:teuthology.orchestra.run.smithi043.stdout:epoch 23 2024-09-11T23:21:31.535 INFO:teuthology.orchestra.run.smithi043.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-11T23:21:31.535 INFO:teuthology.orchestra.run.smithi043.stdout:created 2024-09-11T23:20:00.465830+0000 2024-09-11T23:21:31.535 INFO:teuthology.orchestra.run.smithi043.stdout:modified 2024-09-11T23:20:12.440327+0000 2024-09-11T23:21:31.535 INFO:teuthology.orchestra.run.smithi043.stdout:tableserver 0 2024-09-11T23:21:31.535 INFO:teuthology.orchestra.run.smithi043.stdout:root 0 2024-09-11T23:21:31.535 INFO:teuthology.orchestra.run.smithi043.stdout:session_timeout 300 2024-09-11T23:21:31.535 INFO:teuthology.orchestra.run.smithi043.stdout:session_autoclose 300 2024-09-11T23:21:31.535 INFO:teuthology.orchestra.run.smithi043.stdout:max_file_size 1099511627776 2024-09-11T23:21:31.535 INFO:teuthology.orchestra.run.smithi043.stdout:max_xattr_size 65536 2024-09-11T23:21:31.536 INFO:teuthology.orchestra.run.smithi043.stdout:required_client_features {} 2024-09-11T23:21:31.536 INFO:teuthology.orchestra.run.smithi043.stdout:last_failure 0 2024-09-11T23:21:31.536 INFO:teuthology.orchestra.run.smithi043.stdout:last_failure_osd_epoch 0 2024-09-11T23:21:31.536 INFO:teuthology.orchestra.run.smithi043.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T23:21:31.536 INFO:teuthology.orchestra.run.smithi043.stdout:max_mds 3 2024-09-11T23:21:31.536 INFO:teuthology.orchestra.run.smithi043.stdout:in 0,1,2 2024-09-11T23:21:31.536 INFO:teuthology.orchestra.run.smithi043.stdout:up {0=24497,1=24470,2=24457} 2024-09-11T23:21:31.536 INFO:teuthology.orchestra.run.smithi043.stdout:failed 2024-09-11T23:21:31.536 INFO:teuthology.orchestra.run.smithi043.stdout:damaged 2024-09-11T23:21:31.536 INFO:teuthology.orchestra.run.smithi043.stdout:stopped 2024-09-11T23:21:31.537 INFO:teuthology.orchestra.run.smithi043.stdout:data_pools [3] 2024-09-11T23:21:31.537 INFO:teuthology.orchestra.run.smithi043.stdout:metadata_pool 2 2024-09-11T23:21:31.537 INFO:teuthology.orchestra.run.smithi043.stdout:inline_data disabled 2024-09-11T23:21:31.537 INFO:teuthology.orchestra.run.smithi043.stdout:balancer 2024-09-11T23:21:31.537 INFO:teuthology.orchestra.run.smithi043.stdout:bal_rank_mask -1 2024-09-11T23:21:31.537 INFO:teuthology.orchestra.run.smithi043.stdout:standby_count_wanted 0 2024-09-11T23:21:31.537 INFO:teuthology.orchestra.run.smithi043.stdout:qdb_cluster leader: 24497 members: 24457,24470,24497 2024-09-11T23:21:31.537 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.b{0:24497} state up:active seq 4 addr [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:21:31.537 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.e{0:24449} state up:standby-replay seq 1 addr [v2:172.21.15.90:6834/4233140081,v1:172.21.15.90:6835/4233140081] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:21:31.537 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.h{1:24470} state up:active seq 9 addr [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:21:31.537 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.c{1:24442} state up:standby-replay seq 1 addr [v2:172.21.15.97:6836/191447096,v1:172.21.15.97:6837/191447096] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:21:31.537 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.i{2:24457} state up:active seq 9 addr [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:21:31.538 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.f{2:24436} state up:standby-replay seq 1 addr [v2:172.21.15.97:6834/1640511441,v1:172.21.15.97:6835/1640511441] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:21:31.538 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:21:31.538 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:21:31.538 INFO:teuthology.orchestra.run.smithi043.stdout:Standby daemons: 2024-09-11T23:21:31.538 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:21:31.538 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.j{-1:14526} state up:standby seq 1 addr [v2:172.21.15.43:6834/108369845,v1:172.21.15.43:6835/108369845] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:21:31.538 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.d{-1:14538} state up:standby seq 1 addr [v2:172.21.15.43:6836/3739466479,v1:172.21.15.43:6837/3739466479] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:21:31.538 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.g{-1:14577} state up:standby seq 1 addr [v2:172.21.15.43:6838/2165493077,v1:172.21.15.43:6839/2165493077] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:21:31.538 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.a{-1:14589} state up:standby seq 1 addr [v2:172.21.15.43:6840/3592420960,v1:172.21.15.43:6841/3592420960] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:21:31.538 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.l{-1:24400} state up:standby seq 1 addr [v2:172.21.15.97:6832/2769852264,v1:172.21.15.97:6833/2769852264] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:21:31.538 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.k{-1:24425} state up:standby seq 1 addr [v2:172.21.15.90:6832/2266906537,v1:172.21.15.90:6833/2266906537] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T23:21:31.539 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 23 2024-09-11T23:21:31.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.535+0000 7fd1233e0640 1 -- 172.21.15.43:0/804677101 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd1080a2c00 msgr2=0x7fd1080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:31.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.535+0000 7fd1233e0640 1 --2- 172.21.15.43:0/804677101 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd1080a2c00 0x7fd1080a50c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd10c0058c0 tx=0x7fd10c005850 comp rx=0 tx=0).stop 2024-09-11T23:21:31.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.535+0000 7fd1233e0640 1 -- 172.21.15.43:0/804677101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c101cd0 msgr2=0x7fd11c1030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:31.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.535+0000 7fd1233e0640 1 --2- 172.21.15.43:0/804677101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c101cd0 0x7fd11c1030b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd11800d950 tx=0x7fd11800de10 comp rx=0 tx=0).stop 2024-09-11T23:21:31.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.535+0000 7fd1233e0640 1 -- 172.21.15.43:0/804677101 shutdown_connections 2024-09-11T23:21:31.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.535+0000 7fd1233e0640 1 --2- 172.21.15.43:0/804677101 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fd1080a2c00 0x7fd1080a50c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.535+0000 7fd1233e0640 1 --2- 172.21.15.43:0/804677101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd11c102690 0x7fd11c1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.535+0000 7fd1233e0640 1 --2- 172.21.15.43:0/804677101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c101cd0 0x7fd11c1030b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.535+0000 7fd1233e0640 1 --2- 172.21.15.43:0/804677101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd11c100ad0 0x7fd11c104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:31.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.535+0000 7fd1233e0640 1 -- 172.21.15.43:0/804677101 >> 172.21.15.43:0/804677101 conn(0x7fd11c0fc280 msgr2=0x7fd11c069fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:31.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.535+0000 7fd1233e0640 1 -- 172.21.15.43:0/804677101 shutdown_connections 2024-09-11T23:21:31.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:31.535+0000 7fd1233e0640 1 -- 172.21.15.43:0/804677101 wait complete. 2024-09-11T23:21:32.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:31 smithi043 ceph-mon[30950]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:21:32.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:31 smithi043 ceph-mon[30950]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-11T23:21:32.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:31 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/804677101' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T23:21:32.266 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-11T23:21:32.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:31 smithi090 ceph-mon[35130]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:21:32.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:31 smithi090 ceph-mon[35130]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-11T23:21:32.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:31 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/804677101' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T23:21:32.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:31 smithi097 ceph-mon[35562]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:21:32.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:31 smithi097 ceph-mon[35562]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-11T23:21:32.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:31 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/804677101' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T23:21:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:33.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.022+0000 7f77a9cc4640 1 -- 172.21.15.43:0/3414555086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a4100ad0 msgr2=0x7f77a4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.022+0000 7f77a9cc4640 1 --2- 172.21.15.43:0/3414555086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a4100ad0 0x7f77a4100ed0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7794009900 tx=0x7f779402a410 comp rx=0 tx=0).stop 2024-09-11T23:21:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.022+0000 7f77a9cc4640 1 -- 172.21.15.43:0/3414555086 shutdown_connections 2024-09-11T23:21:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.022+0000 7f77a9cc4640 1 --2- 172.21.15.43:0/3414555086 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77a4102690 0x7f77a4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.022+0000 7f77a9cc4640 1 --2- 172.21.15.43:0/3414555086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a4101cd0 0x7f77a4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.022+0000 7f77a9cc4640 1 --2- 172.21.15.43:0/3414555086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a4100ad0 0x7f77a4100ed0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.022+0000 7f77a9cc4640 1 -- 172.21.15.43:0/3414555086 >> 172.21.15.43:0/3414555086 conn(0x7f77a40fc280 msgr2=0x7f77a40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.022+0000 7f77a9cc4640 1 -- 172.21.15.43:0/3414555086 shutdown_connections 2024-09-11T23:21:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.022+0000 7f77a9cc4640 1 -- 172.21.15.43:0/3414555086 wait complete. 2024-09-11T23:21:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.022+0000 7f77a9cc4640 1 Processor -- start 2024-09-11T23:21:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.022+0000 7f77a9cc4640 1 -- start start 2024-09-11T23:21:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.023+0000 7f77a9cc4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a4100ad0 0x7f77a410b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.023+0000 7f77a9cc4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a4101cd0 0x7f77a4109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.023+0000 7f77a9cc4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77a4102690 0x7f77a4109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.023+0000 7f77a9cc4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f77a410d8e0 con 0x7f77a4101cd0 2024-09-11T23:21:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.023+0000 7f77a9cc4640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f77a410d760 con 0x7f77a4102690 2024-09-11T23:21:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.023+0000 7f77a9cc4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f77a410da60 con 0x7f77a4100ad0 2024-09-11T23:21:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.023+0000 7f77a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a4101cd0 0x7f77a4109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.023+0000 7f77a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a4101cd0 0x7f77a4109aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46442/0 (socket says 172.21.15.43:46442) 2024-09-11T23:21:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.023+0000 7f77a2ffd640 1 -- 172.21.15.43:0/3072020948 learned_addr learned my addr 172.21.15.43:0/3072020948 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.023+0000 7f77a37fe640 1 --2- 172.21.15.43:0/3072020948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a4100ad0 0x7f77a410b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.023+0000 7f77a3fff640 1 --2- 172.21.15.43:0/3072020948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77a4102690 0x7f77a4109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.023+0000 7f77a2ffd640 1 -- 172.21.15.43:0/3072020948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a4100ad0 msgr2=0x7f77a410b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.023+0000 7f77a2ffd640 1 --2- 172.21.15.43:0/3072020948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a4100ad0 0x7f77a410b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.023+0000 7f77a2ffd640 1 -- 172.21.15.43:0/3072020948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77a4102690 msgr2=0x7f77a4109fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.023+0000 7f77a2ffd640 1 --2- 172.21.15.43:0/3072020948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77a4102690 0x7f77a4109fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.023+0000 7f77a2ffd640 1 -- 172.21.15.43:0/3072020948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a410a8a0 con 0x7f77a4101cd0 2024-09-11T23:21:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.023+0000 7f77a2ffd640 1 --2- 172.21.15.43:0/3072020948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a4101cd0 0x7f77a4109aa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f779800d950 tx=0x7f779800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.023+0000 7f77a0ff9640 1 -- 172.21.15.43:0/3072020948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798017070 con 0x7f77a4101cd0 2024-09-11T23:21:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.024+0000 7f77a9cc4640 1 -- 172.21.15.43:0/3072020948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a41a20b0 con 0x7f77a4101cd0 2024-09-11T23:21:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.024+0000 7f77a0ff9640 1 -- 172.21.15.43:0/3072020948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7798009c30 con 0x7f77a4101cd0 2024-09-11T23:21:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.024+0000 7f77a0ff9640 1 -- 172.21.15.43:0/3072020948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779800f6b0 con 0x7f77a4101cd0 2024-09-11T23:21:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.024+0000 7f77a9cc4640 1 -- 172.21.15.43:0/3072020948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a41a25c0 con 0x7f77a4101cd0 2024-09-11T23:21:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.025+0000 7f77a9cc4640 1 -- 172.21.15.43:0/3072020948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a4107ed0 con 0x7f77a4101cd0 2024-09-11T23:21:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.026+0000 7f77a0ff9640 1 -- 172.21.15.43:0/3072020948 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f77980109d0 con 0x7f77a4101cd0 2024-09-11T23:21:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.026+0000 7f77a0ff9640 1 --2- 172.21.15.43:0/3072020948 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f778c0a2b80 0x7f778c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.027+0000 7f77a0ff9640 1 -- 172.21.15.43:0/3072020948 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f77980cebf0 con 0x7f77a4101cd0 2024-09-11T23:21:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.027+0000 7f77a37fe640 1 --2- 172.21.15.43:0/3072020948 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f778c0a2b80 0x7f778c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:33.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.027+0000 7f77a37fe640 1 --2- 172.21.15.43:0/3072020948 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f778c0a2b80 0x7f778c0a5040 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f779402aa00 tx=0x7f77940315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:33.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.031+0000 7f77a0ff9640 1 -- 172.21.15.43:0/3072020948 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7798009de0 con 0x7f77a4101cd0 2024-09-11T23:21:33.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.207+0000 7f77a9cc4640 1 -- 172.21.15.43:0/3072020948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f77a4062df0 con 0x7f77a4101cd0 2024-09-11T23:21:33.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.208+0000 7f77a0ff9640 1 -- 172.21.15.43:0/3072020948 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v84) ==== 56+0+5284 (secure 0 0 0) 0x7f779809b4e0 con 0x7f77a4101cd0 2024-09-11T23:21:33.210 INFO:teuthology.orchestra.run.smithi043.stdout:epoch 84 2024-09-11T23:21:33.210 INFO:teuthology.orchestra.run.smithi043.stdout:fsid 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:21:33.210 INFO:teuthology.orchestra.run.smithi043.stdout:created 2024-09-11T23:12:03.504406+0000 2024-09-11T23:21:33.210 INFO:teuthology.orchestra.run.smithi043.stdout:modified 2024-09-11T23:21:02.348513+0000 2024-09-11T23:21:33.210 INFO:teuthology.orchestra.run.smithi043.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-11T23:21:33.210 INFO:teuthology.orchestra.run.smithi043.stdout:crush_version 26 2024-09-11T23:21:33.210 INFO:teuthology.orchestra.run.smithi043.stdout:full_ratio 0.95 2024-09-11T23:21:33.210 INFO:teuthology.orchestra.run.smithi043.stdout:backfillfull_ratio 0.9 2024-09-11T23:21:33.210 INFO:teuthology.orchestra.run.smithi043.stdout:nearfull_ratio 0.85 2024-09-11T23:21:33.210 INFO:teuthology.orchestra.run.smithi043.stdout:require_min_compat_client luminous 2024-09-11T23:21:33.210 INFO:teuthology.orchestra.run.smithi043.stdout:min_compat_client luminous 2024-09-11T23:21:33.211 INFO:teuthology.orchestra.run.smithi043.stdout:require_osd_release squid 2024-09-11T23:21:33.211 INFO:teuthology.orchestra.run.smithi043.stdout:stretch_mode_enabled false 2024-09-11T23:21:33.211 INFO:teuthology.orchestra.run.smithi043.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 23 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-11T23:21:33.211 INFO:teuthology.orchestra.run.smithi043.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-11T23:21:33.211 INFO:teuthology.orchestra.run.smithi043.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-11T23:21:33.211 INFO:teuthology.orchestra.run.smithi043.stdout:max_osd 12 2024-09-11T23:21:33.211 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0 up in weight 1 up_from 8 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6802/43839997,v1:172.21.15.43:6803/43839997] [v2:172.21.15.43:6804/43839997,v1:172.21.15.43:6805/43839997] exists,up 24076596-3909-4c87-85f1-cbf168070dbb 2024-09-11T23:21:33.211 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1 up in weight 1 up_from 13 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6800/1825149246,v1:172.21.15.90:6801/1825149246] [v2:172.21.15.90:6802/1825149246,v1:172.21.15.90:6803/1825149246] exists,up a0472a57-243c-489d-a247-972ee10a71f8 2024-09-11T23:21:33.211 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2 up in weight 1 up_from 18 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6800/1456846443,v1:172.21.15.97:6801/1456846443] [v2:172.21.15.97:6802/1456846443,v1:172.21.15.97:6803/1456846443] exists,up 40366536-0fcf-45a0-b8e3-49178401dc34 2024-09-11T23:21:33.211 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3 up in weight 1 up_from 26 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6810/1962837804,v1:172.21.15.43:6811/1962837804] [v2:172.21.15.43:6812/1962837804,v1:172.21.15.43:6813/1962837804] exists,up 3062133e-000b-4421-9d53-ac434b9ed9b8 2024-09-11T23:21:33.211 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4 up in weight 1 up_from 33 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6808/181614173,v1:172.21.15.90:6809/181614173] [v2:172.21.15.90:6810/181614173,v1:172.21.15.90:6811/181614173] exists,up 2e73fa4b-f22c-4319-828d-87b91b9f19f7 2024-09-11T23:21:33.211 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5 up in weight 1 up_from 37 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6808/1540039543,v1:172.21.15.97:6809/1540039543] [v2:172.21.15.97:6810/1540039543,v1:172.21.15.97:6811/1540039543] exists,up 1e1ad4f9-e724-440f-b515-3ec1ad97d6e4 2024-09-11T23:21:33.212 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6 up in weight 1 up_from 44 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6818/3785677637,v1:172.21.15.43:6819/3785677637] [v2:172.21.15.43:6820/3785677637,v1:172.21.15.43:6821/3785677637] exists,up d34df8df-f40d-4159-8b29-5401b1b0e431 2024-09-11T23:21:33.212 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7 up in weight 1 up_from 49 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6816/236874962,v1:172.21.15.90:6817/236874962] [v2:172.21.15.90:6818/236874962,v1:172.21.15.90:6819/236874962] exists,up b2e01078-528b-49fd-b000-9f584f784b1d 2024-09-11T23:21:33.212 INFO:teuthology.orchestra.run.smithi043.stdout:osd.8 up in weight 1 up_from 56 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6816/1707327301,v1:172.21.15.97:6817/1707327301] [v2:172.21.15.97:6818/1707327301,v1:172.21.15.97:6819/1707327301] exists,up abef8f27-bf9b-4d63-bfbe-f967fc3d8b17 2024-09-11T23:21:33.212 INFO:teuthology.orchestra.run.smithi043.stdout:osd.9 up in weight 1 up_from 64 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6826/3170306011,v1:172.21.15.43:6827/3170306011] [v2:172.21.15.43:6828/3170306011,v1:172.21.15.43:6829/3170306011] exists,up 190ab8b2-a835-47e6-9b39-16a58331c58a 2024-09-11T23:21:33.212 INFO:teuthology.orchestra.run.smithi043.stdout:osd.10 up in weight 1 up_from 69 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6824/59487041,v1:172.21.15.90:6825/59487041] [v2:172.21.15.90:6826/59487041,v1:172.21.15.90:6827/59487041] exists,up b32ab5fe-466f-4c8e-8696-cd9f820a3263 2024-09-11T23:21:33.212 INFO:teuthology.orchestra.run.smithi043.stdout:osd.11 up in weight 1 up_from 76 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6824/194699315,v1:172.21.15.97:6825/194699315] [v2:172.21.15.97:6826/194699315,v1:172.21.15.97:6827/194699315] exists,up d0faa589-21c8-4a9f-8a73-813aafe6d33c 2024-09-11T23:21:33.212 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 2.a [2,11] 2024-09-11T23:21:33.212 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 2.16 [2,4] 2024-09-11T23:21:33.212 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 2.1d [8,11] 2024-09-11T23:21:33.212 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.2 [5,7] 2024-09-11T23:21:33.212 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.3 [4,9] 2024-09-11T23:21:33.212 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.e [4,6] 2024-09-11T23:21:33.212 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.1d [4,8] 2024-09-11T23:21:33.213 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.1e [3,9] 2024-09-11T23:21:33.213 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.39 [5,8] 2024-09-11T23:21:33.213 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.3c [4,9] 2024-09-11T23:21:33.213 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/76575371 expires 2024-09-12T23:13:00.548389+0000 2024-09-11T23:21:33.213 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/4006124758 expires 2024-09-12T23:13:00.548389+0000 2024-09-11T23:21:33.213 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/120495444 expires 2024-09-12T23:13:00.548389+0000 2024-09-11T23:21:33.213 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6801/3892590401 expires 2024-09-12T23:13:00.548389+0000 2024-09-11T23:21:33.213 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6800/3892590401 expires 2024-09-12T23:13:00.548389+0000 2024-09-11T23:21:33.213 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6801/4015351527 expires 2024-09-12T23:12:35.756598+0000 2024-09-11T23:21:33.213 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6800/4015351527 expires 2024-09-12T23:12:35.756598+0000 2024-09-11T23:21:33.213 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/206044413 expires 2024-09-12T23:12:35.756598+0000 2024-09-11T23:21:33.213 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/1512201905 expires 2024-09-12T23:12:35.756598+0000 2024-09-11T23:21:33.213 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/1375379108 expires 2024-09-12T23:12:35.756598+0000 2024-09-11T23:21:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.211+0000 7f77a9cc4640 1 -- 172.21.15.43:0/3072020948 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f778c0a2b80 msgr2=0x7f778c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.211+0000 7f77a9cc4640 1 --2- 172.21.15.43:0/3072020948 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f778c0a2b80 0x7f778c0a5040 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f779402aa00 tx=0x7f77940315b0 comp rx=0 tx=0).stop 2024-09-11T23:21:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.211+0000 7f77a9cc4640 1 -- 172.21.15.43:0/3072020948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a4101cd0 msgr2=0x7f77a4109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.211+0000 7f77a9cc4640 1 --2- 172.21.15.43:0/3072020948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a4101cd0 0x7f77a4109aa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f779800d950 tx=0x7f779800de10 comp rx=0 tx=0).stop 2024-09-11T23:21:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.211+0000 7f77a9cc4640 1 -- 172.21.15.43:0/3072020948 shutdown_connections 2024-09-11T23:21:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.211+0000 7f77a9cc4640 1 --2- 172.21.15.43:0/3072020948 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f778c0a2b80 0x7f778c0a5040 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.211+0000 7f77a9cc4640 1 --2- 172.21.15.43:0/3072020948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77a4102690 0x7f77a4109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.211+0000 7f77a9cc4640 1 --2- 172.21.15.43:0/3072020948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a4101cd0 0x7f77a4109aa0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.211+0000 7f77a9cc4640 1 --2- 172.21.15.43:0/3072020948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77a4100ad0 0x7f77a410b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.211+0000 7f77a9cc4640 1 -- 172.21.15.43:0/3072020948 >> 172.21.15.43:0/3072020948 conn(0x7f77a40fc280 msgr2=0x7f77a41050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.211+0000 7f77a9cc4640 1 -- 172.21.15.43:0/3072020948 shutdown_connections 2024-09-11T23:21:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:33.211+0000 7f77a9cc4640 1 -- 172.21.15.43:0/3072020948 wait complete. 2024-09-11T23:21:34.021 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-11T23:21:34.034 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:21:34.273 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:33 smithi090 ceph-mon[35130]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-11T23:21:34.273 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:33 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3072020948' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T23:21:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:34.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:33 smithi097 ceph-mon[35562]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-11T23:21:34.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:33 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3072020948' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T23:21:34.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:33 smithi043 ceph-mon[30950]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-11T23:21:34.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:33 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3072020948' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T23:21:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.780+0000 7f7e167d8640 1 -- 172.21.15.43:0/1997969246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e10102570 msgr2=0x7f7e101029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.780+0000 7f7e167d8640 1 --2- 172.21.15.43:0/1997969246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e10102570 0x7f7e101029b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7df8009900 tx=0x7f7df802a480 comp rx=0 tx=0).stop 2024-09-11T23:21:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.780+0000 7f7e167d8640 1 -- 172.21.15.43:0/1997969246 shutdown_connections 2024-09-11T23:21:34.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.780+0000 7f7e167d8640 1 --2- 172.21.15.43:0/1997969246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e10102ef0 0x7f7e1010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:34.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.780+0000 7f7e167d8640 1 --2- 172.21.15.43:0/1997969246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e10102570 0x7f7e101029b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:34.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.780+0000 7f7e167d8640 1 --2- 172.21.15.43:0/1997969246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e10100e20 0x7f7e10101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:34.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.780+0000 7f7e167d8640 1 -- 172.21.15.43:0/1997969246 >> 172.21.15.43:0/1997969246 conn(0x7f7e100fc7d0 msgr2=0x7f7e100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:34.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.780+0000 7f7e167d8640 1 -- 172.21.15.43:0/1997969246 shutdown_connections 2024-09-11T23:21:34.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.780+0000 7f7e167d8640 1 -- 172.21.15.43:0/1997969246 wait complete. 2024-09-11T23:21:34.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.781+0000 7f7e167d8640 1 Processor -- start 2024-09-11T23:21:34.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.781+0000 7f7e167d8640 1 -- start start 2024-09-11T23:21:34.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.781+0000 7f7e167d8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e10100e20 0x7f7e100fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:34.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.781+0000 7f7e167d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e10102570 0x7f7e100fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:34.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.781+0000 7f7e167d8640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e10102ef0 0x7f7e100fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:34.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.781+0000 7f7e167d8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7e10110f50 con 0x7f7e10102570 2024-09-11T23:21:34.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.781+0000 7f7e167d8640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7e10110dd0 con 0x7f7e10102ef0 2024-09-11T23:21:34.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.781+0000 7f7e167d8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7e101110d0 con 0x7f7e10100e20 2024-09-11T23:21:34.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.781+0000 7f7e07fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e10102570 0x7f7e100fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:34.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.781+0000 7f7e07fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e10102570 0x7f7e100fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46470/0 (socket says 172.21.15.43:46470) 2024-09-11T23:21:34.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.781+0000 7f7e07fff640 1 -- 172.21.15.43:0/1643802354 learned_addr learned my addr 172.21.15.43:0/1643802354 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:34.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.781+0000 7f7e0ffff640 1 --2- 172.21.15.43:0/1643802354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e10100e20 0x7f7e100fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:34.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.781+0000 7f7e14d75640 1 --2- 172.21.15.43:0/1643802354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e10102ef0 0x7f7e100fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:34.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.781+0000 7f7e07fff640 1 -- 172.21.15.43:0/1643802354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e10100e20 msgr2=0x7f7e100fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:34.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.781+0000 7f7e07fff640 1 --2- 172.21.15.43:0/1643802354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e10100e20 0x7f7e100fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:34.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.781+0000 7f7e07fff640 1 -- 172.21.15.43:0/1643802354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e10102ef0 msgr2=0x7f7e100fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:34.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.781+0000 7f7e07fff640 1 --2- 172.21.15.43:0/1643802354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e10102ef0 0x7f7e100fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:34.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.781+0000 7f7e07fff640 1 -- 172.21.15.43:0/1643802354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e101a6920 con 0x7f7e10102570 2024-09-11T23:21:34.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.781+0000 7f7e14d75640 1 --2- 172.21.15.43:0/1643802354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e10102ef0 0x7f7e100fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:21:34.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.781+0000 7f7e07fff640 1 --2- 172.21.15.43:0/1643802354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e10102570 0x7f7e100fd1c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7df8006010 tx=0x7f7df8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:34.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.782+0000 7f7e0dffb640 1 -- 172.21.15.43:0/1643802354 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df803f070 con 0x7f7e10102570 2024-09-11T23:21:34.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.782+0000 7f7e0dffb640 1 -- 172.21.15.43:0/1643802354 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7df8031c30 con 0x7f7e10102570 2024-09-11T23:21:34.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.782+0000 7f7e167d8640 1 -- 172.21.15.43:0/1643802354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e101a6bb0 con 0x7f7e10102570 2024-09-11T23:21:34.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.782+0000 7f7e0dffb640 1 -- 172.21.15.43:0/1643802354 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df8035030 con 0x7f7e10102570 2024-09-11T23:21:34.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.782+0000 7f7e167d8640 1 -- 172.21.15.43:0/1643802354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e101a70e0 con 0x7f7e10102570 2024-09-11T23:21:34.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.783+0000 7f7e167d8640 1 -- 172.21.15.43:0/1643802354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e10101200 con 0x7f7e10102570 2024-09-11T23:21:34.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.784+0000 7f7e0dffb640 1 -- 172.21.15.43:0/1643802354 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7df8042070 con 0x7f7e10102570 2024-09-11T23:21:34.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.785+0000 7f7e0dffb640 1 --2- 172.21.15.43:0/1643802354 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f7dfc0a2c00 0x7f7dfc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:34.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.785+0000 7f7e0ffff640 1 --2- 172.21.15.43:0/1643802354 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f7dfc0a2c00 0x7f7dfc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:34.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.785+0000 7f7e0dffb640 1 -- 172.21.15.43:0/1643802354 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f7df80efbb0 con 0x7f7e10102570 2024-09-11T23:21:34.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.785+0000 7f7e0ffff640 1 --2- 172.21.15.43:0/1643802354 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f7dfc0a2c00 0x7f7dfc0a50c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000abe0 tx=0x7f7e0000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:34.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:34.788+0000 7f7e0dffb640 1 -- 172.21.15.43:0/1643802354 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7df80bcaf0 con 0x7f7e10102570 2024-09-11T23:21:35.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:35.002+0000 7f7e167d8640 1 -- 172.21.15.43:0/1643802354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e100fe1b0 con 0x7f7e10102570 2024-09-11T23:21:35.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:35.003+0000 7f7e0dffb640 1 -- 172.21.15.43:0/1643802354 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11896 (secure 0 0 0) 0x7f7e100fe1b0 con 0x7f7e10102570 2024-09-11T23:21:35.004 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:21:35.005 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":23,"btime":"2024-09-11T23:20:12:774166+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:20:12.440327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24457,24470,24497]},"id":1}]} 2024-09-11T23:21:35.005 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 23 2024-09-11T23:21:35.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:35.006+0000 7f7e167d8640 1 -- 172.21.15.43:0/1643802354 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f7dfc0a2c00 msgr2=0x7f7dfc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:35.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:35.006+0000 7f7e167d8640 1 --2- 172.21.15.43:0/1643802354 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f7dfc0a2c00 0x7f7dfc0a50c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000abe0 tx=0x7f7e0000a2d0 comp rx=0 tx=0).stop 2024-09-11T23:21:35.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:35.006+0000 7f7e167d8640 1 -- 172.21.15.43:0/1643802354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e10102570 msgr2=0x7f7e100fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:35.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:35.006+0000 7f7e167d8640 1 --2- 172.21.15.43:0/1643802354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e10102570 0x7f7e100fd1c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7df8006010 tx=0x7f7df8003c10 comp rx=0 tx=0).stop 2024-09-11T23:21:35.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:35.006+0000 7f7e167d8640 1 -- 172.21.15.43:0/1643802354 shutdown_connections 2024-09-11T23:21:35.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:35.006+0000 7f7e167d8640 1 --2- 172.21.15.43:0/1643802354 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f7dfc0a2c00 0x7f7dfc0a50c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:35.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:35.006+0000 7f7e167d8640 1 --2- 172.21.15.43:0/1643802354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e10102ef0 0x7f7e100fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:35.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:35.006+0000 7f7e167d8640 1 --2- 172.21.15.43:0/1643802354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e10102570 0x7f7e100fd1c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:35.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:35.006+0000 7f7e167d8640 1 --2- 172.21.15.43:0/1643802354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e10100e20 0x7f7e100fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:35.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:35.006+0000 7f7e167d8640 1 -- 172.21.15.43:0/1643802354 >> 172.21.15.43:0/1643802354 conn(0x7f7e100fc7d0 msgr2=0x7f7e1006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:35.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:35.006+0000 7f7e167d8640 1 -- 172.21.15.43:0/1643802354 shutdown_connections 2024-09-11T23:21:35.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:35.006+0000 7f7e167d8640 1 -- 172.21.15.43:0/1643802354 wait complete. 2024-09-11T23:21:35.647 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:21:35.913 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:36.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:35 smithi090 ceph-mon[35130]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:21:36.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:35 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1643802354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:36.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:35 smithi097 ceph-mon[35562]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:21:36.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:35 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1643802354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:36.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:35 smithi043 ceph-mon[30950]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:21:36.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:35 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1643802354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:36.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.395+0000 7fe99c256640 1 -- 172.21.15.43:0/2982082553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe994101400 msgr2=0x7fe99410d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.395+0000 7fe99c256640 1 --2- 172.21.15.43:0/2982082553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe994101400 0x7fe99410d930 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009cd0 tx=0x7fe98c02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:21:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.395+0000 7fe99c256640 1 -- 172.21.15.43:0/2982082553 shutdown_connections 2024-09-11T23:21:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.395+0000 7fe99c256640 1 --2- 172.21.15.43:0/2982082553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe994101400 0x7fe99410d930 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.395+0000 7fe99c256640 1 --2- 172.21.15.43:0/2982082553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe994100a60 0x7fe994100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.395+0000 7fe99c256640 1 --2- 172.21.15.43:0/2982082553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe994106a60 0x7fe994106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.395+0000 7fe99c256640 1 -- 172.21.15.43:0/2982082553 >> 172.21.15.43:0/2982082553 conn(0x7fe9940fc780 msgr2=0x7fe9940feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.395+0000 7fe99c256640 1 -- 172.21.15.43:0/2982082553 shutdown_connections 2024-09-11T23:21:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.395+0000 7fe99c256640 1 -- 172.21.15.43:0/2982082553 wait complete. 2024-09-11T23:21:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.396+0000 7fe99c256640 1 Processor -- start 2024-09-11T23:21:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.396+0000 7fe99c256640 1 -- start start 2024-09-11T23:21:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.396+0000 7fe99c256640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe994100a60 0x7fe994198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:36.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.396+0000 7fe99c256640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe994101400 0x7fe994198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:36.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.396+0000 7fe99c256640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe994106a60 0x7fe99419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:36.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.396+0000 7fe99c256640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe994110eb0 con 0x7fe994100a60 2024-09-11T23:21:36.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.396+0000 7fe99c256640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe994110d30 con 0x7fe994106a60 2024-09-11T23:21:36.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.396+0000 7fe99c256640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe994111030 con 0x7fe994101400 2024-09-11T23:21:36.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.396+0000 7fe999ff2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe994100a60 0x7fe994198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:36.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.396+0000 7fe999ff2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe994100a60 0x7fe994198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46484/0 (socket says 172.21.15.43:46484) 2024-09-11T23:21:36.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.396+0000 7fe999ff2640 1 -- 172.21.15.43:0/2586619118 learned_addr learned my addr 172.21.15.43:0/2586619118 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:36.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.396+0000 7fe9997f1640 1 --2- 172.21.15.43:0/2586619118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe994101400 0x7fe994198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:36.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.396+0000 7fe999ff2640 1 -- 172.21.15.43:0/2586619118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe994101400 msgr2=0x7fe994198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:36.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.396+0000 7fe999ff2640 1 --2- 172.21.15.43:0/2586619118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe994101400 0x7fe994198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:36.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.396+0000 7fe999ff2640 1 -- 172.21.15.43:0/2586619118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe994106a60 msgr2=0x7fe99419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:36.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.396+0000 7fe999ff2640 1 --2- 172.21.15.43:0/2586619118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe994106a60 0x7fe99419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:36.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.396+0000 7fe999ff2640 1 -- 172.21.15.43:0/2586619118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99419d2c0 con 0x7fe994100a60 2024-09-11T23:21:36.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.397+0000 7fe999ff2640 1 --2- 172.21.15.43:0/2586619118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe994100a60 0x7fe994198310 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe99000c920 tx=0x7fe99000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:36.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.397+0000 7fe98affd640 1 -- 172.21.15.43:0/2586619118 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990007c70 con 0x7fe994100a60 2024-09-11T23:21:36.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.397+0000 7fe98affd640 1 -- 172.21.15.43:0/2586619118 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe990007e10 con 0x7fe994100a60 2024-09-11T23:21:36.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.397+0000 7fe99c256640 1 -- 172.21.15.43:0/2586619118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99419d5b0 con 0x7fe994100a60 2024-09-11T23:21:36.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.397+0000 7fe98affd640 1 -- 172.21.15.43:0/2586619118 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990018890 con 0x7fe994100a60 2024-09-11T23:21:36.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.397+0000 7fe99c256640 1 -- 172.21.15.43:0/2586619118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9941a4e90 con 0x7fe994100a60 2024-09-11T23:21:36.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.398+0000 7fe99c256640 1 -- 172.21.15.43:0/2586619118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe994100ec0 con 0x7fe994100a60 2024-09-11T23:21:36.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.399+0000 7fe98affd640 1 -- 172.21.15.43:0/2586619118 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe990018a50 con 0x7fe994100a60 2024-09-11T23:21:36.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.400+0000 7fe98affd640 1 --2- 172.21.15.43:0/2586619118 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe9800a2c00 0x7fe9800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:36.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.400+0000 7fe9997f1640 1 --2- 172.21.15.43:0/2586619118 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe9800a2c00 0x7fe9800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:36.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.400+0000 7fe98affd640 1 -- 172.21.15.43:0/2586619118 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7fe9900cf3f0 con 0x7fe994100a60 2024-09-11T23:21:36.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.400+0000 7fe9997f1640 1 --2- 172.21.15.43:0/2586619118 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe9800a2c00 0x7fe9800a50c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe984005fd0 tx=0x7fe984005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:36.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.403+0000 7fe98affd640 1 -- 172.21.15.43:0/2586619118 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe990014030 con 0x7fe994100a60 2024-09-11T23:21:36.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.615+0000 7fe99c256640 1 -- 172.21.15.43:0/2586619118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe994199670 con 0x7fe994100a60 2024-09-11T23:21:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.616+0000 7fe98affd640 1 -- 172.21.15.43:0/2586619118 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11896 (secure 0 0 0) 0x7fe99000de20 con 0x7fe994100a60 2024-09-11T23:21:36.617 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:21:36.617 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":23,"btime":"2024-09-11T23:20:12:774166+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:20:12.440327+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24457,24470,24497]},"id":1}]} 2024-09-11T23:21:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 23 2024-09-11T23:21:36.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.618+0000 7fe99c256640 1 -- 172.21.15.43:0/2586619118 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe9800a2c00 msgr2=0x7fe9800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:36.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.618+0000 7fe99c256640 1 --2- 172.21.15.43:0/2586619118 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe9800a2c00 0x7fe9800a50c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe984005fd0 tx=0x7fe984005850 comp rx=0 tx=0).stop 2024-09-11T23:21:36.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.618+0000 7fe99c256640 1 -- 172.21.15.43:0/2586619118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe994100a60 msgr2=0x7fe994198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:36.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.618+0000 7fe99c256640 1 --2- 172.21.15.43:0/2586619118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe994100a60 0x7fe994198310 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe99000c920 tx=0x7fe99000cde0 comp rx=0 tx=0).stop 2024-09-11T23:21:36.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.618+0000 7fe99c256640 1 -- 172.21.15.43:0/2586619118 shutdown_connections 2024-09-11T23:21:36.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.618+0000 7fe99c256640 1 --2- 172.21.15.43:0/2586619118 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fe9800a2c00 0x7fe9800a50c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:36.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.618+0000 7fe99c256640 1 --2- 172.21.15.43:0/2586619118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe994106a60 0x7fe99419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:36.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.618+0000 7fe99c256640 1 --2- 172.21.15.43:0/2586619118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe994101400 0x7fe994198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:36.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.618+0000 7fe99c256640 1 --2- 172.21.15.43:0/2586619118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe994100a60 0x7fe994198310 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:36.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.618+0000 7fe99c256640 1 -- 172.21.15.43:0/2586619118 >> 172.21.15.43:0/2586619118 conn(0x7fe9940fc780 msgr2=0x7fe99406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:36.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.618+0000 7fe99c256640 1 -- 172.21.15.43:0/2586619118 shutdown_connections 2024-09-11T23:21:36.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:36.618+0000 7fe99c256640 1 -- 172.21.15.43:0/2586619118 wait complete. 2024-09-11T23:21:37.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:36 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2586619118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:37.266 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:21:37.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:36 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2586619118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:37.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:36 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2586619118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:37.535 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:38.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.006+0000 7fa7af7e5640 1 -- 172.21.15.43:0/1058298157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7a8102ef0 msgr2=0x7fa7a810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:38.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.006+0000 7fa7af7e5640 1 --2- 172.21.15.43:0/1058298157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7a8102ef0 0x7fa7a810da40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009ca0 tx=0x7fa7a002a950 comp rx=0 tx=0).stop 2024-09-11T23:21:38.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.006+0000 7fa7af7e5640 1 -- 172.21.15.43:0/1058298157 shutdown_connections 2024-09-11T23:21:38.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.006+0000 7fa7af7e5640 1 --2- 172.21.15.43:0/1058298157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7a8102ef0 0x7fa7a810da40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:38.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.006+0000 7fa7af7e5640 1 --2- 172.21.15.43:0/1058298157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7a8102570 0x7fa7a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:38.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.006+0000 7fa7af7e5640 1 --2- 172.21.15.43:0/1058298157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a8100e20 0x7fa7a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:38.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.006+0000 7fa7af7e5640 1 -- 172.21.15.43:0/1058298157 >> 172.21.15.43:0/1058298157 conn(0x7fa7a80fc7d0 msgr2=0x7fa7a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:38.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.006+0000 7fa7af7e5640 1 -- 172.21.15.43:0/1058298157 shutdown_connections 2024-09-11T23:21:38.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.006+0000 7fa7af7e5640 1 -- 172.21.15.43:0/1058298157 wait complete. 2024-09-11T23:21:38.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.007+0000 7fa7af7e5640 1 Processor -- start 2024-09-11T23:21:38.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.007+0000 7fa7af7e5640 1 -- start start 2024-09-11T23:21:38.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.007+0000 7fa7af7e5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7a8100e20 0x7fa7a81a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:38.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.007+0000 7fa7af7e5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7a8102570 0x7fa7a81a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.007+0000 7fa7af7e5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a8102ef0 0x7fa7a819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.007+0000 7fa7af7e5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a8111020 con 0x7fa7a8102570 2024-09-11T23:21:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.007+0000 7fa7af7e5640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a8110ea0 con 0x7fa7a8100e20 2024-09-11T23:21:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.007+0000 7fa7af7e5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81111a0 con 0x7fa7a8102ef0 2024-09-11T23:21:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.008+0000 7fa7acd80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7a8102570 0x7fa7a81a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.008+0000 7fa7ad581640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7a8100e20 0x7fa7a81a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.008+0000 7fa7acd80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7a8102570 0x7fa7a81a4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46500/0 (socket says 172.21.15.43:46500) 2024-09-11T23:21:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.008+0000 7fa7acd80640 1 -- 172.21.15.43:0/783322429 learned_addr learned my addr 172.21.15.43:0/783322429 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.008+0000 7fa7add82640 1 --2- 172.21.15.43:0/783322429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a8102ef0 0x7fa7a819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.008+0000 7fa7acd80640 1 -- 172.21.15.43:0/783322429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a8102ef0 msgr2=0x7fa7a819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.008+0000 7fa7acd80640 1 --2- 172.21.15.43:0/783322429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a8102ef0 0x7fa7a819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.008+0000 7fa7acd80640 1 -- 172.21.15.43:0/783322429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7a8100e20 msgr2=0x7fa7a81a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:38.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.008+0000 7fa7acd80640 1 --2- 172.21.15.43:0/783322429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7a8100e20 0x7fa7a81a4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:38.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.008+0000 7fa7acd80640 1 -- 172.21.15.43:0/783322429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a819f5c0 con 0x7fa7a8102570 2024-09-11T23:21:38.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.008+0000 7fa7ad581640 1 --2- 172.21.15.43:0/783322429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7a8100e20 0x7fa7a81a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:38.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.008+0000 7fa7acd80640 1 --2- 172.21.15.43:0/783322429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7a8102570 0x7fa7a81a4ec0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa79800b4a0 tx=0x7fa79800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:38.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.008+0000 7fa79e7fc640 1 -- 172.21.15.43:0/783322429 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa798014070 con 0x7fa7a8102570 2024-09-11T23:21:38.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.009+0000 7fa79e7fc640 1 -- 172.21.15.43:0/783322429 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa798010af0 con 0x7fa7a8102570 2024-09-11T23:21:38.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.009+0000 7fa7af7e5640 1 -- 172.21.15.43:0/783322429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a819f8b0 con 0x7fa7a8102570 2024-09-11T23:21:38.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.009+0000 7fa7af7e5640 1 -- 172.21.15.43:0/783322429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a81ab8b0 con 0x7fa7a8102570 2024-09-11T23:21:38.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.009+0000 7fa79e7fc640 1 -- 172.21.15.43:0/783322429 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa798017d20 con 0x7fa7a8102570 2024-09-11T23:21:38.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.010+0000 7fa7af7e5640 1 -- 172.21.15.43:0/783322429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7a81021b0 con 0x7fa7a8102570 2024-09-11T23:21:38.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.011+0000 7fa79e7fc640 1 -- 172.21.15.43:0/783322429 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa798017070 con 0x7fa7a8102570 2024-09-11T23:21:38.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.012+0000 7fa79e7fc640 1 --2- 172.21.15.43:0/783322429 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa7940a2c00 0x7fa7940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:38.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.012+0000 7fa79e7fc640 1 -- 172.21.15.43:0/783322429 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7fa7980ce440 con 0x7fa7a8102570 2024-09-11T23:21:38.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.012+0000 7fa7ad581640 1 --2- 172.21.15.43:0/783322429 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa7940a2c00 0x7fa7940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:38.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.012+0000 7fa7ad581640 1 --2- 172.21.15.43:0/783322429 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa7940a2c00 0x7fa7940a50c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009930 tx=0x7fa7a4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:38.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.016+0000 7fa79e7fc640 1 -- 172.21.15.43:0/783322429 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa79809b380 con 0x7fa7a8102570 2024-09-11T23:21:38.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.192+0000 7fa7af7e5640 1 -- 172.21.15.43:0/783322429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7a81abb40 con 0x7fa7a8102570 2024-09-11T23:21:38.194 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:37 smithi043 ceph-mon[30950]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-11T23:21:38.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.194+0000 7fa79e7fc640 1 -- 172.21.15.43:0/783322429 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22671 (secure 0 0 0) 0x7fa79809ac80 con 0x7fa7a8102570 2024-09-11T23:21:38.195 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:21:38.195 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":84,"fsid":"25c0cd84-7093-11ef-bcea-c7b262605968","created":"2024-09-11T23:12:03.504406+0000","modified":"2024-09-11T23:21:02.348513+0000","last_up_change":"2024-09-11T23:19:07.797873+0000","last_in_change":"2024-09-11T23:18:44.595176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:15:17.058713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:19:56.221406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:19:58.705875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24076596-3909-4c87-85f1-cbf168070dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6803","nonce":43839997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6805","nonce":43839997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6809","nonce":43839997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6807","nonce":43839997}]},"public_addr":"172.21.15.43:6803/43839997","cluster_addr":"172.21.15.43:6805/43839997","heartbeat_back_addr":"172.21.15.43:6809/43839997","heartbeat_front_addr":"172.21.15.43:6807/43839997","state":["exists","up"]},{"osd":1,"uuid":"a0472a57-243c-489d-a247-972ee10a71f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6801","nonce":1825149246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6803","nonce":1825149246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6807","nonce":1825149246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6805","nonce":1825149246}]},"public_addr":"172.21.15.90:6801/1825149246","cluster_addr":"172.21.15.90:6803/1825149246","heartbeat_back_addr":"172.21.15.90:6807/1825149246","heartbeat_front_addr":"172.21.15.90:6805/1825149246","state":["exists","up"]},{"osd":2,"uuid":"40366536-0fcf-45a0-b8e3-49178401dc34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6801","nonce":1456846443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6803","nonce":1456846443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6807","nonce":1456846443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6805","nonce":1456846443}]},"public_addr":"172.21.15.97:6801/1456846443","cluster_addr":"172.21.15.97:6803/1456846443","heartbeat_back_addr":"172.21.15.97:6807/1456846443","heartbeat_front_addr":"172.21.15.97:6805/1456846443","state":["exists","up"]},{"osd":3,"uuid":"3062133e-000b-4421-9d53-ac434b9ed9b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6811","nonce":1962837804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6813","nonce":1962837804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6817","nonce":1962837804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6815","nonce":1962837804}]},"public_addr":"172.21.15.43:6811/1962837804","cluster_addr":"172.21.15.43:6813/1962837804","heartbeat_back_addr":"172.21.15.43:6817/1962837804","heartbeat_front_addr":"172.21.15.43:6815/1962837804","state":["exists","up"]},{"osd":4,"uuid":"2e73fa4b-f22c-4319-828d-87b91b9f19f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6809","nonce":181614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6811","nonce":181614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6815","nonce":181614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6813","nonce":181614173}]},"public_addr":"172.21.15.90:6809/181614173","cluster_addr":"172.21.15.90:6811/181614173","heartbeat_back_addr":"172.21.15.90:6815/181614173","heartbeat_front_addr":"172.21.15.90:6813/181614173","state":["exists","up"]},{"osd":5,"uuid":"1e1ad4f9-e724-440f-b515-3ec1ad97d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6809","nonce":1540039543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6811","nonce":1540039543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6815","nonce":1540039543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6813","nonce":1540039543}]},"public_addr":"172.21.15.97:6809/1540039543","cluster_addr":"172.21.15.97:6811/1540039543","heartbeat_back_addr":"172.21.15.97:6815/1540039543","heartbeat_front_addr":"172.21.15.97:6813/1540039543","state":["exists","up"]},{"osd":6,"uuid":"d34df8df-f40d-4159-8b29-5401b1b0e431","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6819","nonce":3785677637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6821","nonce":3785677637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6825","nonce":3785677637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6823","nonce":3785677637}]},"public_addr":"172.21.15.43:6819/3785677637","cluster_addr":"172.21.15.43:6821/3785677637","heartbeat_back_addr":"172.21.15.43:6825/3785677637","heartbeat_front_addr":"172.21.15.43:6823/3785677637","state":["exists","up"]},{"osd":7,"uuid":"b2e01078-528b-49fd-b000-9f584f784b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6817","nonce":236874962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6819","nonce":236874962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6823","nonce":236874962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6821","nonce":236874962}]},"public_addr":"172.21.15.90:6817/236874962","cluster_addr":"172.21.15.90:6819/236874962","heartbeat_back_addr":"172.21.15.90:6823/236874962","heartbeat_front_addr":"172.21.15.90:6821/236874962","state":["exists","up"]},{"osd":8,"uuid":"abef8f27-bf9b-4d63-bfbe-f967fc3d8b17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6817","nonce":1707327301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6819","nonce":1707327301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6823","nonce":1707327301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6821","nonce":1707327301}]},"public_addr":"172.21.15.97:6817/1707327301","cluster_addr":"172.21.15.97:6819/1707327301","heartbeat_back_addr":"172.21.15.97:6823/1707327301","heartbeat_front_addr":"172.21.15.97:6821/1707327301","state":["exists","up"]},{"osd":9,"uuid":"190ab8b2-a835-47e6-9b39-16a58331c58a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6827","nonce":3170306011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6829","nonce":3170306011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6833","nonce":3170306011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6831","nonce":3170306011}]},"public_addr":"172.21.15.43:6827/3170306011","cluster_addr":"172.21.15.43:6829/3170306011","heartbeat_back_addr":"172.21.15.43:6833/3170306011","heartbeat_front_addr":"172.21.15.43:6831/3170306011","state":["exists","up"]},{"osd":10,"uuid":"b32ab5fe-466f-4c8e-8696-cd9f820a3263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6825","nonce":59487041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6827","nonce":59487041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6831","nonce":59487041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6829","nonce":59487041}]},"public_addr":"172.21.15.90:6825/59487041","cluster_addr":"172.21.15.90:6827/59487041","heartbeat_back_addr":"172.21.15.90:6831/59487041","heartbeat_front_addr":"172.21.15.90:6829/59487041","state":["exists","up"]},{"osd":11,"uuid":"d0faa589-21c8-4a9f-8a73-813aafe6d33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6825","nonce":194699315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6827","nonce":194699315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6831","nonce":194699315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6829","nonce":194699315}]},"public_addr":"172.21.15.97:6825/194699315","cluster_addr":"172.21.15.97:6827/194699315","heartbeat_back_addr":"172.21.15.97:6831/194699315","heartbeat_front_addr":"172.21.15.97:6829/194699315","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:23.381247+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:48.843190+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:13.538979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:40.136983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:05.513790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:31.247090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:56.635221+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:22.667327+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:47.910422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:14.353914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:40.584503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:19:06.476766+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/76575371":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/4006124758":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/120495444":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6800/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:6800/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/206044413":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1512201905":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1375379108":"2024-09-12T23:12:35.756598+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:21:38.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.196+0000 7fa7af7e5640 1 -- 172.21.15.43:0/783322429 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa7940a2c00 msgr2=0x7fa7940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:38.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.196+0000 7fa7af7e5640 1 --2- 172.21.15.43:0/783322429 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa7940a2c00 0x7fa7940a50c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009930 tx=0x7fa7a4008040 comp rx=0 tx=0).stop 2024-09-11T23:21:38.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.196+0000 7fa7af7e5640 1 -- 172.21.15.43:0/783322429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7a8102570 msgr2=0x7fa7a81a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:38.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.196+0000 7fa7af7e5640 1 --2- 172.21.15.43:0/783322429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7a8102570 0x7fa7a81a4ec0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa79800b4a0 tx=0x7fa79800b960 comp rx=0 tx=0).stop 2024-09-11T23:21:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.196+0000 7fa7af7e5640 1 -- 172.21.15.43:0/783322429 shutdown_connections 2024-09-11T23:21:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.196+0000 7fa7af7e5640 1 --2- 172.21.15.43:0/783322429 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa7940a2c00 0x7fa7940a50c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.196+0000 7fa7af7e5640 1 --2- 172.21.15.43:0/783322429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a8102ef0 0x7fa7a819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.196+0000 7fa7af7e5640 1 --2- 172.21.15.43:0/783322429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7a8102570 0x7fa7a81a4ec0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.196+0000 7fa7af7e5640 1 --2- 172.21.15.43:0/783322429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7a8100e20 0x7fa7a81a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.196+0000 7fa7af7e5640 1 -- 172.21.15.43:0/783322429 >> 172.21.15.43:0/783322429 conn(0x7fa7a80fc7d0 msgr2=0x7fa7a806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.196+0000 7fa7af7e5640 1 -- 172.21.15.43:0/783322429 shutdown_connections 2024-09-11T23:21:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:38.196+0000 7fa7af7e5640 1 -- 172.21.15.43:0/783322429 wait complete. 2024-09-11T23:21:38.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:37 smithi090 ceph-mon[35130]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-11T23:21:38.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:37 smithi097 ceph-mon[35562]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-11T23:21:38.910 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T23:20:00.465830+0000', 'modified': '2024-09-11T23:20:12.440327+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24497, 'mds_1': 24470, 'mds_2': 24457}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24436': {'gid': 24436, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6835/1640511441', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 1640511441}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 1640511441}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6837/191447096', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 191447096}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 191447096}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.90:6835/4233140081', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6834', 'nonce': 4233140081}, {'type': 'v1', 'addr': '172.21.15.90:6835', 'nonce': 4233140081}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.97:6839/1424670024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 1424670024}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 1424670024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.90:6837/2946725567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6836', 'nonce': 2946725567}, {'type': 'v1', 'addr': '172.21.15.90:6837', 'nonce': 2946725567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.90:6839/3234895964', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6838', 'nonce': 3234895964}, {'type': 'v1', 'addr': '172.21.15.90:6839', 'nonce': 3234895964}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24497, 'qdb_cluster': [24457, 24470, 24497]} 2024-09-11T23:21:38.910 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-11T23:21:38.910 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T23:21:39.170 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:39.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:38 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/783322429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:21:39.292 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:38 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/783322429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:21:39.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:38 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/783322429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:21:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.657+0000 7f3eb3a49640 1 -- 172.21.15.43:0/990559417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3eac10e480 msgr2=0x7f3eac110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.657+0000 7f3eb3a49640 1 --2- 172.21.15.43:0/990559417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3eac10e480 0x7f3eac110870 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4009ca0 tx=0x7f3ea402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.657+0000 7f3eb3a49640 1 -- 172.21.15.43:0/990559417 shutdown_connections 2024-09-11T23:21:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.657+0000 7f3eb3a49640 1 --2- 172.21.15.43:0/990559417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3eac10e480 0x7f3eac110870 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.657+0000 7f3eb3a49640 1 --2- 172.21.15.43:0/990559417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3eac101280 0x7f3eac10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.657+0000 7f3eb3a49640 1 --2- 172.21.15.43:0/990559417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3eac1008d0 0x7f3eac100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.657+0000 7f3eb3a49640 1 -- 172.21.15.43:0/990559417 >> 172.21.15.43:0/990559417 conn(0x7f3eac0fc7a0 msgr2=0x7f3eac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.657+0000 7f3eb3a49640 1 -- 172.21.15.43:0/990559417 shutdown_connections 2024-09-11T23:21:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.657+0000 7f3eb3a49640 1 -- 172.21.15.43:0/990559417 wait complete. 2024-09-11T23:21:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.658+0000 7f3eb3a49640 1 Processor -- start 2024-09-11T23:21:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.658+0000 7f3eb3a49640 1 -- start start 2024-09-11T23:21:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.658+0000 7f3eb3a49640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3eac1008d0 0x7f3eac19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.658+0000 7f3eb3a49640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3eac101280 0x7f3eac19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.658+0000 7f3eb3a49640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3eac10e480 0x7f3eac19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.658+0000 7f3eb3a49640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3eac114210 con 0x7f3eac1008d0 2024-09-11T23:21:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.658+0000 7f3eb3a49640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3eac114090 con 0x7f3eac101280 2024-09-11T23:21:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.658+0000 7f3eb3a49640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3eac114390 con 0x7f3eac10e480 2024-09-11T23:21:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.658+0000 7f3eb17e5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3eac1008d0 0x7f3eac19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.658+0000 7f3eb17e5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3eac1008d0 0x7f3eac19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46522/0 (socket says 172.21.15.43:46522) 2024-09-11T23:21:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.658+0000 7f3eb17e5640 1 -- 172.21.15.43:0/1985130946 learned_addr learned my addr 172.21.15.43:0/1985130946 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.658+0000 7f3eb1fe6640 1 --2- 172.21.15.43:0/1985130946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3eac10e480 0x7f3eac19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.658+0000 7f3eb0fe4640 1 --2- 172.21.15.43:0/1985130946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3eac101280 0x7f3eac19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.658+0000 7f3eb17e5640 1 -- 172.21.15.43:0/1985130946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3eac10e480 msgr2=0x7f3eac19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.658+0000 7f3eb17e5640 1 --2- 172.21.15.43:0/1985130946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3eac10e480 0x7f3eac19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.658+0000 7f3eb17e5640 1 -- 172.21.15.43:0/1985130946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3eac101280 msgr2=0x7f3eac19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.658+0000 7f3eb17e5640 1 --2- 172.21.15.43:0/1985130946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3eac101280 0x7f3eac19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.658+0000 7f3eb17e5640 1 -- 172.21.15.43:0/1985130946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eac19f640 con 0x7f3eac1008d0 2024-09-11T23:21:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.658+0000 7f3eb0fe4640 1 --2- 172.21.15.43:0/1985130946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3eac101280 0x7f3eac19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:21:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.658+0000 7f3eb1fe6640 1 --2- 172.21.15.43:0/1985130946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3eac10e480 0x7f3eac19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:21:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.658+0000 7f3eb17e5640 1 --2- 172.21.15.43:0/1985130946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3eac1008d0 0x7f3eac19a690 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800c920 tx=0x7f3ea800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.659+0000 7f3ea27fc640 1 -- 172.21.15.43:0/1985130946 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8007c70 con 0x7f3eac1008d0 2024-09-11T23:21:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.659+0000 7f3ea27fc640 1 -- 172.21.15.43:0/1985130946 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ea8007e10 con 0x7f3eac1008d0 2024-09-11T23:21:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.659+0000 7f3eb3a49640 1 -- 172.21.15.43:0/1985130946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eac19f930 con 0x7f3eac1008d0 2024-09-11T23:21:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.659+0000 7f3ea27fc640 1 -- 172.21.15.43:0/1985130946 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8018890 con 0x7f3eac1008d0 2024-09-11T23:21:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.659+0000 7f3eb3a49640 1 -- 172.21.15.43:0/1985130946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eac1a7210 con 0x7f3eac1008d0 2024-09-11T23:21:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.661+0000 7f3ea27fc640 1 -- 172.21.15.43:0/1985130946 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3ea8018a70 con 0x7f3eac1008d0 2024-09-11T23:21:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3eb5e22740 1 Processor -- start 2024-09-11T23:21:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3eb5e22740 1 -- start start 2024-09-11T23:21:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3eb5e22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61229d0 0x556fd6122db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3eb5e22740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556fd61232f0 0x556fd612d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3eb5e22740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd612dbb0 0x556fd612ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3eb5e22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556fd6089090 con 0x556fd61229d0 2024-09-11T23:21:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3eb5e22740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556fd6030a70 con 0x556fd61232f0 2024-09-11T23:21:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3eb5e22740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556fd6085050 con 0x556fd612dbb0 2024-09-11T23:21:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3eb1fe6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61229d0 0x556fd6122db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3ea27fc640 1 --2- 172.21.15.43:0/1985130946 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3e980a2c00 0x7f3e980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3eb1fe6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61229d0 0x556fd6122db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46526/0 (socket says 172.21.15.43:46526) 2024-09-11T23:21:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3eb1fe6640 1 -- 172.21.15.43:0/2301848041 learned_addr learned my addr 172.21.15.43:0/2301848041 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3eb0fe4640 1 --2- 172.21.15.43:0/2301848041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd612dbb0 0x556fd612ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3eb1fe6640 1 --2- 172.21.15.43:0/1985130946 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3e980a2c00 0x7f3e980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3ea27fc640 1 -- 172.21.15.43:0/1985130946 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f3ea80cf230 con 0x7f3eac1008d0 2024-09-11T23:21:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3eb17e5640 1 --2- 172.21.15.43:0/2301848041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556fd61232f0 0x556fd612d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3eb1fe6640 1 -- 172.21.15.43:0/2301848041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd612dbb0 msgr2=0x556fd612ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3eb1fe6640 1 --2- 172.21.15.43:0/2301848041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd612dbb0 0x556fd612ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3eb1fe6640 1 -- 172.21.15.43:0/2301848041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556fd61232f0 msgr2=0x556fd612d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3eb1fe6640 1 --2- 172.21.15.43:0/2301848041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556fd61232f0 0x556fd612d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3eb1fe6640 1 -- 172.21.15.43:0/2301848041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fd603d130 con 0x556fd61229d0 2024-09-11T23:21:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.662+0000 7f3eb0fe4640 1 --2- 172.21.15.43:0/2301848041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd612dbb0 0x556fd612ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.663+0000 7f3eb1fe6640 1 --2- 172.21.15.43:0/2301848041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61229d0 0x556fd6122db0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4005e30 tx=0x7f3ea4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d239f4106ca86666 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.663+0000 7f3eb1fe6640 1 --2- 172.21.15.43:0/1985130946 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3e980a2c00 0x7f3e980a50c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3ea402ade0 tx=0x7f3ea4048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.663+0000 7f3e977fe640 1 -- 172.21.15.43:0/2301848041 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4004040 con 0x556fd61229d0 2024-09-11T23:21:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.663+0000 7f3e977fe640 1 -- 172.21.15.43:0/2301848041 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ea4036070 con 0x556fd61229d0 2024-09-11T23:21:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.663+0000 7f3e977fe640 1 -- 172.21.15.43:0/2301848041 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea402fab0 con 0x556fd61229d0 2024-09-11T23:21:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.663+0000 7f3eb5e22740 1 -- 172.21.15.43:0/2301848041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61229d0 msgr2=0x556fd6122db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.663+0000 7f3eb5e22740 1 --2- 172.21.15.43:0/2301848041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61229d0 0x556fd6122db0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4005e30 tx=0x7f3ea4003970 comp rx=0 tx=0).stop 2024-09-11T23:21:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.664+0000 7f3eb5e22740 1 -- 172.21.15.43:0/2301848041 shutdown_connections 2024-09-11T23:21:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.664+0000 7f3eb5e22740 1 --2- 172.21.15.43:0/2301848041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd612dbb0 0x556fd612ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.664+0000 7f3eb5e22740 1 --2- 172.21.15.43:0/2301848041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556fd61232f0 0x556fd612d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.664+0000 7f3eb5e22740 1 --2- 172.21.15.43:0/2301848041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61229d0 0x556fd6122db0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.664+0000 7f3eb5e22740 1 -- 172.21.15.43:0/2301848041 >> 172.21.15.43:0/2301848041 conn(0x556fd611ef60 msgr2=0x556fd611f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.664+0000 7f3eb5e22740 1 -- 172.21.15.43:0/2301848041 shutdown_connections 2024-09-11T23:21:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.664+0000 7f3eb5e22740 1 -- 172.21.15.43:0/2301848041 wait complete. 2024-09-11T23:21:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.664+0000 7f3eb5e22740 1 -- start start 2024-09-11T23:21:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.664+0000 7f3eb5e22740 1 Processor -- start 2024-09-11T23:21:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.664+0000 7f3eb5e22740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556fd61229d0 0x556fd612d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.664+0000 7f3eb5e22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61232f0 0x556fd612a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.664+0000 7f3eb5e22740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd612dbb0 0x556fd612ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.664+0000 7f3eb5e22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556fd6130720 con 0x556fd61232f0 2024-09-11T23:21:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.664+0000 7f3eb5e22740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556fd6043f20 con 0x556fd61229d0 2024-09-11T23:21:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.664+0000 7f3eb5e22740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556fd61308a0 con 0x556fd612dbb0 2024-09-11T23:21:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.665+0000 7f3eb5e22740 10 client.0 _ll_register_callbacks cb 0x556fd600d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.665+0000 7f3eb0fe4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61232f0 0x556fd612a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.665+0000 7f3eb0fe4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61232f0 0x556fd612a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46542/0 (socket says 172.21.15.43:46542) 2024-09-11T23:21:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.665+0000 7f3eb0fe4640 1 -- 172.21.15.43:0/2903725414 learned_addr learned my addr 172.21.15.43:0/2903725414 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.665+0000 7f3eb17e5640 1 --2- 172.21.15.43:0/2903725414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556fd61229d0 0x556fd612d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.665+0000 7f3eb1fe6640 1 --2- 172.21.15.43:0/2903725414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd612dbb0 0x556fd612ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.665+0000 7f3eb0fe4640 1 -- 172.21.15.43:0/2903725414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd612dbb0 msgr2=0x556fd612ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.665+0000 7f3eb0fe4640 1 --2- 172.21.15.43:0/2903725414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd612dbb0 0x556fd612ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.665+0000 7f3eb0fe4640 1 -- 172.21.15.43:0/2903725414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556fd61229d0 msgr2=0x556fd612d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:39.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.665+0000 7f3eb0fe4640 1 --2- 172.21.15.43:0/2903725414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556fd61229d0 0x556fd612d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.665+0000 7f3eb0fe4640 1 -- 172.21.15.43:0/2903725414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fd5dfa2b0 con 0x556fd61232f0 2024-09-11T23:21:39.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.665+0000 7f3eb1fe6640 1 --2- 172.21.15.43:0/2903725414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd612dbb0 0x556fd612ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:39.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.665+0000 7f3eb0fe4640 1 --2- 172.21.15.43:0/2903725414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61232f0 0x556fd612a7f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00e2a0 tx=0x7f3e9c00e760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98deb4df26f10f75 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:39.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.665+0000 7f3e94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:21:39.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.665+0000 7f3e94ff9640 1 -- 172.21.15.43:0/2903725414 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c018020 con 0x556fd61232f0 2024-09-11T23:21:39.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.665+0000 7f3eb5e22740 1 -- 172.21.15.43:0/2903725414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ea4036070 con 0x556fd61232f0 2024-09-11T23:21:39.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.665+0000 7f3e94ff9640 1 -- 172.21.15.43:0/2903725414 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f3e9c013660 con 0x556fd61232f0 2024-09-11T23:21:39.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.665+0000 7f3e94ff9640 1 -- 172.21.15.43:0/2903725414 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3e9c023050 con 0x556fd61232f0 2024-09-11T23:21:39.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.665+0000 7f3eb5e22740 10 client.15060 fetch_fsmap learned FSMap version 23 2024-09-11T23:21:39.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.666+0000 7f3eb5e22740 1 -- 172.21.15.43:0/2903725414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x556fd613c140 con 0x556fd61232f0 2024-09-11T23:21:39.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.666+0000 7f3e94ff9640 1 -- 172.21.15.43:0/2903725414 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ea4036070 con 0x556fd61232f0 2024-09-11T23:21:39.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.666+0000 7f3e94ff9640 1 -- 172.21.15.43:0/2903725414 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f3e9c014b20 con 0x556fd61232f0 2024-09-11T23:21:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.666+0000 7f3eb5e22740 10 client.15060 fetch_fsmap finished waiting for FSMap version 23 2024-09-11T23:21:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.666+0000 7f3eb5e22740 10 client.15060 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:21:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.666+0000 7f3eb5e22740 20 client.15060 populate_metadata read hostname 'smithi043' 2024-09-11T23:21:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.666+0000 7f3eb5e22740 1 --2- 172.21.15.43:0/2903725414 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x556fd613e800 0x556fd613fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.666+0000 7f3eb5e22740 4 client.15060 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:21:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.666+0000 7f3eb5e22740 1 -- 172.21.15.43:0/2903725414 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fd6096600 con 0x556fd613e800 2024-09-11T23:21:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.666+0000 7f3eb17e5640 1 --2- 172.21.15.43:0/2903725414 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x556fd613e800 0x556fd613fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.667+0000 7f3eb17e5640 1 --2- 172.21.15.43:0/2903725414 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x556fd613e800 0x556fd613fc70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb891a47de173e5e server_cookie=af06c6eb0fc3050 in_seq=0 out_seq=0 2024-09-11T23:21:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.667+0000 7f3e94ff9640 10 client.15060 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:21:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.667+0000 7f3e94ff9640 1 -- 172.21.15.43:0/2903725414 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556fd6096600 con 0x556fd613e800 2024-09-11T23:21:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.667+0000 7f3e94ff9640 10 client.15060 handle_command_reply: tid=0 2024-09-11T23:21:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.667+0000 7f3eb5e22740 1 client.15060 shutdown 2024-09-11T23:21:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.668+0000 7f3eb5e22740 1 -- 172.21.15.43:0/2903725414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61232f0 msgr2=0x556fd612a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:39.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.668+0000 7f3eb5e22740 1 --2- 172.21.15.43:0/2903725414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61232f0 0x556fd612a7f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00e2a0 tx=0x7f3e9c00e760 comp rx=0 tx=0).stop 2024-09-11T23:21:39.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.668+0000 7f3eb5e22740 1 -- 172.21.15.43:0/2903725414 shutdown_connections 2024-09-11T23:21:39.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.668+0000 7f3eb5e22740 1 --2- 172.21.15.43:0/2903725414 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x556fd613e800 0x556fd613fc70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.668+0000 7f3eb5e22740 1 --2- 172.21.15.43:0/2903725414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd612dbb0 0x556fd612ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.668+0000 7f3eb5e22740 1 --2- 172.21.15.43:0/2903725414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61232f0 0x556fd612a7f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.668+0000 7f3eb5e22740 1 --2- 172.21.15.43:0/2903725414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556fd61229d0 0x556fd612d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.668+0000 7f3eb5e22740 1 -- 172.21.15.43:0/2903725414 >> 172.21.15.43:0/2903725414 conn(0x556fd611ef60 msgr2=0x556fd6124c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:39.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.668+0000 7f3e94ff9640 0 client.15060 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:21:39.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.668+0000 7f3eb5e22740 1 -- 172.21.15.43:0/2903725414 shutdown_connections 2024-09-11T23:21:39.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.668+0000 7f3eb5e22740 1 -- 172.21.15.43:0/2903725414 wait complete. 2024-09-11T23:21:39.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.668+0000 7f3eb5e22740 20 client.15060 trim_cache size 0 max 16384 2024-09-11T23:21:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.686+0000 7f3eb5e22740 1 Processor -- start 2024-09-11T23:21:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.686+0000 7f3eb5e22740 1 -- start start 2024-09-11T23:21:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.686+0000 7f3eb5e22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61229d0 0x556fd613e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.686+0000 7f3eb5e22740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd61232f0 0x556fd6147f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.686+0000 7f3eb5e22740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556fd612dbb0 0x556fd6148490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.686+0000 7f3eb5e22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556fd613dfb0 con 0x556fd61229d0 2024-09-11T23:21:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.686+0000 7f3eb5e22740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556fd613e680 con 0x556fd612dbb0 2024-09-11T23:21:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.686+0000 7f3eb5e22740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556fd613ed50 con 0x556fd61232f0 2024-09-11T23:21:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.686+0000 7f3eb17e5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61229d0 0x556fd613e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.686+0000 7f3eb17e5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61229d0 0x556fd613e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46544/0 (socket says 172.21.15.43:46544) 2024-09-11T23:21:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.686+0000 7f3eb17e5640 1 -- 172.21.15.43:0/3650115978 learned_addr learned my addr 172.21.15.43:0/3650115978 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.687+0000 7f3eb0fe4640 1 --2- 172.21.15.43:0/3650115978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd61232f0 0x556fd6147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.687+0000 7f3eb17e5640 1 -- 172.21.15.43:0/3650115978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd61232f0 msgr2=0x556fd6147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.687+0000 7f3eb17e5640 1 --2- 172.21.15.43:0/3650115978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd61232f0 0x556fd6147f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.687+0000 7f3eb17e5640 1 -- 172.21.15.43:0/3650115978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556fd612dbb0 msgr2=0x556fd6148490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.687+0000 7f3eb17e5640 1 --2- 172.21.15.43:0/3650115978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556fd612dbb0 0x556fd6148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.687+0000 7f3eb17e5640 1 -- 172.21.15.43:0/3650115978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fd6128d40 con 0x556fd61229d0 2024-09-11T23:21:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.687+0000 7f3eb17e5640 1 --2- 172.21.15.43:0/3650115978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61229d0 0x556fd613e800 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80ccd60 tx=0x7f3ea80d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a594c13cfc0a3a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.687+0000 7f3e96ffd640 1 -- 172.21.15.43:0/3650115978 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea80b8020 con 0x556fd61229d0 2024-09-11T23:21:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.687+0000 7f3e96ffd640 1 -- 172.21.15.43:0/3650115978 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ea80cce20 con 0x556fd61229d0 2024-09-11T23:21:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.687+0000 7f3e96ffd640 1 -- 172.21.15.43:0/3650115978 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea80980e0 con 0x556fd61229d0 2024-09-11T23:21:39.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.688+0000 7f3eb5e22740 1 -- 172.21.15.43:0/3650115978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61229d0 msgr2=0x556fd613e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:39.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.688+0000 7f3eb5e22740 1 --2- 172.21.15.43:0/3650115978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61229d0 0x556fd613e800 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80ccd60 tx=0x7f3ea80d1040 comp rx=0 tx=0).stop 2024-09-11T23:21:39.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.688+0000 7f3eb5e22740 1 -- 172.21.15.43:0/3650115978 shutdown_connections 2024-09-11T23:21:39.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.688+0000 7f3eb5e22740 1 --2- 172.21.15.43:0/3650115978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556fd612dbb0 0x556fd6148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.688+0000 7f3eb5e22740 1 --2- 172.21.15.43:0/3650115978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd61232f0 0x556fd6147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.688+0000 7f3eb5e22740 1 --2- 172.21.15.43:0/3650115978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61229d0 0x556fd613e800 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.688+0000 7f3eb5e22740 1 -- 172.21.15.43:0/3650115978 >> 172.21.15.43:0/3650115978 conn(0x556fd611ef60 msgr2=0x556fd61290c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.688+0000 7f3eb5e22740 1 -- 172.21.15.43:0/3650115978 shutdown_connections 2024-09-11T23:21:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.688+0000 7f3eb5e22740 1 -- 172.21.15.43:0/3650115978 wait complete. 2024-09-11T23:21:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.688+0000 7f3eb5e22740 1 -- start start 2024-09-11T23:21:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.688+0000 7f3eb5e22740 1 Processor -- start 2024-09-11T23:21:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.689+0000 7f3eb5e22740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd61229d0 0x556fd6143470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.689+0000 7f3eb5e22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61232f0 0x556fd61439b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.689+0000 7f3eb5e22740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556fd612dbb0 0x556fd61a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.689+0000 7f3eb5e22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556fd613faf0 con 0x556fd61232f0 2024-09-11T23:21:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.689+0000 7f3eb5e22740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556fd613f420 con 0x556fd612dbb0 2024-09-11T23:21:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.689+0000 7f3eb5e22740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556fd61401c0 con 0x556fd61229d0 2024-09-11T23:21:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.689+0000 7f3eb5e22740 10 client.0 _ll_register_callbacks cb 0x7f3e9c00c170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.689+0000 7f3eb0fe4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61232f0 0x556fd61439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.689+0000 7f3eb0fe4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61232f0 0x556fd61439b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46550/0 (socket says 172.21.15.43:46550) 2024-09-11T23:21:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.689+0000 7f3eb0fe4640 1 -- 172.21.15.43:0/3352407169 learned_addr learned my addr 172.21.15.43:0/3352407169 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.689+0000 7f3eb17e5640 1 --2- 172.21.15.43:0/3352407169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd61229d0 0x556fd6143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.689+0000 7f3eb1fe6640 1 --2- 172.21.15.43:0/3352407169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556fd612dbb0 0x556fd61a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.689+0000 7f3eb0fe4640 1 -- 172.21.15.43:0/3352407169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd61229d0 msgr2=0x556fd6143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.689+0000 7f3eb0fe4640 1 --2- 172.21.15.43:0/3352407169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd61229d0 0x556fd6143470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.689+0000 7f3eb0fe4640 1 -- 172.21.15.43:0/3352407169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556fd612dbb0 msgr2=0x556fd61a6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.689+0000 7f3eb0fe4640 1 --2- 172.21.15.43:0/3352407169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556fd612dbb0 0x556fd61a6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.689+0000 7f3eb0fe4640 1 -- 172.21.15.43:0/3352407169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fd61a7540 con 0x556fd61232f0 2024-09-11T23:21:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.689+0000 7f3eb1fe6640 1 --2- 172.21.15.43:0/3352407169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556fd612dbb0 0x556fd61a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.689+0000 7f3eb0fe4640 1 --2- 172.21.15.43:0/3352407169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61232f0 0x556fd61439b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x556fd612be30 tx=0x7f3e9c0138f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d186a230772011 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.689+0000 7f3e94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:21:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.690+0000 7f3e94ff9640 1 -- 172.21.15.43:0/3352407169 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c018040 con 0x556fd61232f0 2024-09-11T23:21:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.690+0000 7f3e94ff9640 1 -- 172.21.15.43:0/3352407169 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f3e9c014400 con 0x556fd61232f0 2024-09-11T23:21:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.690+0000 7f3eb5e22740 1 -- 172.21.15.43:0/3352407169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ea80cce20 con 0x556fd61232f0 2024-09-11T23:21:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.690+0000 7f3e94ff9640 1 -- 172.21.15.43:0/3352407169 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3e9c00ec60 con 0x556fd61232f0 2024-09-11T23:21:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.690+0000 7f3eb5e22740 10 client.15066 fetch_fsmap learned FSMap version 23 2024-09-11T23:21:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.690+0000 7f3eb5e22740 1 -- 172.21.15.43:0/3352407169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x556fd61a8800 con 0x556fd61232f0 2024-09-11T23:21:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.690+0000 7f3e94ff9640 1 -- 172.21.15.43:0/3352407169 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ea80cce20 con 0x556fd61232f0 2024-09-11T23:21:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.690+0000 7f3e94ff9640 1 -- 172.21.15.43:0/3352407169 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f3e9c006020 con 0x556fd61232f0 2024-09-11T23:21:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.690+0000 7f3eb5e22740 10 client.15066 fetch_fsmap finished waiting for FSMap version 23 2024-09-11T23:21:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.690+0000 7f3eb5e22740 10 client.15066 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:21:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.690+0000 7f3eb5e22740 20 client.15066 populate_metadata read hostname 'smithi043' 2024-09-11T23:21:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.690+0000 7f3eb5e22740 1 --2- 172.21.15.43:0/3352407169 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x556fd61ad3d0 0x556fd61ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.690+0000 7f3eb5e22740 4 client.15066 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:21:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.690+0000 7f3eb5e22740 1 -- 172.21.15.43:0/3352407169 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "status"}) -- 0x556fd6090b10 con 0x556fd61ad3d0 2024-09-11T23:21:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.691+0000 7f3eb17e5640 1 --2- 172.21.15.43:0/3352407169 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x556fd61ad3d0 0x556fd61ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.691+0000 7f3eb17e5640 1 --2- 172.21.15.43:0/3352407169 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x556fd61ad3d0 0x556fd61ac0e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ce841ff34ea3841 server_cookie=ba4381e562fec917 in_seq=0 out_seq=0 2024-09-11T23:21:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.691+0000 7f3e94ff9640 10 client.15066 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:21:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.691+0000 7f3e94ff9640 1 -- 172.21.15.43:0/3352407169 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x556fd6090b10 con 0x556fd61ad3d0 2024-09-11T23:21:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.691+0000 7f3e94ff9640 10 client.15066 handle_command_reply: tid=0 2024-09-11T23:21:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.691+0000 7f3eb5e22740 1 client.15066 shutdown 2024-09-11T23:21:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.691+0000 7f3eb5e22740 1 -- 172.21.15.43:0/3352407169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61232f0 msgr2=0x556fd61439b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.691+0000 7f3eb5e22740 1 --2- 172.21.15.43:0/3352407169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61232f0 0x556fd61439b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x556fd612be30 tx=0x7f3e9c0138f0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.691+0000 7f3eb5e22740 1 -- 172.21.15.43:0/3352407169 shutdown_connections 2024-09-11T23:21:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.691+0000 7f3eb5e22740 1 --2- 172.21.15.43:0/3352407169 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x556fd61ad3d0 0x556fd61ac0e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.691+0000 7f3eb5e22740 1 --2- 172.21.15.43:0/3352407169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556fd612dbb0 0x556fd61a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.691+0000 7f3eb5e22740 1 --2- 172.21.15.43:0/3352407169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556fd61232f0 0x556fd61439b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.691+0000 7f3eb5e22740 1 --2- 172.21.15.43:0/3352407169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556fd61229d0 0x556fd6143470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.691+0000 7f3eb5e22740 1 -- 172.21.15.43:0/3352407169 >> 172.21.15.43:0/3352407169 conn(0x556fd611ef60 msgr2=0x556fd612be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:39.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.691+0000 7f3e94ff9640 0 client.15066 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:21:39.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.692+0000 7f3eb5e22740 1 -- 172.21.15.43:0/3352407169 shutdown_connections 2024-09-11T23:21:39.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.692+0000 7f3eb5e22740 1 -- 172.21.15.43:0/3352407169 wait complete. 2024-09-11T23:21:39.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.692+0000 7f3eb5e22740 20 client.15066 trim_cache size 0 max 16384 2024-09-11T23:21:39.698 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:21:39.698 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "25c0cd84-7093-11ef-bcea-c7b262605968", 2024-09-11T23:21:39.698 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 2, 2024-09-11T23:21:39.698 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24457, 2024-09-11T23:21:39.698 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-11T23:21:39.699 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-11T23:21:39.699 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-11T23:21:39.699 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 89.973694567999999, 2024-09-11T23:21:39.699 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 23, 2024-09-11T23:21:39.699 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 84, 2024-09-11T23:21:39.699 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 82, 2024-09-11T23:21:39.699 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 119.21818258 2024-09-11T23:21:39.699 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:21:39.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.692+0000 7f3eb3a49640 1 -- 172.21.15.43:0/1985130946 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3e980a2c00 msgr2=0x7f3e980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:39.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.692+0000 7f3eb3a49640 1 --2- 172.21.15.43:0/1985130946 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3e980a2c00 0x7f3e980a50c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3ea402ade0 tx=0x7f3ea4048000 comp rx=0 tx=0).stop 2024-09-11T23:21:39.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.692+0000 7f3eb3a49640 1 -- 172.21.15.43:0/1985130946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3eac1008d0 msgr2=0x7f3eac19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:39.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.692+0000 7f3eb3a49640 1 --2- 172.21.15.43:0/1985130946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3eac1008d0 0x7f3eac19a690 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800c920 tx=0x7f3ea800cde0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.692+0000 7f3eb3a49640 1 -- 172.21.15.43:0/1985130946 shutdown_connections 2024-09-11T23:21:39.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.692+0000 7f3eb3a49640 1 --2- 172.21.15.43:0/1985130946 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f3e980a2c00 0x7f3e980a50c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.692+0000 7f3eb3a49640 1 --2- 172.21.15.43:0/1985130946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3eac10e480 0x7f3eac19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.692+0000 7f3eb3a49640 1 --2- 172.21.15.43:0/1985130946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3eac101280 0x7f3eac19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.692+0000 7f3eb3a49640 1 --2- 172.21.15.43:0/1985130946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3eac1008d0 0x7f3eac19a690 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:39.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.692+0000 7f3eb3a49640 1 -- 172.21.15.43:0/1985130946 >> 172.21.15.43:0/1985130946 conn(0x7f3eac0fc7a0 msgr2=0x7f3eac0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:39.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.692+0000 7f3eb3a49640 1 -- 172.21.15.43:0/1985130946 shutdown_connections 2024-09-11T23:21:39.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:39.692+0000 7f3eb3a49640 1 -- 172.21.15.43:0/1985130946 wait complete. 2024-09-11T23:21:40.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:39 smithi090 ceph-mon[35130]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-11T23:21:40.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:39 smithi097 ceph-mon[35562]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-11T23:21:40.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:39 smithi043 ceph-mon[30950]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-11T23:21:40.580 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T23:21:40.841 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:41.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.365+0000 7f9899c39640 1 -- 172.21.15.43:0/1940569074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9894106a80 msgr2=0x7f9894106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:41.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.365+0000 7f9899c39640 1 --2- 172.21.15.43:0/1940569074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9894106a80 0x7f9894106e60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9884009930 tx=0x7f988402a420 comp rx=0 tx=0).stop 2024-09-11T23:21:41.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.365+0000 7f9899c39640 1 -- 172.21.15.43:0/1940569074 shutdown_connections 2024-09-11T23:21:41.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.365+0000 7f9899c39640 1 --2- 172.21.15.43:0/1940569074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9894101420 0x7f989410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.365+0000 7f9899c39640 1 --2- 172.21.15.43:0/1940569074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9894100a80 0x7f9894100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.365+0000 7f9899c39640 1 --2- 172.21.15.43:0/1940569074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9894106a80 0x7f9894106e60 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.365+0000 7f9899c39640 1 -- 172.21.15.43:0/1940569074 >> 172.21.15.43:0/1940569074 conn(0x7f98940fc7a0 msgr2=0x7f98940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:41.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.365+0000 7f9899c39640 1 -- 172.21.15.43:0/1940569074 shutdown_connections 2024-09-11T23:21:41.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.365+0000 7f9899c39640 1 -- 172.21.15.43:0/1940569074 wait complete. 2024-09-11T23:21:41.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.366+0000 7f9899c39640 1 Processor -- start 2024-09-11T23:21:41.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.366+0000 7f9899c39640 1 -- start start 2024-09-11T23:21:41.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.366+0000 7f9899c39640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9894100a80 0x7f989419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:41.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.366+0000 7f9899c39640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9894101420 0x7f989419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:41.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.366+0000 7f9899c39640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9894106a80 0x7f989419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:41.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.366+0000 7f9899c39640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9894110e80 con 0x7f9894101420 2024-09-11T23:21:41.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.366+0000 7f9899c39640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9894110d00 con 0x7f9894100a80 2024-09-11T23:21:41.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.366+0000 7f9899c39640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9894111000 con 0x7f9894106a80 2024-09-11T23:21:41.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.366+0000 7f9892ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9894101420 0x7f989419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:41.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.366+0000 7f9892ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9894101420 0x7f989419ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46604/0 (socket says 172.21.15.43:46604) 2024-09-11T23:21:41.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.366+0000 7f9892ffd640 1 -- 172.21.15.43:0/2388538745 learned_addr learned my addr 172.21.15.43:0/2388538745 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:41.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.367+0000 7f98937fe640 1 --2- 172.21.15.43:0/2388538745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9894100a80 0x7f989419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:41.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.367+0000 7f9892ffd640 1 -- 172.21.15.43:0/2388538745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9894106a80 msgr2=0x7f989419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:41.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.367+0000 7f9892ffd640 1 --2- 172.21.15.43:0/2388538745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9894106a80 0x7f989419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.367+0000 7f9892ffd640 1 -- 172.21.15.43:0/2388538745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9894100a80 msgr2=0x7f989419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:41.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.367+0000 7f9892ffd640 1 --2- 172.21.15.43:0/2388538745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9894100a80 0x7f989419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.367+0000 7f9892ffd640 1 -- 172.21.15.43:0/2388538745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989419f6c0 con 0x7f9894101420 2024-09-11T23:21:41.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.367+0000 7f98937fe640 1 --2- 172.21.15.43:0/2388538745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9894100a80 0x7f989419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:21:41.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.367+0000 7f9892ffd640 1 --2- 172.21.15.43:0/2388538745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9894101420 0x7f989419ac50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f988800d950 tx=0x7f988800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:41.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.367+0000 7f9890ff9640 1 -- 172.21.15.43:0/2388538745 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888017070 con 0x7f9894101420 2024-09-11T23:21:41.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.367+0000 7f9899c39640 1 -- 172.21.15.43:0/2388538745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f989419f9b0 con 0x7f9894101420 2024-09-11T23:21:41.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.367+0000 7f9890ff9640 1 -- 172.21.15.43:0/2388538745 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9888009c30 con 0x7f9894101420 2024-09-11T23:21:41.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.367+0000 7f9890ff9640 1 -- 172.21.15.43:0/2388538745 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988800f6b0 con 0x7f9894101420 2024-09-11T23:21:41.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.367+0000 7f9899c39640 1 -- 172.21.15.43:0/2388538745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98941a7290 con 0x7f9894101420 2024-09-11T23:21:41.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.369+0000 7f9890ff9640 1 -- 172.21.15.43:0/2388538745 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f98880109d0 con 0x7f9894101420 2024-09-11T23:21:41.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.370+0000 7f989c012740 1 Processor -- start 2024-09-11T23:21:41.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.370+0000 7f989c012740 1 -- start start 2024-09-11T23:21:41.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.370+0000 7f989c012740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854fdf9d0 0x561854fdfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:41.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.370+0000 7f989c012740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561854fe02f0 0x561854fea670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:41.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.370+0000 7f989c012740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561854feabb0 0x561854fecfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:41.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.370+0000 7f989c012740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561854f46090 con 0x561854fdf9d0 2024-09-11T23:21:41.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.370+0000 7f989c012740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561854eeda70 con 0x561854fe02f0 2024-09-11T23:21:41.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.370+0000 7f989c012740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561854f42050 con 0x561854feabb0 2024-09-11T23:21:41.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.370+0000 7f9890ff9640 1 --2- 172.21.15.43:0/2388538745 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f987c0a2b80 0x7f987c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:41.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.370+0000 7f9893fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854fdf9d0 0x561854fdfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:41.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.370+0000 7f9893fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854fdf9d0 0x561854fdfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46612/0 (socket says 172.21.15.43:46612) 2024-09-11T23:21:41.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.370+0000 7f9893fff640 1 -- 172.21.15.43:0/2368721613 learned_addr learned my addr 172.21.15.43:0/2368721613 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:41.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.370+0000 7f9890ff9640 1 -- 172.21.15.43:0/2388538745 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f98880cede0 con 0x7f9894101420 2024-09-11T23:21:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.370+0000 7f9893fff640 1 --2- 172.21.15.43:0/2388538745 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f987c0a2b80 0x7f987c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.370+0000 7f9892ffd640 1 --2- 172.21.15.43:0/2368721613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561854feabb0 0x561854fecfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.370+0000 7f98937fe640 1 --2- 172.21.15.43:0/2368721613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561854fe02f0 0x561854fea670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.370+0000 7f9893fff640 1 -- 172.21.15.43:0/2368721613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561854feabb0 msgr2=0x561854fecfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.370+0000 7f9893fff640 1 --2- 172.21.15.43:0/2368721613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561854feabb0 0x561854fecfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.370+0000 7f9893fff640 1 -- 172.21.15.43:0/2368721613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561854fe02f0 msgr2=0x561854fea670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.370+0000 7f9893fff640 1 --2- 172.21.15.43:0/2368721613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561854fe02f0 0x561854fea670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.370+0000 7f9893fff640 1 -- 172.21.15.43:0/2368721613 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561854efa130 con 0x561854fdf9d0 2024-09-11T23:21:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.371+0000 7f9893fff640 1 --2- 172.21.15.43:0/2368721613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854fdf9d0 0x561854fdfdb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f988c013850 tx=0x7f988c013d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1891c275b0dd1c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.371+0000 7f9879ffb640 1 -- 172.21.15.43:0/2368721613 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c020070 con 0x561854fdf9d0 2024-09-11T23:21:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.371+0000 7f9879ffb640 1 -- 172.21.15.43:0/2368721613 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f988c00d8a0 con 0x561854fdf9d0 2024-09-11T23:21:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.371+0000 7f9879ffb640 1 -- 172.21.15.43:0/2368721613 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c00e5c0 con 0x561854fdf9d0 2024-09-11T23:21:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.371+0000 7f989c012740 1 -- 172.21.15.43:0/2368721613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854fdf9d0 msgr2=0x561854fdfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.371+0000 7f9893fff640 1 --2- 172.21.15.43:0/2388538745 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f987c0a2b80 0x7f987c0a5040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f988c01d000 tx=0x7f988c019000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.372+0000 7f989c012740 1 --2- 172.21.15.43:0/2368721613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854fdf9d0 0x561854fdfdb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f988c013850 tx=0x7f988c013d10 comp rx=0 tx=0).stop 2024-09-11T23:21:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.372+0000 7f989c012740 1 -- 172.21.15.43:0/2368721613 shutdown_connections 2024-09-11T23:21:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.372+0000 7f989c012740 1 --2- 172.21.15.43:0/2368721613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561854feabb0 0x561854fecfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.372+0000 7f989c012740 1 --2- 172.21.15.43:0/2368721613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561854fe02f0 0x561854fea670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.372+0000 7f989c012740 1 --2- 172.21.15.43:0/2368721613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854fdf9d0 0x561854fdfdb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.372+0000 7f989c012740 1 -- 172.21.15.43:0/2368721613 >> 172.21.15.43:0/2368721613 conn(0x561854fdbf60 msgr2=0x561854fdc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.372+0000 7f989c012740 1 -- 172.21.15.43:0/2368721613 shutdown_connections 2024-09-11T23:21:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.372+0000 7f989c012740 1 -- 172.21.15.43:0/2368721613 wait complete. 2024-09-11T23:21:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.372+0000 7f989c012740 1 -- start start 2024-09-11T23:21:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.372+0000 7f989c012740 1 Processor -- start 2024-09-11T23:21:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.373+0000 7f989c012740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561854fdf9d0 0x561854fea430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.373+0000 7f989c012740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561854fe02f0 0x561854fe7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:41.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.373+0000 7f989c012740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854feabb0 0x561854fe7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.373+0000 7f989c012740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561854fed780 con 0x561854feabb0 2024-09-11T23:21:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.373+0000 7f989c012740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561854f00f20 con 0x561854fdf9d0 2024-09-11T23:21:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.373+0000 7f989c012740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561854fed900 con 0x561854fe02f0 2024-09-11T23:21:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.373+0000 7f989c012740 10 client.0 _ll_register_callbacks cb 0x561854eca030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.373+0000 7f9893fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854feabb0 0x561854fe7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.373+0000 7f9893fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854feabb0 0x561854fe7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46620/0 (socket says 172.21.15.43:46620) 2024-09-11T23:21:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.373+0000 7f9893fff640 1 -- 172.21.15.43:0/578844048 learned_addr learned my addr 172.21.15.43:0/578844048 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.373+0000 7f98937fe640 1 --2- 172.21.15.43:0/578844048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561854fdf9d0 0x561854fea430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.373+0000 7f9892ffd640 1 --2- 172.21.15.43:0/578844048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561854fe02f0 0x561854fe7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.373+0000 7f9893fff640 1 -- 172.21.15.43:0/578844048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561854fe02f0 msgr2=0x561854fe7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.373+0000 7f9893fff640 1 --2- 172.21.15.43:0/578844048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561854fe02f0 0x561854fe7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.373+0000 7f9893fff640 1 -- 172.21.15.43:0/578844048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561854fdf9d0 msgr2=0x561854fea430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.373+0000 7f9893fff640 1 --2- 172.21.15.43:0/578844048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561854fdf9d0 0x561854fea430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.373+0000 7f9893fff640 1 -- 172.21.15.43:0/578844048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561854cb72b0 con 0x561854feabb0 2024-09-11T23:21:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.373+0000 7f9893fff640 1 --2- 172.21.15.43:0/578844048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854feabb0 0x561854fe7cc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f988c007fa0 tx=0x7f988c007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d4626cdbc39c39 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.373+0000 7f9872ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:21:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.373+0000 7f9872ffd640 1 -- 172.21.15.43:0/578844048 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c020050 con 0x561854feabb0 2024-09-11T23:21:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.374+0000 7f989c012740 1 -- 172.21.15.43:0/578844048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f988c00d8a0 con 0x561854feabb0 2024-09-11T23:21:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.374+0000 7f9872ffd640 1 -- 172.21.15.43:0/578844048 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f988c00d5c0 con 0x561854feabb0 2024-09-11T23:21:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.374+0000 7f9872ffd640 1 -- 172.21.15.43:0/578844048 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f988c004b60 con 0x561854feabb0 2024-09-11T23:21:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.374+0000 7f989c012740 10 client.15078 fetch_fsmap learned FSMap version 23 2024-09-11T23:21:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.374+0000 7f989c012740 1 -- 172.21.15.43:0/578844048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561854ff9090 con 0x561854feabb0 2024-09-11T23:21:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.374+0000 7f9872ffd640 1 -- 172.21.15.43:0/578844048 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f988c00d8a0 con 0x561854feabb0 2024-09-11T23:21:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.374+0000 7f9872ffd640 1 -- 172.21.15.43:0/578844048 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f988c00f040 con 0x561854feabb0 2024-09-11T23:21:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.374+0000 7f989c012740 10 client.15078 fetch_fsmap finished waiting for FSMap version 23 2024-09-11T23:21:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.374+0000 7f989c012740 10 client.15078 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T23:21:41.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.374+0000 7f989c012740 20 client.15078 populate_metadata read hostname 'smithi043' 2024-09-11T23:21:41.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.374+0000 7f989c012740 1 --2- 172.21.15.43:0/578844048 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x561854ffb560 0x561854ffc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:41.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.374+0000 7f989c012740 4 client.15078 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:21:41.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.374+0000 7f989c012740 1 -- 172.21.15.43:0/578844048 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561854f53600 con 0x561854ffb560 2024-09-11T23:21:41.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.374+0000 7f98937fe640 1 --2- 172.21.15.43:0/578844048 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x561854ffb560 0x561854ffc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:41.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.375+0000 7f9872ffd640 10 client.15078 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:21:41.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.375+0000 7f98937fe640 1 --2- 172.21.15.43:0/578844048 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x561854ffb560 0x561854ffc9d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d55e10f8cfeaec8 server_cookie=18b4aad597a4fd35 in_seq=0 out_seq=0 2024-09-11T23:21:41.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.375+0000 7f9872ffd640 1 -- 172.21.15.43:0/578844048 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561854f53600 con 0x561854ffb560 2024-09-11T23:21:41.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.376+0000 7f9872ffd640 10 client.15078 handle_command_reply: tid=0 2024-09-11T23:21:41.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.376+0000 7f989c012740 1 client.15078 shutdown 2024-09-11T23:21:41.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.376+0000 7f989c012740 1 -- 172.21.15.43:0/578844048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854feabb0 msgr2=0x561854fe7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:41.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.376+0000 7f989c012740 1 --2- 172.21.15.43:0/578844048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854feabb0 0x561854fe7cc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f988c007fa0 tx=0x7f988c007dd0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.376+0000 7f989c012740 1 -- 172.21.15.43:0/578844048 shutdown_connections 2024-09-11T23:21:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.376+0000 7f989c012740 1 --2- 172.21.15.43:0/578844048 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x561854ffb560 0x561854ffc9d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.376+0000 7f989c012740 1 --2- 172.21.15.43:0/578844048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854feabb0 0x561854fe7cc0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.376+0000 7f989c012740 1 --2- 172.21.15.43:0/578844048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561854fe02f0 0x561854fe7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.376+0000 7f989c012740 1 --2- 172.21.15.43:0/578844048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561854fdf9d0 0x561854fea430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.376+0000 7f9872ffd640 0 client.15078 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:21:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.376+0000 7f989c012740 1 -- 172.21.15.43:0/578844048 >> 172.21.15.43:0/578844048 conn(0x561854fdbf60 msgr2=0x561854fe1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.376+0000 7f989c012740 1 -- 172.21.15.43:0/578844048 shutdown_connections 2024-09-11T23:21:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.376+0000 7f989c012740 1 -- 172.21.15.43:0/578844048 wait complete. 2024-09-11T23:21:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.376+0000 7f989c012740 20 client.15078 trim_cache size 0 max 16384 2024-09-11T23:21:41.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.394+0000 7f989c012740 1 Processor -- start 2024-09-11T23:21:41.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.394+0000 7f989c012740 1 -- start start 2024-09-11T23:21:41.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.394+0000 7f989c012740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854fdf9d0 0x561854fe95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:41.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.394+0000 7f989c012740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561854fe02f0 0x561854fe49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:41.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.394+0000 7f989c012740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561854feabb0 0x561854fe4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:41.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.394+0000 7f989c012740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561854ffad10 con 0x561854fdf9d0 2024-09-11T23:21:41.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.394+0000 7f989c012740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561854ffb3e0 con 0x561854feabb0 2024-09-11T23:21:41.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.394+0000 7f989c012740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561854ffbab0 con 0x561854fe02f0 2024-09-11T23:21:41.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.395+0000 7f98937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854fdf9d0 0x561854fe95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:41.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.395+0000 7f98937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854fdf9d0 0x561854fe95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46630/0 (socket says 172.21.15.43:46630) 2024-09-11T23:21:41.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.395+0000 7f98937fe640 1 -- 172.21.15.43:0/3611210424 learned_addr learned my addr 172.21.15.43:0/3611210424 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:41.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.395+0000 7f9893fff640 1 --2- 172.21.15.43:0/3611210424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561854feabb0 0x561854fe4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:41.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.395+0000 7f98937fe640 1 -- 172.21.15.43:0/3611210424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561854fe02f0 msgr2=0x561854fe49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:41.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.395+0000 7f9892ffd640 1 --2- 172.21.15.43:0/3611210424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561854fe02f0 0x561854fe49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:41.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.395+0000 7f98937fe640 1 --2- 172.21.15.43:0/3611210424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561854fe02f0 0x561854fe49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.395+0000 7f98937fe640 1 -- 172.21.15.43:0/3611210424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561854feabb0 msgr2=0x561854fe4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.395+0000 7f98937fe640 1 --2- 172.21.15.43:0/3611210424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561854feabb0 0x561854fe4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.395+0000 7f98937fe640 1 -- 172.21.15.43:0/3611210424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561854fe8540 con 0x561854fdf9d0 2024-09-11T23:21:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.395+0000 7f9893fff640 1 --2- 172.21.15.43:0/3611210424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561854feabb0 0x561854fe4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.395+0000 7f98937fe640 1 --2- 172.21.15.43:0/3611210424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854fdf9d0 0x561854fe95e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9884009930 tx=0x7f9884031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda8353d65e1b29a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.395+0000 7f98797fa640 1 -- 172.21.15.43:0/3611210424 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884035040 con 0x561854fdf9d0 2024-09-11T23:21:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.395+0000 7f98797fa640 1 -- 172.21.15.43:0/3611210424 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9884005520 con 0x561854fdf9d0 2024-09-11T23:21:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.395+0000 7f98797fa640 1 -- 172.21.15.43:0/3611210424 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884004d20 con 0x561854fdf9d0 2024-09-11T23:21:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.396+0000 7f989c012740 1 -- 172.21.15.43:0/3611210424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854fdf9d0 msgr2=0x561854fe95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.396+0000 7f989c012740 1 --2- 172.21.15.43:0/3611210424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854fdf9d0 0x561854fe95e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9884009930 tx=0x7f9884031510 comp rx=0 tx=0).stop 2024-09-11T23:21:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.396+0000 7f989c012740 1 -- 172.21.15.43:0/3611210424 shutdown_connections 2024-09-11T23:21:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.396+0000 7f989c012740 1 --2- 172.21.15.43:0/3611210424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561854feabb0 0x561854fe4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.396+0000 7f989c012740 1 --2- 172.21.15.43:0/3611210424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561854fe02f0 0x561854fe49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.396+0000 7f989c012740 1 --2- 172.21.15.43:0/3611210424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854fdf9d0 0x561854fe95e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.396+0000 7f989c012740 1 -- 172.21.15.43:0/3611210424 >> 172.21.15.43:0/3611210424 conn(0x561854fdbf60 msgr2=0x561854fe8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.396+0000 7f989c012740 1 -- 172.21.15.43:0/3611210424 shutdown_connections 2024-09-11T23:21:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.396+0000 7f989c012740 1 -- 172.21.15.43:0/3611210424 wait complete. 2024-09-11T23:21:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.396+0000 7f989c012740 1 -- start start 2024-09-11T23:21:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.397+0000 7f989c012740 1 Processor -- start 2024-09-11T23:21:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.397+0000 7f989c012740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854fdf9d0 0x561855000010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.397+0000 7f989c012740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561854fe02f0 0x561855000550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.397+0000 7f989c012740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561854feabb0 0x561855063c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.397+0000 7f989c012740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561854ffc850 con 0x561854fdf9d0 2024-09-11T23:21:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.397+0000 7f989c012740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561854ffc180 con 0x561854feabb0 2024-09-11T23:21:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.397+0000 7f989c012740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561854ffcf20 con 0x561854fe02f0 2024-09-11T23:21:41.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.397+0000 7f989c012740 10 client.0 _ll_register_callbacks cb 0x7f988c01e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:41.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.397+0000 7f98937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854fdf9d0 0x561855000010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:41.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.397+0000 7f98937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854fdf9d0 0x561855000010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46642/0 (socket says 172.21.15.43:46642) 2024-09-11T23:21:41.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.397+0000 7f98937fe640 1 -- 172.21.15.43:0/777621104 learned_addr learned my addr 172.21.15.43:0/777621104 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:41.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.397+0000 7f9893fff640 1 --2- 172.21.15.43:0/777621104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561854feabb0 0x561855063c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:41.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.397+0000 7f9892ffd640 1 --2- 172.21.15.43:0/777621104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561854fe02f0 0x561855000550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:41.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.398+0000 7f98937fe640 1 -- 172.21.15.43:0/777621104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561854fe02f0 msgr2=0x561855000550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:41.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.398+0000 7f98937fe640 1 --2- 172.21.15.43:0/777621104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561854fe02f0 0x561855000550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.398+0000 7f98937fe640 1 -- 172.21.15.43:0/777621104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561854feabb0 msgr2=0x561855063c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:41.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.398+0000 7f98937fe640 1 --2- 172.21.15.43:0/777621104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561854feabb0 0x561855063c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.398+0000 7f98937fe640 1 -- 172.21.15.43:0/777621104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618550641a0 con 0x561854fdf9d0 2024-09-11T23:21:41.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.398+0000 7f98937fe640 1 --2- 172.21.15.43:0/777621104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854fdf9d0 0x561855000010 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f988402ab30 tx=0x7f9884004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d707ab6114f3c1b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:41.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.398+0000 7f9872ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:21:41.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.398+0000 7f9872ffd640 1 -- 172.21.15.43:0/777621104 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884050030 con 0x561854fdf9d0 2024-09-11T23:21:41.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.398+0000 7f989c012740 1 -- 172.21.15.43:0/777621104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9884005520 con 0x561854fdf9d0 2024-09-11T23:21:41.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.398+0000 7f9872ffd640 1 -- 172.21.15.43:0/777621104 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f9884042050 con 0x561854fdf9d0 2024-09-11T23:21:41.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.398+0000 7f9872ffd640 1 -- 172.21.15.43:0/777621104 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9884035040 con 0x561854fdf9d0 2024-09-11T23:21:41.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.398+0000 7f989c012740 10 client.15084 fetch_fsmap learned FSMap version 23 2024-09-11T23:21:41.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.398+0000 7f989c012740 1 -- 172.21.15.43:0/777621104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561855065380 con 0x561854fdf9d0 2024-09-11T23:21:41.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.398+0000 7f9872ffd640 1 -- 172.21.15.43:0/777621104 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9884005520 con 0x561854fdf9d0 2024-09-11T23:21:41.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.399+0000 7f9872ffd640 1 -- 172.21.15.43:0/777621104 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f9884003710 con 0x561854fdf9d0 2024-09-11T23:21:41.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.399+0000 7f989c012740 10 client.15084 fetch_fsmap finished waiting for FSMap version 23 2024-09-11T23:21:41.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.399+0000 7f989c012740 10 client.15084 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T23:21:41.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.399+0000 7f989c012740 20 client.15084 populate_metadata read hostname 'smithi043' 2024-09-11T23:21:41.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.399+0000 7f989c012740 1 --2- 172.21.15.43:0/777621104 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x561855069ae0 0x5618550687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.399+0000 7f989c012740 4 client.15084 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:21:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.399+0000 7f989c012740 1 -- 172.21.15.43:0/777621104 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "status"}) -- 0x561854f4db10 con 0x561855069ae0 2024-09-11T23:21:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.399+0000 7f9892ffd640 1 --2- 172.21.15.43:0/777621104 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x561855069ae0 0x5618550687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.399+0000 7f9872ffd640 10 client.15084 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:21:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.399+0000 7f9892ffd640 1 --2- 172.21.15.43:0/777621104 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x561855069ae0 0x5618550687f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9e7e0a0c4fcd1f5 server_cookie=85918b2b164b8ffb in_seq=0 out_seq=0 2024-09-11T23:21:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.399+0000 7f9872ffd640 1 -- 172.21.15.43:0/777621104 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561854f4db10 con 0x561855069ae0 2024-09-11T23:21:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.399+0000 7f9872ffd640 10 client.15084 handle_command_reply: tid=0 2024-09-11T23:21:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.400+0000 7f989c012740 1 client.15084 shutdown 2024-09-11T23:21:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.400+0000 7f989c012740 1 -- 172.21.15.43:0/777621104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854fdf9d0 msgr2=0x561855000010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.400+0000 7f989c012740 1 --2- 172.21.15.43:0/777621104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854fdf9d0 0x561855000010 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f988402ab30 tx=0x7f9884004590 comp rx=0 tx=0).stop 2024-09-11T23:21:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.400+0000 7f989c012740 1 -- 172.21.15.43:0/777621104 shutdown_connections 2024-09-11T23:21:41.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.400+0000 7f989c012740 1 --2- 172.21.15.43:0/777621104 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x561855069ae0 0x5618550687f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.400+0000 7f989c012740 1 --2- 172.21.15.43:0/777621104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561854feabb0 0x561855063c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.400+0000 7f989c012740 1 --2- 172.21.15.43:0/777621104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561854fe02f0 0x561855000550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.400+0000 7f989c012740 1 --2- 172.21.15.43:0/777621104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561854fdf9d0 0x561855000010 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.400+0000 7f989c012740 1 -- 172.21.15.43:0/777621104 >> 172.21.15.43:0/777621104 conn(0x561854fdbf60 msgr2=0x561854fe6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.400+0000 7f9872ffd640 0 client.15084 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:21:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.400+0000 7f989c012740 1 -- 172.21.15.43:0/777621104 shutdown_connections 2024-09-11T23:21:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.400+0000 7f989c012740 1 -- 172.21.15.43:0/777621104 wait complete. 2024-09-11T23:21:41.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.400+0000 7f989c012740 20 client.15084 trim_cache size 0 max 16384 2024-09-11T23:21:41.415 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:21:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "25c0cd84-7093-11ef-bcea-c7b262605968", 2024-09-11T23:21:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 1, 2024-09-11T23:21:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24470, 2024-09-11T23:21:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-11T23:21:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-11T23:21:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-11T23:21:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 92.688121699999996, 2024-09-11T23:21:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 23, 2024-09-11T23:21:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 84, 2024-09-11T23:21:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 82, 2024-09-11T23:21:41.416 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 123.55065425700001 2024-09-11T23:21:41.416 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:21:41.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.400+0000 7f9899c39640 1 -- 172.21.15.43:0/2388538745 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f987c0a2b80 msgr2=0x7f987c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:41.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.400+0000 7f9899c39640 1 --2- 172.21.15.43:0/2388538745 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f987c0a2b80 0x7f987c0a5040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f988c01d000 tx=0x7f988c019000 comp rx=0 tx=0).stop 2024-09-11T23:21:41.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.400+0000 7f9899c39640 1 -- 172.21.15.43:0/2388538745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9894101420 msgr2=0x7f989419ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:41.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.400+0000 7f9899c39640 1 --2- 172.21.15.43:0/2388538745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9894101420 0x7f989419ac50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f988800d950 tx=0x7f988800de10 comp rx=0 tx=0).stop 2024-09-11T23:21:41.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.401+0000 7f9899c39640 1 -- 172.21.15.43:0/2388538745 shutdown_connections 2024-09-11T23:21:41.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.401+0000 7f9899c39640 1 --2- 172.21.15.43:0/2388538745 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f987c0a2b80 0x7f987c0a5040 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.401+0000 7f9899c39640 1 --2- 172.21.15.43:0/2388538745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9894106a80 0x7f989419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.401+0000 7f9899c39640 1 --2- 172.21.15.43:0/2388538745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9894101420 0x7f989419ac50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.401+0000 7f9899c39640 1 --2- 172.21.15.43:0/2388538745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9894100a80 0x7f989419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:41.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.401+0000 7f9899c39640 1 -- 172.21.15.43:0/2388538745 >> 172.21.15.43:0/2388538745 conn(0x7f98940fc7a0 msgr2=0x7f989410c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:41.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.401+0000 7f9899c39640 1 -- 172.21.15.43:0/2388538745 shutdown_connections 2024-09-11T23:21:41.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:41.401+0000 7f9899c39640 1 -- 172.21.15.43:0/2388538745 wait complete. 2024-09-11T23:21:42.250 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:41 smithi043 ceph-mon[30950]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-11T23:21:42.251 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T23:21:42.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:41 smithi090 ceph-mon[35130]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-11T23:21:42.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:41 smithi097 ceph-mon[35562]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-11T23:21:42.513 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:43.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.010+0000 7fba135f7640 1 -- 172.21.15.43:0/1090411722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba0c100e20 msgr2=0x7fba0c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:43.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.010+0000 7fba135f7640 1 --2- 172.21.15.43:0/1090411722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba0c100e20 0x7fba0c101200 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fba08009900 tx=0x7fba0802a480 comp rx=0 tx=0).stop 2024-09-11T23:21:43.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.010+0000 7fba135f7640 1 -- 172.21.15.43:0/1090411722 shutdown_connections 2024-09-11T23:21:43.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.010+0000 7fba135f7640 1 --2- 172.21.15.43:0/1090411722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba0c102ef0 0x7fba0c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.010+0000 7fba135f7640 1 --2- 172.21.15.43:0/1090411722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba0c102570 0x7fba0c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.010+0000 7fba135f7640 1 --2- 172.21.15.43:0/1090411722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba0c100e20 0x7fba0c101200 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.010+0000 7fba135f7640 1 -- 172.21.15.43:0/1090411722 >> 172.21.15.43:0/1090411722 conn(0x7fba0c0fc7d0 msgr2=0x7fba0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:43.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.011+0000 7fba135f7640 1 -- 172.21.15.43:0/1090411722 shutdown_connections 2024-09-11T23:21:43.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.011+0000 7fba135f7640 1 -- 172.21.15.43:0/1090411722 wait complete. 2024-09-11T23:21:43.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.011+0000 7fba135f7640 1 Processor -- start 2024-09-11T23:21:43.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.011+0000 7fba135f7640 1 -- start start 2024-09-11T23:21:43.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.011+0000 7fba135f7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba0c100e20 0x7fba0c197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:43.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.011+0000 7fba135f7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba0c102570 0x7fba0c197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:43.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.011+0000 7fba135f7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba0c102ef0 0x7fba0c19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:43.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.011+0000 7fba135f7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba0c110f50 con 0x7fba0c102ef0 2024-09-11T23:21:43.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.011+0000 7fba135f7640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fba0c110dd0 con 0x7fba0c102570 2024-09-11T23:21:43.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.011+0000 7fba135f7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fba0c1110d0 con 0x7fba0c100e20 2024-09-11T23:21:43.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.012+0000 7fba11b94640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba0c102ef0 0x7fba0c19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:43.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.012+0000 7fba11b94640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba0c102ef0 0x7fba0c19bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46660/0 (socket says 172.21.15.43:46660) 2024-09-11T23:21:43.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.012+0000 7fba11b94640 1 -- 172.21.15.43:0/1957102043 learned_addr learned my addr 172.21.15.43:0/1957102043 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:43.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.012+0000 7fba11393640 1 --2- 172.21.15.43:0/1957102043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba0c100e20 0x7fba0c197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:43.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.012+0000 7fba10b92640 1 --2- 172.21.15.43:0/1957102043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba0c102570 0x7fba0c197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:43.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.012+0000 7fba11b94640 1 -- 172.21.15.43:0/1957102043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba0c100e20 msgr2=0x7fba0c197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:43.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.012+0000 7fba11b94640 1 --2- 172.21.15.43:0/1957102043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba0c100e20 0x7fba0c197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.012+0000 7fba11b94640 1 -- 172.21.15.43:0/1957102043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba0c102570 msgr2=0x7fba0c197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:43.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.012+0000 7fba11b94640 1 --2- 172.21.15.43:0/1957102043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba0c102570 0x7fba0c197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.012+0000 7fba11b94640 1 -- 172.21.15.43:0/1957102043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0c19c2c0 con 0x7fba0c102ef0 2024-09-11T23:21:43.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.012+0000 7fba11b94640 1 --2- 172.21.15.43:0/1957102043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba0c102ef0 0x7fba0c19bbe0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fba0400eaa0 tx=0x7fba0400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:43.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.012+0000 7fba027fc640 1 -- 172.21.15.43:0/1957102043 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04004d20 con 0x7fba0c102ef0 2024-09-11T23:21:43.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.012+0000 7fba135f7640 1 -- 172.21.15.43:0/1957102043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba0c19c5b0 con 0x7fba0c102ef0 2024-09-11T23:21:43.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.012+0000 7fba027fc640 1 -- 172.21.15.43:0/1957102043 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba04005020 con 0x7fba0c102ef0 2024-09-11T23:21:43.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.012+0000 7fba027fc640 1 -- 172.21.15.43:0/1957102043 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba040116d0 con 0x7fba0c102ef0 2024-09-11T23:21:43.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.012+0000 7fba135f7640 1 -- 172.21.15.43:0/1957102043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba0c1a3e90 con 0x7fba0c102ef0 2024-09-11T23:21:43.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.015+0000 7fba027fc640 1 -- 172.21.15.43:0/1957102043 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fba040051d0 con 0x7fba0c102ef0 2024-09-11T23:21:43.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.015+0000 7fba159d0740 1 Processor -- start 2024-09-11T23:21:43.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.015+0000 7fba159d0740 1 -- start start 2024-09-11T23:21:43.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.015+0000 7fba159d0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0d9e39d0 0x557c0d9e3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:43.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.015+0000 7fba159d0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9e42f0 0x557c0d9ee670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:43.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.015+0000 7fba159d0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9eebb0 0x557c0d9f0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:43.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.015+0000 7fba159d0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557c0d94a090 con 0x557c0d9e42f0 2024-09-11T23:21:43.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.015+0000 7fba159d0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557c0d8f1a70 con 0x557c0d9e39d0 2024-09-11T23:21:43.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.015+0000 7fba159d0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557c0d946050 con 0x557c0d9eebb0 2024-09-11T23:21:43.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.015+0000 7fba11393640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9e42f0 0x557c0d9ee670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:43.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.015+0000 7fba11393640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9e42f0 0x557c0d9ee670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46670/0 (socket says 172.21.15.43:46670) 2024-09-11T23:21:43.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.015+0000 7fba027fc640 1 --2- 172.21.15.43:0/1957102043 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb9f80a2b80 0x7fb9f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:43.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.015+0000 7fba11393640 1 -- 172.21.15.43:0/520984289 learned_addr learned my addr 172.21.15.43:0/520984289 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:43.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.015+0000 7fba10b92640 1 --2- 172.21.15.43:0/520984289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0d9e39d0 0x557c0d9e3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:43.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.016+0000 7fba10b92640 1 --2- 172.21.15.43:0/520984289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9eebb0 0x557c0d9f0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:43.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.016+0000 7fba11b94640 1 --2- 172.21.15.43:0/1957102043 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb9f80a2b80 0x7fb9f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:43.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.016+0000 7fba11393640 1 -- 172.21.15.43:0/520984289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9eebb0 msgr2=0x557c0d9f0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:43.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.016+0000 7fba11393640 1 --2- 172.21.15.43:0/520984289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9eebb0 0x557c0d9f0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.016+0000 7fba11393640 1 -- 172.21.15.43:0/520984289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0d9e39d0 msgr2=0x557c0d9e3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:43.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.016+0000 7fba11393640 1 --2- 172.21.15.43:0/520984289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0d9e39d0 0x557c0d9e3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.016+0000 7fba11393640 1 -- 172.21.15.43:0/520984289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c0d8fe130 con 0x557c0d9e42f0 2024-09-11T23:21:43.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.016+0000 7fba10b92640 1 --2- 172.21.15.43:0/520984289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9eebb0 0x557c0d9f0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:43.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.016+0000 7fba027fc640 1 -- 172.21.15.43:0/1957102043 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7fba040cf240 con 0x7fba0c102ef0 2024-09-11T23:21:43.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.016+0000 7fba11393640 1 --2- 172.21.15.43:0/520984289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9e42f0 0x557c0d9ee670 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fba08009900 tx=0x7fba08004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb872b58e06b9fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:43.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.016+0000 7fba11b94640 1 --2- 172.21.15.43:0/1957102043 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb9f80a2b80 0x7fb9f80a5040 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fba04098450 tx=0x7fba04096040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:43.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.016+0000 7fb9f77fe640 1 -- 172.21.15.43:0/520984289 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba08035030 con 0x557c0d9e42f0 2024-09-11T23:21:43.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.016+0000 7fb9f77fe640 1 -- 172.21.15.43:0/520984289 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba08040070 con 0x557c0d9e42f0 2024-09-11T23:21:43.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.016+0000 7fb9f77fe640 1 -- 172.21.15.43:0/520984289 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba08031ad0 con 0x557c0d9e42f0 2024-09-11T23:21:43.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.017+0000 7fba159d0740 1 -- 172.21.15.43:0/520984289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9e42f0 msgr2=0x557c0d9ee670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:43.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.017+0000 7fba159d0740 1 --2- 172.21.15.43:0/520984289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9e42f0 0x557c0d9ee670 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fba08009900 tx=0x7fba08004170 comp rx=0 tx=0).stop 2024-09-11T23:21:43.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.017+0000 7fba159d0740 1 -- 172.21.15.43:0/520984289 shutdown_connections 2024-09-11T23:21:43.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.017+0000 7fba159d0740 1 --2- 172.21.15.43:0/520984289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9eebb0 0x557c0d9f0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.017+0000 7fba159d0740 1 --2- 172.21.15.43:0/520984289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9e42f0 0x557c0d9ee670 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.017+0000 7fba159d0740 1 --2- 172.21.15.43:0/520984289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0d9e39d0 0x557c0d9e3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.017+0000 7fba159d0740 1 -- 172.21.15.43:0/520984289 >> 172.21.15.43:0/520984289 conn(0x557c0d9dff60 msgr2=0x557c0d9e0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:43.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.017+0000 7fba159d0740 1 -- 172.21.15.43:0/520984289 shutdown_connections 2024-09-11T23:21:43.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.017+0000 7fba159d0740 1 -- 172.21.15.43:0/520984289 wait complete. 2024-09-11T23:21:43.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.018+0000 7fba159d0740 1 -- start start 2024-09-11T23:21:43.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.018+0000 7fba159d0740 1 Processor -- start 2024-09-11T23:21:43.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.018+0000 7fba159d0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9e39d0 0x557c0d9ee470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:43.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.018+0000 7fba159d0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0d9e42f0 0x557c0d9eb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:43.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.018+0000 7fba159d0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9eebb0 0x557c0d9ebd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:43.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.018+0000 7fba159d0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557c0d9f1780 con 0x557c0d9eebb0 2024-09-11T23:21:43.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.018+0000 7fba159d0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557c0d904f20 con 0x557c0d9e42f0 2024-09-11T23:21:43.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.018+0000 7fba159d0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557c0d9f1900 con 0x557c0d9e39d0 2024-09-11T23:21:43.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.018+0000 7fba159d0740 10 client.0 _ll_register_callbacks cb 0x557c0d8ce030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:43.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.018+0000 7fba10b92640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9eebb0 0x557c0d9ebd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:43.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.018+0000 7fba10b92640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9eebb0 0x557c0d9ebd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46684/0 (socket says 172.21.15.43:46684) 2024-09-11T23:21:43.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.018+0000 7fba10b92640 1 -- 172.21.15.43:0/2131038410 learned_addr learned my addr 172.21.15.43:0/2131038410 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:43.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.019+0000 7fba11393640 1 --2- 172.21.15.43:0/2131038410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0d9e42f0 0x557c0d9eb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:43.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.019+0000 7fba10b92640 1 -- 172.21.15.43:0/2131038410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9e39d0 msgr2=0x557c0d9ee470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:43.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.019+0000 7fba10b92640 1 --2- 172.21.15.43:0/2131038410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9e39d0 0x557c0d9ee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.019+0000 7fba10b92640 1 -- 172.21.15.43:0/2131038410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0d9e42f0 msgr2=0x557c0d9eb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:43.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.019+0000 7fba10b92640 1 --2- 172.21.15.43:0/2131038410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0d9e42f0 0x557c0d9eb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.019+0000 7fba10b92640 1 -- 172.21.15.43:0/2131038410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c0d6bb2b0 con 0x557c0d9eebb0 2024-09-11T23:21:43.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.019+0000 7fba10b92640 1 --2- 172.21.15.43:0/2131038410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9eebb0 0x557c0d9ebd00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00af30 tx=0x7fb9fc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19958d0e6c698d96 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:43.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.019+0000 7fb9f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:21:43.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.019+0000 7fb9f4ff9640 1 -- 172.21.15.43:0/2131038410 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc005220 con 0x557c0d9eebb0 2024-09-11T23:21:43.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.019+0000 7fba159d0740 1 -- 172.21.15.43:0/2131038410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba08040070 con 0x557c0d9eebb0 2024-09-11T23:21:43.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.019+0000 7fb9f4ff9640 1 -- 172.21.15.43:0/2131038410 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7fb9fc0053c0 con 0x557c0d9eebb0 2024-09-11T23:21:43.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.019+0000 7fb9f4ff9640 1 -- 172.21.15.43:0/2131038410 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb9fc0055c0 con 0x557c0d9eebb0 2024-09-11T23:21:43.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.019+0000 7fba159d0740 10 client.15096 fetch_fsmap learned FSMap version 23 2024-09-11T23:21:43.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.019+0000 7fba159d0740 1 -- 172.21.15.43:0/2131038410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x557c0d9fd180 con 0x557c0d9eebb0 2024-09-11T23:21:43.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.020+0000 7fb9f4ff9640 1 -- 172.21.15.43:0/2131038410 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba08040070 con 0x557c0d9eebb0 2024-09-11T23:21:43.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.020+0000 7fb9f4ff9640 1 -- 172.21.15.43:0/2131038410 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fb9fc01b070 con 0x557c0d9eebb0 2024-09-11T23:21:43.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.020+0000 7fba159d0740 10 client.15096 fetch_fsmap finished waiting for FSMap version 23 2024-09-11T23:21:43.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.020+0000 7fba159d0740 10 client.15096 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:21:43.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.020+0000 7fba159d0740 20 client.15096 populate_metadata read hostname 'smithi043' 2024-09-11T23:21:43.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.020+0000 7fba159d0740 1 --2- 172.21.15.43:0/2131038410 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x557c0d9ff750 0x557c0da00bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:43.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.020+0000 7fba159d0740 4 client.15096 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:21:43.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.020+0000 7fba159d0740 1 -- 172.21.15.43:0/2131038410 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c0d957600 con 0x557c0d9ff750 2024-09-11T23:21:43.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.020+0000 7fba11393640 1 --2- 172.21.15.43:0/2131038410 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x557c0d9ff750 0x557c0da00bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:43.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.021+0000 7fb9f4ff9640 10 client.15096 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:21:43.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.021+0000 7fba11393640 1 --2- 172.21.15.43:0/2131038410 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x557c0d9ff750 0x557c0da00bc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2679310d2359e3a server_cookie=6d816a88e0ee9f2c in_seq=0 out_seq=0 2024-09-11T23:21:43.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.021+0000 7fb9f4ff9640 1 -- 172.21.15.43:0/2131038410 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c0d957600 con 0x557c0d9ff750 2024-09-11T23:21:43.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.021+0000 7fb9f4ff9640 10 client.15096 handle_command_reply: tid=0 2024-09-11T23:21:43.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.021+0000 7fba159d0740 1 client.15096 shutdown 2024-09-11T23:21:43.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.022+0000 7fba159d0740 1 -- 172.21.15.43:0/2131038410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9eebb0 msgr2=0x557c0d9ebd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:43.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.022+0000 7fba159d0740 1 --2- 172.21.15.43:0/2131038410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9eebb0 0x557c0d9ebd00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00af30 tx=0x7fb9fc00f790 comp rx=0 tx=0).stop 2024-09-11T23:21:43.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.022+0000 7fba159d0740 1 -- 172.21.15.43:0/2131038410 shutdown_connections 2024-09-11T23:21:43.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.022+0000 7fba159d0740 1 --2- 172.21.15.43:0/2131038410 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x557c0d9ff750 0x557c0da00bc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.022+0000 7fba159d0740 1 --2- 172.21.15.43:0/2131038410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9eebb0 0x557c0d9ebd00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.022+0000 7fba159d0740 1 --2- 172.21.15.43:0/2131038410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0d9e42f0 0x557c0d9eb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.022+0000 7fb9f4ff9640 0 client.15096 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:21:43.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.022+0000 7fba159d0740 1 --2- 172.21.15.43:0/2131038410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9e39d0 0x557c0d9ee470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.022+0000 7fba159d0740 1 -- 172.21.15.43:0/2131038410 >> 172.21.15.43:0/2131038410 conn(0x557c0d9dff60 msgr2=0x557c0d9e5c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:43.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.022+0000 7fba159d0740 1 -- 172.21.15.43:0/2131038410 shutdown_connections 2024-09-11T23:21:43.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.022+0000 7fba159d0740 1 -- 172.21.15.43:0/2131038410 wait complete. 2024-09-11T23:21:43.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.022+0000 7fba159d0740 20 client.15096 trim_cache size 0 max 16384 2024-09-11T23:21:43.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.040+0000 7fba159d0740 1 Processor -- start 2024-09-11T23:21:43.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.040+0000 7fba159d0740 1 -- start start 2024-09-11T23:21:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.040+0000 7fba159d0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0d9e39d0 0x557c0da00bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.040+0000 7fba159d0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9e42f0 0x557c0d9eb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.040+0000 7fba159d0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9eebb0 0x557c0d9ebcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.040+0000 7fba159d0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557c0d9fef00 con 0x557c0d9e42f0 2024-09-11T23:21:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.040+0000 7fba159d0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557c0d9ff5d0 con 0x557c0d9e39d0 2024-09-11T23:21:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.040+0000 7fba159d0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557c0d9ffca0 con 0x557c0d9eebb0 2024-09-11T23:21:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.040+0000 7fba11393640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9e42f0 0x557c0d9eb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.040+0000 7fba11393640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9e42f0 0x557c0d9eb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46694/0 (socket says 172.21.15.43:46694) 2024-09-11T23:21:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.040+0000 7fba11393640 1 -- 172.21.15.43:0/3907489971 learned_addr learned my addr 172.21.15.43:0/3907489971 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.040+0000 7fba10b92640 1 --2- 172.21.15.43:0/3907489971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0d9e39d0 0x557c0da00bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:43.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.040+0000 7fba11393640 1 -- 172.21.15.43:0/3907489971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9eebb0 msgr2=0x557c0d9ebcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:43.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.040+0000 7fba10b92640 1 --2- 172.21.15.43:0/3907489971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9eebb0 0x557c0d9ebcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:43.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.040+0000 7fba11393640 1 --2- 172.21.15.43:0/3907489971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9eebb0 0x557c0d9ebcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.040+0000 7fba11393640 1 -- 172.21.15.43:0/3907489971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0d9e39d0 msgr2=0x557c0da00bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:43.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.040+0000 7fba11393640 1 --2- 172.21.15.43:0/3907489971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0d9e39d0 0x557c0da00bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.040+0000 7fba11393640 1 -- 172.21.15.43:0/3907489971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c0d9ee7e0 con 0x557c0d9e42f0 2024-09-11T23:21:43.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.041+0000 7fba10b92640 1 --2- 172.21.15.43:0/3907489971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9eebb0 0x557c0d9ebcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:21:43.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.041+0000 7fba11393640 1 --2- 172.21.15.43:0/3907489971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9e42f0 0x557c0d9eb780 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fba08006010 tx=0x7fba0803e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0357c1f2dfc6f7c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:43.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.041+0000 7fb9f6ffd640 1 -- 172.21.15.43:0/3907489971 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba08035040 con 0x557c0d9e42f0 2024-09-11T23:21:43.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.041+0000 7fb9f6ffd640 1 -- 172.21.15.43:0/3907489971 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba08040070 con 0x557c0d9e42f0 2024-09-11T23:21:43.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.041+0000 7fb9f6ffd640 1 -- 172.21.15.43:0/3907489971 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba08050ab0 con 0x557c0d9e42f0 2024-09-11T23:21:43.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.041+0000 7fba159d0740 1 -- 172.21.15.43:0/3907489971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9e42f0 msgr2=0x557c0d9eb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:43.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.042+0000 7fba159d0740 1 --2- 172.21.15.43:0/3907489971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9e42f0 0x557c0d9eb780 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fba08006010 tx=0x7fba0803e040 comp rx=0 tx=0).stop 2024-09-11T23:21:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.042+0000 7fba159d0740 1 -- 172.21.15.43:0/3907489971 shutdown_connections 2024-09-11T23:21:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.042+0000 7fba159d0740 1 --2- 172.21.15.43:0/3907489971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9eebb0 0x557c0d9ebcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.042+0000 7fba159d0740 1 --2- 172.21.15.43:0/3907489971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9e42f0 0x557c0d9eb780 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.042+0000 7fba159d0740 1 --2- 172.21.15.43:0/3907489971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0d9e39d0 0x557c0da00bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.042+0000 7fba159d0740 1 -- 172.21.15.43:0/3907489971 >> 172.21.15.43:0/3907489971 conn(0x557c0d9dff60 msgr2=0x557c0d9ead00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.042+0000 7fba159d0740 1 -- 172.21.15.43:0/3907489971 shutdown_connections 2024-09-11T23:21:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.042+0000 7fba159d0740 1 -- 172.21.15.43:0/3907489971 wait complete. 2024-09-11T23:21:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.042+0000 7fba159d0740 1 -- start start 2024-09-11T23:21:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.042+0000 7fba159d0740 1 Processor -- start 2024-09-11T23:21:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.042+0000 7fba159d0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9e39d0 0x557c0da06080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.042+0000 7fba159d0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9e42f0 0x557c0da065c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.042+0000 7fba159d0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0d9eebb0 0x557c0da06b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:43.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.042+0000 7fba159d0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557c0da00a40 con 0x557c0d9e42f0 2024-09-11T23:21:43.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.042+0000 7fba159d0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557c0da00370 con 0x557c0d9eebb0 2024-09-11T23:21:43.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.043+0000 7fba159d0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557c0da01110 con 0x557c0d9e39d0 2024-09-11T23:21:43.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.043+0000 7fba159d0740 10 client.0 _ll_register_callbacks cb 0x7fb9fc00adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:43.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.043+0000 7fba11393640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9e42f0 0x557c0da065c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:43.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.043+0000 7fba11393640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9e42f0 0x557c0da065c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46696/0 (socket says 172.21.15.43:46696) 2024-09-11T23:21:43.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.043+0000 7fba11393640 1 -- 172.21.15.43:0/476910562 learned_addr learned my addr 172.21.15.43:0/476910562 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:43.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.043+0000 7fba10b92640 1 --2- 172.21.15.43:0/476910562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9e39d0 0x557c0da06080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:43.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.043+0000 7fba10b92640 1 --2- 172.21.15.43:0/476910562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0d9eebb0 0x557c0da06b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:43.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.043+0000 7fba11393640 1 -- 172.21.15.43:0/476910562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9e39d0 msgr2=0x557c0da06080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:43.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.043+0000 7fba11393640 1 --2- 172.21.15.43:0/476910562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9e39d0 0x557c0da06080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.043+0000 7fba11393640 1 -- 172.21.15.43:0/476910562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0d9eebb0 msgr2=0x557c0da06b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:43.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.043+0000 7fba11393640 1 --2- 172.21.15.43:0/476910562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0d9eebb0 0x557c0da06b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.043+0000 7fba11393640 1 -- 172.21.15.43:0/476910562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c0da69f70 con 0x557c0d9e42f0 2024-09-11T23:21:43.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.043+0000 7fba10b92640 1 --2- 172.21.15.43:0/476910562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9e39d0 0x557c0da06080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:43.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.043+0000 7fba11393640 1 --2- 172.21.15.43:0/476910562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9e42f0 0x557c0da065c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fba08009340 tx=0x7fba08005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9080014f45ae6f3f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:43.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.043+0000 7fb9f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:21:43.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.043+0000 7fb9f4ff9640 1 -- 172.21.15.43:0/476910562 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba08035030 con 0x557c0d9e42f0 2024-09-11T23:21:43.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.043+0000 7fb9f4ff9640 1 -- 172.21.15.43:0/476910562 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7fba08050c40 con 0x557c0d9e42f0 2024-09-11T23:21:43.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.043+0000 7fba159d0740 1 -- 172.21.15.43:0/476910562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba08040070 con 0x557c0d9e42f0 2024-09-11T23:21:43.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.044+0000 7fb9f4ff9640 1 -- 172.21.15.43:0/476910562 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fba08068070 con 0x557c0d9e42f0 2024-09-11T23:21:43.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.044+0000 7fba159d0740 10 client.15102 fetch_fsmap learned FSMap version 23 2024-09-11T23:21:43.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.044+0000 7fba159d0740 1 -- 172.21.15.43:0/476910562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x557c0da6b1a0 con 0x557c0d9e42f0 2024-09-11T23:21:43.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.044+0000 7fb9f4ff9640 1 -- 172.21.15.43:0/476910562 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba08040070 con 0x557c0d9e42f0 2024-09-11T23:21:43.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.044+0000 7fb9f4ff9640 1 -- 172.21.15.43:0/476910562 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fba08031840 con 0x557c0d9e42f0 2024-09-11T23:21:43.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.044+0000 7fba159d0740 10 client.15102 fetch_fsmap finished waiting for FSMap version 23 2024-09-11T23:21:43.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.044+0000 7fba159d0740 10 client.15102 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:21:43.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.044+0000 7fba159d0740 20 client.15102 populate_metadata read hostname 'smithi043' 2024-09-11T23:21:43.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.044+0000 7fba159d0740 1 --2- 172.21.15.43:0/476910562 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x557c0da6f6b0 0x557c0da6e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:43.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.044+0000 7fba159d0740 4 client.15102 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:21:43.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.044+0000 7fba159d0740 1 -- 172.21.15.43:0/476910562 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "status"}) -- 0x557c0d951b10 con 0x557c0da6f6b0 2024-09-11T23:21:43.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.044+0000 7fba10b92640 1 --2- 172.21.15.43:0/476910562 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x557c0da6f6b0 0x557c0da6e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:43.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.045+0000 7fba10b92640 1 --2- 172.21.15.43:0/476910562 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x557c0da6f6b0 0x557c0da6e3c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=739885daf697811f server_cookie=cbec9c740677f732 in_seq=0 out_seq=0 2024-09-11T23:21:43.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.045+0000 7fb9f4ff9640 10 client.15102 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:21:43.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.045+0000 7fb9f4ff9640 1 -- 172.21.15.43:0/476910562 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x557c0d951b10 con 0x557c0da6f6b0 2024-09-11T23:21:43.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.045+0000 7fb9f4ff9640 10 client.15102 handle_command_reply: tid=0 2024-09-11T23:21:43.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.045+0000 7fba159d0740 1 client.15102 shutdown 2024-09-11T23:21:43.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.045+0000 7fba159d0740 1 -- 172.21.15.43:0/476910562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9e42f0 msgr2=0x557c0da065c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:43.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.045+0000 7fba159d0740 1 --2- 172.21.15.43:0/476910562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9e42f0 0x557c0da065c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fba08009340 tx=0x7fba08005a40 comp rx=0 tx=0).stop 2024-09-11T23:21:43.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.045+0000 7fba159d0740 1 -- 172.21.15.43:0/476910562 shutdown_connections 2024-09-11T23:21:43.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.045+0000 7fba159d0740 1 --2- 172.21.15.43:0/476910562 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x557c0da6f6b0 0x557c0da6e3c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.045+0000 7fba159d0740 1 --2- 172.21.15.43:0/476910562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0d9eebb0 0x557c0da06b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.045+0000 7fba159d0740 1 --2- 172.21.15.43:0/476910562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557c0d9e42f0 0x557c0da065c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.045+0000 7fba159d0740 1 --2- 172.21.15.43:0/476910562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557c0d9e39d0 0x557c0da06080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.045+0000 7fb9f4ff9640 0 client.15102 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:21:43.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.045+0000 7fba159d0740 1 -- 172.21.15.43:0/476910562 >> 172.21.15.43:0/476910562 conn(0x557c0d9dff60 msgr2=0x557c0d9eb780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:43.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.045+0000 7fba159d0740 1 -- 172.21.15.43:0/476910562 shutdown_connections 2024-09-11T23:21:43.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.045+0000 7fba159d0740 1 -- 172.21.15.43:0/476910562 wait complete. 2024-09-11T23:21:43.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.046+0000 7fba159d0740 20 client.15102 trim_cache size 0 max 16384 2024-09-11T23:21:43.054 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:21:43.054 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "25c0cd84-7093-11ef-bcea-c7b262605968", 2024-09-11T23:21:43.054 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-11T23:21:43.054 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24497, 2024-09-11T23:21:43.054 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-11T23:21:43.054 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-11T23:21:43.055 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-11T23:21:43.055 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 102.310981344, 2024-09-11T23:21:43.055 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 23, 2024-09-11T23:21:43.055 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 84, 2024-09-11T23:21:43.055 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 82, 2024-09-11T23:21:43.055 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 113.33210595 2024-09-11T23:21:43.055 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:21:43.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.046+0000 7fba135f7640 1 -- 172.21.15.43:0/1957102043 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb9f80a2b80 msgr2=0x7fb9f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:43.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.046+0000 7fba135f7640 1 --2- 172.21.15.43:0/1957102043 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb9f80a2b80 0x7fb9f80a5040 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fba04098450 tx=0x7fba04096040 comp rx=0 tx=0).stop 2024-09-11T23:21:43.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.046+0000 7fba135f7640 1 -- 172.21.15.43:0/1957102043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba0c102ef0 msgr2=0x7fba0c19bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:43.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.046+0000 7fba135f7640 1 --2- 172.21.15.43:0/1957102043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba0c102ef0 0x7fba0c19bbe0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fba0400eaa0 tx=0x7fba0400eea0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.046+0000 7fba135f7640 1 -- 172.21.15.43:0/1957102043 shutdown_connections 2024-09-11T23:21:43.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.046+0000 7fba135f7640 1 --2- 172.21.15.43:0/1957102043 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fb9f80a2b80 0x7fb9f80a5040 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.046+0000 7fba135f7640 1 --2- 172.21.15.43:0/1957102043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba0c102ef0 0x7fba0c19bbe0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.046+0000 7fba135f7640 1 --2- 172.21.15.43:0/1957102043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba0c102570 0x7fba0c197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.046+0000 7fba135f7640 1 --2- 172.21.15.43:0/1957102043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba0c100e20 0x7fba0c197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:43.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.046+0000 7fba135f7640 1 -- 172.21.15.43:0/1957102043 >> 172.21.15.43:0/1957102043 conn(0x7fba0c0fc7d0 msgr2=0x7fba0c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:43.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.046+0000 7fba135f7640 1 -- 172.21.15.43:0/1957102043 shutdown_connections 2024-09-11T23:21:43.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:43.046+0000 7fba135f7640 1 -- 172.21.15.43:0/1957102043 wait complete. 2024-09-11T23:21:43.899 INFO:teuthology.run_tasks:Running task sequential... 2024-09-11T23:21:43.909 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-11T23:21:43.909 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-11T23:21:43.909 INFO:teuthology.task.print:Enabling mgr modules 2024-09-11T23:21:43.909 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-11T23:21:43.910 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T23:21:43.910 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-09-11T23:21:43.911 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-11T23:21:44.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.017+0000 7f37be85e640 1 -- 172.21.15.43:0/4141401152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37b8101b40 msgr2=0x7f37b8101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:44.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.017+0000 7f37be85e640 1 --2- 172.21.15.43:0/4141401152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37b8101b40 0x7f37b8101f40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009900 tx=0x7f37a802a410 comp rx=0 tx=0).stop 2024-09-11T23:21:44.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.017+0000 7f37be85e640 1 -- 172.21.15.43:0/4141401152 shutdown_connections 2024-09-11T23:21:44.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.017+0000 7f37be85e640 1 --2- 172.21.15.43:0/4141401152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b81036f0 0x7f37b8109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:44.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.017+0000 7f37be85e640 1 --2- 172.21.15.43:0/4141401152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37b8102d30 0x7f37b81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:44.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.017+0000 7f37be85e640 1 --2- 172.21.15.43:0/4141401152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37b8101b40 0x7f37b8101f40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:44.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.017+0000 7f37be85e640 1 -- 172.21.15.43:0/4141401152 >> 172.21.15.43:0/4141401152 conn(0x7f37b80fd2e0 msgr2=0x7f37b80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:44.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.017+0000 7f37be85e640 1 -- 172.21.15.43:0/4141401152 shutdown_connections 2024-09-11T23:21:44.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.017+0000 7f37be85e640 1 -- 172.21.15.43:0/4141401152 wait complete. 2024-09-11T23:21:44.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.018+0000 7f37be85e640 1 Processor -- start 2024-09-11T23:21:44.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.018+0000 7f37be85e640 1 -- start start 2024-09-11T23:21:44.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.018+0000 7f37be85e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b8101b40 0x7f37b8197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:44.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.018+0000 7f37be85e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37b8102d30 0x7f37b8198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:44.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.018+0000 7f37be85e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37b81036f0 0x7f37b819f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:44.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.018+0000 7f37be85e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f37b810d820 con 0x7f37b8102d30 2024-09-11T23:21:44.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.018+0000 7f37be85e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f37b810d6a0 con 0x7f37b8101b40 2024-09-11T23:21:44.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.018+0000 7f37be85e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f37b810d9a0 con 0x7f37b81036f0 2024-09-11T23:21:44.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.018+0000 7f37b77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37b8102d30 0x7f37b8198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:44.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.018+0000 7f37b77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37b8102d30 0x7f37b8198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46728/0 (socket says 172.21.15.43:46728) 2024-09-11T23:21:44.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.018+0000 7f37b77fe640 1 -- 172.21.15.43:0/3532475046 learned_addr learned my addr 172.21.15.43:0/3532475046 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:44.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.019+0000 7f37b7fff640 1 --2- 172.21.15.43:0/3532475046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b8101b40 0x7f37b8197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:44.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.019+0000 7f37b77fe640 1 -- 172.21.15.43:0/3532475046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37b81036f0 msgr2=0x7f37b819f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:44.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.019+0000 7f37b77fe640 1 --2- 172.21.15.43:0/3532475046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37b81036f0 0x7f37b819f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:44.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.019+0000 7f37b77fe640 1 -- 172.21.15.43:0/3532475046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b8101b40 msgr2=0x7f37b8197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:44.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.019+0000 7f37b77fe640 1 --2- 172.21.15.43:0/3532475046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b8101b40 0x7f37b8197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:44.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.019+0000 7f37b77fe640 1 -- 172.21.15.43:0/3532475046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b819fb50 con 0x7f37b8102d30 2024-09-11T23:21:44.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.019+0000 7f37b7fff640 1 --2- 172.21.15.43:0/3532475046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b8101b40 0x7f37b8197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:44.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.019+0000 7f37b77fe640 1 --2- 172.21.15.43:0/3532475046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37b8102d30 0x7f37b8198470 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00b4a0 tx=0x7f37ac00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:44.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.019+0000 7f37b57fa640 1 -- 172.21.15.43:0/3532475046 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac014070 con 0x7f37b8102d30 2024-09-11T23:21:44.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.019+0000 7f37b57fa640 1 -- 172.21.15.43:0/3532475046 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37ac010af0 con 0x7f37b8102d30 2024-09-11T23:21:44.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.019+0000 7f37be85e640 1 -- 172.21.15.43:0/3532475046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b819fe40 con 0x7f37b8102d30 2024-09-11T23:21:44.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.019+0000 7f37b57fa640 1 -- 172.21.15.43:0/3532475046 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac017d10 con 0x7f37b8102d30 2024-09-11T23:21:44.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.019+0000 7f37be85e640 1 -- 172.21.15.43:0/3532475046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b81a0330 con 0x7f37b8102d30 2024-09-11T23:21:44.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.020+0000 7f37be85e640 1 -- 172.21.15.43:0/3532475046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37b8108fd0 con 0x7f37b8102d30 2024-09-11T23:21:44.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.021+0000 7f37b57fa640 1 -- 172.21.15.43:0/3532475046 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f37ac017070 con 0x7f37b8102d30 2024-09-11T23:21:44.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.022+0000 7f37b57fa640 1 --2- 172.21.15.43:0/3532475046 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f37a00a2b80 0x7f37a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:44.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.022+0000 7f37b7fff640 1 --2- 172.21.15.43:0/3532475046 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f37a00a2b80 0x7f37a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:44.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.022+0000 7f37b57fa640 1 -- 172.21.15.43:0/3532475046 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f37ac0ce540 con 0x7f37b8102d30 2024-09-11T23:21:44.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.022+0000 7f37b7fff640 1 --2- 172.21.15.43:0/3532475046 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f37a00a2b80 0x7f37a00a5040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f37a8006010 tx=0x7f37a8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:44.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.026+0000 7f37b57fa640 1 -- 172.21.15.43:0/3532475046 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f37ac09b420 con 0x7f37b8102d30 2024-09-11T23:21:44.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.253+0000 7f37be85e640 1 -- 172.21.15.43:0/3532475046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f37b8062e90 con 0x7f37b8102d30 2024-09-11T23:21:44.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:43 smithi090 ceph-mon[35130]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:21:44.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:43 smithi097 ceph-mon[35562]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:21:44.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:43 smithi043 ceph-mon[30950]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T23:21:44.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:44.997+0000 7f37b57fa640 1 -- 172.21.15.43:0/3532475046 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f37ac09ad20 con 0x7f37b8102d30 2024-09-11T23:21:45.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.000+0000 7f37be85e640 1 -- 172.21.15.43:0/3532475046 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f37a00a2b80 msgr2=0x7f37a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:45.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.000+0000 7f37be85e640 1 --2- 172.21.15.43:0/3532475046 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f37a00a2b80 0x7f37a00a5040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f37a8006010 tx=0x7f37a8000bc0 comp rx=0 tx=0).stop 2024-09-11T23:21:45.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.000+0000 7f37be85e640 1 -- 172.21.15.43:0/3532475046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37b8102d30 msgr2=0x7f37b8198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:45.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.000+0000 7f37be85e640 1 --2- 172.21.15.43:0/3532475046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37b8102d30 0x7f37b8198470 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00b4a0 tx=0x7f37ac00b960 comp rx=0 tx=0).stop 2024-09-11T23:21:45.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.000+0000 7f37be85e640 1 -- 172.21.15.43:0/3532475046 shutdown_connections 2024-09-11T23:21:45.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.000+0000 7f37be85e640 1 --2- 172.21.15.43:0/3532475046 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f37a00a2b80 0x7f37a00a5040 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:45.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.000+0000 7f37be85e640 1 --2- 172.21.15.43:0/3532475046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37b81036f0 0x7f37b819f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:45.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.000+0000 7f37be85e640 1 --2- 172.21.15.43:0/3532475046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37b8102d30 0x7f37b8198470 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:45.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.000+0000 7f37be85e640 1 --2- 172.21.15.43:0/3532475046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b8101b40 0x7f37b8197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:45.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.000+0000 7f37be85e640 1 -- 172.21.15.43:0/3532475046 >> 172.21.15.43:0/3532475046 conn(0x7f37b80fd2e0 msgr2=0x7f37b81061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:45.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.000+0000 7f37be85e640 1 -- 172.21.15.43:0/3532475046 shutdown_connections 2024-09-11T23:21:45.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.001+0000 7f37be85e640 1 -- 172.21.15.43:0/3532475046 wait complete. 2024-09-11T23:21:45.020 INFO:teuthology.run_tasks:Running task kclient... 2024-09-11T23:21:45.046 INFO:tasks.kclient:Mounting kernel clients... 2024-09-11T23:21:45.047 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-11T23:21:45.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:21:45.047 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-11T23:21:45.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:21:45.065 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-09-11T23:21:45.134 INFO:teuthology.orchestra.run.smithi043.stderr:Cannot find device "ceph-brx" 2024-09-11T23:21:45.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:21:45.136 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-11T23:21:45.136 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-11T23:21:45.136 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T23:21:45.136 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-11T23:21:45.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:44 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3532475046' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T23:21:45.270 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:45 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: ignoring --setuser ceph since I am not root 2024-09-11T23:21:45.270 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:45 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: ignoring --setgroup ceph since I am not root 2024-09-11T23:21:45.270 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:45 smithi090 ceph-mgr[36498]: -- 172.21.15.90:0/895243194 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561e209110e0 con 0x561e208ef400 2024-09-11T23:21:45.270 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:45 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:45.148+0000 7ff0630bd040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:21:45.270 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:45 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:45.204+0000 7ff0630bd040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:21:45.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:44 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3532475046' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T23:21:45.301 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:45 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: ignoring --setuser ceph since I am not root 2024-09-11T23:21:45.301 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:45 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: ignoring --setgroup ceph since I am not root 2024-09-11T23:21:45.301 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:45 smithi097 ceph-mgr[37371]: -- 172.21.15.97:0/4202442663 <== mon.1 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5639c91430e0 con 0x5639c9121400 2024-09-11T23:21:45.301 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:45 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:45.152+0000 7f6dddd90040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:21:45.301 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:45 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:45.208+0000 7f6dddd90040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:21:45.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:44 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3532475046' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T23:21:45.360 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:45 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: ignoring --setuser ceph since I am not root 2024-09-11T23:21:45.360 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:45 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: ignoring --setgroup ceph since I am not root 2024-09-11T23:21:45.360 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:45 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:45.159+0000 7fe617aa9040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T23:21:45.360 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:45 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:45.220+0000 7fe617aa9040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T23:21:45.436 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:45.699 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:45 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:45.380+0000 7fe617aa9040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:21:45.699 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:45 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:45.455+0000 7fe617aa9040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:21:45.699 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:45 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:45.612+0000 7fe617aa9040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:21:45.700 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:45 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:45.351+0000 7ff0630bd040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:21:45.700 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:45 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:45.417+0000 7ff0630bd040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:21:45.701 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:45 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:45.562+0000 7ff0630bd040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:21:45.701 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:45 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:45.700+0000 7ff0630bd040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:21:45.710 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:45 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:45.357+0000 7f6dddd90040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T23:21:45.710 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:45 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:45.423+0000 7f6dddd90040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T23:21:45.710 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:45 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:45.569+0000 7f6dddd90040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:21:45.710 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:45 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:45.709+0000 7f6dddd90040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:21:45.999 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:45 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:45.766+0000 7fe617aa9040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T23:21:46.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.929+0000 7f6841cbc640 1 -- 172.21.15.43:0/4262004707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f683c1036e0 msgr2=0x7f683c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:46.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.929+0000 7f6841cbc640 1 --2- 172.21.15.43:0/4262004707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f683c1036e0 0x7f683c107b90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6824009930 tx=0x7f682402a420 comp rx=0 tx=0).stop 2024-09-11T23:21:46.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.929+0000 7f6841cbc640 1 -- 172.21.15.43:0/4262004707 shutdown_connections 2024-09-11T23:21:46.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.929+0000 7f6841cbc640 1 --2- 172.21.15.43:0/4262004707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f683c1082a0 0x7f683c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:46.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.929+0000 7f6841cbc640 1 --2- 172.21.15.43:0/4262004707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f683c1036e0 0x7f683c107b90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:46.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.929+0000 7f6841cbc640 1 --2- 172.21.15.43:0/4262004707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f683c102d30 0x7f683c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:46.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.929+0000 7f6841cbc640 1 -- 172.21.15.43:0/4262004707 >> 172.21.15.43:0/4262004707 conn(0x7f683c0fc7d0 msgr2=0x7f683c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:46.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.929+0000 7f6841cbc640 1 -- 172.21.15.43:0/4262004707 shutdown_connections 2024-09-11T23:21:46.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.929+0000 7f6841cbc640 1 -- 172.21.15.43:0/4262004707 wait complete. 2024-09-11T23:21:46.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.929+0000 7f6841cbc640 1 Processor -- start 2024-09-11T23:21:46.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.930+0000 7f6841cbc640 1 -- start start 2024-09-11T23:21:46.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.930+0000 7f6841cbc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f683c102d30 0x7f683c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:46.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.930+0000 7f6841cbc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f683c1036e0 0x7f683c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:46.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.930+0000 7f6841cbc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f683c1082a0 0x7f683c19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:46.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.930+0000 7f6841cbc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1120a0 con 0x7f683c102d30 2024-09-11T23:21:46.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.930+0000 7f6841cbc640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f683c111f20 con 0x7f683c1082a0 2024-09-11T23:21:46.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.930+0000 7f6841cbc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f683c112220 con 0x7f683c1036e0 2024-09-11T23:21:46.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.930+0000 7f683b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f683c102d30 0x7f683c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:46.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.930+0000 7f683b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f683c102d30 0x7f683c198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46760/0 (socket says 172.21.15.43:46760) 2024-09-11T23:21:46.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.930+0000 7f683b7fe640 1 -- 172.21.15.43:0/3832486534 learned_addr learned my addr 172.21.15.43:0/3832486534 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:46.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.930+0000 7f6833fff640 1 --2- 172.21.15.43:0/3832486534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f683c1036e0 0x7f683c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:46.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.930+0000 7f683bfff640 1 --2- 172.21.15.43:0/3832486534 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f683c1082a0 0x7f683c19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:46.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.930+0000 7f683b7fe640 1 -- 172.21.15.43:0/3832486534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f683c1036e0 msgr2=0x7f683c198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:46.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.930+0000 7f683b7fe640 1 --2- 172.21.15.43:0/3832486534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f683c1036e0 0x7f683c198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:46.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.930+0000 7f683b7fe640 1 -- 172.21.15.43:0/3832486534 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f683c1082a0 msgr2=0x7f683c19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:46.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.930+0000 7f683b7fe640 1 --2- 172.21.15.43:0/3832486534 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f683c1082a0 0x7f683c19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:46.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.930+0000 7f683b7fe640 1 -- 172.21.15.43:0/3832486534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683c19d5f0 con 0x7f683c102d30 2024-09-11T23:21:46.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.931+0000 7f683b7fe640 1 --2- 172.21.15.43:0/3832486534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f683c102d30 0x7f683c198640 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f682c00e9f0 tx=0x7f682c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:46.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.931+0000 7f68397fa640 1 -- 172.21.15.43:0/3832486534 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c00cd80 con 0x7f683c102d30 2024-09-11T23:21:46.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.931+0000 7f6841cbc640 1 -- 172.21.15.43:0/3832486534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683c19d8e0 con 0x7f683c102d30 2024-09-11T23:21:46.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.931+0000 7f68397fa640 1 -- 172.21.15.43:0/3832486534 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f682c010910 con 0x7f683c102d30 2024-09-11T23:21:46.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.931+0000 7f68397fa640 1 -- 172.21.15.43:0/3832486534 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c0188f0 con 0x7f683c102d30 2024-09-11T23:21:46.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.931+0000 7f6841cbc640 1 -- 172.21.15.43:0/3832486534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f683c1a51c0 con 0x7f683c102d30 2024-09-11T23:21:46.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.932+0000 7f6841cbc640 1 -- 172.21.15.43:0/3832486534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f683c103110 con 0x7f683c102d30 2024-09-11T23:21:46.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.933+0000 7f68397fa640 1 -- 172.21.15.43:0/3832486534 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f682c010ac0 con 0x7f683c102d30 2024-09-11T23:21:46.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.934+0000 7f68397fa640 1 --2- 172.21.15.43:0/3832486534 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f68280a2bd0 0x7f68280a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:46.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.934+0000 7f6833fff640 1 -- 172.21.15.43:0/3832486534 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f68280a2bd0 msgr2=0x7f68280a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/2370495020 2024-09-11T23:21:46.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.934+0000 7f68397fa640 1 -- 172.21.15.43:0/3832486534 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f682c0cf850 con 0x7f683c102d30 2024-09-11T23:21:46.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.934+0000 7f6833fff640 1 --2- 172.21.15.43:0/3832486534 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f68280a2bd0 0x7f68280a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:21:46.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:45.941+0000 7f68397fa640 1 -- 172.21.15.43:0/3832486534 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f682c09c140 con 0x7f683c102d30 2024-09-11T23:21:46.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:46.135+0000 7f6833fff640 1 -- 172.21.15.43:0/3832486534 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f68280a2bd0 msgr2=0x7f68280a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/2370495020 2024-09-11T23:21:46.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:46.135+0000 7f6833fff640 1 --2- 172.21.15.43:0/3832486534 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f68280a2bd0 0x7f68280a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:21:46.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:46.157+0000 7f6841cbc640 1 -- 172.21.15.43:0/3832486534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f683c10a2b0 con 0x7f683c102d30 2024-09-11T23:21:46.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:46.157+0000 7f68397fa640 1 -- 172.21.15.43:0/3832486534 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7f682c0a0440 con 0x7f683c102d30 2024-09-11T23:21:46.158 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T23:21:46.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:46.159+0000 7f6841cbc640 1 -- 172.21.15.43:0/3832486534 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f68280a2bd0 msgr2=0x7f68280a5090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:21:46.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:46.159+0000 7f6841cbc640 1 --2- 172.21.15.43:0/3832486534 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f68280a2bd0 0x7f68280a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:46.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:46.159+0000 7f6841cbc640 1 -- 172.21.15.43:0/3832486534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f683c102d30 msgr2=0x7f683c198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:46.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:46.159+0000 7f6841cbc640 1 --2- 172.21.15.43:0/3832486534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f683c102d30 0x7f683c198640 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f682c00e9f0 tx=0x7f682c00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:21:46.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:46.159+0000 7f6841cbc640 1 -- 172.21.15.43:0/3832486534 shutdown_connections 2024-09-11T23:21:46.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:46.159+0000 7f6841cbc640 1 --2- 172.21.15.43:0/3832486534 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f68280a2bd0 0x7f68280a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:46.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:46.159+0000 7f6841cbc640 1 --2- 172.21.15.43:0/3832486534 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f683c1082a0 0x7f683c19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:46.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:46.159+0000 7f6841cbc640 1 --2- 172.21.15.43:0/3832486534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f683c1036e0 0x7f683c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:46.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:46.159+0000 7f6841cbc640 1 --2- 172.21.15.43:0/3832486534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f683c102d30 0x7f683c198640 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:46.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:46.159+0000 7f6841cbc640 1 -- 172.21.15.43:0/3832486534 >> 172.21.15.43:0/3832486534 conn(0x7f683c0fc7d0 msgr2=0x7f683c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:46.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:46.160+0000 7f6841cbc640 1 -- 172.21.15.43:0/3832486534 shutdown_connections 2024-09-11T23:21:46.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:46.160+0000 7f6841cbc640 1 -- 172.21.15.43:0/3832486534 wait complete. 2024-09-11T23:21:46.257 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:46 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3532475046' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T23:21:46.257 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:46 smithi090 ceph-mon[35130]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-11T23:21:46.257 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:46 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:46.016+0000 7ff0630bd040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:21:46.258 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:46 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:46.076+0000 7ff0630bd040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:21:46.262 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:45 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3532475046' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T23:21:46.262 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:45 smithi097 ceph-mon[35562]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-11T23:21:46.262 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:46 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:46.020+0000 7f6dddd90040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:21:46.262 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:46 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:46.080+0000 7f6dddd90040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:21:46.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:46 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3532475046' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T23:21:46.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:46 smithi043 ceph-mon[30950]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-11T23:21:46.360 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:46 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:46.106+0000 7fe617aa9040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T23:21:46.360 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:46 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:46.171+0000 7fe617aa9040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T23:21:46.519 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:46 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:46.257+0000 7ff0630bd040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:21:46.550 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:46 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:46.262+0000 7f6dddd90040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:21:46.860 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:46 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:46.367+0000 7fe617aa9040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T23:21:46.999 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-11T23:21:46.999 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T23:21:46.999 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-11T23:21:46.999 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T23:21:47.000 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T23:21:47.000 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-11T23:21:47.000 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-11T23:21:47.000 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T23:21:47.000 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T23:21:47.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:21:47.000 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-11T23:21:47.017 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T23:21:47.017 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T23:21:47.017 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T23:21:47.018 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-11T23:21:47.018 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-11T23:21:47.018 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-11T23:21:47.018 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T23:21:47.018 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-11T23:21:47.018 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-11T23:21:47.018 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T23:21:47.018 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-11T23:21:47.018 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-11T23:21:47.018 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T23:21:47.018 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-11T23:21:47.018 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-11T23:21:47.019 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-11T23:21:47.019 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42211sec preferred_lft 42211sec 2024-09-11T23:21:47.019 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-11T23:21:47.019 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-11T23:21:47.019 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T23:21:47.019 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-11T23:21:47.019 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-11T23:21:47.019 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-11T23:21:47.020 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T23:21:47.020 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-11T23:21:47.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add name ceph-brx type bridge 2024-09-11T23:21:47.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr flush dev ceph-brx 2024-09-11T23:21:47.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set ceph-brx up 2024-09-11T23:21:47.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-11T23:21:47.020 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-11T23:21:47.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:21:47.224 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:47 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3832486534' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T23:21:47.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:21:47.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:21:47.246 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T23:21:47.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:47 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3832486534' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T23:21:47.270 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:47 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:47.135+0000 7ff0630bd040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:21:47.270 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:47 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:47.202+0000 7ff0630bd040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:21:47.271 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-11T23:21:47.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:21:47.273 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-11T23:21:47.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:47 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3832486534' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T23:21:47.301 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:47 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:47.138+0000 7f6dddd90040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:21:47.301 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:47 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:47.205+0000 7f6dddd90040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:21:47.351 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-11T23:21:47.351 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T23:21:47.351 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-11T23:21:47.351 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-11T23:21:47.351 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T23:21:47.352 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T23:21:47.352 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-11T23:21:47.352 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-11T23:21:47.352 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-11T23:21:47.352 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-11T23:21:47.352 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-11T23:21:47.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:21:47.567 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:47 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:47.314+0000 7fe617aa9040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T23:21:47.567 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:47 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:47.385+0000 7fe617aa9040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T23:21:47.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:21:47.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:21:47.677 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-11T23:21:47.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:21:47.695 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2024-09-11T23:21:47.751 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T23:21:47.751 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-11T23:21:47.751 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T23:21:47.751 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-11T23:21:47.752 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-11T23:21:47.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:21:47.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:21:47.872 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-11T23:21:47.872 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T23:21:47.872 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-11T23:21:47.872 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-11T23:21:47.872 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-11T23:21:47.872 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-11T23:21:47.873 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-11T23:21:47.873 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-11T23:21:47.873 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-11T23:21:47.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:21:47.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:21:48.001 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T23:21:48.001 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-11T23:21:48.001 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-11T23:21:48.001 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.0 master ceph-brx 2024-09-11T23:21:48.001 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-11T23:21:48.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:21:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:21:48.083 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T23:21:48.083 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T23:21:48.099 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T23:21:48.099 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T23:21:48.100 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T23:21:48.157 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-11T23:21:48.157 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-11T23:21:48.158 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/2b469744-514b-43ef-9c93-a53c45581893 2024-09-11T23:21:48.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:21:48.158 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/2b469744-514b-43ef-9c93-a53c45581893 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-11T23:21:48.168 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:47 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:47.910+0000 7f6dddd90040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:21:48.168 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:48 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:48.026+0000 7f6dddd90040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:21:48.171 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:47 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:47.913+0000 7ff0630bd040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:21:48.172 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:48 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:48.028+0000 7ff0630bd040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:21:48.263 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-11T23:21:48.263 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-09-11T23:21:48.400 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-11T23:21:48.400 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-09-11T23:21:48.400 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.90:3300,172.21.15.97:3300" 2024-09-11T23:21:48.400 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with new device syntax: 0@25c0cd84-7093-11ef-bcea-c7b262605968.cephfs=/volumes/qa/sv_1/2b469744-514b-43ef-9c93-a53c45581893 2024-09-11T23:21:48.400 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.43:3300/172.21.15.90:3300/172.21.15.97:3300" will pass to kernel 2024-09-11T23:21:48.401 INFO:teuthology.orchestra.run.smithi043.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-11T23:21:48.401 INFO:teuthology.orchestra.run.smithi043.stdout: You just mounted a file system that supports labels which does not 2024-09-11T23:21:48.401 INFO:teuthology.orchestra.run.smithi043.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-11T23:21:48.401 INFO:teuthology.orchestra.run.smithi043.stdout: applications will generate AVC messages and not be allowed access to 2024-09-11T23:21:48.401 INFO:teuthology.orchestra.run.smithi043.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-11T23:21:48.403 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T23:21:48.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:21:48.403 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T23:21:48.426 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:48 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:48.187+0000 7fe617aa9040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T23:21:48.426 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:48 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:48.312+0000 7fe617aa9040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T23:21:48.431 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T23:21:48.446 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:48 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:48.168+0000 7f6dddd90040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:21:48.446 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:48 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:48.230+0000 7f6dddd90040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:21:48.446 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:48 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:48.376+0000 7f6dddd90040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:21:48.446 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:48 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:48.445+0000 7f6dddd90040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:21:48.449 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:48 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:48.172+0000 7ff0630bd040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:21:48.449 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:48 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:48.233+0000 7ff0630bd040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:21:48.450 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:48 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:48.381+0000 7ff0630bd040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:21:48.450 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:48 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:48.449+0000 7ff0630bd040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:21:48.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T23:21:48.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:21:48.530 INFO:teuthology.orchestra.run.smithi043.stdout:client24820 2024-09-11T23:21:48.531 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T23:21:48.531 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T23:21:48.548 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T23:21:48.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T23:21:48.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:21:48.646 INFO:teuthology.orchestra.run.smithi043.stdout:client24820 2024-09-11T23:21:48.647 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T23:21:48.691 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:48 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:48.465+0000 7fe617aa9040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T23:21:48.691 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:48 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:48.531+0000 7fe617aa9040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T23:21:48.691 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:48 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:48.691+0000 7fe617aa9040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T23:21:48.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T23:21:48.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:21:48.706 INFO:teuthology.orchestra.run.smithi043.stdout:client24820 2024-09-11T23:21:48.707 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/25c0cd84-7093-11ef-bcea-c7b262605968.client24820/status') 2024-09-11T23:21:48.727 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:48 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:48.505+0000 7f6dddd90040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:21:48.728 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:48 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:48.662+0000 7f6dddd90040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:21:48.728 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:48 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:48.727+0000 7f6dddd90040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:21:48.732 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:48 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:48.509+0000 7ff0630bd040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:21:48.732 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:48 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:48.667+0000 7ff0630bd040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:21:48.732 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:48 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:48.732+0000 7ff0630bd040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:21:48.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T23:21:48.768 INFO:teuthology.orchestra.run.smithi043.stdout:instance: client.24820 (3)172.21.15.43:0/2979000034 2024-09-11T23:21:48.768 INFO:teuthology.orchestra.run.smithi043.stdout:blocklisted: false 2024-09-11T23:21:48.768 INFO:teuthology.orchestra.run.smithi043.stderr:0+1 records in 2024-09-11T23:21:48.768 INFO:teuthology.orchestra.run.smithi043.stderr:0+1 records out 2024-09-11T23:21:48.769 INFO:teuthology.orchestra.run.smithi043.stderr:71 bytes copied, 2.8384e-05 s, 2.5 MB/s 2024-09-11T23:21:48.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T23:21:48.776 INFO:teuthology.run_tasks:Running task exec... 2024-09-11T23:21:48.784 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T23:21:48.784 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-09-11T23:21:48.784 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-11T23:21:48.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.891+0000 7fa26245e640 1 -- 172.21.15.43:0/3859741838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa25c102d50 msgr2=0x7fa25c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:48.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.891+0000 7fa26245e640 1 --2- 172.21.15.43:0/3859741838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa25c102d50 0x7fa25c1031d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa244009900 tx=0x7fa24402a410 comp rx=0 tx=0).stop 2024-09-11T23:21:48.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.891+0000 7fa26245e640 1 -- 172.21.15.43:0/3859741838 shutdown_connections 2024-09-11T23:21:48.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.891+0000 7fa26245e640 1 --2- 172.21.15.43:0/3859741838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa25c103710 0x7fa25c109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:48.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.891+0000 7fa26245e640 1 --2- 172.21.15.43:0/3859741838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa25c102d50 0x7fa25c1031d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:48.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.891+0000 7fa26245e640 1 --2- 172.21.15.43:0/3859741838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa25c101b60 0x7fa25c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:48.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.891+0000 7fa26245e640 1 -- 172.21.15.43:0/3859741838 >> 172.21.15.43:0/3859741838 conn(0x7fa25c0fd2e0 msgr2=0x7fa25c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:48.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.891+0000 7fa26245e640 1 -- 172.21.15.43:0/3859741838 shutdown_connections 2024-09-11T23:21:48.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.891+0000 7fa26245e640 1 -- 172.21.15.43:0/3859741838 wait complete. 2024-09-11T23:21:48.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.892+0000 7fa26245e640 1 Processor -- start 2024-09-11T23:21:48.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.892+0000 7fa26245e640 1 -- start start 2024-09-11T23:21:48.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.892+0000 7fa26245e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa25c101b60 0x7fa25c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:48.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.892+0000 7fa26245e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa25c102d50 0x7fa25c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:48.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.893+0000 7fa26245e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa25c103710 0x7fa25c19f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:48.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.893+0000 7fa26245e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c10d810 con 0x7fa25c103710 2024-09-11T23:21:48.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.893+0000 7fa26245e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c10d690 con 0x7fa25c102d50 2024-09-11T23:21:48.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.893+0000 7fa26245e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c10d990 con 0x7fa25c101b60 2024-09-11T23:21:48.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.893+0000 7fa25bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa25c101b60 0x7fa25c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:48.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.893+0000 7fa2609fb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa25c103710 0x7fa25c19f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:48.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.893+0000 7fa25bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa25c101b60 0x7fa25c197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:39006/0 (socket says 172.21.15.43:39006) 2024-09-11T23:21:48.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.893+0000 7fa25bfff640 1 -- 172.21.15.43:0/352015156 learned_addr learned my addr 172.21.15.43:0/352015156 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:48.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.893+0000 7fa25b7fe640 1 --2- 172.21.15.43:0/352015156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa25c102d50 0x7fa25c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:48.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.893+0000 7fa25bfff640 1 -- 172.21.15.43:0/352015156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa25c102d50 msgr2=0x7fa25c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:48.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.893+0000 7fa25bfff640 1 --2- 172.21.15.43:0/352015156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa25c102d50 0x7fa25c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:48.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.893+0000 7fa25bfff640 1 -- 172.21.15.43:0/352015156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa25c103710 msgr2=0x7fa25c19f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:48.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.893+0000 7fa25bfff640 1 --2- 172.21.15.43:0/352015156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa25c103710 0x7fa25c19f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:48.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.893+0000 7fa25bfff640 1 -- 172.21.15.43:0/352015156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25c19fac0 con 0x7fa25c101b60 2024-09-11T23:21:48.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.893+0000 7fa2609fb640 1 --2- 172.21.15.43:0/352015156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa25c103710 0x7fa25c19f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:21:48.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.893+0000 7fa25bfff640 1 --2- 172.21.15.43:0/352015156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa25c101b60 0x7fa25c197ea0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00e9c0 tx=0x7fa24c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:48.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.893+0000 7fa2597fa640 1 -- 172.21.15.43:0/352015156 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c00cde0 con 0x7fa25c101b60 2024-09-11T23:21:48.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.893+0000 7fa2597fa640 1 -- 172.21.15.43:0/352015156 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa24c010910 con 0x7fa25c101b60 2024-09-11T23:21:48.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.893+0000 7fa26245e640 1 -- 172.21.15.43:0/352015156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25c19fdb0 con 0x7fa25c101b60 2024-09-11T23:21:48.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.893+0000 7fa2597fa640 1 -- 172.21.15.43:0/352015156 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c018a00 con 0x7fa25c101b60 2024-09-11T23:21:48.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.893+0000 7fa26245e640 1 -- 172.21.15.43:0/352015156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa25c1a0210 con 0x7fa25c101b60 2024-09-11T23:21:48.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.894+0000 7fa26245e640 1 -- 172.21.15.43:0/352015156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa25c108f40 con 0x7fa25c101b60 2024-09-11T23:21:48.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.896+0000 7fa2597fa640 1 -- 172.21.15.43:0/352015156 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fa24c01f020 con 0x7fa25c101b60 2024-09-11T23:21:48.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.896+0000 7fa2597fa640 1 --2- 172.21.15.43:0/352015156 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa2480a2bd0 0x7fa2480a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:48.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.896+0000 7fa25b7fe640 1 -- 172.21.15.43:0/352015156 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa2480a2bd0 msgr2=0x7fa2480a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/2370495020 2024-09-11T23:21:48.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.896+0000 7fa2597fa640 1 -- 172.21.15.43:0/352015156 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7fa24c0cf340 con 0x7fa25c101b60 2024-09-11T23:21:48.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.896+0000 7fa25b7fe640 1 --2- 172.21.15.43:0/352015156 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa2480a2bd0 0x7fa2480a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:21:48.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:48.901+0000 7fa2597fa640 1 -- 172.21.15.43:0/352015156 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa24c09c220 con 0x7fa25c101b60 2024-09-11T23:21:48.995 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:48 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:48.795+0000 7f6dddd90040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:21:48.995 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:48 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:48.877+0000 7f6dddd90040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:21:48.995 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:48 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:48.937+0000 7f6dddd90040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:21:48.995 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:48 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:48.995+0000 7f6dddd90040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:21:49.001 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:48 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:48.800+0000 7ff0630bd040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:21:49.001 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:48 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:48.882+0000 7ff0630bd040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:21:49.001 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:48 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:48.942+0000 7ff0630bd040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:21:49.001 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:49 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:49.001+0000 7ff0630bd040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:21:49.002 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:48 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:48.766+0000 7fe617aa9040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T23:21:49.002 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:48 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:48.831+0000 7fe617aa9040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T23:21:49.003 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:49 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:49.002+0000 7fe617aa9040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T23:21:49.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.097+0000 7fa25b7fe640 1 -- 172.21.15.43:0/352015156 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa2480a2bd0 msgr2=0x7fa2480a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/2370495020 2024-09-11T23:21:49.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.097+0000 7fa25b7fe640 1 --2- 172.21.15.43:0/352015156 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa2480a2bd0 0x7fa2480a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:21:49.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.115+0000 7fa26245e640 1 -- 172.21.15.43:0/352015156 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fa25c062e90 con 0x7fa25c101b60 2024-09-11T23:21:49.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.176+0000 7fa2597fa640 1 -- 172.21.15.43:0/352015156 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v24) ==== 118+0+0 (secure 0 0 0) 0x7fa24c09bb20 con 0x7fa25c101b60 2024-09-11T23:21:49.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.178+0000 7fa26245e640 1 -- 172.21.15.43:0/352015156 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa2480a2bd0 msgr2=0x7fa2480a5090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:21:49.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.178+0000 7fa26245e640 1 --2- 172.21.15.43:0/352015156 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa2480a2bd0 0x7fa2480a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:49.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.178+0000 7fa26245e640 1 -- 172.21.15.43:0/352015156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa25c101b60 msgr2=0x7fa25c197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:49.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.178+0000 7fa26245e640 1 --2- 172.21.15.43:0/352015156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa25c101b60 0x7fa25c197ea0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00e9c0 tx=0x7fa24c00ee80 comp rx=0 tx=0).stop 2024-09-11T23:21:49.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.178+0000 7fa26245e640 1 -- 172.21.15.43:0/352015156 shutdown_connections 2024-09-11T23:21:49.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.178+0000 7fa26245e640 1 --2- 172.21.15.43:0/352015156 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7fa2480a2bd0 0x7fa2480a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:49.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.178+0000 7fa26245e640 1 --2- 172.21.15.43:0/352015156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa25c103710 0x7fa25c19f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:49.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.178+0000 7fa26245e640 1 --2- 172.21.15.43:0/352015156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa25c102d50 0x7fa25c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:49.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.178+0000 7fa26245e640 1 --2- 172.21.15.43:0/352015156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa25c101b60 0x7fa25c197ea0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:49.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.178+0000 7fa26245e640 1 -- 172.21.15.43:0/352015156 >> 172.21.15.43:0/352015156 conn(0x7fa25c0fd2e0 msgr2=0x7fa25c106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:49.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.178+0000 7fa26245e640 1 -- 172.21.15.43:0/352015156 shutdown_connections 2024-09-11T23:21:49.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.178+0000 7fa26245e640 1 -- 172.21.15.43:0/352015156 wait complete. 2024-09-11T23:21:49.189 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-11T23:21:49.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:49 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/352015156' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-11T23:21:49.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:49 smithi090 ceph-mon[35130]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-11T23:21:49.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.297+0000 7f681738d640 1 -- 172.21.15.43:0/2903201954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6810107810 msgr2=0x7f6810109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:49.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.297+0000 7f681738d640 1 --2- 172.21.15.43:0/2903201954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6810107810 0x7f6810109c80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6808009ca0 tx=0x7f680802a8c0 comp rx=0 tx=0).stop 2024-09-11T23:21:49.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.297+0000 7f681738d640 1 -- 172.21.15.43:0/2903201954 shutdown_connections 2024-09-11T23:21:49.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.297+0000 7f681738d640 1 --2- 172.21.15.43:0/2903201954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6810107810 0x7f6810109c80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:49.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.297+0000 7f681738d640 1 --2- 172.21.15.43:0/2903201954 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6810104ee0 0x7f68101072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:49.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.297+0000 7f681738d640 1 --2- 172.21.15.43:0/2903201954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68101025b0 0x7f68101049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:49.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.297+0000 7f681738d640 1 -- 172.21.15.43:0/2903201954 >> 172.21.15.43:0/2903201954 conn(0x7f68100fc190 msgr2=0x7f68100fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:49.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.297+0000 7f681738d640 1 -- 172.21.15.43:0/2903201954 shutdown_connections 2024-09-11T23:21:49.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.297+0000 7f681738d640 1 -- 172.21.15.43:0/2903201954 wait complete. 2024-09-11T23:21:49.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.298+0000 7f681738d640 1 Processor -- start 2024-09-11T23:21:49.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.298+0000 7f681738d640 1 -- start start 2024-09-11T23:21:49.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.298+0000 7f681738d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68101025b0 0x7f6810197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:49.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.298+0000 7f681738d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6810104ee0 0x7f6810198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:49.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.298+0000 7f681738d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6810107810 0x7f681019f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:49.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.298+0000 7f681738d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f681010d840 con 0x7f6810107810 2024-09-11T23:21:49.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.298+0000 7f681738d640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f681010d6c0 con 0x7f6810104ee0 2024-09-11T23:21:49.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.298+0000 7f681738d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f681010d9c0 con 0x7f68101025b0 2024-09-11T23:21:49.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.298+0000 7f681592a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6810107810 0x7f681019f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:49.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.298+0000 7f681592a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6810107810 0x7f681019f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46798/0 (socket says 172.21.15.43:46798) 2024-09-11T23:21:49.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.298+0000 7f681592a640 1 -- 172.21.15.43:0/2825633444 learned_addr learned my addr 172.21.15.43:0/2825633444 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:49.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.298+0000 7f6814928640 1 --2- 172.21.15.43:0/2825633444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6810104ee0 0x7f6810198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:49.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.298+0000 7f6815129640 1 --2- 172.21.15.43:0/2825633444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68101025b0 0x7f6810197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:49.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.299+0000 7f681592a640 1 -- 172.21.15.43:0/2825633444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68101025b0 msgr2=0x7f6810197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:49.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.299+0000 7f681592a640 1 --2- 172.21.15.43:0/2825633444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68101025b0 0x7f6810197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:49.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.299+0000 7f681592a640 1 -- 172.21.15.43:0/2825633444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6810104ee0 msgr2=0x7f6810198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:49.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.299+0000 7f681592a640 1 --2- 172.21.15.43:0/2825633444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6810104ee0 0x7f6810198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:49.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.299+0000 7f681592a640 1 -- 172.21.15.43:0/2825633444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681019faf0 con 0x7f6810107810 2024-09-11T23:21:49.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.299+0000 7f6814928640 1 --2- 172.21.15.43:0/2825633444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6810104ee0 0x7f6810198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:21:49.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.299+0000 7f681592a640 1 --2- 172.21.15.43:0/2825633444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6810107810 0x7f681019f480 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6808031ae0 tx=0x7f6808031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:49.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.299+0000 7f68067fc640 1 -- 172.21.15.43:0/2825633444 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6808003b70 con 0x7f6810107810 2024-09-11T23:21:49.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.299+0000 7f68067fc640 1 -- 172.21.15.43:0/2825633444 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6808003d10 con 0x7f6810107810 2024-09-11T23:21:49.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.299+0000 7f681738d640 1 -- 172.21.15.43:0/2825633444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f681019fd80 con 0x7f6810107810 2024-09-11T23:21:49.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.299+0000 7f68067fc640 1 -- 172.21.15.43:0/2825633444 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680802fb70 con 0x7f6810107810 2024-09-11T23:21:49.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.299+0000 7f681738d640 1 -- 172.21.15.43:0/2825633444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68101a0270 con 0x7f6810107810 2024-09-11T23:21:49.303 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:49 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/352015156' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-11T23:21:49.303 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:49 smithi043 ceph-mon[30950]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-11T23:21:49.303 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:49 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:49.071+0000 7fe617aa9040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T23:21:49.303 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:49 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:49.142+0000 7fe617aa9040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T23:21:49.303 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:49 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:49.234+0000 7fe617aa9040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T23:21:49.303 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:49 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:49.299+0000 7fe617aa9040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T23:21:49.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:49 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/352015156' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-11T23:21:49.304 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:49 smithi097 ceph-mon[35562]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-11T23:21:49.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.300+0000 7f681738d640 1 -- 172.21.15.43:0/2825633444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68100fdc80 con 0x7f6810107810 2024-09-11T23:21:49.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.302+0000 7f68067fc640 1 -- 172.21.15.43:0/2825633444 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f680802fd10 con 0x7f6810107810 2024-09-11T23:21:49.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.302+0000 7f68067fc640 1 --2- 172.21.15.43:0/2825633444 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f67fc0a2bd0 0x7f67fc0a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:49.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.302+0000 7f6815129640 1 -- 172.21.15.43:0/2825633444 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f67fc0a2bd0 msgr2=0x7f67fc0a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/2370495020 2024-09-11T23:21:49.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.303+0000 7f68067fc640 1 -- 172.21.15.43:0/2825633444 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8269+0+0 (secure 0 0 0) 0x7f68080f05c0 con 0x7f6810107810 2024-09-11T23:21:49.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.303+0000 7f6815129640 1 --2- 172.21.15.43:0/2825633444 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f67fc0a2bd0 0x7f67fc0a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T23:21:49.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.306+0000 7f68067fc640 1 -- 172.21.15.43:0/2825633444 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f68080bd500 con 0x7f6810107810 2024-09-11T23:21:49.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.504+0000 7f6815129640 1 -- 172.21.15.43:0/2825633444 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f67fc0a2bd0 msgr2=0x7f67fc0a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/2370495020 2024-09-11T23:21:49.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.504+0000 7f6815129640 1 --2- 172.21.15.43:0/2825633444 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f67fc0a2bd0 0x7f67fc0a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T23:21:49.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.509+0000 7f681738d640 1 -- 172.21.15.43:0/2825633444 --> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f68100626d0 con 0x7f67fc0a2bd0 2024-09-11T23:21:49.610 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:49 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:49.363+0000 7fe617aa9040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T23:21:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.905+0000 7f6815129640 1 -- 172.21.15.43:0/2825633444 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f67fc0a2bd0 msgr2=0x7f67fc0a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/2370495020 2024-09-11T23:21:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:49.905+0000 7f6815129640 1 --2- 172.21.15.43:0/2825633444 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f67fc0a2bd0 0x7f67fc0a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-11T23:21:50.051 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:21:49 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z[37367]: 2024-09-11T23:21:49.783+0000 7f6dddd90040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:21:50.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:50.184+0000 7f68067fc640 1 -- 172.21.15.43:0/2825633444 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 20) ==== 137850+0+0 (secure 0 0 0) 0x7f68080b9470 con 0x7f6810107810 2024-09-11T23:21:50.189 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:21:49 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y[36494]: 2024-09-11T23:21:49.794+0000 7ff0630bd040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:21:50.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:50.214+0000 7f68067fc640 1 -- 172.21.15.43:0/2825633444 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 21) ==== 137518+0+0 (secure 0 0 0) 0x7f6808004640 con 0x7f6810107810 2024-09-11T23:21:50.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:50.215+0000 7f68067fc640 1 -- 172.21.15.43:0/2825633444 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f67fc0a2bd0 msgr2=0x7f67fc0a5090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:21:50.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:50.215+0000 7f68067fc640 1 --2- 172.21.15.43:0/2825633444 >> [v2:172.21.15.43:6800/2370495020,v1:172.21.15.43:6801/2370495020] conn(0x7f67fc0a2bd0 0x7f67fc0a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:50.445 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:50.203+0000 7fe617aa9040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T23:21:50.445 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-mon[30950]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-11T23:21:50.446 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:21:50.446 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-mon[30950]: Standby manager daemon z restarted 2024-09-11T23:21:50.446 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-mon[30950]: Standby manager daemon z started 2024-09-11T23:21:50.446 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-mon[30950]: from='mgr.? 172.21.15.97:0/1757673918' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:21:50.446 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-mon[30950]: from='mgr.? 172.21.15.97:0/1757673918' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:21:50.446 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-mon[30950]: from='mgr.? 172.21.15.97:0/1757673918' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:21:50.446 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-mon[30950]: from='mgr.? 172.21.15.97:0/1757673918' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:21:50.446 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-mon[30950]: Standby manager daemon y restarted 2024-09-11T23:21:50.446 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-mon[30950]: Standby manager daemon y started 2024-09-11T23:21:50.446 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-mon[30950]: from='mgr.? 172.21.15.90:0/1575441462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:21:50.446 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-mon[30950]: from='mgr.? 172.21.15.90:0/1575441462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:21:50.446 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-mon[30950]: from='mgr.? 172.21.15.90:0/1575441462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:21:50.447 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-mon[30950]: from='mgr.? 172.21.15.90:0/1575441462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:21:50.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:50 smithi090 ceph-mon[35130]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-11T23:21:50.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:50 smithi090 ceph-mon[35130]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:21:50.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:50 smithi090 ceph-mon[35130]: Standby manager daemon z restarted 2024-09-11T23:21:50.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:50 smithi090 ceph-mon[35130]: Standby manager daemon z started 2024-09-11T23:21:50.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:50 smithi090 ceph-mon[35130]: from='mgr.? 172.21.15.97:0/1757673918' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:21:50.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:50 smithi090 ceph-mon[35130]: from='mgr.? 172.21.15.97:0/1757673918' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:21:50.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:50 smithi090 ceph-mon[35130]: from='mgr.? 172.21.15.97:0/1757673918' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:21:50.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:50 smithi090 ceph-mon[35130]: from='mgr.? 172.21.15.97:0/1757673918' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:21:50.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:50 smithi090 ceph-mon[35130]: Standby manager daemon y restarted 2024-09-11T23:21:50.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:50 smithi090 ceph-mon[35130]: Standby manager daemon y started 2024-09-11T23:21:50.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:50 smithi090 ceph-mon[35130]: from='mgr.? 172.21.15.90:0/1575441462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:21:50.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:50 smithi090 ceph-mon[35130]: from='mgr.? 172.21.15.90:0/1575441462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:21:50.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:50 smithi090 ceph-mon[35130]: from='mgr.? 172.21.15.90:0/1575441462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:21:50.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:50 smithi090 ceph-mon[35130]: from='mgr.? 172.21.15.90:0/1575441462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:21:50.529 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:50 smithi097 ceph-mon[35562]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-11T23:21:50.529 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:50 smithi097 ceph-mon[35562]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:21:50.529 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:50 smithi097 ceph-mon[35562]: Standby manager daemon z restarted 2024-09-11T23:21:50.529 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:50 smithi097 ceph-mon[35562]: Standby manager daemon z started 2024-09-11T23:21:50.530 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:50 smithi097 ceph-mon[35562]: from='mgr.? 172.21.15.97:0/1757673918' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T23:21:50.530 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:50 smithi097 ceph-mon[35562]: from='mgr.? 172.21.15.97:0/1757673918' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:21:50.530 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:50 smithi097 ceph-mon[35562]: from='mgr.? 172.21.15.97:0/1757673918' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T23:21:50.530 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:50 smithi097 ceph-mon[35562]: from='mgr.? 172.21.15.97:0/1757673918' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:21:50.530 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:50 smithi097 ceph-mon[35562]: Standby manager daemon y restarted 2024-09-11T23:21:50.530 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:50 smithi097 ceph-mon[35562]: Standby manager daemon y started 2024-09-11T23:21:50.530 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:50 smithi097 ceph-mon[35562]: from='mgr.? 172.21.15.90:0/1575441462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T23:21:50.530 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:50 smithi097 ceph-mon[35562]: from='mgr.? 172.21.15.90:0/1575441462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T23:21:50.530 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:50 smithi097 ceph-mon[35562]: from='mgr.? 172.21.15.90:0/1575441462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T23:21:50.530 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:50 smithi097 ceph-mon[35562]: from='mgr.? 172.21.15.90:0/1575441462' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T23:21:50.776 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:50.445+0000 7fe582401640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:21:50.776 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:50.445+0000 7fe582401640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:21:50.776 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:50.445+0000 7fe582401640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:21:50.776 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:50.445+0000 7fe582401640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:21:50.776 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:50.445+0000 7fe582401640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:21:50.777 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:50.446+0000 7fe57f3bb640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:21:50.777 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:50.446+0000 7fe57f3bb640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:21:50.777 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:50.446+0000 7fe57f3bb640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:21:50.777 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:50.446+0000 7fe57f3bb640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:21:50.777 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:50 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:50.446+0000 7fe57f3bb640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:21:51.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.224+0000 7f68067fc640 1 -- 172.21.15.43:0/2825633444 <== mon.0 v2:172.21.15.43:3300/0 9 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f68080ee0f0 con 0x7f6810107810 2024-09-11T23:21:51.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.225+0000 7f68067fc640 1 --2- 172.21.15.43:0/2825633444 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f67fc034aa0 0x7f67fc034e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:51.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.225+0000 7f68067fc640 1 -- 172.21.15.43:0/2825633444 --> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f680807e000 con 0x7f67fc034aa0 2024-09-11T23:21:51.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.228+0000 7f6815129640 1 --2- 172.21.15.43:0/2825633444 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f67fc034aa0 0x7f67fc034e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:51.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.231+0000 7f6815129640 1 --2- 172.21.15.43:0/2825633444 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f67fc034aa0 0x7f67fc034e80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f680c0097d0 tx=0x7f680c005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:51.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.248+0000 7f68067fc640 1 -- 172.21.15.43:0/2825633444 <== mgr.15126 v2:172.21.15.43:6800/2510386404 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f680807e000 con 0x7f67fc034aa0 2024-09-11T23:21:51.248 INFO:teuthology.orchestra.run.smithi043.stdout:{} 2024-09-11T23:21:51.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.250+0000 7f681738d640 1 -- 172.21.15.43:0/2825633444 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f67fc034aa0 msgr2=0x7f67fc034e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:51.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.250+0000 7f681738d640 1 --2- 172.21.15.43:0/2825633444 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f67fc034aa0 0x7f67fc034e80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f680c0097d0 tx=0x7f680c005f90 comp rx=0 tx=0).stop 2024-09-11T23:21:51.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.250+0000 7f681738d640 1 -- 172.21.15.43:0/2825633444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6810107810 msgr2=0x7f681019f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:51.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.250+0000 7f681738d640 1 --2- 172.21.15.43:0/2825633444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6810107810 0x7f681019f480 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6808031ae0 tx=0x7f6808031750 comp rx=0 tx=0).stop 2024-09-11T23:21:51.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.250+0000 7f681592a640 1 -- 172.21.15.43:0/2825633444 reap_dead start 2024-09-11T23:21:51.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.251+0000 7f681738d640 1 -- 172.21.15.43:0/2825633444 shutdown_connections 2024-09-11T23:21:51.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.251+0000 7f681738d640 1 -- 172.21.15.43:0/2825633444 >> 172.21.15.43:0/2825633444 conn(0x7f68100fc190 msgr2=0x7f6810102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:51.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.251+0000 7f681738d640 1 -- 172.21.15.43:0/2825633444 shutdown_connections 2024-09-11T23:21:51.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.251+0000 7f681738d640 1 -- 172.21.15.43:0/2825633444 wait complete. 2024-09-11T23:21:51.262 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-11T23:21:51.275 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-11T23:21:51.319 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T23:21:51.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.423+0000 7fa6b9e0e640 1 -- 172.21.15.43:0/2186412515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6b41025d0 msgr2=0x7fa6b41049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:51.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.423+0000 7fa6b9e0e640 1 --2- 172.21.15.43:0/2186412515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6b41025d0 0x7fa6b41049c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009900 tx=0x7fa6a402a410 comp rx=0 tx=0).stop 2024-09-11T23:21:51.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.423+0000 7fa6b9e0e640 1 -- 172.21.15.43:0/2186412515 shutdown_connections 2024-09-11T23:21:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.423+0000 7fa6b9e0e640 1 --2- 172.21.15.43:0/2186412515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6b4107830 0x7fa6b4109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.423+0000 7fa6b9e0e640 1 --2- 172.21.15.43:0/2186412515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6b4104f00 0x7fa6b41072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.423+0000 7fa6b9e0e640 1 --2- 172.21.15.43:0/2186412515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6b41025d0 0x7fa6b41049c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.423+0000 7fa6b9e0e640 1 -- 172.21.15.43:0/2186412515 >> 172.21.15.43:0/2186412515 conn(0x7fa6b40fc1f0 msgr2=0x7fa6b40fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.423+0000 7fa6b9e0e640 1 -- 172.21.15.43:0/2186412515 shutdown_connections 2024-09-11T23:21:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.423+0000 7fa6b9e0e640 1 -- 172.21.15.43:0/2186412515 wait complete. 2024-09-11T23:21:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.424+0000 7fa6b9e0e640 1 Processor -- start 2024-09-11T23:21:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.424+0000 7fa6b9e0e640 1 -- start start 2024-09-11T23:21:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.424+0000 7fa6b9e0e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6b41025d0 0x7fa6b4101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.424+0000 7fa6b9e0e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6b4104f00 0x7fa6b4100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.424+0000 7fa6b9e0e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6b4107830 0x7fa6b4100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.424+0000 7fa6b9e0e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b410d830 con 0x7fa6b41025d0 2024-09-11T23:21:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.424+0000 7fa6b9e0e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b410d6b0 con 0x7fa6b4107830 2024-09-11T23:21:51.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.424+0000 7fa6b9e0e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b410d9b0 con 0x7fa6b4104f00 2024-09-11T23:21:51.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.424+0000 7fa6b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6b41025d0 0x7fa6b4101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:51.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.424+0000 7fa6b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6b41025d0 0x7fa6b4101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45250/0 (socket says 172.21.15.43:45250) 2024-09-11T23:21:51.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.424+0000 7fa6b37fe640 1 -- 172.21.15.43:0/722880093 learned_addr learned my addr 172.21.15.43:0/722880093 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:51.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.424+0000 7fa6b2ffd640 1 --2- 172.21.15.43:0/722880093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6b4104f00 0x7fa6b4100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:51.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.424+0000 7fa6b3fff640 1 --2- 172.21.15.43:0/722880093 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6b4107830 0x7fa6b4100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:51.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.425+0000 7fa6b37fe640 1 -- 172.21.15.43:0/722880093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6b4104f00 msgr2=0x7fa6b4100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:51.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.425+0000 7fa6b37fe640 1 --2- 172.21.15.43:0/722880093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6b4104f00 0x7fa6b4100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:51.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.425+0000 7fa6b37fe640 1 -- 172.21.15.43:0/722880093 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6b4107830 msgr2=0x7fa6b4100ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:51.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.425+0000 7fa6b37fe640 1 --2- 172.21.15.43:0/722880093 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6b4107830 0x7fa6b4100ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:51.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.425+0000 7fa6b37fe640 1 -- 172.21.15.43:0/722880093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b4101350 con 0x7fa6b41025d0 2024-09-11T23:21:51.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.425+0000 7fa6b37fe640 1 --2- 172.21.15.43:0/722880093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6b41025d0 0x7fa6b4101f40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009230 tx=0x7fa6a40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:51.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.425+0000 7fa6b0ff9640 1 -- 172.21.15.43:0/722880093 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a403f030 con 0x7fa6b41025d0 2024-09-11T23:21:51.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.425+0000 7fa6b0ff9640 1 -- 172.21.15.43:0/722880093 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a4003ce0 con 0x7fa6b41025d0 2024-09-11T23:21:51.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.425+0000 7fa6b9e0e640 1 -- 172.21.15.43:0/722880093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b41a1fb0 con 0x7fa6b41025d0 2024-09-11T23:21:51.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.425+0000 7fa6b0ff9640 1 -- 172.21.15.43:0/722880093 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4035070 con 0x7fa6b41025d0 2024-09-11T23:21:51.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.425+0000 7fa6b9e0e640 1 -- 172.21.15.43:0/722880093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b41a23c0 con 0x7fa6b41025d0 2024-09-11T23:21:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.426+0000 7fa6b9e0e640 1 -- 172.21.15.43:0/722880093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b40fdce0 con 0x7fa6b41025d0 2024-09-11T23:21:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.427+0000 7fa6b0ff9640 1 -- 172.21.15.43:0/722880093 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7fa6a4042030 con 0x7fa6b41025d0 2024-09-11T23:21:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.428+0000 7fa6b0ff9640 1 --2- 172.21.15.43:0/722880093 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa69c0a2990 0x7fa69c0a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.428+0000 7fa6b0ff9640 1 -- 172.21.15.43:0/722880093 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fa6a40ef0d0 con 0x7fa6b41025d0 2024-09-11T23:21:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.428+0000 7fa6b2ffd640 1 --2- 172.21.15.43:0/722880093 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa69c0a2990 0x7fa69c0a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.429+0000 7fa6b2ffd640 1 --2- 172.21.15.43:0/722880093 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa69c0a2990 0x7fa69c0a4e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4101cb0 tx=0x7fa6a8005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:51.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.436+0000 7fa6b0ff9640 1 -- 172.21.15.43:0/722880093 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa6a40bbc80 con 0x7fa6b41025d0 2024-09-11T23:21:51.454 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:51.238+0000 7fe57c375640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:21:51.454 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:51.238+0000 7fe57c375640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:21:51.455 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:51.238+0000 7fe57c375640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:21:51.455 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:51.238+0000 7fe57c375640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:21:51.455 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:51.238+0000 7fe57c375640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:21:51.455 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-09-11T23:21:51.455 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: Active manager daemon x restarted 2024-09-11T23:21:51.455 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: Activating manager daemon x 2024-09-11T23:21:51.455 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T23:21:51.456 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: mgrmap e21: x(active, starting, since 0.0117587s), standbys: z, y 2024-09-11T23:21:51.456 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:21:51.456 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:21:51.456 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:21:51.456 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:21:51.456 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:21:51.456 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:21:51.456 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:21:51.456 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:21:51.456 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:21:51.456 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:21:51.456 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:21:51.457 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:21:51.457 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:21:51.457 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:21:51.457 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:21:51.457 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:21:51.457 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:21:51.457 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:21:51.457 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:21:51.457 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:21:51.457 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:21:51.457 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:21:51.457 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:21:51.457 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:21:51.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:21:51.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:21:51.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:21:51.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:21:51.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:21:51.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:21:51.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:21:51.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:21:51.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:21:51.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: Manager daemon x is now available 2024-09-11T23:21:51.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:21:51.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:21:51.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:51 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:21:51.494 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-09-11T23:21:51.494 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: Active manager daemon x restarted 2024-09-11T23:21:51.494 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: Activating manager daemon x 2024-09-11T23:21:51.494 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T23:21:51.494 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: mgrmap e21: x(active, starting, since 0.0117587s), standbys: z, y 2024-09-11T23:21:51.494 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:21:51.495 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:21:51.495 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:21:51.495 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:21:51.495 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:21:51.495 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:21:51.495 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:21:51.495 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:21:51.495 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:21:51.495 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:21:51.495 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:21:51.495 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:21:51.495 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:21:51.496 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:21:51.496 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:21:51.496 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:21:51.496 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:21:51.496 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:21:51.496 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:21:51.496 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:21:51.496 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:21:51.496 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:21:51.496 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:21:51.496 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:21:51.496 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:21:51.496 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:21:51.497 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:21:51.497 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:21:51.497 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:21:51.497 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:21:51.497 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:21:51.497 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:21:51.497 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:21:51.497 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: Manager daemon x is now available 2024-09-11T23:21:51.497 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:21:51.497 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:21:51.497 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:51 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:21:51.539 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-09-11T23:21:51.540 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: Active manager daemon x restarted 2024-09-11T23:21:51.540 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: Activating manager daemon x 2024-09-11T23:21:51.540 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T23:21:51.540 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: mgrmap e21: x(active, starting, since 0.0117587s), standbys: z, y 2024-09-11T23:21:51.540 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T23:21:51.540 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T23:21:51.540 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T23:21:51.540 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T23:21:51.540 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T23:21:51.540 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:21:51.540 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:21:51.541 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:21:51.541 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:21:51.541 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:21:51.541 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:21:51.541 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:21:51.541 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:21:51.541 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:21:51.541 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:21:51.541 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T23:21:51.541 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T23:21:51.541 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T23:21:51.541 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T23:21:51.542 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T23:21:51.542 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T23:21:51.542 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T23:21:51.542 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T23:21:51.542 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T23:21:51.542 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T23:21:51.542 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T23:21:51.542 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T23:21:51.542 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T23:21:51.542 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T23:21:51.542 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T23:21:51.542 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T23:21:51.543 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T23:21:51.543 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T23:21:51.543 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: Manager daemon x is now available 2024-09-11T23:21:51.543 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:21:51.543 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T23:21:51.543 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:51 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T23:21:51.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.618+0000 7fa6b9e0e640 1 -- 172.21.15.43:0/722880093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6b4062f30 con 0x7fa6b41025d0 2024-09-11T23:21:51.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.620+0000 7fa6b0ff9640 1 -- 172.21.15.43:0/722880093 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+23048 (secure 0 0 0) 0x7fa6a40bff80 con 0x7fa6b41025d0 2024-09-11T23:21:51.621 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:21:51.621 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":85,"fsid":"25c0cd84-7093-11ef-bcea-c7b262605968","created":"2024-09-11T23:12:03.504406+0000","modified":"2024-09-11T23:21:50.205389+0000","last_up_change":"2024-09-11T23:19:07.797873+0000","last_in_change":"2024-09-11T23:18:44.595176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:15:17.058713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:19:56.221406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:19:58.705875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24076596-3909-4c87-85f1-cbf168070dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6803","nonce":43839997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6805","nonce":43839997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6809","nonce":43839997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6807","nonce":43839997}]},"public_addr":"172.21.15.43:6803/43839997","cluster_addr":"172.21.15.43:6805/43839997","heartbeat_back_addr":"172.21.15.43:6809/43839997","heartbeat_front_addr":"172.21.15.43:6807/43839997","state":["exists","up"]},{"osd":1,"uuid":"a0472a57-243c-489d-a247-972ee10a71f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6801","nonce":1825149246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6803","nonce":1825149246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6807","nonce":1825149246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6805","nonce":1825149246}]},"public_addr":"172.21.15.90:6801/1825149246","cluster_addr":"172.21.15.90:6803/1825149246","heartbeat_back_addr":"172.21.15.90:6807/1825149246","heartbeat_front_addr":"172.21.15.90:6805/1825149246","state":["exists","up"]},{"osd":2,"uuid":"40366536-0fcf-45a0-b8e3-49178401dc34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6801","nonce":1456846443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6803","nonce":1456846443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6807","nonce":1456846443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6805","nonce":1456846443}]},"public_addr":"172.21.15.97:6801/1456846443","cluster_addr":"172.21.15.97:6803/1456846443","heartbeat_back_addr":"172.21.15.97:6807/1456846443","heartbeat_front_addr":"172.21.15.97:6805/1456846443","state":["exists","up"]},{"osd":3,"uuid":"3062133e-000b-4421-9d53-ac434b9ed9b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6811","nonce":1962837804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6813","nonce":1962837804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6817","nonce":1962837804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6815","nonce":1962837804}]},"public_addr":"172.21.15.43:6811/1962837804","cluster_addr":"172.21.15.43:6813/1962837804","heartbeat_back_addr":"172.21.15.43:6817/1962837804","heartbeat_front_addr":"172.21.15.43:6815/1962837804","state":["exists","up"]},{"osd":4,"uuid":"2e73fa4b-f22c-4319-828d-87b91b9f19f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6809","nonce":181614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6811","nonce":181614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6815","nonce":181614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6813","nonce":181614173}]},"public_addr":"172.21.15.90:6809/181614173","cluster_addr":"172.21.15.90:6811/181614173","heartbeat_back_addr":"172.21.15.90:6815/181614173","heartbeat_front_addr":"172.21.15.90:6813/181614173","state":["exists","up"]},{"osd":5,"uuid":"1e1ad4f9-e724-440f-b515-3ec1ad97d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6809","nonce":1540039543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6811","nonce":1540039543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6815","nonce":1540039543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6813","nonce":1540039543}]},"public_addr":"172.21.15.97:6809/1540039543","cluster_addr":"172.21.15.97:6811/1540039543","heartbeat_back_addr":"172.21.15.97:6815/1540039543","heartbeat_front_addr":"172.21.15.97:6813/1540039543","state":["exists","up"]},{"osd":6,"uuid":"d34df8df-f40d-4159-8b29-5401b1b0e431","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6819","nonce":3785677637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6821","nonce":3785677637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6825","nonce":3785677637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6823","nonce":3785677637}]},"public_addr":"172.21.15.43:6819/3785677637","cluster_addr":"172.21.15.43:6821/3785677637","heartbeat_back_addr":"172.21.15.43:6825/3785677637","heartbeat_front_addr":"172.21.15.43:6823/3785677637","state":["exists","up"]},{"osd":7,"uuid":"b2e01078-528b-49fd-b000-9f584f784b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6817","nonce":236874962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6819","nonce":236874962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6823","nonce":236874962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6821","nonce":236874962}]},"public_addr":"172.21.15.90:6817/236874962","cluster_addr":"172.21.15.90:6819/236874962","heartbeat_back_addr":"172.21.15.90:6823/236874962","heartbeat_front_addr":"172.21.15.90:6821/236874962","state":["exists","up"]},{"osd":8,"uuid":"abef8f27-bf9b-4d63-bfbe-f967fc3d8b17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6817","nonce":1707327301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6819","nonce":1707327301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6823","nonce":1707327301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6821","nonce":1707327301}]},"public_addr":"172.21.15.97:6817/1707327301","cluster_addr":"172.21.15.97:6819/1707327301","heartbeat_back_addr":"172.21.15.97:6823/1707327301","heartbeat_front_addr":"172.21.15.97:6821/1707327301","state":["exists","up"]},{"osd":9,"uuid":"190ab8b2-a835-47e6-9b39-16a58331c58a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6827","nonce":3170306011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6829","nonce":3170306011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6833","nonce":3170306011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6831","nonce":3170306011}]},"public_addr":"172.21.15.43:6827/3170306011","cluster_addr":"172.21.15.43:6829/3170306011","heartbeat_back_addr":"172.21.15.43:6833/3170306011","heartbeat_front_addr":"172.21.15.43:6831/3170306011","state":["exists","up"]},{"osd":10,"uuid":"b32ab5fe-466f-4c8e-8696-cd9f820a3263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6825","nonce":59487041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6827","nonce":59487041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6831","nonce":59487041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6829","nonce":59487041}]},"public_addr":"172.21.15.90:6825/59487041","cluster_addr":"172.21.15.90:6827/59487041","heartbeat_back_addr":"172.21.15.90:6831/59487041","heartbeat_front_addr":"172.21.15.90:6829/59487041","state":["exists","up"]},{"osd":11,"uuid":"d0faa589-21c8-4a9f-8a73-813aafe6d33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6825","nonce":194699315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6827","nonce":194699315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6831","nonce":194699315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6829","nonce":194699315}]},"public_addr":"172.21.15.97:6825/194699315","cluster_addr":"172.21.15.97:6827/194699315","heartbeat_back_addr":"172.21.15.97:6831/194699315","heartbeat_front_addr":"172.21.15.97:6829/194699315","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:23.381247+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:48.843190+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:13.538979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:40.136983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:05.513790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:31.247090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:56.635221+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:22.667327+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:47.910422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:14.353914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:40.584503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:19:06.476766+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/2262294782":"2024-09-12T23:21:50.205362+0000","172.21.15.43:0/2398981254":"2024-09-12T23:21:50.205362+0000","172.21.15.43:0/481605249":"2024-09-12T23:21:50.205362+0000","172.21.15.43:0/1375379108":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1512201905":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/206044413":"2024-09-12T23:12:35.756598+0000","172.21.15.43:6800/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1579053924":"2024-09-12T23:21:50.205362+0000","172.21.15.43:6801/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:6800/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/2370495020":"2024-09-12T23:21:50.205362+0000","172.21.15.43:0/4006124758":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/76575371":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/120495444":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6800/2370495020":"2024-09-12T23:21:50.205362+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:21:51.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.622+0000 7fa6b9e0e640 1 -- 172.21.15.43:0/722880093 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa69c0a2990 msgr2=0x7fa69c0a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:51.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.622+0000 7fa6b9e0e640 1 --2- 172.21.15.43:0/722880093 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa69c0a2990 0x7fa69c0a4e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4101cb0 tx=0x7fa6a8005f90 comp rx=0 tx=0).stop 2024-09-11T23:21:51.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.622+0000 7fa6b9e0e640 1 -- 172.21.15.43:0/722880093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6b41025d0 msgr2=0x7fa6b4101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:51.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.622+0000 7fa6b9e0e640 1 --2- 172.21.15.43:0/722880093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6b41025d0 0x7fa6b4101f40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009230 tx=0x7fa6a40042f0 comp rx=0 tx=0).stop 2024-09-11T23:21:51.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.622+0000 7fa6b9e0e640 1 -- 172.21.15.43:0/722880093 shutdown_connections 2024-09-11T23:21:51.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.622+0000 7fa6b9e0e640 1 --2- 172.21.15.43:0/722880093 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa69c0a2990 0x7fa69c0a4e50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:51.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.622+0000 7fa6b9e0e640 1 --2- 172.21.15.43:0/722880093 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6b4107830 0x7fa6b4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:51.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.622+0000 7fa6b9e0e640 1 --2- 172.21.15.43:0/722880093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6b4104f00 0x7fa6b4100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:51.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.622+0000 7fa6b9e0e640 1 --2- 172.21.15.43:0/722880093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6b41025d0 0x7fa6b4101f40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:51.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.622+0000 7fa6b9e0e640 1 -- 172.21.15.43:0/722880093 >> 172.21.15.43:0/722880093 conn(0x7fa6b40fc1f0 msgr2=0x7fa6b4103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:51.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.622+0000 7fa6b9e0e640 1 -- 172.21.15.43:0/722880093 shutdown_connections 2024-09-11T23:21:51.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.622+0000 7fa6b9e0e640 1 -- 172.21.15.43:0/722880093 wait complete. 2024-09-11T23:21:51.633 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T23:15:17.058713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-11T23:19:56.221406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-11T23:19:58.705875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T23:21:51.634 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T23:21:51.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.741+0000 7fb8b1103640 1 -- 172.21.15.43:0/1488529720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8ac1025e0 msgr2=0x7fb8ac108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:51.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.741+0000 7fb8b1103640 1 --2- 172.21.15.43:0/1488529720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8ac1025e0 0x7fb8ac108e70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009ca0 tx=0x7fb8a402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:21:51.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.741+0000 7fb8b1103640 1 -- 172.21.15.43:0/1488529720 shutdown_connections 2024-09-11T23:21:51.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.741+0000 7fb8b1103640 1 --2- 172.21.15.43:0/1488529720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8ac1025e0 0x7fb8ac108e70 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:51.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.741+0000 7fb8b1103640 1 --2- 172.21.15.43:0/1488529720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8ac101c20 0x7fb8ac1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:51.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.741+0000 7fb8b1103640 1 --2- 172.21.15.43:0/1488529720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8ac100a30 0x7fb8ac100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:51.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.741+0000 7fb8b1103640 1 -- 172.21.15.43:0/1488529720 >> 172.21.15.43:0/1488529720 conn(0x7fb8ac0fc190 msgr2=0x7fb8ac0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:51.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.741+0000 7fb8b1103640 1 -- 172.21.15.43:0/1488529720 shutdown_connections 2024-09-11T23:21:51.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.741+0000 7fb8b1103640 1 -- 172.21.15.43:0/1488529720 wait complete. 2024-09-11T23:21:51.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.741+0000 7fb8b1103640 1 Processor -- start 2024-09-11T23:21:51.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.742+0000 7fb8b1103640 1 -- start start 2024-09-11T23:21:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.742+0000 7fb8b1103640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8ac100a30 0x7fb8ac197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.742+0000 7fb8b1103640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8ac101c20 0x7fb8ac198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.742+0000 7fb8b1103640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8ac1025e0 0x7fb8ac19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.742+0000 7fb8b1103640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac10d800 con 0x7fb8ac101c20 2024-09-11T23:21:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.742+0000 7fb8b1103640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac10d680 con 0x7fb8ac100a30 2024-09-11T23:21:51.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.742+0000 7fb8b1103640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac10d980 con 0x7fb8ac1025e0 2024-09-11T23:21:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.742+0000 7fb8aa59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8ac101c20 0x7fb8ac198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.742+0000 7fb8aa59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8ac101c20 0x7fb8ac198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45266/0 (socket says 172.21.15.43:45266) 2024-09-11T23:21:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.742+0000 7fb8aad9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8ac100a30 0x7fb8ac197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.742+0000 7fb8aa59c640 1 -- 172.21.15.43:0/2408251243 learned_addr learned my addr 172.21.15.43:0/2408251243 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.742+0000 7fb8ab59e640 1 --2- 172.21.15.43:0/2408251243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8ac1025e0 0x7fb8ac19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.742+0000 7fb8aad9d640 1 -- 172.21.15.43:0/2408251243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8ac1025e0 msgr2=0x7fb8ac19f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.742+0000 7fb8aad9d640 1 --2- 172.21.15.43:0/2408251243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8ac1025e0 0x7fb8ac19f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.742+0000 7fb8aad9d640 1 -- 172.21.15.43:0/2408251243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8ac101c20 msgr2=0x7fb8ac198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.742+0000 7fb8aad9d640 1 --2- 172.21.15.43:0/2408251243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8ac101c20 0x7fb8ac198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.743+0000 7fb8aad9d640 1 -- 172.21.15.43:0/2408251243 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8ac19fa40 con 0x7fb8ac100a30 2024-09-11T23:21:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.743+0000 7fb8aad9d640 1 --2- 172.21.15.43:0/2408251243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8ac100a30 0x7fb8ac197e20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00d6a0 tx=0x7fb89c00db60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.744+0000 7fb893fff640 1 -- 172.21.15.43:0/2408251243 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c017070 con 0x7fb8ac100a30 2024-09-11T23:21:51.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.744+0000 7fb893fff640 1 -- 172.21.15.43:0/2408251243 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb89c009d00 con 0x7fb8ac100a30 2024-09-11T23:21:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.744+0000 7fb893fff640 1 -- 172.21.15.43:0/2408251243 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c003c60 con 0x7fb8ac100a30 2024-09-11T23:21:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.744+0000 7fb8b1103640 1 -- 172.21.15.43:0/2408251243 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8ac19fd30 con 0x7fb8ac100a30 2024-09-11T23:21:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.744+0000 7fb8b1103640 1 -- 172.21.15.43:0/2408251243 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8ac1a0210 con 0x7fb8ac100a30 2024-09-11T23:21:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.745+0000 7fb8b1103640 1 -- 172.21.15.43:0/2408251243 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8ac107dd0 con 0x7fb8ac100a30 2024-09-11T23:21:51.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.747+0000 7fb893fff640 1 -- 172.21.15.43:0/2408251243 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7fb89c009820 con 0x7fb8ac100a30 2024-09-11T23:21:51.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.748+0000 7fb893fff640 1 --2- 172.21.15.43:0/2408251243 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb8940a2a10 0x7fb8940a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:51.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.748+0000 7fb893fff640 1 -- 172.21.15.43:0/2408251243 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb89c0ce8a0 con 0x7fb8ac100a30 2024-09-11T23:21:51.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.748+0000 7fb8aa59c640 1 --2- 172.21.15.43:0/2408251243 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb8940a2a10 0x7fb8940a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:51.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.748+0000 7fb8aa59c640 1 --2- 172.21.15.43:0/2408251243 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb8940a2a10 0x7fb8940a4ed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac199330 tx=0x7fb8a0005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.750+0000 7fb893fff640 1 -- 172.21.15.43:0/2408251243 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb89c09ba50 con 0x7fb8ac100a30 2024-09-11T23:21:51.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.927+0000 7fb8b1103640 1 -- 172.21.15.43:0/2408251243 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb8ac062ef0 con 0x7fb8ac100a30 2024-09-11T23:21:51.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.928+0000 7fb893fff640 1 -- 172.21.15.43:0/2408251243 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7fb89c09b350 con 0x7fb8ac100a30 2024-09-11T23:21:51.928 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-09-11T23:21:51.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.930+0000 7fb8b1103640 1 -- 172.21.15.43:0/2408251243 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb8940a2a10 msgr2=0x7fb8940a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:51.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.930+0000 7fb8b1103640 1 --2- 172.21.15.43:0/2408251243 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb8940a2a10 0x7fb8940a4ed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac199330 tx=0x7fb8a0005f90 comp rx=0 tx=0).stop 2024-09-11T23:21:51.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.930+0000 7fb8b1103640 1 -- 172.21.15.43:0/2408251243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8ac100a30 msgr2=0x7fb8ac197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:51.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.930+0000 7fb8b1103640 1 --2- 172.21.15.43:0/2408251243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8ac100a30 0x7fb8ac197e20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00d6a0 tx=0x7fb89c00db60 comp rx=0 tx=0).stop 2024-09-11T23:21:51.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.930+0000 7fb8b1103640 1 -- 172.21.15.43:0/2408251243 shutdown_connections 2024-09-11T23:21:51.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.930+0000 7fb8b1103640 1 --2- 172.21.15.43:0/2408251243 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb8940a2a10 0x7fb8940a4ed0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:51.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.930+0000 7fb8b1103640 1 --2- 172.21.15.43:0/2408251243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8ac1025e0 0x7fb8ac19f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:51.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.930+0000 7fb8b1103640 1 --2- 172.21.15.43:0/2408251243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8ac101c20 0x7fb8ac198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:51.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.930+0000 7fb8b1103640 1 --2- 172.21.15.43:0/2408251243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8ac100a30 0x7fb8ac197e20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:51.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.930+0000 7fb8b1103640 1 -- 172.21.15.43:0/2408251243 >> 172.21.15.43:0/2408251243 conn(0x7fb8ac0fc190 msgr2=0x7fb8ac104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:51.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.930+0000 7fb8b1103640 1 -- 172.21.15.43:0/2408251243 shutdown_connections 2024-09-11T23:21:51.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:51.930+0000 7fb8b1103640 1 -- 172.21.15.43:0/2408251243 wait complete. 2024-09-11T23:21:51.944 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-11T23:21:52.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.065+0000 7f6b5b5af640 1 -- 172.21.15.43:0/904513286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b54100a30 msgr2=0x7f6b54100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:52.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.065+0000 7f6b5b5af640 1 --2- 172.21.15.43:0/904513286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b54100a30 0x7f6b54100e30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009900 tx=0x7f6b5002a410 comp rx=0 tx=0).stop 2024-09-11T23:21:52.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.065+0000 7f6b5b5af640 1 -- 172.21.15.43:0/904513286 shutdown_connections 2024-09-11T23:21:52.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.065+0000 7f6b5b5af640 1 --2- 172.21.15.43:0/904513286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b541025e0 0x7f6b54108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:52.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.065+0000 7f6b5b5af640 1 --2- 172.21.15.43:0/904513286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b54101c20 0x7f6b541020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:52.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.065+0000 7f6b5b5af640 1 --2- 172.21.15.43:0/904513286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b54100a30 0x7f6b54100e30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:52.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.065+0000 7f6b5b5af640 1 -- 172.21.15.43:0/904513286 >> 172.21.15.43:0/904513286 conn(0x7f6b540fc190 msgr2=0x7f6b540fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:52.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.065+0000 7f6b5b5af640 1 -- 172.21.15.43:0/904513286 shutdown_connections 2024-09-11T23:21:52.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.065+0000 7f6b5b5af640 1 -- 172.21.15.43:0/904513286 wait complete. 2024-09-11T23:21:52.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.066+0000 7f6b5b5af640 1 Processor -- start 2024-09-11T23:21:52.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.066+0000 7f6b5b5af640 1 -- start start 2024-09-11T23:21:52.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.066+0000 7f6b5b5af640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b54100a30 0x7f6b541049c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:52.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.066+0000 7f6b5b5af640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b54101c20 0x7f6b54103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:52.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.066+0000 7f6b5b5af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b541025e0 0x7f6b54103540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:52.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.066+0000 7f6b5b5af640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5410c6b0 con 0x7f6b54101c20 2024-09-11T23:21:52.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.066+0000 7f6b5b5af640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5410c530 con 0x7f6b54100a30 2024-09-11T23:21:52.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.066+0000 7f6b5b5af640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5410c830 con 0x7f6b541025e0 2024-09-11T23:21:52.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.066+0000 7f6b58b4a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b54101c20 0x7f6b54103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:52.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.066+0000 7f6b58b4a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b54101c20 0x7f6b54103000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45288/0 (socket says 172.21.15.43:45288) 2024-09-11T23:21:52.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.066+0000 7f6b58b4a640 1 -- 172.21.15.43:0/1306702972 learned_addr learned my addr 172.21.15.43:0/1306702972 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:52.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.066+0000 7f6b5934b640 1 --2- 172.21.15.43:0/1306702972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b54100a30 0x7f6b541049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:52.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.066+0000 7f6b59b4c640 1 --2- 172.21.15.43:0/1306702972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b541025e0 0x7f6b54103540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:52.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.066+0000 7f6b58b4a640 1 -- 172.21.15.43:0/1306702972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b541025e0 msgr2=0x7f6b54103540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:52.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.066+0000 7f6b58b4a640 1 --2- 172.21.15.43:0/1306702972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b541025e0 0x7f6b54103540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:52.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.066+0000 7f6b58b4a640 1 -- 172.21.15.43:0/1306702972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b54100a30 msgr2=0x7f6b541049c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:52.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.066+0000 7f6b58b4a640 1 --2- 172.21.15.43:0/1306702972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b54100a30 0x7f6b541049c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:52.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.066+0000 7f6b58b4a640 1 -- 172.21.15.43:0/1306702972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b54103dd0 con 0x7f6b54101c20 2024-09-11T23:21:52.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.067+0000 7f6b58b4a640 1 --2- 172.21.15.43:0/1306702972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b54101c20 0x7f6b54103000 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400e9f0 tx=0x7f6b4400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:52.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.067+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/1306702972 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4400cd80 con 0x7f6b54101c20 2024-09-11T23:21:52.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.067+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/1306702972 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b44005500 con 0x7f6b54101c20 2024-09-11T23:21:52.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.067+0000 7f6b5b5af640 1 -- 172.21.15.43:0/1306702972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b541a1fd0 con 0x7f6b54101c20 2024-09-11T23:21:52.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.067+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/1306702972 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b44010690 con 0x7f6b54101c20 2024-09-11T23:21:52.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.067+0000 7f6b5b5af640 1 -- 172.21.15.43:0/1306702972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b541a23e0 con 0x7f6b54101c20 2024-09-11T23:21:52.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.068+0000 7f6b5b5af640 1 -- 172.21.15.43:0/1306702972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b541a26d0 con 0x7f6b54101c20 2024-09-11T23:21:52.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.069+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/1306702972 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f6b440046c0 con 0x7f6b54101c20 2024-09-11T23:21:52.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.070+0000 7f6b4a7fc640 1 --2- 172.21.15.43:0/1306702972 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6b400a2990 0x7f6b400a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:52.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.071+0000 7f6b5934b640 1 --2- 172.21.15.43:0/1306702972 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6b400a2990 0x7f6b400a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:52.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.071+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/1306702972 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f6b440cf300 con 0x7f6b54101c20 2024-09-11T23:21:52.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.071+0000 7f6b5934b640 1 --2- 172.21.15.43:0/1306702972 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6b400a2990 0x7f6b400a4e50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6b50005ab0 tx=0x7f6b50000bc0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:52.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.073+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/1306702972 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b440541b0 con 0x7f6b54101c20 2024-09-11T23:21:52.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.227+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/1306702972 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 23) ==== 138004+0+0 (secure 0 0 0) 0x7f6b44098360 con 0x7f6b54101c20 2024-09-11T23:21:52.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.280+0000 7f6b5b5af640 1 -- 172.21.15.43:0/1306702972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6b541a28a0 con 0x7f6b54101c20 2024-09-11T23:21:52.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.281+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/1306702972 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) ==== 104+0+11 (secure 0 0 0) 0x7f6b4409c3f0 con 0x7f6b54101c20 2024-09-11T23:21:52.281 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-09-11T23:21:52.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.283+0000 7f6b5b5af640 1 -- 172.21.15.43:0/1306702972 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6b400a2990 msgr2=0x7f6b400a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:52.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.283+0000 7f6b5b5af640 1 --2- 172.21.15.43:0/1306702972 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6b400a2990 0x7f6b400a4e50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6b50005ab0 tx=0x7f6b50000bc0 comp rx=0 tx=0).stop 2024-09-11T23:21:52.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.283+0000 7f6b5b5af640 1 -- 172.21.15.43:0/1306702972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b54101c20 msgr2=0x7f6b54103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:52.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.283+0000 7f6b5b5af640 1 --2- 172.21.15.43:0/1306702972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b54101c20 0x7f6b54103000 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400e9f0 tx=0x7f6b4400edf0 comp rx=0 tx=0).stop 2024-09-11T23:21:52.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.283+0000 7f6b5b5af640 1 -- 172.21.15.43:0/1306702972 shutdown_connections 2024-09-11T23:21:52.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.283+0000 7f6b5b5af640 1 --2- 172.21.15.43:0/1306702972 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6b400a2990 0x7f6b400a4e50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:52.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.283+0000 7f6b5b5af640 1 --2- 172.21.15.43:0/1306702972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b541025e0 0x7f6b54103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:52.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.283+0000 7f6b5b5af640 1 --2- 172.21.15.43:0/1306702972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b54101c20 0x7f6b54103000 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:52.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.283+0000 7f6b5b5af640 1 --2- 172.21.15.43:0/1306702972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b54100a30 0x7f6b541049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:52.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.283+0000 7f6b5b5af640 1 -- 172.21.15.43:0/1306702972 >> 172.21.15.43:0/1306702972 conn(0x7f6b540fc190 msgr2=0x7f6b54069520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:52.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.283+0000 7f6b5b5af640 1 -- 172.21.15.43:0/1306702972 shutdown_connections 2024-09-11T23:21:52.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.283+0000 7f6b5b5af640 1 -- 172.21.15.43:0/1306702972 wait complete. 2024-09-11T23:21:52.298 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-11T23:21:52.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.413+0000 7fb281201640 1 -- 172.21.15.43:0/3717567594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb27c10e0f0 msgr2=0x7fb27c114980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:52.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.413+0000 7fb281201640 1 --2- 172.21.15.43:0/3717567594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb27c10e0f0 0x7fb27c114980 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb274009900 tx=0x7fb27402a410 comp rx=0 tx=0).stop 2024-09-11T23:21:52.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.413+0000 7fb281201640 1 -- 172.21.15.43:0/3717567594 shutdown_connections 2024-09-11T23:21:52.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.413+0000 7fb281201640 1 --2- 172.21.15.43:0/3717567594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb27c10e0f0 0x7fb27c114980 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:52.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.413+0000 7fb281201640 1 --2- 172.21.15.43:0/3717567594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb27c10d730 0x7fb27c10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:52.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.413+0000 7fb281201640 1 --2- 172.21.15.43:0/3717567594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb27c10c570 0x7fb27c10c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:52.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.413+0000 7fb281201640 1 -- 172.21.15.43:0/3717567594 >> 172.21.15.43:0/3717567594 conn(0x7fb27c06c1e0 msgr2=0x7fb27c06c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:52.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.413+0000 7fb281201640 1 -- 172.21.15.43:0/3717567594 shutdown_connections 2024-09-11T23:21:52.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.413+0000 7fb281201640 1 -- 172.21.15.43:0/3717567594 wait complete. 2024-09-11T23:21:52.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.414+0000 7fb281201640 1 Processor -- start 2024-09-11T23:21:52.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.414+0000 7fb281201640 1 -- start start 2024-09-11T23:21:52.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.414+0000 7fb281201640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb27c10c570 0x7fb27c1a2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:52.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.414+0000 7fb281201640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb27c10d730 0x7fb27c1a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:52.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.414+0000 7fb281201640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb27c10e0f0 0x7fb27c1a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:52.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.414+0000 7fb281201640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c1181d0 con 0x7fb27c10d730 2024-09-11T23:21:52.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.414+0000 7fb281201640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c118050 con 0x7fb27c10c570 2024-09-11T23:21:52.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.414+0000 7fb281201640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c118350 con 0x7fb27c10e0f0 2024-09-11T23:21:52.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.414+0000 7fb27a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb27c10d730 0x7fb27c1a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:52.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.414+0000 7fb27ad9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb27c10c570 0x7fb27c1a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:52.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.414+0000 7fb27b59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb27c10e0f0 0x7fb27c1a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:52.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.414+0000 7fb27ad9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb27c10c570 0x7fb27c1a2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:55984/0 (socket says 172.21.15.43:55984) 2024-09-11T23:21:52.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.414+0000 7fb27a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb27c10d730 0x7fb27c1a2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45332/0 (socket says 172.21.15.43:45332) 2024-09-11T23:21:52.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.414+0000 7fb27ad9d640 1 -- 172.21.15.43:0/1243222537 learned_addr learned my addr 172.21.15.43:0/1243222537 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:52.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.415+0000 7fb27b59e640 1 -- 172.21.15.43:0/1243222537 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb27c10c570 msgr2=0x7fb27c1a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:52.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.415+0000 7fb27b59e640 1 --2- 172.21.15.43:0/1243222537 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb27c10c570 0x7fb27c1a2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:52.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.415+0000 7fb27b59e640 1 -- 172.21.15.43:0/1243222537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb27c10d730 msgr2=0x7fb27c1a2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:52.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.415+0000 7fb27b59e640 1 --2- 172.21.15.43:0/1243222537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb27c10d730 0x7fb27c1a2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:52.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.415+0000 7fb27b59e640 1 -- 172.21.15.43:0/1243222537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27c1aa490 con 0x7fb27c10e0f0 2024-09-11T23:21:52.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.415+0000 7fb27a59c640 1 --2- 172.21.15.43:0/1243222537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb27c10d730 0x7fb27c1a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:21:52.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.415+0000 7fb27b59e640 1 --2- 172.21.15.43:0/1243222537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb27c10e0f0 0x7fb27c1a9e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb2740314e0 tx=0x7fb2740042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:52.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.415+0000 7fb263fff640 1 -- 172.21.15.43:0/1243222537 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27403f030 con 0x7fb27c10e0f0 2024-09-11T23:21:52.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.415+0000 7fb263fff640 1 -- 172.21.15.43:0/1243222537 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb274003ce0 con 0x7fb27c10e0f0 2024-09-11T23:21:52.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.415+0000 7fb281201640 1 -- 172.21.15.43:0/1243222537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27c1aa720 con 0x7fb27c10e0f0 2024-09-11T23:21:52.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.415+0000 7fb263fff640 1 -- 172.21.15.43:0/1243222537 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb274035070 con 0x7fb27c10e0f0 2024-09-11T23:21:52.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.415+0000 7fb281201640 1 -- 172.21.15.43:0/1243222537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb27c1aac10 con 0x7fb27c10e0f0 2024-09-11T23:21:52.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.416+0000 7fb281201640 1 -- 172.21.15.43:0/1243222537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb27c113920 con 0x7fb27c10e0f0 2024-09-11T23:21:52.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.417+0000 7fb263fff640 1 -- 172.21.15.43:0/1243222537 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 23) ==== 138004+0+0 (secure 0 0 0) 0x7fb274042030 con 0x7fb27c10e0f0 2024-09-11T23:21:52.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.418+0000 7fb263fff640 1 --2- 172.21.15.43:0/1243222537 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb2640a2d90 0x7fb2640a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:52.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.418+0000 7fb27ad9d640 1 --2- 172.21.15.43:0/1243222537 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb2640a2d90 0x7fb2640a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:52.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.418+0000 7fb263fff640 1 -- 172.21.15.43:0/1243222537 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb2740ef9a0 con 0x7fb27c10e0f0 2024-09-11T23:21:52.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.418+0000 7fb27ad9d640 1 --2- 172.21.15.43:0/1243222537 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb2640a2d90 0x7fb2640a5250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb26c005f40 tx=0x7fb26c009500 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:52.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.422+0000 7fb263fff640 1 -- 172.21.15.43:0/1243222537 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2740bbe00 con 0x7fb27c10e0f0 2024-09-11T23:21:52.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:52 smithi090 ceph-mon[35130]: mgrmap e22: x(active, since 1.02033s), standbys: z, y 2024-09-11T23:21:52.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:52 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/722880093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:21:52.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:52 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2408251243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:21:52.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:52 smithi090 ceph-mon[35130]: [11/Sep/2024:23:21:52] ENGINE Bus STARTING 2024-09-11T23:21:52.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:52 smithi097 ceph-mon[35562]: mgrmap e22: x(active, since 1.02033s), standbys: z, y 2024-09-11T23:21:52.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:52 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/722880093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:21:52.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:52 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2408251243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:21:52.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:52 smithi097 ceph-mon[35562]: [11/Sep/2024:23:21:52] ENGINE Bus STARTING 2024-09-11T23:21:52.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.601+0000 7fb281201640 1 -- 172.21.15.43:0/1243222537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb27c02d860 con 0x7fb27c10e0f0 2024-09-11T23:21:52.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.601+0000 7fb263fff640 1 -- 172.21.15.43:0/1243222537 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) ==== 100+0+11 (secure 0 0 0) 0x7fb2740c0100 con 0x7fb27c10e0f0 2024-09-11T23:21:52.601 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-09-11T23:21:52.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.603+0000 7fb281201640 1 -- 172.21.15.43:0/1243222537 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb2640a2d90 msgr2=0x7fb2640a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:52.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.603+0000 7fb281201640 1 --2- 172.21.15.43:0/1243222537 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb2640a2d90 0x7fb2640a5250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb26c005f40 tx=0x7fb26c009500 comp rx=0 tx=0).stop 2024-09-11T23:21:52.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.603+0000 7fb281201640 1 -- 172.21.15.43:0/1243222537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb27c10e0f0 msgr2=0x7fb27c1a9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:52.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.603+0000 7fb281201640 1 --2- 172.21.15.43:0/1243222537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb27c10e0f0 0x7fb27c1a9e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb2740314e0 tx=0x7fb2740042f0 comp rx=0 tx=0).stop 2024-09-11T23:21:52.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.603+0000 7fb281201640 1 -- 172.21.15.43:0/1243222537 shutdown_connections 2024-09-11T23:21:52.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.603+0000 7fb281201640 1 --2- 172.21.15.43:0/1243222537 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb2640a2d90 0x7fb2640a5250 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:52.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.604+0000 7fb281201640 1 --2- 172.21.15.43:0/1243222537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb27c10e0f0 0x7fb27c1a9e20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:52.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.604+0000 7fb281201640 1 --2- 172.21.15.43:0/1243222537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb27c10d730 0x7fb27c1a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:52.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.604+0000 7fb281201640 1 --2- 172.21.15.43:0/1243222537 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb27c10c570 0x7fb27c1a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:52.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.604+0000 7fb281201640 1 -- 172.21.15.43:0/1243222537 >> 172.21.15.43:0/1243222537 conn(0x7fb27c06c1e0 msgr2=0x7fb27c110af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:52.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.604+0000 7fb281201640 1 -- 172.21.15.43:0/1243222537 shutdown_connections 2024-09-11T23:21:52.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:52.604+0000 7fb281201640 1 -- 172.21.15.43:0/1243222537 wait complete. 2024-09-11T23:21:52.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-mon[30950]: mgrmap e22: x(active, since 1.02033s), standbys: z, y 2024-09-11T23:21:52.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/722880093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:21:52.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2408251243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:21:52.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-mon[30950]: [11/Sep/2024:23:21:52] ENGINE Bus STARTING 2024-09-11T23:21:52.611 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:52.280+0000 7fe5612c0640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:23:21:52] ENGINE Error in HTTPServer.serve 2024-09-11T23:21:52.611 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: Traceback (most recent call last): 2024-09-11T23:21:52.611 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:21:52.611 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: self._connections.run(self.expiration_interval) 2024-09-11T23:21:52.611 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:21:52.611 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: self._run(expiration_interval) 2024-09-11T23:21:52.611 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:21:52.611 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:21:52.611 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:21:52.611 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:21:52.611 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:21:52.611 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: s = self.context.wrap_socket( 2024-09-11T23:21:52.611 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:21:52.612 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: return self.sslsocket_class._create( 2024-09-11T23:21:52.612 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:21:52.612 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: self.do_handshake() 2024-09-11T23:21:52.612 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:21:52.612 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: self._sslobj.do_handshake() 2024-09-11T23:21:52.612 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:21:52.612 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:21:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:21:52.616 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-11T23:21:52.616 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:21:52.877 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:53.307 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: [11/Sep/2024:23:21:52] ENGINE Serving on http://172.21.15.43:8765 2024-09-11T23:21:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:21:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-11T23:21:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: [11/Sep/2024:23:21:52] ENGINE Serving on https://172.21.15.43:7150 2024-09-11T23:21:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: [11/Sep/2024:23:21:52] ENGINE Bus STARTED 2024-09-11T23:21:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1306702972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:21:53.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: [11/Sep/2024:23:21:52] ENGINE Error in HTTPServer.serve 2024-09-11T23:21:53.315 INFO:journalctl@ceph.mon.a.smithi043.stdout: Traceback (most recent call last): 2024-09-11T23:21:53.315 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:21:53.315 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._connections.run(self.expiration_interval) 2024-09-11T23:21:53.315 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:21:53.315 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._run(expiration_interval) 2024-09-11T23:21:53.315 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:21:53.315 INFO:journalctl@ceph.mon.a.smithi043.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:21:53.315 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:21:53.315 INFO:journalctl@ceph.mon.a.smithi043.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:21:53.315 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:21:53.315 INFO:journalctl@ceph.mon.a.smithi043.stdout: s = self.context.wrap_socket( 2024-09-11T23:21:53.315 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:21:53.316 INFO:journalctl@ceph.mon.a.smithi043.stdout: return self.sslsocket_class._create( 2024-09-11T23:21:53.316 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:21:53.316 INFO:journalctl@ceph.mon.a.smithi043.stdout: self.do_handshake() 2024-09-11T23:21:53.316 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:21:53.316 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._sslobj.do_handshake() 2024-09-11T23:21:53.316 INFO:journalctl@ceph.mon.a.smithi043.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:21:53.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1243222537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:21:53.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.317 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.317 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.317 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.317 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.317 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.317 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.317 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: Adjusting osd_memory_target on smithi097 to 176.0M 2024-09-11T23:21:53.317 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.317 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: Unable to set osd_memory_target on smithi097 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-11T23:21:53.317 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.317 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: Adjusting osd_memory_target on smithi090 to 176.0M 2024-09-11T23:21:53.317 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: Unable to set osd_memory_target on smithi090 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-11T23:21:53.317 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:53 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.395+0000 7f43dface640 1 -- 172.21.15.43:0/675432903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d8108290 msgr2=0x7f43d810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:53.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.395+0000 7f43dface640 1 --2- 172.21.15.43:0/675432903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d8108290 0x7f43d810ec30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009930 tx=0x7f43d002a420 comp rx=0 tx=0).stop 2024-09-11T23:21:53.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.395+0000 7f43dface640 1 -- 172.21.15.43:0/675432903 shutdown_connections 2024-09-11T23:21:53.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.395+0000 7f43dface640 1 --2- 172.21.15.43:0/675432903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d8108290 0x7f43d810ec30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.395+0000 7f43dface640 1 --2- 172.21.15.43:0/675432903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43d8103740 0x7f43d8107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.395+0000 7f43dface640 1 --2- 172.21.15.43:0/675432903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43d8102d90 0x7f43d8103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.395+0000 7f43dface640 1 -- 172.21.15.43:0/675432903 >> 172.21.15.43:0/675432903 conn(0x7f43d80fc7d0 msgr2=0x7f43d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:53.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.395+0000 7f43dface640 1 -- 172.21.15.43:0/675432903 shutdown_connections 2024-09-11T23:21:53.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.395+0000 7f43dface640 1 -- 172.21.15.43:0/675432903 wait complete. 2024-09-11T23:21:53.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.396+0000 7f43dface640 1 Processor -- start 2024-09-11T23:21:53.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.396+0000 7f43dface640 1 -- start start 2024-09-11T23:21:53.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.396+0000 7f43dface640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d8102d90 0x7f43d8198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:53.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.396+0000 7f43dface640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43d8103740 0x7f43d8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:53.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.396+0000 7f43dface640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43d8108290 0x7f43d819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:53.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.396+0000 7f43dface640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43d8112090 con 0x7f43d8102d90 2024-09-11T23:21:53.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.396+0000 7f43dface640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f43d8111f10 con 0x7f43d8108290 2024-09-11T23:21:53.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.396+0000 7f43dface640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f43d8112210 con 0x7f43d8103740 2024-09-11T23:21:53.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.396+0000 7f43dd86a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d8102d90 0x7f43d8198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:53.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.396+0000 7f43dd86a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d8102d90 0x7f43d8198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45362/0 (socket says 172.21.15.43:45362) 2024-09-11T23:21:53.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.396+0000 7f43dd86a640 1 -- 172.21.15.43:0/3728865702 learned_addr learned my addr 172.21.15.43:0/3728865702 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:53.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.396+0000 7f43de06b640 1 --2- 172.21.15.43:0/3728865702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43d8108290 0x7f43d819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:53.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.397+0000 7f43dd069640 1 --2- 172.21.15.43:0/3728865702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43d8103740 0x7f43d8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:53.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.397+0000 7f43dd86a640 1 -- 172.21.15.43:0/3728865702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43d8103740 msgr2=0x7f43d8198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:53.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.397+0000 7f43dd86a640 1 --2- 172.21.15.43:0/3728865702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43d8103740 0x7f43d8198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.397+0000 7f43dd86a640 1 -- 172.21.15.43:0/3728865702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43d8108290 msgr2=0x7f43d819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:53.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.397+0000 7f43dd86a640 1 --2- 172.21.15.43:0/3728865702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43d8108290 0x7f43d819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.397+0000 7f43dd86a640 1 -- 172.21.15.43:0/3728865702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d819d500 con 0x7f43d8102d90 2024-09-11T23:21:53.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.397+0000 7f43de06b640 1 --2- 172.21.15.43:0/3728865702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43d8108290 0x7f43d819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:53.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.397+0000 7f43dd86a640 1 --2- 172.21.15.43:0/3728865702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d8102d90 0x7f43d8198550 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f43c800ebc0 tx=0x7f43c800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:53.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.397+0000 7f43d6ffd640 1 -- 172.21.15.43:0/3728865702 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8010630 con 0x7f43d8102d90 2024-09-11T23:21:53.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.397+0000 7f43d6ffd640 1 -- 172.21.15.43:0/3728865702 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43c8010cb0 con 0x7f43d8102d90 2024-09-11T23:21:53.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.397+0000 7f43d6ffd640 1 -- 172.21.15.43:0/3728865702 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8018bd0 con 0x7f43d8102d90 2024-09-11T23:21:53.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.397+0000 7f43dface640 1 -- 172.21.15.43:0/3728865702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d819d7f0 con 0x7f43d8102d90 2024-09-11T23:21:53.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.397+0000 7f43dface640 1 -- 172.21.15.43:0/3728865702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d81a50d0 con 0x7f43d8102d90 2024-09-11T23:21:53.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.399+0000 7f43dface640 1 -- 172.21.15.43:0/3728865702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d8103170 con 0x7f43d8102d90 2024-09-11T23:21:53.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.400+0000 7f43d6ffd640 1 -- 172.21.15.43:0/3728865702 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 138004+0+0 (secure 0 0 0) 0x7f43c80107d0 con 0x7f43d8102d90 2024-09-11T23:21:53.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.400+0000 7f43d6ffd640 1 --2- 172.21.15.43:0/3728865702 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f43c00a2d80 0x7f43c00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:53.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.400+0000 7f43d6ffd640 1 -- 172.21.15.43:0/3728865702 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f43c80cfc80 con 0x7f43d8102d90 2024-09-11T23:21:53.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.400+0000 7f43dd069640 1 --2- 172.21.15.43:0/3728865702 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f43c00a2d80 0x7f43c00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:53.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.401+0000 7f43dd069640 1 --2- 172.21.15.43:0/3728865702 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f43c00a2d80 0x7f43c00a5240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0095f0 tx=0x7f43cc005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:53.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.405+0000 7f43d6ffd640 1 -- 172.21.15.43:0/3728865702 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f43c809c730 con 0x7f43d8102d90 2024-09-11T23:21:53.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: [11/Sep/2024:23:21:52] ENGINE Serving on http://172.21.15.43:8765 2024-09-11T23:21:53.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:21:53.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-11T23:21:53.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: [11/Sep/2024:23:21:52] ENGINE Serving on https://172.21.15.43:7150 2024-09-11T23:21:53.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: [11/Sep/2024:23:21:52] ENGINE Bus STARTED 2024-09-11T23:21:53.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1306702972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:21:53.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: [11/Sep/2024:23:21:52] ENGINE Error in HTTPServer.serve 2024-09-11T23:21:53.521 INFO:journalctl@ceph.mon.b.smithi090.stdout: Traceback (most recent call last): 2024-09-11T23:21:53.521 INFO:journalctl@ceph.mon.b.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:21:53.521 INFO:journalctl@ceph.mon.b.smithi090.stdout: self._connections.run(self.expiration_interval) 2024-09-11T23:21:53.521 INFO:journalctl@ceph.mon.b.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:21:53.521 INFO:journalctl@ceph.mon.b.smithi090.stdout: self._run(expiration_interval) 2024-09-11T23:21:53.522 INFO:journalctl@ceph.mon.b.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:21:53.522 INFO:journalctl@ceph.mon.b.smithi090.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:21:53.522 INFO:journalctl@ceph.mon.b.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:21:53.522 INFO:journalctl@ceph.mon.b.smithi090.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:21:53.522 INFO:journalctl@ceph.mon.b.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:21:53.522 INFO:journalctl@ceph.mon.b.smithi090.stdout: s = self.context.wrap_socket( 2024-09-11T23:21:53.522 INFO:journalctl@ceph.mon.b.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:21:53.522 INFO:journalctl@ceph.mon.b.smithi090.stdout: return self.sslsocket_class._create( 2024-09-11T23:21:53.522 INFO:journalctl@ceph.mon.b.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:21:53.522 INFO:journalctl@ceph.mon.b.smithi090.stdout: self.do_handshake() 2024-09-11T23:21:53.522 INFO:journalctl@ceph.mon.b.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:21:53.523 INFO:journalctl@ceph.mon.b.smithi090.stdout: self._sslobj.do_handshake() 2024-09-11T23:21:53.523 INFO:journalctl@ceph.mon.b.smithi090.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:21:53.523 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.523 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.523 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1243222537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:21:53.523 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.523 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.523 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.523 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.523 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.523 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: Adjusting osd_memory_target on smithi097 to 176.0M 2024-09-11T23:21:53.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: Unable to set osd_memory_target on smithi097 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-11T23:21:53.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: Adjusting osd_memory_target on smithi090 to 176.0M 2024-09-11T23:21:53.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: Unable to set osd_memory_target on smithi090 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-11T23:21:53.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:53 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: [11/Sep/2024:23:21:52] ENGINE Serving on http://172.21.15.43:8765 2024-09-11T23:21:53.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:21:53.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-11T23:21:53.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: [11/Sep/2024:23:21:52] ENGINE Serving on https://172.21.15.43:7150 2024-09-11T23:21:53.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: [11/Sep/2024:23:21:52] ENGINE Bus STARTED 2024-09-11T23:21:53.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1306702972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:21:53.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: [11/Sep/2024:23:21:52] ENGINE Error in HTTPServer.serve 2024-09-11T23:21:53.552 INFO:journalctl@ceph.mon.c.smithi097.stdout: Traceback (most recent call last): 2024-09-11T23:21:53.552 INFO:journalctl@ceph.mon.c.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T23:21:53.552 INFO:journalctl@ceph.mon.c.smithi097.stdout: self._connections.run(self.expiration_interval) 2024-09-11T23:21:53.552 INFO:journalctl@ceph.mon.c.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T23:21:53.552 INFO:journalctl@ceph.mon.c.smithi097.stdout: self._run(expiration_interval) 2024-09-11T23:21:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T23:21:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T23:21:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T23:21:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T23:21:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T23:21:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout: s = self.context.wrap_socket( 2024-09-11T23:21:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T23:21:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout: return self.sslsocket_class._create( 2024-09-11T23:21:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T23:21:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout: self.do_handshake() 2024-09-11T23:21:53.553 INFO:journalctl@ceph.mon.c.smithi097.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T23:21:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout: self._sslobj.do_handshake() 2024-09-11T23:21:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T23:21:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1243222537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:21:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: Adjusting osd_memory_target on smithi097 to 176.0M 2024-09-11T23:21:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: Unable to set osd_memory_target on smithi097 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-11T23:21:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: Adjusting osd_memory_target on smithi090 to 176.0M 2024-09-11T23:21:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: Unable to set osd_memory_target on smithi090 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-11T23:21:53.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:53 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:53.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.642+0000 7f43dface640 1 -- 172.21.15.43:0/3728865702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43d810a2a0 con 0x7f43d8102d90 2024-09-11T23:21:53.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.643+0000 7f43d6ffd640 1 -- 172.21.15.43:0/3728865702 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11896 (secure 0 0 0) 0x7f43c809c030 con 0x7f43d8102d90 2024-09-11T23:21:53.643 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:21:53.643 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"btime":"2024-09-11T23:21:49:167551+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:21:49.117342+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:21:53.643 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 24 2024-09-11T23:21:53.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.645+0000 7f43dface640 1 -- 172.21.15.43:0/3728865702 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f43c00a2d80 msgr2=0x7f43c00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:53.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.645+0000 7f43dface640 1 --2- 172.21.15.43:0/3728865702 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f43c00a2d80 0x7f43c00a5240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0095f0 tx=0x7f43cc005f90 comp rx=0 tx=0).stop 2024-09-11T23:21:53.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.645+0000 7f43dface640 1 -- 172.21.15.43:0/3728865702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d8102d90 msgr2=0x7f43d8198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:53.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.645+0000 7f43dface640 1 --2- 172.21.15.43:0/3728865702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d8102d90 0x7f43d8198550 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f43c800ebc0 tx=0x7f43c800c6a0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.645+0000 7f43dface640 1 -- 172.21.15.43:0/3728865702 shutdown_connections 2024-09-11T23:21:53.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.645+0000 7f43dface640 1 --2- 172.21.15.43:0/3728865702 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f43c00a2d80 0x7f43c00a5240 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.645+0000 7f43dface640 1 --2- 172.21.15.43:0/3728865702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43d8108290 0x7f43d819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.645+0000 7f43dface640 1 --2- 172.21.15.43:0/3728865702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43d8103740 0x7f43d8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.645+0000 7f43dface640 1 --2- 172.21.15.43:0/3728865702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d8102d90 0x7f43d8198550 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:53.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.645+0000 7f43dface640 1 -- 172.21.15.43:0/3728865702 >> 172.21.15.43:0/3728865702 conn(0x7f43d80fc7d0 msgr2=0x7f43d8105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:53.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.645+0000 7f43dface640 1 -- 172.21.15.43:0/3728865702 shutdown_connections 2024-09-11T23:21:53.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:53.645+0000 7f43dface640 1 -- 172.21.15.43:0/3728865702 wait complete. 2024-09-11T23:21:54.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:54.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: Adjusting osd_memory_target on smithi043 to 177.0M 2024-09-11T23:21:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: Unable to set osd_memory_target on smithi043 to 185687756: error parsing value: Value '185687756' is below minimum 939524096 2024-09-11T23:21:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:21:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:21:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-11T23:21:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-11T23:21:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: Updating smithi097:/etc/ceph/ceph.conf 2024-09-11T23:21:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: Updating smithi090:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:21:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: Updating smithi097:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:21:54.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: Updating smithi043:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:21:54.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3728865702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:54.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:21:54.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:21:54.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:21:54.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: Updating smithi090:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:21:54.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: Updating smithi097:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:21:54.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: Updating smithi043:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:21:54.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:54.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:54.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:54.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:54 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:54.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:54.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:54.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:54.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:54.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:54.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: Adjusting osd_memory_target on smithi043 to 177.0M 2024-09-11T23:21:54.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: Unable to set osd_memory_target on smithi043 to 185687756: error parsing value: Value '185687756' is below minimum 939524096 2024-09-11T23:21:54.522 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:21:54.522 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:21:54.522 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-11T23:21:54.522 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-11T23:21:54.522 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: Updating smithi097:/etc/ceph/ceph.conf 2024-09-11T23:21:54.522 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: Updating smithi090:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:21:54.523 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: Updating smithi097:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:21:54.523 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: Updating smithi043:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:21:54.523 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3728865702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:54.523 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:21:54.523 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:21:54.523 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:21:54.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: Updating smithi090:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:21:54.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: Updating smithi097:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:21:54.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: Updating smithi043:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:21:54.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:54.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:54.525 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:54.525 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:54 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:54.527 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-11T23:21:54.527 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-11T23:21:54.527 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T23:21:54.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:54.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:54.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:54.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:54.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:21:54.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: Adjusting osd_memory_target on smithi043 to 177.0M 2024-09-11T23:21:54.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: Unable to set osd_memory_target on smithi043 to 185687756: error parsing value: Value '185687756' is below minimum 939524096 2024-09-11T23:21:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:21:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:21:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-11T23:21:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-11T23:21:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: Updating smithi097:/etc/ceph/ceph.conf 2024-09-11T23:21:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: Updating smithi090:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:21:54.553 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: Updating smithi097:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:21:54.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: Updating smithi043:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T23:21:54.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3728865702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:54.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:21:54.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:21:54.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-11T23:21:54.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: Updating smithi090:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:21:54.554 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: Updating smithi097:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:21:54.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: Updating smithi043:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T23:21:54.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:54.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:54.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:54.555 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:54 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:54.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.632+0000 7ff9094a5640 1 -- 172.21.15.43:0/3074618816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9041009d0 msgr2=0x7ff904100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.632+0000 7ff9094a5640 1 --2- 172.21.15.43:0/3074618816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9041009d0 0x7ff904100e30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009930 tx=0x7ff8ec02a420 comp rx=0 tx=0).stop 2024-09-11T23:21:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.633+0000 7ff9094a5640 1 -- 172.21.15.43:0/3074618816 shutdown_connections 2024-09-11T23:21:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.633+0000 7ff9094a5640 1 --2- 172.21.15.43:0/3074618816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff904101370 0x7ff90410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.633+0000 7ff9094a5640 1 --2- 172.21.15.43:0/3074618816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9041009d0 0x7ff904100e30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.633+0000 7ff9094a5640 1 --2- 172.21.15.43:0/3074618816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9041069e0 0x7ff904106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.633+0000 7ff9094a5640 1 -- 172.21.15.43:0/3074618816 >> 172.21.15.43:0/3074618816 conn(0x7ff9040fc6b0 msgr2=0x7ff9040fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.633+0000 7ff9094a5640 1 -- 172.21.15.43:0/3074618816 shutdown_connections 2024-09-11T23:21:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.633+0000 7ff9094a5640 1 -- 172.21.15.43:0/3074618816 wait complete. 2024-09-11T23:21:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.633+0000 7ff9094a5640 1 Processor -- start 2024-09-11T23:21:54.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.633+0000 7ff9094a5640 1 -- start start 2024-09-11T23:21:54.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.633+0000 7ff9094a5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9041009d0 0x7ff904198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:54.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.633+0000 7ff9094a5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff904101370 0x7ff904198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:54.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.633+0000 7ff9094a5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9041069e0 0x7ff90419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:54.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.633+0000 7ff9094a5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff904110e20 con 0x7ff904101370 2024-09-11T23:21:54.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.633+0000 7ff9094a5640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff904110ca0 con 0x7ff9041009d0 2024-09-11T23:21:54.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.633+0000 7ff9094a5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff904110fa0 con 0x7ff9041069e0 2024-09-11T23:21:54.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.634+0000 7ff8fbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff904101370 0x7ff904198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:54.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.634+0000 7ff8fbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff904101370 0x7ff904198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45370/0 (socket says 172.21.15.43:45370) 2024-09-11T23:21:54.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.634+0000 7ff8fbfff640 1 -- 172.21.15.43:0/2938020185 learned_addr learned my addr 172.21.15.43:0/2938020185 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:54.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.634+0000 7ff902ffd640 1 --2- 172.21.15.43:0/2938020185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9041009d0 0x7ff904198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:54.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.634+0000 7ff9037fe640 1 --2- 172.21.15.43:0/2938020185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9041069e0 0x7ff90419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:54.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.634+0000 7ff8fbfff640 1 -- 172.21.15.43:0/2938020185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9041069e0 msgr2=0x7ff90419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:54.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.634+0000 7ff8fbfff640 1 --2- 172.21.15.43:0/2938020185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9041069e0 0x7ff90419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.634+0000 7ff8fbfff640 1 -- 172.21.15.43:0/2938020185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9041009d0 msgr2=0x7ff904198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:54.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.634+0000 7ff8fbfff640 1 --2- 172.21.15.43:0/2938020185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9041009d0 0x7ff904198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.634+0000 7ff8fbfff640 1 -- 172.21.15.43:0/2938020185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff90419d430 con 0x7ff904101370 2024-09-11T23:21:54.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.634+0000 7ff902ffd640 1 --2- 172.21.15.43:0/2938020185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9041009d0 0x7ff904198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:21:54.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.634+0000 7ff8fbfff640 1 --2- 172.21.15.43:0/2938020185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff904101370 0x7ff904198980 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009230 tx=0x7ff8ec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:54.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.634+0000 7ff900ff9640 1 -- 172.21.15.43:0/2938020185 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec03f030 con 0x7ff904101370 2024-09-11T23:21:54.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.634+0000 7ff900ff9640 1 -- 172.21.15.43:0/2938020185 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8ec003ad0 con 0x7ff904101370 2024-09-11T23:21:54.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.634+0000 7ff900ff9640 1 -- 172.21.15.43:0/2938020185 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec035070 con 0x7ff904101370 2024-09-11T23:21:54.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.634+0000 7ff9094a5640 1 -- 172.21.15.43:0/2938020185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff90419d6c0 con 0x7ff904101370 2024-09-11T23:21:54.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.634+0000 7ff9094a5640 1 -- 172.21.15.43:0/2938020185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9041a4fb0 con 0x7ff904101370 2024-09-11T23:21:54.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.635+0000 7ff9094a5640 1 -- 172.21.15.43:0/2938020185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff904102110 con 0x7ff904101370 2024-09-11T23:21:54.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.636+0000 7ff900ff9640 1 -- 172.21.15.43:0/2938020185 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 138004+0+0 (secure 0 0 0) 0x7ff8ec042030 con 0x7ff904101370 2024-09-11T23:21:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.637+0000 7ff900ff9640 1 --2- 172.21.15.43:0/2938020185 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff8f00a2d90 0x7ff8f00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.637+0000 7ff900ff9640 1 -- 172.21.15.43:0/2938020185 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff8ec0ef8b0 con 0x7ff904101370 2024-09-11T23:21:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.637+0000 7ff902ffd640 1 --2- 172.21.15.43:0/2938020185 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff8f00a2d90 0x7ff8f00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.638+0000 7ff902ffd640 1 --2- 172.21.15.43:0/2938020185 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff8f00a2d90 0x7ff8f00a5250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400bf50 tx=0x7ff8f400b390 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:54.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.641+0000 7ff900ff9640 1 -- 172.21.15.43:0/2938020185 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff8ec0bbcb0 con 0x7ff904101370 2024-09-11T23:21:54.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.814+0000 7ff9094a5640 1 -- 172.21.15.43:0/2938020185 --> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff9040fcd50 con 0x7ff8f00a2d90 2024-09-11T23:21:54.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.821+0000 7ff900ff9640 1 -- 172.21.15.43:0/2938020185 <== mgr.15126 v2:172.21.15.43:6800/2510386404 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258002 (secure 0 0 0) 0x7ff9040fcd50 con 0x7ff8f00a2d90 2024-09-11T23:21:54.826 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:21:54.829 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-11T23:21:54.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.824+0000 7ff9094a5640 1 -- 172.21.15.43:0/2938020185 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff8f00a2d90 msgr2=0x7ff8f00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:54.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.824+0000 7ff9094a5640 1 --2- 172.21.15.43:0/2938020185 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff8f00a2d90 0x7ff8f00a5250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400bf50 tx=0x7ff8f400b390 comp rx=0 tx=0).stop 2024-09-11T23:21:54.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.824+0000 7ff9094a5640 1 -- 172.21.15.43:0/2938020185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff904101370 msgr2=0x7ff904198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:54.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.824+0000 7ff9094a5640 1 --2- 172.21.15.43:0/2938020185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff904101370 0x7ff904198980 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009230 tx=0x7ff8ec004270 comp rx=0 tx=0).stop 2024-09-11T23:21:54.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.824+0000 7ff9094a5640 1 -- 172.21.15.43:0/2938020185 shutdown_connections 2024-09-11T23:21:54.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.824+0000 7ff9094a5640 1 --2- 172.21.15.43:0/2938020185 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff8f00a2d90 0x7ff8f00a5250 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.824+0000 7ff9094a5640 1 --2- 172.21.15.43:0/2938020185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9041069e0 0x7ff90419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.824+0000 7ff9094a5640 1 --2- 172.21.15.43:0/2938020185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff904101370 0x7ff904198980 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.824+0000 7ff9094a5640 1 --2- 172.21.15.43:0/2938020185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9041009d0 0x7ff904198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.824+0000 7ff9094a5640 1 -- 172.21.15.43:0/2938020185 >> 172.21.15.43:0/2938020185 conn(0x7ff9040fc6b0 msgr2=0x7ff9041047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:54.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.825+0000 7ff9094a5640 1 -- 172.21.15.43:0/2938020185 shutdown_connections 2024-09-11T23:21:54.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.825+0000 7ff9094a5640 1 -- 172.21.15.43:0/2938020185 wait complete. 2024-09-11T23:21:54.836 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-11T23:21:54.223395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187806,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":867,"num_read_kb":594,"num_write":474,"num_write_kb":4377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369056,"kb_used_data":11460,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572768,"statfs":{"total":1151940427776,"available":1151562514432,"internally_reserved":0,"allocated":11735040,"data_stored":8948064,"data_compressed":283387,"data_compressed_allocated":372736,"data_compressed_original":3596288,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987422"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598561+0000","last_change":"2024-09-11T23:19:59.734378+0000","last_active":"2024-09-11T23:21:50.598561+0000","last_peered":"2024-09-11T23:21:50.598561+0000","last_clean":"2024-09-11T23:21:50.598561+0000","last_became_active":"2024-09-11T23:19:59.734067+0000","last_became_peered":"2024-09-11T23:19:59.734067+0000","last_unstale":"2024-09-11T23:21:50.598561+0000","last_undegraded":"2024-09-11T23:21:50.598561+0000","last_fullsized":"2024-09-11T23:21:50.598561+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:28:18.384453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.454939+0000","last_change":"2024-09-11T23:19:59.733892+0000","last_active":"2024-09-11T23:21:50.454939+0000","last_peered":"2024-09-11T23:21:50.454939+0000","last_clean":"2024-09-11T23:21:50.454939+0000","last_became_active":"2024-09-11T23:19:59.733540+0000","last_became_peered":"2024-09-11T23:19:59.733540+0000","last_unstale":"2024-09-11T23:21:50.454939+0000","last_undegraded":"2024-09-11T23:21:50.454939+0000","last_fullsized":"2024-09-11T23:21:50.454939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:42.186458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455001+0000","last_change":"2024-09-11T23:19:59.739443+0000","last_active":"2024-09-11T23:21:50.455001+0000","last_peered":"2024-09-11T23:21:50.455001+0000","last_clean":"2024-09-11T23:21:50.455001+0000","last_became_active":"2024-09-11T23:19:59.739025+0000","last_became_peered":"2024-09-11T23:19:59.739025+0000","last_unstale":"2024-09-11T23:21:50.455001+0000","last_undegraded":"2024-09-11T23:21:50.455001+0000","last_fullsized":"2024-09-11T23:21:50.455001+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:40.819321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297376+0000","last_change":"2024-09-11T23:21:02.363511+0000","last_active":"2024-09-11T23:21:50.297376+0000","last_peered":"2024-09-11T23:21:50.297376+0000","last_clean":"2024-09-11T23:21:50.297376+0000","last_became_active":"2024-09-11T23:21:02.363156+0000","last_became_peered":"2024-09-11T23:21:02.363156+0000","last_unstale":"2024-09-11T23:21:50.297376+0000","last_undegraded":"2024-09-11T23:21:50.297376+0000","last_fullsized":"2024-09-11T23:21:50.297376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:17.059509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299585+0000","last_change":"2024-09-11T23:19:59.738662+0000","last_active":"2024-09-11T23:21:50.299585+0000","last_peered":"2024-09-11T23:21:50.299585+0000","last_clean":"2024-09-11T23:21:50.299585+0000","last_became_active":"2024-09-11T23:19:59.738320+0000","last_became_peered":"2024-09-11T23:19:59.738320+0000","last_unstale":"2024-09-11T23:21:50.299585+0000","last_undegraded":"2024-09-11T23:21:50.299585+0000","last_fullsized":"2024-09-11T23:21:50.299585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:15.645179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.310947+0000","last_change":"2024-09-11T23:19:59.731442+0000","last_active":"2024-09-11T23:21:50.310947+0000","last_peered":"2024-09-11T23:21:50.310947+0000","last_clean":"2024-09-11T23:21:50.310947+0000","last_became_active":"2024-09-11T23:19:59.731079+0000","last_became_peered":"2024-09-11T23:19:59.731079+0000","last_unstale":"2024-09-11T23:21:50.310947+0000","last_undegraded":"2024-09-11T23:21:50.310947+0000","last_fullsized":"2024-09-11T23:21:50.310947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:32.764991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455210+0000","last_change":"2024-09-11T23:21:02.359992+0000","last_active":"2024-09-11T23:21:50.455210+0000","last_peered":"2024-09-11T23:21:50.455210+0000","last_clean":"2024-09-11T23:21:50.455210+0000","last_became_active":"2024-09-11T23:21:02.359564+0000","last_became_peered":"2024-09-11T23:21:02.359564+0000","last_unstale":"2024-09-11T23:21:50.455210+0000","last_undegraded":"2024-09-11T23:21:50.455210+0000","last_fullsized":"2024-09-11T23:21:50.455210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:53.633219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300215+0000","last_change":"2024-09-11T23:19:59.740733+0000","last_active":"2024-09-11T23:21:50.300215+0000","last_peered":"2024-09-11T23:21:50.300215+0000","last_clean":"2024-09-11T23:21:50.300215+0000","last_became_active":"2024-09-11T23:19:59.740438+0000","last_became_peered":"2024-09-11T23:19:59.740438+0000","last_unstale":"2024-09-11T23:21:50.300215+0000","last_undegraded":"2024-09-11T23:21:50.300215+0000","last_fullsized":"2024-09-11T23:21:50.300215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:27.063305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.212024+0000","last_change":"2024-09-11T23:19:59.742439+0000","last_active":"2024-09-11T23:21:50.212024+0000","last_peered":"2024-09-11T23:21:50.212024+0000","last_clean":"2024-09-11T23:21:50.212024+0000","last_became_active":"2024-09-11T23:19:59.741983+0000","last_became_peered":"2024-09-11T23:19:59.741983+0000","last_unstale":"2024-09-11T23:21:50.212024+0000","last_undegraded":"2024-09-11T23:21:50.212024+0000","last_fullsized":"2024-09-11T23:21:50.212024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:39.477167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311317+0000","last_change":"2024-09-11T23:19:59.732544+0000","last_active":"2024-09-11T23:21:50.311317+0000","last_peered":"2024-09-11T23:21:50.311317+0000","last_clean":"2024-09-11T23:21:50.311317+0000","last_became_active":"2024-09-11T23:19:59.732205+0000","last_became_peered":"2024-09-11T23:19:59.732205+0000","last_unstale":"2024-09-11T23:21:50.311317+0000","last_undegraded":"2024-09-11T23:21:50.311317+0000","last_fullsized":"2024-09-11T23:21:50.311317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:22.494344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455518+0000","last_change":"2024-09-11T23:19:59.741857+0000","last_active":"2024-09-11T23:21:50.455518+0000","last_peered":"2024-09-11T23:21:50.455518+0000","last_clean":"2024-09-11T23:21:50.455518+0000","last_became_active":"2024-09-11T23:19:59.741484+0000","last_became_peered":"2024-09-11T23:19:59.741484+0000","last_unstale":"2024-09-11T23:21:50.455518+0000","last_undegraded":"2024-09-11T23:21:50.455518+0000","last_fullsized":"2024-09-11T23:21:50.455518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:52.998258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.453834+0000","last_change":"2024-09-11T23:19:59.736042+0000","last_active":"2024-09-11T23:21:50.453834+0000","last_peered":"2024-09-11T23:21:50.453834+0000","last_clean":"2024-09-11T23:21:50.453834+0000","last_became_active":"2024-09-11T23:19:59.735731+0000","last_became_peered":"2024-09-11T23:19:59.735731+0000","last_unstale":"2024-09-11T23:21:50.453834+0000","last_undegraded":"2024-09-11T23:21:50.453834+0000","last_fullsized":"2024-09-11T23:21:50.453834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:19.085804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210129+0000","last_change":"2024-09-11T23:19:59.742049+0000","last_active":"2024-09-11T23:21:50.210129+0000","last_peered":"2024-09-11T23:21:50.210129+0000","last_clean":"2024-09-11T23:21:50.210129+0000","last_became_active":"2024-09-11T23:19:59.740974+0000","last_became_peered":"2024-09-11T23:19:59.740974+0000","last_unstale":"2024-09-11T23:21:50.210129+0000","last_undegraded":"2024-09-11T23:21:50.210129+0000","last_fullsized":"2024-09-11T23:21:50.210129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:10.443113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.454569+0000","last_change":"2024-09-11T23:19:59.734270+0000","last_active":"2024-09-11T23:21:50.454569+0000","last_peered":"2024-09-11T23:21:50.454569+0000","last_clean":"2024-09-11T23:21:50.454569+0000","last_became_active":"2024-09-11T23:19:59.733967+0000","last_became_peered":"2024-09-11T23:19:59.733967+0000","last_unstale":"2024-09-11T23:21:50.454569+0000","last_undegraded":"2024-09-11T23:21:50.454569+0000","last_fullsized":"2024-09-11T23:21:50.454569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:02.185757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.454215+0000","last_change":"2024-09-11T23:19:59.734152+0000","last_active":"2024-09-11T23:21:50.454215+0000","last_peered":"2024-09-11T23:21:50.454215+0000","last_clean":"2024-09-11T23:21:50.454215+0000","last_became_active":"2024-09-11T23:19:59.733163+0000","last_became_peered":"2024-09-11T23:19:59.733163+0000","last_unstale":"2024-09-11T23:21:50.454215+0000","last_undegraded":"2024-09-11T23:21:50.454215+0000","last_fullsized":"2024-09-11T23:21:50.454215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:22.370591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211142+0000","last_change":"2024-09-11T23:19:59.735517+0000","last_active":"2024-09-11T23:21:50.211142+0000","last_peered":"2024-09-11T23:21:50.211142+0000","last_clean":"2024-09-11T23:21:50.211142+0000","last_became_active":"2024-09-11T23:19:59.735182+0000","last_became_peered":"2024-09-11T23:19:59.735182+0000","last_unstale":"2024-09-11T23:21:50.211142+0000","last_undegraded":"2024-09-11T23:21:50.211142+0000","last_fullsized":"2024-09-11T23:21:50.211142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:25.989416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598585+0000","last_change":"2024-09-11T23:19:59.733579+0000","last_active":"2024-09-11T23:21:50.598585+0000","last_peered":"2024-09-11T23:21:50.598585+0000","last_clean":"2024-09-11T23:21:50.598585+0000","last_became_active":"2024-09-11T23:19:59.733245+0000","last_became_peered":"2024-09-11T23:19:59.733245+0000","last_unstale":"2024-09-11T23:21:50.598585+0000","last_undegraded":"2024-09-11T23:21:50.598585+0000","last_fullsized":"2024-09-11T23:21:50.598585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:22.361158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297955+0000","last_change":"2024-09-11T23:19:59.734354+0000","last_active":"2024-09-11T23:21:50.297955+0000","last_peered":"2024-09-11T23:21:50.297955+0000","last_clean":"2024-09-11T23:21:50.297955+0000","last_became_active":"2024-09-11T23:19:59.733988+0000","last_became_peered":"2024-09-11T23:19:59.733988+0000","last_unstale":"2024-09-11T23:21:50.297955+0000","last_undegraded":"2024-09-11T23:21:50.297955+0000","last_fullsized":"2024-09-11T23:21:50.297955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:41.401496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.212032+0000","last_change":"2024-09-11T23:19:59.730318+0000","last_active":"2024-09-11T23:21:50.212032+0000","last_peered":"2024-09-11T23:21:50.212032+0000","last_clean":"2024-09-11T23:21:50.212032+0000","last_became_active":"2024-09-11T23:19:59.730009+0000","last_became_peered":"2024-09-11T23:19:59.730009+0000","last_unstale":"2024-09-11T23:21:50.212032+0000","last_undegraded":"2024-09-11T23:21:50.212032+0000","last_fullsized":"2024-09-11T23:21:50.212032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:01:38.531967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598328+0000","last_change":"2024-09-11T23:19:59.737254+0000","last_active":"2024-09-11T23:21:50.598328+0000","last_peered":"2024-09-11T23:21:50.598328+0000","last_clean":"2024-09-11T23:21:50.598328+0000","last_became_active":"2024-09-11T23:19:59.736936+0000","last_became_peered":"2024-09-11T23:19:59.736936+0000","last_unstale":"2024-09-11T23:21:50.598328+0000","last_undegraded":"2024-09-11T23:21:50.598328+0000","last_fullsized":"2024-09-11T23:21:50.598328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:00.361468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210147+0000","last_change":"2024-09-11T23:19:59.741617+0000","last_active":"2024-09-11T23:21:50.210147+0000","last_peered":"2024-09-11T23:21:50.210147+0000","last_clean":"2024-09-11T23:21:50.210147+0000","last_became_active":"2024-09-11T23:19:59.740358+0000","last_became_peered":"2024-09-11T23:19:59.740358+0000","last_unstale":"2024-09-11T23:21:50.210147+0000","last_undegraded":"2024-09-11T23:21:50.210147+0000","last_fullsized":"2024-09-11T23:21:50.210147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:18.541105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211292+0000","last_change":"2024-09-11T23:19:59.736437+0000","last_active":"2024-09-11T23:21:50.211292+0000","last_peered":"2024-09-11T23:21:50.211292+0000","last_clean":"2024-09-11T23:21:50.211292+0000","last_became_active":"2024-09-11T23:19:59.736029+0000","last_became_peered":"2024-09-11T23:19:59.736029+0000","last_unstale":"2024-09-11T23:21:50.211292+0000","last_undegraded":"2024-09-11T23:21:50.211292+0000","last_fullsized":"2024-09-11T23:21:50.211292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:29.226939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300441+0000","last_change":"2024-09-11T23:19:59.736104+0000","last_active":"2024-09-11T23:21:50.300441+0000","last_peered":"2024-09-11T23:21:50.300441+0000","last_clean":"2024-09-11T23:21:50.300441+0000","last_became_active":"2024-09-11T23:19:59.735760+0000","last_became_peered":"2024-09-11T23:19:59.735760+0000","last_unstale":"2024-09-11T23:21:50.300441+0000","last_undegraded":"2024-09-11T23:21:50.300441+0000","last_fullsized":"2024-09-11T23:21:50.300441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:56.554313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308481+0000","last_change":"2024-09-11T23:19:59.735455+0000","last_active":"2024-09-11T23:21:50.308481+0000","last_peered":"2024-09-11T23:21:50.308481+0000","last_clean":"2024-09-11T23:21:50.308481+0000","last_became_active":"2024-09-11T23:19:59.735078+0000","last_became_peered":"2024-09-11T23:19:59.735078+0000","last_unstale":"2024-09-11T23:21:50.308481+0000","last_undegraded":"2024-09-11T23:21:50.308481+0000","last_fullsized":"2024-09-11T23:21:50.308481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:21:16.336006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.301909+0000","last_change":"2024-09-11T23:19:59.745997+0000","last_active":"2024-09-11T23:21:50.301909+0000","last_peered":"2024-09-11T23:21:50.301909+0000","last_clean":"2024-09-11T23:21:50.301909+0000","last_became_active":"2024-09-11T23:19:59.745715+0000","last_became_peered":"2024-09-11T23:19:59.745715+0000","last_unstale":"2024-09-11T23:21:50.301909+0000","last_undegraded":"2024-09-11T23:21:50.301909+0000","last_fullsized":"2024-09-11T23:21:50.301909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:34.776229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.212119+0000","last_change":"2024-09-11T23:19:59.734181+0000","last_active":"2024-09-11T23:21:50.212119+0000","last_peered":"2024-09-11T23:21:50.212119+0000","last_clean":"2024-09-11T23:21:50.212119+0000","last_became_active":"2024-09-11T23:19:59.733793+0000","last_became_peered":"2024-09-11T23:19:59.733793+0000","last_unstale":"2024-09-11T23:21:50.212119+0000","last_undegraded":"2024-09-11T23:21:50.212119+0000","last_fullsized":"2024-09-11T23:21:50.212119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:36.906308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598526+0000","last_change":"2024-09-11T23:19:59.738713+0000","last_active":"2024-09-11T23:21:50.598526+0000","last_peered":"2024-09-11T23:21:50.598526+0000","last_clean":"2024-09-11T23:21:50.598526+0000","last_became_active":"2024-09-11T23:19:59.737579+0000","last_became_peered":"2024-09-11T23:19:59.737579+0000","last_unstale":"2024-09-11T23:21:50.598526+0000","last_undegraded":"2024-09-11T23:21:50.598526+0000","last_fullsized":"2024-09-11T23:21:50.598526+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:47.135775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210205+0000","last_change":"2024-09-11T23:19:59.741381+0000","last_active":"2024-09-11T23:21:50.210205+0000","last_peered":"2024-09-11T23:21:50.210205+0000","last_clean":"2024-09-11T23:21:50.210205+0000","last_became_active":"2024-09-11T23:19:59.740914+0000","last_became_peered":"2024-09-11T23:19:59.740914+0000","last_unstale":"2024-09-11T23:21:50.210205+0000","last_undegraded":"2024-09-11T23:21:50.210205+0000","last_fullsized":"2024-09-11T23:21:50.210205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:24:04.365704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455133+0000","last_change":"2024-09-11T23:19:59.736840+0000","last_active":"2024-09-11T23:21:50.455133+0000","last_peered":"2024-09-11T23:21:50.455133+0000","last_clean":"2024-09-11T23:21:50.455133+0000","last_became_active":"2024-09-11T23:19:59.736554+0000","last_became_peered":"2024-09-11T23:19:59.736554+0000","last_unstale":"2024-09-11T23:21:50.455133+0000","last_undegraded":"2024-09-11T23:21:50.455133+0000","last_fullsized":"2024-09-11T23:21:50.455133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:48.508369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211738+0000","last_change":"2024-09-11T23:19:59.735719+0000","last_active":"2024-09-11T23:21:50.211738+0000","last_peered":"2024-09-11T23:21:50.211738+0000","last_clean":"2024-09-11T23:21:50.211738+0000","last_became_active":"2024-09-11T23:19:59.735397+0000","last_became_peered":"2024-09-11T23:19:59.735397+0000","last_unstale":"2024-09-11T23:21:50.211738+0000","last_undegraded":"2024-09-11T23:21:50.211738+0000","last_fullsized":"2024-09-11T23:21:50.211738+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:26:19.392730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211747+0000","last_change":"2024-09-11T23:19:59.737704+0000","last_active":"2024-09-11T23:21:50.211747+0000","last_peered":"2024-09-11T23:21:50.211747+0000","last_clean":"2024-09-11T23:21:50.211747+0000","last_became_active":"2024-09-11T23:19:59.737176+0000","last_became_peered":"2024-09-11T23:19:59.737176+0000","last_unstale":"2024-09-11T23:21:50.211747+0000","last_undegraded":"2024-09-11T23:21:50.211747+0000","last_fullsized":"2024-09-11T23:21:50.211747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:22.938171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598248+0000","last_change":"2024-09-11T23:19:59.735499+0000","last_active":"2024-09-11T23:21:50.598248+0000","last_peered":"2024-09-11T23:21:50.598248+0000","last_clean":"2024-09-11T23:21:50.598248+0000","last_became_active":"2024-09-11T23:19:59.735088+0000","last_became_peered":"2024-09-11T23:19:59.735088+0000","last_unstale":"2024-09-11T23:21:50.598248+0000","last_undegraded":"2024-09-11T23:21:50.598248+0000","last_fullsized":"2024-09-11T23:21:50.598248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:12.633104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299549+0000","last_change":"2024-09-11T23:19:59.743972+0000","last_active":"2024-09-11T23:21:50.299549+0000","last_peered":"2024-09-11T23:21:50.299549+0000","last_clean":"2024-09-11T23:21:50.299549+0000","last_became_active":"2024-09-11T23:19:59.743675+0000","last_became_peered":"2024-09-11T23:19:59.743675+0000","last_unstale":"2024-09-11T23:21:50.299549+0000","last_undegraded":"2024-09-11T23:21:50.299549+0000","last_fullsized":"2024-09-11T23:21:50.299549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:46.645696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":134,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.866819+0000","last_change":"2024-09-11T23:19:57.615472+0000","last_active":"2024-09-11T23:21:50.866819+0000","last_peered":"2024-09-11T23:21:50.866819+0000","last_clean":"2024-09-11T23:21:50.866819+0000","last_became_active":"2024-09-11T23:19:57.615104+0000","last_became_peered":"2024-09-11T23:19:57.615104+0000","last_unstale":"2024-09-11T23:21:50.866819+0000","last_undegraded":"2024-09-11T23:21:50.866819+0000","last_fullsized":"2024-09-11T23:21:50.866819+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:03.601363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308144+0000","last_change":"2024-09-11T23:19:59.741027+0000","last_active":"2024-09-11T23:21:50.308144+0000","last_peered":"2024-09-11T23:21:50.308144+0000","last_clean":"2024-09-11T23:21:50.308144+0000","last_became_active":"2024-09-11T23:19:59.740681+0000","last_became_peered":"2024-09-11T23:19:59.740681+0000","last_unstale":"2024-09-11T23:21:50.308144+0000","last_undegraded":"2024-09-11T23:21:50.308144+0000","last_fullsized":"2024-09-11T23:21:50.308144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:36.948147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311025+0000","last_change":"2024-09-11T23:19:57.612730+0000","last_active":"2024-09-11T23:21:50.311025+0000","last_peered":"2024-09-11T23:21:50.311025+0000","last_clean":"2024-09-11T23:21:50.311025+0000","last_became_active":"2024-09-11T23:19:57.612369+0000","last_became_peered":"2024-09-11T23:19:57.612369+0000","last_unstale":"2024-09-11T23:21:50.311025+0000","last_undegraded":"2024-09-11T23:21:50.311025+0000","last_fullsized":"2024-09-11T23:21:50.311025+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:17.823616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.209881+0000","last_change":"2024-09-11T23:19:57.611470+0000","last_active":"2024-09-11T23:21:50.209881+0000","last_peered":"2024-09-11T23:21:50.209881+0000","last_clean":"2024-09-11T23:21:50.209881+0000","last_became_active":"2024-09-11T23:19:57.611141+0000","last_became_peered":"2024-09-11T23:19:57.611141+0000","last_unstale":"2024-09-11T23:21:50.209881+0000","last_undegraded":"2024-09-11T23:21:50.209881+0000","last_fullsized":"2024-09-11T23:21:50.209881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:43.173955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300408+0000","last_change":"2024-09-11T23:19:59.732375+0000","last_active":"2024-09-11T23:21:50.300408+0000","last_peered":"2024-09-11T23:21:50.300408+0000","last_clean":"2024-09-11T23:21:50.300408+0000","last_became_active":"2024-09-11T23:19:59.731793+0000","last_became_peered":"2024-09-11T23:19:59.731793+0000","last_unstale":"2024-09-11T23:21:50.300408+0000","last_undegraded":"2024-09-11T23:21:50.300408+0000","last_fullsized":"2024-09-11T23:21:50.300408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:02:47.278646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308399+0000","last_change":"2024-09-11T23:19:57.614062+0000","last_active":"2024-09-11T23:21:50.308399+0000","last_peered":"2024-09-11T23:21:50.308399+0000","last_clean":"2024-09-11T23:21:50.308399+0000","last_became_active":"2024-09-11T23:19:57.613749+0000","last_became_peered":"2024-09-11T23:19:57.613749+0000","last_unstale":"2024-09-11T23:21:50.308399+0000","last_undegraded":"2024-09-11T23:21:50.308399+0000","last_fullsized":"2024-09-11T23:21:50.308399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:45:22.299379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297086+0000","last_change":"2024-09-11T23:19:59.730758+0000","last_active":"2024-09-11T23:21:50.297086+0000","last_peered":"2024-09-11T23:21:50.297086+0000","last_clean":"2024-09-11T23:21:50.297086+0000","last_became_active":"2024-09-11T23:19:59.730423+0000","last_became_peered":"2024-09-11T23:19:59.730423+0000","last_unstale":"2024-09-11T23:21:50.297086+0000","last_undegraded":"2024-09-11T23:21:50.297086+0000","last_fullsized":"2024-09-11T23:21:50.297086+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:30.679451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":129,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.459375+0000","last_change":"2024-09-11T23:19:57.615574+0000","last_active":"2024-09-11T23:21:50.459375+0000","last_peered":"2024-09-11T23:21:50.459375+0000","last_clean":"2024-09-11T23:21:50.459375+0000","last_became_active":"2024-09-11T23:19:57.615192+0000","last_became_peered":"2024-09-11T23:19:57.615192+0000","last_unstale":"2024-09-11T23:21:50.459375+0000","last_undegraded":"2024-09-11T23:21:50.459375+0000","last_fullsized":"2024-09-11T23:21:50.459375+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:29.186899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211749+0000","last_change":"2024-09-11T23:19:59.738433+0000","last_active":"2024-09-11T23:21:50.211749+0000","last_peered":"2024-09-11T23:21:50.211749+0000","last_clean":"2024-09-11T23:21:50.211749+0000","last_became_active":"2024-09-11T23:19:59.738081+0000","last_became_peered":"2024-09-11T23:19:59.738081+0000","last_unstale":"2024-09-11T23:21:50.211749+0000","last_undegraded":"2024-09-11T23:21:50.211749+0000","last_fullsized":"2024-09-11T23:21:50.211749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:10:46.786454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455225+0000","last_change":"2024-09-11T23:21:02.437498+0000","last_active":"2024-09-11T23:21:50.455225+0000","last_peered":"2024-09-11T23:21:50.455225+0000","last_clean":"2024-09-11T23:21:50.455225+0000","last_became_active":"2024-09-11T23:21:02.434895+0000","last_became_peered":"2024-09-11T23:21:02.434895+0000","last_unstale":"2024-09-11T23:21:50.455225+0000","last_undegraded":"2024-09-11T23:21:50.455225+0000","last_fullsized":"2024-09-11T23:21:50.455225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:33.142041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455584+0000","last_change":"2024-09-11T23:19:59.741081+0000","last_active":"2024-09-11T23:21:50.455584+0000","last_peered":"2024-09-11T23:21:50.455584+0000","last_clean":"2024-09-11T23:21:50.455584+0000","last_became_active":"2024-09-11T23:19:59.739984+0000","last_became_peered":"2024-09-11T23:19:59.739984+0000","last_unstale":"2024-09-11T23:21:50.455584+0000","last_undegraded":"2024-09-11T23:21:50.455584+0000","last_fullsized":"2024-09-11T23:21:50.455584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:03.513121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308186+0000","last_change":"2024-09-11T23:19:57.823272+0000","last_active":"2024-09-11T23:21:50.308186+0000","last_peered":"2024-09-11T23:21:50.308186+0000","last_clean":"2024-09-11T23:21:50.308186+0000","last_became_active":"2024-09-11T23:19:57.822957+0000","last_became_peered":"2024-09-11T23:19:57.822957+0000","last_unstale":"2024-09-11T23:21:50.308186+0000","last_undegraded":"2024-09-11T23:21:50.308186+0000","last_fullsized":"2024-09-11T23:21:50.308186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:06.779385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211510+0000","last_change":"2024-09-11T23:19:59.738398+0000","last_active":"2024-09-11T23:21:50.211510+0000","last_peered":"2024-09-11T23:21:50.211510+0000","last_clean":"2024-09-11T23:21:50.211510+0000","last_became_active":"2024-09-11T23:19:59.738042+0000","last_became_peered":"2024-09-11T23:19:59.738042+0000","last_unstale":"2024-09-11T23:21:50.211510+0000","last_undegraded":"2024-09-11T23:21:50.211510+0000","last_fullsized":"2024-09-11T23:21:50.211510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:05.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'34","reported_seq":271,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.867174+0000","last_change":"2024-09-11T23:19:57.615036+0000","last_active":"2024-09-11T23:21:50.867174+0000","last_peered":"2024-09-11T23:21:50.867174+0000","last_clean":"2024-09-11T23:21:50.867174+0000","last_became_active":"2024-09-11T23:19:57.614641+0000","last_became_peered":"2024-09-11T23:19:57.614641+0000","last_unstale":"2024-09-11T23:21:50.867174+0000","last_undegraded":"2024-09-11T23:21:50.867174+0000","last_fullsized":"2024-09-11T23:21:50.867174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:05.430508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67065,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":213,"num_read_kb":172,"num_write":35,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299857+0000","last_change":"2024-09-11T23:19:59.741597+0000","last_active":"2024-09-11T23:21:50.299857+0000","last_peered":"2024-09-11T23:21:50.299857+0000","last_clean":"2024-09-11T23:21:50.299857+0000","last_became_active":"2024-09-11T23:19:59.741296+0000","last_became_peered":"2024-09-11T23:19:59.741296+0000","last_unstale":"2024-09-11T23:21:50.299857+0000","last_undegraded":"2024-09-11T23:21:50.299857+0000","last_fullsized":"2024-09-11T23:21:50.299857+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:06:20.742916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211837+0000","last_change":"2024-09-11T23:19:57.617550+0000","last_active":"2024-09-11T23:21:50.211837+0000","last_peered":"2024-09-11T23:21:50.211837+0000","last_clean":"2024-09-11T23:21:50.211837+0000","last_became_active":"2024-09-11T23:19:57.617211+0000","last_became_peered":"2024-09-11T23:19:57.617211+0000","last_unstale":"2024-09-11T23:21:50.211837+0000","last_undegraded":"2024-09-11T23:21:50.211837+0000","last_fullsized":"2024-09-11T23:21:50.211837+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:19.707742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210301+0000","last_change":"2024-09-11T23:19:59.742070+0000","last_active":"2024-09-11T23:21:50.210301+0000","last_peered":"2024-09-11T23:21:50.210301+0000","last_clean":"2024-09-11T23:21:50.210301+0000","last_became_active":"2024-09-11T23:19:59.739367+0000","last_became_peered":"2024-09-11T23:19:59.739367+0000","last_unstale":"2024-09-11T23:21:50.210301+0000","last_undegraded":"2024-09-11T23:21:50.210301+0000","last_fullsized":"2024-09-11T23:21:50.210301+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:27.850136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308168+0000","last_change":"2024-09-11T23:19:57.618054+0000","last_active":"2024-09-11T23:21:50.308168+0000","last_peered":"2024-09-11T23:21:50.308168+0000","last_clean":"2024-09-11T23:21:50.308168+0000","last_became_active":"2024-09-11T23:19:57.617702+0000","last_became_peered":"2024-09-11T23:19:57.617702+0000","last_unstale":"2024-09-11T23:21:50.308168+0000","last_undegraded":"2024-09-11T23:21:50.308168+0000","last_fullsized":"2024-09-11T23:21:50.308168+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:40.171851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455548+0000","last_change":"2024-09-11T23:19:59.744000+0000","last_active":"2024-09-11T23:21:50.455548+0000","last_peered":"2024-09-11T23:21:50.455548+0000","last_clean":"2024-09-11T23:21:50.455548+0000","last_became_active":"2024-09-11T23:19:59.743629+0000","last_became_peered":"2024-09-11T23:19:59.743629+0000","last_unstale":"2024-09-11T23:21:50.455548+0000","last_undegraded":"2024-09-11T23:21:50.455548+0000","last_fullsized":"2024-09-11T23:21:50.455548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:20:04.133258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'7","reported_seq":137,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.867227+0000","last_change":"2024-09-11T23:19:57.615117+0000","last_active":"2024-09-11T23:21:50.867227+0000","last_peered":"2024-09-11T23:21:50.867227+0000","last_clean":"2024-09-11T23:21:50.867227+0000","last_became_active":"2024-09-11T23:19:57.614758+0000","last_became_peered":"2024-09-11T23:19:57.614758+0000","last_unstale":"2024-09-11T23:21:50.867227+0000","last_undegraded":"2024-09-11T23:21:50.867227+0000","last_fullsized":"2024-09-11T23:21:50.867227+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:58.113986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":6,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455281+0000","last_change":"2024-09-11T23:19:59.739403+0000","last_active":"2024-09-11T23:21:50.455281+0000","last_peered":"2024-09-11T23:21:50.455281+0000","last_clean":"2024-09-11T23:21:50.455281+0000","last_became_active":"2024-09-11T23:19:59.739078+0000","last_became_peered":"2024-09-11T23:19:59.739078+0000","last_unstale":"2024-09-11T23:21:50.455281+0000","last_undegraded":"2024-09-11T23:21:50.455281+0000","last_fullsized":"2024-09-11T23:21:50.455281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:46.168720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299488+0000","last_change":"2024-09-11T23:19:57.823323+0000","last_active":"2024-09-11T23:21:50.299488+0000","last_peered":"2024-09-11T23:21:50.299488+0000","last_clean":"2024-09-11T23:21:50.299488+0000","last_became_active":"2024-09-11T23:19:57.822947+0000","last_became_peered":"2024-09-11T23:19:57.822947+0000","last_unstale":"2024-09-11T23:21:50.299488+0000","last_undegraded":"2024-09-11T23:21:50.299488+0000","last_fullsized":"2024-09-11T23:21:50.299488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:12.542911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308091+0000","last_change":"2024-09-11T23:19:59.741163+0000","last_active":"2024-09-11T23:21:50.308091+0000","last_peered":"2024-09-11T23:21:50.308091+0000","last_clean":"2024-09-11T23:21:50.308091+0000","last_became_active":"2024-09-11T23:19:59.740633+0000","last_became_peered":"2024-09-11T23:19:59.740633+0000","last_unstale":"2024-09-11T23:21:50.308091+0000","last_undegraded":"2024-09-11T23:21:50.308091+0000","last_fullsized":"2024-09-11T23:21:50.308091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:26.301775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299941+0000","last_change":"2024-09-11T23:19:57.615804+0000","last_active":"2024-09-11T23:21:50.299941+0000","last_peered":"2024-09-11T23:21:50.299941+0000","last_clean":"2024-09-11T23:21:50.299941+0000","last_became_active":"2024-09-11T23:19:57.615443+0000","last_became_peered":"2024-09-11T23:19:57.615443+0000","last_unstale":"2024-09-11T23:21:50.299941+0000","last_undegraded":"2024-09-11T23:21:50.299941+0000","last_fullsized":"2024-09-11T23:21:50.299941+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:39.296873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598177+0000","last_change":"2024-09-11T23:21:02.360875+0000","last_active":"2024-09-11T23:21:50.598177+0000","last_peered":"2024-09-11T23:21:50.598177+0000","last_clean":"2024-09-11T23:21:50.598177+0000","last_became_active":"2024-09-11T23:21:02.360480+0000","last_became_peered":"2024-09-11T23:21:02.360480+0000","last_unstale":"2024-09-11T23:21:50.598177+0000","last_undegraded":"2024-09-11T23:21:50.598177+0000","last_fullsized":"2024-09-11T23:21:50.598177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:04.141587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210448+0000","last_change":"2024-09-11T23:19:57.612873+0000","last_active":"2024-09-11T23:21:50.210448+0000","last_peered":"2024-09-11T23:21:50.210448+0000","last_clean":"2024-09-11T23:21:50.210448+0000","last_became_active":"2024-09-11T23:19:57.612566+0000","last_became_peered":"2024-09-11T23:19:57.612566+0000","last_unstale":"2024-09-11T23:21:50.210448+0000","last_undegraded":"2024-09-11T23:21:50.210448+0000","last_fullsized":"2024-09-11T23:21:50.210448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:46.218212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455770+0000","last_change":"2024-09-11T23:19:59.736352+0000","last_active":"2024-09-11T23:21:50.455770+0000","last_peered":"2024-09-11T23:21:50.455770+0000","last_clean":"2024-09-11T23:21:50.455770+0000","last_became_active":"2024-09-11T23:19:59.735402+0000","last_became_peered":"2024-09-11T23:19:59.735402+0000","last_unstale":"2024-09-11T23:21:50.455770+0000","last_undegraded":"2024-09-11T23:21:50.455770+0000","last_fullsized":"2024-09-11T23:21:50.455770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:41.854526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'39","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.412363+0000","last_change":"2024-09-11T23:19:57.612439+0000","last_active":"2024-09-11T23:21:50.412363+0000","last_peered":"2024-09-11T23:21:50.412363+0000","last_clean":"2024-09-11T23:21:50.412363+0000","last_became_active":"2024-09-11T23:19:57.612030+0000","last_became_peered":"2024-09-11T23:19:57.612030+0000","last_unstale":"2024-09-11T23:21:50.412363+0000","last_undegraded":"2024-09-11T23:21:50.412363+0000","last_fullsized":"2024-09-11T23:21:50.412363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:36:24.579246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455480+0000","last_change":"2024-09-11T23:19:59.741241+0000","last_active":"2024-09-11T23:21:50.455480+0000","last_peered":"2024-09-11T23:21:50.455480+0000","last_clean":"2024-09-11T23:21:50.455480+0000","last_became_active":"2024-09-11T23:19:59.740265+0000","last_became_peered":"2024-09-11T23:19:59.740265+0000","last_unstale":"2024-09-11T23:21:50.455480+0000","last_undegraded":"2024-09-11T23:21:50.455480+0000","last_fullsized":"2024-09-11T23:21:50.455480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:00.676762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211975+0000","last_change":"2024-09-11T23:19:57.825238+0000","last_active":"2024-09-11T23:21:50.211975+0000","last_peered":"2024-09-11T23:21:50.211975+0000","last_clean":"2024-09-11T23:21:50.211975+0000","last_became_active":"2024-09-11T23:19:57.824732+0000","last_became_peered":"2024-09-11T23:19:57.824732+0000","last_unstale":"2024-09-11T23:21:50.211975+0000","last_undegraded":"2024-09-11T23:21:50.211975+0000","last_fullsized":"2024-09-11T23:21:50.211975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:24.465863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455634+0000","last_change":"2024-09-11T23:19:59.741338+0000","last_active":"2024-09-11T23:21:50.455634+0000","last_peered":"2024-09-11T23:21:50.455634+0000","last_clean":"2024-09-11T23:21:50.455634+0000","last_became_active":"2024-09-11T23:19:59.741019+0000","last_became_peered":"2024-09-11T23:19:59.741019+0000","last_unstale":"2024-09-11T23:21:50.455634+0000","last_undegraded":"2024-09-11T23:21:50.455634+0000","last_fullsized":"2024-09-11T23:21:50.455634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:36:03.480006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":351,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455427+0000","last_change":"2024-09-11T23:18:45.606646+0000","last_active":"2024-09-11T23:21:50.455427+0000","last_peered":"2024-09-11T23:21:50.455427+0000","last_clean":"2024-09-11T23:21:50.455427+0000","last_became_active":"2024-09-11T23:18:45.606065+0000","last_became_peered":"2024-09-11T23:18:45.606065+0000","last_unstale":"2024-09-11T23:21:50.455427+0000","last_undegraded":"2024-09-11T23:21:50.455427+0000","last_fullsized":"2024-09-11T23:21:50.455427+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:15:17.148562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:15:17.148562+0000","last_clean_scrub_stamp":"2024-09-11T23:15:17.148562+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:07.992971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.453802+0000","last_change":"2024-09-11T23:19:57.612584+0000","last_active":"2024-09-11T23:21:50.453802+0000","last_peered":"2024-09-11T23:21:50.453802+0000","last_clean":"2024-09-11T23:21:50.453802+0000","last_became_active":"2024-09-11T23:19:57.612211+0000","last_became_peered":"2024-09-11T23:19:57.612211+0000","last_unstale":"2024-09-11T23:21:50.453802+0000","last_undegraded":"2024-09-11T23:21:50.453802+0000","last_fullsized":"2024-09-11T23:21:50.453802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:04.026233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300938+0000","last_change":"2024-09-11T23:21:02.359249+0000","last_active":"2024-09-11T23:21:50.300938+0000","last_peered":"2024-09-11T23:21:50.300938+0000","last_clean":"2024-09-11T23:21:50.300938+0000","last_became_active":"2024-09-11T23:21:02.358906+0000","last_became_peered":"2024-09-11T23:21:02.358906+0000","last_unstale":"2024-09-11T23:21:50.300938+0000","last_undegraded":"2024-09-11T23:21:50.300938+0000","last_fullsized":"2024-09-11T23:21:50.300938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:05.973581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455432+0000","last_change":"2024-09-11T23:19:57.618283+0000","last_active":"2024-09-11T23:21:50.455432+0000","last_peered":"2024-09-11T23:21:50.455432+0000","last_clean":"2024-09-11T23:21:50.455432+0000","last_became_active":"2024-09-11T23:19:57.617942+0000","last_became_peered":"2024-09-11T23:19:57.617942+0000","last_unstale":"2024-09-11T23:21:50.455432+0000","last_undegraded":"2024-09-11T23:21:50.455432+0000","last_fullsized":"2024-09-11T23:21:50.455432+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:30.832602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.209864+0000","last_change":"2024-09-11T23:19:59.740730+0000","last_active":"2024-09-11T23:21:50.209864+0000","last_peered":"2024-09-11T23:21:50.209864+0000","last_clean":"2024-09-11T23:21:50.209864+0000","last_became_active":"2024-09-11T23:19:59.740369+0000","last_became_peered":"2024-09-11T23:19:59.740369+0000","last_unstale":"2024-09-11T23:21:50.209864+0000","last_undegraded":"2024-09-11T23:21:50.209864+0000","last_fullsized":"2024-09-11T23:21:50.209864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:50.264644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":131,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.454901+0000","last_change":"2024-09-11T23:19:57.823253+0000","last_active":"2024-09-11T23:21:50.454901+0000","last_peered":"2024-09-11T23:21:50.454901+0000","last_clean":"2024-09-11T23:21:50.454901+0000","last_became_active":"2024-09-11T23:19:57.822895+0000","last_became_peered":"2024-09-11T23:19:57.822895+0000","last_unstale":"2024-09-11T23:21:50.454901+0000","last_undegraded":"2024-09-11T23:21:50.454901+0000","last_fullsized":"2024-09-11T23:21:50.454901+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:22.998371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308142+0000","last_change":"2024-09-11T23:19:59.742248+0000","last_active":"2024-09-11T23:21:50.308142+0000","last_peered":"2024-09-11T23:21:50.308142+0000","last_clean":"2024-09-11T23:21:50.308142+0000","last_became_active":"2024-09-11T23:19:59.741810+0000","last_became_peered":"2024-09-11T23:19:59.741810+0000","last_unstale":"2024-09-11T23:21:50.308142+0000","last_undegraded":"2024-09-11T23:21:50.308142+0000","last_fullsized":"2024-09-11T23:21:50.308142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:19:52.720286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455504+0000","last_change":"2024-09-11T23:19:57.616244+0000","last_active":"2024-09-11T23:21:50.455504+0000","last_peered":"2024-09-11T23:21:50.455504+0000","last_clean":"2024-09-11T23:21:50.455504+0000","last_became_active":"2024-09-11T23:19:57.615868+0000","last_became_peered":"2024-09-11T23:19:57.615868+0000","last_unstale":"2024-09-11T23:21:50.455504+0000","last_undegraded":"2024-09-11T23:21:50.455504+0000","last_fullsized":"2024-09-11T23:21:50.455504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:42.510896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311069+0000","last_change":"2024-09-11T23:21:02.434265+0000","last_active":"2024-09-11T23:21:50.311069+0000","last_peered":"2024-09-11T23:21:50.311069+0000","last_clean":"2024-09-11T23:21:50.311069+0000","last_became_active":"2024-09-11T23:21:02.433840+0000","last_became_peered":"2024-09-11T23:21:02.433840+0000","last_unstale":"2024-09-11T23:21:50.311069+0000","last_undegraded":"2024-09-11T23:21:50.311069+0000","last_fullsized":"2024-09-11T23:21:50.311069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:04.619695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297618+0000","last_change":"2024-09-11T23:19:57.824334+0000","last_active":"2024-09-11T23:21:50.297618+0000","last_peered":"2024-09-11T23:21:50.297618+0000","last_clean":"2024-09-11T23:21:50.297618+0000","last_became_active":"2024-09-11T23:19:57.823972+0000","last_became_peered":"2024-09-11T23:19:57.823972+0000","last_unstale":"2024-09-11T23:21:50.297618+0000","last_undegraded":"2024-09-11T23:21:50.297618+0000","last_fullsized":"2024-09-11T23:21:50.297618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:39.183281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311164+0000","last_change":"2024-09-11T23:19:59.728064+0000","last_active":"2024-09-11T23:21:50.311164+0000","last_peered":"2024-09-11T23:21:50.311164+0000","last_clean":"2024-09-11T23:21:50.311164+0000","last_became_active":"2024-09-11T23:19:59.727732+0000","last_became_peered":"2024-09-11T23:19:59.727732+0000","last_unstale":"2024-09-11T23:21:50.311164+0000","last_undegraded":"2024-09-11T23:21:50.311164+0000","last_fullsized":"2024-09-11T23:21:50.311164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:15.383974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299855+0000","last_change":"2024-09-11T23:19:57.824362+0000","last_active":"2024-09-11T23:21:50.299855+0000","last_peered":"2024-09-11T23:21:50.299855+0000","last_clean":"2024-09-11T23:21:50.299855+0000","last_became_active":"2024-09-11T23:19:57.823753+0000","last_became_peered":"2024-09-11T23:19:57.823753+0000","last_unstale":"2024-09-11T23:21:50.299855+0000","last_undegraded":"2024-09-11T23:21:50.299855+0000","last_fullsized":"2024-09-11T23:21:50.299855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:56.357268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210323+0000","last_change":"2024-09-11T23:19:59.730623+0000","last_active":"2024-09-11T23:21:50.210323+0000","last_peered":"2024-09-11T23:21:50.210323+0000","last_clean":"2024-09-11T23:21:50.210323+0000","last_became_active":"2024-09-11T23:19:59.730307+0000","last_became_peered":"2024-09-11T23:19:59.730307+0000","last_unstale":"2024-09-11T23:21:50.210323+0000","last_undegraded":"2024-09-11T23:21:50.210323+0000","last_fullsized":"2024-09-11T23:21:50.210323+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:52.874884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'48","reported_seq":95,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.418264+0000","last_change":"2024-09-11T23:19:57.611730+0000","last_active":"2024-09-11T23:21:50.418264+0000","last_peered":"2024-09-11T23:21:50.418264+0000","last_clean":"2024-09-11T23:21:50.418264+0000","last_became_active":"2024-09-11T23:19:57.611394+0000","last_became_peered":"2024-09-11T23:19:57.611394+0000","last_unstale":"2024-09-11T23:21:50.418264+0000","last_undegraded":"2024-09-11T23:21:50.418264+0000","last_fullsized":"2024-09-11T23:21:50.418264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:15.518573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.302035+0000","last_change":"2024-09-11T23:19:59.741396+0000","last_active":"2024-09-11T23:21:50.302035+0000","last_peered":"2024-09-11T23:21:50.302035+0000","last_clean":"2024-09-11T23:21:50.302035+0000","last_became_active":"2024-09-11T23:19:59.741102+0000","last_became_peered":"2024-09-11T23:19:59.741102+0000","last_unstale":"2024-09-11T23:21:50.302035+0000","last_undegraded":"2024-09-11T23:21:50.302035+0000","last_fullsized":"2024-09-11T23:21:50.302035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:08.759512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.453918+0000","last_change":"2024-09-11T23:19:57.824812+0000","last_active":"2024-09-11T23:21:50.453918+0000","last_peered":"2024-09-11T23:21:50.453918+0000","last_clean":"2024-09-11T23:21:50.453918+0000","last_became_active":"2024-09-11T23:19:57.824454+0000","last_became_peered":"2024-09-11T23:19:57.824454+0000","last_unstale":"2024-09-11T23:21:50.453918+0000","last_undegraded":"2024-09-11T23:21:50.453918+0000","last_fullsized":"2024-09-11T23:21:50.453918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:25.275573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455034+0000","last_change":"2024-09-11T23:19:59.735857+0000","last_active":"2024-09-11T23:21:50.455034+0000","last_peered":"2024-09-11T23:21:50.455034+0000","last_clean":"2024-09-11T23:21:50.455034+0000","last_became_active":"2024-09-11T23:19:59.735541+0000","last_became_peered":"2024-09-11T23:19:59.735541+0000","last_unstale":"2024-09-11T23:21:50.455034+0000","last_undegraded":"2024-09-11T23:21:50.455034+0000","last_fullsized":"2024-09-11T23:21:50.455034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:21.600490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211231+0000","last_change":"2024-09-11T23:19:57.823468+0000","last_active":"2024-09-11T23:21:50.211231+0000","last_peered":"2024-09-11T23:21:50.211231+0000","last_clean":"2024-09-11T23:21:50.211231+0000","last_became_active":"2024-09-11T23:19:57.823154+0000","last_became_peered":"2024-09-11T23:19:57.823154+0000","last_unstale":"2024-09-11T23:21:50.211231+0000","last_undegraded":"2024-09-11T23:21:50.211231+0000","last_fullsized":"2024-09-11T23:21:50.211231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:28.770627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299509+0000","last_change":"2024-09-11T23:19:59.741263+0000","last_active":"2024-09-11T23:21:50.299509+0000","last_peered":"2024-09-11T23:21:50.299509+0000","last_clean":"2024-09-11T23:21:50.299509+0000","last_became_active":"2024-09-11T23:19:59.740938+0000","last_became_peered":"2024-09-11T23:19:59.740938+0000","last_unstale":"2024-09-11T23:21:50.299509+0000","last_undegraded":"2024-09-11T23:21:50.299509+0000","last_fullsized":"2024-09-11T23:21:50.299509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:54.768652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.301723+0000","last_change":"2024-09-11T23:19:57.612517+0000","last_active":"2024-09-11T23:21:50.301723+0000","last_peered":"2024-09-11T23:21:50.301723+0000","last_clean":"2024-09-11T23:21:50.301723+0000","last_became_active":"2024-09-11T23:19:57.612188+0000","last_became_peered":"2024-09-11T23:19:57.612188+0000","last_unstale":"2024-09-11T23:21:50.301723+0000","last_undegraded":"2024-09-11T23:21:50.301723+0000","last_fullsized":"2024-09-11T23:21:50.301723+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:52.896413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598315+0000","last_change":"2024-09-11T23:19:59.732520+0000","last_active":"2024-09-11T23:21:50.598315+0000","last_peered":"2024-09-11T23:21:50.598315+0000","last_clean":"2024-09-11T23:21:50.598315+0000","last_became_active":"2024-09-11T23:19:59.732173+0000","last_became_peered":"2024-09-11T23:19:59.732173+0000","last_unstale":"2024-09-11T23:21:50.598315+0000","last_undegraded":"2024-09-11T23:21:50.598315+0000","last_fullsized":"2024-09-11T23:21:50.598315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:04.626312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598177+0000","last_change":"2024-09-11T23:21:02.365533+0000","last_active":"2024-09-11T23:21:50.598177+0000","last_peered":"2024-09-11T23:21:50.598177+0000","last_clean":"2024-09-11T23:21:50.598177+0000","last_became_active":"2024-09-11T23:21:02.359345+0000","last_became_peered":"2024-09-11T23:21:02.359345+0000","last_unstale":"2024-09-11T23:21:50.598177+0000","last_undegraded":"2024-09-11T23:21:50.598177+0000","last_fullsized":"2024-09-11T23:21:50.598177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:27.195666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.301266+0000","last_change":"2024-09-11T23:19:59.734667+0000","last_active":"2024-09-11T23:21:50.301266+0000","last_peered":"2024-09-11T23:21:50.301266+0000","last_clean":"2024-09-11T23:21:50.301266+0000","last_became_active":"2024-09-11T23:19:59.733754+0000","last_became_peered":"2024-09-11T23:19:59.733754+0000","last_unstale":"2024-09-11T23:21:50.301266+0000","last_undegraded":"2024-09-11T23:21:50.301266+0000","last_fullsized":"2024-09-11T23:21:50.301266+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:57.392272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299623+0000","last_change":"2024-09-11T23:19:57.614579+0000","last_active":"2024-09-11T23:21:50.299623+0000","last_peered":"2024-09-11T23:21:50.299623+0000","last_clean":"2024-09-11T23:21:50.299623+0000","last_became_active":"2024-09-11T23:19:57.614227+0000","last_became_peered":"2024-09-11T23:19:57.614227+0000","last_unstale":"2024-09-11T23:21:50.299623+0000","last_undegraded":"2024-09-11T23:21:50.299623+0000","last_fullsized":"2024-09-11T23:21:50.299623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:47.952920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455414+0000","last_change":"2024-09-11T23:19:59.744219+0000","last_active":"2024-09-11T23:21:50.455414+0000","last_peered":"2024-09-11T23:21:50.455414+0000","last_clean":"2024-09-11T23:21:50.455414+0000","last_became_active":"2024-09-11T23:19:59.743914+0000","last_became_peered":"2024-09-11T23:19:59.743914+0000","last_unstale":"2024-09-11T23:21:50.455414+0000","last_undegraded":"2024-09-11T23:21:50.455414+0000","last_fullsized":"2024-09-11T23:21:50.455414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:52.149588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308372+0000","last_change":"2024-09-11T23:21:02.361647+0000","last_active":"2024-09-11T23:21:50.308372+0000","last_peered":"2024-09-11T23:21:50.308372+0000","last_clean":"2024-09-11T23:21:50.308372+0000","last_became_active":"2024-09-11T23:21:02.361319+0000","last_became_peered":"2024-09-11T23:21:02.361319+0000","last_unstale":"2024-09-11T23:21:50.308372+0000","last_undegraded":"2024-09-11T23:21:50.308372+0000","last_fullsized":"2024-09-11T23:21:50.308372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:45.125525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299796+0000","last_change":"2024-09-11T23:19:57.610389+0000","last_active":"2024-09-11T23:21:50.299796+0000","last_peered":"2024-09-11T23:21:50.299796+0000","last_clean":"2024-09-11T23:21:50.299796+0000","last_became_active":"2024-09-11T23:19:57.609911+0000","last_became_peered":"2024-09-11T23:19:57.609911+0000","last_unstale":"2024-09-11T23:21:50.299796+0000","last_undegraded":"2024-09-11T23:21:50.299796+0000","last_fullsized":"2024-09-11T23:21:50.299796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:23.949929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211496+0000","last_change":"2024-09-11T23:19:59.736867+0000","last_active":"2024-09-11T23:21:50.211496+0000","last_peered":"2024-09-11T23:21:50.211496+0000","last_clean":"2024-09-11T23:21:50.211496+0000","last_became_active":"2024-09-11T23:19:59.736341+0000","last_became_peered":"2024-09-11T23:19:59.736341+0000","last_unstale":"2024-09-11T23:21:50.211496+0000","last_undegraded":"2024-09-11T23:21:50.211496+0000","last_fullsized":"2024-09-11T23:21:50.211496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:57:51.015235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598177+0000","last_change":"2024-09-11T23:21:02.368959+0000","last_active":"2024-09-11T23:21:50.598177+0000","last_peered":"2024-09-11T23:21:50.598177+0000","last_clean":"2024-09-11T23:21:50.598177+0000","last_became_active":"2024-09-11T23:21:02.363263+0000","last_became_peered":"2024-09-11T23:21:02.363263+0000","last_unstale":"2024-09-11T23:21:50.598177+0000","last_undegraded":"2024-09-11T23:21:50.598177+0000","last_fullsized":"2024-09-11T23:21:50.598177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:29.827330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455355+0000","last_change":"2024-09-11T23:19:59.742049+0000","last_active":"2024-09-11T23:21:50.455355+0000","last_peered":"2024-09-11T23:21:50.455355+0000","last_clean":"2024-09-11T23:21:50.455355+0000","last_became_active":"2024-09-11T23:19:59.741763+0000","last_became_peered":"2024-09-11T23:19:59.741763+0000","last_unstale":"2024-09-11T23:21:50.455355+0000","last_undegraded":"2024-09-11T23:21:50.455355+0000","last_fullsized":"2024-09-11T23:21:50.455355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:48.698896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.302105+0000","last_change":"2024-09-11T23:19:57.822949+0000","last_active":"2024-09-11T23:21:50.302105+0000","last_peered":"2024-09-11T23:21:50.302105+0000","last_clean":"2024-09-11T23:21:50.302105+0000","last_became_active":"2024-09-11T23:19:57.822564+0000","last_became_peered":"2024-09-11T23:19:57.822564+0000","last_unstale":"2024-09-11T23:21:50.302105+0000","last_undegraded":"2024-09-11T23:21:50.302105+0000","last_fullsized":"2024-09-11T23:21:50.302105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:29.532109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297740+0000","last_change":"2024-09-11T23:21:02.363639+0000","last_active":"2024-09-11T23:21:50.297740+0000","last_peered":"2024-09-11T23:21:50.297740+0000","last_clean":"2024-09-11T23:21:50.297740+0000","last_became_active":"2024-09-11T23:21:02.363247+0000","last_became_peered":"2024-09-11T23:21:02.363247+0000","last_unstale":"2024-09-11T23:21:50.297740+0000","last_undegraded":"2024-09-11T23:21:50.297740+0000","last_fullsized":"2024-09-11T23:21:50.297740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:18:58.906939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297845+0000","last_change":"2024-09-11T23:19:57.613323+0000","last_active":"2024-09-11T23:21:50.297845+0000","last_peered":"2024-09-11T23:21:50.297845+0000","last_clean":"2024-09-11T23:21:50.297845+0000","last_became_active":"2024-09-11T23:19:57.612985+0000","last_became_peered":"2024-09-11T23:19:57.612985+0000","last_unstale":"2024-09-11T23:21:50.297845+0000","last_undegraded":"2024-09-11T23:21:50.297845+0000","last_fullsized":"2024-09-11T23:21:50.297845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:13:09.179547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597204,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":731,"num_read_kb":475,"num_write":291,"num_write_kb":1437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1791612,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108948,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514531,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":441815,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743463,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":1242266,"data_compressed":51171,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906990,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":962560,"data_stored":844252,"data_compressed":27845,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168627,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":844172,"data_compressed":27808,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397560,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":987136,"data_stored":1044208,"data_compressed":45015,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561085,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":450425,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790018,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":1174172,"data_compressed":51171,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":452182,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149772,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":975552,"data_compressed":34001,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":519026,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":995328,"data_stored":517283,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":442711,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133469,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":201563,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:21:54.841 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T23:21:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.979+0000 7ff07b8dd640 1 -- 172.21.15.43:0/975598621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0741080c0 msgr2=0x7ff07410ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.979+0000 7ff07b8dd640 1 --2- 172.21.15.43:0/975598621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0741080c0 0x7ff07410ea10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009cd0 tx=0x7ff06c02a970 comp rx=0 tx=0).stop 2024-09-11T23:21:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.979+0000 7ff07b8dd640 1 -- 172.21.15.43:0/975598621 shutdown_connections 2024-09-11T23:21:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.979+0000 7ff07b8dd640 1 --2- 172.21.15.43:0/975598621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0741080c0 0x7ff07410ea10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.979+0000 7ff07b8dd640 1 --2- 172.21.15.43:0/975598621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff074101120 0x7ff074107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.979+0000 7ff07b8dd640 1 --2- 172.21.15.43:0/975598621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff074100800 0x7ff074100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.979+0000 7ff07b8dd640 1 -- 172.21.15.43:0/975598621 >> 172.21.15.43:0/975598621 conn(0x7ff074075e80 msgr2=0x7ff074076290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.979+0000 7ff07b8dd640 1 -- 172.21.15.43:0/975598621 shutdown_connections 2024-09-11T23:21:54.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.979+0000 7ff07b8dd640 1 -- 172.21.15.43:0/975598621 wait complete. 2024-09-11T23:21:54.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.980+0000 7ff07b8dd640 1 Processor -- start 2024-09-11T23:21:54.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.980+0000 7ff07b8dd640 1 -- start start 2024-09-11T23:21:54.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.980+0000 7ff07b8dd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff074100800 0x7ff074198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:54.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.980+0000 7ff07b8dd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff074101120 0x7ff074198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:54.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.980+0000 7ff07b8dd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0741080c0 0x7ff07419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:54.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.980+0000 7ff07b8dd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff074111ef0 con 0x7ff074101120 2024-09-11T23:21:54.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.980+0000 7ff07b8dd640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff074111d70 con 0x7ff0741080c0 2024-09-11T23:21:54.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.980+0000 7ff07b8dd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff074112070 con 0x7ff074100800 2024-09-11T23:21:54.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.980+0000 7ff078e78640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff074101120 0x7ff074198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:54.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.980+0000 7ff078e78640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff074101120 0x7ff074198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45392/0 (socket says 172.21.15.43:45392) 2024-09-11T23:21:54.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.980+0000 7ff078e78640 1 -- 172.21.15.43:0/186657444 learned_addr learned my addr 172.21.15.43:0/186657444 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:54.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.980+0000 7ff079679640 1 --2- 172.21.15.43:0/186657444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff074100800 0x7ff074198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:54.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.980+0000 7ff079e7a640 1 --2- 172.21.15.43:0/186657444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0741080c0 0x7ff07419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:54.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.980+0000 7ff078e78640 1 -- 172.21.15.43:0/186657444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff074100800 msgr2=0x7ff074198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:54.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.980+0000 7ff078e78640 1 --2- 172.21.15.43:0/186657444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff074100800 0x7ff074198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.980+0000 7ff078e78640 1 -- 172.21.15.43:0/186657444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0741080c0 msgr2=0x7ff07419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:54.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.980+0000 7ff078e78640 1 --2- 172.21.15.43:0/186657444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0741080c0 0x7ff07419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:54.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.980+0000 7ff078e78640 1 -- 172.21.15.43:0/186657444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07419d330 con 0x7ff074101120 2024-09-11T23:21:54.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.980+0000 7ff078e78640 1 --2- 172.21.15.43:0/186657444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff074101120 0x7ff074198880 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff06400c970 tx=0x7ff06400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:54.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.980+0000 7ff06a7fc640 1 -- 172.21.15.43:0/186657444 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064014070 con 0x7ff074101120 2024-09-11T23:21:54.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.981+0000 7ff07b8dd640 1 -- 172.21.15.43:0/186657444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff07419d620 con 0x7ff074101120 2024-09-11T23:21:54.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.981+0000 7ff06a7fc640 1 -- 172.21.15.43:0/186657444 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff064004440 con 0x7ff074101120 2024-09-11T23:21:54.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.981+0000 7ff06a7fc640 1 -- 172.21.15.43:0/186657444 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06400f450 con 0x7ff074101120 2024-09-11T23:21:54.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.981+0000 7ff07b8dd640 1 -- 172.21.15.43:0/186657444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0741a4eb0 con 0x7ff074101120 2024-09-11T23:21:54.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.981+0000 7ff07b8dd640 1 -- 172.21.15.43:0/186657444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff074100be0 con 0x7ff074101120 2024-09-11T23:21:54.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.984+0000 7ff06a7fc640 1 -- 172.21.15.43:0/186657444 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 23) ==== 138004+0+0 (secure 0 0 0) 0x7ff064017070 con 0x7ff074101120 2024-09-11T23:21:54.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.984+0000 7ff06a7fc640 1 --2- 172.21.15.43:0/186657444 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff0600a2d90 0x7ff0600a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:54.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.985+0000 7ff06a7fc640 1 -- 172.21.15.43:0/186657444 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff0640cf800 con 0x7ff074101120 2024-09-11T23:21:54.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.985+0000 7ff079679640 1 --2- 172.21.15.43:0/186657444 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff0600a2d90 0x7ff0600a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:54.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.985+0000 7ff079679640 1 --2- 172.21.15.43:0/186657444 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff0600a2d90 0x7ff0600a5250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff07000bbe0 tx=0x7ff07000b390 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:54.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:54.987+0000 7ff06a7fc640 1 -- 172.21.15.43:0/186657444 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff06401e020 con 0x7ff074101120 2024-09-11T23:21:55.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.160+0000 7ff07b8dd640 1 -- 172.21.15.43:0/186657444 --> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff0740ffd70 con 0x7ff0600a2d90 2024-09-11T23:21:55.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.166+0000 7ff06a7fc640 1 -- 172.21.15.43:0/186657444 <== mgr.15126 v2:172.21.15.43:6800/2510386404 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258002 (secure 0 0 0) 0x7ff0740ffd70 con 0x7ff0600a2d90 2024-09-11T23:21:55.169 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:21:55.171 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-11T23:21:55.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.170+0000 7ff07b8dd640 1 -- 172.21.15.43:0/186657444 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff0600a2d90 msgr2=0x7ff0600a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:55.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.170+0000 7ff07b8dd640 1 --2- 172.21.15.43:0/186657444 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff0600a2d90 0x7ff0600a5250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff07000bbe0 tx=0x7ff07000b390 comp rx=0 tx=0).stop 2024-09-11T23:21:55.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.170+0000 7ff07b8dd640 1 -- 172.21.15.43:0/186657444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff074101120 msgr2=0x7ff074198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:55.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.170+0000 7ff07b8dd640 1 --2- 172.21.15.43:0/186657444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff074101120 0x7ff074198880 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff06400c970 tx=0x7ff06400ce30 comp rx=0 tx=0).stop 2024-09-11T23:21:55.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.170+0000 7ff07b8dd640 1 -- 172.21.15.43:0/186657444 shutdown_connections 2024-09-11T23:21:55.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.170+0000 7ff07b8dd640 1 --2- 172.21.15.43:0/186657444 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff0600a2d90 0x7ff0600a5250 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:55.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.170+0000 7ff07b8dd640 1 --2- 172.21.15.43:0/186657444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0741080c0 0x7ff07419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:55.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.170+0000 7ff07b8dd640 1 --2- 172.21.15.43:0/186657444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff074101120 0x7ff074198880 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:55.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.170+0000 7ff07b8dd640 1 --2- 172.21.15.43:0/186657444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff074100800 0x7ff074198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:55.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.170+0000 7ff07b8dd640 1 -- 172.21.15.43:0/186657444 >> 172.21.15.43:0/186657444 conn(0x7ff074075e80 msgr2=0x7ff0740ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:55.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.170+0000 7ff07b8dd640 1 -- 172.21.15.43:0/186657444 shutdown_connections 2024-09-11T23:21:55.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.170+0000 7ff07b8dd640 1 -- 172.21.15.43:0/186657444 wait complete. 2024-09-11T23:21:55.181 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-11T23:21:54.223395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187806,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":867,"num_read_kb":594,"num_write":474,"num_write_kb":4377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369056,"kb_used_data":11460,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572768,"statfs":{"total":1151940427776,"available":1151562514432,"internally_reserved":0,"allocated":11735040,"data_stored":8948064,"data_compressed":283387,"data_compressed_allocated":372736,"data_compressed_original":3596288,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987422"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598561+0000","last_change":"2024-09-11T23:19:59.734378+0000","last_active":"2024-09-11T23:21:50.598561+0000","last_peered":"2024-09-11T23:21:50.598561+0000","last_clean":"2024-09-11T23:21:50.598561+0000","last_became_active":"2024-09-11T23:19:59.734067+0000","last_became_peered":"2024-09-11T23:19:59.734067+0000","last_unstale":"2024-09-11T23:21:50.598561+0000","last_undegraded":"2024-09-11T23:21:50.598561+0000","last_fullsized":"2024-09-11T23:21:50.598561+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:28:18.384453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.454939+0000","last_change":"2024-09-11T23:19:59.733892+0000","last_active":"2024-09-11T23:21:50.454939+0000","last_peered":"2024-09-11T23:21:50.454939+0000","last_clean":"2024-09-11T23:21:50.454939+0000","last_became_active":"2024-09-11T23:19:59.733540+0000","last_became_peered":"2024-09-11T23:19:59.733540+0000","last_unstale":"2024-09-11T23:21:50.454939+0000","last_undegraded":"2024-09-11T23:21:50.454939+0000","last_fullsized":"2024-09-11T23:21:50.454939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:42.186458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455001+0000","last_change":"2024-09-11T23:19:59.739443+0000","last_active":"2024-09-11T23:21:50.455001+0000","last_peered":"2024-09-11T23:21:50.455001+0000","last_clean":"2024-09-11T23:21:50.455001+0000","last_became_active":"2024-09-11T23:19:59.739025+0000","last_became_peered":"2024-09-11T23:19:59.739025+0000","last_unstale":"2024-09-11T23:21:50.455001+0000","last_undegraded":"2024-09-11T23:21:50.455001+0000","last_fullsized":"2024-09-11T23:21:50.455001+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:40.819321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297376+0000","last_change":"2024-09-11T23:21:02.363511+0000","last_active":"2024-09-11T23:21:50.297376+0000","last_peered":"2024-09-11T23:21:50.297376+0000","last_clean":"2024-09-11T23:21:50.297376+0000","last_became_active":"2024-09-11T23:21:02.363156+0000","last_became_peered":"2024-09-11T23:21:02.363156+0000","last_unstale":"2024-09-11T23:21:50.297376+0000","last_undegraded":"2024-09-11T23:21:50.297376+0000","last_fullsized":"2024-09-11T23:21:50.297376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:17.059509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299585+0000","last_change":"2024-09-11T23:19:59.738662+0000","last_active":"2024-09-11T23:21:50.299585+0000","last_peered":"2024-09-11T23:21:50.299585+0000","last_clean":"2024-09-11T23:21:50.299585+0000","last_became_active":"2024-09-11T23:19:59.738320+0000","last_became_peered":"2024-09-11T23:19:59.738320+0000","last_unstale":"2024-09-11T23:21:50.299585+0000","last_undegraded":"2024-09-11T23:21:50.299585+0000","last_fullsized":"2024-09-11T23:21:50.299585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:15.645179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.310947+0000","last_change":"2024-09-11T23:19:59.731442+0000","last_active":"2024-09-11T23:21:50.310947+0000","last_peered":"2024-09-11T23:21:50.310947+0000","last_clean":"2024-09-11T23:21:50.310947+0000","last_became_active":"2024-09-11T23:19:59.731079+0000","last_became_peered":"2024-09-11T23:19:59.731079+0000","last_unstale":"2024-09-11T23:21:50.310947+0000","last_undegraded":"2024-09-11T23:21:50.310947+0000","last_fullsized":"2024-09-11T23:21:50.310947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:32.764991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455210+0000","last_change":"2024-09-11T23:21:02.359992+0000","last_active":"2024-09-11T23:21:50.455210+0000","last_peered":"2024-09-11T23:21:50.455210+0000","last_clean":"2024-09-11T23:21:50.455210+0000","last_became_active":"2024-09-11T23:21:02.359564+0000","last_became_peered":"2024-09-11T23:21:02.359564+0000","last_unstale":"2024-09-11T23:21:50.455210+0000","last_undegraded":"2024-09-11T23:21:50.455210+0000","last_fullsized":"2024-09-11T23:21:50.455210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:53.633219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300215+0000","last_change":"2024-09-11T23:19:59.740733+0000","last_active":"2024-09-11T23:21:50.300215+0000","last_peered":"2024-09-11T23:21:50.300215+0000","last_clean":"2024-09-11T23:21:50.300215+0000","last_became_active":"2024-09-11T23:19:59.740438+0000","last_became_peered":"2024-09-11T23:19:59.740438+0000","last_unstale":"2024-09-11T23:21:50.300215+0000","last_undegraded":"2024-09-11T23:21:50.300215+0000","last_fullsized":"2024-09-11T23:21:50.300215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:27.063305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.212024+0000","last_change":"2024-09-11T23:19:59.742439+0000","last_active":"2024-09-11T23:21:50.212024+0000","last_peered":"2024-09-11T23:21:50.212024+0000","last_clean":"2024-09-11T23:21:50.212024+0000","last_became_active":"2024-09-11T23:19:59.741983+0000","last_became_peered":"2024-09-11T23:19:59.741983+0000","last_unstale":"2024-09-11T23:21:50.212024+0000","last_undegraded":"2024-09-11T23:21:50.212024+0000","last_fullsized":"2024-09-11T23:21:50.212024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:39.477167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311317+0000","last_change":"2024-09-11T23:19:59.732544+0000","last_active":"2024-09-11T23:21:50.311317+0000","last_peered":"2024-09-11T23:21:50.311317+0000","last_clean":"2024-09-11T23:21:50.311317+0000","last_became_active":"2024-09-11T23:19:59.732205+0000","last_became_peered":"2024-09-11T23:19:59.732205+0000","last_unstale":"2024-09-11T23:21:50.311317+0000","last_undegraded":"2024-09-11T23:21:50.311317+0000","last_fullsized":"2024-09-11T23:21:50.311317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:22.494344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455518+0000","last_change":"2024-09-11T23:19:59.741857+0000","last_active":"2024-09-11T23:21:50.455518+0000","last_peered":"2024-09-11T23:21:50.455518+0000","last_clean":"2024-09-11T23:21:50.455518+0000","last_became_active":"2024-09-11T23:19:59.741484+0000","last_became_peered":"2024-09-11T23:19:59.741484+0000","last_unstale":"2024-09-11T23:21:50.455518+0000","last_undegraded":"2024-09-11T23:21:50.455518+0000","last_fullsized":"2024-09-11T23:21:50.455518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:52.998258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.453834+0000","last_change":"2024-09-11T23:19:59.736042+0000","last_active":"2024-09-11T23:21:50.453834+0000","last_peered":"2024-09-11T23:21:50.453834+0000","last_clean":"2024-09-11T23:21:50.453834+0000","last_became_active":"2024-09-11T23:19:59.735731+0000","last_became_peered":"2024-09-11T23:19:59.735731+0000","last_unstale":"2024-09-11T23:21:50.453834+0000","last_undegraded":"2024-09-11T23:21:50.453834+0000","last_fullsized":"2024-09-11T23:21:50.453834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:19.085804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210129+0000","last_change":"2024-09-11T23:19:59.742049+0000","last_active":"2024-09-11T23:21:50.210129+0000","last_peered":"2024-09-11T23:21:50.210129+0000","last_clean":"2024-09-11T23:21:50.210129+0000","last_became_active":"2024-09-11T23:19:59.740974+0000","last_became_peered":"2024-09-11T23:19:59.740974+0000","last_unstale":"2024-09-11T23:21:50.210129+0000","last_undegraded":"2024-09-11T23:21:50.210129+0000","last_fullsized":"2024-09-11T23:21:50.210129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:10.443113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.454569+0000","last_change":"2024-09-11T23:19:59.734270+0000","last_active":"2024-09-11T23:21:50.454569+0000","last_peered":"2024-09-11T23:21:50.454569+0000","last_clean":"2024-09-11T23:21:50.454569+0000","last_became_active":"2024-09-11T23:19:59.733967+0000","last_became_peered":"2024-09-11T23:19:59.733967+0000","last_unstale":"2024-09-11T23:21:50.454569+0000","last_undegraded":"2024-09-11T23:21:50.454569+0000","last_fullsized":"2024-09-11T23:21:50.454569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:02.185757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.454215+0000","last_change":"2024-09-11T23:19:59.734152+0000","last_active":"2024-09-11T23:21:50.454215+0000","last_peered":"2024-09-11T23:21:50.454215+0000","last_clean":"2024-09-11T23:21:50.454215+0000","last_became_active":"2024-09-11T23:19:59.733163+0000","last_became_peered":"2024-09-11T23:19:59.733163+0000","last_unstale":"2024-09-11T23:21:50.454215+0000","last_undegraded":"2024-09-11T23:21:50.454215+0000","last_fullsized":"2024-09-11T23:21:50.454215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:22.370591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211142+0000","last_change":"2024-09-11T23:19:59.735517+0000","last_active":"2024-09-11T23:21:50.211142+0000","last_peered":"2024-09-11T23:21:50.211142+0000","last_clean":"2024-09-11T23:21:50.211142+0000","last_became_active":"2024-09-11T23:19:59.735182+0000","last_became_peered":"2024-09-11T23:19:59.735182+0000","last_unstale":"2024-09-11T23:21:50.211142+0000","last_undegraded":"2024-09-11T23:21:50.211142+0000","last_fullsized":"2024-09-11T23:21:50.211142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:25.989416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598585+0000","last_change":"2024-09-11T23:19:59.733579+0000","last_active":"2024-09-11T23:21:50.598585+0000","last_peered":"2024-09-11T23:21:50.598585+0000","last_clean":"2024-09-11T23:21:50.598585+0000","last_became_active":"2024-09-11T23:19:59.733245+0000","last_became_peered":"2024-09-11T23:19:59.733245+0000","last_unstale":"2024-09-11T23:21:50.598585+0000","last_undegraded":"2024-09-11T23:21:50.598585+0000","last_fullsized":"2024-09-11T23:21:50.598585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:22.361158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297955+0000","last_change":"2024-09-11T23:19:59.734354+0000","last_active":"2024-09-11T23:21:50.297955+0000","last_peered":"2024-09-11T23:21:50.297955+0000","last_clean":"2024-09-11T23:21:50.297955+0000","last_became_active":"2024-09-11T23:19:59.733988+0000","last_became_peered":"2024-09-11T23:19:59.733988+0000","last_unstale":"2024-09-11T23:21:50.297955+0000","last_undegraded":"2024-09-11T23:21:50.297955+0000","last_fullsized":"2024-09-11T23:21:50.297955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:41.401496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.212032+0000","last_change":"2024-09-11T23:19:59.730318+0000","last_active":"2024-09-11T23:21:50.212032+0000","last_peered":"2024-09-11T23:21:50.212032+0000","last_clean":"2024-09-11T23:21:50.212032+0000","last_became_active":"2024-09-11T23:19:59.730009+0000","last_became_peered":"2024-09-11T23:19:59.730009+0000","last_unstale":"2024-09-11T23:21:50.212032+0000","last_undegraded":"2024-09-11T23:21:50.212032+0000","last_fullsized":"2024-09-11T23:21:50.212032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:01:38.531967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598328+0000","last_change":"2024-09-11T23:19:59.737254+0000","last_active":"2024-09-11T23:21:50.598328+0000","last_peered":"2024-09-11T23:21:50.598328+0000","last_clean":"2024-09-11T23:21:50.598328+0000","last_became_active":"2024-09-11T23:19:59.736936+0000","last_became_peered":"2024-09-11T23:19:59.736936+0000","last_unstale":"2024-09-11T23:21:50.598328+0000","last_undegraded":"2024-09-11T23:21:50.598328+0000","last_fullsized":"2024-09-11T23:21:50.598328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:00.361468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210147+0000","last_change":"2024-09-11T23:19:59.741617+0000","last_active":"2024-09-11T23:21:50.210147+0000","last_peered":"2024-09-11T23:21:50.210147+0000","last_clean":"2024-09-11T23:21:50.210147+0000","last_became_active":"2024-09-11T23:19:59.740358+0000","last_became_peered":"2024-09-11T23:19:59.740358+0000","last_unstale":"2024-09-11T23:21:50.210147+0000","last_undegraded":"2024-09-11T23:21:50.210147+0000","last_fullsized":"2024-09-11T23:21:50.210147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:18.541105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211292+0000","last_change":"2024-09-11T23:19:59.736437+0000","last_active":"2024-09-11T23:21:50.211292+0000","last_peered":"2024-09-11T23:21:50.211292+0000","last_clean":"2024-09-11T23:21:50.211292+0000","last_became_active":"2024-09-11T23:19:59.736029+0000","last_became_peered":"2024-09-11T23:19:59.736029+0000","last_unstale":"2024-09-11T23:21:50.211292+0000","last_undegraded":"2024-09-11T23:21:50.211292+0000","last_fullsized":"2024-09-11T23:21:50.211292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:29.226939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300441+0000","last_change":"2024-09-11T23:19:59.736104+0000","last_active":"2024-09-11T23:21:50.300441+0000","last_peered":"2024-09-11T23:21:50.300441+0000","last_clean":"2024-09-11T23:21:50.300441+0000","last_became_active":"2024-09-11T23:19:59.735760+0000","last_became_peered":"2024-09-11T23:19:59.735760+0000","last_unstale":"2024-09-11T23:21:50.300441+0000","last_undegraded":"2024-09-11T23:21:50.300441+0000","last_fullsized":"2024-09-11T23:21:50.300441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:56.554313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308481+0000","last_change":"2024-09-11T23:19:59.735455+0000","last_active":"2024-09-11T23:21:50.308481+0000","last_peered":"2024-09-11T23:21:50.308481+0000","last_clean":"2024-09-11T23:21:50.308481+0000","last_became_active":"2024-09-11T23:19:59.735078+0000","last_became_peered":"2024-09-11T23:19:59.735078+0000","last_unstale":"2024-09-11T23:21:50.308481+0000","last_undegraded":"2024-09-11T23:21:50.308481+0000","last_fullsized":"2024-09-11T23:21:50.308481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:21:16.336006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.301909+0000","last_change":"2024-09-11T23:19:59.745997+0000","last_active":"2024-09-11T23:21:50.301909+0000","last_peered":"2024-09-11T23:21:50.301909+0000","last_clean":"2024-09-11T23:21:50.301909+0000","last_became_active":"2024-09-11T23:19:59.745715+0000","last_became_peered":"2024-09-11T23:19:59.745715+0000","last_unstale":"2024-09-11T23:21:50.301909+0000","last_undegraded":"2024-09-11T23:21:50.301909+0000","last_fullsized":"2024-09-11T23:21:50.301909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:34.776229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.212119+0000","last_change":"2024-09-11T23:19:59.734181+0000","last_active":"2024-09-11T23:21:50.212119+0000","last_peered":"2024-09-11T23:21:50.212119+0000","last_clean":"2024-09-11T23:21:50.212119+0000","last_became_active":"2024-09-11T23:19:59.733793+0000","last_became_peered":"2024-09-11T23:19:59.733793+0000","last_unstale":"2024-09-11T23:21:50.212119+0000","last_undegraded":"2024-09-11T23:21:50.212119+0000","last_fullsized":"2024-09-11T23:21:50.212119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:36.906308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598526+0000","last_change":"2024-09-11T23:19:59.738713+0000","last_active":"2024-09-11T23:21:50.598526+0000","last_peered":"2024-09-11T23:21:50.598526+0000","last_clean":"2024-09-11T23:21:50.598526+0000","last_became_active":"2024-09-11T23:19:59.737579+0000","last_became_peered":"2024-09-11T23:19:59.737579+0000","last_unstale":"2024-09-11T23:21:50.598526+0000","last_undegraded":"2024-09-11T23:21:50.598526+0000","last_fullsized":"2024-09-11T23:21:50.598526+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:47.135775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210205+0000","last_change":"2024-09-11T23:19:59.741381+0000","last_active":"2024-09-11T23:21:50.210205+0000","last_peered":"2024-09-11T23:21:50.210205+0000","last_clean":"2024-09-11T23:21:50.210205+0000","last_became_active":"2024-09-11T23:19:59.740914+0000","last_became_peered":"2024-09-11T23:19:59.740914+0000","last_unstale":"2024-09-11T23:21:50.210205+0000","last_undegraded":"2024-09-11T23:21:50.210205+0000","last_fullsized":"2024-09-11T23:21:50.210205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:24:04.365704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455133+0000","last_change":"2024-09-11T23:19:59.736840+0000","last_active":"2024-09-11T23:21:50.455133+0000","last_peered":"2024-09-11T23:21:50.455133+0000","last_clean":"2024-09-11T23:21:50.455133+0000","last_became_active":"2024-09-11T23:19:59.736554+0000","last_became_peered":"2024-09-11T23:19:59.736554+0000","last_unstale":"2024-09-11T23:21:50.455133+0000","last_undegraded":"2024-09-11T23:21:50.455133+0000","last_fullsized":"2024-09-11T23:21:50.455133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:48.508369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211738+0000","last_change":"2024-09-11T23:19:59.735719+0000","last_active":"2024-09-11T23:21:50.211738+0000","last_peered":"2024-09-11T23:21:50.211738+0000","last_clean":"2024-09-11T23:21:50.211738+0000","last_became_active":"2024-09-11T23:19:59.735397+0000","last_became_peered":"2024-09-11T23:19:59.735397+0000","last_unstale":"2024-09-11T23:21:50.211738+0000","last_undegraded":"2024-09-11T23:21:50.211738+0000","last_fullsized":"2024-09-11T23:21:50.211738+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:26:19.392730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211747+0000","last_change":"2024-09-11T23:19:59.737704+0000","last_active":"2024-09-11T23:21:50.211747+0000","last_peered":"2024-09-11T23:21:50.211747+0000","last_clean":"2024-09-11T23:21:50.211747+0000","last_became_active":"2024-09-11T23:19:59.737176+0000","last_became_peered":"2024-09-11T23:19:59.737176+0000","last_unstale":"2024-09-11T23:21:50.211747+0000","last_undegraded":"2024-09-11T23:21:50.211747+0000","last_fullsized":"2024-09-11T23:21:50.211747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:22.938171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598248+0000","last_change":"2024-09-11T23:19:59.735499+0000","last_active":"2024-09-11T23:21:50.598248+0000","last_peered":"2024-09-11T23:21:50.598248+0000","last_clean":"2024-09-11T23:21:50.598248+0000","last_became_active":"2024-09-11T23:19:59.735088+0000","last_became_peered":"2024-09-11T23:19:59.735088+0000","last_unstale":"2024-09-11T23:21:50.598248+0000","last_undegraded":"2024-09-11T23:21:50.598248+0000","last_fullsized":"2024-09-11T23:21:50.598248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:12.633104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299549+0000","last_change":"2024-09-11T23:19:59.743972+0000","last_active":"2024-09-11T23:21:50.299549+0000","last_peered":"2024-09-11T23:21:50.299549+0000","last_clean":"2024-09-11T23:21:50.299549+0000","last_became_active":"2024-09-11T23:19:59.743675+0000","last_became_peered":"2024-09-11T23:19:59.743675+0000","last_unstale":"2024-09-11T23:21:50.299549+0000","last_undegraded":"2024-09-11T23:21:50.299549+0000","last_fullsized":"2024-09-11T23:21:50.299549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:46.645696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":134,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.866819+0000","last_change":"2024-09-11T23:19:57.615472+0000","last_active":"2024-09-11T23:21:50.866819+0000","last_peered":"2024-09-11T23:21:50.866819+0000","last_clean":"2024-09-11T23:21:50.866819+0000","last_became_active":"2024-09-11T23:19:57.615104+0000","last_became_peered":"2024-09-11T23:19:57.615104+0000","last_unstale":"2024-09-11T23:21:50.866819+0000","last_undegraded":"2024-09-11T23:21:50.866819+0000","last_fullsized":"2024-09-11T23:21:50.866819+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:03.601363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308144+0000","last_change":"2024-09-11T23:19:59.741027+0000","last_active":"2024-09-11T23:21:50.308144+0000","last_peered":"2024-09-11T23:21:50.308144+0000","last_clean":"2024-09-11T23:21:50.308144+0000","last_became_active":"2024-09-11T23:19:59.740681+0000","last_became_peered":"2024-09-11T23:19:59.740681+0000","last_unstale":"2024-09-11T23:21:50.308144+0000","last_undegraded":"2024-09-11T23:21:50.308144+0000","last_fullsized":"2024-09-11T23:21:50.308144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:36.948147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311025+0000","last_change":"2024-09-11T23:19:57.612730+0000","last_active":"2024-09-11T23:21:50.311025+0000","last_peered":"2024-09-11T23:21:50.311025+0000","last_clean":"2024-09-11T23:21:50.311025+0000","last_became_active":"2024-09-11T23:19:57.612369+0000","last_became_peered":"2024-09-11T23:19:57.612369+0000","last_unstale":"2024-09-11T23:21:50.311025+0000","last_undegraded":"2024-09-11T23:21:50.311025+0000","last_fullsized":"2024-09-11T23:21:50.311025+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:17.823616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.209881+0000","last_change":"2024-09-11T23:19:57.611470+0000","last_active":"2024-09-11T23:21:50.209881+0000","last_peered":"2024-09-11T23:21:50.209881+0000","last_clean":"2024-09-11T23:21:50.209881+0000","last_became_active":"2024-09-11T23:19:57.611141+0000","last_became_peered":"2024-09-11T23:19:57.611141+0000","last_unstale":"2024-09-11T23:21:50.209881+0000","last_undegraded":"2024-09-11T23:21:50.209881+0000","last_fullsized":"2024-09-11T23:21:50.209881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:43.173955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300408+0000","last_change":"2024-09-11T23:19:59.732375+0000","last_active":"2024-09-11T23:21:50.300408+0000","last_peered":"2024-09-11T23:21:50.300408+0000","last_clean":"2024-09-11T23:21:50.300408+0000","last_became_active":"2024-09-11T23:19:59.731793+0000","last_became_peered":"2024-09-11T23:19:59.731793+0000","last_unstale":"2024-09-11T23:21:50.300408+0000","last_undegraded":"2024-09-11T23:21:50.300408+0000","last_fullsized":"2024-09-11T23:21:50.300408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:02:47.278646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308399+0000","last_change":"2024-09-11T23:19:57.614062+0000","last_active":"2024-09-11T23:21:50.308399+0000","last_peered":"2024-09-11T23:21:50.308399+0000","last_clean":"2024-09-11T23:21:50.308399+0000","last_became_active":"2024-09-11T23:19:57.613749+0000","last_became_peered":"2024-09-11T23:19:57.613749+0000","last_unstale":"2024-09-11T23:21:50.308399+0000","last_undegraded":"2024-09-11T23:21:50.308399+0000","last_fullsized":"2024-09-11T23:21:50.308399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:45:22.299379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297086+0000","last_change":"2024-09-11T23:19:59.730758+0000","last_active":"2024-09-11T23:21:50.297086+0000","last_peered":"2024-09-11T23:21:50.297086+0000","last_clean":"2024-09-11T23:21:50.297086+0000","last_became_active":"2024-09-11T23:19:59.730423+0000","last_became_peered":"2024-09-11T23:19:59.730423+0000","last_unstale":"2024-09-11T23:21:50.297086+0000","last_undegraded":"2024-09-11T23:21:50.297086+0000","last_fullsized":"2024-09-11T23:21:50.297086+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:30.679451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":129,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.459375+0000","last_change":"2024-09-11T23:19:57.615574+0000","last_active":"2024-09-11T23:21:50.459375+0000","last_peered":"2024-09-11T23:21:50.459375+0000","last_clean":"2024-09-11T23:21:50.459375+0000","last_became_active":"2024-09-11T23:19:57.615192+0000","last_became_peered":"2024-09-11T23:19:57.615192+0000","last_unstale":"2024-09-11T23:21:50.459375+0000","last_undegraded":"2024-09-11T23:21:50.459375+0000","last_fullsized":"2024-09-11T23:21:50.459375+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:29.186899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211749+0000","last_change":"2024-09-11T23:19:59.738433+0000","last_active":"2024-09-11T23:21:50.211749+0000","last_peered":"2024-09-11T23:21:50.211749+0000","last_clean":"2024-09-11T23:21:50.211749+0000","last_became_active":"2024-09-11T23:19:59.738081+0000","last_became_peered":"2024-09-11T23:19:59.738081+0000","last_unstale":"2024-09-11T23:21:50.211749+0000","last_undegraded":"2024-09-11T23:21:50.211749+0000","last_fullsized":"2024-09-11T23:21:50.211749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:10:46.786454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455225+0000","last_change":"2024-09-11T23:21:02.437498+0000","last_active":"2024-09-11T23:21:50.455225+0000","last_peered":"2024-09-11T23:21:50.455225+0000","last_clean":"2024-09-11T23:21:50.455225+0000","last_became_active":"2024-09-11T23:21:02.434895+0000","last_became_peered":"2024-09-11T23:21:02.434895+0000","last_unstale":"2024-09-11T23:21:50.455225+0000","last_undegraded":"2024-09-11T23:21:50.455225+0000","last_fullsized":"2024-09-11T23:21:50.455225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:33.142041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455584+0000","last_change":"2024-09-11T23:19:59.741081+0000","last_active":"2024-09-11T23:21:50.455584+0000","last_peered":"2024-09-11T23:21:50.455584+0000","last_clean":"2024-09-11T23:21:50.455584+0000","last_became_active":"2024-09-11T23:19:59.739984+0000","last_became_peered":"2024-09-11T23:19:59.739984+0000","last_unstale":"2024-09-11T23:21:50.455584+0000","last_undegraded":"2024-09-11T23:21:50.455584+0000","last_fullsized":"2024-09-11T23:21:50.455584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:03.513121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308186+0000","last_change":"2024-09-11T23:19:57.823272+0000","last_active":"2024-09-11T23:21:50.308186+0000","last_peered":"2024-09-11T23:21:50.308186+0000","last_clean":"2024-09-11T23:21:50.308186+0000","last_became_active":"2024-09-11T23:19:57.822957+0000","last_became_peered":"2024-09-11T23:19:57.822957+0000","last_unstale":"2024-09-11T23:21:50.308186+0000","last_undegraded":"2024-09-11T23:21:50.308186+0000","last_fullsized":"2024-09-11T23:21:50.308186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:06.779385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211510+0000","last_change":"2024-09-11T23:19:59.738398+0000","last_active":"2024-09-11T23:21:50.211510+0000","last_peered":"2024-09-11T23:21:50.211510+0000","last_clean":"2024-09-11T23:21:50.211510+0000","last_became_active":"2024-09-11T23:19:59.738042+0000","last_became_peered":"2024-09-11T23:19:59.738042+0000","last_unstale":"2024-09-11T23:21:50.211510+0000","last_undegraded":"2024-09-11T23:21:50.211510+0000","last_fullsized":"2024-09-11T23:21:50.211510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:05.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'34","reported_seq":271,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.867174+0000","last_change":"2024-09-11T23:19:57.615036+0000","last_active":"2024-09-11T23:21:50.867174+0000","last_peered":"2024-09-11T23:21:50.867174+0000","last_clean":"2024-09-11T23:21:50.867174+0000","last_became_active":"2024-09-11T23:19:57.614641+0000","last_became_peered":"2024-09-11T23:19:57.614641+0000","last_unstale":"2024-09-11T23:21:50.867174+0000","last_undegraded":"2024-09-11T23:21:50.867174+0000","last_fullsized":"2024-09-11T23:21:50.867174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:05.430508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67065,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":213,"num_read_kb":172,"num_write":35,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299857+0000","last_change":"2024-09-11T23:19:59.741597+0000","last_active":"2024-09-11T23:21:50.299857+0000","last_peered":"2024-09-11T23:21:50.299857+0000","last_clean":"2024-09-11T23:21:50.299857+0000","last_became_active":"2024-09-11T23:19:59.741296+0000","last_became_peered":"2024-09-11T23:19:59.741296+0000","last_unstale":"2024-09-11T23:21:50.299857+0000","last_undegraded":"2024-09-11T23:21:50.299857+0000","last_fullsized":"2024-09-11T23:21:50.299857+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:06:20.742916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211837+0000","last_change":"2024-09-11T23:19:57.617550+0000","last_active":"2024-09-11T23:21:50.211837+0000","last_peered":"2024-09-11T23:21:50.211837+0000","last_clean":"2024-09-11T23:21:50.211837+0000","last_became_active":"2024-09-11T23:19:57.617211+0000","last_became_peered":"2024-09-11T23:19:57.617211+0000","last_unstale":"2024-09-11T23:21:50.211837+0000","last_undegraded":"2024-09-11T23:21:50.211837+0000","last_fullsized":"2024-09-11T23:21:50.211837+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:19.707742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210301+0000","last_change":"2024-09-11T23:19:59.742070+0000","last_active":"2024-09-11T23:21:50.210301+0000","last_peered":"2024-09-11T23:21:50.210301+0000","last_clean":"2024-09-11T23:21:50.210301+0000","last_became_active":"2024-09-11T23:19:59.739367+0000","last_became_peered":"2024-09-11T23:19:59.739367+0000","last_unstale":"2024-09-11T23:21:50.210301+0000","last_undegraded":"2024-09-11T23:21:50.210301+0000","last_fullsized":"2024-09-11T23:21:50.210301+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:27.850136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308168+0000","last_change":"2024-09-11T23:19:57.618054+0000","last_active":"2024-09-11T23:21:50.308168+0000","last_peered":"2024-09-11T23:21:50.308168+0000","last_clean":"2024-09-11T23:21:50.308168+0000","last_became_active":"2024-09-11T23:19:57.617702+0000","last_became_peered":"2024-09-11T23:19:57.617702+0000","last_unstale":"2024-09-11T23:21:50.308168+0000","last_undegraded":"2024-09-11T23:21:50.308168+0000","last_fullsized":"2024-09-11T23:21:50.308168+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:40.171851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455548+0000","last_change":"2024-09-11T23:19:59.744000+0000","last_active":"2024-09-11T23:21:50.455548+0000","last_peered":"2024-09-11T23:21:50.455548+0000","last_clean":"2024-09-11T23:21:50.455548+0000","last_became_active":"2024-09-11T23:19:59.743629+0000","last_became_peered":"2024-09-11T23:19:59.743629+0000","last_unstale":"2024-09-11T23:21:50.455548+0000","last_undegraded":"2024-09-11T23:21:50.455548+0000","last_fullsized":"2024-09-11T23:21:50.455548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:20:04.133258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'7","reported_seq":137,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.867227+0000","last_change":"2024-09-11T23:19:57.615117+0000","last_active":"2024-09-11T23:21:50.867227+0000","last_peered":"2024-09-11T23:21:50.867227+0000","last_clean":"2024-09-11T23:21:50.867227+0000","last_became_active":"2024-09-11T23:19:57.614758+0000","last_became_peered":"2024-09-11T23:19:57.614758+0000","last_unstale":"2024-09-11T23:21:50.867227+0000","last_undegraded":"2024-09-11T23:21:50.867227+0000","last_fullsized":"2024-09-11T23:21:50.867227+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:58.113986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":6,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455281+0000","last_change":"2024-09-11T23:19:59.739403+0000","last_active":"2024-09-11T23:21:50.455281+0000","last_peered":"2024-09-11T23:21:50.455281+0000","last_clean":"2024-09-11T23:21:50.455281+0000","last_became_active":"2024-09-11T23:19:59.739078+0000","last_became_peered":"2024-09-11T23:19:59.739078+0000","last_unstale":"2024-09-11T23:21:50.455281+0000","last_undegraded":"2024-09-11T23:21:50.455281+0000","last_fullsized":"2024-09-11T23:21:50.455281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:46.168720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299488+0000","last_change":"2024-09-11T23:19:57.823323+0000","last_active":"2024-09-11T23:21:50.299488+0000","last_peered":"2024-09-11T23:21:50.299488+0000","last_clean":"2024-09-11T23:21:50.299488+0000","last_became_active":"2024-09-11T23:19:57.822947+0000","last_became_peered":"2024-09-11T23:19:57.822947+0000","last_unstale":"2024-09-11T23:21:50.299488+0000","last_undegraded":"2024-09-11T23:21:50.299488+0000","last_fullsized":"2024-09-11T23:21:50.299488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:12.542911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308091+0000","last_change":"2024-09-11T23:19:59.741163+0000","last_active":"2024-09-11T23:21:50.308091+0000","last_peered":"2024-09-11T23:21:50.308091+0000","last_clean":"2024-09-11T23:21:50.308091+0000","last_became_active":"2024-09-11T23:19:59.740633+0000","last_became_peered":"2024-09-11T23:19:59.740633+0000","last_unstale":"2024-09-11T23:21:50.308091+0000","last_undegraded":"2024-09-11T23:21:50.308091+0000","last_fullsized":"2024-09-11T23:21:50.308091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:26.301775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299941+0000","last_change":"2024-09-11T23:19:57.615804+0000","last_active":"2024-09-11T23:21:50.299941+0000","last_peered":"2024-09-11T23:21:50.299941+0000","last_clean":"2024-09-11T23:21:50.299941+0000","last_became_active":"2024-09-11T23:19:57.615443+0000","last_became_peered":"2024-09-11T23:19:57.615443+0000","last_unstale":"2024-09-11T23:21:50.299941+0000","last_undegraded":"2024-09-11T23:21:50.299941+0000","last_fullsized":"2024-09-11T23:21:50.299941+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:39.296873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598177+0000","last_change":"2024-09-11T23:21:02.360875+0000","last_active":"2024-09-11T23:21:50.598177+0000","last_peered":"2024-09-11T23:21:50.598177+0000","last_clean":"2024-09-11T23:21:50.598177+0000","last_became_active":"2024-09-11T23:21:02.360480+0000","last_became_peered":"2024-09-11T23:21:02.360480+0000","last_unstale":"2024-09-11T23:21:50.598177+0000","last_undegraded":"2024-09-11T23:21:50.598177+0000","last_fullsized":"2024-09-11T23:21:50.598177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:04.141587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210448+0000","last_change":"2024-09-11T23:19:57.612873+0000","last_active":"2024-09-11T23:21:50.210448+0000","last_peered":"2024-09-11T23:21:50.210448+0000","last_clean":"2024-09-11T23:21:50.210448+0000","last_became_active":"2024-09-11T23:19:57.612566+0000","last_became_peered":"2024-09-11T23:19:57.612566+0000","last_unstale":"2024-09-11T23:21:50.210448+0000","last_undegraded":"2024-09-11T23:21:50.210448+0000","last_fullsized":"2024-09-11T23:21:50.210448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:46.218212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455770+0000","last_change":"2024-09-11T23:19:59.736352+0000","last_active":"2024-09-11T23:21:50.455770+0000","last_peered":"2024-09-11T23:21:50.455770+0000","last_clean":"2024-09-11T23:21:50.455770+0000","last_became_active":"2024-09-11T23:19:59.735402+0000","last_became_peered":"2024-09-11T23:19:59.735402+0000","last_unstale":"2024-09-11T23:21:50.455770+0000","last_undegraded":"2024-09-11T23:21:50.455770+0000","last_fullsized":"2024-09-11T23:21:50.455770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:41.854526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'39","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.412363+0000","last_change":"2024-09-11T23:19:57.612439+0000","last_active":"2024-09-11T23:21:50.412363+0000","last_peered":"2024-09-11T23:21:50.412363+0000","last_clean":"2024-09-11T23:21:50.412363+0000","last_became_active":"2024-09-11T23:19:57.612030+0000","last_became_peered":"2024-09-11T23:19:57.612030+0000","last_unstale":"2024-09-11T23:21:50.412363+0000","last_undegraded":"2024-09-11T23:21:50.412363+0000","last_fullsized":"2024-09-11T23:21:50.412363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:36:24.579246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455480+0000","last_change":"2024-09-11T23:19:59.741241+0000","last_active":"2024-09-11T23:21:50.455480+0000","last_peered":"2024-09-11T23:21:50.455480+0000","last_clean":"2024-09-11T23:21:50.455480+0000","last_became_active":"2024-09-11T23:19:59.740265+0000","last_became_peered":"2024-09-11T23:19:59.740265+0000","last_unstale":"2024-09-11T23:21:50.455480+0000","last_undegraded":"2024-09-11T23:21:50.455480+0000","last_fullsized":"2024-09-11T23:21:50.455480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:00.676762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211975+0000","last_change":"2024-09-11T23:19:57.825238+0000","last_active":"2024-09-11T23:21:50.211975+0000","last_peered":"2024-09-11T23:21:50.211975+0000","last_clean":"2024-09-11T23:21:50.211975+0000","last_became_active":"2024-09-11T23:19:57.824732+0000","last_became_peered":"2024-09-11T23:19:57.824732+0000","last_unstale":"2024-09-11T23:21:50.211975+0000","last_undegraded":"2024-09-11T23:21:50.211975+0000","last_fullsized":"2024-09-11T23:21:50.211975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:24.465863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455634+0000","last_change":"2024-09-11T23:19:59.741338+0000","last_active":"2024-09-11T23:21:50.455634+0000","last_peered":"2024-09-11T23:21:50.455634+0000","last_clean":"2024-09-11T23:21:50.455634+0000","last_became_active":"2024-09-11T23:19:59.741019+0000","last_became_peered":"2024-09-11T23:19:59.741019+0000","last_unstale":"2024-09-11T23:21:50.455634+0000","last_undegraded":"2024-09-11T23:21:50.455634+0000","last_fullsized":"2024-09-11T23:21:50.455634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:36:03.480006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":351,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455427+0000","last_change":"2024-09-11T23:18:45.606646+0000","last_active":"2024-09-11T23:21:50.455427+0000","last_peered":"2024-09-11T23:21:50.455427+0000","last_clean":"2024-09-11T23:21:50.455427+0000","last_became_active":"2024-09-11T23:18:45.606065+0000","last_became_peered":"2024-09-11T23:18:45.606065+0000","last_unstale":"2024-09-11T23:21:50.455427+0000","last_undegraded":"2024-09-11T23:21:50.455427+0000","last_fullsized":"2024-09-11T23:21:50.455427+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:15:17.148562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:15:17.148562+0000","last_clean_scrub_stamp":"2024-09-11T23:15:17.148562+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:07.992971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.453802+0000","last_change":"2024-09-11T23:19:57.612584+0000","last_active":"2024-09-11T23:21:50.453802+0000","last_peered":"2024-09-11T23:21:50.453802+0000","last_clean":"2024-09-11T23:21:50.453802+0000","last_became_active":"2024-09-11T23:19:57.612211+0000","last_became_peered":"2024-09-11T23:19:57.612211+0000","last_unstale":"2024-09-11T23:21:50.453802+0000","last_undegraded":"2024-09-11T23:21:50.453802+0000","last_fullsized":"2024-09-11T23:21:50.453802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:04.026233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300938+0000","last_change":"2024-09-11T23:21:02.359249+0000","last_active":"2024-09-11T23:21:50.300938+0000","last_peered":"2024-09-11T23:21:50.300938+0000","last_clean":"2024-09-11T23:21:50.300938+0000","last_became_active":"2024-09-11T23:21:02.358906+0000","last_became_peered":"2024-09-11T23:21:02.358906+0000","last_unstale":"2024-09-11T23:21:50.300938+0000","last_undegraded":"2024-09-11T23:21:50.300938+0000","last_fullsized":"2024-09-11T23:21:50.300938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:05.973581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455432+0000","last_change":"2024-09-11T23:19:57.618283+0000","last_active":"2024-09-11T23:21:50.455432+0000","last_peered":"2024-09-11T23:21:50.455432+0000","last_clean":"2024-09-11T23:21:50.455432+0000","last_became_active":"2024-09-11T23:19:57.617942+0000","last_became_peered":"2024-09-11T23:19:57.617942+0000","last_unstale":"2024-09-11T23:21:50.455432+0000","last_undegraded":"2024-09-11T23:21:50.455432+0000","last_fullsized":"2024-09-11T23:21:50.455432+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:30.832602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.209864+0000","last_change":"2024-09-11T23:19:59.740730+0000","last_active":"2024-09-11T23:21:50.209864+0000","last_peered":"2024-09-11T23:21:50.209864+0000","last_clean":"2024-09-11T23:21:50.209864+0000","last_became_active":"2024-09-11T23:19:59.740369+0000","last_became_peered":"2024-09-11T23:19:59.740369+0000","last_unstale":"2024-09-11T23:21:50.209864+0000","last_undegraded":"2024-09-11T23:21:50.209864+0000","last_fullsized":"2024-09-11T23:21:50.209864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:50.264644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":131,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.454901+0000","last_change":"2024-09-11T23:19:57.823253+0000","last_active":"2024-09-11T23:21:50.454901+0000","last_peered":"2024-09-11T23:21:50.454901+0000","last_clean":"2024-09-11T23:21:50.454901+0000","last_became_active":"2024-09-11T23:19:57.822895+0000","last_became_peered":"2024-09-11T23:19:57.822895+0000","last_unstale":"2024-09-11T23:21:50.454901+0000","last_undegraded":"2024-09-11T23:21:50.454901+0000","last_fullsized":"2024-09-11T23:21:50.454901+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:22.998371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308142+0000","last_change":"2024-09-11T23:19:59.742248+0000","last_active":"2024-09-11T23:21:50.308142+0000","last_peered":"2024-09-11T23:21:50.308142+0000","last_clean":"2024-09-11T23:21:50.308142+0000","last_became_active":"2024-09-11T23:19:59.741810+0000","last_became_peered":"2024-09-11T23:19:59.741810+0000","last_unstale":"2024-09-11T23:21:50.308142+0000","last_undegraded":"2024-09-11T23:21:50.308142+0000","last_fullsized":"2024-09-11T23:21:50.308142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:19:52.720286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455504+0000","last_change":"2024-09-11T23:19:57.616244+0000","last_active":"2024-09-11T23:21:50.455504+0000","last_peered":"2024-09-11T23:21:50.455504+0000","last_clean":"2024-09-11T23:21:50.455504+0000","last_became_active":"2024-09-11T23:19:57.615868+0000","last_became_peered":"2024-09-11T23:19:57.615868+0000","last_unstale":"2024-09-11T23:21:50.455504+0000","last_undegraded":"2024-09-11T23:21:50.455504+0000","last_fullsized":"2024-09-11T23:21:50.455504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:42.510896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311069+0000","last_change":"2024-09-11T23:21:02.434265+0000","last_active":"2024-09-11T23:21:50.311069+0000","last_peered":"2024-09-11T23:21:50.311069+0000","last_clean":"2024-09-11T23:21:50.311069+0000","last_became_active":"2024-09-11T23:21:02.433840+0000","last_became_peered":"2024-09-11T23:21:02.433840+0000","last_unstale":"2024-09-11T23:21:50.311069+0000","last_undegraded":"2024-09-11T23:21:50.311069+0000","last_fullsized":"2024-09-11T23:21:50.311069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:04.619695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297618+0000","last_change":"2024-09-11T23:19:57.824334+0000","last_active":"2024-09-11T23:21:50.297618+0000","last_peered":"2024-09-11T23:21:50.297618+0000","last_clean":"2024-09-11T23:21:50.297618+0000","last_became_active":"2024-09-11T23:19:57.823972+0000","last_became_peered":"2024-09-11T23:19:57.823972+0000","last_unstale":"2024-09-11T23:21:50.297618+0000","last_undegraded":"2024-09-11T23:21:50.297618+0000","last_fullsized":"2024-09-11T23:21:50.297618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:39.183281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311164+0000","last_change":"2024-09-11T23:19:59.728064+0000","last_active":"2024-09-11T23:21:50.311164+0000","last_peered":"2024-09-11T23:21:50.311164+0000","last_clean":"2024-09-11T23:21:50.311164+0000","last_became_active":"2024-09-11T23:19:59.727732+0000","last_became_peered":"2024-09-11T23:19:59.727732+0000","last_unstale":"2024-09-11T23:21:50.311164+0000","last_undegraded":"2024-09-11T23:21:50.311164+0000","last_fullsized":"2024-09-11T23:21:50.311164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:15.383974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299855+0000","last_change":"2024-09-11T23:19:57.824362+0000","last_active":"2024-09-11T23:21:50.299855+0000","last_peered":"2024-09-11T23:21:50.299855+0000","last_clean":"2024-09-11T23:21:50.299855+0000","last_became_active":"2024-09-11T23:19:57.823753+0000","last_became_peered":"2024-09-11T23:19:57.823753+0000","last_unstale":"2024-09-11T23:21:50.299855+0000","last_undegraded":"2024-09-11T23:21:50.299855+0000","last_fullsized":"2024-09-11T23:21:50.299855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:56.357268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210323+0000","last_change":"2024-09-11T23:19:59.730623+0000","last_active":"2024-09-11T23:21:50.210323+0000","last_peered":"2024-09-11T23:21:50.210323+0000","last_clean":"2024-09-11T23:21:50.210323+0000","last_became_active":"2024-09-11T23:19:59.730307+0000","last_became_peered":"2024-09-11T23:19:59.730307+0000","last_unstale":"2024-09-11T23:21:50.210323+0000","last_undegraded":"2024-09-11T23:21:50.210323+0000","last_fullsized":"2024-09-11T23:21:50.210323+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:52.874884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'48","reported_seq":95,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.418264+0000","last_change":"2024-09-11T23:19:57.611730+0000","last_active":"2024-09-11T23:21:50.418264+0000","last_peered":"2024-09-11T23:21:50.418264+0000","last_clean":"2024-09-11T23:21:50.418264+0000","last_became_active":"2024-09-11T23:19:57.611394+0000","last_became_peered":"2024-09-11T23:19:57.611394+0000","last_unstale":"2024-09-11T23:21:50.418264+0000","last_undegraded":"2024-09-11T23:21:50.418264+0000","last_fullsized":"2024-09-11T23:21:50.418264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:15.518573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.302035+0000","last_change":"2024-09-11T23:19:59.741396+0000","last_active":"2024-09-11T23:21:50.302035+0000","last_peered":"2024-09-11T23:21:50.302035+0000","last_clean":"2024-09-11T23:21:50.302035+0000","last_became_active":"2024-09-11T23:19:59.741102+0000","last_became_peered":"2024-09-11T23:19:59.741102+0000","last_unstale":"2024-09-11T23:21:50.302035+0000","last_undegraded":"2024-09-11T23:21:50.302035+0000","last_fullsized":"2024-09-11T23:21:50.302035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:08.759512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.453918+0000","last_change":"2024-09-11T23:19:57.824812+0000","last_active":"2024-09-11T23:21:50.453918+0000","last_peered":"2024-09-11T23:21:50.453918+0000","last_clean":"2024-09-11T23:21:50.453918+0000","last_became_active":"2024-09-11T23:19:57.824454+0000","last_became_peered":"2024-09-11T23:19:57.824454+0000","last_unstale":"2024-09-11T23:21:50.453918+0000","last_undegraded":"2024-09-11T23:21:50.453918+0000","last_fullsized":"2024-09-11T23:21:50.453918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:25.275573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455034+0000","last_change":"2024-09-11T23:19:59.735857+0000","last_active":"2024-09-11T23:21:50.455034+0000","last_peered":"2024-09-11T23:21:50.455034+0000","last_clean":"2024-09-11T23:21:50.455034+0000","last_became_active":"2024-09-11T23:19:59.735541+0000","last_became_peered":"2024-09-11T23:19:59.735541+0000","last_unstale":"2024-09-11T23:21:50.455034+0000","last_undegraded":"2024-09-11T23:21:50.455034+0000","last_fullsized":"2024-09-11T23:21:50.455034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:21.600490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211231+0000","last_change":"2024-09-11T23:19:57.823468+0000","last_active":"2024-09-11T23:21:50.211231+0000","last_peered":"2024-09-11T23:21:50.211231+0000","last_clean":"2024-09-11T23:21:50.211231+0000","last_became_active":"2024-09-11T23:19:57.823154+0000","last_became_peered":"2024-09-11T23:19:57.823154+0000","last_unstale":"2024-09-11T23:21:50.211231+0000","last_undegraded":"2024-09-11T23:21:50.211231+0000","last_fullsized":"2024-09-11T23:21:50.211231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:28.770627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299509+0000","last_change":"2024-09-11T23:19:59.741263+0000","last_active":"2024-09-11T23:21:50.299509+0000","last_peered":"2024-09-11T23:21:50.299509+0000","last_clean":"2024-09-11T23:21:50.299509+0000","last_became_active":"2024-09-11T23:19:59.740938+0000","last_became_peered":"2024-09-11T23:19:59.740938+0000","last_unstale":"2024-09-11T23:21:50.299509+0000","last_undegraded":"2024-09-11T23:21:50.299509+0000","last_fullsized":"2024-09-11T23:21:50.299509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:54.768652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.301723+0000","last_change":"2024-09-11T23:19:57.612517+0000","last_active":"2024-09-11T23:21:50.301723+0000","last_peered":"2024-09-11T23:21:50.301723+0000","last_clean":"2024-09-11T23:21:50.301723+0000","last_became_active":"2024-09-11T23:19:57.612188+0000","last_became_peered":"2024-09-11T23:19:57.612188+0000","last_unstale":"2024-09-11T23:21:50.301723+0000","last_undegraded":"2024-09-11T23:21:50.301723+0000","last_fullsized":"2024-09-11T23:21:50.301723+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:52.896413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598315+0000","last_change":"2024-09-11T23:19:59.732520+0000","last_active":"2024-09-11T23:21:50.598315+0000","last_peered":"2024-09-11T23:21:50.598315+0000","last_clean":"2024-09-11T23:21:50.598315+0000","last_became_active":"2024-09-11T23:19:59.732173+0000","last_became_peered":"2024-09-11T23:19:59.732173+0000","last_unstale":"2024-09-11T23:21:50.598315+0000","last_undegraded":"2024-09-11T23:21:50.598315+0000","last_fullsized":"2024-09-11T23:21:50.598315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:04.626312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598177+0000","last_change":"2024-09-11T23:21:02.365533+0000","last_active":"2024-09-11T23:21:50.598177+0000","last_peered":"2024-09-11T23:21:50.598177+0000","last_clean":"2024-09-11T23:21:50.598177+0000","last_became_active":"2024-09-11T23:21:02.359345+0000","last_became_peered":"2024-09-11T23:21:02.359345+0000","last_unstale":"2024-09-11T23:21:50.598177+0000","last_undegraded":"2024-09-11T23:21:50.598177+0000","last_fullsized":"2024-09-11T23:21:50.598177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:27.195666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.301266+0000","last_change":"2024-09-11T23:19:59.734667+0000","last_active":"2024-09-11T23:21:50.301266+0000","last_peered":"2024-09-11T23:21:50.301266+0000","last_clean":"2024-09-11T23:21:50.301266+0000","last_became_active":"2024-09-11T23:19:59.733754+0000","last_became_peered":"2024-09-11T23:19:59.733754+0000","last_unstale":"2024-09-11T23:21:50.301266+0000","last_undegraded":"2024-09-11T23:21:50.301266+0000","last_fullsized":"2024-09-11T23:21:50.301266+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:57.392272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299623+0000","last_change":"2024-09-11T23:19:57.614579+0000","last_active":"2024-09-11T23:21:50.299623+0000","last_peered":"2024-09-11T23:21:50.299623+0000","last_clean":"2024-09-11T23:21:50.299623+0000","last_became_active":"2024-09-11T23:19:57.614227+0000","last_became_peered":"2024-09-11T23:19:57.614227+0000","last_unstale":"2024-09-11T23:21:50.299623+0000","last_undegraded":"2024-09-11T23:21:50.299623+0000","last_fullsized":"2024-09-11T23:21:50.299623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:47.952920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455414+0000","last_change":"2024-09-11T23:19:59.744219+0000","last_active":"2024-09-11T23:21:50.455414+0000","last_peered":"2024-09-11T23:21:50.455414+0000","last_clean":"2024-09-11T23:21:50.455414+0000","last_became_active":"2024-09-11T23:19:59.743914+0000","last_became_peered":"2024-09-11T23:19:59.743914+0000","last_unstale":"2024-09-11T23:21:50.455414+0000","last_undegraded":"2024-09-11T23:21:50.455414+0000","last_fullsized":"2024-09-11T23:21:50.455414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:52.149588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308372+0000","last_change":"2024-09-11T23:21:02.361647+0000","last_active":"2024-09-11T23:21:50.308372+0000","last_peered":"2024-09-11T23:21:50.308372+0000","last_clean":"2024-09-11T23:21:50.308372+0000","last_became_active":"2024-09-11T23:21:02.361319+0000","last_became_peered":"2024-09-11T23:21:02.361319+0000","last_unstale":"2024-09-11T23:21:50.308372+0000","last_undegraded":"2024-09-11T23:21:50.308372+0000","last_fullsized":"2024-09-11T23:21:50.308372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:45.125525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299796+0000","last_change":"2024-09-11T23:19:57.610389+0000","last_active":"2024-09-11T23:21:50.299796+0000","last_peered":"2024-09-11T23:21:50.299796+0000","last_clean":"2024-09-11T23:21:50.299796+0000","last_became_active":"2024-09-11T23:19:57.609911+0000","last_became_peered":"2024-09-11T23:19:57.609911+0000","last_unstale":"2024-09-11T23:21:50.299796+0000","last_undegraded":"2024-09-11T23:21:50.299796+0000","last_fullsized":"2024-09-11T23:21:50.299796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:23.949929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211496+0000","last_change":"2024-09-11T23:19:59.736867+0000","last_active":"2024-09-11T23:21:50.211496+0000","last_peered":"2024-09-11T23:21:50.211496+0000","last_clean":"2024-09-11T23:21:50.211496+0000","last_became_active":"2024-09-11T23:19:59.736341+0000","last_became_peered":"2024-09-11T23:19:59.736341+0000","last_unstale":"2024-09-11T23:21:50.211496+0000","last_undegraded":"2024-09-11T23:21:50.211496+0000","last_fullsized":"2024-09-11T23:21:50.211496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:57:51.015235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598177+0000","last_change":"2024-09-11T23:21:02.368959+0000","last_active":"2024-09-11T23:21:50.598177+0000","last_peered":"2024-09-11T23:21:50.598177+0000","last_clean":"2024-09-11T23:21:50.598177+0000","last_became_active":"2024-09-11T23:21:02.363263+0000","last_became_peered":"2024-09-11T23:21:02.363263+0000","last_unstale":"2024-09-11T23:21:50.598177+0000","last_undegraded":"2024-09-11T23:21:50.598177+0000","last_fullsized":"2024-09-11T23:21:50.598177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:29.827330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455355+0000","last_change":"2024-09-11T23:19:59.742049+0000","last_active":"2024-09-11T23:21:50.455355+0000","last_peered":"2024-09-11T23:21:50.455355+0000","last_clean":"2024-09-11T23:21:50.455355+0000","last_became_active":"2024-09-11T23:19:59.741763+0000","last_became_peered":"2024-09-11T23:19:59.741763+0000","last_unstale":"2024-09-11T23:21:50.455355+0000","last_undegraded":"2024-09-11T23:21:50.455355+0000","last_fullsized":"2024-09-11T23:21:50.455355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:48.698896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.302105+0000","last_change":"2024-09-11T23:19:57.822949+0000","last_active":"2024-09-11T23:21:50.302105+0000","last_peered":"2024-09-11T23:21:50.302105+0000","last_clean":"2024-09-11T23:21:50.302105+0000","last_became_active":"2024-09-11T23:19:57.822564+0000","last_became_peered":"2024-09-11T23:19:57.822564+0000","last_unstale":"2024-09-11T23:21:50.302105+0000","last_undegraded":"2024-09-11T23:21:50.302105+0000","last_fullsized":"2024-09-11T23:21:50.302105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:29.532109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297740+0000","last_change":"2024-09-11T23:21:02.363639+0000","last_active":"2024-09-11T23:21:50.297740+0000","last_peered":"2024-09-11T23:21:50.297740+0000","last_clean":"2024-09-11T23:21:50.297740+0000","last_became_active":"2024-09-11T23:21:02.363247+0000","last_became_peered":"2024-09-11T23:21:02.363247+0000","last_unstale":"2024-09-11T23:21:50.297740+0000","last_undegraded":"2024-09-11T23:21:50.297740+0000","last_fullsized":"2024-09-11T23:21:50.297740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:18:58.906939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297845+0000","last_change":"2024-09-11T23:19:57.613323+0000","last_active":"2024-09-11T23:21:50.297845+0000","last_peered":"2024-09-11T23:21:50.297845+0000","last_clean":"2024-09-11T23:21:50.297845+0000","last_became_active":"2024-09-11T23:19:57.612985+0000","last_became_peered":"2024-09-11T23:19:57.612985+0000","last_unstale":"2024-09-11T23:21:50.297845+0000","last_undegraded":"2024-09-11T23:21:50.297845+0000","last_fullsized":"2024-09-11T23:21:50.297845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:13:09.179547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597204,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":731,"num_read_kb":475,"num_write":291,"num_write_kb":1437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1791612,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108948,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514531,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":441815,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743463,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":1242266,"data_compressed":51171,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906990,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":962560,"data_stored":844252,"data_compressed":27845,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168627,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":844172,"data_compressed":27808,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397560,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":987136,"data_stored":1044208,"data_compressed":45015,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561085,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":450425,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790018,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":1174172,"data_compressed":51171,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":452182,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149772,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":975552,"data_compressed":34001,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":519026,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":995328,"data_stored":517283,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":442711,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133469,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":201563,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:21:55.185 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-11T23:21:55.185 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T23:21:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.328+0000 7f52608f1640 1 -- 172.21.15.43:0/2141275074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f525c102cf0 msgr2=0x7f525c1030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.328+0000 7f52608f1640 1 --2- 172.21.15.43:0/2141275074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f525c102cf0 0x7f525c1030d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f524c009900 tx=0x7f524c02a480 comp rx=0 tx=0).stop 2024-09-11T23:21:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.328+0000 7f52608f1640 1 -- 172.21.15.43:0/2141275074 shutdown_connections 2024-09-11T23:21:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.328+0000 7f52608f1640 1 --2- 172.21.15.43:0/2141275074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f525c10c0c0 0x7f525c10e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.328+0000 7f52608f1640 1 --2- 172.21.15.43:0/2141275074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f525c103610 0x7f525c10baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.328+0000 7f52608f1640 1 --2- 172.21.15.43:0/2141275074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f525c102cf0 0x7f525c1030d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.328+0000 7f52608f1640 1 -- 172.21.15.43:0/2141275074 >> 172.21.15.43:0/2141275074 conn(0x7f525c077a10 msgr2=0x7f525c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.328+0000 7f52608f1640 1 -- 172.21.15.43:0/2141275074 shutdown_connections 2024-09-11T23:21:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.328+0000 7f52608f1640 1 -- 172.21.15.43:0/2141275074 wait complete. 2024-09-11T23:21:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.329+0000 7f52608f1640 1 Processor -- start 2024-09-11T23:21:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.329+0000 7f52608f1640 1 -- start start 2024-09-11T23:21:55.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.329+0000 7f52608f1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f525c102cf0 0x7f525c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:55.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.329+0000 7f52608f1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f525c103610 0x7f525c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:55.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.329+0000 7f52608f1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f525c10c0c0 0x7f525c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:55.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.329+0000 7f52608f1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f525c111f20 con 0x7f525c102cf0 2024-09-11T23:21:55.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.329+0000 7f52608f1640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f525c111da0 con 0x7f525c10c0c0 2024-09-11T23:21:55.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.329+0000 7f52608f1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f525c1120a0 con 0x7f525c103610 2024-09-11T23:21:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.329+0000 7f525a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f525c102cf0 0x7f525c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.329+0000 7f525a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f525c102cf0 0x7f525c198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45414/0 (socket says 172.21.15.43:45414) 2024-09-11T23:21:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.329+0000 7f525a59c640 1 -- 172.21.15.43:0/3525975727 learned_addr learned my addr 172.21.15.43:0/3525975727 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.329+0000 7f5253fff640 1 --2- 172.21.15.43:0/3525975727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f525c103610 0x7f525c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.329+0000 7f525ad9d640 1 --2- 172.21.15.43:0/3525975727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f525c10c0c0 0x7f525c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.329+0000 7f525a59c640 1 -- 172.21.15.43:0/3525975727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f525c103610 msgr2=0x7f525c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.329+0000 7f525a59c640 1 --2- 172.21.15.43:0/3525975727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f525c103610 0x7f525c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.329+0000 7f525a59c640 1 -- 172.21.15.43:0/3525975727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f525c10c0c0 msgr2=0x7f525c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.329+0000 7f525a59c640 1 --2- 172.21.15.43:0/3525975727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f525c10c0c0 0x7f525c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.329+0000 7f525a59c640 1 -- 172.21.15.43:0/3525975727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525c19d420 con 0x7f525c102cf0 2024-09-11T23:21:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.330+0000 7f525a59c640 1 --2- 172.21.15.43:0/3525975727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f525c102cf0 0x7f525c198430 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f524c006010 tx=0x7f524c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.330+0000 7f52537fe640 1 -- 172.21.15.43:0/3525975727 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c03f030 con 0x7f525c102cf0 2024-09-11T23:21:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.330+0000 7f52537fe640 1 -- 172.21.15.43:0/3525975727 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f524c003ce0 con 0x7f525c102cf0 2024-09-11T23:21:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.330+0000 7f52608f1640 1 -- 172.21.15.43:0/3525975727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f525c19d6b0 con 0x7f525c102cf0 2024-09-11T23:21:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.330+0000 7f52537fe640 1 -- 172.21.15.43:0/3525975727 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c035070 con 0x7f525c102cf0 2024-09-11T23:21:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.330+0000 7f52608f1640 1 -- 172.21.15.43:0/3525975727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f525c1a4fa0 con 0x7f525c102cf0 2024-09-11T23:21:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.331+0000 7f52608f1640 1 -- 172.21.15.43:0/3525975727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f525c1030d0 con 0x7f525c102cf0 2024-09-11T23:21:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.332+0000 7f52537fe640 1 -- 172.21.15.43:0/3525975727 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f524c042030 con 0x7f525c102cf0 2024-09-11T23:21:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.333+0000 7f52537fe640 1 --2- 172.21.15.43:0/3525975727 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f52480a2e20 0x7f52480a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.333+0000 7f5253fff640 1 --2- 172.21.15.43:0/3525975727 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f52480a2e20 0x7f52480a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.333+0000 7f52537fe640 1 -- 172.21.15.43:0/3525975727 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f524c0efce0 con 0x7f525c102cf0 2024-09-11T23:21:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.334+0000 7f5253fff640 1 --2- 172.21.15.43:0/3525975727 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f52480a2e20 0x7f52480a52e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f525c10e530 tx=0x7f5244009110 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:55.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.337+0000 7f52537fe640 1 -- 172.21.15.43:0/3525975727 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f524c0bc780 con 0x7f525c102cf0 2024-09-11T23:21:55.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:55 smithi043 ceph-mon[30950]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:21:55.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:55 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:55.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:55 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:55.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:55 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:55.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:55 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:21:55.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:55 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:21:55.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:55 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:21:55.463 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:55 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:55.463 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:55 smithi043 ceph-mon[30950]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:21:55.463 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:55 smithi043 ceph-mon[30950]: mgrmap e24: x(active, since 5s), standbys: z, y 2024-09-11T23:21:55.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.510+0000 7f52608f1640 1 -- 172.21.15.43:0/3525975727 --> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f525c062630 con 0x7f52480a2e20 2024-09-11T23:21:55.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.517+0000 7f52537fe640 1 -- 172.21.15.43:0/3525975727 <== mgr.15126 v2:172.21.15.43:6800/2510386404 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258002 (secure 0 0 0) 0x7f525c062630 con 0x7f52480a2e20 2024-09-11T23:21:55.519 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:21:55.521 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-11T23:21:55.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.520+0000 7f52608f1640 1 -- 172.21.15.43:0/3525975727 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f52480a2e20 msgr2=0x7f52480a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:55.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.520+0000 7f52608f1640 1 --2- 172.21.15.43:0/3525975727 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f52480a2e20 0x7f52480a52e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f525c10e530 tx=0x7f5244009110 comp rx=0 tx=0).stop 2024-09-11T23:21:55.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.520+0000 7f52608f1640 1 -- 172.21.15.43:0/3525975727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f525c102cf0 msgr2=0x7f525c198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:55.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.520+0000 7f52608f1640 1 --2- 172.21.15.43:0/3525975727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f525c102cf0 0x7f525c198430 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f524c006010 tx=0x7f524c0042f0 comp rx=0 tx=0).stop 2024-09-11T23:21:55.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.520+0000 7f52608f1640 1 -- 172.21.15.43:0/3525975727 shutdown_connections 2024-09-11T23:21:55.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.520+0000 7f52608f1640 1 --2- 172.21.15.43:0/3525975727 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f52480a2e20 0x7f52480a52e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:55.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.520+0000 7f52608f1640 1 --2- 172.21.15.43:0/3525975727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f525c10c0c0 0x7f525c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:55.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.520+0000 7f52608f1640 1 --2- 172.21.15.43:0/3525975727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f525c103610 0x7f525c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:55.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.520+0000 7f52608f1640 1 --2- 172.21.15.43:0/3525975727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f525c102cf0 0x7f525c198430 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:55.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.520+0000 7f52608f1640 1 -- 172.21.15.43:0/3525975727 >> 172.21.15.43:0/3525975727 conn(0x7f525c077a10 msgr2=0x7f525c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:55.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.520+0000 7f52608f1640 1 -- 172.21.15.43:0/3525975727 shutdown_connections 2024-09-11T23:21:55.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:55.520+0000 7f52608f1640 1 -- 172.21.15.43:0/3525975727 wait complete. 2024-09-11T23:21:55.523 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:55 smithi090 ceph-mon[35130]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:21:55.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:55 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:55.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:55 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:55.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:55 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:55.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:55 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:21:55.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:55 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:21:55.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:55 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:21:55.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:55 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:55.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:55 smithi090 ceph-mon[35130]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:21:55.524 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:55 smithi090 ceph-mon[35130]: mgrmap e24: x(active, since 5s), standbys: z, y 2024-09-11T23:21:55.531 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-11T23:21:54.223395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187806,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":867,"num_read_kb":594,"num_write":474,"num_write_kb":4377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369056,"kb_used_data":11460,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572768,"statfs":{"total":1151940427776,"available":1151562514432,"internally_reserved":0,"allocated":11735040,"data_stored":8948064,"data_compressed":283387,"data_compressed_allocated":372736,"data_compressed_original":3596288,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987422"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598561+0000","last_change":"2024-09-11T23:19:59.734378+0000","last_active":"2024-09-11T23:21:50.598561+0000","last_peered":"2024-09-11T23:21:50.598561+0000","last_clean":"2024-09-11T23:21:50.598561+0000","last_became_active":"2024-09-11T23:19:59.734067+0000","last_became_peered":"2024-09-11T23:19:59.734067+0000","last_unstale":"2024-09-11T23:21:50.598561+0000","last_undegraded":"2024-09-11T23:21:50.598561+0000","last_fullsized":"2024-09-11T23:21:50.598561+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:28:18.384453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.454939+0000","last_change":"2024-09-11T23:19:59.733892+0000","last_active":"2024-09-11T23:21:50.454939+0000","last_peered":"2024-09-11T23:21:50.454939+0000","last_clean":"2024-09-11T23:21:50.454939+0000","last_became_active":"2024-09-11T23:19:59.733540+0000","last_became_peered":"2024-09-11T23:19:59.733540+0000","last_unstale":"2024-09-11T23:21:50.454939+0000","last_undegraded":"2024-09-11T23:21:50.454939+0000","last_fullsized":"2024-09-11T23:21:50.454939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:42.186458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455001+0000","last_change":"2024-09-11T23:19:59.739443+0000","last_active":"2024-09-11T23:21:50.455001+0000","last_peered":"2024-09-11T23:21:50.455001+0000","last_clean":"2024-09-11T23:21:50.455001+0000","last_became_active":"2024-09-11T23:19:59.739025+0000","last_became_peered":"2024-09-11T23:19:59.739025+0000","last_unstale":"2024-09-11T23:21:50.455001+0000","last_undegraded":"2024-09-11T23:21:50.455001+0000","last_fullsized":"2024-09-11T23:21:50.455001+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:40.819321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297376+0000","last_change":"2024-09-11T23:21:02.363511+0000","last_active":"2024-09-11T23:21:50.297376+0000","last_peered":"2024-09-11T23:21:50.297376+0000","last_clean":"2024-09-11T23:21:50.297376+0000","last_became_active":"2024-09-11T23:21:02.363156+0000","last_became_peered":"2024-09-11T23:21:02.363156+0000","last_unstale":"2024-09-11T23:21:50.297376+0000","last_undegraded":"2024-09-11T23:21:50.297376+0000","last_fullsized":"2024-09-11T23:21:50.297376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:17.059509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299585+0000","last_change":"2024-09-11T23:19:59.738662+0000","last_active":"2024-09-11T23:21:50.299585+0000","last_peered":"2024-09-11T23:21:50.299585+0000","last_clean":"2024-09-11T23:21:50.299585+0000","last_became_active":"2024-09-11T23:19:59.738320+0000","last_became_peered":"2024-09-11T23:19:59.738320+0000","last_unstale":"2024-09-11T23:21:50.299585+0000","last_undegraded":"2024-09-11T23:21:50.299585+0000","last_fullsized":"2024-09-11T23:21:50.299585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:15.645179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.310947+0000","last_change":"2024-09-11T23:19:59.731442+0000","last_active":"2024-09-11T23:21:50.310947+0000","last_peered":"2024-09-11T23:21:50.310947+0000","last_clean":"2024-09-11T23:21:50.310947+0000","last_became_active":"2024-09-11T23:19:59.731079+0000","last_became_peered":"2024-09-11T23:19:59.731079+0000","last_unstale":"2024-09-11T23:21:50.310947+0000","last_undegraded":"2024-09-11T23:21:50.310947+0000","last_fullsized":"2024-09-11T23:21:50.310947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:32.764991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455210+0000","last_change":"2024-09-11T23:21:02.359992+0000","last_active":"2024-09-11T23:21:50.455210+0000","last_peered":"2024-09-11T23:21:50.455210+0000","last_clean":"2024-09-11T23:21:50.455210+0000","last_became_active":"2024-09-11T23:21:02.359564+0000","last_became_peered":"2024-09-11T23:21:02.359564+0000","last_unstale":"2024-09-11T23:21:50.455210+0000","last_undegraded":"2024-09-11T23:21:50.455210+0000","last_fullsized":"2024-09-11T23:21:50.455210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:53.633219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300215+0000","last_change":"2024-09-11T23:19:59.740733+0000","last_active":"2024-09-11T23:21:50.300215+0000","last_peered":"2024-09-11T23:21:50.300215+0000","last_clean":"2024-09-11T23:21:50.300215+0000","last_became_active":"2024-09-11T23:19:59.740438+0000","last_became_peered":"2024-09-11T23:19:59.740438+0000","last_unstale":"2024-09-11T23:21:50.300215+0000","last_undegraded":"2024-09-11T23:21:50.300215+0000","last_fullsized":"2024-09-11T23:21:50.300215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:27.063305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.212024+0000","last_change":"2024-09-11T23:19:59.742439+0000","last_active":"2024-09-11T23:21:50.212024+0000","last_peered":"2024-09-11T23:21:50.212024+0000","last_clean":"2024-09-11T23:21:50.212024+0000","last_became_active":"2024-09-11T23:19:59.741983+0000","last_became_peered":"2024-09-11T23:19:59.741983+0000","last_unstale":"2024-09-11T23:21:50.212024+0000","last_undegraded":"2024-09-11T23:21:50.212024+0000","last_fullsized":"2024-09-11T23:21:50.212024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:39.477167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311317+0000","last_change":"2024-09-11T23:19:59.732544+0000","last_active":"2024-09-11T23:21:50.311317+0000","last_peered":"2024-09-11T23:21:50.311317+0000","last_clean":"2024-09-11T23:21:50.311317+0000","last_became_active":"2024-09-11T23:19:59.732205+0000","last_became_peered":"2024-09-11T23:19:59.732205+0000","last_unstale":"2024-09-11T23:21:50.311317+0000","last_undegraded":"2024-09-11T23:21:50.311317+0000","last_fullsized":"2024-09-11T23:21:50.311317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:22.494344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455518+0000","last_change":"2024-09-11T23:19:59.741857+0000","last_active":"2024-09-11T23:21:50.455518+0000","last_peered":"2024-09-11T23:21:50.455518+0000","last_clean":"2024-09-11T23:21:50.455518+0000","last_became_active":"2024-09-11T23:19:59.741484+0000","last_became_peered":"2024-09-11T23:19:59.741484+0000","last_unstale":"2024-09-11T23:21:50.455518+0000","last_undegraded":"2024-09-11T23:21:50.455518+0000","last_fullsized":"2024-09-11T23:21:50.455518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:52.998258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.453834+0000","last_change":"2024-09-11T23:19:59.736042+0000","last_active":"2024-09-11T23:21:50.453834+0000","last_peered":"2024-09-11T23:21:50.453834+0000","last_clean":"2024-09-11T23:21:50.453834+0000","last_became_active":"2024-09-11T23:19:59.735731+0000","last_became_peered":"2024-09-11T23:19:59.735731+0000","last_unstale":"2024-09-11T23:21:50.453834+0000","last_undegraded":"2024-09-11T23:21:50.453834+0000","last_fullsized":"2024-09-11T23:21:50.453834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:19.085804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210129+0000","last_change":"2024-09-11T23:19:59.742049+0000","last_active":"2024-09-11T23:21:50.210129+0000","last_peered":"2024-09-11T23:21:50.210129+0000","last_clean":"2024-09-11T23:21:50.210129+0000","last_became_active":"2024-09-11T23:19:59.740974+0000","last_became_peered":"2024-09-11T23:19:59.740974+0000","last_unstale":"2024-09-11T23:21:50.210129+0000","last_undegraded":"2024-09-11T23:21:50.210129+0000","last_fullsized":"2024-09-11T23:21:50.210129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:10.443113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.454569+0000","last_change":"2024-09-11T23:19:59.734270+0000","last_active":"2024-09-11T23:21:50.454569+0000","last_peered":"2024-09-11T23:21:50.454569+0000","last_clean":"2024-09-11T23:21:50.454569+0000","last_became_active":"2024-09-11T23:19:59.733967+0000","last_became_peered":"2024-09-11T23:19:59.733967+0000","last_unstale":"2024-09-11T23:21:50.454569+0000","last_undegraded":"2024-09-11T23:21:50.454569+0000","last_fullsized":"2024-09-11T23:21:50.454569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:02.185757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.454215+0000","last_change":"2024-09-11T23:19:59.734152+0000","last_active":"2024-09-11T23:21:50.454215+0000","last_peered":"2024-09-11T23:21:50.454215+0000","last_clean":"2024-09-11T23:21:50.454215+0000","last_became_active":"2024-09-11T23:19:59.733163+0000","last_became_peered":"2024-09-11T23:19:59.733163+0000","last_unstale":"2024-09-11T23:21:50.454215+0000","last_undegraded":"2024-09-11T23:21:50.454215+0000","last_fullsized":"2024-09-11T23:21:50.454215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:22.370591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211142+0000","last_change":"2024-09-11T23:19:59.735517+0000","last_active":"2024-09-11T23:21:50.211142+0000","last_peered":"2024-09-11T23:21:50.211142+0000","last_clean":"2024-09-11T23:21:50.211142+0000","last_became_active":"2024-09-11T23:19:59.735182+0000","last_became_peered":"2024-09-11T23:19:59.735182+0000","last_unstale":"2024-09-11T23:21:50.211142+0000","last_undegraded":"2024-09-11T23:21:50.211142+0000","last_fullsized":"2024-09-11T23:21:50.211142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:25.989416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598585+0000","last_change":"2024-09-11T23:19:59.733579+0000","last_active":"2024-09-11T23:21:50.598585+0000","last_peered":"2024-09-11T23:21:50.598585+0000","last_clean":"2024-09-11T23:21:50.598585+0000","last_became_active":"2024-09-11T23:19:59.733245+0000","last_became_peered":"2024-09-11T23:19:59.733245+0000","last_unstale":"2024-09-11T23:21:50.598585+0000","last_undegraded":"2024-09-11T23:21:50.598585+0000","last_fullsized":"2024-09-11T23:21:50.598585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:22.361158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297955+0000","last_change":"2024-09-11T23:19:59.734354+0000","last_active":"2024-09-11T23:21:50.297955+0000","last_peered":"2024-09-11T23:21:50.297955+0000","last_clean":"2024-09-11T23:21:50.297955+0000","last_became_active":"2024-09-11T23:19:59.733988+0000","last_became_peered":"2024-09-11T23:19:59.733988+0000","last_unstale":"2024-09-11T23:21:50.297955+0000","last_undegraded":"2024-09-11T23:21:50.297955+0000","last_fullsized":"2024-09-11T23:21:50.297955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:41.401496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.212032+0000","last_change":"2024-09-11T23:19:59.730318+0000","last_active":"2024-09-11T23:21:50.212032+0000","last_peered":"2024-09-11T23:21:50.212032+0000","last_clean":"2024-09-11T23:21:50.212032+0000","last_became_active":"2024-09-11T23:19:59.730009+0000","last_became_peered":"2024-09-11T23:19:59.730009+0000","last_unstale":"2024-09-11T23:21:50.212032+0000","last_undegraded":"2024-09-11T23:21:50.212032+0000","last_fullsized":"2024-09-11T23:21:50.212032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:01:38.531967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598328+0000","last_change":"2024-09-11T23:19:59.737254+0000","last_active":"2024-09-11T23:21:50.598328+0000","last_peered":"2024-09-11T23:21:50.598328+0000","last_clean":"2024-09-11T23:21:50.598328+0000","last_became_active":"2024-09-11T23:19:59.736936+0000","last_became_peered":"2024-09-11T23:19:59.736936+0000","last_unstale":"2024-09-11T23:21:50.598328+0000","last_undegraded":"2024-09-11T23:21:50.598328+0000","last_fullsized":"2024-09-11T23:21:50.598328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:00.361468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210147+0000","last_change":"2024-09-11T23:19:59.741617+0000","last_active":"2024-09-11T23:21:50.210147+0000","last_peered":"2024-09-11T23:21:50.210147+0000","last_clean":"2024-09-11T23:21:50.210147+0000","last_became_active":"2024-09-11T23:19:59.740358+0000","last_became_peered":"2024-09-11T23:19:59.740358+0000","last_unstale":"2024-09-11T23:21:50.210147+0000","last_undegraded":"2024-09-11T23:21:50.210147+0000","last_fullsized":"2024-09-11T23:21:50.210147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:18.541105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211292+0000","last_change":"2024-09-11T23:19:59.736437+0000","last_active":"2024-09-11T23:21:50.211292+0000","last_peered":"2024-09-11T23:21:50.211292+0000","last_clean":"2024-09-11T23:21:50.211292+0000","last_became_active":"2024-09-11T23:19:59.736029+0000","last_became_peered":"2024-09-11T23:19:59.736029+0000","last_unstale":"2024-09-11T23:21:50.211292+0000","last_undegraded":"2024-09-11T23:21:50.211292+0000","last_fullsized":"2024-09-11T23:21:50.211292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:29.226939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300441+0000","last_change":"2024-09-11T23:19:59.736104+0000","last_active":"2024-09-11T23:21:50.300441+0000","last_peered":"2024-09-11T23:21:50.300441+0000","last_clean":"2024-09-11T23:21:50.300441+0000","last_became_active":"2024-09-11T23:19:59.735760+0000","last_became_peered":"2024-09-11T23:19:59.735760+0000","last_unstale":"2024-09-11T23:21:50.300441+0000","last_undegraded":"2024-09-11T23:21:50.300441+0000","last_fullsized":"2024-09-11T23:21:50.300441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:56.554313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308481+0000","last_change":"2024-09-11T23:19:59.735455+0000","last_active":"2024-09-11T23:21:50.308481+0000","last_peered":"2024-09-11T23:21:50.308481+0000","last_clean":"2024-09-11T23:21:50.308481+0000","last_became_active":"2024-09-11T23:19:59.735078+0000","last_became_peered":"2024-09-11T23:19:59.735078+0000","last_unstale":"2024-09-11T23:21:50.308481+0000","last_undegraded":"2024-09-11T23:21:50.308481+0000","last_fullsized":"2024-09-11T23:21:50.308481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:21:16.336006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.301909+0000","last_change":"2024-09-11T23:19:59.745997+0000","last_active":"2024-09-11T23:21:50.301909+0000","last_peered":"2024-09-11T23:21:50.301909+0000","last_clean":"2024-09-11T23:21:50.301909+0000","last_became_active":"2024-09-11T23:19:59.745715+0000","last_became_peered":"2024-09-11T23:19:59.745715+0000","last_unstale":"2024-09-11T23:21:50.301909+0000","last_undegraded":"2024-09-11T23:21:50.301909+0000","last_fullsized":"2024-09-11T23:21:50.301909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:34.776229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.212119+0000","last_change":"2024-09-11T23:19:59.734181+0000","last_active":"2024-09-11T23:21:50.212119+0000","last_peered":"2024-09-11T23:21:50.212119+0000","last_clean":"2024-09-11T23:21:50.212119+0000","last_became_active":"2024-09-11T23:19:59.733793+0000","last_became_peered":"2024-09-11T23:19:59.733793+0000","last_unstale":"2024-09-11T23:21:50.212119+0000","last_undegraded":"2024-09-11T23:21:50.212119+0000","last_fullsized":"2024-09-11T23:21:50.212119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:36.906308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598526+0000","last_change":"2024-09-11T23:19:59.738713+0000","last_active":"2024-09-11T23:21:50.598526+0000","last_peered":"2024-09-11T23:21:50.598526+0000","last_clean":"2024-09-11T23:21:50.598526+0000","last_became_active":"2024-09-11T23:19:59.737579+0000","last_became_peered":"2024-09-11T23:19:59.737579+0000","last_unstale":"2024-09-11T23:21:50.598526+0000","last_undegraded":"2024-09-11T23:21:50.598526+0000","last_fullsized":"2024-09-11T23:21:50.598526+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:47.135775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210205+0000","last_change":"2024-09-11T23:19:59.741381+0000","last_active":"2024-09-11T23:21:50.210205+0000","last_peered":"2024-09-11T23:21:50.210205+0000","last_clean":"2024-09-11T23:21:50.210205+0000","last_became_active":"2024-09-11T23:19:59.740914+0000","last_became_peered":"2024-09-11T23:19:59.740914+0000","last_unstale":"2024-09-11T23:21:50.210205+0000","last_undegraded":"2024-09-11T23:21:50.210205+0000","last_fullsized":"2024-09-11T23:21:50.210205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:24:04.365704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455133+0000","last_change":"2024-09-11T23:19:59.736840+0000","last_active":"2024-09-11T23:21:50.455133+0000","last_peered":"2024-09-11T23:21:50.455133+0000","last_clean":"2024-09-11T23:21:50.455133+0000","last_became_active":"2024-09-11T23:19:59.736554+0000","last_became_peered":"2024-09-11T23:19:59.736554+0000","last_unstale":"2024-09-11T23:21:50.455133+0000","last_undegraded":"2024-09-11T23:21:50.455133+0000","last_fullsized":"2024-09-11T23:21:50.455133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:48.508369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211738+0000","last_change":"2024-09-11T23:19:59.735719+0000","last_active":"2024-09-11T23:21:50.211738+0000","last_peered":"2024-09-11T23:21:50.211738+0000","last_clean":"2024-09-11T23:21:50.211738+0000","last_became_active":"2024-09-11T23:19:59.735397+0000","last_became_peered":"2024-09-11T23:19:59.735397+0000","last_unstale":"2024-09-11T23:21:50.211738+0000","last_undegraded":"2024-09-11T23:21:50.211738+0000","last_fullsized":"2024-09-11T23:21:50.211738+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:26:19.392730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211747+0000","last_change":"2024-09-11T23:19:59.737704+0000","last_active":"2024-09-11T23:21:50.211747+0000","last_peered":"2024-09-11T23:21:50.211747+0000","last_clean":"2024-09-11T23:21:50.211747+0000","last_became_active":"2024-09-11T23:19:59.737176+0000","last_became_peered":"2024-09-11T23:19:59.737176+0000","last_unstale":"2024-09-11T23:21:50.211747+0000","last_undegraded":"2024-09-11T23:21:50.211747+0000","last_fullsized":"2024-09-11T23:21:50.211747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:22.938171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598248+0000","last_change":"2024-09-11T23:19:59.735499+0000","last_active":"2024-09-11T23:21:50.598248+0000","last_peered":"2024-09-11T23:21:50.598248+0000","last_clean":"2024-09-11T23:21:50.598248+0000","last_became_active":"2024-09-11T23:19:59.735088+0000","last_became_peered":"2024-09-11T23:19:59.735088+0000","last_unstale":"2024-09-11T23:21:50.598248+0000","last_undegraded":"2024-09-11T23:21:50.598248+0000","last_fullsized":"2024-09-11T23:21:50.598248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:12.633104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299549+0000","last_change":"2024-09-11T23:19:59.743972+0000","last_active":"2024-09-11T23:21:50.299549+0000","last_peered":"2024-09-11T23:21:50.299549+0000","last_clean":"2024-09-11T23:21:50.299549+0000","last_became_active":"2024-09-11T23:19:59.743675+0000","last_became_peered":"2024-09-11T23:19:59.743675+0000","last_unstale":"2024-09-11T23:21:50.299549+0000","last_undegraded":"2024-09-11T23:21:50.299549+0000","last_fullsized":"2024-09-11T23:21:50.299549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:46.645696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":134,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.866819+0000","last_change":"2024-09-11T23:19:57.615472+0000","last_active":"2024-09-11T23:21:50.866819+0000","last_peered":"2024-09-11T23:21:50.866819+0000","last_clean":"2024-09-11T23:21:50.866819+0000","last_became_active":"2024-09-11T23:19:57.615104+0000","last_became_peered":"2024-09-11T23:19:57.615104+0000","last_unstale":"2024-09-11T23:21:50.866819+0000","last_undegraded":"2024-09-11T23:21:50.866819+0000","last_fullsized":"2024-09-11T23:21:50.866819+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:03.601363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308144+0000","last_change":"2024-09-11T23:19:59.741027+0000","last_active":"2024-09-11T23:21:50.308144+0000","last_peered":"2024-09-11T23:21:50.308144+0000","last_clean":"2024-09-11T23:21:50.308144+0000","last_became_active":"2024-09-11T23:19:59.740681+0000","last_became_peered":"2024-09-11T23:19:59.740681+0000","last_unstale":"2024-09-11T23:21:50.308144+0000","last_undegraded":"2024-09-11T23:21:50.308144+0000","last_fullsized":"2024-09-11T23:21:50.308144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:36.948147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311025+0000","last_change":"2024-09-11T23:19:57.612730+0000","last_active":"2024-09-11T23:21:50.311025+0000","last_peered":"2024-09-11T23:21:50.311025+0000","last_clean":"2024-09-11T23:21:50.311025+0000","last_became_active":"2024-09-11T23:19:57.612369+0000","last_became_peered":"2024-09-11T23:19:57.612369+0000","last_unstale":"2024-09-11T23:21:50.311025+0000","last_undegraded":"2024-09-11T23:21:50.311025+0000","last_fullsized":"2024-09-11T23:21:50.311025+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:17.823616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.209881+0000","last_change":"2024-09-11T23:19:57.611470+0000","last_active":"2024-09-11T23:21:50.209881+0000","last_peered":"2024-09-11T23:21:50.209881+0000","last_clean":"2024-09-11T23:21:50.209881+0000","last_became_active":"2024-09-11T23:19:57.611141+0000","last_became_peered":"2024-09-11T23:19:57.611141+0000","last_unstale":"2024-09-11T23:21:50.209881+0000","last_undegraded":"2024-09-11T23:21:50.209881+0000","last_fullsized":"2024-09-11T23:21:50.209881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:43.173955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300408+0000","last_change":"2024-09-11T23:19:59.732375+0000","last_active":"2024-09-11T23:21:50.300408+0000","last_peered":"2024-09-11T23:21:50.300408+0000","last_clean":"2024-09-11T23:21:50.300408+0000","last_became_active":"2024-09-11T23:19:59.731793+0000","last_became_peered":"2024-09-11T23:19:59.731793+0000","last_unstale":"2024-09-11T23:21:50.300408+0000","last_undegraded":"2024-09-11T23:21:50.300408+0000","last_fullsized":"2024-09-11T23:21:50.300408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:02:47.278646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308399+0000","last_change":"2024-09-11T23:19:57.614062+0000","last_active":"2024-09-11T23:21:50.308399+0000","last_peered":"2024-09-11T23:21:50.308399+0000","last_clean":"2024-09-11T23:21:50.308399+0000","last_became_active":"2024-09-11T23:19:57.613749+0000","last_became_peered":"2024-09-11T23:19:57.613749+0000","last_unstale":"2024-09-11T23:21:50.308399+0000","last_undegraded":"2024-09-11T23:21:50.308399+0000","last_fullsized":"2024-09-11T23:21:50.308399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:45:22.299379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297086+0000","last_change":"2024-09-11T23:19:59.730758+0000","last_active":"2024-09-11T23:21:50.297086+0000","last_peered":"2024-09-11T23:21:50.297086+0000","last_clean":"2024-09-11T23:21:50.297086+0000","last_became_active":"2024-09-11T23:19:59.730423+0000","last_became_peered":"2024-09-11T23:19:59.730423+0000","last_unstale":"2024-09-11T23:21:50.297086+0000","last_undegraded":"2024-09-11T23:21:50.297086+0000","last_fullsized":"2024-09-11T23:21:50.297086+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:30.679451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":129,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.459375+0000","last_change":"2024-09-11T23:19:57.615574+0000","last_active":"2024-09-11T23:21:50.459375+0000","last_peered":"2024-09-11T23:21:50.459375+0000","last_clean":"2024-09-11T23:21:50.459375+0000","last_became_active":"2024-09-11T23:19:57.615192+0000","last_became_peered":"2024-09-11T23:19:57.615192+0000","last_unstale":"2024-09-11T23:21:50.459375+0000","last_undegraded":"2024-09-11T23:21:50.459375+0000","last_fullsized":"2024-09-11T23:21:50.459375+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:29.186899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211749+0000","last_change":"2024-09-11T23:19:59.738433+0000","last_active":"2024-09-11T23:21:50.211749+0000","last_peered":"2024-09-11T23:21:50.211749+0000","last_clean":"2024-09-11T23:21:50.211749+0000","last_became_active":"2024-09-11T23:19:59.738081+0000","last_became_peered":"2024-09-11T23:19:59.738081+0000","last_unstale":"2024-09-11T23:21:50.211749+0000","last_undegraded":"2024-09-11T23:21:50.211749+0000","last_fullsized":"2024-09-11T23:21:50.211749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:10:46.786454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455225+0000","last_change":"2024-09-11T23:21:02.437498+0000","last_active":"2024-09-11T23:21:50.455225+0000","last_peered":"2024-09-11T23:21:50.455225+0000","last_clean":"2024-09-11T23:21:50.455225+0000","last_became_active":"2024-09-11T23:21:02.434895+0000","last_became_peered":"2024-09-11T23:21:02.434895+0000","last_unstale":"2024-09-11T23:21:50.455225+0000","last_undegraded":"2024-09-11T23:21:50.455225+0000","last_fullsized":"2024-09-11T23:21:50.455225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:33.142041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455584+0000","last_change":"2024-09-11T23:19:59.741081+0000","last_active":"2024-09-11T23:21:50.455584+0000","last_peered":"2024-09-11T23:21:50.455584+0000","last_clean":"2024-09-11T23:21:50.455584+0000","last_became_active":"2024-09-11T23:19:59.739984+0000","last_became_peered":"2024-09-11T23:19:59.739984+0000","last_unstale":"2024-09-11T23:21:50.455584+0000","last_undegraded":"2024-09-11T23:21:50.455584+0000","last_fullsized":"2024-09-11T23:21:50.455584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:03.513121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308186+0000","last_change":"2024-09-11T23:19:57.823272+0000","last_active":"2024-09-11T23:21:50.308186+0000","last_peered":"2024-09-11T23:21:50.308186+0000","last_clean":"2024-09-11T23:21:50.308186+0000","last_became_active":"2024-09-11T23:19:57.822957+0000","last_became_peered":"2024-09-11T23:19:57.822957+0000","last_unstale":"2024-09-11T23:21:50.308186+0000","last_undegraded":"2024-09-11T23:21:50.308186+0000","last_fullsized":"2024-09-11T23:21:50.308186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:06.779385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211510+0000","last_change":"2024-09-11T23:19:59.738398+0000","last_active":"2024-09-11T23:21:50.211510+0000","last_peered":"2024-09-11T23:21:50.211510+0000","last_clean":"2024-09-11T23:21:50.211510+0000","last_became_active":"2024-09-11T23:19:59.738042+0000","last_became_peered":"2024-09-11T23:19:59.738042+0000","last_unstale":"2024-09-11T23:21:50.211510+0000","last_undegraded":"2024-09-11T23:21:50.211510+0000","last_fullsized":"2024-09-11T23:21:50.211510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:05.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'34","reported_seq":271,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.867174+0000","last_change":"2024-09-11T23:19:57.615036+0000","last_active":"2024-09-11T23:21:50.867174+0000","last_peered":"2024-09-11T23:21:50.867174+0000","last_clean":"2024-09-11T23:21:50.867174+0000","last_became_active":"2024-09-11T23:19:57.614641+0000","last_became_peered":"2024-09-11T23:19:57.614641+0000","last_unstale":"2024-09-11T23:21:50.867174+0000","last_undegraded":"2024-09-11T23:21:50.867174+0000","last_fullsized":"2024-09-11T23:21:50.867174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:05.430508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67065,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":213,"num_read_kb":172,"num_write":35,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299857+0000","last_change":"2024-09-11T23:19:59.741597+0000","last_active":"2024-09-11T23:21:50.299857+0000","last_peered":"2024-09-11T23:21:50.299857+0000","last_clean":"2024-09-11T23:21:50.299857+0000","last_became_active":"2024-09-11T23:19:59.741296+0000","last_became_peered":"2024-09-11T23:19:59.741296+0000","last_unstale":"2024-09-11T23:21:50.299857+0000","last_undegraded":"2024-09-11T23:21:50.299857+0000","last_fullsized":"2024-09-11T23:21:50.299857+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:06:20.742916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211837+0000","last_change":"2024-09-11T23:19:57.617550+0000","last_active":"2024-09-11T23:21:50.211837+0000","last_peered":"2024-09-11T23:21:50.211837+0000","last_clean":"2024-09-11T23:21:50.211837+0000","last_became_active":"2024-09-11T23:19:57.617211+0000","last_became_peered":"2024-09-11T23:19:57.617211+0000","last_unstale":"2024-09-11T23:21:50.211837+0000","last_undegraded":"2024-09-11T23:21:50.211837+0000","last_fullsized":"2024-09-11T23:21:50.211837+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:19.707742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210301+0000","last_change":"2024-09-11T23:19:59.742070+0000","last_active":"2024-09-11T23:21:50.210301+0000","last_peered":"2024-09-11T23:21:50.210301+0000","last_clean":"2024-09-11T23:21:50.210301+0000","last_became_active":"2024-09-11T23:19:59.739367+0000","last_became_peered":"2024-09-11T23:19:59.739367+0000","last_unstale":"2024-09-11T23:21:50.210301+0000","last_undegraded":"2024-09-11T23:21:50.210301+0000","last_fullsized":"2024-09-11T23:21:50.210301+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:27.850136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308168+0000","last_change":"2024-09-11T23:19:57.618054+0000","last_active":"2024-09-11T23:21:50.308168+0000","last_peered":"2024-09-11T23:21:50.308168+0000","last_clean":"2024-09-11T23:21:50.308168+0000","last_became_active":"2024-09-11T23:19:57.617702+0000","last_became_peered":"2024-09-11T23:19:57.617702+0000","last_unstale":"2024-09-11T23:21:50.308168+0000","last_undegraded":"2024-09-11T23:21:50.308168+0000","last_fullsized":"2024-09-11T23:21:50.308168+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:40.171851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455548+0000","last_change":"2024-09-11T23:19:59.744000+0000","last_active":"2024-09-11T23:21:50.455548+0000","last_peered":"2024-09-11T23:21:50.455548+0000","last_clean":"2024-09-11T23:21:50.455548+0000","last_became_active":"2024-09-11T23:19:59.743629+0000","last_became_peered":"2024-09-11T23:19:59.743629+0000","last_unstale":"2024-09-11T23:21:50.455548+0000","last_undegraded":"2024-09-11T23:21:50.455548+0000","last_fullsized":"2024-09-11T23:21:50.455548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:20:04.133258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'7","reported_seq":137,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.867227+0000","last_change":"2024-09-11T23:19:57.615117+0000","last_active":"2024-09-11T23:21:50.867227+0000","last_peered":"2024-09-11T23:21:50.867227+0000","last_clean":"2024-09-11T23:21:50.867227+0000","last_became_active":"2024-09-11T23:19:57.614758+0000","last_became_peered":"2024-09-11T23:19:57.614758+0000","last_unstale":"2024-09-11T23:21:50.867227+0000","last_undegraded":"2024-09-11T23:21:50.867227+0000","last_fullsized":"2024-09-11T23:21:50.867227+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:58.113986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":6,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455281+0000","last_change":"2024-09-11T23:19:59.739403+0000","last_active":"2024-09-11T23:21:50.455281+0000","last_peered":"2024-09-11T23:21:50.455281+0000","last_clean":"2024-09-11T23:21:50.455281+0000","last_became_active":"2024-09-11T23:19:59.739078+0000","last_became_peered":"2024-09-11T23:19:59.739078+0000","last_unstale":"2024-09-11T23:21:50.455281+0000","last_undegraded":"2024-09-11T23:21:50.455281+0000","last_fullsized":"2024-09-11T23:21:50.455281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:46.168720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299488+0000","last_change":"2024-09-11T23:19:57.823323+0000","last_active":"2024-09-11T23:21:50.299488+0000","last_peered":"2024-09-11T23:21:50.299488+0000","last_clean":"2024-09-11T23:21:50.299488+0000","last_became_active":"2024-09-11T23:19:57.822947+0000","last_became_peered":"2024-09-11T23:19:57.822947+0000","last_unstale":"2024-09-11T23:21:50.299488+0000","last_undegraded":"2024-09-11T23:21:50.299488+0000","last_fullsized":"2024-09-11T23:21:50.299488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:12.542911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308091+0000","last_change":"2024-09-11T23:19:59.741163+0000","last_active":"2024-09-11T23:21:50.308091+0000","last_peered":"2024-09-11T23:21:50.308091+0000","last_clean":"2024-09-11T23:21:50.308091+0000","last_became_active":"2024-09-11T23:19:59.740633+0000","last_became_peered":"2024-09-11T23:19:59.740633+0000","last_unstale":"2024-09-11T23:21:50.308091+0000","last_undegraded":"2024-09-11T23:21:50.308091+0000","last_fullsized":"2024-09-11T23:21:50.308091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:26.301775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299941+0000","last_change":"2024-09-11T23:19:57.615804+0000","last_active":"2024-09-11T23:21:50.299941+0000","last_peered":"2024-09-11T23:21:50.299941+0000","last_clean":"2024-09-11T23:21:50.299941+0000","last_became_active":"2024-09-11T23:19:57.615443+0000","last_became_peered":"2024-09-11T23:19:57.615443+0000","last_unstale":"2024-09-11T23:21:50.299941+0000","last_undegraded":"2024-09-11T23:21:50.299941+0000","last_fullsized":"2024-09-11T23:21:50.299941+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:39.296873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598177+0000","last_change":"2024-09-11T23:21:02.360875+0000","last_active":"2024-09-11T23:21:50.598177+0000","last_peered":"2024-09-11T23:21:50.598177+0000","last_clean":"2024-09-11T23:21:50.598177+0000","last_became_active":"2024-09-11T23:21:02.360480+0000","last_became_peered":"2024-09-11T23:21:02.360480+0000","last_unstale":"2024-09-11T23:21:50.598177+0000","last_undegraded":"2024-09-11T23:21:50.598177+0000","last_fullsized":"2024-09-11T23:21:50.598177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:04.141587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210448+0000","last_change":"2024-09-11T23:19:57.612873+0000","last_active":"2024-09-11T23:21:50.210448+0000","last_peered":"2024-09-11T23:21:50.210448+0000","last_clean":"2024-09-11T23:21:50.210448+0000","last_became_active":"2024-09-11T23:19:57.612566+0000","last_became_peered":"2024-09-11T23:19:57.612566+0000","last_unstale":"2024-09-11T23:21:50.210448+0000","last_undegraded":"2024-09-11T23:21:50.210448+0000","last_fullsized":"2024-09-11T23:21:50.210448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:46.218212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455770+0000","last_change":"2024-09-11T23:19:59.736352+0000","last_active":"2024-09-11T23:21:50.455770+0000","last_peered":"2024-09-11T23:21:50.455770+0000","last_clean":"2024-09-11T23:21:50.455770+0000","last_became_active":"2024-09-11T23:19:59.735402+0000","last_became_peered":"2024-09-11T23:19:59.735402+0000","last_unstale":"2024-09-11T23:21:50.455770+0000","last_undegraded":"2024-09-11T23:21:50.455770+0000","last_fullsized":"2024-09-11T23:21:50.455770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:41.854526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'39","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.412363+0000","last_change":"2024-09-11T23:19:57.612439+0000","last_active":"2024-09-11T23:21:50.412363+0000","last_peered":"2024-09-11T23:21:50.412363+0000","last_clean":"2024-09-11T23:21:50.412363+0000","last_became_active":"2024-09-11T23:19:57.612030+0000","last_became_peered":"2024-09-11T23:19:57.612030+0000","last_unstale":"2024-09-11T23:21:50.412363+0000","last_undegraded":"2024-09-11T23:21:50.412363+0000","last_fullsized":"2024-09-11T23:21:50.412363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:36:24.579246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455480+0000","last_change":"2024-09-11T23:19:59.741241+0000","last_active":"2024-09-11T23:21:50.455480+0000","last_peered":"2024-09-11T23:21:50.455480+0000","last_clean":"2024-09-11T23:21:50.455480+0000","last_became_active":"2024-09-11T23:19:59.740265+0000","last_became_peered":"2024-09-11T23:19:59.740265+0000","last_unstale":"2024-09-11T23:21:50.455480+0000","last_undegraded":"2024-09-11T23:21:50.455480+0000","last_fullsized":"2024-09-11T23:21:50.455480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:00.676762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211975+0000","last_change":"2024-09-11T23:19:57.825238+0000","last_active":"2024-09-11T23:21:50.211975+0000","last_peered":"2024-09-11T23:21:50.211975+0000","last_clean":"2024-09-11T23:21:50.211975+0000","last_became_active":"2024-09-11T23:19:57.824732+0000","last_became_peered":"2024-09-11T23:19:57.824732+0000","last_unstale":"2024-09-11T23:21:50.211975+0000","last_undegraded":"2024-09-11T23:21:50.211975+0000","last_fullsized":"2024-09-11T23:21:50.211975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:24.465863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455634+0000","last_change":"2024-09-11T23:19:59.741338+0000","last_active":"2024-09-11T23:21:50.455634+0000","last_peered":"2024-09-11T23:21:50.455634+0000","last_clean":"2024-09-11T23:21:50.455634+0000","last_became_active":"2024-09-11T23:19:59.741019+0000","last_became_peered":"2024-09-11T23:19:59.741019+0000","last_unstale":"2024-09-11T23:21:50.455634+0000","last_undegraded":"2024-09-11T23:21:50.455634+0000","last_fullsized":"2024-09-11T23:21:50.455634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:36:03.480006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":351,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455427+0000","last_change":"2024-09-11T23:18:45.606646+0000","last_active":"2024-09-11T23:21:50.455427+0000","last_peered":"2024-09-11T23:21:50.455427+0000","last_clean":"2024-09-11T23:21:50.455427+0000","last_became_active":"2024-09-11T23:18:45.606065+0000","last_became_peered":"2024-09-11T23:18:45.606065+0000","last_unstale":"2024-09-11T23:21:50.455427+0000","last_undegraded":"2024-09-11T23:21:50.455427+0000","last_fullsized":"2024-09-11T23:21:50.455427+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:15:17.148562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:15:17.148562+0000","last_clean_scrub_stamp":"2024-09-11T23:15:17.148562+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:07.992971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.453802+0000","last_change":"2024-09-11T23:19:57.612584+0000","last_active":"2024-09-11T23:21:50.453802+0000","last_peered":"2024-09-11T23:21:50.453802+0000","last_clean":"2024-09-11T23:21:50.453802+0000","last_became_active":"2024-09-11T23:19:57.612211+0000","last_became_peered":"2024-09-11T23:19:57.612211+0000","last_unstale":"2024-09-11T23:21:50.453802+0000","last_undegraded":"2024-09-11T23:21:50.453802+0000","last_fullsized":"2024-09-11T23:21:50.453802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:04.026233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300938+0000","last_change":"2024-09-11T23:21:02.359249+0000","last_active":"2024-09-11T23:21:50.300938+0000","last_peered":"2024-09-11T23:21:50.300938+0000","last_clean":"2024-09-11T23:21:50.300938+0000","last_became_active":"2024-09-11T23:21:02.358906+0000","last_became_peered":"2024-09-11T23:21:02.358906+0000","last_unstale":"2024-09-11T23:21:50.300938+0000","last_undegraded":"2024-09-11T23:21:50.300938+0000","last_fullsized":"2024-09-11T23:21:50.300938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:05.973581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455432+0000","last_change":"2024-09-11T23:19:57.618283+0000","last_active":"2024-09-11T23:21:50.455432+0000","last_peered":"2024-09-11T23:21:50.455432+0000","last_clean":"2024-09-11T23:21:50.455432+0000","last_became_active":"2024-09-11T23:19:57.617942+0000","last_became_peered":"2024-09-11T23:19:57.617942+0000","last_unstale":"2024-09-11T23:21:50.455432+0000","last_undegraded":"2024-09-11T23:21:50.455432+0000","last_fullsized":"2024-09-11T23:21:50.455432+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:30.832602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.209864+0000","last_change":"2024-09-11T23:19:59.740730+0000","last_active":"2024-09-11T23:21:50.209864+0000","last_peered":"2024-09-11T23:21:50.209864+0000","last_clean":"2024-09-11T23:21:50.209864+0000","last_became_active":"2024-09-11T23:19:59.740369+0000","last_became_peered":"2024-09-11T23:19:59.740369+0000","last_unstale":"2024-09-11T23:21:50.209864+0000","last_undegraded":"2024-09-11T23:21:50.209864+0000","last_fullsized":"2024-09-11T23:21:50.209864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:50.264644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":131,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.454901+0000","last_change":"2024-09-11T23:19:57.823253+0000","last_active":"2024-09-11T23:21:50.454901+0000","last_peered":"2024-09-11T23:21:50.454901+0000","last_clean":"2024-09-11T23:21:50.454901+0000","last_became_active":"2024-09-11T23:19:57.822895+0000","last_became_peered":"2024-09-11T23:19:57.822895+0000","last_unstale":"2024-09-11T23:21:50.454901+0000","last_undegraded":"2024-09-11T23:21:50.454901+0000","last_fullsized":"2024-09-11T23:21:50.454901+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:22.998371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308142+0000","last_change":"2024-09-11T23:19:59.742248+0000","last_active":"2024-09-11T23:21:50.308142+0000","last_peered":"2024-09-11T23:21:50.308142+0000","last_clean":"2024-09-11T23:21:50.308142+0000","last_became_active":"2024-09-11T23:19:59.741810+0000","last_became_peered":"2024-09-11T23:19:59.741810+0000","last_unstale":"2024-09-11T23:21:50.308142+0000","last_undegraded":"2024-09-11T23:21:50.308142+0000","last_fullsized":"2024-09-11T23:21:50.308142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:19:52.720286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455504+0000","last_change":"2024-09-11T23:19:57.616244+0000","last_active":"2024-09-11T23:21:50.455504+0000","last_peered":"2024-09-11T23:21:50.455504+0000","last_clean":"2024-09-11T23:21:50.455504+0000","last_became_active":"2024-09-11T23:19:57.615868+0000","last_became_peered":"2024-09-11T23:19:57.615868+0000","last_unstale":"2024-09-11T23:21:50.455504+0000","last_undegraded":"2024-09-11T23:21:50.455504+0000","last_fullsized":"2024-09-11T23:21:50.455504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:42.510896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311069+0000","last_change":"2024-09-11T23:21:02.434265+0000","last_active":"2024-09-11T23:21:50.311069+0000","last_peered":"2024-09-11T23:21:50.311069+0000","last_clean":"2024-09-11T23:21:50.311069+0000","last_became_active":"2024-09-11T23:21:02.433840+0000","last_became_peered":"2024-09-11T23:21:02.433840+0000","last_unstale":"2024-09-11T23:21:50.311069+0000","last_undegraded":"2024-09-11T23:21:50.311069+0000","last_fullsized":"2024-09-11T23:21:50.311069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:04.619695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297618+0000","last_change":"2024-09-11T23:19:57.824334+0000","last_active":"2024-09-11T23:21:50.297618+0000","last_peered":"2024-09-11T23:21:50.297618+0000","last_clean":"2024-09-11T23:21:50.297618+0000","last_became_active":"2024-09-11T23:19:57.823972+0000","last_became_peered":"2024-09-11T23:19:57.823972+0000","last_unstale":"2024-09-11T23:21:50.297618+0000","last_undegraded":"2024-09-11T23:21:50.297618+0000","last_fullsized":"2024-09-11T23:21:50.297618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:39.183281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311164+0000","last_change":"2024-09-11T23:19:59.728064+0000","last_active":"2024-09-11T23:21:50.311164+0000","last_peered":"2024-09-11T23:21:50.311164+0000","last_clean":"2024-09-11T23:21:50.311164+0000","last_became_active":"2024-09-11T23:19:59.727732+0000","last_became_peered":"2024-09-11T23:19:59.727732+0000","last_unstale":"2024-09-11T23:21:50.311164+0000","last_undegraded":"2024-09-11T23:21:50.311164+0000","last_fullsized":"2024-09-11T23:21:50.311164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:15.383974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299855+0000","last_change":"2024-09-11T23:19:57.824362+0000","last_active":"2024-09-11T23:21:50.299855+0000","last_peered":"2024-09-11T23:21:50.299855+0000","last_clean":"2024-09-11T23:21:50.299855+0000","last_became_active":"2024-09-11T23:19:57.823753+0000","last_became_peered":"2024-09-11T23:19:57.823753+0000","last_unstale":"2024-09-11T23:21:50.299855+0000","last_undegraded":"2024-09-11T23:21:50.299855+0000","last_fullsized":"2024-09-11T23:21:50.299855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:56.357268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210323+0000","last_change":"2024-09-11T23:19:59.730623+0000","last_active":"2024-09-11T23:21:50.210323+0000","last_peered":"2024-09-11T23:21:50.210323+0000","last_clean":"2024-09-11T23:21:50.210323+0000","last_became_active":"2024-09-11T23:19:59.730307+0000","last_became_peered":"2024-09-11T23:19:59.730307+0000","last_unstale":"2024-09-11T23:21:50.210323+0000","last_undegraded":"2024-09-11T23:21:50.210323+0000","last_fullsized":"2024-09-11T23:21:50.210323+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:52.874884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'48","reported_seq":95,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.418264+0000","last_change":"2024-09-11T23:19:57.611730+0000","last_active":"2024-09-11T23:21:50.418264+0000","last_peered":"2024-09-11T23:21:50.418264+0000","last_clean":"2024-09-11T23:21:50.418264+0000","last_became_active":"2024-09-11T23:19:57.611394+0000","last_became_peered":"2024-09-11T23:19:57.611394+0000","last_unstale":"2024-09-11T23:21:50.418264+0000","last_undegraded":"2024-09-11T23:21:50.418264+0000","last_fullsized":"2024-09-11T23:21:50.418264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:15.518573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.302035+0000","last_change":"2024-09-11T23:19:59.741396+0000","last_active":"2024-09-11T23:21:50.302035+0000","last_peered":"2024-09-11T23:21:50.302035+0000","last_clean":"2024-09-11T23:21:50.302035+0000","last_became_active":"2024-09-11T23:19:59.741102+0000","last_became_peered":"2024-09-11T23:19:59.741102+0000","last_unstale":"2024-09-11T23:21:50.302035+0000","last_undegraded":"2024-09-11T23:21:50.302035+0000","last_fullsized":"2024-09-11T23:21:50.302035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:08.759512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.453918+0000","last_change":"2024-09-11T23:19:57.824812+0000","last_active":"2024-09-11T23:21:50.453918+0000","last_peered":"2024-09-11T23:21:50.453918+0000","last_clean":"2024-09-11T23:21:50.453918+0000","last_became_active":"2024-09-11T23:19:57.824454+0000","last_became_peered":"2024-09-11T23:19:57.824454+0000","last_unstale":"2024-09-11T23:21:50.453918+0000","last_undegraded":"2024-09-11T23:21:50.453918+0000","last_fullsized":"2024-09-11T23:21:50.453918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:25.275573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455034+0000","last_change":"2024-09-11T23:19:59.735857+0000","last_active":"2024-09-11T23:21:50.455034+0000","last_peered":"2024-09-11T23:21:50.455034+0000","last_clean":"2024-09-11T23:21:50.455034+0000","last_became_active":"2024-09-11T23:19:59.735541+0000","last_became_peered":"2024-09-11T23:19:59.735541+0000","last_unstale":"2024-09-11T23:21:50.455034+0000","last_undegraded":"2024-09-11T23:21:50.455034+0000","last_fullsized":"2024-09-11T23:21:50.455034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:21.600490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211231+0000","last_change":"2024-09-11T23:19:57.823468+0000","last_active":"2024-09-11T23:21:50.211231+0000","last_peered":"2024-09-11T23:21:50.211231+0000","last_clean":"2024-09-11T23:21:50.211231+0000","last_became_active":"2024-09-11T23:19:57.823154+0000","last_became_peered":"2024-09-11T23:19:57.823154+0000","last_unstale":"2024-09-11T23:21:50.211231+0000","last_undegraded":"2024-09-11T23:21:50.211231+0000","last_fullsized":"2024-09-11T23:21:50.211231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:28.770627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299509+0000","last_change":"2024-09-11T23:19:59.741263+0000","last_active":"2024-09-11T23:21:50.299509+0000","last_peered":"2024-09-11T23:21:50.299509+0000","last_clean":"2024-09-11T23:21:50.299509+0000","last_became_active":"2024-09-11T23:19:59.740938+0000","last_became_peered":"2024-09-11T23:19:59.740938+0000","last_unstale":"2024-09-11T23:21:50.299509+0000","last_undegraded":"2024-09-11T23:21:50.299509+0000","last_fullsized":"2024-09-11T23:21:50.299509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:54.768652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.301723+0000","last_change":"2024-09-11T23:19:57.612517+0000","last_active":"2024-09-11T23:21:50.301723+0000","last_peered":"2024-09-11T23:21:50.301723+0000","last_clean":"2024-09-11T23:21:50.301723+0000","last_became_active":"2024-09-11T23:19:57.612188+0000","last_became_peered":"2024-09-11T23:19:57.612188+0000","last_unstale":"2024-09-11T23:21:50.301723+0000","last_undegraded":"2024-09-11T23:21:50.301723+0000","last_fullsized":"2024-09-11T23:21:50.301723+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:52.896413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598315+0000","last_change":"2024-09-11T23:19:59.732520+0000","last_active":"2024-09-11T23:21:50.598315+0000","last_peered":"2024-09-11T23:21:50.598315+0000","last_clean":"2024-09-11T23:21:50.598315+0000","last_became_active":"2024-09-11T23:19:59.732173+0000","last_became_peered":"2024-09-11T23:19:59.732173+0000","last_unstale":"2024-09-11T23:21:50.598315+0000","last_undegraded":"2024-09-11T23:21:50.598315+0000","last_fullsized":"2024-09-11T23:21:50.598315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:04.626312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598177+0000","last_change":"2024-09-11T23:21:02.365533+0000","last_active":"2024-09-11T23:21:50.598177+0000","last_peered":"2024-09-11T23:21:50.598177+0000","last_clean":"2024-09-11T23:21:50.598177+0000","last_became_active":"2024-09-11T23:21:02.359345+0000","last_became_peered":"2024-09-11T23:21:02.359345+0000","last_unstale":"2024-09-11T23:21:50.598177+0000","last_undegraded":"2024-09-11T23:21:50.598177+0000","last_fullsized":"2024-09-11T23:21:50.598177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:27.195666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.301266+0000","last_change":"2024-09-11T23:19:59.734667+0000","last_active":"2024-09-11T23:21:50.301266+0000","last_peered":"2024-09-11T23:21:50.301266+0000","last_clean":"2024-09-11T23:21:50.301266+0000","last_became_active":"2024-09-11T23:19:59.733754+0000","last_became_peered":"2024-09-11T23:19:59.733754+0000","last_unstale":"2024-09-11T23:21:50.301266+0000","last_undegraded":"2024-09-11T23:21:50.301266+0000","last_fullsized":"2024-09-11T23:21:50.301266+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:57.392272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299623+0000","last_change":"2024-09-11T23:19:57.614579+0000","last_active":"2024-09-11T23:21:50.299623+0000","last_peered":"2024-09-11T23:21:50.299623+0000","last_clean":"2024-09-11T23:21:50.299623+0000","last_became_active":"2024-09-11T23:19:57.614227+0000","last_became_peered":"2024-09-11T23:19:57.614227+0000","last_unstale":"2024-09-11T23:21:50.299623+0000","last_undegraded":"2024-09-11T23:21:50.299623+0000","last_fullsized":"2024-09-11T23:21:50.299623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:47.952920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455414+0000","last_change":"2024-09-11T23:19:59.744219+0000","last_active":"2024-09-11T23:21:50.455414+0000","last_peered":"2024-09-11T23:21:50.455414+0000","last_clean":"2024-09-11T23:21:50.455414+0000","last_became_active":"2024-09-11T23:19:59.743914+0000","last_became_peered":"2024-09-11T23:19:59.743914+0000","last_unstale":"2024-09-11T23:21:50.455414+0000","last_undegraded":"2024-09-11T23:21:50.455414+0000","last_fullsized":"2024-09-11T23:21:50.455414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:52.149588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308372+0000","last_change":"2024-09-11T23:21:02.361647+0000","last_active":"2024-09-11T23:21:50.308372+0000","last_peered":"2024-09-11T23:21:50.308372+0000","last_clean":"2024-09-11T23:21:50.308372+0000","last_became_active":"2024-09-11T23:21:02.361319+0000","last_became_peered":"2024-09-11T23:21:02.361319+0000","last_unstale":"2024-09-11T23:21:50.308372+0000","last_undegraded":"2024-09-11T23:21:50.308372+0000","last_fullsized":"2024-09-11T23:21:50.308372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:45.125525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299796+0000","last_change":"2024-09-11T23:19:57.610389+0000","last_active":"2024-09-11T23:21:50.299796+0000","last_peered":"2024-09-11T23:21:50.299796+0000","last_clean":"2024-09-11T23:21:50.299796+0000","last_became_active":"2024-09-11T23:19:57.609911+0000","last_became_peered":"2024-09-11T23:19:57.609911+0000","last_unstale":"2024-09-11T23:21:50.299796+0000","last_undegraded":"2024-09-11T23:21:50.299796+0000","last_fullsized":"2024-09-11T23:21:50.299796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:23.949929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211496+0000","last_change":"2024-09-11T23:19:59.736867+0000","last_active":"2024-09-11T23:21:50.211496+0000","last_peered":"2024-09-11T23:21:50.211496+0000","last_clean":"2024-09-11T23:21:50.211496+0000","last_became_active":"2024-09-11T23:19:59.736341+0000","last_became_peered":"2024-09-11T23:19:59.736341+0000","last_unstale":"2024-09-11T23:21:50.211496+0000","last_undegraded":"2024-09-11T23:21:50.211496+0000","last_fullsized":"2024-09-11T23:21:50.211496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:57:51.015235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598177+0000","last_change":"2024-09-11T23:21:02.368959+0000","last_active":"2024-09-11T23:21:50.598177+0000","last_peered":"2024-09-11T23:21:50.598177+0000","last_clean":"2024-09-11T23:21:50.598177+0000","last_became_active":"2024-09-11T23:21:02.363263+0000","last_became_peered":"2024-09-11T23:21:02.363263+0000","last_unstale":"2024-09-11T23:21:50.598177+0000","last_undegraded":"2024-09-11T23:21:50.598177+0000","last_fullsized":"2024-09-11T23:21:50.598177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:29.827330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455355+0000","last_change":"2024-09-11T23:19:59.742049+0000","last_active":"2024-09-11T23:21:50.455355+0000","last_peered":"2024-09-11T23:21:50.455355+0000","last_clean":"2024-09-11T23:21:50.455355+0000","last_became_active":"2024-09-11T23:19:59.741763+0000","last_became_peered":"2024-09-11T23:19:59.741763+0000","last_unstale":"2024-09-11T23:21:50.455355+0000","last_undegraded":"2024-09-11T23:21:50.455355+0000","last_fullsized":"2024-09-11T23:21:50.455355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:48.698896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.302105+0000","last_change":"2024-09-11T23:19:57.822949+0000","last_active":"2024-09-11T23:21:50.302105+0000","last_peered":"2024-09-11T23:21:50.302105+0000","last_clean":"2024-09-11T23:21:50.302105+0000","last_became_active":"2024-09-11T23:19:57.822564+0000","last_became_peered":"2024-09-11T23:19:57.822564+0000","last_unstale":"2024-09-11T23:21:50.302105+0000","last_undegraded":"2024-09-11T23:21:50.302105+0000","last_fullsized":"2024-09-11T23:21:50.302105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:29.532109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297740+0000","last_change":"2024-09-11T23:21:02.363639+0000","last_active":"2024-09-11T23:21:50.297740+0000","last_peered":"2024-09-11T23:21:50.297740+0000","last_clean":"2024-09-11T23:21:50.297740+0000","last_became_active":"2024-09-11T23:21:02.363247+0000","last_became_peered":"2024-09-11T23:21:02.363247+0000","last_unstale":"2024-09-11T23:21:50.297740+0000","last_undegraded":"2024-09-11T23:21:50.297740+0000","last_fullsized":"2024-09-11T23:21:50.297740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:18:58.906939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297845+0000","last_change":"2024-09-11T23:19:57.613323+0000","last_active":"2024-09-11T23:21:50.297845+0000","last_peered":"2024-09-11T23:21:50.297845+0000","last_clean":"2024-09-11T23:21:50.297845+0000","last_became_active":"2024-09-11T23:19:57.612985+0000","last_became_peered":"2024-09-11T23:19:57.612985+0000","last_unstale":"2024-09-11T23:21:50.297845+0000","last_undegraded":"2024-09-11T23:21:50.297845+0000","last_fullsized":"2024-09-11T23:21:50.297845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:13:09.179547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597204,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":731,"num_read_kb":475,"num_write":291,"num_write_kb":1437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1791612,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108948,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514531,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":441815,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743463,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":1242266,"data_compressed":51171,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906990,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":962560,"data_stored":844252,"data_compressed":27845,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168627,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":844172,"data_compressed":27808,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397560,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":987136,"data_stored":1044208,"data_compressed":45015,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561085,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":450425,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790018,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":1174172,"data_compressed":51171,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":452182,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149772,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":975552,"data_compressed":34001,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":519026,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":995328,"data_stored":517283,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":442711,"data_compressed":5790,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133469,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":201563,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:21:55.535 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:21:55.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:55 smithi097 ceph-mon[35562]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:21:55.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:55 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:55.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:55 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:55.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:55 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:55.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:55 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:21:55.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:55 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:21:55.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:55 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:21:55.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:55 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:21:55.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:55 smithi097 ceph-mon[35562]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:21:55.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:55 smithi097 ceph-mon[35562]: mgrmap e24: x(active, since 5s), standbys: z, y 2024-09-11T23:21:55.851 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:56.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.372+0000 7fdd41d77640 1 -- 172.21.15.43:0/1155185540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd3c102570 msgr2=0x7fdd3c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:56.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.372+0000 7fdd41d77640 1 --2- 172.21.15.43:0/1155185540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd3c102570 0x7fdd3c1029b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009900 tx=0x7fdd3002a480 comp rx=0 tx=0).stop 2024-09-11T23:21:56.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.372+0000 7fdd41d77640 1 -- 172.21.15.43:0/1155185540 shutdown_connections 2024-09-11T23:21:56.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.372+0000 7fdd41d77640 1 --2- 172.21.15.43:0/1155185540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd3c102ef0 0x7fdd3c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:56.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.372+0000 7fdd41d77640 1 --2- 172.21.15.43:0/1155185540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd3c102570 0x7fdd3c1029b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:56.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.372+0000 7fdd41d77640 1 --2- 172.21.15.43:0/1155185540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3c100e20 0x7fdd3c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:56.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.372+0000 7fdd41d77640 1 -- 172.21.15.43:0/1155185540 >> 172.21.15.43:0/1155185540 conn(0x7fdd3c0fc7d0 msgr2=0x7fdd3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:56.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.373+0000 7fdd41d77640 1 -- 172.21.15.43:0/1155185540 shutdown_connections 2024-09-11T23:21:56.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.373+0000 7fdd41d77640 1 -- 172.21.15.43:0/1155185540 wait complete. 2024-09-11T23:21:56.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.373+0000 7fdd41d77640 1 Processor -- start 2024-09-11T23:21:56.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.373+0000 7fdd41d77640 1 -- start start 2024-09-11T23:21:56.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.374+0000 7fdd41d77640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd3c100e20 0x7fdd3c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:56.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.374+0000 7fdd41d77640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd3c102570 0x7fdd3c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:56.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.374+0000 7fdd41d77640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3c102ef0 0x7fdd3c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:56.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.374+0000 7fdd41d77640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c110fd0 con 0x7fdd3c102570 2024-09-11T23:21:56.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.374+0000 7fdd41d77640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c110e50 con 0x7fdd3c100e20 2024-09-11T23:21:56.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.374+0000 7fdd41d77640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c111150 con 0x7fdd3c102ef0 2024-09-11T23:21:56.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.374+0000 7fdd3affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd3c102570 0x7fdd3c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:56.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.374+0000 7fdd3affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd3c102570 0x7fdd3c198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45432/0 (socket says 172.21.15.43:45432) 2024-09-11T23:21:56.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.374+0000 7fdd3affd640 1 -- 172.21.15.43:0/121357734 learned_addr learned my addr 172.21.15.43:0/121357734 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:56.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.374+0000 7fdd3b7fe640 1 --2- 172.21.15.43:0/121357734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd3c100e20 0x7fdd3c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:56.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.374+0000 7fdd3bfff640 1 --2- 172.21.15.43:0/121357734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3c102ef0 0x7fdd3c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:56.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.374+0000 7fdd3affd640 1 -- 172.21.15.43:0/121357734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3c102ef0 msgr2=0x7fdd3c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:56.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.374+0000 7fdd3affd640 1 --2- 172.21.15.43:0/121357734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3c102ef0 0x7fdd3c19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:56.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.374+0000 7fdd3affd640 1 -- 172.21.15.43:0/121357734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd3c100e20 msgr2=0x7fdd3c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:56.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.374+0000 7fdd3affd640 1 --2- 172.21.15.43:0/121357734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd3c100e20 0x7fdd3c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:56.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.374+0000 7fdd3affd640 1 -- 172.21.15.43:0/121357734 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3c19d4c0 con 0x7fdd3c102570 2024-09-11T23:21:56.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.374+0000 7fdd3b7fe640 1 --2- 172.21.15.43:0/121357734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd3c100e20 0x7fdd3c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:21:56.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.374+0000 7fdd3affd640 1 --2- 172.21.15.43:0/121357734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd3c102570 0x7fdd3c198a50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdd3002aa70 tx=0x7fdd30003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.375+0000 7fdd38ff9640 1 -- 172.21.15.43:0/121357734 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3003f070 con 0x7fdd3c102570 2024-09-11T23:21:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.375+0000 7fdd41d77640 1 -- 172.21.15.43:0/121357734 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd3c19d750 con 0x7fdd3c102570 2024-09-11T23:21:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.375+0000 7fdd38ff9640 1 -- 172.21.15.43:0/121357734 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd30031c30 con 0x7fdd3c102570 2024-09-11T23:21:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.375+0000 7fdd38ff9640 1 -- 172.21.15.43:0/121357734 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd30035030 con 0x7fdd3c102570 2024-09-11T23:21:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.375+0000 7fdd41d77640 1 -- 172.21.15.43:0/121357734 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd3c1a5090 con 0x7fdd3c102570 2024-09-11T23:21:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.376+0000 7fdd41d77640 1 -- 172.21.15.43:0/121357734 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd3c101200 con 0x7fdd3c102570 2024-09-11T23:21:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.377+0000 7fdd38ff9640 1 -- 172.21.15.43:0/121357734 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fdd30042070 con 0x7fdd3c102570 2024-09-11T23:21:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.378+0000 7fdd38ff9640 1 --2- 172.21.15.43:0/121357734 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdd240a2e10 0x7fdd240a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.378+0000 7fdd3b7fe640 1 --2- 172.21.15.43:0/121357734 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdd240a2e10 0x7fdd240a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.378+0000 7fdd38ff9640 1 -- 172.21.15.43:0/121357734 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fdd300f04b0 con 0x7fdd3c102570 2024-09-11T23:21:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.378+0000 7fdd3b7fe640 1 --2- 172.21.15.43:0/121357734 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdd240a2e10 0x7fdd240a52d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c009b50 tx=0x7fdd2c005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.381+0000 7fdd38ff9640 1 -- 172.21.15.43:0/121357734 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdd300bcf50 con 0x7fdd3c102570 2024-09-11T23:21:56.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:56 smithi090 ceph-mon[35130]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:21:56.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:56 smithi090 ceph-mon[35130]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:21:56.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:56 smithi097 ceph-mon[35562]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:21:56.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:56 smithi097 ceph-mon[35562]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:21:56.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.596+0000 7fdd41d77640 1 -- 172.21.15.43:0/121357734 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd3c199870 con 0x7fdd3c102570 2024-09-11T23:21:56.597 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:56 smithi043 ceph-mon[30950]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:21:56.597 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:56 smithi043 ceph-mon[30950]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:21:56.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.597+0000 7fdd38ff9640 1 -- 172.21.15.43:0/121357734 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11896 (secure 0 0 0) 0x7fdd3c199870 con 0x7fdd3c102570 2024-09-11T23:21:56.597 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:21:56.597 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"btime":"2024-09-11T23:21:49:167551+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:21:49.117342+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:21:56.598 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 24 2024-09-11T23:21:56.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.599+0000 7fdd41d77640 1 -- 172.21.15.43:0/121357734 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdd240a2e10 msgr2=0x7fdd240a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:56.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.599+0000 7fdd41d77640 1 --2- 172.21.15.43:0/121357734 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdd240a2e10 0x7fdd240a52d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c009b50 tx=0x7fdd2c005f90 comp rx=0 tx=0).stop 2024-09-11T23:21:56.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.599+0000 7fdd41d77640 1 -- 172.21.15.43:0/121357734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd3c102570 msgr2=0x7fdd3c198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:56.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.599+0000 7fdd41d77640 1 --2- 172.21.15.43:0/121357734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd3c102570 0x7fdd3c198a50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdd3002aa70 tx=0x7fdd30003c10 comp rx=0 tx=0).stop 2024-09-11T23:21:56.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.599+0000 7fdd41d77640 1 -- 172.21.15.43:0/121357734 shutdown_connections 2024-09-11T23:21:56.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.599+0000 7fdd41d77640 1 --2- 172.21.15.43:0/121357734 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdd240a2e10 0x7fdd240a52d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:56.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.599+0000 7fdd41d77640 1 --2- 172.21.15.43:0/121357734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3c102ef0 0x7fdd3c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:56.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.599+0000 7fdd41d77640 1 --2- 172.21.15.43:0/121357734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd3c102570 0x7fdd3c198a50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:56.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.599+0000 7fdd41d77640 1 --2- 172.21.15.43:0/121357734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd3c100e20 0x7fdd3c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:56.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.599+0000 7fdd41d77640 1 -- 172.21.15.43:0/121357734 >> 172.21.15.43:0/121357734 conn(0x7fdd3c0fc7d0 msgr2=0x7fdd3c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:56.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.599+0000 7fdd41d77640 1 -- 172.21.15.43:0/121357734 shutdown_connections 2024-09-11T23:21:56.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:56.599+0000 7fdd41d77640 1 -- 172.21.15.43:0/121357734 wait complete. 2024-09-11T23:21:57.289 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:57 smithi043 ceph-mon[30950]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:21:57.289 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:57 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/121357734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:57.427 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:21:57.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:57 smithi090 ceph-mon[35130]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:21:57.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:57 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/121357734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:57.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:57 smithi097 ceph-mon[35562]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:21:57.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:57 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/121357734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:21:57.692 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:58.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.154+0000 7f3347ca7640 1 -- 172.21.15.43:0/3124659117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3340100ca0 msgr2=0x7f3340106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:58.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.154+0000 7f3347ca7640 1 --2- 172.21.15.43:0/3124659117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3340100ca0 0x7f3340106840 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3330009930 tx=0x7f333002a420 comp rx=0 tx=0).stop 2024-09-11T23:21:58.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.154+0000 7f3347ca7640 1 -- 172.21.15.43:0/3124659117 shutdown_connections 2024-09-11T23:21:58.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.154+0000 7f3347ca7640 1 --2- 172.21.15.43:0/3124659117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3340106f50 0x7f334010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:58.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.154+0000 7f3347ca7640 1 --2- 172.21.15.43:0/3124659117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3340100ca0 0x7f3340106840 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:58.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.154+0000 7f3347ca7640 1 --2- 172.21.15.43:0/3124659117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33401002f0 0x7f33401006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:58.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.154+0000 7f3347ca7640 1 -- 172.21.15.43:0/3124659117 >> 172.21.15.43:0/3124659117 conn(0x7f3340077210 msgr2=0x7f33400fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:58.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.154+0000 7f3347ca7640 1 -- 172.21.15.43:0/3124659117 shutdown_connections 2024-09-11T23:21:58.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.154+0000 7f3347ca7640 1 -- 172.21.15.43:0/3124659117 wait complete. 2024-09-11T23:21:58.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.155+0000 7f3347ca7640 1 Processor -- start 2024-09-11T23:21:58.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.155+0000 7f3347ca7640 1 -- start start 2024-09-11T23:21:58.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.155+0000 7f3347ca7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33401002f0 0x7f3340101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:58.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.155+0000 7f3347ca7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3340100ca0 0x7f33401017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:58.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.155+0000 7f3347ca7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3340106f50 0x7f3340101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:58.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.155+0000 7f3347ca7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3340111fa0 con 0x7f33401002f0 2024-09-11T23:21:58.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.155+0000 7f3347ca7640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3340111e20 con 0x7f3340100ca0 2024-09-11T23:21:58.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.155+0000 7f3347ca7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3340112120 con 0x7f3340106f50 2024-09-11T23:21:58.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.155+0000 7f3345a43640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33401002f0 0x7f3340101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:58.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.155+0000 7f3345a43640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33401002f0 0x7f3340101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45448/0 (socket says 172.21.15.43:45448) 2024-09-11T23:21:58.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.155+0000 7f3345a43640 1 -- 172.21.15.43:0/3335694015 learned_addr learned my addr 172.21.15.43:0/3335694015 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:58.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.155+0000 7f3345242640 1 --2- 172.21.15.43:0/3335694015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3340100ca0 0x7f33401017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:58.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.155+0000 7f3346244640 1 --2- 172.21.15.43:0/3335694015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3340106f50 0x7f3340101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:58.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.156+0000 7f3345a43640 1 -- 172.21.15.43:0/3335694015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3340106f50 msgr2=0x7f3340101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:58.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.156+0000 7f3345a43640 1 --2- 172.21.15.43:0/3335694015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3340106f50 0x7f3340101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:58.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.156+0000 7f3345a43640 1 -- 172.21.15.43:0/3335694015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3340100ca0 msgr2=0x7f33401017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:58.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.156+0000 7f3345a43640 1 --2- 172.21.15.43:0/3335694015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3340100ca0 0x7f33401017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:58.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.156+0000 7f3345a43640 1 -- 172.21.15.43:0/3335694015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33401a6a40 con 0x7f33401002f0 2024-09-11T23:21:58.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.156+0000 7f3345242640 1 --2- 172.21.15.43:0/3335694015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3340100ca0 0x7f33401017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:21:58.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.156+0000 7f3345a43640 1 --2- 172.21.15.43:0/3335694015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33401002f0 0x7f3340101270 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f333c00d950 tx=0x7f333c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:58.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.156+0000 7f3336ffd640 1 -- 172.21.15.43:0/3335694015 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c017070 con 0x7f33401002f0 2024-09-11T23:21:58.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.156+0000 7f3347ca7640 1 -- 172.21.15.43:0/3335694015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33401a6d30 con 0x7f33401002f0 2024-09-11T23:21:58.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.156+0000 7f3336ffd640 1 -- 172.21.15.43:0/3335694015 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f333c009c30 con 0x7f33401002f0 2024-09-11T23:21:58.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.156+0000 7f3336ffd640 1 -- 172.21.15.43:0/3335694015 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c00f6b0 con 0x7f33401002f0 2024-09-11T23:21:58.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.156+0000 7f3347ca7640 1 -- 172.21.15.43:0/3335694015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33401a71e0 con 0x7f33401002f0 2024-09-11T23:21:58.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.157+0000 7f3347ca7640 1 -- 172.21.15.43:0/3335694015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33401006d0 con 0x7f33401002f0 2024-09-11T23:21:58.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.158+0000 7f3336ffd640 1 -- 172.21.15.43:0/3335694015 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f333c0109d0 con 0x7f33401002f0 2024-09-11T23:21:58.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.159+0000 7f3336ffd640 1 --2- 172.21.15.43:0/3335694015 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f332c0a2e10 0x7f332c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:58.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.159+0000 7f3345242640 1 --2- 172.21.15.43:0/3335694015 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f332c0a2e10 0x7f332c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:58.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.159+0000 7f3336ffd640 1 -- 172.21.15.43:0/3335694015 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f333c0cf430 con 0x7f33401002f0 2024-09-11T23:21:58.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.159+0000 7f3345242640 1 --2- 172.21.15.43:0/3335694015 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f332c0a2e10 0x7f332c0a52d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3330006010 tx=0x7f3330005a20 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:58.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.162+0000 7f3336ffd640 1 -- 172.21.15.43:0/3335694015 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f333c009de0 con 0x7f33401002f0 2024-09-11T23:21:58.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.340+0000 7f3347ca7640 1 -- 172.21.15.43:0/3335694015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33401027a0 con 0x7f33401002f0 2024-09-11T23:21:58.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.342+0000 7f3336ffd640 1 -- 172.21.15.43:0/3335694015 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+23048 (secure 0 0 0) 0x7f333c09bda0 con 0x7f33401002f0 2024-09-11T23:21:58.343 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:21:58.343 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":85,"fsid":"25c0cd84-7093-11ef-bcea-c7b262605968","created":"2024-09-11T23:12:03.504406+0000","modified":"2024-09-11T23:21:50.205389+0000","last_up_change":"2024-09-11T23:19:07.797873+0000","last_in_change":"2024-09-11T23:18:44.595176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:15:17.058713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:19:56.221406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:19:58.705875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24076596-3909-4c87-85f1-cbf168070dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6803","nonce":43839997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6805","nonce":43839997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6809","nonce":43839997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6807","nonce":43839997}]},"public_addr":"172.21.15.43:6803/43839997","cluster_addr":"172.21.15.43:6805/43839997","heartbeat_back_addr":"172.21.15.43:6809/43839997","heartbeat_front_addr":"172.21.15.43:6807/43839997","state":["exists","up"]},{"osd":1,"uuid":"a0472a57-243c-489d-a247-972ee10a71f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6801","nonce":1825149246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6803","nonce":1825149246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6807","nonce":1825149246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6805","nonce":1825149246}]},"public_addr":"172.21.15.90:6801/1825149246","cluster_addr":"172.21.15.90:6803/1825149246","heartbeat_back_addr":"172.21.15.90:6807/1825149246","heartbeat_front_addr":"172.21.15.90:6805/1825149246","state":["exists","up"]},{"osd":2,"uuid":"40366536-0fcf-45a0-b8e3-49178401dc34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6801","nonce":1456846443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6803","nonce":1456846443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6807","nonce":1456846443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6805","nonce":1456846443}]},"public_addr":"172.21.15.97:6801/1456846443","cluster_addr":"172.21.15.97:6803/1456846443","heartbeat_back_addr":"172.21.15.97:6807/1456846443","heartbeat_front_addr":"172.21.15.97:6805/1456846443","state":["exists","up"]},{"osd":3,"uuid":"3062133e-000b-4421-9d53-ac434b9ed9b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6811","nonce":1962837804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6813","nonce":1962837804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6817","nonce":1962837804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6815","nonce":1962837804}]},"public_addr":"172.21.15.43:6811/1962837804","cluster_addr":"172.21.15.43:6813/1962837804","heartbeat_back_addr":"172.21.15.43:6817/1962837804","heartbeat_front_addr":"172.21.15.43:6815/1962837804","state":["exists","up"]},{"osd":4,"uuid":"2e73fa4b-f22c-4319-828d-87b91b9f19f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6809","nonce":181614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6811","nonce":181614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6815","nonce":181614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6813","nonce":181614173}]},"public_addr":"172.21.15.90:6809/181614173","cluster_addr":"172.21.15.90:6811/181614173","heartbeat_back_addr":"172.21.15.90:6815/181614173","heartbeat_front_addr":"172.21.15.90:6813/181614173","state":["exists","up"]},{"osd":5,"uuid":"1e1ad4f9-e724-440f-b515-3ec1ad97d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6809","nonce":1540039543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6811","nonce":1540039543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6815","nonce":1540039543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6813","nonce":1540039543}]},"public_addr":"172.21.15.97:6809/1540039543","cluster_addr":"172.21.15.97:6811/1540039543","heartbeat_back_addr":"172.21.15.97:6815/1540039543","heartbeat_front_addr":"172.21.15.97:6813/1540039543","state":["exists","up"]},{"osd":6,"uuid":"d34df8df-f40d-4159-8b29-5401b1b0e431","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6819","nonce":3785677637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6821","nonce":3785677637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6825","nonce":3785677637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6823","nonce":3785677637}]},"public_addr":"172.21.15.43:6819/3785677637","cluster_addr":"172.21.15.43:6821/3785677637","heartbeat_back_addr":"172.21.15.43:6825/3785677637","heartbeat_front_addr":"172.21.15.43:6823/3785677637","state":["exists","up"]},{"osd":7,"uuid":"b2e01078-528b-49fd-b000-9f584f784b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6817","nonce":236874962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6819","nonce":236874962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6823","nonce":236874962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6821","nonce":236874962}]},"public_addr":"172.21.15.90:6817/236874962","cluster_addr":"172.21.15.90:6819/236874962","heartbeat_back_addr":"172.21.15.90:6823/236874962","heartbeat_front_addr":"172.21.15.90:6821/236874962","state":["exists","up"]},{"osd":8,"uuid":"abef8f27-bf9b-4d63-bfbe-f967fc3d8b17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6817","nonce":1707327301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6819","nonce":1707327301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6823","nonce":1707327301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6821","nonce":1707327301}]},"public_addr":"172.21.15.97:6817/1707327301","cluster_addr":"172.21.15.97:6819/1707327301","heartbeat_back_addr":"172.21.15.97:6823/1707327301","heartbeat_front_addr":"172.21.15.97:6821/1707327301","state":["exists","up"]},{"osd":9,"uuid":"190ab8b2-a835-47e6-9b39-16a58331c58a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6827","nonce":3170306011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6829","nonce":3170306011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6833","nonce":3170306011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6831","nonce":3170306011}]},"public_addr":"172.21.15.43:6827/3170306011","cluster_addr":"172.21.15.43:6829/3170306011","heartbeat_back_addr":"172.21.15.43:6833/3170306011","heartbeat_front_addr":"172.21.15.43:6831/3170306011","state":["exists","up"]},{"osd":10,"uuid":"b32ab5fe-466f-4c8e-8696-cd9f820a3263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6825","nonce":59487041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6827","nonce":59487041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6831","nonce":59487041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6829","nonce":59487041}]},"public_addr":"172.21.15.90:6825/59487041","cluster_addr":"172.21.15.90:6827/59487041","heartbeat_back_addr":"172.21.15.90:6831/59487041","heartbeat_front_addr":"172.21.15.90:6829/59487041","state":["exists","up"]},{"osd":11,"uuid":"d0faa589-21c8-4a9f-8a73-813aafe6d33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6825","nonce":194699315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6827","nonce":194699315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6831","nonce":194699315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6829","nonce":194699315}]},"public_addr":"172.21.15.97:6825/194699315","cluster_addr":"172.21.15.97:6827/194699315","heartbeat_back_addr":"172.21.15.97:6831/194699315","heartbeat_front_addr":"172.21.15.97:6829/194699315","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:23.381247+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:48.843190+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:13.538979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:40.136983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:05.513790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:31.247090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:56.635221+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:22.667327+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:47.910422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:14.353914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:40.584503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:19:06.476766+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/2262294782":"2024-09-12T23:21:50.205362+0000","172.21.15.43:0/2398981254":"2024-09-12T23:21:50.205362+0000","172.21.15.43:0/481605249":"2024-09-12T23:21:50.205362+0000","172.21.15.43:0/1375379108":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1512201905":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/206044413":"2024-09-12T23:12:35.756598+0000","172.21.15.43:6800/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1579053924":"2024-09-12T23:21:50.205362+0000","172.21.15.43:6801/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:6800/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/2370495020":"2024-09-12T23:21:50.205362+0000","172.21.15.43:0/4006124758":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/76575371":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/120495444":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6800/2370495020":"2024-09-12T23:21:50.205362+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:21:58.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.345+0000 7f3347ca7640 1 -- 172.21.15.43:0/3335694015 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f332c0a2e10 msgr2=0x7f332c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:58.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.345+0000 7f3347ca7640 1 --2- 172.21.15.43:0/3335694015 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f332c0a2e10 0x7f332c0a52d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3330006010 tx=0x7f3330005a20 comp rx=0 tx=0).stop 2024-09-11T23:21:58.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.345+0000 7f3347ca7640 1 -- 172.21.15.43:0/3335694015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33401002f0 msgr2=0x7f3340101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:58.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.345+0000 7f3347ca7640 1 --2- 172.21.15.43:0/3335694015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33401002f0 0x7f3340101270 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f333c00d950 tx=0x7f333c00de10 comp rx=0 tx=0).stop 2024-09-11T23:21:58.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.345+0000 7f3347ca7640 1 -- 172.21.15.43:0/3335694015 shutdown_connections 2024-09-11T23:21:58.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.345+0000 7f3347ca7640 1 --2- 172.21.15.43:0/3335694015 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f332c0a2e10 0x7f332c0a52d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:58.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.345+0000 7f3347ca7640 1 --2- 172.21.15.43:0/3335694015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3340106f50 0x7f3340101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:58.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.345+0000 7f3347ca7640 1 --2- 172.21.15.43:0/3335694015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3340100ca0 0x7f33401017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:58.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.345+0000 7f3347ca7640 1 --2- 172.21.15.43:0/3335694015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33401002f0 0x7f3340101270 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:58.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.345+0000 7f3347ca7640 1 -- 172.21.15.43:0/3335694015 >> 172.21.15.43:0/3335694015 conn(0x7f3340077210 msgr2=0x7f33400fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:58.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.345+0000 7f3347ca7640 1 -- 172.21.15.43:0/3335694015 shutdown_connections 2024-09-11T23:21:58.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:58.345+0000 7f3347ca7640 1 -- 172.21.15.43:0/3335694015 wait complete. 2024-09-11T23:21:59.063 DEBUG:tasks.fwd_scrub:Yielding 2024-09-11T23:21:59.064 INFO:teuthology.run_tasks:Running task exec... 2024-09-11T23:21:59.073 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T23:21:59.074 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-09-11T23:21:59.074 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-11T23:21:59.074 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-11T23:21:59.074 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:21:59.075 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:21:59.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.191+0000 7f2b437c4640 1 -- 172.21.15.43:0/2270369340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b3c1009a0 msgr2=0x7f2b3c100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.191+0000 7f2b437c4640 1 --2- 172.21.15.43:0/2270369340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b3c1009a0 0x7f2b3c100e00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009930 tx=0x7f2b2c02a420 comp rx=0 tx=0).stop 2024-09-11T23:21:59.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.191+0000 7f2b437c4640 1 -- 172.21.15.43:0/2270369340 shutdown_connections 2024-09-11T23:21:59.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.191+0000 7f2b437c4640 1 --2- 172.21.15.43:0/2270369340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b3c101340 0x7f2b3c10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.191+0000 7f2b437c4640 1 --2- 172.21.15.43:0/2270369340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b3c1009a0 0x7f2b3c100e00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.191+0000 7f2b437c4640 1 --2- 172.21.15.43:0/2270369340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b3c1069b0 0x7f2b3c106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.191+0000 7f2b437c4640 1 -- 172.21.15.43:0/2270369340 >> 172.21.15.43:0/2270369340 conn(0x7f2b3c0779a0 msgr2=0x7f2b3c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:59.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.192+0000 7f2b437c4640 1 -- 172.21.15.43:0/2270369340 shutdown_connections 2024-09-11T23:21:59.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.192+0000 7f2b437c4640 1 -- 172.21.15.43:0/2270369340 wait complete. 2024-09-11T23:21:59.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.192+0000 7f2b437c4640 1 Processor -- start 2024-09-11T23:21:59.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.192+0000 7f2b437c4640 1 -- start start 2024-09-11T23:21:59.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.192+0000 7f2b437c4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b3c1009a0 0x7f2b3c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.192+0000 7f2b437c4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b3c101340 0x7f2b3c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.193+0000 7f2b437c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b3c1069b0 0x7f2b3c1a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.193+0000 7f2b437c4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c110e00 con 0x7f2b3c1069b0 2024-09-11T23:21:59.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.193+0000 7f2b437c4640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c110c80 con 0x7f2b3c101340 2024-09-11T23:21:59.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.193+0000 7f2b437c4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c110f80 con 0x7f2b3c1009a0 2024-09-11T23:21:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.193+0000 7f2b41d61640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b3c1069b0 0x7f2b3c1a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.193+0000 7f2b41d61640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b3c1069b0 0x7f2b3c1a42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45474/0 (socket says 172.21.15.43:45474) 2024-09-11T23:21:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.193+0000 7f2b41d61640 1 -- 172.21.15.43:0/206956693 learned_addr learned my addr 172.21.15.43:0/206956693 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.193+0000 7f2b41560640 1 --2- 172.21.15.43:0/206956693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b3c1009a0 0x7f2b3c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.193+0000 7f2b40d5f640 1 --2- 172.21.15.43:0/206956693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b3c101340 0x7f2b3c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.193+0000 7f2b41d61640 1 -- 172.21.15.43:0/206956693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b3c1009a0 msgr2=0x7f2b3c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.193+0000 7f2b41d61640 1 --2- 172.21.15.43:0/206956693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b3c1009a0 0x7f2b3c199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.193+0000 7f2b41d61640 1 -- 172.21.15.43:0/206956693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b3c101340 msgr2=0x7f2b3c1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.193+0000 7f2b41d61640 1 --2- 172.21.15.43:0/206956693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b3c101340 0x7f2b3c1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.193+0000 7f2b41d61640 1 -- 172.21.15.43:0/206956693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b3c1a4830 con 0x7f2b3c1069b0 2024-09-11T23:21:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.193+0000 7f2b41d61640 1 --2- 172.21.15.43:0/206956693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b3c1069b0 0x7f2b3c1a42f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400eaa0 tx=0x7f2b3400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:59.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.193+0000 7f2b327fc640 1 -- 172.21.15.43:0/206956693 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3400cdb0 con 0x7f2b3c1069b0 2024-09-11T23:21:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.193+0000 7f2b327fc640 1 -- 172.21.15.43:0/206956693 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b34005020 con 0x7f2b3c1069b0 2024-09-11T23:21:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.193+0000 7f2b437c4640 1 -- 172.21.15.43:0/206956693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b3c1a4a00 con 0x7f2b3c1069b0 2024-09-11T23:21:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.193+0000 7f2b327fc640 1 -- 172.21.15.43:0/206956693 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b340116b0 con 0x7f2b3c1069b0 2024-09-11T23:21:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.194+0000 7f2b437c4640 1 -- 172.21.15.43:0/206956693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b3c1a4f20 con 0x7f2b3c1069b0 2024-09-11T23:21:59.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.194+0000 7f2b437c4640 1 -- 172.21.15.43:0/206956693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b3c100e00 con 0x7f2b3c1069b0 2024-09-11T23:21:59.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.196+0000 7f2b327fc640 1 -- 172.21.15.43:0/206956693 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f2b34005560 con 0x7f2b3c1069b0 2024-09-11T23:21:59.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.196+0000 7f2b327fc640 1 --2- 172.21.15.43:0/206956693 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f2b280a2e90 0x7f2b280a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.197+0000 7f2b327fc640 1 -- 172.21.15.43:0/206956693 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f2b340cfbc0 con 0x7f2b3c1069b0 2024-09-11T23:21:59.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.197+0000 7f2b41560640 1 --2- 172.21.15.43:0/206956693 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f2b280a2e90 0x7f2b280a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.197+0000 7f2b41560640 1 --2- 172.21.15.43:0/206956693 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f2b280a2e90 0x7f2b280a5350 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800bb60 tx=0x7f2b3800b390 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.200+0000 7f2b327fc640 1 -- 172.21.15.43:0/206956693 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2b34016030 con 0x7f2b3c1069b0 2024-09-11T23:21:59.345 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:21:59.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:59 smithi043 ceph-mon[30950]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 879 B/s wr, 22 op/s 2024-09-11T23:21:59.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:21:59 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3335694015' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:21:59.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.385+0000 7f2b437c4640 1 -- 172.21.15.43:0/206956693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f2b3c06a000 con 0x7f2b3c1069b0 2024-09-11T23:21:59.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.389+0000 7f2b327fc640 1 -- 172.21.15.43:0/206956693 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v28)=0 v28) ==== 129+0+0 (secure 0 0 0) 0x7f2b3c06a000 con 0x7f2b3c1069b0 2024-09-11T23:21:59.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.391+0000 7f2b437c4640 1 -- 172.21.15.43:0/206956693 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f2b280a2e90 msgr2=0x7f2b280a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.391+0000 7f2b437c4640 1 --2- 172.21.15.43:0/206956693 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f2b280a2e90 0x7f2b280a5350 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800bb60 tx=0x7f2b3800b390 comp rx=0 tx=0).stop 2024-09-11T23:21:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.391+0000 7f2b437c4640 1 -- 172.21.15.43:0/206956693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b3c1069b0 msgr2=0x7f2b3c1a42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.391+0000 7f2b437c4640 1 --2- 172.21.15.43:0/206956693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b3c1069b0 0x7f2b3c1a42f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400eaa0 tx=0x7f2b3400ef60 comp rx=0 tx=0).stop 2024-09-11T23:21:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.391+0000 7f2b437c4640 1 -- 172.21.15.43:0/206956693 shutdown_connections 2024-09-11T23:21:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.391+0000 7f2b437c4640 1 --2- 172.21.15.43:0/206956693 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f2b280a2e90 0x7f2b280a5350 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.391+0000 7f2b437c4640 1 --2- 172.21.15.43:0/206956693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b3c1069b0 0x7f2b3c1a42f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.391+0000 7f2b437c4640 1 --2- 172.21.15.43:0/206956693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b3c101340 0x7f2b3c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.391+0000 7f2b437c4640 1 --2- 172.21.15.43:0/206956693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b3c1009a0 0x7f2b3c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.391+0000 7f2b437c4640 1 -- 172.21.15.43:0/206956693 >> 172.21.15.43:0/206956693 conn(0x7f2b3c0779a0 msgr2=0x7f2b3c103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.391+0000 7f2b437c4640 1 -- 172.21.15.43:0/206956693 shutdown_connections 2024-09-11T23:21:59.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.391+0000 7f2b437c4640 1 -- 172.21.15.43:0/206956693 wait complete. 2024-09-11T23:21:59.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-11T23:21:59.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.511+0000 7fc198653640 1 -- 172.21.15.43:0/1950457192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc190102470 msgr2=0x7fc19010e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.511+0000 7fc198653640 1 --2- 172.21.15.43:0/1950457192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc190102470 0x7fc19010e9a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc188009cd0 tx=0x7fc18802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.511+0000 7fc198653640 1 -- 172.21.15.43:0/1950457192 shutdown_connections 2024-09-11T23:21:59.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.511+0000 7fc198653640 1 --2- 172.21.15.43:0/1950457192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc190102470 0x7fc19010e9a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.511+0000 7fc198653640 1 --2- 172.21.15.43:0/1950457192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc190101ad0 0x7fc190101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.511+0000 7fc198653640 1 --2- 172.21.15.43:0/1950457192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc190107b70 0x7fc190107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.511+0000 7fc198653640 1 -- 172.21.15.43:0/1950457192 >> 172.21.15.43:0/1950457192 conn(0x7fc1900fd7d0 msgr2=0x7fc1900ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:59.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.511+0000 7fc198653640 1 -- 172.21.15.43:0/1950457192 shutdown_connections 2024-09-11T23:21:59.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.511+0000 7fc198653640 1 -- 172.21.15.43:0/1950457192 wait complete. 2024-09-11T23:21:59.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.512+0000 7fc198653640 1 Processor -- start 2024-09-11T23:21:59.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.512+0000 7fc198653640 1 -- start start 2024-09-11T23:21:59.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.512+0000 7fc198653640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc190101ad0 0x7fc1900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.512+0000 7fc198653640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc190102470 0x7fc190074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.512+0000 7fc198653640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc190107b70 0x7fc190074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.512+0000 7fc198653640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc190111f20 con 0x7fc190107b70 2024-09-11T23:21:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.512+0000 7fc198653640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc190111da0 con 0x7fc190102470 2024-09-11T23:21:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.512+0000 7fc198653640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901120a0 con 0x7fc190101ad0 2024-09-11T23:21:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.512+0000 7fc196bf0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc190107b70 0x7fc190074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.512+0000 7fc196bf0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc190107b70 0x7fc190074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45504/0 (socket says 172.21.15.43:45504) 2024-09-11T23:21:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.512+0000 7fc196bf0640 1 -- 172.21.15.43:0/3028607516 learned_addr learned my addr 172.21.15.43:0/3028607516 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.512+0000 7fc1963ef640 1 --2- 172.21.15.43:0/3028607516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc190101ad0 0x7fc1900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.512+0000 7fc195bee640 1 --2- 172.21.15.43:0/3028607516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc190102470 0x7fc190074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.512+0000 7fc196bf0640 1 -- 172.21.15.43:0/3028607516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc190101ad0 msgr2=0x7fc1900744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.512+0000 7fc196bf0640 1 --2- 172.21.15.43:0/3028607516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc190101ad0 0x7fc1900744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.512+0000 7fc196bf0640 1 -- 172.21.15.43:0/3028607516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc190102470 msgr2=0x7fc190074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.512+0000 7fc196bf0640 1 --2- 172.21.15.43:0/3028607516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc190102470 0x7fc190074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.512+0000 7fc196bf0640 1 -- 172.21.15.43:0/3028607516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1901a68b0 con 0x7fc190107b70 2024-09-11T23:21:59.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.512+0000 7fc195bee640 1 --2- 172.21.15.43:0/3028607516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc190102470 0x7fc190074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:21:59.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.513+0000 7fc196bf0640 1 --2- 172.21.15.43:0/3028607516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc190107b70 0x7fc190074f70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc188005cc0 tx=0x7fc188032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:59.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.513+0000 7fc1837fe640 1 -- 172.21.15.43:0/3028607516 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188004040 con 0x7fc190107b70 2024-09-11T23:21:59.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.513+0000 7fc1837fe640 1 -- 172.21.15.43:0/3028607516 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1880046c0 con 0x7fc190107b70 2024-09-11T23:21:59.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.513+0000 7fc198653640 1 -- 172.21.15.43:0/3028607516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1901a6b40 con 0x7fc190107b70 2024-09-11T23:21:59.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.513+0000 7fc1837fe640 1 -- 172.21.15.43:0/3028607516 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188030e00 con 0x7fc190107b70 2024-09-11T23:21:59.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.513+0000 7fc198653640 1 -- 172.21.15.43:0/3028607516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1901a7090 con 0x7fc190107b70 2024-09-11T23:21:59.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.514+0000 7fc198653640 1 -- 172.21.15.43:0/3028607516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc190101f30 con 0x7fc190107b70 2024-09-11T23:21:59.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.515+0000 7fc1837fe640 1 -- 172.21.15.43:0/3028607516 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fc1880305a0 con 0x7fc190107b70 2024-09-11T23:21:59.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.516+0000 7fc1837fe640 1 --2- 172.21.15.43:0/3028607516 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc17c0a2e10 0x7fc17c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.516+0000 7fc1837fe640 1 -- 172.21.15.43:0/3028607516 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fc18802e030 con 0x7fc190107b70 2024-09-11T23:21:59.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.516+0000 7fc1963ef640 1 --2- 172.21.15.43:0/3028607516 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc17c0a2e10 0x7fc17c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.517+0000 7fc1963ef640 1 --2- 172.21.15.43:0/3028607516 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc17c0a2e10 0x7fc17c0a52d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc184005d00 tx=0x7fc18400b410 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:59.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:59 smithi090 ceph-mon[35130]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 879 B/s wr, 22 op/s 2024-09-11T23:21:59.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:21:59 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3335694015' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:21:59.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.520+0000 7fc1837fe640 1 -- 172.21.15.43:0/3028607516 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc18804b060 con 0x7fc190107b70 2024-09-11T23:21:59.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:59 smithi097 ceph-mon[35562]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 879 B/s wr, 22 op/s 2024-09-11T23:21:59.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:21:59 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3335694015' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:21:59.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.702+0000 7fc198653640 1 -- 172.21.15.43:0/3028607516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fc1901a72c0 con 0x7fc190107b70 2024-09-11T23:21:59.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.705+0000 7fc1837fe640 1 -- 172.21.15.43:0/3028607516 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7fc1880bda40 con 0x7fc190107b70 2024-09-11T23:21:59.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.708+0000 7fc198653640 1 -- 172.21.15.43:0/3028607516 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc17c0a2e10 msgr2=0x7fc17c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.708+0000 7fc198653640 1 --2- 172.21.15.43:0/3028607516 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc17c0a2e10 0x7fc17c0a52d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc184005d00 tx=0x7fc18400b410 comp rx=0 tx=0).stop 2024-09-11T23:21:59.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.708+0000 7fc198653640 1 -- 172.21.15.43:0/3028607516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc190107b70 msgr2=0x7fc190074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.708+0000 7fc198653640 1 --2- 172.21.15.43:0/3028607516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc190107b70 0x7fc190074f70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc188005cc0 tx=0x7fc188032f40 comp rx=0 tx=0).stop 2024-09-11T23:21:59.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.708+0000 7fc198653640 1 -- 172.21.15.43:0/3028607516 shutdown_connections 2024-09-11T23:21:59.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.708+0000 7fc198653640 1 --2- 172.21.15.43:0/3028607516 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc17c0a2e10 0x7fc17c0a52d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.708+0000 7fc198653640 1 --2- 172.21.15.43:0/3028607516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc190107b70 0x7fc190074f70 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.708+0000 7fc198653640 1 --2- 172.21.15.43:0/3028607516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc190102470 0x7fc190074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.708+0000 7fc198653640 1 --2- 172.21.15.43:0/3028607516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc190101ad0 0x7fc1900744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.708+0000 7fc198653640 1 -- 172.21.15.43:0/3028607516 >> 172.21.15.43:0/3028607516 conn(0x7fc1900fd7d0 msgr2=0x7fc1900fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:59.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.708+0000 7fc198653640 1 -- 172.21.15.43:0/3028607516 shutdown_connections 2024-09-11T23:21:59.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.708+0000 7fc198653640 1 -- 172.21.15.43:0/3028607516 wait complete. 2024-09-11T23:21:59.720 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-11T23:21:59.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.830+0000 7ff6abaf1640 1 -- 172.21.15.43:0/3134667953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6a41069b0 msgr2=0x7ff6a4106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.830+0000 7ff6abaf1640 1 --2- 172.21.15.43:0/3134667953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6a41069b0 0x7ff6a4106d90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009930 tx=0x7ff6a002a420 comp rx=0 tx=0).stop 2024-09-11T23:21:59.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.830+0000 7ff6abaf1640 1 -- 172.21.15.43:0/3134667953 shutdown_connections 2024-09-11T23:21:59.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.830+0000 7ff6abaf1640 1 --2- 172.21.15.43:0/3134667953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6a4101340 0x7ff6a410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.830+0000 7ff6abaf1640 1 --2- 172.21.15.43:0/3134667953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6a41009a0 0x7ff6a4100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.830+0000 7ff6abaf1640 1 --2- 172.21.15.43:0/3134667953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6a41069b0 0x7ff6a4106d90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.830+0000 7ff6abaf1640 1 -- 172.21.15.43:0/3134667953 >> 172.21.15.43:0/3134667953 conn(0x7ff6a40779a0 msgr2=0x7ff6a40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:59.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.830+0000 7ff6abaf1640 1 -- 172.21.15.43:0/3134667953 shutdown_connections 2024-09-11T23:21:59.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.830+0000 7ff6abaf1640 1 -- 172.21.15.43:0/3134667953 wait complete. 2024-09-11T23:21:59.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.831+0000 7ff6abaf1640 1 Processor -- start 2024-09-11T23:21:59.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.831+0000 7ff6abaf1640 1 -- start start 2024-09-11T23:21:59.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.831+0000 7ff6abaf1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6a41009a0 0x7ff6a4199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.831+0000 7ff6abaf1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6a4101340 0x7ff6a41998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.831+0000 7ff6abaf1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6a41069b0 0x7ff6a4199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.831+0000 7ff6abaf1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a4110df0 con 0x7ff6a41009a0 2024-09-11T23:21:59.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.831+0000 7ff6abaf1640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a4110c70 con 0x7ff6a41069b0 2024-09-11T23:21:59.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.831+0000 7ff6abaf1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a4110f70 con 0x7ff6a4101340 2024-09-11T23:21:59.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.831+0000 7ff6a988d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6a41009a0 0x7ff6a4199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.831+0000 7ff6a988d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6a41009a0 0x7ff6a4199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45520/0 (socket says 172.21.15.43:45520) 2024-09-11T23:21:59.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.831+0000 7ff6a988d640 1 -- 172.21.15.43:0/621378119 learned_addr learned my addr 172.21.15.43:0/621378119 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:59.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.831+0000 7ff6a908c640 1 --2- 172.21.15.43:0/621378119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6a4101340 0x7ff6a41998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.831+0000 7ff6aa08e640 1 --2- 172.21.15.43:0/621378119 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6a41069b0 0x7ff6a4199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.831+0000 7ff6a988d640 1 -- 172.21.15.43:0/621378119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6a4101340 msgr2=0x7ff6a41998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.831+0000 7ff6a988d640 1 --2- 172.21.15.43:0/621378119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6a4101340 0x7ff6a41998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.831+0000 7ff6a988d640 1 -- 172.21.15.43:0/621378119 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6a41069b0 msgr2=0x7ff6a4199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.831+0000 7ff6a988d640 1 --2- 172.21.15.43:0/621378119 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6a41069b0 0x7ff6a4199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.831+0000 7ff6a988d640 1 -- 172.21.15.43:0/621378119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a41a4760 con 0x7ff6a41009a0 2024-09-11T23:21:59.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.832+0000 7ff6a988d640 1 --2- 172.21.15.43:0/621378119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6a41009a0 0x7ff6a4199360 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009230 tx=0x7ff6a0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:59.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.832+0000 7ff696ffd640 1 -- 172.21.15.43:0/621378119 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a003f030 con 0x7ff6a41009a0 2024-09-11T23:21:59.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.832+0000 7ff696ffd640 1 -- 172.21.15.43:0/621378119 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6a0004440 con 0x7ff6a41009a0 2024-09-11T23:21:59.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.832+0000 7ff696ffd640 1 -- 172.21.15.43:0/621378119 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0035070 con 0x7ff6a41009a0 2024-09-11T23:21:59.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.832+0000 7ff6abaf1640 1 -- 172.21.15.43:0/621378119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a41a49f0 con 0x7ff6a41009a0 2024-09-11T23:21:59.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.832+0000 7ff6abaf1640 1 -- 172.21.15.43:0/621378119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a41a4f30 con 0x7ff6a41009a0 2024-09-11T23:21:59.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.833+0000 7ff6abaf1640 1 -- 172.21.15.43:0/621378119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a41020e0 con 0x7ff6a41009a0 2024-09-11T23:21:59.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.834+0000 7ff696ffd640 1 -- 172.21.15.43:0/621378119 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7ff6a0042030 con 0x7ff6a41009a0 2024-09-11T23:21:59.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.835+0000 7ff696ffd640 1 --2- 172.21.15.43:0/621378119 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff6900a2e90 0x7ff6900a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.835+0000 7ff696ffd640 1 -- 172.21.15.43:0/621378119 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff6a00efdd0 con 0x7ff6a41009a0 2024-09-11T23:21:59.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.835+0000 7ff6a908c640 1 --2- 172.21.15.43:0/621378119 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff6900a2e90 0x7ff6900a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.836+0000 7ff6a908c640 1 --2- 172.21.15.43:0/621378119 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff6900a2e90 0x7ff6900a5350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff6a40fffa0 tx=0x7ff698005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:59.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.839+0000 7ff696ffd640 1 -- 172.21.15.43:0/621378119 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff6a00bc810 con 0x7ff6a41009a0 2024-09-11T23:21:59.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.867+0000 7f4c8b5e8640 1 -- 172.21.15.43:0/2065857809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c84102790 msgr2=0x7f4c841067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.867+0000 7f4c8b5e8640 1 --2- 172.21.15.43:0/2065857809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c84102790 0x7f4c841067e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009900 tx=0x7f4c7402a410 comp rx=0 tx=0).stop 2024-09-11T23:21:59.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.868+0000 7f4c8b5e8640 1 -- 172.21.15.43:0/2065857809 shutdown_connections 2024-09-11T23:21:59.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.868+0000 7f4c8b5e8640 1 --2- 172.21.15.43:0/2065857809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c84106f10 0x7f4c8410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.868+0000 7f4c8b5e8640 1 --2- 172.21.15.43:0/2065857809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c84102790 0x7f4c841067e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.868+0000 7f4c8b5e8640 1 --2- 172.21.15.43:0/2065857809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c84101de0 0x7f4c841021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.868+0000 7f4c8b5e8640 1 -- 172.21.15.43:0/2065857809 >> 172.21.15.43:0/2065857809 conn(0x7f4c840fd8e0 msgr2=0x7f4c840ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.868+0000 7f4c8b5e8640 1 -- 172.21.15.43:0/2065857809 shutdown_connections 2024-09-11T23:21:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.868+0000 7f4c8b5e8640 1 -- 172.21.15.43:0/2065857809 wait complete. 2024-09-11T23:21:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.868+0000 7f4c8b5e8640 1 Processor -- start 2024-09-11T23:21:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.868+0000 7f4c8b5e8640 1 -- start start 2024-09-11T23:21:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.868+0000 7f4c8b5e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c84101de0 0x7f4c840744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.868+0000 7f4c8b5e8640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c84102790 0x7f4c84074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.869+0000 7f4c8b5e8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c84106f10 0x7f4c84074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.869+0000 7f4c8b5e8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4c84111fc0 con 0x7f4c84101de0 2024-09-11T23:21:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.869+0000 7f4c8b5e8640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4c84111e40 con 0x7f4c84102790 2024-09-11T23:21:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.869+0000 7f4c8b5e8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4c84112140 con 0x7f4c84106f10 2024-09-11T23:21:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.869+0000 7f4c89384640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c84101de0 0x7f4c840744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.869+0000 7f4c89384640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c84101de0 0x7f4c840744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45532/0 (socket says 172.21.15.43:45532) 2024-09-11T23:21:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.869+0000 7f4c89384640 1 -- 172.21.15.43:0/1814491583 learned_addr learned my addr 172.21.15.43:0/1814491583 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.869+0000 7f4c88b83640 1 --2- 172.21.15.43:0/1814491583 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c84102790 0x7f4c84074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.869+0000 7f4c89b85640 1 --2- 172.21.15.43:0/1814491583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c84106f10 0x7f4c84074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.869+0000 7f4c89384640 1 -- 172.21.15.43:0/1814491583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c84106f10 msgr2=0x7f4c84074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.869+0000 7f4c89384640 1 --2- 172.21.15.43:0/1814491583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c84106f10 0x7f4c84074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.869+0000 7f4c89384640 1 -- 172.21.15.43:0/1814491583 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c84102790 msgr2=0x7f4c84074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.869+0000 7f4c89384640 1 --2- 172.21.15.43:0/1814491583 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c84102790 0x7f4c84074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.869+0000 7f4c89384640 1 -- 172.21.15.43:0/1814491583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c841a6860 con 0x7f4c84101de0 2024-09-11T23:21:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.869+0000 7f4c89384640 1 --2- 172.21.15.43:0/1814491583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c84101de0 0x7f4c840744f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000c970 tx=0x7f4c8000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.869+0000 7f4c7a7fc640 1 -- 172.21.15.43:0/1814491583 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80007bb0 con 0x7f4c84101de0 2024-09-11T23:21:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.869+0000 7f4c7a7fc640 1 -- 172.21.15.43:0/1814491583 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c80007d50 con 0x7f4c84101de0 2024-09-11T23:21:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.869+0000 7f4c8b5e8640 1 -- 172.21.15.43:0/1814491583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c841a6b50 con 0x7f4c84101de0 2024-09-11T23:21:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.870+0000 7f4c7a7fc640 1 -- 172.21.15.43:0/1814491583 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8000f450 con 0x7f4c84101de0 2024-09-11T23:21:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.870+0000 7f4c8b5e8640 1 -- 172.21.15.43:0/1814491583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c841a7060 con 0x7f4c84101de0 2024-09-11T23:21:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.872+0000 7f4c7a7fc640 1 -- 172.21.15.43:0/1814491583 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f4c80005020 con 0x7f4c84101de0 2024-09-11T23:21:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.872+0000 7f4c8d9c1740 1 Processor -- start 2024-09-11T23:21:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.873+0000 7f4c8d9c1740 1 -- start start 2024-09-11T23:21:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.873+0000 7f4c8d9c1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c130813b60 0x55c130813f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.873+0000 7f4c8d9c1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c130814480 0x55c13081e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.873+0000 7f4c8d9c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c13081ed40 0x55c130821130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.873+0000 7f4c8d9c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c130779280 con 0x55c13081ed40 2024-09-11T23:21:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.873+0000 7f4c8d9c1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c130720b10 con 0x55c130813b60 2024-09-11T23:21:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.873+0000 7f4c8d9c1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c130775240 con 0x55c130814480 2024-09-11T23:21:59.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.873+0000 7f4c88b83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c13081ed40 0x55c130821130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.873+0000 7f4c88b83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c13081ed40 0x55c130821130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45548/0 (socket says 172.21.15.43:45548) 2024-09-11T23:21:59.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.873+0000 7f4c88b83640 1 -- 172.21.15.43:0/3377748040 learned_addr learned my addr 172.21.15.43:0/3377748040 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:59.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.873+0000 7f4c89384640 1 --2- 172.21.15.43:0/3377748040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c130814480 0x55c13081e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.873+0000 7f4c7a7fc640 1 --2- 172.21.15.43:0/1814491583 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f4c700a2e90 0x7f4c700a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.873+0000 7f4c89b85640 1 --2- 172.21.15.43:0/3377748040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c130813b60 0x55c130813f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.873+0000 7f4c89b85640 1 --2- 172.21.15.43:0/1814491583 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f4c700a2e90 0x7f4c700a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.873+0000 7f4c88b83640 1 -- 172.21.15.43:0/3377748040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c130814480 msgr2=0x55c13081e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:59.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.873+0000 7f4c88b83640 1 --2- 172.21.15.43:0/3377748040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c130814480 0x55c13081e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.873+0000 7f4c88b83640 1 -- 172.21.15.43:0/3377748040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c130813b60 msgr2=0x55c130813f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:59.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.873+0000 7f4c88b83640 1 --2- 172.21.15.43:0/3377748040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c130813b60 0x55c130813f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.873+0000 7f4c88b83640 1 -- 172.21.15.43:0/3377748040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c13072d1f0 con 0x55c13081ed40 2024-09-11T23:21:59.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.874+0000 7f4c7a7fc640 1 -- 172.21.15.43:0/1814491583 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f4c800cf420 con 0x7f4c84101de0 2024-09-11T23:21:59.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.874+0000 7f4c88b83640 1 --2- 172.21.15.43:0/3377748040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c13081ed40 0x55c130821130 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009230 tx=0x7f4c74004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f7637525684b1d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:59.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.874+0000 7f4c6f7fe640 1 -- 172.21.15.43:0/3377748040 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74040070 con 0x55c13081ed40 2024-09-11T23:21:59.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.874+0000 7f4c89b85640 1 --2- 172.21.15.43:0/1814491583 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f4c700a2e90 0x7f4c700a5350 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0077f0 tx=0x7f4c7c007520 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:59.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.874+0000 7f4c6f7fe640 1 -- 172.21.15.43:0/3377748040 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c740047a0 con 0x55c13081ed40 2024-09-11T23:21:59.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.874+0000 7f4c6f7fe640 1 -- 172.21.15.43:0/3377748040 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74035040 con 0x55c13081ed40 2024-09-11T23:21:59.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.875+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/3377748040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c13081ed40 msgr2=0x55c130821130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.875+0000 7f4c8d9c1740 1 --2- 172.21.15.43:0/3377748040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c13081ed40 0x55c130821130 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009230 tx=0x7f4c74004310 comp rx=0 tx=0).stop 2024-09-11T23:21:59.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.875+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/3377748040 shutdown_connections 2024-09-11T23:21:59.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.875+0000 7f4c8d9c1740 1 --2- 172.21.15.43:0/3377748040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c13081ed40 0x55c130821130 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.875+0000 7f4c8d9c1740 1 --2- 172.21.15.43:0/3377748040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c130814480 0x55c13081e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.875+0000 7f4c8d9c1740 1 --2- 172.21.15.43:0/3377748040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c130813b60 0x55c130813f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.875+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/3377748040 >> 172.21.15.43:0/3377748040 conn(0x55c1308100f0 msgr2=0x55c1308104d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:59.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.875+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/3377748040 shutdown_connections 2024-09-11T23:21:59.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.875+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/3377748040 wait complete. 2024-09-11T23:21:59.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.875+0000 7f4c8d9c1740 1 -- start start 2024-09-11T23:21:59.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.875+0000 7f4c8d9c1740 1 Processor -- start 2024-09-11T23:21:59.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c8d9c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c130813b60 0x55c13081bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c8d9c1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c130814480 0x55c13081c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c8d9c1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c13081ed40 0x55c13081c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c8d9c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c130821910 con 0x55c130813b60 2024-09-11T23:21:59.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c8d9c1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c130733f10 con 0x55c13081ed40 2024-09-11T23:21:59.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c8d9c1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c130821a90 con 0x55c130814480 2024-09-11T23:21:59.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c89384640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c130813b60 0x55c13081bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c8d9c1740 10 client.0 _ll_register_callbacks cb 0x55c1306fca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:59.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c89384640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c130813b60 0x55c13081bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45560/0 (socket says 172.21.15.43:45560) 2024-09-11T23:21:59.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c89384640 1 -- 172.21.15.43:0/2610907615 learned_addr learned my addr 172.21.15.43:0/2610907615 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:59.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c88b83640 1 --2- 172.21.15.43:0/2610907615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c130814480 0x55c13081c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c89384640 1 -- 172.21.15.43:0/2610907615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c130814480 msgr2=0x55c13081c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:59.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c89384640 1 --2- 172.21.15.43:0/2610907615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c130814480 0x55c13081c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c89384640 1 -- 172.21.15.43:0/2610907615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c13081ed40 msgr2=0x55c13081c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:59.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c89b85640 1 --2- 172.21.15.43:0/2610907615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c13081ed40 0x55c13081c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c89384640 1 --2- 172.21.15.43:0/2610907615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c13081ed40 0x55c13081c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c89384640 1 -- 172.21.15.43:0/2610907615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1304ea6a0 con 0x55c130813b60 2024-09-11T23:21:59.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c89b85640 1 --2- 172.21.15.43:0/2610907615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c13081ed40 0x55c13081c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:21:59.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c89384640 1 --2- 172.21.15.43:0/2610907615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c130813b60 0x55c13081bba0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4c80098110 tx=0x7f4c8000a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6891747228a3b275 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:59.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:21:59.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c6cff9640 1 -- 172.21.15.43:0/2610907615 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8001d050 con 0x55c130813b60 2024-09-11T23:21:59.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.876+0000 7f4c6cff9640 1 -- 172.21.15.43:0/2610907615 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f4c80005310 con 0x55c130813b60 2024-09-11T23:21:59.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.877+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/2610907615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c740047a0 con 0x55c130813b60 2024-09-11T23:21:59.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.877+0000 7f4c6cff9640 1 -- 172.21.15.43:0/2610907615 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4c8009fa40 con 0x55c130813b60 2024-09-11T23:21:59.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.877+0000 7f4c8d9c1740 10 client.15279 fetch_fsmap learned FSMap version 24 2024-09-11T23:21:59.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.877+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/2610907615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c13082d550 con 0x55c130813b60 2024-09-11T23:21:59.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.877+0000 7f4c6cff9640 1 -- 172.21.15.43:0/2610907615 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c740047a0 con 0x55c130813b60 2024-09-11T23:21:59.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.877+0000 7f4c6cff9640 1 -- 172.21.15.43:0/2610907615 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f4c800cce00 con 0x55c130813b60 2024-09-11T23:21:59.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.877+0000 7f4c8d9c1740 10 client.15279 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:21:59.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.877+0000 7f4c8d9c1740 10 client.15279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:21:59.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.877+0000 7f4c8d9c1740 20 client.15279 populate_metadata read hostname 'smithi043' 2024-09-11T23:21:59.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.877+0000 7f4c8d9c1740 1 --2- 172.21.15.43:0/2610907615 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c13082ea10 0x55c130830e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.877+0000 7f4c8d9c1740 4 client.15279 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:21:59.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.877+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/2610907615 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1307867f0 con 0x55c13082ea10 2024-09-11T23:21:59.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.878+0000 7f4c88b83640 1 --2- 172.21.15.43:0/2610907615 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c13082ea10 0x55c130830e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.878+0000 7f4c6cff9640 10 client.15279 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:21:59.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.878+0000 7f4c88b83640 1 --2- 172.21.15.43:0/2610907615 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c13082ea10 0x55c130830e20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1a38f974bc56fd2 server_cookie=43cf4d352e6fbc79 in_seq=0 out_seq=0 2024-09-11T23:21:59.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.879+0000 7f4c6cff9640 1 -- 172.21.15.43:0/2610907615 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c1307867f0 con 0x55c13082ea10 2024-09-11T23:21:59.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.879+0000 7f4c6cff9640 10 client.15279 handle_command_reply: tid=0 2024-09-11T23:21:59.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.879+0000 7f4c8d9c1740 1 client.15279 shutdown 2024-09-11T23:21:59.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.879+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/2610907615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c130813b60 msgr2=0x55c13081bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.879+0000 7f4c8d9c1740 1 --2- 172.21.15.43:0/2610907615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c130813b60 0x55c13081bba0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4c80098110 tx=0x7f4c8000a570 comp rx=0 tx=0).stop 2024-09-11T23:21:59.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.879+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/2610907615 shutdown_connections 2024-09-11T23:21:59.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.879+0000 7f4c8d9c1740 1 --2- 172.21.15.43:0/2610907615 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c13082ea10 0x55c130830e20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.879+0000 7f4c8d9c1740 1 --2- 172.21.15.43:0/2610907615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c13081ed40 0x55c13081c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.879+0000 7f4c8d9c1740 1 --2- 172.21.15.43:0/2610907615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c130814480 0x55c13081c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.879+0000 7f4c8d9c1740 1 --2- 172.21.15.43:0/2610907615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c130813b60 0x55c13081bba0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.879+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/2610907615 >> 172.21.15.43:0/2610907615 conn(0x55c1308100f0 msgr2=0x55c130815dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:59.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.879+0000 7f4c6cff9640 0 client.15279 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:21:59.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.879+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/2610907615 shutdown_connections 2024-09-11T23:21:59.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.879+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/2610907615 wait complete. 2024-09-11T23:21:59.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.879+0000 7f4c8d9c1740 20 client.15279 trim_cache size 0 max 16384 2024-09-11T23:21:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.897+0000 7f4c8d9c1740 1 Processor -- start 2024-09-11T23:21:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.897+0000 7f4c8d9c1740 1 -- start start 2024-09-11T23:21:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.897+0000 7f4c8d9c1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c130813b60 0x55c130815cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.897+0000 7f4c8d9c1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c130814480 0x55c130836690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.897+0000 7f4c8d9c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c13081ed40 0x55c130836bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.897+0000 7f4c8d9c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4c841b2250 con 0x55c13081ed40 2024-09-11T23:21:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.897+0000 7f4c8d9c1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4c841afb20 con 0x55c130813b60 2024-09-11T23:21:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.897+0000 7f4c8d9c1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4c841ad970 con 0x55c130814480 2024-09-11T23:21:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.898+0000 7f4c89b85640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c13081ed40 0x55c130836bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.898+0000 7f4c89b85640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c13081ed40 0x55c130836bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45568/0 (socket says 172.21.15.43:45568) 2024-09-11T23:21:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.898+0000 7f4c89b85640 1 -- 172.21.15.43:0/1553069173 learned_addr learned my addr 172.21.15.43:0/1553069173 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.898+0000 7f4c88b83640 1 --2- 172.21.15.43:0/1553069173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c130814480 0x55c130836690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.898+0000 7f4c89384640 1 --2- 172.21.15.43:0/1553069173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c130813b60 0x55c130815cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.898+0000 7f4c89b85640 1 -- 172.21.15.43:0/1553069173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c130814480 msgr2=0x55c130836690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.898+0000 7f4c89b85640 1 --2- 172.21.15.43:0/1553069173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c130814480 0x55c130836690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.898+0000 7f4c89b85640 1 -- 172.21.15.43:0/1553069173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c130813b60 msgr2=0x55c130815cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.898+0000 7f4c89b85640 1 --2- 172.21.15.43:0/1553069173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c130813b60 0x55c130815cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.898+0000 7f4c89b85640 1 -- 172.21.15.43:0/1553069173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c13080b830 con 0x55c13081ed40 2024-09-11T23:21:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.898+0000 7f4c89b85640 1 --2- 172.21.15.43:0/1553069173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c13081ed40 0x55c130836bd0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c014f40 tx=0x7f4c7c0078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7a26127accd790 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.898+0000 7f4c6effd640 1 -- 172.21.15.43:0/1553069173 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c024070 con 0x55c13081ed40 2024-09-11T23:21:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.898+0000 7f4c6effd640 1 -- 172.21.15.43:0/1553069173 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c7c004d20 con 0x55c13081ed40 2024-09-11T23:21:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.899+0000 7f4c6effd640 1 -- 172.21.15.43:0/1553069173 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c005520 con 0x55c13081ed40 2024-09-11T23:21:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.899+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/1553069173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c13081ed40 msgr2=0x55c130836bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.899+0000 7f4c8d9c1740 1 --2- 172.21.15.43:0/1553069173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c13081ed40 0x55c130836bd0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c014f40 tx=0x7f4c7c0078c0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.899+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/1553069173 shutdown_connections 2024-09-11T23:21:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.899+0000 7f4c8d9c1740 1 --2- 172.21.15.43:0/1553069173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c13081ed40 0x55c130836bd0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.899+0000 7f4c8d9c1740 1 --2- 172.21.15.43:0/1553069173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c130814480 0x55c130836690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.899+0000 7f4c8d9c1740 1 --2- 172.21.15.43:0/1553069173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c130813b60 0x55c130815cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.899+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/1553069173 >> 172.21.15.43:0/1553069173 conn(0x55c1308100f0 msgr2=0x55c1308155b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.899+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/1553069173 shutdown_connections 2024-09-11T23:21:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.899+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/1553069173 wait complete. 2024-09-11T23:21:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.900+0000 7f4c8d9c1740 1 -- start start 2024-09-11T23:21:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.900+0000 7f4c8d9c1740 1 Processor -- start 2024-09-11T23:21:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.900+0000 7f4c8d9c1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c130813b60 0x55c1308273f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.900+0000 7f4c8d9c1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c130814480 0x55c1308188b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.900+0000 7f4c8d9c1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c13081ed40 0x55c130818df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.900+0000 7f4c8d9c1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4c841adfe0 con 0x55c13081ed40 2024-09-11T23:21:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.900+0000 7f4c8d9c1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4c841af450 con 0x55c130813b60 2024-09-11T23:21:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.900+0000 7f4c8d9c1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4c841ae6b0 con 0x55c130814480 2024-09-11T23:21:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.900+0000 7f4c89b85640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c13081ed40 0x55c130818df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.900+0000 7f4c8d9c1740 10 client.0 _ll_register_callbacks cb 0x7f4c8000c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:21:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.900+0000 7f4c89b85640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c13081ed40 0x55c130818df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45578/0 (socket says 172.21.15.43:45578) 2024-09-11T23:21:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.900+0000 7f4c89b85640 1 -- 172.21.15.43:0/2839417637 learned_addr learned my addr 172.21.15.43:0/2839417637 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:21:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.901+0000 7f4c89384640 1 --2- 172.21.15.43:0/2839417637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c130813b60 0x55c1308273f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.901+0000 7f4c88b83640 1 --2- 172.21.15.43:0/2839417637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c130814480 0x55c1308188b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.901+0000 7f4c89b85640 1 -- 172.21.15.43:0/2839417637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c130814480 msgr2=0x55c1308188b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.901+0000 7f4c89b85640 1 --2- 172.21.15.43:0/2839417637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c130814480 0x55c1308188b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.901+0000 7f4c89b85640 1 -- 172.21.15.43:0/2839417637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c130813b60 msgr2=0x55c1308273f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:21:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.901+0000 7f4c89b85640 1 --2- 172.21.15.43:0/2839417637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c130813b60 0x55c1308273f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.901+0000 7f4c89b85640 1 -- 172.21.15.43:0/2839417637 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c130819600 con 0x55c13081ed40 2024-09-11T23:21:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.901+0000 7f4c89384640 1 --2- 172.21.15.43:0/2839417637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c130813b60 0x55c1308273f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:21:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.901+0000 7f4c89b85640 1 --2- 172.21.15.43:0/2839417637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c13081ed40 0x55c130818df0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c014f10 tx=0x7f4c7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cae47693910496 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:21:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.901+0000 7f4c6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:21:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.901+0000 7f4c6cff9640 1 -- 172.21.15.43:0/2839417637 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c03e060 con 0x55c13081ed40 2024-09-11T23:21:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.901+0000 7f4c6cff9640 1 -- 172.21.15.43:0/2839417637 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f4c7c005370 con 0x55c13081ed40 2024-09-11T23:21:59.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.901+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/2839417637 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c7c004d20 con 0x55c13081ed40 2024-09-11T23:21:59.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.902+0000 7f4c6cff9640 1 -- 172.21.15.43:0/2839417637 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4c7c024050 con 0x55c13081ed40 2024-09-11T23:21:59.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.902+0000 7f4c8d9c1740 10 client.15285 fetch_fsmap learned FSMap version 24 2024-09-11T23:21:59.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.902+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/2839417637 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c1308996e0 con 0x55c13081ed40 2024-09-11T23:21:59.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.902+0000 7f4c6cff9640 1 -- 172.21.15.43:0/2839417637 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c7c003ec0 con 0x55c13081ed40 2024-09-11T23:21:59.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.902+0000 7f4c6cff9640 1 -- 172.21.15.43:0/2839417637 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f4c7c020440 con 0x55c13081ed40 2024-09-11T23:21:59.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.902+0000 7f4c8d9c1740 10 client.15285 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:21:59.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.902+0000 7f4c8d9c1740 10 client.15285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:21:59.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.902+0000 7f4c8d9c1740 20 client.15285 populate_metadata read hostname 'smithi043' 2024-09-11T23:21:59.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.902+0000 7f4c8d9c1740 1 --2- 172.21.15.43:0/2839417637 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c130899cf0 0x55c13089c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:21:59.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.902+0000 7f4c8d9c1740 4 client.15285 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:21:59.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.902+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/2839417637 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f4c7c004d20 con 0x55c130899cf0 2024-09-11T23:21:59.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.902+0000 7f4c89384640 1 --2- 172.21.15.43:0/2839417637 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c130899cf0 0x55c13089c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:21:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.903+0000 7f4c6cff9640 10 client.15285 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:21:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.903+0000 7f4c89384640 1 --2- 172.21.15.43:0/2839417637 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c130899cf0 0x55c13089c110 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e6e9b09654f0e1d server_cookie=48272f77e89f714c in_seq=0 out_seq=0 2024-09-11T23:21:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.904+0000 7f4c6cff9640 1 -- 172.21.15.43:0/2839417637 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f4c7c004d20 con 0x55c130899cf0 2024-09-11T23:21:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.904+0000 7f4c6cff9640 10 client.15285 handle_command_reply: tid=0 2024-09-11T23:21:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.904+0000 7f4c8d9c1740 1 client.15285 shutdown 2024-09-11T23:21:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.904+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/2839417637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c13081ed40 msgr2=0x55c130818df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.904+0000 7f4c8d9c1740 1 --2- 172.21.15.43:0/2839417637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c13081ed40 0x55c130818df0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c014f10 tx=0x7f4c7c004a40 comp rx=0 tx=0).stop 2024-09-11T23:21:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.904+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/2839417637 shutdown_connections 2024-09-11T23:21:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.904+0000 7f4c8d9c1740 1 --2- 172.21.15.43:0/2839417637 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c130899cf0 0x55c13089c110 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.904+0000 7f4c6cff9640 0 client.15285 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:21:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.904+0000 7f4c8d9c1740 1 --2- 172.21.15.43:0/2839417637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c13081ed40 0x55c130818df0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.904+0000 7f4c8d9c1740 1 --2- 172.21.15.43:0/2839417637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c130814480 0x55c1308188b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.904+0000 7f4c8d9c1740 1 --2- 172.21.15.43:0/2839417637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c130813b60 0x55c1308273f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.904+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/2839417637 >> 172.21.15.43:0/2839417637 conn(0x55c1308100f0 msgr2=0x55c13080f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:59.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.904+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/2839417637 shutdown_connections 2024-09-11T23:21:59.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.904+0000 7f4c8d9c1740 1 -- 172.21.15.43:0/2839417637 wait complete. 2024-09-11T23:21:59.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.905+0000 7f4c8d9c1740 20 client.15285 trim_cache size 0 max 16384 2024-09-11T23:21:59.911 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:21:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-11T23:21:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "76241474-3f11-44ed-bdea-435862600675", 2024-09-11T23:21:59.911 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-11T23:21:59.911 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:21:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.905+0000 7f4c8b5e8640 1 -- 172.21.15.43:0/1814491583 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f4c700a2e90 msgr2=0x7f4c700a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.905+0000 7f4c8b5e8640 1 --2- 172.21.15.43:0/1814491583 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f4c700a2e90 0x7f4c700a5350 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0077f0 tx=0x7f4c7c007520 comp rx=0 tx=0).stop 2024-09-11T23:21:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.905+0000 7f4c8b5e8640 1 -- 172.21.15.43:0/1814491583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c84101de0 msgr2=0x7f4c840744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:21:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.905+0000 7f4c8b5e8640 1 --2- 172.21.15.43:0/1814491583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c84101de0 0x7f4c840744f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000c970 tx=0x7f4c8000ce30 comp rx=0 tx=0).stop 2024-09-11T23:21:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.905+0000 7f4c8b5e8640 1 -- 172.21.15.43:0/1814491583 shutdown_connections 2024-09-11T23:21:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.905+0000 7f4c8b5e8640 1 --2- 172.21.15.43:0/1814491583 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f4c700a2e90 0x7f4c700a5350 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.905+0000 7f4c8b5e8640 1 --2- 172.21.15.43:0/1814491583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c84106f10 0x7f4c84074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.905+0000 7f4c8b5e8640 1 --2- 172.21.15.43:0/1814491583 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c84102790 0x7f4c84074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.905+0000 7f4c8b5e8640 1 --2- 172.21.15.43:0/1814491583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c84101de0 0x7f4c840744f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:21:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.905+0000 7f4c8b5e8640 1 -- 172.21.15.43:0/1814491583 >> 172.21.15.43:0/1814491583 conn(0x7f4c840fd8e0 msgr2=0x7f4c840fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:21:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.905+0000 7f4c8b5e8640 1 -- 172.21.15.43:0/1814491583 shutdown_connections 2024-09-11T23:21:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:21:59.906+0000 7f4c8b5e8640 1 -- 172.21.15.43:0/1814491583 wait complete. 2024-09-11T23:22:00.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.031+0000 7ff6abaf1640 1 -- 172.21.15.43:0/621378119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7ff6a4100e00 con 0x7ff6a41009a0 2024-09-11T23:22:00.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.035+0000 7ff696ffd640 1 -- 172.21.15.43:0/621378119 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7ff6a00bc110 con 0x7ff6a41009a0 2024-09-11T23:22:00.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.037+0000 7ff6abaf1640 1 -- 172.21.15.43:0/621378119 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff6900a2e90 msgr2=0x7ff6900a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:00.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.037+0000 7ff6abaf1640 1 --2- 172.21.15.43:0/621378119 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff6900a2e90 0x7ff6900a5350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff6a40fffa0 tx=0x7ff698005f90 comp rx=0 tx=0).stop 2024-09-11T23:22:00.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.037+0000 7ff6abaf1640 1 -- 172.21.15.43:0/621378119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6a41009a0 msgr2=0x7ff6a4199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:00.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.037+0000 7ff6abaf1640 1 --2- 172.21.15.43:0/621378119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6a41009a0 0x7ff6a4199360 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009230 tx=0x7ff6a0004040 comp rx=0 tx=0).stop 2024-09-11T23:22:00.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.037+0000 7ff6abaf1640 1 -- 172.21.15.43:0/621378119 shutdown_connections 2024-09-11T23:22:00.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.037+0000 7ff6abaf1640 1 --2- 172.21.15.43:0/621378119 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff6900a2e90 0x7ff6900a5350 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.037+0000 7ff6abaf1640 1 --2- 172.21.15.43:0/621378119 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6a41069b0 0x7ff6a4199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.037+0000 7ff6abaf1640 1 --2- 172.21.15.43:0/621378119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6a4101340 0x7ff6a41998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.037+0000 7ff6abaf1640 1 --2- 172.21.15.43:0/621378119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6a41009a0 0x7ff6a4199360 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.037+0000 7ff6abaf1640 1 -- 172.21.15.43:0/621378119 >> 172.21.15.43:0/621378119 conn(0x7ff6a40779a0 msgr2=0x7ff6a4103930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:00.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.038+0000 7ff6abaf1640 1 -- 172.21.15.43:0/621378119 shutdown_connections 2024-09-11T23:22:00.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.038+0000 7ff6abaf1640 1 -- 172.21.15.43:0/621378119 wait complete. 2024-09-11T23:22:00.049 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-11T23:22:00.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.155+0000 7f3eefadd640 1 -- 172.21.15.43:0/114769034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee81011e0 msgr2=0x7f3ee810dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:00.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.155+0000 7f3eefadd640 1 --2- 172.21.15.43:0/114769034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee81011e0 0x7f3ee810dea0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3edc009900 tx=0x7f3edc02a410 comp rx=0 tx=0).stop 2024-09-11T23:22:00.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.155+0000 7f3eefadd640 1 -- 172.21.15.43:0/114769034 shutdown_connections 2024-09-11T23:22:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.155+0000 7f3eefadd640 1 --2- 172.21.15.43:0/114769034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee810e3e0 0x7f3ee81107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.155+0000 7f3eefadd640 1 --2- 172.21.15.43:0/114769034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee81011e0 0x7f3ee810dea0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.155+0000 7f3eefadd640 1 --2- 172.21.15.43:0/114769034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ee81008c0 0x7f3ee8100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.155+0000 7f3eefadd640 1 -- 172.21.15.43:0/114769034 >> 172.21.15.43:0/114769034 conn(0x7f3ee80fc6c0 msgr2=0x7f3ee80feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.155+0000 7f3eefadd640 1 -- 172.21.15.43:0/114769034 shutdown_connections 2024-09-11T23:22:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.155+0000 7f3eefadd640 1 -- 172.21.15.43:0/114769034 wait complete. 2024-09-11T23:22:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.156+0000 7f3eefadd640 1 Processor -- start 2024-09-11T23:22:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.156+0000 7f3eefadd640 1 -- start start 2024-09-11T23:22:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.156+0000 7f3eefadd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee81008c0 0x7f3ee819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.156+0000 7f3eefadd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ee81011e0 0x7f3ee819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.156+0000 7f3eefadd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee810e3e0 0x7f3ee819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.156+0000 7f3eefadd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee81140e0 con 0x7f3ee81008c0 2024-09-11T23:22:00.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.156+0000 7f3eefadd640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee8113f60 con 0x7f3ee81011e0 2024-09-11T23:22:00.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.156+0000 7f3eefadd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee8114260 con 0x7f3ee810e3e0 2024-09-11T23:22:00.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.156+0000 7f3eed879640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee81008c0 0x7f3ee819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:00.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.156+0000 7f3eed879640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee81008c0 0x7f3ee819a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45594/0 (socket says 172.21.15.43:45594) 2024-09-11T23:22:00.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.156+0000 7f3eed879640 1 -- 172.21.15.43:0/2438588184 learned_addr learned my addr 172.21.15.43:0/2438588184 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:00.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.156+0000 7f3eee07a640 1 --2- 172.21.15.43:0/2438588184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee810e3e0 0x7f3ee819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:00.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.156+0000 7f3eed078640 1 --2- 172.21.15.43:0/2438588184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ee81011e0 0x7f3ee819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:00.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.156+0000 7f3eed879640 1 -- 172.21.15.43:0/2438588184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee810e3e0 msgr2=0x7f3ee819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:00.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.156+0000 7f3eed879640 1 --2- 172.21.15.43:0/2438588184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee810e3e0 0x7f3ee819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.156+0000 7f3eed879640 1 -- 172.21.15.43:0/2438588184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ee81011e0 msgr2=0x7f3ee819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:00.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.156+0000 7f3eed879640 1 --2- 172.21.15.43:0/2438588184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ee81011e0 0x7f3ee819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.156+0000 7f3eed879640 1 -- 172.21.15.43:0/2438588184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee819f670 con 0x7f3ee81008c0 2024-09-11T23:22:00.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.156+0000 7f3eed078640 1 --2- 172.21.15.43:0/2438588184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ee81011e0 0x7f3ee819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:22:00.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.156+0000 7f3eee07a640 1 --2- 172.21.15.43:0/2438588184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee810e3e0 0x7f3ee819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:00.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.157+0000 7f3eed879640 1 --2- 172.21.15.43:0/2438588184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee81008c0 0x7f3ee819a680 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000d950 tx=0x7f3ee000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:00.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.157+0000 7f3ee6ffd640 1 -- 172.21.15.43:0/2438588184 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0017070 con 0x7f3ee81008c0 2024-09-11T23:22:00.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.157+0000 7f3ee6ffd640 1 -- 172.21.15.43:0/2438588184 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ee0009c30 con 0x7f3ee81008c0 2024-09-11T23:22:00.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.157+0000 7f3eefadd640 1 -- 172.21.15.43:0/2438588184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee819f960 con 0x7f3ee81008c0 2024-09-11T23:22:00.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.157+0000 7f3ee6ffd640 1 -- 172.21.15.43:0/2438588184 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee000f6b0 con 0x7f3ee81008c0 2024-09-11T23:22:00.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.157+0000 7f3eefadd640 1 -- 172.21.15.43:0/2438588184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee81a71f0 con 0x7f3ee81008c0 2024-09-11T23:22:00.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.158+0000 7f3eefadd640 1 -- 172.21.15.43:0/2438588184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee8100ca0 con 0x7f3ee81008c0 2024-09-11T23:22:00.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.159+0000 7f3ee6ffd640 1 -- 172.21.15.43:0/2438588184 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f3ee00109d0 con 0x7f3ee81008c0 2024-09-11T23:22:00.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.160+0000 7f3ee6ffd640 1 --2- 172.21.15.43:0/2438588184 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f3ed00a2e10 0x7f3ed00a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:00.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.160+0000 7f3eed078640 1 --2- 172.21.15.43:0/2438588184 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f3ed00a2e10 0x7f3ed00a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:00.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.160+0000 7f3ee6ffd640 1 -- 172.21.15.43:0/2438588184 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f3ee00cf430 con 0x7f3ee81008c0 2024-09-11T23:22:00.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.160+0000 7f3eed078640 1 --2- 172.21.15.43:0/2438588184 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f3ed00a2e10 0x7f3ed00a52d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3edc000c00 tx=0x7f3edc032c70 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:00.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.163+0000 7f3ee6ffd640 1 -- 172.21.15.43:0/2438588184 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ee0009de0 con 0x7f3ee81008c0 2024-09-11T23:22:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.362+0000 7f3eefadd640 1 -- 172.21.15.43:0/2438588184 --> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f3ee8062630 con 0x7f3ed00a2e10 2024-09-11T23:22:00.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.427+0000 7f3ee6ffd640 1 -- 172.21.15.43:0/2438588184 <== mgr.15126 v2:172.21.15.43:6800/2510386404 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3ee8062630 con 0x7f3ed00a2e10 2024-09-11T23:22:00.428 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path / 2024-09-11T23:22:00.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.429+0000 7f3eefadd640 1 -- 172.21.15.43:0/2438588184 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f3ed00a2e10 msgr2=0x7f3ed00a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:00.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.429+0000 7f3eefadd640 1 --2- 172.21.15.43:0/2438588184 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f3ed00a2e10 0x7f3ed00a52d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3edc000c00 tx=0x7f3edc032c70 comp rx=0 tx=0).stop 2024-09-11T23:22:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.429+0000 7f3eefadd640 1 -- 172.21.15.43:0/2438588184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee81008c0 msgr2=0x7f3ee819a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.429+0000 7f3eefadd640 1 --2- 172.21.15.43:0/2438588184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee81008c0 0x7f3ee819a680 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000d950 tx=0x7f3ee000de10 comp rx=0 tx=0).stop 2024-09-11T23:22:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.429+0000 7f3eefadd640 1 -- 172.21.15.43:0/2438588184 shutdown_connections 2024-09-11T23:22:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.429+0000 7f3eefadd640 1 --2- 172.21.15.43:0/2438588184 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f3ed00a2e10 0x7f3ed00a52d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.429+0000 7f3eefadd640 1 --2- 172.21.15.43:0/2438588184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ee810e3e0 0x7f3ee819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.429+0000 7f3eefadd640 1 --2- 172.21.15.43:0/2438588184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ee81011e0 0x7f3ee819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.429+0000 7f3eefadd640 1 --2- 172.21.15.43:0/2438588184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee81008c0 0x7f3ee819a680 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.429+0000 7f3eefadd640 1 -- 172.21.15.43:0/2438588184 >> 172.21.15.43:0/2438588184 conn(0x7f3ee80fc6c0 msgr2=0x7f3ee80feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.429+0000 7f3eefadd640 1 -- 172.21.15.43:0/2438588184 shutdown_connections 2024-09-11T23:22:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.429+0000 7f3eefadd640 1 -- 172.21.15.43:0/2438588184 wait complete. 2024-09-11T23:22:00.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-11T23:22:00.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.548+0000 7fb25c677640 1 -- 172.21.15.43:0/1489584696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb25410c0d0 msgr2=0x7fb25410e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:00.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.548+0000 7fb25c677640 1 --2- 172.21.15.43:0/1489584696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb25410c0d0 0x7fb25410e540 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009ca0 tx=0x7fb24c02a950 comp rx=0 tx=0).stop 2024-09-11T23:22:00.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.548+0000 7fb25c677640 1 -- 172.21.15.43:0/1489584696 shutdown_connections 2024-09-11T23:22:00.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.548+0000 7fb25c677640 1 --2- 172.21.15.43:0/1489584696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb25410c0d0 0x7fb25410e540 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.548+0000 7fb25c677640 1 --2- 172.21.15.43:0/1489584696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb254103620 0x7fb25410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.548+0000 7fb25c677640 1 --2- 172.21.15.43:0/1489584696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb254102d00 0x7fb2541030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.548+0000 7fb25c677640 1 -- 172.21.15.43:0/1489584696 >> 172.21.15.43:0/1489584696 conn(0x7fb2540779a0 msgr2=0x7fb2540fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:00.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.548+0000 7fb25c677640 1 -- 172.21.15.43:0/1489584696 shutdown_connections 2024-09-11T23:22:00.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.548+0000 7fb25c677640 1 -- 172.21.15.43:0/1489584696 wait complete. 2024-09-11T23:22:00.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.548+0000 7fb25c677640 1 Processor -- start 2024-09-11T23:22:00.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.548+0000 7fb25c677640 1 -- start start 2024-09-11T23:22:00.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb25c677640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb254102d00 0x7fb254198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:00.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb25c677640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb254103620 0x7fb2541988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:00.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb25c677640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb25410c0d0 0x7fb25419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:00.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb25c677640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb254111ea0 con 0x7fb254103620 2024-09-11T23:22:00.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb25c677640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb254111d20 con 0x7fb25410c0d0 2024-09-11T23:22:00.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb25c677640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb254112020 con 0x7fb254102d00 2024-09-11T23:22:00.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb259c12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb254103620 0x7fb2541988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:00.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb259c12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb254103620 0x7fb2541988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49826/0 (socket says 172.21.15.43:49826) 2024-09-11T23:22:00.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb259c12640 1 -- 172.21.15.43:0/3536175387 learned_addr learned my addr 172.21.15.43:0/3536175387 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:00.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb25ac14640 1 --2- 172.21.15.43:0/3536175387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb25410c0d0 0x7fb25419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb25a413640 1 --2- 172.21.15.43:0/3536175387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb254102d00 0x7fb254198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb259c12640 1 -- 172.21.15.43:0/3536175387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb254102d00 msgr2=0x7fb254198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb259c12640 1 --2- 172.21.15.43:0/3536175387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb254102d00 0x7fb254198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb259c12640 1 -- 172.21.15.43:0/3536175387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb25410c0d0 msgr2=0x7fb25419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb259c12640 1 --2- 172.21.15.43:0/3536175387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb25410c0d0 0x7fb25419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb259c12640 1 -- 172.21.15.43:0/3536175387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25419d380 con 0x7fb254103620 2024-09-11T23:22:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb25a413640 1 --2- 172.21.15.43:0/3536175387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb254102d00 0x7fb254198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:22:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb25ac14640 1 --2- 172.21.15.43:0/3536175387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb25410c0d0 0x7fb25419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb259c12640 1 --2- 172.21.15.43:0/3536175387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb254103620 0x7fb2541988d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb24400d770 tx=0x7fb24400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:00.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb24b7fe640 1 -- 172.21.15.43:0/3536175387 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb244017070 con 0x7fb254103620 2024-09-11T23:22:00.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb24b7fe640 1 -- 172.21.15.43:0/3536175387 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb244005500 con 0x7fb254103620 2024-09-11T23:22:00.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.549+0000 7fb24b7fe640 1 -- 172.21.15.43:0/3536175387 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb244004e00 con 0x7fb254103620 2024-09-11T23:22:00.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.550+0000 7fb25c677640 1 -- 172.21.15.43:0/3536175387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb25419d670 con 0x7fb254103620 2024-09-11T23:22:00.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.550+0000 7fb25c677640 1 -- 172.21.15.43:0/3536175387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2541a4f00 con 0x7fb254103620 2024-09-11T23:22:00.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.551+0000 7fb25c677640 1 -- 172.21.15.43:0/3536175387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb25406a260 con 0x7fb254103620 2024-09-11T23:22:00.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.552+0000 7fb24b7fe640 1 -- 172.21.15.43:0/3536175387 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fb244004fa0 con 0x7fb254103620 2024-09-11T23:22:00.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.552+0000 7fb24b7fe640 1 --2- 172.21.15.43:0/3536175387 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb2400a2e90 0x7fb2400a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:00.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.552+0000 7fb25a413640 1 --2- 172.21.15.43:0/3536175387 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb2400a2e90 0x7fb2400a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:00.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.553+0000 7fb24b7fe640 1 -- 172.21.15.43:0/3536175387 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb2440cf170 con 0x7fb254103620 2024-09-11T23:22:00.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.553+0000 7fb25a413640 1 --2- 172.21.15.43:0/3536175387 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb2400a2e90 0x7fb2400a5350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb2500058c0 tx=0x7fb250005d30 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:00.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.556+0000 7fb24b7fe640 1 -- 172.21.15.43:0/3536175387 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb24409bae0 con 0x7fb254103620 2024-09-11T23:22:00.704 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:00 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/206956693' entity='client.admin' 2024-09-11T23:22:00.704 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:00 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:22:00.704 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:00 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:22:00.704 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:00 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:22:00.704 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:00 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:22:00.704 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:00 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3028607516' entity='client.admin' 2024-09-11T23:22:00.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:00 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:22:00.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:00 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:22:00.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:00 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:22:00.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:00 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:22:00.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:00 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/621378119' entity='client.admin' 2024-09-11T23:22:00.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:00 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:22:00.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:00 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:22:00.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:00 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:22:00.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:00 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:22:00.730 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:22:00.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.758+0000 7fb25c677640 1 -- 172.21.15.43:0/3536175387 --> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fb254062630 con 0x7fb2400a2e90 2024-09-11T23:22:00.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:00 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/206956693' entity='client.admin' 2024-09-11T23:22:00.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:00 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:22:00.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:00 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:22:00.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:00 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:22:00.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:00 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:22:00.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:00 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3028607516' entity='client.admin' 2024-09-11T23:22:00.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:00 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:22:00.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:00 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:22:00.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:00 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:22:00.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:00 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:22:00.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:00 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/621378119' entity='client.admin' 2024-09-11T23:22:00.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:00 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:22:00.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:00 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:22:00.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:00 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:22:00.771 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:00 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:22:00.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:00 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/206956693' entity='client.admin' 2024-09-11T23:22:00.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:00 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:22:00.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:00 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:22:00.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:00 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:22:00.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:00 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:22:00.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:00 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3028607516' entity='client.admin' 2024-09-11T23:22:00.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:00 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:22:00.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:00 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:22:00.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:00 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:22:00.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:00 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:22:00.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:00 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/621378119' entity='client.admin' 2024-09-11T23:22:00.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:00 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:22:00.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:00 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:22:00.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:00 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:22:00.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:00 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:22:00.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.822+0000 7fb24b7fe640 1 -- 172.21.15.43:0/3536175387 <== mgr.15126 v2:172.21.15.43:6800/2510386404 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fb254062630 con 0x7fb2400a2e90 2024-09-11T23:22:00.823 INFO:teuthology.orchestra.run.smithi043.stdout:Retention added to path / 2024-09-11T23:22:00.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.824+0000 7fb25c677640 1 -- 172.21.15.43:0/3536175387 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb2400a2e90 msgr2=0x7fb2400a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:00.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.824+0000 7fb25c677640 1 --2- 172.21.15.43:0/3536175387 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb2400a2e90 0x7fb2400a5350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb2500058c0 tx=0x7fb250005d30 comp rx=0 tx=0).stop 2024-09-11T23:22:00.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.824+0000 7fb25c677640 1 -- 172.21.15.43:0/3536175387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb254103620 msgr2=0x7fb2541988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:00.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.824+0000 7fb25c677640 1 --2- 172.21.15.43:0/3536175387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb254103620 0x7fb2541988d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb24400d770 tx=0x7fb24400dc30 comp rx=0 tx=0).stop 2024-09-11T23:22:00.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.824+0000 7fb25c677640 1 -- 172.21.15.43:0/3536175387 shutdown_connections 2024-09-11T23:22:00.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.824+0000 7fb25c677640 1 --2- 172.21.15.43:0/3536175387 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb2400a2e90 0x7fb2400a5350 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.824+0000 7fb25c677640 1 --2- 172.21.15.43:0/3536175387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb25410c0d0 0x7fb25419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.824+0000 7fb25c677640 1 --2- 172.21.15.43:0/3536175387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb254103620 0x7fb2541988d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.824+0000 7fb25c677640 1 --2- 172.21.15.43:0/3536175387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb254102d00 0x7fb254198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.824+0000 7fb25c677640 1 -- 172.21.15.43:0/3536175387 >> 172.21.15.43:0/3536175387 conn(0x7fb2540779a0 msgr2=0x7fb2540fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:00.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.824+0000 7fb25c677640 1 -- 172.21.15.43:0/3536175387 shutdown_connections 2024-09-11T23:22:00.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.824+0000 7fb25c677640 1 -- 172.21.15.43:0/3536175387 wait complete. 2024-09-11T23:22:00.836 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-11T23:22:00.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.985+0000 7f9474a3c640 1 -- 172.21.15.43:0/2873545894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94701035c0 msgr2=0x7f947010baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:00.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.985+0000 7f9474a3c640 1 --2- 172.21.15.43:0/2873545894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94701035c0 0x7f947010baa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9464009900 tx=0x7f946402a480 comp rx=0 tx=0).stop 2024-09-11T23:22:00.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.985+0000 7f9474a3c640 1 -- 172.21.15.43:0/2873545894 shutdown_connections 2024-09-11T23:22:00.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.985+0000 7f9474a3c640 1 --2- 172.21.15.43:0/2873545894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f947010c0f0 0x7f947010e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.985+0000 7f9474a3c640 1 --2- 172.21.15.43:0/2873545894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94701035c0 0x7f947010baa0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.985+0000 7f9474a3c640 1 --2- 172.21.15.43:0/2873545894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9470102ca0 0x7f9470103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.985+0000 7f9474a3c640 1 -- 172.21.15.43:0/2873545894 >> 172.21.15.43:0/2873545894 conn(0x7f94700fc6b0 msgr2=0x7f94700fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.985+0000 7f9474a3c640 1 -- 172.21.15.43:0/2873545894 shutdown_connections 2024-09-11T23:22:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.985+0000 7f9474a3c640 1 -- 172.21.15.43:0/2873545894 wait complete. 2024-09-11T23:22:00.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.986+0000 7f9474a3c640 1 Processor -- start 2024-09-11T23:22:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.986+0000 7f9474a3c640 1 -- start start 2024-09-11T23:22:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.986+0000 7f9474a3c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9470102ca0 0x7f94701008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.986+0000 7f9474a3c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94701035c0 0x7f9470100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.986+0000 7f9474a3c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947010c0f0 0x7f9470101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.986+0000 7f9474a3c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9470111ed0 con 0x7f947010c0f0 2024-09-11T23:22:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.986+0000 7f9474a3c640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9470111d50 con 0x7f94701035c0 2024-09-11T23:22:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.986+0000 7f9474a3c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9470112050 con 0x7f9470102ca0 2024-09-11T23:22:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.986+0000 7f946ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947010c0f0 0x7f9470101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.986+0000 7f946ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947010c0f0 0x7f9470101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49844/0 (socket says 172.21.15.43:49844) 2024-09-11T23:22:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.986+0000 7f946ed9d640 1 -- 172.21.15.43:0/2218870773 learned_addr learned my addr 172.21.15.43:0/2218870773 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.986+0000 7f946e59c640 1 --2- 172.21.15.43:0/2218870773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9470102ca0 0x7f94701008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.986+0000 7f946dd9b640 1 --2- 172.21.15.43:0/2218870773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94701035c0 0x7f9470100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:00.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.986+0000 7f946ed9d640 1 -- 172.21.15.43:0/2218870773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9470102ca0 msgr2=0x7f94701008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:00.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.986+0000 7f946ed9d640 1 --2- 172.21.15.43:0/2218870773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9470102ca0 0x7f94701008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.986+0000 7f946ed9d640 1 -- 172.21.15.43:0/2218870773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94701035c0 msgr2=0x7f9470100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:00.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.986+0000 7f946ed9d640 1 --2- 172.21.15.43:0/2218870773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94701035c0 0x7f9470100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:00.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.986+0000 7f946ed9d640 1 -- 172.21.15.43:0/2218870773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94701a6950 con 0x7f947010c0f0 2024-09-11T23:22:00.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.987+0000 7f946ed9d640 1 --2- 172.21.15.43:0/2218870773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947010c0f0 0x7f9470101370 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f946800cad0 tx=0x7f946800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:00.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.987+0000 7f945f7fe640 1 -- 172.21.15.43:0/2218870773 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946800ae20 con 0x7f947010c0f0 2024-09-11T23:22:00.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.987+0000 7f945f7fe640 1 -- 172.21.15.43:0/2218870773 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9468005020 con 0x7f947010c0f0 2024-09-11T23:22:00.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.987+0000 7f9474a3c640 1 -- 172.21.15.43:0/2218870773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94701a6c40 con 0x7f947010c0f0 2024-09-11T23:22:00.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.987+0000 7f945f7fe640 1 -- 172.21.15.43:0/2218870773 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946800f6f0 con 0x7f947010c0f0 2024-09-11T23:22:00.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.987+0000 7f9474a3c640 1 -- 172.21.15.43:0/2218870773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94701a7110 con 0x7f947010c0f0 2024-09-11T23:22:00.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.988+0000 7f9474a3c640 1 -- 172.21.15.43:0/2218870773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9470103080 con 0x7f947010c0f0 2024-09-11T23:22:00.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.990+0000 7f945f7fe640 1 -- 172.21.15.43:0/2218870773 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f94680051d0 con 0x7f947010c0f0 2024-09-11T23:22:00.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.991+0000 7f945f7fe640 1 --2- 172.21.15.43:0/2218870773 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f94580a2e10 0x7f94580a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:00.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.991+0000 7f945f7fe640 1 -- 172.21.15.43:0/2218870773 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f94680cfa60 con 0x7f947010c0f0 2024-09-11T23:22:00.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.991+0000 7f946e59c640 1 --2- 172.21.15.43:0/2218870773 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f94580a2e10 0x7f94580a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:00.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.992+0000 7f946e59c640 1 --2- 172.21.15.43:0/2218870773 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f94580a2e10 0x7f94580a52d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f946000bbe0 tx=0x7f946000b390 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:00.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:00.994+0000 7f945f7fe640 1 -- 172.21.15.43:0/2218870773 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9468014030 con 0x7f947010c0f0 2024-09-11T23:22:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:01.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.194+0000 7f9474a3c640 1 -- 172.21.15.43:0/2218870773 --> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f9470062630 con 0x7f94580a2e10 2024-09-11T23:22:01.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.201+0000 7f945f7fe640 1 -- 172.21.15.43:0/2218870773 <== mgr.15126 v2:172.21.15.43:6800/2510386404 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f9470062630 con 0x7f94580a2e10 2024-09-11T23:22:01.202 INFO:teuthology.orchestra.run.smithi043.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-11T00:00:00", "created": "2024-09-11T23:22:00", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-11T23:22:01.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.203+0000 7f9474a3c640 1 -- 172.21.15.43:0/2218870773 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f94580a2e10 msgr2=0x7f94580a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:01.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.203+0000 7f9474a3c640 1 --2- 172.21.15.43:0/2218870773 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f94580a2e10 0x7f94580a52d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f946000bbe0 tx=0x7f946000b390 comp rx=0 tx=0).stop 2024-09-11T23:22:01.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.203+0000 7f9474a3c640 1 -- 172.21.15.43:0/2218870773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947010c0f0 msgr2=0x7f9470101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:01.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.203+0000 7f9474a3c640 1 --2- 172.21.15.43:0/2218870773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947010c0f0 0x7f9470101370 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f946800cad0 tx=0x7f946800cf90 comp rx=0 tx=0).stop 2024-09-11T23:22:01.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.203+0000 7f9474a3c640 1 -- 172.21.15.43:0/2218870773 shutdown_connections 2024-09-11T23:22:01.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.203+0000 7f9474a3c640 1 --2- 172.21.15.43:0/2218870773 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f94580a2e10 0x7f94580a52d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.203+0000 7f9474a3c640 1 --2- 172.21.15.43:0/2218870773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947010c0f0 0x7f9470101370 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.203+0000 7f9474a3c640 1 --2- 172.21.15.43:0/2218870773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94701035c0 0x7f9470100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.203+0000 7f9474a3c640 1 --2- 172.21.15.43:0/2218870773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9470102ca0 0x7f94701008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.203+0000 7f9474a3c640 1 -- 172.21.15.43:0/2218870773 >> 172.21.15.43:0/2218870773 conn(0x7f94700fc6b0 msgr2=0x7f94700fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:01.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.204+0000 7f9474a3c640 1 -- 172.21.15.43:0/2218870773 shutdown_connections 2024-09-11T23:22:01.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.204+0000 7f9474a3c640 1 -- 172.21.15.43:0/2218870773 wait complete. 2024-09-11T23:22:01.214 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-11T23:22:01.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.323+0000 7f236eb4b640 1 -- 172.21.15.43:0/2248827291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2368101370 msgr2=0x7f236810d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:01.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.323+0000 7f236eb4b640 1 --2- 172.21.15.43:0/2248827291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2368101370 0x7f236810d8a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2360009cd0 tx=0x7f236002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.323+0000 7f236eb4b640 1 -- 172.21.15.43:0/2248827291 shutdown_connections 2024-09-11T23:22:01.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.323+0000 7f236eb4b640 1 --2- 172.21.15.43:0/2248827291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2368101370 0x7f236810d8a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.323+0000 7f236eb4b640 1 --2- 172.21.15.43:0/2248827291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23681009d0 0x7f2368100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.323+0000 7f236eb4b640 1 --2- 172.21.15.43:0/2248827291 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23681069e0 0x7f2368106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.323+0000 7f236eb4b640 1 -- 172.21.15.43:0/2248827291 >> 172.21.15.43:0/2248827291 conn(0x7f23680fc6b0 msgr2=0x7f23680fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:01.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.323+0000 7f236eb4b640 1 -- 172.21.15.43:0/2248827291 shutdown_connections 2024-09-11T23:22:01.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.323+0000 7f236eb4b640 1 -- 172.21.15.43:0/2248827291 wait complete. 2024-09-11T23:22:01.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.324+0000 7f236eb4b640 1 Processor -- start 2024-09-11T23:22:01.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.324+0000 7f236eb4b640 1 -- start start 2024-09-11T23:22:01.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.324+0000 7f236eb4b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23681009d0 0x7f23681972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.324+0000 7f236eb4b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2368101370 0x7f23681977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.324+0000 7f236eb4b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23681069e0 0x7f236819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.324+0000 7f236eb4b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2368110e20 con 0x7f2368101370 2024-09-11T23:22:01.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.324+0000 7f236eb4b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2368110ca0 con 0x7f23681009d0 2024-09-11T23:22:01.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.324+0000 7f236eb4b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2368110fa0 con 0x7f23681069e0 2024-09-11T23:22:01.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.325+0000 7f2367fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2368101370 0x7f23681977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:01.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.325+0000 7f2367fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2368101370 0x7f23681977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49868/0 (socket says 172.21.15.43:49868) 2024-09-11T23:22:01.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.325+0000 7f2367fff640 1 -- 172.21.15.43:0/1047545738 learned_addr learned my addr 172.21.15.43:0/1047545738 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:01.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.325+0000 7f236d0e8640 1 --2- 172.21.15.43:0/1047545738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23681069e0 0x7f236819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:01.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.325+0000 7f2367fff640 1 -- 172.21.15.43:0/1047545738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23681069e0 msgr2=0x7f236819bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:01.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.325+0000 7f2367fff640 1 --2- 172.21.15.43:0/1047545738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23681069e0 0x7f236819bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.325+0000 7f2367fff640 1 -- 172.21.15.43:0/1047545738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23681009d0 msgr2=0x7f23681972b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:22:01.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.325+0000 7f2367fff640 1 --2- 172.21.15.43:0/1047545738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23681009d0 0x7f23681972b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.325+0000 7f2367fff640 1 -- 172.21.15.43:0/1047545738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236819c2a0 con 0x7f2368101370 2024-09-11T23:22:01.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.325+0000 7f2367fff640 1 --2- 172.21.15.43:0/1047545738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2368101370 0x7f23681977f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f235800e9f0 tx=0x7f235800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:01.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.325+0000 7f2365ffb640 1 -- 172.21.15.43:0/1047545738 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235800cd80 con 0x7f2368101370 2024-09-11T23:22:01.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.325+0000 7f2365ffb640 1 -- 172.21.15.43:0/1047545738 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2358005500 con 0x7f2368101370 2024-09-11T23:22:01.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.325+0000 7f236eb4b640 1 -- 172.21.15.43:0/1047545738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f236819c590 con 0x7f2368101370 2024-09-11T23:22:01.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.325+0000 7f2365ffb640 1 -- 172.21.15.43:0/1047545738 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2358010690 con 0x7f2368101370 2024-09-11T23:22:01.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.325+0000 7f236eb4b640 1 -- 172.21.15.43:0/1047545738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23681a3e20 con 0x7f2368101370 2024-09-11T23:22:01.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.327+0000 7f236eb4b640 1 -- 172.21.15.43:0/1047545738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2368100e30 con 0x7f2368101370 2024-09-11T23:22:01.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.328+0000 7f2365ffb640 1 -- 172.21.15.43:0/1047545738 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f23580046c0 con 0x7f2368101370 2024-09-11T23:22:01.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.328+0000 7f2365ffb640 1 --2- 172.21.15.43:0/1047545738 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f23540a2e10 0x7f23540a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.329+0000 7f236c8e7640 1 --2- 172.21.15.43:0/1047545738 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f23540a2e10 0x7f23540a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:01.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.329+0000 7f2365ffb640 1 -- 172.21.15.43:0/1047545738 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f23580cfc40 con 0x7f2368101370 2024-09-11T23:22:01.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.329+0000 7f236c8e7640 1 --2- 172.21.15.43:0/1047545738 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f23540a2e10 0x7f23540a52d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f23500099e0 tx=0x7f2350005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:01.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.332+0000 7f2365ffb640 1 -- 172.21.15.43:0/1047545738 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f235809c610 con 0x7f2368101370 2024-09-11T23:22:01.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.541+0000 7f236eb4b640 1 -- 172.21.15.43:0/1047545738 --> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f23680fd450 con 0x7f23540a2e10 2024-09-11T23:22:01.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.549+0000 7f2365ffb640 1 -- 172.21.15.43:0/1047545738 <== mgr.15126 v2:172.21.15.43:6800/2510386404 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f23680fd450 con 0x7f23540a2e10 2024-09-11T23:22:01.550 INFO:teuthology.orchestra.run.smithi043.stdout:/ 1m 6m3h 2024-09-11T23:22:01.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.485+0000 7fb5d2c90640 1 -- 172.21.15.43:0/435854925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5cc100e20 msgr2=0x7fb5cc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:01.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.485+0000 7fb5d2c90640 1 --2- 172.21.15.43:0/435854925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5cc100e20 0x7fb5cc101200 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8009900 tx=0x7fb5c802a480 comp rx=0 tx=0).stop 2024-09-11T23:22:01.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.485+0000 7fb5d2c90640 1 -- 172.21.15.43:0/435854925 shutdown_connections 2024-09-11T23:22:01.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.485+0000 7fb5d2c90640 1 --2- 172.21.15.43:0/435854925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc102ef0 0x7fb5cc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.485+0000 7fb5d2c90640 1 --2- 172.21.15.43:0/435854925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5cc102570 0x7fb5cc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.485+0000 7fb5d2c90640 1 --2- 172.21.15.43:0/435854925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5cc100e20 0x7fb5cc101200 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.485+0000 7fb5d2c90640 1 -- 172.21.15.43:0/435854925 >> 172.21.15.43:0/435854925 conn(0x7fb5cc0fc7d0 msgr2=0x7fb5cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:01.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.485+0000 7fb5d2c90640 1 -- 172.21.15.43:0/435854925 shutdown_connections 2024-09-11T23:22:01.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.485+0000 7fb5d2c90640 1 -- 172.21.15.43:0/435854925 wait complete. 2024-09-11T23:22:01.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.486+0000 7fb5d2c90640 1 Processor -- start 2024-09-11T23:22:01.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.486+0000 7fb5d2c90640 1 -- start start 2024-09-11T23:22:01.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.486+0000 7fb5d2c90640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5cc100e20 0x7fb5cc19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.486+0000 7fb5d2c90640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5cc102570 0x7fb5cc19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.486+0000 7fb5d2c90640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc102ef0 0x7fb5cc19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.486+0000 7fb5d2c90640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc110f80 con 0x7fb5cc100e20 2024-09-11T23:22:01.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.486+0000 7fb5d2c90640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc110e00 con 0x7fb5cc102570 2024-09-11T23:22:01.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.486+0000 7fb5d2c90640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc111100 con 0x7fb5cc102ef0 2024-09-11T23:22:01.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.486+0000 7fb5d0a2c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5cc100e20 0x7fb5cc19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:01.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.486+0000 7fb5d0a2c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5cc100e20 0x7fb5cc19a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49890/0 (socket says 172.21.15.43:49890) 2024-09-11T23:22:01.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.486+0000 7fb5d0a2c640 1 -- 172.21.15.43:0/1124650936 learned_addr learned my addr 172.21.15.43:0/1124650936 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:01.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.486+0000 7fb5d122d640 1 --2- 172.21.15.43:0/1124650936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc102ef0 0x7fb5cc19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:01.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.486+0000 7fb5c3fff640 1 --2- 172.21.15.43:0/1124650936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5cc102570 0x7fb5cc19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:01.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.486+0000 7fb5d0a2c640 1 -- 172.21.15.43:0/1124650936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc102ef0 msgr2=0x7fb5cc19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:01.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.486+0000 7fb5d0a2c640 1 --2- 172.21.15.43:0/1124650936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc102ef0 0x7fb5cc19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.486+0000 7fb5d0a2c640 1 -- 172.21.15.43:0/1124650936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5cc102570 msgr2=0x7fb5cc19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:01.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.486+0000 7fb5d0a2c640 1 --2- 172.21.15.43:0/1124650936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5cc102570 0x7fb5cc19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.486+0000 7fb5d0a2c640 1 -- 172.21.15.43:0/1124650936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5cc19f6a0 con 0x7fb5cc100e20 2024-09-11T23:22:01.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.486+0000 7fb5d0a2c640 1 --2- 172.21.15.43:0/1124650936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5cc100e20 0x7fb5cc19a6f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8006010 tx=0x7fb5c8003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:01.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.487+0000 7fb5c1ffb640 1 -- 172.21.15.43:0/1124650936 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c803f070 con 0x7fb5cc100e20 2024-09-11T23:22:01.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.487+0000 7fb5c1ffb640 1 -- 172.21.15.43:0/1124650936 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5c8031e30 con 0x7fb5cc100e20 2024-09-11T23:22:01.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.487+0000 7fb5c1ffb640 1 -- 172.21.15.43:0/1124650936 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8035030 con 0x7fb5cc100e20 2024-09-11T23:22:01.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.487+0000 7fb5d2c90640 1 -- 172.21.15.43:0/1124650936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5cc19f930 con 0x7fb5cc100e20 2024-09-11T23:22:01.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.487+0000 7fb5d2c90640 1 -- 172.21.15.43:0/1124650936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5cc1a7270 con 0x7fb5cc100e20 2024-09-11T23:22:01.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.489+0000 7fb5c1ffb640 1 -- 172.21.15.43:0/1124650936 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fb5c8042070 con 0x7fb5cc100e20 2024-09-11T23:22:01.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.490+0000 7fb5c1ffb640 1 --2- 172.21.15.43:0/1124650936 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb5b40a2e90 0x7fb5b40a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.490+0000 7fb5c3fff640 1 --2- 172.21.15.43:0/1124650936 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb5b40a2e90 0x7fb5b40a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:01.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.490+0000 7fb5c1ffb640 1 -- 172.21.15.43:0/1124650936 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb5c80efe50 con 0x7fb5cc100e20 2024-09-11T23:22:01.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.490+0000 7fb5c3fff640 1 --2- 172.21.15.43:0/1124650936 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb5b40a2e90 0x7fb5b40a5350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc104c10 tx=0x7fb5bc00a2d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:01.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.491+0000 7fb5d5069740 1 Processor -- start 2024-09-11T23:22:01.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.491+0000 7fb5d5069740 1 -- start start 2024-09-11T23:22:01.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.491+0000 7fb5d5069740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc47cae4a0 0x55bc47cae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.491+0000 7fb5d5069740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47caedc0 0x55bc47cb9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.491+0000 7fb5d5069740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47cb9680 0x55bc47cbba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.491+0000 7fb5d5069740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bc47c142e0 con 0x55bc47cb9680 2024-09-11T23:22:01.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.491+0000 7fb5d5069740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bc47bbbce0 con 0x55bc47caedc0 2024-09-11T23:22:01.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.491+0000 7fb5d5069740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bc47c102a0 con 0x55bc47cae4a0 2024-09-11T23:22:01.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.491+0000 7fb5d122d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47cb9680 0x55bc47cbba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:01.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.491+0000 7fb5d122d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47cb9680 0x55bc47cbba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49894/0 (socket says 172.21.15.43:49894) 2024-09-11T23:22:01.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.491+0000 7fb5d122d640 1 -- 172.21.15.43:0/2547880654 learned_addr learned my addr 172.21.15.43:0/2547880654 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:01.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.491+0000 7fb5d0a2c640 1 --2- 172.21.15.43:0/2547880654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc47cae4a0 0x55bc47cae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:01.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.491+0000 7fb5c3fff640 1 --2- 172.21.15.43:0/2547880654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47caedc0 0x55bc47cb9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:01.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.491+0000 7fb5d122d640 1 -- 172.21.15.43:0/2547880654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc47cae4a0 msgr2=0x55bc47cae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:01.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.491+0000 7fb5d122d640 1 --2- 172.21.15.43:0/2547880654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc47cae4a0 0x55bc47cae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.491+0000 7fb5d122d640 1 -- 172.21.15.43:0/2547880654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47caedc0 msgr2=0x55bc47cb9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:01.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.491+0000 7fb5d122d640 1 --2- 172.21.15.43:0/2547880654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47caedc0 0x55bc47cb9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.491+0000 7fb5d122d640 1 -- 172.21.15.43:0/2547880654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc47ba0440 con 0x55bc47cb9680 2024-09-11T23:22:01.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.491+0000 7fb5c3fff640 1 --2- 172.21.15.43:0/2547880654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47caedc0 0x55bc47cb9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:22:01.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.492+0000 7fb5d122d640 1 --2- 172.21.15.43:0/2547880654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47cb9680 0x55bc47cbba70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb5c400ae90 tx=0x7fb5c400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c5f745a2c5d9da server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:01.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.492+0000 7fb5b2ffd640 1 -- 172.21.15.43:0/2547880654 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4003b70 con 0x55bc47cb9680 2024-09-11T23:22:01.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.492+0000 7fb5b2ffd640 1 -- 172.21.15.43:0/2547880654 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5c4003d10 con 0x55bc47cb9680 2024-09-11T23:22:01.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.492+0000 7fb5b2ffd640 1 -- 172.21.15.43:0/2547880654 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4010c70 con 0x55bc47cb9680 2024-09-11T23:22:01.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.493+0000 7fb5d5069740 1 -- 172.21.15.43:0/2547880654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47cb9680 msgr2=0x55bc47cbba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:01.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.493+0000 7fb5d5069740 1 --2- 172.21.15.43:0/2547880654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47cb9680 0x55bc47cbba70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb5c400ae90 tx=0x7fb5c400f790 comp rx=0 tx=0).stop 2024-09-11T23:22:01.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.493+0000 7fb5d5069740 1 -- 172.21.15.43:0/2547880654 shutdown_connections 2024-09-11T23:22:01.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.493+0000 7fb5d5069740 1 --2- 172.21.15.43:0/2547880654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47cb9680 0x55bc47cbba70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.493+0000 7fb5d5069740 1 --2- 172.21.15.43:0/2547880654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47caedc0 0x55bc47cb9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.493+0000 7fb5d5069740 1 --2- 172.21.15.43:0/2547880654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc47cae4a0 0x55bc47cae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.493+0000 7fb5d5069740 1 -- 172.21.15.43:0/2547880654 >> 172.21.15.43:0/2547880654 conn(0x55bc47caaa30 msgr2=0x55bc47caae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:01.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.493+0000 7fb5d5069740 1 -- 172.21.15.43:0/2547880654 shutdown_connections 2024-09-11T23:22:01.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.493+0000 7fb5d5069740 1 -- 172.21.15.43:0/2547880654 wait complete. 2024-09-11T23:22:01.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.493+0000 7fb5d5069740 1 -- start start 2024-09-11T23:22:01.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.493+0000 7fb5d5069740 1 Processor -- start 2024-09-11T23:22:01.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.494+0000 7fb5d5069740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47cae4a0 0x55bc47cb8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.558 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:22:01.559 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:22:01.559 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-09-11T23:22:01.559 INFO:teuthology.orchestra.run.smithi043.stdout: "76241474-3f11-44ed-bdea-435862600675": { 2024-09-11T23:22:01.559 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-09-11T23:22:01.559 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "76241474-3f11-44ed-bdea-435862600675", 2024-09-11T23:22:01.559 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-09-11T23:22:01.559 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:22:01.560 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:22:01.560 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:22:01.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.494+0000 7fb5d5069740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47caedc0 0x55bc47cb6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.494+0000 7fb5d5069740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc47cb9680 0x55bc47cb6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.494+0000 7fb5d5069740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bc47cbc250 con 0x55bc47cae4a0 2024-09-11T23:22:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.494+0000 7fb5d5069740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bc47bcf050 con 0x55bc47caedc0 2024-09-11T23:22:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.494+0000 7fb5d5069740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bc47cbc3d0 con 0x55bc47cb9680 2024-09-11T23:22:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.494+0000 7fb5d5069740 10 client.0 _ll_register_callbacks cb 0x55bc47c0f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.494+0000 7fb5d0a2c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47cae4a0 0x55bc47cb8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.494+0000 7fb5d0a2c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47cae4a0 0x55bc47cb8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49902/0 (socket says 172.21.15.43:49902) 2024-09-11T23:22:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.494+0000 7fb5d0a2c640 1 -- 172.21.15.43:0/313709012 learned_addr learned my addr 172.21.15.43:0/313709012 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.494+0000 7fb5d122d640 1 --2- 172.21.15.43:0/313709012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc47cb9680 0x55bc47cb6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.494+0000 7fb5c3fff640 1 --2- 172.21.15.43:0/313709012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47caedc0 0x55bc47cb6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.494+0000 7fb5d0a2c640 1 -- 172.21.15.43:0/313709012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc47cb9680 msgr2=0x55bc47cb6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.494+0000 7fb5d0a2c640 1 --2- 172.21.15.43:0/313709012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc47cb9680 0x55bc47cb6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.494+0000 7fb5d0a2c640 1 -- 172.21.15.43:0/313709012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47caedc0 msgr2=0x55bc47cb6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.494+0000 7fb5d0a2c640 1 --2- 172.21.15.43:0/313709012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47caedc0 0x55bc47cb6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.494+0000 7fb5d0a2c640 1 -- 172.21.15.43:0/313709012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc479853c0 con 0x55bc47cae4a0 2024-09-11T23:22:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.494+0000 7fb5c3fff640 1 --2- 172.21.15.43:0/313709012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47caedc0 0x55bc47cb6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.494+0000 7fb5d0a2c640 1 --2- 172.21.15.43:0/313709012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47cae4a0 0x55bc47cb8fa0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80bbfa0 tx=0x7fb5c80bc160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25fe14900033471 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.494+0000 7fb5abfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.495+0000 7fb5abfff640 1 -- 172.21.15.43:0/313709012 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8049040 con 0x55bc47cae4a0 2024-09-11T23:22:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.495+0000 7fb5abfff640 1 -- 172.21.15.43:0/313709012 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb5c80c0120 con 0x55bc47cae4a0 2024-09-11T23:22:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.495+0000 7fb5d5069740 1 -- 172.21.15.43:0/313709012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5c4003d10 con 0x55bc47cae4a0 2024-09-11T23:22:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.495+0000 7fb5abfff640 1 -- 172.21.15.43:0/313709012 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb5c80c05a0 con 0x55bc47cae4a0 2024-09-11T23:22:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.495+0000 7fb5d5069740 10 client.15321 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.495+0000 7fb5d5069740 1 -- 172.21.15.43:0/313709012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bc47cc7b40 con 0x55bc47cae4a0 2024-09-11T23:22:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.495+0000 7fb5abfff640 1 -- 172.21.15.43:0/313709012 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5c4003d10 con 0x55bc47cae4a0 2024-09-11T23:22:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.495+0000 7fb5abfff640 1 -- 172.21.15.43:0/313709012 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb5c80dc070 con 0x55bc47cae4a0 2024-09-11T23:22:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.495+0000 7fb5d5069740 10 client.15321 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.495+0000 7fb5d5069740 10 client.15321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.495+0000 7fb5d5069740 20 client.15321 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.495+0000 7fb5d5069740 1 --2- 172.21.15.43:0/313709012 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bc47cc9040 0x55bc47ccb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.495+0000 7fb5d5069740 4 client.15321 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:22:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.495+0000 7fb5d5069740 1 -- 172.21.15.43:0/313709012 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc47c21850 con 0x55bc47cc9040 2024-09-11T23:22:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.496+0000 7fb5c3fff640 1 --2- 172.21.15.43:0/313709012 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bc47cc9040 0x55bc47ccb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.496+0000 7fb5abfff640 10 client.15321 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.496+0000 7fb5c3fff640 1 --2- 172.21.15.43:0/313709012 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bc47cc9040 0x55bc47ccb490 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4054df2386b8f359 server_cookie=e3d8237376a215bc in_seq=0 out_seq=0 2024-09-11T23:22:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.497+0000 7fb5abfff640 1 -- 172.21.15.43:0/313709012 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc47c21850 con 0x55bc47cc9040 2024-09-11T23:22:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.497+0000 7fb5abfff640 10 client.15321 handle_command_reply: tid=0 2024-09-11T23:22:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.497+0000 7fb5d5069740 1 client.15321 shutdown 2024-09-11T23:22:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.497+0000 7fb5d5069740 1 -- 172.21.15.43:0/313709012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47cae4a0 msgr2=0x55bc47cb8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.497+0000 7fb5d5069740 1 --2- 172.21.15.43:0/313709012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47cae4a0 0x55bc47cb8fa0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80bbfa0 tx=0x7fb5c80bc160 comp rx=0 tx=0).stop 2024-09-11T23:22:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.497+0000 7fb5d5069740 1 -- 172.21.15.43:0/313709012 shutdown_connections 2024-09-11T23:22:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.497+0000 7fb5d5069740 1 --2- 172.21.15.43:0/313709012 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bc47cc9040 0x55bc47ccb490 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.497+0000 7fb5d5069740 1 --2- 172.21.15.43:0/313709012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc47cb9680 0x55bc47cb6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.497+0000 7fb5d5069740 1 --2- 172.21.15.43:0/313709012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47caedc0 0x55bc47cb6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.497+0000 7fb5abfff640 0 client.15321 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.497+0000 7fb5d5069740 1 --2- 172.21.15.43:0/313709012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47cae4a0 0x55bc47cb8fa0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.497+0000 7fb5d5069740 1 -- 172.21.15.43:0/313709012 >> 172.21.15.43:0/313709012 conn(0x55bc47caaa30 msgr2=0x55bc47cb0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.497+0000 7fb5d5069740 1 -- 172.21.15.43:0/313709012 shutdown_connections 2024-09-11T23:22:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.497+0000 7fb5d5069740 1 -- 172.21.15.43:0/313709012 wait complete. 2024-09-11T23:22:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.497+0000 7fb5d5069740 20 client.15321 trim_cache size 0 max 16384 2024-09-11T23:22:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.515+0000 7fb5d5069740 1 Processor -- start 2024-09-11T23:22:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.515+0000 7fb5d5069740 1 -- start start 2024-09-11T23:22:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.515+0000 7fb5d5069740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47caaa30 0x55bc47ca9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.515+0000 7fb5d5069740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc47cae4a0 0x55bc47cd0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.515+0000 7fb5d5069740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47caedc0 0x55bc47cd12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.515+0000 7fb5d5069740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc1b23e0 con 0x55bc47caedc0 2024-09-11T23:22:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.515+0000 7fb5d5069740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc1afcb0 con 0x55bc47caaa30 2024-09-11T23:22:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.515+0000 7fb5d5069740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc1adb00 con 0x55bc47cae4a0 2024-09-11T23:22:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.515+0000 7fb5d122d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47caedc0 0x55bc47cd12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.515+0000 7fb5d122d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47caedc0 0x55bc47cd12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49918/0 (socket says 172.21.15.43:49918) 2024-09-11T23:22:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.515+0000 7fb5d122d640 1 -- 172.21.15.43:0/1238099278 learned_addr learned my addr 172.21.15.43:0/1238099278 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.515+0000 7fb5d0a2c640 1 --2- 172.21.15.43:0/1238099278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47caaa30 0x55bc47ca9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.515+0000 7fb5c3fff640 1 --2- 172.21.15.43:0/1238099278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc47cae4a0 0x55bc47cd0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.515+0000 7fb5d122d640 1 -- 172.21.15.43:0/1238099278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc47cae4a0 msgr2=0x55bc47cd0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:01.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.515+0000 7fb5d122d640 1 --2- 172.21.15.43:0/1238099278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc47cae4a0 0x55bc47cd0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.515+0000 7fb5d122d640 1 -- 172.21.15.43:0/1238099278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47caaa30 msgr2=0x55bc47ca9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:01.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.515+0000 7fb5d122d640 1 --2- 172.21.15.43:0/1238099278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47caaa30 0x55bc47ca9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.515+0000 7fb5d122d640 1 -- 172.21.15.43:0/1238099278 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc47caebf0 con 0x55bc47caedc0 2024-09-11T23:22:01.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.515+0000 7fb5c3fff640 1 --2- 172.21.15.43:0/1238099278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc47cae4a0 0x55bc47cd0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:22:01.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.516+0000 7fb5d122d640 1 --2- 172.21.15.43:0/1238099278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47caedc0 0x55bc47cd12e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55bc47cabdb0 tx=0x7fb5c4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74b569be91b95f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:01.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.516+0000 7fb5b27fc640 1 -- 172.21.15.43:0/1238099278 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c401d040 con 0x55bc47caedc0 2024-09-11T23:22:01.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.516+0000 7fb5b27fc640 1 -- 172.21.15.43:0/1238099278 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5c4027070 con 0x55bc47caedc0 2024-09-11T23:22:01.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.516+0000 7fb5b27fc640 1 -- 172.21.15.43:0/1238099278 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c40052b0 con 0x55bc47caedc0 2024-09-11T23:22:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.517+0000 7fb5d5069740 1 -- 172.21.15.43:0/1238099278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47caedc0 msgr2=0x55bc47cd12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.517+0000 7fb5d5069740 1 --2- 172.21.15.43:0/1238099278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47caedc0 0x55bc47cd12e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55bc47cabdb0 tx=0x7fb5c4005bd0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.517+0000 7fb5d5069740 1 -- 172.21.15.43:0/1238099278 shutdown_connections 2024-09-11T23:22:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.517+0000 7fb5d5069740 1 --2- 172.21.15.43:0/1238099278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47caedc0 0x55bc47cd12e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.517+0000 7fb5d5069740 1 --2- 172.21.15.43:0/1238099278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc47cae4a0 0x55bc47cd0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.517+0000 7fb5d5069740 1 --2- 172.21.15.43:0/1238099278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47caaa30 0x55bc47ca9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.517+0000 7fb5d5069740 1 -- 172.21.15.43:0/1238099278 >> 172.21.15.43:0/1238099278 conn(0x55bc47cb68f0 msgr2=0x55bc47ca9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.517+0000 7fb5d5069740 1 -- 172.21.15.43:0/1238099278 shutdown_connections 2024-09-11T23:22:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.517+0000 7fb5d5069740 1 -- 172.21.15.43:0/1238099278 wait complete. 2024-09-11T23:22:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.517+0000 7fb5d5069740 1 -- start start 2024-09-11T23:22:01.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.517+0000 7fb5d5069740 1 Processor -- start 2024-09-11T23:22:01.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.518+0000 7fb5d5069740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47caaa30 0x55bc47cc1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.518+0000 7fb5d5069740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47cae4a0 0x55bc47cbedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.518+0000 7fb5d5069740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc47caedc0 0x55bc47cbf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.518+0000 7fb5d5069740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc1ae170 con 0x55bc47caaa30 2024-09-11T23:22:01.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.518+0000 7fb5d5069740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc1af5e0 con 0x55bc47cae4a0 2024-09-11T23:22:01.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.518+0000 7fb5d5069740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc1ae840 con 0x55bc47caedc0 2024-09-11T23:22:01.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.518+0000 7fb5d5069740 10 client.0 _ll_register_callbacks cb 0x7fb5c8005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:01.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.518+0000 7fb5d0a2c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47caaa30 0x55bc47cc1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:01.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.518+0000 7fb5d0a2c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47caaa30 0x55bc47cc1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49920/0 (socket says 172.21.15.43:49920) 2024-09-11T23:22:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.518+0000 7fb5d0a2c640 1 -- 172.21.15.43:0/542649024 learned_addr learned my addr 172.21.15.43:0/542649024 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.518+0000 7fb5c3fff640 1 --2- 172.21.15.43:0/542649024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47cae4a0 0x55bc47cbedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.518+0000 7fb5d122d640 1 --2- 172.21.15.43:0/542649024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc47caedc0 0x55bc47cbf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.518+0000 7fb5d0a2c640 1 -- 172.21.15.43:0/542649024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc47caedc0 msgr2=0x55bc47cbf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.518+0000 7fb5d0a2c640 1 --2- 172.21.15.43:0/542649024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc47caedc0 0x55bc47cbf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.518+0000 7fb5d0a2c640 1 -- 172.21.15.43:0/542649024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47cae4a0 msgr2=0x55bc47cbedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.518+0000 7fb5d0a2c640 1 --2- 172.21.15.43:0/542649024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47cae4a0 0x55bc47cbedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.518+0000 7fb5d0a2c640 1 -- 172.21.15.43:0/542649024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc47cbf870 con 0x55bc47caaa30 2024-09-11T23:22:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.519+0000 7fb5d0a2c640 1 --2- 172.21.15.43:0/542649024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47caaa30 0x55bc47cc1aa0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8005ab0 tx=0x7fb5c803b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba73fa473ecf470 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.519+0000 7fb5abfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.519+0000 7fb5abfff640 1 -- 172.21.15.43:0/542649024 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8049070 con 0x55bc47caaa30 2024-09-11T23:22:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.519+0000 7fb5abfff640 1 -- 172.21.15.43:0/542649024 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb5c80c0320 con 0x55bc47caaa30 2024-09-11T23:22:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.519+0000 7fb5d5069740 1 -- 172.21.15.43:0/542649024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5c4027070 con 0x55bc47caaa30 2024-09-11T23:22:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.519+0000 7fb5abfff640 1 -- 172.21.15.43:0/542649024 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb5c80b8540 con 0x55bc47caaa30 2024-09-11T23:22:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.519+0000 7fb5d5069740 10 client.15327 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.519+0000 7fb5d5069740 1 -- 172.21.15.43:0/542649024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bc47cbf870 con 0x55bc47caaa30 2024-09-11T23:22:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.519+0000 7fb5abfff640 1 -- 172.21.15.43:0/542649024 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5c4027070 con 0x55bc47caaa30 2024-09-11T23:22:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.519+0000 7fb5abfff640 1 -- 172.21.15.43:0/542649024 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb5c80ed770 con 0x55bc47caaa30 2024-09-11T23:22:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.519+0000 7fb5d5069740 10 client.15327 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.519+0000 7fb5d5069740 10 client.15327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.519+0000 7fb5d5069740 20 client.15327 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.519+0000 7fb5d5069740 1 --2- 172.21.15.43:0/542649024 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bc47cb9680 0x55bc47d35e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.519+0000 7fb5d5069740 4 client.15327 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:22:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.519+0000 7fb5d5069740 1 -- 172.21.15.43:0/542649024 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bc47c1bd60 con 0x55bc47cb9680 2024-09-11T23:22:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.520+0000 7fb5c3fff640 1 --2- 172.21.15.43:0/542649024 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bc47cb9680 0x55bc47d35e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.520+0000 7fb5c3fff640 1 --2- 172.21.15.43:0/542649024 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bc47cb9680 0x55bc47d35e60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1455b79bbdea24b server_cookie=1b36adc4d8bbafd in_seq=0 out_seq=0 2024-09-11T23:22:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.520+0000 7fb5abfff640 10 client.15327 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:01.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.520+0000 7fb5abfff640 1 -- 172.21.15.43:0/542649024 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bc47c1bd60 con 0x55bc47cb9680 2024-09-11T23:22:01.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.520+0000 7fb5abfff640 10 client.15327 handle_command_reply: tid=0 2024-09-11T23:22:01.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.520+0000 7fb5d5069740 1 client.15327 shutdown 2024-09-11T23:22:01.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.520+0000 7fb5d5069740 1 -- 172.21.15.43:0/542649024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47caaa30 msgr2=0x55bc47cc1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:01.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.520+0000 7fb5d5069740 1 --2- 172.21.15.43:0/542649024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47caaa30 0x55bc47cc1aa0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8005ab0 tx=0x7fb5c803b990 comp rx=0 tx=0).stop 2024-09-11T23:22:01.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.520+0000 7fb5d5069740 1 -- 172.21.15.43:0/542649024 shutdown_connections 2024-09-11T23:22:01.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.520+0000 7fb5d5069740 1 --2- 172.21.15.43:0/542649024 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bc47cb9680 0x55bc47d35e60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.520+0000 7fb5d5069740 1 --2- 172.21.15.43:0/542649024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc47caedc0 0x55bc47cbf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.520+0000 7fb5abfff640 0 client.15327 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:01.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.520+0000 7fb5d5069740 1 --2- 172.21.15.43:0/542649024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc47cae4a0 0x55bc47cbedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.520+0000 7fb5d5069740 1 --2- 172.21.15.43:0/542649024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bc47caaa30 0x55bc47cc1aa0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.520+0000 7fb5d5069740 1 -- 172.21.15.43:0/542649024 >> 172.21.15.43:0/542649024 conn(0x55bc47cb68f0 msgr2=0x55bc47ca9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:01.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.521+0000 7fb5d5069740 1 -- 172.21.15.43:0/542649024 shutdown_connections 2024-09-11T23:22:01.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.521+0000 7fb5d5069740 1 -- 172.21.15.43:0/542649024 wait complete. 2024-09-11T23:22:01.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.521+0000 7fb5d5069740 20 client.15327 trim_cache size 0 max 16384 2024-09-11T23:22:01.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.521+0000 7fb5d2c90640 1 -- 172.21.15.43:0/1124650936 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb5b40a2e90 msgr2=0x7fb5b40a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:01.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.521+0000 7fb5d2c90640 1 --2- 172.21.15.43:0/1124650936 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb5b40a2e90 0x7fb5b40a5350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc104c10 tx=0x7fb5bc00a2d0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.521+0000 7fb5d2c90640 1 -- 172.21.15.43:0/1124650936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5cc100e20 msgr2=0x7fb5cc19a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:01.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.521+0000 7fb5d2c90640 1 --2- 172.21.15.43:0/1124650936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5cc100e20 0x7fb5cc19a6f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8006010 tx=0x7fb5c8003c80 comp rx=0 tx=0).stop 2024-09-11T23:22:01.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.522+0000 7fb5d2c90640 1 -- 172.21.15.43:0/1124650936 shutdown_connections 2024-09-11T23:22:01.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.522+0000 7fb5d2c90640 1 --2- 172.21.15.43:0/1124650936 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb5b40a2e90 0x7fb5b40a5350 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.522+0000 7fb5d2c90640 1 --2- 172.21.15.43:0/1124650936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc102ef0 0x7fb5cc19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.522+0000 7fb5d2c90640 1 --2- 172.21.15.43:0/1124650936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5cc102570 0x7fb5cc19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.522+0000 7fb5d2c90640 1 --2- 172.21.15.43:0/1124650936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5cc100e20 0x7fb5cc19a6f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.522+0000 7fb5d2c90640 1 -- 172.21.15.43:0/1124650936 >> 172.21.15.43:0/1124650936 conn(0x7fb5cc0fc7d0 msgr2=0x7fb5cc06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:01.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.522+0000 7fb5d2c90640 1 -- 172.21.15.43:0/1124650936 shutdown_connections 2024-09-11T23:22:01.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.522+0000 7fb5d2c90640 1 -- 172.21.15.43:0/1124650936 wait complete. 2024-09-11T23:22:01.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.552+0000 7f236eb4b640 1 -- 172.21.15.43:0/1047545738 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f23540a2e10 msgr2=0x7f23540a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:01.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.552+0000 7f236eb4b640 1 --2- 172.21.15.43:0/1047545738 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f23540a2e10 0x7f23540a52d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f23500099e0 tx=0x7f2350005f90 comp rx=0 tx=0).stop 2024-09-11T23:22:01.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.552+0000 7f236eb4b640 1 -- 172.21.15.43:0/1047545738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2368101370 msgr2=0x7f23681977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:01.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.552+0000 7f236eb4b640 1 --2- 172.21.15.43:0/1047545738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2368101370 0x7f23681977f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f235800e9f0 tx=0x7f235800edf0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.552+0000 7f236eb4b640 1 -- 172.21.15.43:0/1047545738 shutdown_connections 2024-09-11T23:22:01.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.552+0000 7f236eb4b640 1 --2- 172.21.15.43:0/1047545738 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f23540a2e10 0x7f23540a52d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.552+0000 7f236eb4b640 1 --2- 172.21.15.43:0/1047545738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23681069e0 0x7f236819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.552+0000 7f236eb4b640 1 --2- 172.21.15.43:0/1047545738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2368101370 0x7f23681977f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.552+0000 7f236eb4b640 1 --2- 172.21.15.43:0/1047545738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23681009d0 0x7f23681972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:01.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.552+0000 7f236eb4b640 1 -- 172.21.15.43:0/1047545738 >> 172.21.15.43:0/1047545738 conn(0x7f23680fc6b0 msgr2=0x7f23680fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:01.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.553+0000 7f236eb4b640 1 -- 172.21.15.43:0/1047545738 shutdown_connections 2024-09-11T23:22:01.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:01.553+0000 7f236eb4b640 1 -- 172.21.15.43:0/1047545738 wait complete. 2024-09-11T23:22:01.577 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-11T23:22:01.605 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-11T23:22:01.615 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-11T23:22:01.659 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-09-11T23:22:01.659 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-11T23:22:01.662 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-11T23:22:01.696 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T23:22:01.696 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:22:01.697 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:01 smithi043 ceph-mon[30950]: scrub queued for path: / 2024-09-11T23:22:01.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:01 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:22:01.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:01 smithi043 ceph-mon[30950]: scrub summary: active paths [/] 2024-09-11T23:22:01.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:01 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:22:01.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:01 smithi043 ceph-mon[30950]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 683 B/s wr, 17 op/s 2024-09-11T23:22:01.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:01 smithi043 ceph-mon[30950]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:22:01.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:01 smithi043 ceph-mon[30950]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:22:01.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:01 smithi090 ceph-mon[35130]: scrub queued for path: / 2024-09-11T23:22:01.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:01 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:22:01.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:01 smithi090 ceph-mon[35130]: scrub summary: active paths [/] 2024-09-11T23:22:01.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:01 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:22:01.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:01 smithi090 ceph-mon[35130]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 683 B/s wr, 17 op/s 2024-09-11T23:22:01.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:01 smithi090 ceph-mon[35130]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:22:01.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:01 smithi090 ceph-mon[35130]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:22:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:01 smithi097 ceph-mon[35562]: scrub queued for path: / 2024-09-11T23:22:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:01 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:22:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:01 smithi097 ceph-mon[35562]: scrub summary: active paths [/] 2024-09-11T23:22:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:01 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:22:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:01 smithi097 ceph-mon[35562]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 683 B/s wr, 17 op/s 2024-09-11T23:22:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:01 smithi097 ceph-mon[35562]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:22:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:01 smithi097 ceph-mon[35562]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:22:02.342 INFO:tasks.cephfs.filesystem:scrub status for tag:76241474-3f11-44ed-bdea-435862600675 - {'path': '/', 'tag': '76241474-3f11-44ed-bdea-435862600675', 'options': 'recursive,force'} 2024-09-11T23:22:02.388 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:02.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:02 smithi043 ceph-mon[30950]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:22:02.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:02 smithi043 ceph-mon[30950]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:22:02.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:02 smithi090 ceph-mon[35130]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:22:02.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:02 smithi090 ceph-mon[35130]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:22:02.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:02 smithi097 ceph-mon[35562]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:22:02.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:02 smithi097 ceph-mon[35562]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:22:02.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.908+0000 7f27d3923640 1 -- 172.21.15.43:0/3609302194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc102de0 msgr2=0x7f27cc1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:02.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.908+0000 7f27d3923640 1 --2- 172.21.15.43:0/3609302194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc102de0 0x7f27cc1031c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009900 tx=0x7f27c802a480 comp rx=0 tx=0).stop 2024-09-11T23:22:02.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.908+0000 7f27d3923640 1 -- 172.21.15.43:0/3609302194 shutdown_connections 2024-09-11T23:22:02.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.908+0000 7f27d3923640 1 --2- 172.21.15.43:0/3609302194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27cc10c180 0x7f27cc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:02.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.908+0000 7f27d3923640 1 --2- 172.21.15.43:0/3609302194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27cc103700 0x7f27cc10bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:02.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.908+0000 7f27d3923640 1 --2- 172.21.15.43:0/3609302194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc102de0 0x7f27cc1031c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:02.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.908+0000 7f27d3923640 1 -- 172.21.15.43:0/3609302194 >> 172.21.15.43:0/3609302194 conn(0x7f27cc0fc7a0 msgr2=0x7f27cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:02.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.908+0000 7f27d3923640 1 -- 172.21.15.43:0/3609302194 shutdown_connections 2024-09-11T23:22:02.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.908+0000 7f27d3923640 1 -- 172.21.15.43:0/3609302194 wait complete. 2024-09-11T23:22:02.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.909+0000 7f27d3923640 1 Processor -- start 2024-09-11T23:22:02.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.909+0000 7f27d3923640 1 -- start start 2024-09-11T23:22:02.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.909+0000 7f27d3923640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc102de0 0x7f27cc198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:02.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.909+0000 7f27d3923640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27cc103700 0x7f27cc198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:02.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.909+0000 7f27d3923640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27cc10c180 0x7f27cc19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:02.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.909+0000 7f27d3923640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc111f70 con 0x7f27cc102de0 2024-09-11T23:22:02.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.909+0000 7f27d3923640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc111df0 con 0x7f27cc10c180 2024-09-11T23:22:02.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.909+0000 7f27d3923640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc1120f0 con 0x7f27cc103700 2024-09-11T23:22:02.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.909+0000 7f27d16bf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc102de0 0x7f27cc198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:02.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.909+0000 7f27d16bf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc102de0 0x7f27cc198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49934/0 (socket says 172.21.15.43:49934) 2024-09-11T23:22:02.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.909+0000 7f27d16bf640 1 -- 172.21.15.43:0/1741269639 learned_addr learned my addr 172.21.15.43:0/1741269639 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:02.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.909+0000 7f27d0ebe640 1 --2- 172.21.15.43:0/1741269639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27cc103700 0x7f27cc198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:02.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.909+0000 7f27d1ec0640 1 --2- 172.21.15.43:0/1741269639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27cc10c180 0x7f27cc19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:02.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.909+0000 7f27d16bf640 1 -- 172.21.15.43:0/1741269639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27cc103700 msgr2=0x7f27cc198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:02.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.909+0000 7f27d16bf640 1 --2- 172.21.15.43:0/1741269639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27cc103700 0x7f27cc198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:02.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.909+0000 7f27d16bf640 1 -- 172.21.15.43:0/1741269639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27cc10c180 msgr2=0x7f27cc19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:02.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.909+0000 7f27d16bf640 1 --2- 172.21.15.43:0/1741269639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27cc10c180 0x7f27cc19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:02.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.910+0000 7f27d16bf640 1 -- 172.21.15.43:0/1741269639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27cc19d3c0 con 0x7f27cc102de0 2024-09-11T23:22:02.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.910+0000 7f27d16bf640 1 --2- 172.21.15.43:0/1741269639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc102de0 0x7f27cc198410 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f27c8005ab0 tx=0x7f27c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:02.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.910+0000 7f27c27fc640 1 -- 172.21.15.43:0/1741269639 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c803f030 con 0x7f27cc102de0 2024-09-11T23:22:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.910+0000 7f27c27fc640 1 -- 172.21.15.43:0/1741269639 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27c8003ce0 con 0x7f27cc102de0 2024-09-11T23:22:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.910+0000 7f27d3923640 1 -- 172.21.15.43:0/1741269639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27cc19d650 con 0x7f27cc102de0 2024-09-11T23:22:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.910+0000 7f27c27fc640 1 -- 172.21.15.43:0/1741269639 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8035070 con 0x7f27cc102de0 2024-09-11T23:22:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.910+0000 7f27d3923640 1 -- 172.21.15.43:0/1741269639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27cc1a4f90 con 0x7f27cc102de0 2024-09-11T23:22:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.911+0000 7f27d3923640 1 -- 172.21.15.43:0/1741269639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27cc06afd0 con 0x7f27cc102de0 2024-09-11T23:22:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.912+0000 7f27c27fc640 1 -- 172.21.15.43:0/1741269639 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f27c8042030 con 0x7f27cc102de0 2024-09-11T23:22:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.913+0000 7f27c27fc640 1 --2- 172.21.15.43:0/1741269639 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f27b80a2e10 0x7f27b80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.913+0000 7f27d0ebe640 1 --2- 172.21.15.43:0/1741269639 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f27b80a2e10 0x7f27b80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.913+0000 7f27c27fc640 1 -- 172.21.15.43:0/1741269639 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f27c80efff0 con 0x7f27cc102de0 2024-09-11T23:22:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.913+0000 7f27d0ebe640 1 --2- 172.21.15.43:0/1741269639 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f27b80a2e10 0x7f27b80a52d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f27cc10d5b0 tx=0x7f27bc00a4c0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:02.917+0000 7f27c27fc640 1 -- 172.21.15.43:0/1741269639 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f27c80bca90 con 0x7f27cc102de0 2024-09-11T23:22:03.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:03.090+0000 7f27d3923640 1 -- 172.21.15.43:0/1741269639 --> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f27cc062630 con 0x7f27b80a2e10 2024-09-11T23:22:03.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:03.099+0000 7f27c27fc640 1 -- 172.21.15.43:0/1741269639 <== mgr.15126 v2:172.21.15.43:6800/2510386404 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258039 (secure 0 0 0) 0x7f27cc062630 con 0x7f27b80a2e10 2024-09-11T23:22:03.102 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:22:03.103 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-11T23:22:03.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:03.103+0000 7f27d3923640 1 -- 172.21.15.43:0/1741269639 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f27b80a2e10 msgr2=0x7f27b80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:03.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:03.103+0000 7f27d3923640 1 --2- 172.21.15.43:0/1741269639 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f27b80a2e10 0x7f27b80a52d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f27cc10d5b0 tx=0x7f27bc00a4c0 comp rx=0 tx=0).stop 2024-09-11T23:22:03.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:03.103+0000 7f27d3923640 1 -- 172.21.15.43:0/1741269639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc102de0 msgr2=0x7f27cc198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:03.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:03.103+0000 7f27d3923640 1 --2- 172.21.15.43:0/1741269639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc102de0 0x7f27cc198410 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f27c8005ab0 tx=0x7f27c80042f0 comp rx=0 tx=0).stop 2024-09-11T23:22:03.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:03.103+0000 7f27d3923640 1 -- 172.21.15.43:0/1741269639 shutdown_connections 2024-09-11T23:22:03.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:03.103+0000 7f27d3923640 1 --2- 172.21.15.43:0/1741269639 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f27b80a2e10 0x7f27b80a52d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:03.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:03.103+0000 7f27d3923640 1 --2- 172.21.15.43:0/1741269639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27cc10c180 0x7f27cc19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:03.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:03.103+0000 7f27d3923640 1 --2- 172.21.15.43:0/1741269639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27cc103700 0x7f27cc198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:03.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:03.103+0000 7f27d3923640 1 --2- 172.21.15.43:0/1741269639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc102de0 0x7f27cc198410 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:03.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:03.103+0000 7f27d3923640 1 -- 172.21.15.43:0/1741269639 >> 172.21.15.43:0/1741269639 conn(0x7f27cc0fc7a0 msgr2=0x7f27cc0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:03.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:03.103+0000 7f27d3923640 1 -- 172.21.15.43:0/1741269639 shutdown_connections 2024-09-11T23:22:03.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:03.103+0000 7f27d3923640 1 -- 172.21.15.43:0/1741269639 wait complete. 2024-09-11T23:22:03.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:03 smithi090 ceph-mon[35130]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 50 op/s 2024-09-11T23:22:03.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:03 smithi097 ceph-mon[35562]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 50 op/s 2024-09-11T23:22:03.807 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:03 smithi043 ceph-mon[30950]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 50 op/s 2024-09-11T23:22:03.837 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-11T23:22:02.225202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1387744,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1275,"num_read_kb":1062,"num_write":621,"num_write_kb":5297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":958,"ondisk_log_size":958,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369160,"kb_used_data":11556,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572664,"statfs":{"total":1151940427776,"available":1151562407936,"internally_reserved":0,"allocated":11833344,"data_stored":9565848,"data_compressed":316419,"data_compressed_allocated":417792,"data_compressed_original":4165632,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":199938,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":408,"num_read_kb":468,"num_write":147,"num_write_kb":920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.989229"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598561+0000","last_change":"2024-09-11T23:19:59.734378+0000","last_active":"2024-09-11T23:21:50.598561+0000","last_peered":"2024-09-11T23:21:50.598561+0000","last_clean":"2024-09-11T23:21:50.598561+0000","last_became_active":"2024-09-11T23:19:59.734067+0000","last_became_peered":"2024-09-11T23:19:59.734067+0000","last_unstale":"2024-09-11T23:21:50.598561+0000","last_undegraded":"2024-09-11T23:21:50.598561+0000","last_fullsized":"2024-09-11T23:21:50.598561+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:28:18.384453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.454939+0000","last_change":"2024-09-11T23:19:59.733892+0000","last_active":"2024-09-11T23:21:50.454939+0000","last_peered":"2024-09-11T23:21:50.454939+0000","last_clean":"2024-09-11T23:21:50.454939+0000","last_became_active":"2024-09-11T23:19:59.733540+0000","last_became_peered":"2024-09-11T23:19:59.733540+0000","last_unstale":"2024-09-11T23:21:50.454939+0000","last_undegraded":"2024-09-11T23:21:50.454939+0000","last_fullsized":"2024-09-11T23:21:50.454939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:42.186458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455001+0000","last_change":"2024-09-11T23:19:59.739443+0000","last_active":"2024-09-11T23:21:50.455001+0000","last_peered":"2024-09-11T23:21:50.455001+0000","last_clean":"2024-09-11T23:21:50.455001+0000","last_became_active":"2024-09-11T23:19:59.739025+0000","last_became_peered":"2024-09-11T23:19:59.739025+0000","last_unstale":"2024-09-11T23:21:50.455001+0000","last_undegraded":"2024-09-11T23:21:50.455001+0000","last_fullsized":"2024-09-11T23:21:50.455001+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:40.819321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297376+0000","last_change":"2024-09-11T23:21:02.363511+0000","last_active":"2024-09-11T23:21:50.297376+0000","last_peered":"2024-09-11T23:21:50.297376+0000","last_clean":"2024-09-11T23:21:50.297376+0000","last_became_active":"2024-09-11T23:21:02.363156+0000","last_became_peered":"2024-09-11T23:21:02.363156+0000","last_unstale":"2024-09-11T23:21:50.297376+0000","last_undegraded":"2024-09-11T23:21:50.297376+0000","last_fullsized":"2024-09-11T23:21:50.297376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:17.059509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299585+0000","last_change":"2024-09-11T23:19:59.738662+0000","last_active":"2024-09-11T23:21:50.299585+0000","last_peered":"2024-09-11T23:21:50.299585+0000","last_clean":"2024-09-11T23:21:50.299585+0000","last_became_active":"2024-09-11T23:19:59.738320+0000","last_became_peered":"2024-09-11T23:19:59.738320+0000","last_unstale":"2024-09-11T23:21:50.299585+0000","last_undegraded":"2024-09-11T23:21:50.299585+0000","last_fullsized":"2024-09-11T23:21:50.299585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:15.645179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.310947+0000","last_change":"2024-09-11T23:19:59.731442+0000","last_active":"2024-09-11T23:21:50.310947+0000","last_peered":"2024-09-11T23:21:50.310947+0000","last_clean":"2024-09-11T23:21:50.310947+0000","last_became_active":"2024-09-11T23:19:59.731079+0000","last_became_peered":"2024-09-11T23:19:59.731079+0000","last_unstale":"2024-09-11T23:21:50.310947+0000","last_undegraded":"2024-09-11T23:21:50.310947+0000","last_fullsized":"2024-09-11T23:21:50.310947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:32.764991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455210+0000","last_change":"2024-09-11T23:21:02.359992+0000","last_active":"2024-09-11T23:21:50.455210+0000","last_peered":"2024-09-11T23:21:50.455210+0000","last_clean":"2024-09-11T23:21:50.455210+0000","last_became_active":"2024-09-11T23:21:02.359564+0000","last_became_peered":"2024-09-11T23:21:02.359564+0000","last_unstale":"2024-09-11T23:21:50.455210+0000","last_undegraded":"2024-09-11T23:21:50.455210+0000","last_fullsized":"2024-09-11T23:21:50.455210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:53.633219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300215+0000","last_change":"2024-09-11T23:19:59.740733+0000","last_active":"2024-09-11T23:21:50.300215+0000","last_peered":"2024-09-11T23:21:50.300215+0000","last_clean":"2024-09-11T23:21:50.300215+0000","last_became_active":"2024-09-11T23:19:59.740438+0000","last_became_peered":"2024-09-11T23:19:59.740438+0000","last_unstale":"2024-09-11T23:21:50.300215+0000","last_undegraded":"2024-09-11T23:21:50.300215+0000","last_fullsized":"2024-09-11T23:21:50.300215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:27.063305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.212024+0000","last_change":"2024-09-11T23:19:59.742439+0000","last_active":"2024-09-11T23:21:50.212024+0000","last_peered":"2024-09-11T23:21:50.212024+0000","last_clean":"2024-09-11T23:21:50.212024+0000","last_became_active":"2024-09-11T23:19:59.741983+0000","last_became_peered":"2024-09-11T23:19:59.741983+0000","last_unstale":"2024-09-11T23:21:50.212024+0000","last_undegraded":"2024-09-11T23:21:50.212024+0000","last_fullsized":"2024-09-11T23:21:50.212024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:39.477167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311317+0000","last_change":"2024-09-11T23:19:59.732544+0000","last_active":"2024-09-11T23:21:50.311317+0000","last_peered":"2024-09-11T23:21:50.311317+0000","last_clean":"2024-09-11T23:21:50.311317+0000","last_became_active":"2024-09-11T23:19:59.732205+0000","last_became_peered":"2024-09-11T23:19:59.732205+0000","last_unstale":"2024-09-11T23:21:50.311317+0000","last_undegraded":"2024-09-11T23:21:50.311317+0000","last_fullsized":"2024-09-11T23:21:50.311317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:22.494344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455518+0000","last_change":"2024-09-11T23:19:59.741857+0000","last_active":"2024-09-11T23:21:50.455518+0000","last_peered":"2024-09-11T23:21:50.455518+0000","last_clean":"2024-09-11T23:21:50.455518+0000","last_became_active":"2024-09-11T23:19:59.741484+0000","last_became_peered":"2024-09-11T23:19:59.741484+0000","last_unstale":"2024-09-11T23:21:50.455518+0000","last_undegraded":"2024-09-11T23:21:50.455518+0000","last_fullsized":"2024-09-11T23:21:50.455518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:52.998258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.453834+0000","last_change":"2024-09-11T23:19:59.736042+0000","last_active":"2024-09-11T23:21:50.453834+0000","last_peered":"2024-09-11T23:21:50.453834+0000","last_clean":"2024-09-11T23:21:50.453834+0000","last_became_active":"2024-09-11T23:19:59.735731+0000","last_became_peered":"2024-09-11T23:19:59.735731+0000","last_unstale":"2024-09-11T23:21:50.453834+0000","last_undegraded":"2024-09-11T23:21:50.453834+0000","last_fullsized":"2024-09-11T23:21:50.453834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:19.085804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210129+0000","last_change":"2024-09-11T23:19:59.742049+0000","last_active":"2024-09-11T23:21:50.210129+0000","last_peered":"2024-09-11T23:21:50.210129+0000","last_clean":"2024-09-11T23:21:50.210129+0000","last_became_active":"2024-09-11T23:19:59.740974+0000","last_became_peered":"2024-09-11T23:19:59.740974+0000","last_unstale":"2024-09-11T23:21:50.210129+0000","last_undegraded":"2024-09-11T23:21:50.210129+0000","last_fullsized":"2024-09-11T23:21:50.210129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:10.443113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.454569+0000","last_change":"2024-09-11T23:19:59.734270+0000","last_active":"2024-09-11T23:21:50.454569+0000","last_peered":"2024-09-11T23:21:50.454569+0000","last_clean":"2024-09-11T23:21:50.454569+0000","last_became_active":"2024-09-11T23:19:59.733967+0000","last_became_peered":"2024-09-11T23:19:59.733967+0000","last_unstale":"2024-09-11T23:21:50.454569+0000","last_undegraded":"2024-09-11T23:21:50.454569+0000","last_fullsized":"2024-09-11T23:21:50.454569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:02.185757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.454215+0000","last_change":"2024-09-11T23:19:59.734152+0000","last_active":"2024-09-11T23:21:50.454215+0000","last_peered":"2024-09-11T23:21:50.454215+0000","last_clean":"2024-09-11T23:21:50.454215+0000","last_became_active":"2024-09-11T23:19:59.733163+0000","last_became_peered":"2024-09-11T23:19:59.733163+0000","last_unstale":"2024-09-11T23:21:50.454215+0000","last_undegraded":"2024-09-11T23:21:50.454215+0000","last_fullsized":"2024-09-11T23:21:50.454215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:22.370591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211142+0000","last_change":"2024-09-11T23:19:59.735517+0000","last_active":"2024-09-11T23:21:50.211142+0000","last_peered":"2024-09-11T23:21:50.211142+0000","last_clean":"2024-09-11T23:21:50.211142+0000","last_became_active":"2024-09-11T23:19:59.735182+0000","last_became_peered":"2024-09-11T23:19:59.735182+0000","last_unstale":"2024-09-11T23:21:50.211142+0000","last_undegraded":"2024-09-11T23:21:50.211142+0000","last_fullsized":"2024-09-11T23:21:50.211142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:25.989416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598585+0000","last_change":"2024-09-11T23:19:59.733579+0000","last_active":"2024-09-11T23:21:50.598585+0000","last_peered":"2024-09-11T23:21:50.598585+0000","last_clean":"2024-09-11T23:21:50.598585+0000","last_became_active":"2024-09-11T23:19:59.733245+0000","last_became_peered":"2024-09-11T23:19:59.733245+0000","last_unstale":"2024-09-11T23:21:50.598585+0000","last_undegraded":"2024-09-11T23:21:50.598585+0000","last_fullsized":"2024-09-11T23:21:50.598585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:22.361158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297955+0000","last_change":"2024-09-11T23:19:59.734354+0000","last_active":"2024-09-11T23:21:50.297955+0000","last_peered":"2024-09-11T23:21:50.297955+0000","last_clean":"2024-09-11T23:21:50.297955+0000","last_became_active":"2024-09-11T23:19:59.733988+0000","last_became_peered":"2024-09-11T23:19:59.733988+0000","last_unstale":"2024-09-11T23:21:50.297955+0000","last_undegraded":"2024-09-11T23:21:50.297955+0000","last_fullsized":"2024-09-11T23:21:50.297955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:41.401496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.212032+0000","last_change":"2024-09-11T23:19:59.730318+0000","last_active":"2024-09-11T23:21:50.212032+0000","last_peered":"2024-09-11T23:21:50.212032+0000","last_clean":"2024-09-11T23:21:50.212032+0000","last_became_active":"2024-09-11T23:19:59.730009+0000","last_became_peered":"2024-09-11T23:19:59.730009+0000","last_unstale":"2024-09-11T23:21:50.212032+0000","last_undegraded":"2024-09-11T23:21:50.212032+0000","last_fullsized":"2024-09-11T23:21:50.212032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:01:38.531967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598328+0000","last_change":"2024-09-11T23:19:59.737254+0000","last_active":"2024-09-11T23:21:50.598328+0000","last_peered":"2024-09-11T23:21:50.598328+0000","last_clean":"2024-09-11T23:21:50.598328+0000","last_became_active":"2024-09-11T23:19:59.736936+0000","last_became_peered":"2024-09-11T23:19:59.736936+0000","last_unstale":"2024-09-11T23:21:50.598328+0000","last_undegraded":"2024-09-11T23:21:50.598328+0000","last_fullsized":"2024-09-11T23:21:50.598328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:00.361468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210147+0000","last_change":"2024-09-11T23:19:59.741617+0000","last_active":"2024-09-11T23:21:50.210147+0000","last_peered":"2024-09-11T23:21:50.210147+0000","last_clean":"2024-09-11T23:21:50.210147+0000","last_became_active":"2024-09-11T23:19:59.740358+0000","last_became_peered":"2024-09-11T23:19:59.740358+0000","last_unstale":"2024-09-11T23:21:50.210147+0000","last_undegraded":"2024-09-11T23:21:50.210147+0000","last_fullsized":"2024-09-11T23:21:50.210147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:18.541105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211292+0000","last_change":"2024-09-11T23:19:59.736437+0000","last_active":"2024-09-11T23:21:50.211292+0000","last_peered":"2024-09-11T23:21:50.211292+0000","last_clean":"2024-09-11T23:21:50.211292+0000","last_became_active":"2024-09-11T23:19:59.736029+0000","last_became_peered":"2024-09-11T23:19:59.736029+0000","last_unstale":"2024-09-11T23:21:50.211292+0000","last_undegraded":"2024-09-11T23:21:50.211292+0000","last_fullsized":"2024-09-11T23:21:50.211292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:29.226939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300441+0000","last_change":"2024-09-11T23:19:59.736104+0000","last_active":"2024-09-11T23:21:50.300441+0000","last_peered":"2024-09-11T23:21:50.300441+0000","last_clean":"2024-09-11T23:21:50.300441+0000","last_became_active":"2024-09-11T23:19:59.735760+0000","last_became_peered":"2024-09-11T23:19:59.735760+0000","last_unstale":"2024-09-11T23:21:50.300441+0000","last_undegraded":"2024-09-11T23:21:50.300441+0000","last_fullsized":"2024-09-11T23:21:50.300441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:56.554313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308481+0000","last_change":"2024-09-11T23:19:59.735455+0000","last_active":"2024-09-11T23:21:50.308481+0000","last_peered":"2024-09-11T23:21:50.308481+0000","last_clean":"2024-09-11T23:21:50.308481+0000","last_became_active":"2024-09-11T23:19:59.735078+0000","last_became_peered":"2024-09-11T23:19:59.735078+0000","last_unstale":"2024-09-11T23:21:50.308481+0000","last_undegraded":"2024-09-11T23:21:50.308481+0000","last_fullsized":"2024-09-11T23:21:50.308481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:21:16.336006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.301909+0000","last_change":"2024-09-11T23:19:59.745997+0000","last_active":"2024-09-11T23:21:50.301909+0000","last_peered":"2024-09-11T23:21:50.301909+0000","last_clean":"2024-09-11T23:21:50.301909+0000","last_became_active":"2024-09-11T23:19:59.745715+0000","last_became_peered":"2024-09-11T23:19:59.745715+0000","last_unstale":"2024-09-11T23:21:50.301909+0000","last_undegraded":"2024-09-11T23:21:50.301909+0000","last_fullsized":"2024-09-11T23:21:50.301909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:34.776229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.212119+0000","last_change":"2024-09-11T23:19:59.734181+0000","last_active":"2024-09-11T23:21:50.212119+0000","last_peered":"2024-09-11T23:21:50.212119+0000","last_clean":"2024-09-11T23:21:50.212119+0000","last_became_active":"2024-09-11T23:19:59.733793+0000","last_became_peered":"2024-09-11T23:19:59.733793+0000","last_unstale":"2024-09-11T23:21:50.212119+0000","last_undegraded":"2024-09-11T23:21:50.212119+0000","last_fullsized":"2024-09-11T23:21:50.212119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:36.906308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598526+0000","last_change":"2024-09-11T23:19:59.738713+0000","last_active":"2024-09-11T23:21:50.598526+0000","last_peered":"2024-09-11T23:21:50.598526+0000","last_clean":"2024-09-11T23:21:50.598526+0000","last_became_active":"2024-09-11T23:19:59.737579+0000","last_became_peered":"2024-09-11T23:19:59.737579+0000","last_unstale":"2024-09-11T23:21:50.598526+0000","last_undegraded":"2024-09-11T23:21:50.598526+0000","last_fullsized":"2024-09-11T23:21:50.598526+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:47.135775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210205+0000","last_change":"2024-09-11T23:19:59.741381+0000","last_active":"2024-09-11T23:21:50.210205+0000","last_peered":"2024-09-11T23:21:50.210205+0000","last_clean":"2024-09-11T23:21:50.210205+0000","last_became_active":"2024-09-11T23:19:59.740914+0000","last_became_peered":"2024-09-11T23:19:59.740914+0000","last_unstale":"2024-09-11T23:21:50.210205+0000","last_undegraded":"2024-09-11T23:21:50.210205+0000","last_fullsized":"2024-09-11T23:21:50.210205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:24:04.365704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455133+0000","last_change":"2024-09-11T23:19:59.736840+0000","last_active":"2024-09-11T23:21:50.455133+0000","last_peered":"2024-09-11T23:21:50.455133+0000","last_clean":"2024-09-11T23:21:50.455133+0000","last_became_active":"2024-09-11T23:19:59.736554+0000","last_became_peered":"2024-09-11T23:19:59.736554+0000","last_unstale":"2024-09-11T23:21:50.455133+0000","last_undegraded":"2024-09-11T23:21:50.455133+0000","last_fullsized":"2024-09-11T23:21:50.455133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:48.508369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211738+0000","last_change":"2024-09-11T23:19:59.735719+0000","last_active":"2024-09-11T23:21:50.211738+0000","last_peered":"2024-09-11T23:21:50.211738+0000","last_clean":"2024-09-11T23:21:50.211738+0000","last_became_active":"2024-09-11T23:19:59.735397+0000","last_became_peered":"2024-09-11T23:19:59.735397+0000","last_unstale":"2024-09-11T23:21:50.211738+0000","last_undegraded":"2024-09-11T23:21:50.211738+0000","last_fullsized":"2024-09-11T23:21:50.211738+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:26:19.392730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211747+0000","last_change":"2024-09-11T23:19:59.737704+0000","last_active":"2024-09-11T23:21:50.211747+0000","last_peered":"2024-09-11T23:21:50.211747+0000","last_clean":"2024-09-11T23:21:50.211747+0000","last_became_active":"2024-09-11T23:19:59.737176+0000","last_became_peered":"2024-09-11T23:19:59.737176+0000","last_unstale":"2024-09-11T23:21:50.211747+0000","last_undegraded":"2024-09-11T23:21:50.211747+0000","last_fullsized":"2024-09-11T23:21:50.211747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:22.938171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598301+0000","last_change":"2024-09-11T23:19:59.735499+0000","last_active":"2024-09-11T23:21:50.598301+0000","last_peered":"2024-09-11T23:21:50.598301+0000","last_clean":"2024-09-11T23:21:50.598301+0000","last_became_active":"2024-09-11T23:19:59.735088+0000","last_became_peered":"2024-09-11T23:19:59.735088+0000","last_unstale":"2024-09-11T23:21:50.598301+0000","last_undegraded":"2024-09-11T23:21:50.598301+0000","last_fullsized":"2024-09-11T23:21:50.598301+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:12.633104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299549+0000","last_change":"2024-09-11T23:19:59.743972+0000","last_active":"2024-09-11T23:21:50.299549+0000","last_peered":"2024-09-11T23:21:50.299549+0000","last_clean":"2024-09-11T23:21:50.299549+0000","last_became_active":"2024-09-11T23:19:59.743675+0000","last_became_peered":"2024-09-11T23:19:59.743675+0000","last_unstale":"2024-09-11T23:21:50.299549+0000","last_undegraded":"2024-09-11T23:21:50.299549+0000","last_fullsized":"2024-09-11T23:21:50.299549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:46.645696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":144,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:59.879602+0000","last_change":"2024-09-11T23:19:57.615472+0000","last_active":"2024-09-11T23:21:59.879602+0000","last_peered":"2024-09-11T23:21:59.879602+0000","last_clean":"2024-09-11T23:21:59.879602+0000","last_became_active":"2024-09-11T23:19:57.615104+0000","last_became_peered":"2024-09-11T23:19:57.615104+0000","last_unstale":"2024-09-11T23:21:59.879602+0000","last_undegraded":"2024-09-11T23:21:59.879602+0000","last_fullsized":"2024-09-11T23:21:59.879602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:03.601363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308144+0000","last_change":"2024-09-11T23:19:59.741027+0000","last_active":"2024-09-11T23:21:50.308144+0000","last_peered":"2024-09-11T23:21:50.308144+0000","last_clean":"2024-09-11T23:21:50.308144+0000","last_became_active":"2024-09-11T23:19:59.740681+0000","last_became_peered":"2024-09-11T23:19:59.740681+0000","last_unstale":"2024-09-11T23:21:50.308144+0000","last_undegraded":"2024-09-11T23:21:50.308144+0000","last_fullsized":"2024-09-11T23:21:50.308144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:36.948147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311025+0000","last_change":"2024-09-11T23:19:57.612730+0000","last_active":"2024-09-11T23:21:50.311025+0000","last_peered":"2024-09-11T23:21:50.311025+0000","last_clean":"2024-09-11T23:21:50.311025+0000","last_became_active":"2024-09-11T23:19:57.612369+0000","last_became_peered":"2024-09-11T23:19:57.612369+0000","last_unstale":"2024-09-11T23:21:50.311025+0000","last_undegraded":"2024-09-11T23:21:50.311025+0000","last_fullsized":"2024-09-11T23:21:50.311025+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:17.823616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.209881+0000","last_change":"2024-09-11T23:19:57.611470+0000","last_active":"2024-09-11T23:21:50.209881+0000","last_peered":"2024-09-11T23:21:50.209881+0000","last_clean":"2024-09-11T23:21:50.209881+0000","last_became_active":"2024-09-11T23:19:57.611141+0000","last_became_peered":"2024-09-11T23:19:57.611141+0000","last_unstale":"2024-09-11T23:21:50.209881+0000","last_undegraded":"2024-09-11T23:21:50.209881+0000","last_fullsized":"2024-09-11T23:21:50.209881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:43.173955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300408+0000","last_change":"2024-09-11T23:19:59.732375+0000","last_active":"2024-09-11T23:21:50.300408+0000","last_peered":"2024-09-11T23:21:50.300408+0000","last_clean":"2024-09-11T23:21:50.300408+0000","last_became_active":"2024-09-11T23:19:59.731793+0000","last_became_peered":"2024-09-11T23:19:59.731793+0000","last_unstale":"2024-09-11T23:21:50.300408+0000","last_undegraded":"2024-09-11T23:21:50.300408+0000","last_fullsized":"2024-09-11T23:21:50.300408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:02:47.278646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308399+0000","last_change":"2024-09-11T23:19:57.614062+0000","last_active":"2024-09-11T23:21:50.308399+0000","last_peered":"2024-09-11T23:21:50.308399+0000","last_clean":"2024-09-11T23:21:50.308399+0000","last_became_active":"2024-09-11T23:19:57.613749+0000","last_became_peered":"2024-09-11T23:19:57.613749+0000","last_unstale":"2024-09-11T23:21:50.308399+0000","last_undegraded":"2024-09-11T23:21:50.308399+0000","last_fullsized":"2024-09-11T23:21:50.308399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:45:22.299379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297086+0000","last_change":"2024-09-11T23:19:59.730758+0000","last_active":"2024-09-11T23:21:50.297086+0000","last_peered":"2024-09-11T23:21:50.297086+0000","last_clean":"2024-09-11T23:21:50.297086+0000","last_became_active":"2024-09-11T23:19:59.730423+0000","last_became_peered":"2024-09-11T23:19:59.730423+0000","last_unstale":"2024-09-11T23:21:50.297086+0000","last_undegraded":"2024-09-11T23:21:50.297086+0000","last_fullsized":"2024-09-11T23:21:50.297086+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:30.679451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":139,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:59.879280+0000","last_change":"2024-09-11T23:19:57.615574+0000","last_active":"2024-09-11T23:21:59.879280+0000","last_peered":"2024-09-11T23:21:59.879280+0000","last_clean":"2024-09-11T23:21:59.879280+0000","last_became_active":"2024-09-11T23:19:57.615192+0000","last_became_peered":"2024-09-11T23:19:57.615192+0000","last_unstale":"2024-09-11T23:21:59.879280+0000","last_undegraded":"2024-09-11T23:21:59.879280+0000","last_fullsized":"2024-09-11T23:21:59.879280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:29.186899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211749+0000","last_change":"2024-09-11T23:19:59.738433+0000","last_active":"2024-09-11T23:21:50.211749+0000","last_peered":"2024-09-11T23:21:50.211749+0000","last_clean":"2024-09-11T23:21:50.211749+0000","last_became_active":"2024-09-11T23:19:59.738081+0000","last_became_peered":"2024-09-11T23:19:59.738081+0000","last_unstale":"2024-09-11T23:21:50.211749+0000","last_undegraded":"2024-09-11T23:21:50.211749+0000","last_fullsized":"2024-09-11T23:21:50.211749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:10:46.786454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455225+0000","last_change":"2024-09-11T23:21:02.437498+0000","last_active":"2024-09-11T23:21:50.455225+0000","last_peered":"2024-09-11T23:21:50.455225+0000","last_clean":"2024-09-11T23:21:50.455225+0000","last_became_active":"2024-09-11T23:21:02.434895+0000","last_became_peered":"2024-09-11T23:21:02.434895+0000","last_unstale":"2024-09-11T23:21:50.455225+0000","last_undegraded":"2024-09-11T23:21:50.455225+0000","last_fullsized":"2024-09-11T23:21:50.455225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:33.142041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455584+0000","last_change":"2024-09-11T23:19:59.741081+0000","last_active":"2024-09-11T23:21:50.455584+0000","last_peered":"2024-09-11T23:21:50.455584+0000","last_clean":"2024-09-11T23:21:50.455584+0000","last_became_active":"2024-09-11T23:19:59.739984+0000","last_became_peered":"2024-09-11T23:19:59.739984+0000","last_unstale":"2024-09-11T23:21:50.455584+0000","last_undegraded":"2024-09-11T23:21:50.455584+0000","last_fullsized":"2024-09-11T23:21:50.455584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:03.513121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308186+0000","last_change":"2024-09-11T23:19:57.823272+0000","last_active":"2024-09-11T23:21:50.308186+0000","last_peered":"2024-09-11T23:21:50.308186+0000","last_clean":"2024-09-11T23:21:50.308186+0000","last_became_active":"2024-09-11T23:19:57.822957+0000","last_became_peered":"2024-09-11T23:19:57.822957+0000","last_unstale":"2024-09-11T23:21:50.308186+0000","last_undegraded":"2024-09-11T23:21:50.308186+0000","last_fullsized":"2024-09-11T23:21:50.308186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:06.779385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211510+0000","last_change":"2024-09-11T23:19:59.738398+0000","last_active":"2024-09-11T23:21:50.211510+0000","last_peered":"2024-09-11T23:21:50.211510+0000","last_clean":"2024-09-11T23:21:50.211510+0000","last_became_active":"2024-09-11T23:19:59.738042+0000","last_became_peered":"2024-09-11T23:19:59.738042+0000","last_unstale":"2024-09-11T23:21:50.211510+0000","last_undegraded":"2024-09-11T23:21:50.211510+0000","last_fullsized":"2024-09-11T23:21:50.211510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:05.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'36","reported_seq":296,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:22:00.880783+0000","last_change":"2024-09-11T23:19:57.615036+0000","last_active":"2024-09-11T23:22:00.880783+0000","last_peered":"2024-09-11T23:22:00.880783+0000","last_clean":"2024-09-11T23:22:00.880783+0000","last_became_active":"2024-09-11T23:19:57.614641+0000","last_became_peered":"2024-09-11T23:19:57.614641+0000","last_unstale":"2024-09-11T23:22:00.880783+0000","last_undegraded":"2024-09-11T23:22:00.880783+0000","last_fullsized":"2024-09-11T23:22:00.880783+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:05.430508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69978,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":236,"num_read_kb":187,"num_write":37,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299857+0000","last_change":"2024-09-11T23:19:59.741597+0000","last_active":"2024-09-11T23:21:50.299857+0000","last_peered":"2024-09-11T23:21:50.299857+0000","last_clean":"2024-09-11T23:21:50.299857+0000","last_became_active":"2024-09-11T23:19:59.741296+0000","last_became_peered":"2024-09-11T23:19:59.741296+0000","last_unstale":"2024-09-11T23:21:50.299857+0000","last_undegraded":"2024-09-11T23:21:50.299857+0000","last_fullsized":"2024-09-11T23:21:50.299857+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:06:20.742916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211837+0000","last_change":"2024-09-11T23:19:57.617550+0000","last_active":"2024-09-11T23:21:50.211837+0000","last_peered":"2024-09-11T23:21:50.211837+0000","last_clean":"2024-09-11T23:21:50.211837+0000","last_became_active":"2024-09-11T23:19:57.617211+0000","last_became_peered":"2024-09-11T23:19:57.617211+0000","last_unstale":"2024-09-11T23:21:50.211837+0000","last_undegraded":"2024-09-11T23:21:50.211837+0000","last_fullsized":"2024-09-11T23:21:50.211837+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:19.707742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210301+0000","last_change":"2024-09-11T23:19:59.742070+0000","last_active":"2024-09-11T23:21:50.210301+0000","last_peered":"2024-09-11T23:21:50.210301+0000","last_clean":"2024-09-11T23:21:50.210301+0000","last_became_active":"2024-09-11T23:19:59.739367+0000","last_became_peered":"2024-09-11T23:19:59.739367+0000","last_unstale":"2024-09-11T23:21:50.210301+0000","last_undegraded":"2024-09-11T23:21:50.210301+0000","last_fullsized":"2024-09-11T23:21:50.210301+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:27.850136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308168+0000","last_change":"2024-09-11T23:19:57.618054+0000","last_active":"2024-09-11T23:21:50.308168+0000","last_peered":"2024-09-11T23:21:50.308168+0000","last_clean":"2024-09-11T23:21:50.308168+0000","last_became_active":"2024-09-11T23:19:57.617702+0000","last_became_peered":"2024-09-11T23:19:57.617702+0000","last_unstale":"2024-09-11T23:21:50.308168+0000","last_undegraded":"2024-09-11T23:21:50.308168+0000","last_fullsized":"2024-09-11T23:21:50.308168+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:40.171851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455548+0000","last_change":"2024-09-11T23:19:59.744000+0000","last_active":"2024-09-11T23:21:50.455548+0000","last_peered":"2024-09-11T23:21:50.455548+0000","last_clean":"2024-09-11T23:21:50.455548+0000","last_became_active":"2024-09-11T23:19:59.743629+0000","last_became_peered":"2024-09-11T23:19:59.743629+0000","last_unstale":"2024-09-11T23:21:50.455548+0000","last_undegraded":"2024-09-11T23:21:50.455548+0000","last_fullsized":"2024-09-11T23:21:50.455548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:20:04.133258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'8","reported_seq":149,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:59.879721+0000","last_change":"2024-09-11T23:19:57.615117+0000","last_active":"2024-09-11T23:21:59.879721+0000","last_peered":"2024-09-11T23:21:59.879721+0000","last_clean":"2024-09-11T23:21:59.879721+0000","last_became_active":"2024-09-11T23:19:57.614758+0000","last_became_peered":"2024-09-11T23:19:57.614758+0000","last_unstale":"2024-09-11T23:21:59.879721+0000","last_undegraded":"2024-09-11T23:21:59.879721+0000","last_fullsized":"2024-09-11T23:21:59.879721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:58.113986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2301,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":7,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455281+0000","last_change":"2024-09-11T23:19:59.739403+0000","last_active":"2024-09-11T23:21:50.455281+0000","last_peered":"2024-09-11T23:21:50.455281+0000","last_clean":"2024-09-11T23:21:50.455281+0000","last_became_active":"2024-09-11T23:19:59.739078+0000","last_became_peered":"2024-09-11T23:19:59.739078+0000","last_unstale":"2024-09-11T23:21:50.455281+0000","last_undegraded":"2024-09-11T23:21:50.455281+0000","last_fullsized":"2024-09-11T23:21:50.455281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:46.168720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299488+0000","last_change":"2024-09-11T23:19:57.823323+0000","last_active":"2024-09-11T23:21:50.299488+0000","last_peered":"2024-09-11T23:21:50.299488+0000","last_clean":"2024-09-11T23:21:50.299488+0000","last_became_active":"2024-09-11T23:19:57.822947+0000","last_became_peered":"2024-09-11T23:19:57.822947+0000","last_unstale":"2024-09-11T23:21:50.299488+0000","last_undegraded":"2024-09-11T23:21:50.299488+0000","last_fullsized":"2024-09-11T23:21:50.299488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:12.542911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308091+0000","last_change":"2024-09-11T23:19:59.741163+0000","last_active":"2024-09-11T23:21:50.308091+0000","last_peered":"2024-09-11T23:21:50.308091+0000","last_clean":"2024-09-11T23:21:50.308091+0000","last_became_active":"2024-09-11T23:19:59.740633+0000","last_became_peered":"2024-09-11T23:19:59.740633+0000","last_unstale":"2024-09-11T23:21:50.308091+0000","last_undegraded":"2024-09-11T23:21:50.308091+0000","last_fullsized":"2024-09-11T23:21:50.308091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:26.301775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299941+0000","last_change":"2024-09-11T23:19:57.615804+0000","last_active":"2024-09-11T23:21:50.299941+0000","last_peered":"2024-09-11T23:21:50.299941+0000","last_clean":"2024-09-11T23:21:50.299941+0000","last_became_active":"2024-09-11T23:19:57.615443+0000","last_became_peered":"2024-09-11T23:19:57.615443+0000","last_unstale":"2024-09-11T23:21:50.299941+0000","last_undegraded":"2024-09-11T23:21:50.299941+0000","last_fullsized":"2024-09-11T23:21:50.299941+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:39.296873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598177+0000","last_change":"2024-09-11T23:21:02.360875+0000","last_active":"2024-09-11T23:21:50.598177+0000","last_peered":"2024-09-11T23:21:50.598177+0000","last_clean":"2024-09-11T23:21:50.598177+0000","last_became_active":"2024-09-11T23:21:02.360480+0000","last_became_peered":"2024-09-11T23:21:02.360480+0000","last_unstale":"2024-09-11T23:21:50.598177+0000","last_undegraded":"2024-09-11T23:21:50.598177+0000","last_fullsized":"2024-09-11T23:21:50.598177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:04.141587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210448+0000","last_change":"2024-09-11T23:19:57.612873+0000","last_active":"2024-09-11T23:21:50.210448+0000","last_peered":"2024-09-11T23:21:50.210448+0000","last_clean":"2024-09-11T23:21:50.210448+0000","last_became_active":"2024-09-11T23:19:57.612566+0000","last_became_peered":"2024-09-11T23:19:57.612566+0000","last_unstale":"2024-09-11T23:21:50.210448+0000","last_undegraded":"2024-09-11T23:21:50.210448+0000","last_fullsized":"2024-09-11T23:21:50.210448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:46.218212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455770+0000","last_change":"2024-09-11T23:19:59.736352+0000","last_active":"2024-09-11T23:21:50.455770+0000","last_peered":"2024-09-11T23:21:50.455770+0000","last_clean":"2024-09-11T23:21:50.455770+0000","last_became_active":"2024-09-11T23:19:59.735402+0000","last_became_peered":"2024-09-11T23:19:59.735402+0000","last_unstale":"2024-09-11T23:21:50.455770+0000","last_undegraded":"2024-09-11T23:21:50.455770+0000","last_fullsized":"2024-09-11T23:21:50.455770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:41.854526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'65","reported_seq":119,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:22:00.820163+0000","last_change":"2024-09-11T23:19:57.612439+0000","last_active":"2024-09-11T23:22:00.820163+0000","last_peered":"2024-09-11T23:22:00.820163+0000","last_clean":"2024-09-11T23:22:00.820163+0000","last_became_active":"2024-09-11T23:19:57.612030+0000","last_became_peered":"2024-09-11T23:19:57.612030+0000","last_unstale":"2024-09-11T23:22:00.820163+0000","last_undegraded":"2024-09-11T23:22:00.820163+0000","last_fullsized":"2024-09-11T23:22:00.820163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:36:24.579246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455480+0000","last_change":"2024-09-11T23:19:59.741241+0000","last_active":"2024-09-11T23:21:50.455480+0000","last_peered":"2024-09-11T23:21:50.455480+0000","last_clean":"2024-09-11T23:21:50.455480+0000","last_became_active":"2024-09-11T23:19:59.740265+0000","last_became_peered":"2024-09-11T23:19:59.740265+0000","last_unstale":"2024-09-11T23:21:50.455480+0000","last_undegraded":"2024-09-11T23:21:50.455480+0000","last_fullsized":"2024-09-11T23:21:50.455480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:00.676762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211975+0000","last_change":"2024-09-11T23:19:57.825238+0000","last_active":"2024-09-11T23:21:50.211975+0000","last_peered":"2024-09-11T23:21:50.211975+0000","last_clean":"2024-09-11T23:21:50.211975+0000","last_became_active":"2024-09-11T23:19:57.824732+0000","last_became_peered":"2024-09-11T23:19:57.824732+0000","last_unstale":"2024-09-11T23:21:50.211975+0000","last_undegraded":"2024-09-11T23:21:50.211975+0000","last_fullsized":"2024-09-11T23:21:50.211975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:24.465863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455634+0000","last_change":"2024-09-11T23:19:59.741338+0000","last_active":"2024-09-11T23:21:50.455634+0000","last_peered":"2024-09-11T23:21:50.455634+0000","last_clean":"2024-09-11T23:21:50.455634+0000","last_became_active":"2024-09-11T23:19:59.741019+0000","last_became_peered":"2024-09-11T23:19:59.741019+0000","last_unstale":"2024-09-11T23:21:50.455634+0000","last_undegraded":"2024-09-11T23:21:50.455634+0000","last_fullsized":"2024-09-11T23:21:50.455634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:36:03.480006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"85'148","reported_seq":395,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:52.311759+0000","last_change":"2024-09-11T23:18:45.606646+0000","last_active":"2024-09-11T23:21:52.311759+0000","last_peered":"2024-09-11T23:21:52.311759+0000","last_clean":"2024-09-11T23:21:52.311759+0000","last_became_active":"2024-09-11T23:18:45.606065+0000","last_became_peered":"2024-09-11T23:18:45.606065+0000","last_unstale":"2024-09-11T23:21:52.311759+0000","last_undegraded":"2024-09-11T23:21:52.311759+0000","last_fullsized":"2024-09-11T23:21:52.311759+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:15:17.148562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:15:17.148562+0000","last_clean_scrub_stamp":"2024-09-11T23:15:17.148562+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:07.992971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.453802+0000","last_change":"2024-09-11T23:19:57.612584+0000","last_active":"2024-09-11T23:21:50.453802+0000","last_peered":"2024-09-11T23:21:50.453802+0000","last_clean":"2024-09-11T23:21:50.453802+0000","last_became_active":"2024-09-11T23:19:57.612211+0000","last_became_peered":"2024-09-11T23:19:57.612211+0000","last_unstale":"2024-09-11T23:21:50.453802+0000","last_undegraded":"2024-09-11T23:21:50.453802+0000","last_fullsized":"2024-09-11T23:21:50.453802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:04.026233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300938+0000","last_change":"2024-09-11T23:21:02.359249+0000","last_active":"2024-09-11T23:21:50.300938+0000","last_peered":"2024-09-11T23:21:50.300938+0000","last_clean":"2024-09-11T23:21:50.300938+0000","last_became_active":"2024-09-11T23:21:02.358906+0000","last_became_peered":"2024-09-11T23:21:02.358906+0000","last_unstale":"2024-09-11T23:21:50.300938+0000","last_undegraded":"2024-09-11T23:21:50.300938+0000","last_fullsized":"2024-09-11T23:21:50.300938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:05.973581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455482+0000","last_change":"2024-09-11T23:19:57.618283+0000","last_active":"2024-09-11T23:21:50.455482+0000","last_peered":"2024-09-11T23:21:50.455482+0000","last_clean":"2024-09-11T23:21:50.455482+0000","last_became_active":"2024-09-11T23:19:57.617942+0000","last_became_peered":"2024-09-11T23:19:57.617942+0000","last_unstale":"2024-09-11T23:21:50.455482+0000","last_undegraded":"2024-09-11T23:21:50.455482+0000","last_fullsized":"2024-09-11T23:21:50.455482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:30.832602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.209864+0000","last_change":"2024-09-11T23:19:59.740730+0000","last_active":"2024-09-11T23:21:50.209864+0000","last_peered":"2024-09-11T23:21:50.209864+0000","last_clean":"2024-09-11T23:21:50.209864+0000","last_became_active":"2024-09-11T23:19:59.740369+0000","last_became_peered":"2024-09-11T23:19:59.740369+0000","last_unstale":"2024-09-11T23:21:50.209864+0000","last_undegraded":"2024-09-11T23:21:50.209864+0000","last_fullsized":"2024-09-11T23:21:50.209864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:50.264644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":142,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:22:00.880529+0000","last_change":"2024-09-11T23:19:57.823253+0000","last_active":"2024-09-11T23:22:00.880529+0000","last_peered":"2024-09-11T23:22:00.880529+0000","last_clean":"2024-09-11T23:22:00.880529+0000","last_became_active":"2024-09-11T23:19:57.822895+0000","last_became_peered":"2024-09-11T23:19:57.822895+0000","last_unstale":"2024-09-11T23:22:00.880529+0000","last_undegraded":"2024-09-11T23:22:00.880529+0000","last_fullsized":"2024-09-11T23:22:00.880529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:22.998371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308142+0000","last_change":"2024-09-11T23:19:59.742248+0000","last_active":"2024-09-11T23:21:50.308142+0000","last_peered":"2024-09-11T23:21:50.308142+0000","last_clean":"2024-09-11T23:21:50.308142+0000","last_became_active":"2024-09-11T23:19:59.741810+0000","last_became_peered":"2024-09-11T23:19:59.741810+0000","last_unstale":"2024-09-11T23:21:50.308142+0000","last_undegraded":"2024-09-11T23:21:50.308142+0000","last_fullsized":"2024-09-11T23:21:50.308142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:19:52.720286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455554+0000","last_change":"2024-09-11T23:19:57.616244+0000","last_active":"2024-09-11T23:21:50.455554+0000","last_peered":"2024-09-11T23:21:50.455554+0000","last_clean":"2024-09-11T23:21:50.455554+0000","last_became_active":"2024-09-11T23:19:57.615868+0000","last_became_peered":"2024-09-11T23:19:57.615868+0000","last_unstale":"2024-09-11T23:21:50.455554+0000","last_undegraded":"2024-09-11T23:21:50.455554+0000","last_fullsized":"2024-09-11T23:21:50.455554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:42.510896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311069+0000","last_change":"2024-09-11T23:21:02.434265+0000","last_active":"2024-09-11T23:21:50.311069+0000","last_peered":"2024-09-11T23:21:50.311069+0000","last_clean":"2024-09-11T23:21:50.311069+0000","last_became_active":"2024-09-11T23:21:02.433840+0000","last_became_peered":"2024-09-11T23:21:02.433840+0000","last_unstale":"2024-09-11T23:21:50.311069+0000","last_undegraded":"2024-09-11T23:21:50.311069+0000","last_fullsized":"2024-09-11T23:21:50.311069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:04.619695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297618+0000","last_change":"2024-09-11T23:19:57.824334+0000","last_active":"2024-09-11T23:21:50.297618+0000","last_peered":"2024-09-11T23:21:50.297618+0000","last_clean":"2024-09-11T23:21:50.297618+0000","last_became_active":"2024-09-11T23:19:57.823972+0000","last_became_peered":"2024-09-11T23:19:57.823972+0000","last_unstale":"2024-09-11T23:21:50.297618+0000","last_undegraded":"2024-09-11T23:21:50.297618+0000","last_fullsized":"2024-09-11T23:21:50.297618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:39.183281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311164+0000","last_change":"2024-09-11T23:19:59.728064+0000","last_active":"2024-09-11T23:21:50.311164+0000","last_peered":"2024-09-11T23:21:50.311164+0000","last_clean":"2024-09-11T23:21:50.311164+0000","last_became_active":"2024-09-11T23:19:59.727732+0000","last_became_peered":"2024-09-11T23:19:59.727732+0000","last_unstale":"2024-09-11T23:21:50.311164+0000","last_undegraded":"2024-09-11T23:21:50.311164+0000","last_fullsized":"2024-09-11T23:21:50.311164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:15.383974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299855+0000","last_change":"2024-09-11T23:19:57.824362+0000","last_active":"2024-09-11T23:21:50.299855+0000","last_peered":"2024-09-11T23:21:50.299855+0000","last_clean":"2024-09-11T23:21:50.299855+0000","last_became_active":"2024-09-11T23:19:57.823753+0000","last_became_peered":"2024-09-11T23:19:57.823753+0000","last_unstale":"2024-09-11T23:21:50.299855+0000","last_undegraded":"2024-09-11T23:21:50.299855+0000","last_fullsized":"2024-09-11T23:21:50.299855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:56.357268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210323+0000","last_change":"2024-09-11T23:19:59.730623+0000","last_active":"2024-09-11T23:21:50.210323+0000","last_peered":"2024-09-11T23:21:50.210323+0000","last_clean":"2024-09-11T23:21:50.210323+0000","last_became_active":"2024-09-11T23:19:59.730307+0000","last_became_peered":"2024-09-11T23:19:59.730307+0000","last_unstale":"2024-09-11T23:21:50.210323+0000","last_undegraded":"2024-09-11T23:21:50.210323+0000","last_fullsized":"2024-09-11T23:21:50.210323+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:52.874884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'100","reported_seq":213,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:22:01.200407+0000","last_change":"2024-09-11T23:19:57.611730+0000","last_active":"2024-09-11T23:22:01.200407+0000","last_peered":"2024-09-11T23:22:01.200407+0000","last_clean":"2024-09-11T23:22:01.200407+0000","last_became_active":"2024-09-11T23:19:57.611394+0000","last_became_peered":"2024-09-11T23:19:57.611394+0000","last_unstale":"2024-09-11T23:22:01.200407+0000","last_undegraded":"2024-09-11T23:22:01.200407+0000","last_fullsized":"2024-09-11T23:22:01.200407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:15.518573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":183,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.302035+0000","last_change":"2024-09-11T23:19:59.741396+0000","last_active":"2024-09-11T23:21:50.302035+0000","last_peered":"2024-09-11T23:21:50.302035+0000","last_clean":"2024-09-11T23:21:50.302035+0000","last_became_active":"2024-09-11T23:19:59.741102+0000","last_became_peered":"2024-09-11T23:19:59.741102+0000","last_unstale":"2024-09-11T23:21:50.302035+0000","last_undegraded":"2024-09-11T23:21:50.302035+0000","last_fullsized":"2024-09-11T23:21:50.302035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:08.759512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.453918+0000","last_change":"2024-09-11T23:19:57.824812+0000","last_active":"2024-09-11T23:21:50.453918+0000","last_peered":"2024-09-11T23:21:50.453918+0000","last_clean":"2024-09-11T23:21:50.453918+0000","last_became_active":"2024-09-11T23:19:57.824454+0000","last_became_peered":"2024-09-11T23:19:57.824454+0000","last_unstale":"2024-09-11T23:21:50.453918+0000","last_undegraded":"2024-09-11T23:21:50.453918+0000","last_fullsized":"2024-09-11T23:21:50.453918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:25.275573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455034+0000","last_change":"2024-09-11T23:19:59.735857+0000","last_active":"2024-09-11T23:21:50.455034+0000","last_peered":"2024-09-11T23:21:50.455034+0000","last_clean":"2024-09-11T23:21:50.455034+0000","last_became_active":"2024-09-11T23:19:59.735541+0000","last_became_peered":"2024-09-11T23:19:59.735541+0000","last_unstale":"2024-09-11T23:21:50.455034+0000","last_undegraded":"2024-09-11T23:21:50.455034+0000","last_fullsized":"2024-09-11T23:21:50.455034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:21.600490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211231+0000","last_change":"2024-09-11T23:19:57.823468+0000","last_active":"2024-09-11T23:21:50.211231+0000","last_peered":"2024-09-11T23:21:50.211231+0000","last_clean":"2024-09-11T23:21:50.211231+0000","last_became_active":"2024-09-11T23:19:57.823154+0000","last_became_peered":"2024-09-11T23:19:57.823154+0000","last_unstale":"2024-09-11T23:21:50.211231+0000","last_undegraded":"2024-09-11T23:21:50.211231+0000","last_fullsized":"2024-09-11T23:21:50.211231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:28.770627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299509+0000","last_change":"2024-09-11T23:19:59.741263+0000","last_active":"2024-09-11T23:21:50.299509+0000","last_peered":"2024-09-11T23:21:50.299509+0000","last_clean":"2024-09-11T23:21:50.299509+0000","last_became_active":"2024-09-11T23:19:59.740938+0000","last_became_peered":"2024-09-11T23:19:59.740938+0000","last_unstale":"2024-09-11T23:21:50.299509+0000","last_undegraded":"2024-09-11T23:21:50.299509+0000","last_fullsized":"2024-09-11T23:21:50.299509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:54.768652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.301723+0000","last_change":"2024-09-11T23:19:57.612517+0000","last_active":"2024-09-11T23:21:50.301723+0000","last_peered":"2024-09-11T23:21:50.301723+0000","last_clean":"2024-09-11T23:21:50.301723+0000","last_became_active":"2024-09-11T23:19:57.612188+0000","last_became_peered":"2024-09-11T23:19:57.612188+0000","last_unstale":"2024-09-11T23:21:50.301723+0000","last_undegraded":"2024-09-11T23:21:50.301723+0000","last_fullsized":"2024-09-11T23:21:50.301723+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:52.896413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598315+0000","last_change":"2024-09-11T23:19:59.732520+0000","last_active":"2024-09-11T23:21:50.598315+0000","last_peered":"2024-09-11T23:21:50.598315+0000","last_clean":"2024-09-11T23:21:50.598315+0000","last_became_active":"2024-09-11T23:19:59.732173+0000","last_became_peered":"2024-09-11T23:19:59.732173+0000","last_unstale":"2024-09-11T23:21:50.598315+0000","last_undegraded":"2024-09-11T23:21:50.598315+0000","last_fullsized":"2024-09-11T23:21:50.598315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:04.626312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598177+0000","last_change":"2024-09-11T23:21:02.365533+0000","last_active":"2024-09-11T23:21:50.598177+0000","last_peered":"2024-09-11T23:21:50.598177+0000","last_clean":"2024-09-11T23:21:50.598177+0000","last_became_active":"2024-09-11T23:21:02.359345+0000","last_became_peered":"2024-09-11T23:21:02.359345+0000","last_unstale":"2024-09-11T23:21:50.598177+0000","last_undegraded":"2024-09-11T23:21:50.598177+0000","last_fullsized":"2024-09-11T23:21:50.598177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:27.195666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.301266+0000","last_change":"2024-09-11T23:19:59.734667+0000","last_active":"2024-09-11T23:21:50.301266+0000","last_peered":"2024-09-11T23:21:50.301266+0000","last_clean":"2024-09-11T23:21:50.301266+0000","last_became_active":"2024-09-11T23:19:59.733754+0000","last_became_peered":"2024-09-11T23:19:59.733754+0000","last_unstale":"2024-09-11T23:21:50.301266+0000","last_undegraded":"2024-09-11T23:21:50.301266+0000","last_fullsized":"2024-09-11T23:21:50.301266+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:57.392272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299623+0000","last_change":"2024-09-11T23:19:57.614579+0000","last_active":"2024-09-11T23:21:50.299623+0000","last_peered":"2024-09-11T23:21:50.299623+0000","last_clean":"2024-09-11T23:21:50.299623+0000","last_became_active":"2024-09-11T23:19:57.614227+0000","last_became_peered":"2024-09-11T23:19:57.614227+0000","last_unstale":"2024-09-11T23:21:50.299623+0000","last_undegraded":"2024-09-11T23:21:50.299623+0000","last_fullsized":"2024-09-11T23:21:50.299623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:47.952920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455414+0000","last_change":"2024-09-11T23:19:59.744219+0000","last_active":"2024-09-11T23:21:50.455414+0000","last_peered":"2024-09-11T23:21:50.455414+0000","last_clean":"2024-09-11T23:21:50.455414+0000","last_became_active":"2024-09-11T23:19:59.743914+0000","last_became_peered":"2024-09-11T23:19:59.743914+0000","last_unstale":"2024-09-11T23:21:50.455414+0000","last_undegraded":"2024-09-11T23:21:50.455414+0000","last_fullsized":"2024-09-11T23:21:50.455414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:52.149588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308372+0000","last_change":"2024-09-11T23:21:02.361647+0000","last_active":"2024-09-11T23:21:50.308372+0000","last_peered":"2024-09-11T23:21:50.308372+0000","last_clean":"2024-09-11T23:21:50.308372+0000","last_became_active":"2024-09-11T23:21:02.361319+0000","last_became_peered":"2024-09-11T23:21:02.361319+0000","last_unstale":"2024-09-11T23:21:50.308372+0000","last_undegraded":"2024-09-11T23:21:50.308372+0000","last_fullsized":"2024-09-11T23:21:50.308372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:45.125525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299796+0000","last_change":"2024-09-11T23:19:57.610389+0000","last_active":"2024-09-11T23:21:50.299796+0000","last_peered":"2024-09-11T23:21:50.299796+0000","last_clean":"2024-09-11T23:21:50.299796+0000","last_became_active":"2024-09-11T23:19:57.609911+0000","last_became_peered":"2024-09-11T23:19:57.609911+0000","last_unstale":"2024-09-11T23:21:50.299796+0000","last_undegraded":"2024-09-11T23:21:50.299796+0000","last_fullsized":"2024-09-11T23:21:50.299796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:23.949929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211496+0000","last_change":"2024-09-11T23:19:59.736867+0000","last_active":"2024-09-11T23:21:50.211496+0000","last_peered":"2024-09-11T23:21:50.211496+0000","last_clean":"2024-09-11T23:21:50.211496+0000","last_became_active":"2024-09-11T23:19:59.736341+0000","last_became_peered":"2024-09-11T23:19:59.736341+0000","last_unstale":"2024-09-11T23:21:50.211496+0000","last_undegraded":"2024-09-11T23:21:50.211496+0000","last_fullsized":"2024-09-11T23:21:50.211496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:57:51.015235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598177+0000","last_change":"2024-09-11T23:21:02.368959+0000","last_active":"2024-09-11T23:21:50.598177+0000","last_peered":"2024-09-11T23:21:50.598177+0000","last_clean":"2024-09-11T23:21:50.598177+0000","last_became_active":"2024-09-11T23:21:02.363263+0000","last_became_peered":"2024-09-11T23:21:02.363263+0000","last_unstale":"2024-09-11T23:21:50.598177+0000","last_undegraded":"2024-09-11T23:21:50.598177+0000","last_fullsized":"2024-09-11T23:21:50.598177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:29.827330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455355+0000","last_change":"2024-09-11T23:19:59.742049+0000","last_active":"2024-09-11T23:21:50.455355+0000","last_peered":"2024-09-11T23:21:50.455355+0000","last_clean":"2024-09-11T23:21:50.455355+0000","last_became_active":"2024-09-11T23:19:59.741763+0000","last_became_peered":"2024-09-11T23:19:59.741763+0000","last_unstale":"2024-09-11T23:21:50.455355+0000","last_undegraded":"2024-09-11T23:21:50.455355+0000","last_fullsized":"2024-09-11T23:21:50.455355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:48.698896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.302151+0000","last_change":"2024-09-11T23:19:57.822949+0000","last_active":"2024-09-11T23:21:50.302151+0000","last_peered":"2024-09-11T23:21:50.302151+0000","last_clean":"2024-09-11T23:21:50.302151+0000","last_became_active":"2024-09-11T23:19:57.822564+0000","last_became_peered":"2024-09-11T23:19:57.822564+0000","last_unstale":"2024-09-11T23:21:50.302151+0000","last_undegraded":"2024-09-11T23:21:50.302151+0000","last_fullsized":"2024-09-11T23:21:50.302151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:29.532109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297740+0000","last_change":"2024-09-11T23:21:02.363639+0000","last_active":"2024-09-11T23:21:50.297740+0000","last_peered":"2024-09-11T23:21:50.297740+0000","last_clean":"2024-09-11T23:21:50.297740+0000","last_became_active":"2024-09-11T23:21:02.363247+0000","last_became_peered":"2024-09-11T23:21:02.363247+0000","last_unstale":"2024-09-11T23:21:50.297740+0000","last_undegraded":"2024-09-11T23:21:50.297740+0000","last_fullsized":"2024-09-11T23:21:50.297740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:18:58.906939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297895+0000","last_change":"2024-09-11T23:19:57.613323+0000","last_active":"2024-09-11T23:21:50.297895+0000","last_peered":"2024-09-11T23:21:50.297895+0000","last_clean":"2024-09-11T23:21:50.297895+0000","last_became_active":"2024-09-11T23:19:57.612985+0000","last_became_peered":"2024-09-11T23:19:57.612985+0000","last_unstale":"2024-09-11T23:21:50.297895+0000","last_undegraded":"2024-09-11T23:21:50.297895+0000","last_fullsized":"2024-09-11T23:21:50.297895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:13:09.179547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":797142,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1038,"num_read_kb":732,"num_write":418,"num_write_kb":2357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":2391426,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108948,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514533,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":450800,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743465,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1126400,"data_stored":1442204,"data_compressed":60217,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906992,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":844252,"data_compressed":27988,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168629,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":844172,"data_compressed":27974,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":1044601,"data_compressed":45015,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561087,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":450425,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1044480,"data_stored":1370804,"data_compressed":60217,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":452575,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1011712,"data_stored":1172184,"data_compressed":42814,"data_compressed_allocated":57344,"data_compressed_original":724992,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":521939,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":520196,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":946176,"data_stored":451696,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":330101,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":401501,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:22:03.842 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:22:04.098 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:04.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.559+0000 7f8485d51640 1 -- 172.21.15.43:0/1559892894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8480101de0 msgr2=0x7f84801021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:04.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.559+0000 7f8485d51640 1 --2- 172.21.15.43:0/1559892894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8480101de0 0x7f84801021c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8470009900 tx=0x7f847002a410 comp rx=0 tx=0).stop 2024-09-11T23:22:04.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.559+0000 7f8485d51640 1 -- 172.21.15.43:0/1559892894 shutdown_connections 2024-09-11T23:22:04.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.559+0000 7f8485d51640 1 --2- 172.21.15.43:0/1559892894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8480106f10 0x7f848010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:04.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.559+0000 7f8485d51640 1 --2- 172.21.15.43:0/1559892894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8480102790 0x7f84801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:04.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.559+0000 7f8485d51640 1 --2- 172.21.15.43:0/1559892894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8480101de0 0x7f84801021c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:04.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.560+0000 7f8485d51640 1 -- 172.21.15.43:0/1559892894 >> 172.21.15.43:0/1559892894 conn(0x7f84800fd8c0 msgr2=0x7f84800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.560+0000 7f8485d51640 1 -- 172.21.15.43:0/1559892894 shutdown_connections 2024-09-11T23:22:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.560+0000 7f8485d51640 1 -- 172.21.15.43:0/1559892894 wait complete. 2024-09-11T23:22:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.560+0000 7f8485d51640 1 Processor -- start 2024-09-11T23:22:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.560+0000 7f8485d51640 1 -- start start 2024-09-11T23:22:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.561+0000 7f8485d51640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8480101de0 0x7f848019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.561+0000 7f8485d51640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8480102790 0x7f848019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.561+0000 7f8485d51640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8480106f10 0x7f848019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.561+0000 7f8485d51640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8480111f70 con 0x7f8480106f10 2024-09-11T23:22:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.561+0000 7f8485d51640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8480111df0 con 0x7f8480102790 2024-09-11T23:22:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.561+0000 7f8485d51640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f84801120f0 con 0x7f8480101de0 2024-09-11T23:22:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.561+0000 7f847ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8480106f10 0x7f848019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.561+0000 7f847ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8480106f10 0x7f848019ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49952/0 (socket says 172.21.15.43:49952) 2024-09-11T23:22:04.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.561+0000 7f847ffff640 1 -- 172.21.15.43:0/2683070733 learned_addr learned my addr 172.21.15.43:0/2683070733 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:04.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.561+0000 7f847f7fe640 1 --2- 172.21.15.43:0/2683070733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8480101de0 0x7f848019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:04.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.561+0000 7f847effd640 1 --2- 172.21.15.43:0/2683070733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8480102790 0x7f848019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:04.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.561+0000 7f847ffff640 1 -- 172.21.15.43:0/2683070733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8480101de0 msgr2=0x7f848019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:04.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.561+0000 7f847ffff640 1 --2- 172.21.15.43:0/2683070733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8480101de0 0x7f848019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:04.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.561+0000 7f847ffff640 1 -- 172.21.15.43:0/2683070733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8480102790 msgr2=0x7f848019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:04.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.561+0000 7f847ffff640 1 --2- 172.21.15.43:0/2683070733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8480102790 0x7f848019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:04.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.561+0000 7f847ffff640 1 -- 172.21.15.43:0/2683070733 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848019f640 con 0x7f8480106f10 2024-09-11T23:22:04.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.561+0000 7f847ffff640 1 --2- 172.21.15.43:0/2683070733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8480106f10 0x7f848019ef60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f847800eb10 tx=0x7f847800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:04.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.561+0000 7f847cff9640 1 -- 172.21.15.43:0/2683070733 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478004b20 con 0x7f8480106f10 2024-09-11T23:22:04.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.561+0000 7f847cff9640 1 -- 172.21.15.43:0/2683070733 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8478004cc0 con 0x7f8480106f10 2024-09-11T23:22:04.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.561+0000 7f8485d51640 1 -- 172.21.15.43:0/2683070733 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848019f930 con 0x7f8480106f10 2024-09-11T23:22:04.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.561+0000 7f847cff9640 1 -- 172.21.15.43:0/2683070733 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478010760 con 0x7f8480106f10 2024-09-11T23:22:04.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.562+0000 7f8485d51640 1 -- 172.21.15.43:0/2683070733 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84801a7210 con 0x7f8480106f10 2024-09-11T23:22:04.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.562+0000 7f8485d51640 1 -- 172.21.15.43:0/2683070733 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84801021c0 con 0x7f8480106f10 2024-09-11T23:22:04.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.564+0000 7f847cff9640 1 -- 172.21.15.43:0/2683070733 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f8478005020 con 0x7f8480106f10 2024-09-11T23:22:04.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.564+0000 7f847cff9640 1 --2- 172.21.15.43:0/2683070733 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f84680a2e10 0x7f84680a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:04.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.564+0000 7f847f7fe640 1 --2- 172.21.15.43:0/2683070733 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f84680a2e10 0x7f84680a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:04.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.564+0000 7f847cff9640 1 -- 172.21.15.43:0/2683070733 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f84780cfbf0 con 0x7f8480106f10 2024-09-11T23:22:04.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.565+0000 7f847f7fe640 1 --2- 172.21.15.43:0/2683070733 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f84680a2e10 0x7f84680a52d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8470009230 tx=0x7f84700058d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:04.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.569+0000 7f847cff9640 1 -- 172.21.15.43:0/2683070733 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f847809c5c0 con 0x7f8480106f10 2024-09-11T23:22:04.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.742+0000 7f8485d51640 1 -- 172.21.15.43:0/2683070733 --> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f84800fec00 con 0x7f84680a2e10 2024-09-11T23:22:04.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.749+0000 7f847cff9640 1 -- 172.21.15.43:0/2683070733 <== mgr.15126 v2:172.21.15.43:6800/2510386404 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258040 (secure 0 0 0) 0x7f84800fec00 con 0x7f84680a2e10 2024-09-11T23:22:04.751 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:22:04.752 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-11T23:22:04.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.752+0000 7f8485d51640 1 -- 172.21.15.43:0/2683070733 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f84680a2e10 msgr2=0x7f84680a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:04.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.752+0000 7f8485d51640 1 --2- 172.21.15.43:0/2683070733 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f84680a2e10 0x7f84680a52d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8470009230 tx=0x7f84700058d0 comp rx=0 tx=0).stop 2024-09-11T23:22:04.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.752+0000 7f8485d51640 1 -- 172.21.15.43:0/2683070733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8480106f10 msgr2=0x7f848019ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:04.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.752+0000 7f8485d51640 1 --2- 172.21.15.43:0/2683070733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8480106f10 0x7f848019ef60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f847800eb10 tx=0x7f847800c2d0 comp rx=0 tx=0).stop 2024-09-11T23:22:04.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.752+0000 7f8485d51640 1 -- 172.21.15.43:0/2683070733 shutdown_connections 2024-09-11T23:22:04.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.752+0000 7f8485d51640 1 --2- 172.21.15.43:0/2683070733 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f84680a2e10 0x7f84680a52d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:04.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.752+0000 7f8485d51640 1 --2- 172.21.15.43:0/2683070733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8480106f10 0x7f848019ef60 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:04.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.752+0000 7f8485d51640 1 --2- 172.21.15.43:0/2683070733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8480102790 0x7f848019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:04.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.752+0000 7f8485d51640 1 --2- 172.21.15.43:0/2683070733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8480101de0 0x7f848019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:04.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.752+0000 7f8485d51640 1 -- 172.21.15.43:0/2683070733 >> 172.21.15.43:0/2683070733 conn(0x7f84800fd8c0 msgr2=0x7f84800ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:04.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.753+0000 7f8485d51640 1 -- 172.21.15.43:0/2683070733 shutdown_connections 2024-09-11T23:22:04.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:04.753+0000 7f8485d51640 1 -- 172.21.15.43:0/2683070733 wait complete. 2024-09-11T23:22:05.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:04 smithi090 ceph-mon[35130]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:22:05.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:04 smithi097 ceph-mon[35562]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:22:05.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:04 smithi043 ceph-mon[30950]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:22:05.454 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-11T23:22:04.225708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1387744,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1275,"num_read_kb":1062,"num_write":621,"num_write_kb":5297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":958,"ondisk_log_size":958,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369160,"kb_used_data":11556,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572664,"statfs":{"total":1151940427776,"available":1151562407936,"internally_reserved":0,"allocated":11833344,"data_stored":9565848,"data_compressed":316419,"data_compressed_allocated":417792,"data_compressed_original":4165632,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":199938,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":408,"num_read_kb":468,"num_write":147,"num_write_kb":920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002760"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598561+0000","last_change":"2024-09-11T23:19:59.734378+0000","last_active":"2024-09-11T23:21:50.598561+0000","last_peered":"2024-09-11T23:21:50.598561+0000","last_clean":"2024-09-11T23:21:50.598561+0000","last_became_active":"2024-09-11T23:19:59.734067+0000","last_became_peered":"2024-09-11T23:19:59.734067+0000","last_unstale":"2024-09-11T23:21:50.598561+0000","last_undegraded":"2024-09-11T23:21:50.598561+0000","last_fullsized":"2024-09-11T23:21:50.598561+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:28:18.384453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.454939+0000","last_change":"2024-09-11T23:19:59.733892+0000","last_active":"2024-09-11T23:21:50.454939+0000","last_peered":"2024-09-11T23:21:50.454939+0000","last_clean":"2024-09-11T23:21:50.454939+0000","last_became_active":"2024-09-11T23:19:59.733540+0000","last_became_peered":"2024-09-11T23:19:59.733540+0000","last_unstale":"2024-09-11T23:21:50.454939+0000","last_undegraded":"2024-09-11T23:21:50.454939+0000","last_fullsized":"2024-09-11T23:21:50.454939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:42.186458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455001+0000","last_change":"2024-09-11T23:19:59.739443+0000","last_active":"2024-09-11T23:21:50.455001+0000","last_peered":"2024-09-11T23:21:50.455001+0000","last_clean":"2024-09-11T23:21:50.455001+0000","last_became_active":"2024-09-11T23:19:59.739025+0000","last_became_peered":"2024-09-11T23:19:59.739025+0000","last_unstale":"2024-09-11T23:21:50.455001+0000","last_undegraded":"2024-09-11T23:21:50.455001+0000","last_fullsized":"2024-09-11T23:21:50.455001+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:40.819321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297376+0000","last_change":"2024-09-11T23:21:02.363511+0000","last_active":"2024-09-11T23:21:50.297376+0000","last_peered":"2024-09-11T23:21:50.297376+0000","last_clean":"2024-09-11T23:21:50.297376+0000","last_became_active":"2024-09-11T23:21:02.363156+0000","last_became_peered":"2024-09-11T23:21:02.363156+0000","last_unstale":"2024-09-11T23:21:50.297376+0000","last_undegraded":"2024-09-11T23:21:50.297376+0000","last_fullsized":"2024-09-11T23:21:50.297376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:17.059509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299585+0000","last_change":"2024-09-11T23:19:59.738662+0000","last_active":"2024-09-11T23:21:50.299585+0000","last_peered":"2024-09-11T23:21:50.299585+0000","last_clean":"2024-09-11T23:21:50.299585+0000","last_became_active":"2024-09-11T23:19:59.738320+0000","last_became_peered":"2024-09-11T23:19:59.738320+0000","last_unstale":"2024-09-11T23:21:50.299585+0000","last_undegraded":"2024-09-11T23:21:50.299585+0000","last_fullsized":"2024-09-11T23:21:50.299585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:15.645179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.310947+0000","last_change":"2024-09-11T23:19:59.731442+0000","last_active":"2024-09-11T23:21:50.310947+0000","last_peered":"2024-09-11T23:21:50.310947+0000","last_clean":"2024-09-11T23:21:50.310947+0000","last_became_active":"2024-09-11T23:19:59.731079+0000","last_became_peered":"2024-09-11T23:19:59.731079+0000","last_unstale":"2024-09-11T23:21:50.310947+0000","last_undegraded":"2024-09-11T23:21:50.310947+0000","last_fullsized":"2024-09-11T23:21:50.310947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:32.764991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455210+0000","last_change":"2024-09-11T23:21:02.359992+0000","last_active":"2024-09-11T23:21:50.455210+0000","last_peered":"2024-09-11T23:21:50.455210+0000","last_clean":"2024-09-11T23:21:50.455210+0000","last_became_active":"2024-09-11T23:21:02.359564+0000","last_became_peered":"2024-09-11T23:21:02.359564+0000","last_unstale":"2024-09-11T23:21:50.455210+0000","last_undegraded":"2024-09-11T23:21:50.455210+0000","last_fullsized":"2024-09-11T23:21:50.455210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:53.633219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300215+0000","last_change":"2024-09-11T23:19:59.740733+0000","last_active":"2024-09-11T23:21:50.300215+0000","last_peered":"2024-09-11T23:21:50.300215+0000","last_clean":"2024-09-11T23:21:50.300215+0000","last_became_active":"2024-09-11T23:19:59.740438+0000","last_became_peered":"2024-09-11T23:19:59.740438+0000","last_unstale":"2024-09-11T23:21:50.300215+0000","last_undegraded":"2024-09-11T23:21:50.300215+0000","last_fullsized":"2024-09-11T23:21:50.300215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:27.063305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.212024+0000","last_change":"2024-09-11T23:19:59.742439+0000","last_active":"2024-09-11T23:21:50.212024+0000","last_peered":"2024-09-11T23:21:50.212024+0000","last_clean":"2024-09-11T23:21:50.212024+0000","last_became_active":"2024-09-11T23:19:59.741983+0000","last_became_peered":"2024-09-11T23:19:59.741983+0000","last_unstale":"2024-09-11T23:21:50.212024+0000","last_undegraded":"2024-09-11T23:21:50.212024+0000","last_fullsized":"2024-09-11T23:21:50.212024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:39.477167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311317+0000","last_change":"2024-09-11T23:19:59.732544+0000","last_active":"2024-09-11T23:21:50.311317+0000","last_peered":"2024-09-11T23:21:50.311317+0000","last_clean":"2024-09-11T23:21:50.311317+0000","last_became_active":"2024-09-11T23:19:59.732205+0000","last_became_peered":"2024-09-11T23:19:59.732205+0000","last_unstale":"2024-09-11T23:21:50.311317+0000","last_undegraded":"2024-09-11T23:21:50.311317+0000","last_fullsized":"2024-09-11T23:21:50.311317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:22.494344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455518+0000","last_change":"2024-09-11T23:19:59.741857+0000","last_active":"2024-09-11T23:21:50.455518+0000","last_peered":"2024-09-11T23:21:50.455518+0000","last_clean":"2024-09-11T23:21:50.455518+0000","last_became_active":"2024-09-11T23:19:59.741484+0000","last_became_peered":"2024-09-11T23:19:59.741484+0000","last_unstale":"2024-09-11T23:21:50.455518+0000","last_undegraded":"2024-09-11T23:21:50.455518+0000","last_fullsized":"2024-09-11T23:21:50.455518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:52.998258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.453834+0000","last_change":"2024-09-11T23:19:59.736042+0000","last_active":"2024-09-11T23:21:50.453834+0000","last_peered":"2024-09-11T23:21:50.453834+0000","last_clean":"2024-09-11T23:21:50.453834+0000","last_became_active":"2024-09-11T23:19:59.735731+0000","last_became_peered":"2024-09-11T23:19:59.735731+0000","last_unstale":"2024-09-11T23:21:50.453834+0000","last_undegraded":"2024-09-11T23:21:50.453834+0000","last_fullsized":"2024-09-11T23:21:50.453834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:19.085804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210129+0000","last_change":"2024-09-11T23:19:59.742049+0000","last_active":"2024-09-11T23:21:50.210129+0000","last_peered":"2024-09-11T23:21:50.210129+0000","last_clean":"2024-09-11T23:21:50.210129+0000","last_became_active":"2024-09-11T23:19:59.740974+0000","last_became_peered":"2024-09-11T23:19:59.740974+0000","last_unstale":"2024-09-11T23:21:50.210129+0000","last_undegraded":"2024-09-11T23:21:50.210129+0000","last_fullsized":"2024-09-11T23:21:50.210129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:10.443113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.454569+0000","last_change":"2024-09-11T23:19:59.734270+0000","last_active":"2024-09-11T23:21:50.454569+0000","last_peered":"2024-09-11T23:21:50.454569+0000","last_clean":"2024-09-11T23:21:50.454569+0000","last_became_active":"2024-09-11T23:19:59.733967+0000","last_became_peered":"2024-09-11T23:19:59.733967+0000","last_unstale":"2024-09-11T23:21:50.454569+0000","last_undegraded":"2024-09-11T23:21:50.454569+0000","last_fullsized":"2024-09-11T23:21:50.454569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:02.185757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.454215+0000","last_change":"2024-09-11T23:19:59.734152+0000","last_active":"2024-09-11T23:21:50.454215+0000","last_peered":"2024-09-11T23:21:50.454215+0000","last_clean":"2024-09-11T23:21:50.454215+0000","last_became_active":"2024-09-11T23:19:59.733163+0000","last_became_peered":"2024-09-11T23:19:59.733163+0000","last_unstale":"2024-09-11T23:21:50.454215+0000","last_undegraded":"2024-09-11T23:21:50.454215+0000","last_fullsized":"2024-09-11T23:21:50.454215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:22.370591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211142+0000","last_change":"2024-09-11T23:19:59.735517+0000","last_active":"2024-09-11T23:21:50.211142+0000","last_peered":"2024-09-11T23:21:50.211142+0000","last_clean":"2024-09-11T23:21:50.211142+0000","last_became_active":"2024-09-11T23:19:59.735182+0000","last_became_peered":"2024-09-11T23:19:59.735182+0000","last_unstale":"2024-09-11T23:21:50.211142+0000","last_undegraded":"2024-09-11T23:21:50.211142+0000","last_fullsized":"2024-09-11T23:21:50.211142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:25.989416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598585+0000","last_change":"2024-09-11T23:19:59.733579+0000","last_active":"2024-09-11T23:21:50.598585+0000","last_peered":"2024-09-11T23:21:50.598585+0000","last_clean":"2024-09-11T23:21:50.598585+0000","last_became_active":"2024-09-11T23:19:59.733245+0000","last_became_peered":"2024-09-11T23:19:59.733245+0000","last_unstale":"2024-09-11T23:21:50.598585+0000","last_undegraded":"2024-09-11T23:21:50.598585+0000","last_fullsized":"2024-09-11T23:21:50.598585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:22.361158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297955+0000","last_change":"2024-09-11T23:19:59.734354+0000","last_active":"2024-09-11T23:21:50.297955+0000","last_peered":"2024-09-11T23:21:50.297955+0000","last_clean":"2024-09-11T23:21:50.297955+0000","last_became_active":"2024-09-11T23:19:59.733988+0000","last_became_peered":"2024-09-11T23:19:59.733988+0000","last_unstale":"2024-09-11T23:21:50.297955+0000","last_undegraded":"2024-09-11T23:21:50.297955+0000","last_fullsized":"2024-09-11T23:21:50.297955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:41.401496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.212032+0000","last_change":"2024-09-11T23:19:59.730318+0000","last_active":"2024-09-11T23:21:50.212032+0000","last_peered":"2024-09-11T23:21:50.212032+0000","last_clean":"2024-09-11T23:21:50.212032+0000","last_became_active":"2024-09-11T23:19:59.730009+0000","last_became_peered":"2024-09-11T23:19:59.730009+0000","last_unstale":"2024-09-11T23:21:50.212032+0000","last_undegraded":"2024-09-11T23:21:50.212032+0000","last_fullsized":"2024-09-11T23:21:50.212032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:01:38.531967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598328+0000","last_change":"2024-09-11T23:19:59.737254+0000","last_active":"2024-09-11T23:21:50.598328+0000","last_peered":"2024-09-11T23:21:50.598328+0000","last_clean":"2024-09-11T23:21:50.598328+0000","last_became_active":"2024-09-11T23:19:59.736936+0000","last_became_peered":"2024-09-11T23:19:59.736936+0000","last_unstale":"2024-09-11T23:21:50.598328+0000","last_undegraded":"2024-09-11T23:21:50.598328+0000","last_fullsized":"2024-09-11T23:21:50.598328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:00.361468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210147+0000","last_change":"2024-09-11T23:19:59.741617+0000","last_active":"2024-09-11T23:21:50.210147+0000","last_peered":"2024-09-11T23:21:50.210147+0000","last_clean":"2024-09-11T23:21:50.210147+0000","last_became_active":"2024-09-11T23:19:59.740358+0000","last_became_peered":"2024-09-11T23:19:59.740358+0000","last_unstale":"2024-09-11T23:21:50.210147+0000","last_undegraded":"2024-09-11T23:21:50.210147+0000","last_fullsized":"2024-09-11T23:21:50.210147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:18.541105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211292+0000","last_change":"2024-09-11T23:19:59.736437+0000","last_active":"2024-09-11T23:21:50.211292+0000","last_peered":"2024-09-11T23:21:50.211292+0000","last_clean":"2024-09-11T23:21:50.211292+0000","last_became_active":"2024-09-11T23:19:59.736029+0000","last_became_peered":"2024-09-11T23:19:59.736029+0000","last_unstale":"2024-09-11T23:21:50.211292+0000","last_undegraded":"2024-09-11T23:21:50.211292+0000","last_fullsized":"2024-09-11T23:21:50.211292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:29.226939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300441+0000","last_change":"2024-09-11T23:19:59.736104+0000","last_active":"2024-09-11T23:21:50.300441+0000","last_peered":"2024-09-11T23:21:50.300441+0000","last_clean":"2024-09-11T23:21:50.300441+0000","last_became_active":"2024-09-11T23:19:59.735760+0000","last_became_peered":"2024-09-11T23:19:59.735760+0000","last_unstale":"2024-09-11T23:21:50.300441+0000","last_undegraded":"2024-09-11T23:21:50.300441+0000","last_fullsized":"2024-09-11T23:21:50.300441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:56.554313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308481+0000","last_change":"2024-09-11T23:19:59.735455+0000","last_active":"2024-09-11T23:21:50.308481+0000","last_peered":"2024-09-11T23:21:50.308481+0000","last_clean":"2024-09-11T23:21:50.308481+0000","last_became_active":"2024-09-11T23:19:59.735078+0000","last_became_peered":"2024-09-11T23:19:59.735078+0000","last_unstale":"2024-09-11T23:21:50.308481+0000","last_undegraded":"2024-09-11T23:21:50.308481+0000","last_fullsized":"2024-09-11T23:21:50.308481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:21:16.336006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.301909+0000","last_change":"2024-09-11T23:19:59.745997+0000","last_active":"2024-09-11T23:21:50.301909+0000","last_peered":"2024-09-11T23:21:50.301909+0000","last_clean":"2024-09-11T23:21:50.301909+0000","last_became_active":"2024-09-11T23:19:59.745715+0000","last_became_peered":"2024-09-11T23:19:59.745715+0000","last_unstale":"2024-09-11T23:21:50.301909+0000","last_undegraded":"2024-09-11T23:21:50.301909+0000","last_fullsized":"2024-09-11T23:21:50.301909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:34.776229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.212119+0000","last_change":"2024-09-11T23:19:59.734181+0000","last_active":"2024-09-11T23:21:50.212119+0000","last_peered":"2024-09-11T23:21:50.212119+0000","last_clean":"2024-09-11T23:21:50.212119+0000","last_became_active":"2024-09-11T23:19:59.733793+0000","last_became_peered":"2024-09-11T23:19:59.733793+0000","last_unstale":"2024-09-11T23:21:50.212119+0000","last_undegraded":"2024-09-11T23:21:50.212119+0000","last_fullsized":"2024-09-11T23:21:50.212119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:36.906308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598526+0000","last_change":"2024-09-11T23:19:59.738713+0000","last_active":"2024-09-11T23:21:50.598526+0000","last_peered":"2024-09-11T23:21:50.598526+0000","last_clean":"2024-09-11T23:21:50.598526+0000","last_became_active":"2024-09-11T23:19:59.737579+0000","last_became_peered":"2024-09-11T23:19:59.737579+0000","last_unstale":"2024-09-11T23:21:50.598526+0000","last_undegraded":"2024-09-11T23:21:50.598526+0000","last_fullsized":"2024-09-11T23:21:50.598526+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:47.135775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210205+0000","last_change":"2024-09-11T23:19:59.741381+0000","last_active":"2024-09-11T23:21:50.210205+0000","last_peered":"2024-09-11T23:21:50.210205+0000","last_clean":"2024-09-11T23:21:50.210205+0000","last_became_active":"2024-09-11T23:19:59.740914+0000","last_became_peered":"2024-09-11T23:19:59.740914+0000","last_unstale":"2024-09-11T23:21:50.210205+0000","last_undegraded":"2024-09-11T23:21:50.210205+0000","last_fullsized":"2024-09-11T23:21:50.210205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:24:04.365704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455133+0000","last_change":"2024-09-11T23:19:59.736840+0000","last_active":"2024-09-11T23:21:50.455133+0000","last_peered":"2024-09-11T23:21:50.455133+0000","last_clean":"2024-09-11T23:21:50.455133+0000","last_became_active":"2024-09-11T23:19:59.736554+0000","last_became_peered":"2024-09-11T23:19:59.736554+0000","last_unstale":"2024-09-11T23:21:50.455133+0000","last_undegraded":"2024-09-11T23:21:50.455133+0000","last_fullsized":"2024-09-11T23:21:50.455133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:48.508369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211738+0000","last_change":"2024-09-11T23:19:59.735719+0000","last_active":"2024-09-11T23:21:50.211738+0000","last_peered":"2024-09-11T23:21:50.211738+0000","last_clean":"2024-09-11T23:21:50.211738+0000","last_became_active":"2024-09-11T23:19:59.735397+0000","last_became_peered":"2024-09-11T23:19:59.735397+0000","last_unstale":"2024-09-11T23:21:50.211738+0000","last_undegraded":"2024-09-11T23:21:50.211738+0000","last_fullsized":"2024-09-11T23:21:50.211738+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:26:19.392730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211747+0000","last_change":"2024-09-11T23:19:59.737704+0000","last_active":"2024-09-11T23:21:50.211747+0000","last_peered":"2024-09-11T23:21:50.211747+0000","last_clean":"2024-09-11T23:21:50.211747+0000","last_became_active":"2024-09-11T23:19:59.737176+0000","last_became_peered":"2024-09-11T23:19:59.737176+0000","last_unstale":"2024-09-11T23:21:50.211747+0000","last_undegraded":"2024-09-11T23:21:50.211747+0000","last_fullsized":"2024-09-11T23:21:50.211747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:22.938171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598301+0000","last_change":"2024-09-11T23:19:59.735499+0000","last_active":"2024-09-11T23:21:50.598301+0000","last_peered":"2024-09-11T23:21:50.598301+0000","last_clean":"2024-09-11T23:21:50.598301+0000","last_became_active":"2024-09-11T23:19:59.735088+0000","last_became_peered":"2024-09-11T23:19:59.735088+0000","last_unstale":"2024-09-11T23:21:50.598301+0000","last_undegraded":"2024-09-11T23:21:50.598301+0000","last_fullsized":"2024-09-11T23:21:50.598301+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:12.633104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299549+0000","last_change":"2024-09-11T23:19:59.743972+0000","last_active":"2024-09-11T23:21:50.299549+0000","last_peered":"2024-09-11T23:21:50.299549+0000","last_clean":"2024-09-11T23:21:50.299549+0000","last_became_active":"2024-09-11T23:19:59.743675+0000","last_became_peered":"2024-09-11T23:19:59.743675+0000","last_unstale":"2024-09-11T23:21:50.299549+0000","last_undegraded":"2024-09-11T23:21:50.299549+0000","last_fullsized":"2024-09-11T23:21:50.299549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:46.645696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":144,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:59.879602+0000","last_change":"2024-09-11T23:19:57.615472+0000","last_active":"2024-09-11T23:21:59.879602+0000","last_peered":"2024-09-11T23:21:59.879602+0000","last_clean":"2024-09-11T23:21:59.879602+0000","last_became_active":"2024-09-11T23:19:57.615104+0000","last_became_peered":"2024-09-11T23:19:57.615104+0000","last_unstale":"2024-09-11T23:21:59.879602+0000","last_undegraded":"2024-09-11T23:21:59.879602+0000","last_fullsized":"2024-09-11T23:21:59.879602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:03.601363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308144+0000","last_change":"2024-09-11T23:19:59.741027+0000","last_active":"2024-09-11T23:21:50.308144+0000","last_peered":"2024-09-11T23:21:50.308144+0000","last_clean":"2024-09-11T23:21:50.308144+0000","last_became_active":"2024-09-11T23:19:59.740681+0000","last_became_peered":"2024-09-11T23:19:59.740681+0000","last_unstale":"2024-09-11T23:21:50.308144+0000","last_undegraded":"2024-09-11T23:21:50.308144+0000","last_fullsized":"2024-09-11T23:21:50.308144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:36.948147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311025+0000","last_change":"2024-09-11T23:19:57.612730+0000","last_active":"2024-09-11T23:21:50.311025+0000","last_peered":"2024-09-11T23:21:50.311025+0000","last_clean":"2024-09-11T23:21:50.311025+0000","last_became_active":"2024-09-11T23:19:57.612369+0000","last_became_peered":"2024-09-11T23:19:57.612369+0000","last_unstale":"2024-09-11T23:21:50.311025+0000","last_undegraded":"2024-09-11T23:21:50.311025+0000","last_fullsized":"2024-09-11T23:21:50.311025+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:17.823616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.209881+0000","last_change":"2024-09-11T23:19:57.611470+0000","last_active":"2024-09-11T23:21:50.209881+0000","last_peered":"2024-09-11T23:21:50.209881+0000","last_clean":"2024-09-11T23:21:50.209881+0000","last_became_active":"2024-09-11T23:19:57.611141+0000","last_became_peered":"2024-09-11T23:19:57.611141+0000","last_unstale":"2024-09-11T23:21:50.209881+0000","last_undegraded":"2024-09-11T23:21:50.209881+0000","last_fullsized":"2024-09-11T23:21:50.209881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:43.173955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300408+0000","last_change":"2024-09-11T23:19:59.732375+0000","last_active":"2024-09-11T23:21:50.300408+0000","last_peered":"2024-09-11T23:21:50.300408+0000","last_clean":"2024-09-11T23:21:50.300408+0000","last_became_active":"2024-09-11T23:19:59.731793+0000","last_became_peered":"2024-09-11T23:19:59.731793+0000","last_unstale":"2024-09-11T23:21:50.300408+0000","last_undegraded":"2024-09-11T23:21:50.300408+0000","last_fullsized":"2024-09-11T23:21:50.300408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:02:47.278646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308399+0000","last_change":"2024-09-11T23:19:57.614062+0000","last_active":"2024-09-11T23:21:50.308399+0000","last_peered":"2024-09-11T23:21:50.308399+0000","last_clean":"2024-09-11T23:21:50.308399+0000","last_became_active":"2024-09-11T23:19:57.613749+0000","last_became_peered":"2024-09-11T23:19:57.613749+0000","last_unstale":"2024-09-11T23:21:50.308399+0000","last_undegraded":"2024-09-11T23:21:50.308399+0000","last_fullsized":"2024-09-11T23:21:50.308399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:45:22.299379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297086+0000","last_change":"2024-09-11T23:19:59.730758+0000","last_active":"2024-09-11T23:21:50.297086+0000","last_peered":"2024-09-11T23:21:50.297086+0000","last_clean":"2024-09-11T23:21:50.297086+0000","last_became_active":"2024-09-11T23:19:59.730423+0000","last_became_peered":"2024-09-11T23:19:59.730423+0000","last_unstale":"2024-09-11T23:21:50.297086+0000","last_undegraded":"2024-09-11T23:21:50.297086+0000","last_fullsized":"2024-09-11T23:21:50.297086+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:30.679451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":139,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:59.879280+0000","last_change":"2024-09-11T23:19:57.615574+0000","last_active":"2024-09-11T23:21:59.879280+0000","last_peered":"2024-09-11T23:21:59.879280+0000","last_clean":"2024-09-11T23:21:59.879280+0000","last_became_active":"2024-09-11T23:19:57.615192+0000","last_became_peered":"2024-09-11T23:19:57.615192+0000","last_unstale":"2024-09-11T23:21:59.879280+0000","last_undegraded":"2024-09-11T23:21:59.879280+0000","last_fullsized":"2024-09-11T23:21:59.879280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:29.186899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211749+0000","last_change":"2024-09-11T23:19:59.738433+0000","last_active":"2024-09-11T23:21:50.211749+0000","last_peered":"2024-09-11T23:21:50.211749+0000","last_clean":"2024-09-11T23:21:50.211749+0000","last_became_active":"2024-09-11T23:19:59.738081+0000","last_became_peered":"2024-09-11T23:19:59.738081+0000","last_unstale":"2024-09-11T23:21:50.211749+0000","last_undegraded":"2024-09-11T23:21:50.211749+0000","last_fullsized":"2024-09-11T23:21:50.211749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:10:46.786454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455225+0000","last_change":"2024-09-11T23:21:02.437498+0000","last_active":"2024-09-11T23:21:50.455225+0000","last_peered":"2024-09-11T23:21:50.455225+0000","last_clean":"2024-09-11T23:21:50.455225+0000","last_became_active":"2024-09-11T23:21:02.434895+0000","last_became_peered":"2024-09-11T23:21:02.434895+0000","last_unstale":"2024-09-11T23:21:50.455225+0000","last_undegraded":"2024-09-11T23:21:50.455225+0000","last_fullsized":"2024-09-11T23:21:50.455225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:33.142041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455584+0000","last_change":"2024-09-11T23:19:59.741081+0000","last_active":"2024-09-11T23:21:50.455584+0000","last_peered":"2024-09-11T23:21:50.455584+0000","last_clean":"2024-09-11T23:21:50.455584+0000","last_became_active":"2024-09-11T23:19:59.739984+0000","last_became_peered":"2024-09-11T23:19:59.739984+0000","last_unstale":"2024-09-11T23:21:50.455584+0000","last_undegraded":"2024-09-11T23:21:50.455584+0000","last_fullsized":"2024-09-11T23:21:50.455584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:03.513121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308186+0000","last_change":"2024-09-11T23:19:57.823272+0000","last_active":"2024-09-11T23:21:50.308186+0000","last_peered":"2024-09-11T23:21:50.308186+0000","last_clean":"2024-09-11T23:21:50.308186+0000","last_became_active":"2024-09-11T23:19:57.822957+0000","last_became_peered":"2024-09-11T23:19:57.822957+0000","last_unstale":"2024-09-11T23:21:50.308186+0000","last_undegraded":"2024-09-11T23:21:50.308186+0000","last_fullsized":"2024-09-11T23:21:50.308186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:06.779385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211510+0000","last_change":"2024-09-11T23:19:59.738398+0000","last_active":"2024-09-11T23:21:50.211510+0000","last_peered":"2024-09-11T23:21:50.211510+0000","last_clean":"2024-09-11T23:21:50.211510+0000","last_became_active":"2024-09-11T23:19:59.738042+0000","last_became_peered":"2024-09-11T23:19:59.738042+0000","last_unstale":"2024-09-11T23:21:50.211510+0000","last_undegraded":"2024-09-11T23:21:50.211510+0000","last_fullsized":"2024-09-11T23:21:50.211510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:05.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'36","reported_seq":296,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:22:00.880783+0000","last_change":"2024-09-11T23:19:57.615036+0000","last_active":"2024-09-11T23:22:00.880783+0000","last_peered":"2024-09-11T23:22:00.880783+0000","last_clean":"2024-09-11T23:22:00.880783+0000","last_became_active":"2024-09-11T23:19:57.614641+0000","last_became_peered":"2024-09-11T23:19:57.614641+0000","last_unstale":"2024-09-11T23:22:00.880783+0000","last_undegraded":"2024-09-11T23:22:00.880783+0000","last_fullsized":"2024-09-11T23:22:00.880783+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:05.430508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69978,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":236,"num_read_kb":187,"num_write":37,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299857+0000","last_change":"2024-09-11T23:19:59.741597+0000","last_active":"2024-09-11T23:21:50.299857+0000","last_peered":"2024-09-11T23:21:50.299857+0000","last_clean":"2024-09-11T23:21:50.299857+0000","last_became_active":"2024-09-11T23:19:59.741296+0000","last_became_peered":"2024-09-11T23:19:59.741296+0000","last_unstale":"2024-09-11T23:21:50.299857+0000","last_undegraded":"2024-09-11T23:21:50.299857+0000","last_fullsized":"2024-09-11T23:21:50.299857+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:06:20.742916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211837+0000","last_change":"2024-09-11T23:19:57.617550+0000","last_active":"2024-09-11T23:21:50.211837+0000","last_peered":"2024-09-11T23:21:50.211837+0000","last_clean":"2024-09-11T23:21:50.211837+0000","last_became_active":"2024-09-11T23:19:57.617211+0000","last_became_peered":"2024-09-11T23:19:57.617211+0000","last_unstale":"2024-09-11T23:21:50.211837+0000","last_undegraded":"2024-09-11T23:21:50.211837+0000","last_fullsized":"2024-09-11T23:21:50.211837+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:19.707742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210301+0000","last_change":"2024-09-11T23:19:59.742070+0000","last_active":"2024-09-11T23:21:50.210301+0000","last_peered":"2024-09-11T23:21:50.210301+0000","last_clean":"2024-09-11T23:21:50.210301+0000","last_became_active":"2024-09-11T23:19:59.739367+0000","last_became_peered":"2024-09-11T23:19:59.739367+0000","last_unstale":"2024-09-11T23:21:50.210301+0000","last_undegraded":"2024-09-11T23:21:50.210301+0000","last_fullsized":"2024-09-11T23:21:50.210301+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:27.850136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308168+0000","last_change":"2024-09-11T23:19:57.618054+0000","last_active":"2024-09-11T23:21:50.308168+0000","last_peered":"2024-09-11T23:21:50.308168+0000","last_clean":"2024-09-11T23:21:50.308168+0000","last_became_active":"2024-09-11T23:19:57.617702+0000","last_became_peered":"2024-09-11T23:19:57.617702+0000","last_unstale":"2024-09-11T23:21:50.308168+0000","last_undegraded":"2024-09-11T23:21:50.308168+0000","last_fullsized":"2024-09-11T23:21:50.308168+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:40.171851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455548+0000","last_change":"2024-09-11T23:19:59.744000+0000","last_active":"2024-09-11T23:21:50.455548+0000","last_peered":"2024-09-11T23:21:50.455548+0000","last_clean":"2024-09-11T23:21:50.455548+0000","last_became_active":"2024-09-11T23:19:59.743629+0000","last_became_peered":"2024-09-11T23:19:59.743629+0000","last_unstale":"2024-09-11T23:21:50.455548+0000","last_undegraded":"2024-09-11T23:21:50.455548+0000","last_fullsized":"2024-09-11T23:21:50.455548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:20:04.133258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'8","reported_seq":149,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:59.879721+0000","last_change":"2024-09-11T23:19:57.615117+0000","last_active":"2024-09-11T23:21:59.879721+0000","last_peered":"2024-09-11T23:21:59.879721+0000","last_clean":"2024-09-11T23:21:59.879721+0000","last_became_active":"2024-09-11T23:19:57.614758+0000","last_became_peered":"2024-09-11T23:19:57.614758+0000","last_unstale":"2024-09-11T23:21:59.879721+0000","last_undegraded":"2024-09-11T23:21:59.879721+0000","last_fullsized":"2024-09-11T23:21:59.879721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:58.113986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2301,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":7,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455281+0000","last_change":"2024-09-11T23:19:59.739403+0000","last_active":"2024-09-11T23:21:50.455281+0000","last_peered":"2024-09-11T23:21:50.455281+0000","last_clean":"2024-09-11T23:21:50.455281+0000","last_became_active":"2024-09-11T23:19:59.739078+0000","last_became_peered":"2024-09-11T23:19:59.739078+0000","last_unstale":"2024-09-11T23:21:50.455281+0000","last_undegraded":"2024-09-11T23:21:50.455281+0000","last_fullsized":"2024-09-11T23:21:50.455281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:46.168720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299488+0000","last_change":"2024-09-11T23:19:57.823323+0000","last_active":"2024-09-11T23:21:50.299488+0000","last_peered":"2024-09-11T23:21:50.299488+0000","last_clean":"2024-09-11T23:21:50.299488+0000","last_became_active":"2024-09-11T23:19:57.822947+0000","last_became_peered":"2024-09-11T23:19:57.822947+0000","last_unstale":"2024-09-11T23:21:50.299488+0000","last_undegraded":"2024-09-11T23:21:50.299488+0000","last_fullsized":"2024-09-11T23:21:50.299488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:12.542911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308091+0000","last_change":"2024-09-11T23:19:59.741163+0000","last_active":"2024-09-11T23:21:50.308091+0000","last_peered":"2024-09-11T23:21:50.308091+0000","last_clean":"2024-09-11T23:21:50.308091+0000","last_became_active":"2024-09-11T23:19:59.740633+0000","last_became_peered":"2024-09-11T23:19:59.740633+0000","last_unstale":"2024-09-11T23:21:50.308091+0000","last_undegraded":"2024-09-11T23:21:50.308091+0000","last_fullsized":"2024-09-11T23:21:50.308091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:26.301775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299941+0000","last_change":"2024-09-11T23:19:57.615804+0000","last_active":"2024-09-11T23:21:50.299941+0000","last_peered":"2024-09-11T23:21:50.299941+0000","last_clean":"2024-09-11T23:21:50.299941+0000","last_became_active":"2024-09-11T23:19:57.615443+0000","last_became_peered":"2024-09-11T23:19:57.615443+0000","last_unstale":"2024-09-11T23:21:50.299941+0000","last_undegraded":"2024-09-11T23:21:50.299941+0000","last_fullsized":"2024-09-11T23:21:50.299941+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:39.296873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598177+0000","last_change":"2024-09-11T23:21:02.360875+0000","last_active":"2024-09-11T23:21:50.598177+0000","last_peered":"2024-09-11T23:21:50.598177+0000","last_clean":"2024-09-11T23:21:50.598177+0000","last_became_active":"2024-09-11T23:21:02.360480+0000","last_became_peered":"2024-09-11T23:21:02.360480+0000","last_unstale":"2024-09-11T23:21:50.598177+0000","last_undegraded":"2024-09-11T23:21:50.598177+0000","last_fullsized":"2024-09-11T23:21:50.598177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:04.141587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210448+0000","last_change":"2024-09-11T23:19:57.612873+0000","last_active":"2024-09-11T23:21:50.210448+0000","last_peered":"2024-09-11T23:21:50.210448+0000","last_clean":"2024-09-11T23:21:50.210448+0000","last_became_active":"2024-09-11T23:19:57.612566+0000","last_became_peered":"2024-09-11T23:19:57.612566+0000","last_unstale":"2024-09-11T23:21:50.210448+0000","last_undegraded":"2024-09-11T23:21:50.210448+0000","last_fullsized":"2024-09-11T23:21:50.210448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:46.218212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455770+0000","last_change":"2024-09-11T23:19:59.736352+0000","last_active":"2024-09-11T23:21:50.455770+0000","last_peered":"2024-09-11T23:21:50.455770+0000","last_clean":"2024-09-11T23:21:50.455770+0000","last_became_active":"2024-09-11T23:19:59.735402+0000","last_became_peered":"2024-09-11T23:19:59.735402+0000","last_unstale":"2024-09-11T23:21:50.455770+0000","last_undegraded":"2024-09-11T23:21:50.455770+0000","last_fullsized":"2024-09-11T23:21:50.455770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:41.854526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'65","reported_seq":119,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:22:00.820163+0000","last_change":"2024-09-11T23:19:57.612439+0000","last_active":"2024-09-11T23:22:00.820163+0000","last_peered":"2024-09-11T23:22:00.820163+0000","last_clean":"2024-09-11T23:22:00.820163+0000","last_became_active":"2024-09-11T23:19:57.612030+0000","last_became_peered":"2024-09-11T23:19:57.612030+0000","last_unstale":"2024-09-11T23:22:00.820163+0000","last_undegraded":"2024-09-11T23:22:00.820163+0000","last_fullsized":"2024-09-11T23:22:00.820163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:36:24.579246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455480+0000","last_change":"2024-09-11T23:19:59.741241+0000","last_active":"2024-09-11T23:21:50.455480+0000","last_peered":"2024-09-11T23:21:50.455480+0000","last_clean":"2024-09-11T23:21:50.455480+0000","last_became_active":"2024-09-11T23:19:59.740265+0000","last_became_peered":"2024-09-11T23:19:59.740265+0000","last_unstale":"2024-09-11T23:21:50.455480+0000","last_undegraded":"2024-09-11T23:21:50.455480+0000","last_fullsized":"2024-09-11T23:21:50.455480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:00.676762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211975+0000","last_change":"2024-09-11T23:19:57.825238+0000","last_active":"2024-09-11T23:21:50.211975+0000","last_peered":"2024-09-11T23:21:50.211975+0000","last_clean":"2024-09-11T23:21:50.211975+0000","last_became_active":"2024-09-11T23:19:57.824732+0000","last_became_peered":"2024-09-11T23:19:57.824732+0000","last_unstale":"2024-09-11T23:21:50.211975+0000","last_undegraded":"2024-09-11T23:21:50.211975+0000","last_fullsized":"2024-09-11T23:21:50.211975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:24.465863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455634+0000","last_change":"2024-09-11T23:19:59.741338+0000","last_active":"2024-09-11T23:21:50.455634+0000","last_peered":"2024-09-11T23:21:50.455634+0000","last_clean":"2024-09-11T23:21:50.455634+0000","last_became_active":"2024-09-11T23:19:59.741019+0000","last_became_peered":"2024-09-11T23:19:59.741019+0000","last_unstale":"2024-09-11T23:21:50.455634+0000","last_undegraded":"2024-09-11T23:21:50.455634+0000","last_fullsized":"2024-09-11T23:21:50.455634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:36:03.480006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"85'148","reported_seq":395,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:52.311759+0000","last_change":"2024-09-11T23:18:45.606646+0000","last_active":"2024-09-11T23:21:52.311759+0000","last_peered":"2024-09-11T23:21:52.311759+0000","last_clean":"2024-09-11T23:21:52.311759+0000","last_became_active":"2024-09-11T23:18:45.606065+0000","last_became_peered":"2024-09-11T23:18:45.606065+0000","last_unstale":"2024-09-11T23:21:52.311759+0000","last_undegraded":"2024-09-11T23:21:52.311759+0000","last_fullsized":"2024-09-11T23:21:52.311759+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:15:17.148562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:15:17.148562+0000","last_clean_scrub_stamp":"2024-09-11T23:15:17.148562+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:07.992971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.453802+0000","last_change":"2024-09-11T23:19:57.612584+0000","last_active":"2024-09-11T23:21:50.453802+0000","last_peered":"2024-09-11T23:21:50.453802+0000","last_clean":"2024-09-11T23:21:50.453802+0000","last_became_active":"2024-09-11T23:19:57.612211+0000","last_became_peered":"2024-09-11T23:19:57.612211+0000","last_unstale":"2024-09-11T23:21:50.453802+0000","last_undegraded":"2024-09-11T23:21:50.453802+0000","last_fullsized":"2024-09-11T23:21:50.453802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:04.026233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300938+0000","last_change":"2024-09-11T23:21:02.359249+0000","last_active":"2024-09-11T23:21:50.300938+0000","last_peered":"2024-09-11T23:21:50.300938+0000","last_clean":"2024-09-11T23:21:50.300938+0000","last_became_active":"2024-09-11T23:21:02.358906+0000","last_became_peered":"2024-09-11T23:21:02.358906+0000","last_unstale":"2024-09-11T23:21:50.300938+0000","last_undegraded":"2024-09-11T23:21:50.300938+0000","last_fullsized":"2024-09-11T23:21:50.300938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:05.973581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455482+0000","last_change":"2024-09-11T23:19:57.618283+0000","last_active":"2024-09-11T23:21:50.455482+0000","last_peered":"2024-09-11T23:21:50.455482+0000","last_clean":"2024-09-11T23:21:50.455482+0000","last_became_active":"2024-09-11T23:19:57.617942+0000","last_became_peered":"2024-09-11T23:19:57.617942+0000","last_unstale":"2024-09-11T23:21:50.455482+0000","last_undegraded":"2024-09-11T23:21:50.455482+0000","last_fullsized":"2024-09-11T23:21:50.455482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:30.832602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.209864+0000","last_change":"2024-09-11T23:19:59.740730+0000","last_active":"2024-09-11T23:21:50.209864+0000","last_peered":"2024-09-11T23:21:50.209864+0000","last_clean":"2024-09-11T23:21:50.209864+0000","last_became_active":"2024-09-11T23:19:59.740369+0000","last_became_peered":"2024-09-11T23:19:59.740369+0000","last_unstale":"2024-09-11T23:21:50.209864+0000","last_undegraded":"2024-09-11T23:21:50.209864+0000","last_fullsized":"2024-09-11T23:21:50.209864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:50.264644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":142,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:22:00.880529+0000","last_change":"2024-09-11T23:19:57.823253+0000","last_active":"2024-09-11T23:22:00.880529+0000","last_peered":"2024-09-11T23:22:00.880529+0000","last_clean":"2024-09-11T23:22:00.880529+0000","last_became_active":"2024-09-11T23:19:57.822895+0000","last_became_peered":"2024-09-11T23:19:57.822895+0000","last_unstale":"2024-09-11T23:22:00.880529+0000","last_undegraded":"2024-09-11T23:22:00.880529+0000","last_fullsized":"2024-09-11T23:22:00.880529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:22.998371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308142+0000","last_change":"2024-09-11T23:19:59.742248+0000","last_active":"2024-09-11T23:21:50.308142+0000","last_peered":"2024-09-11T23:21:50.308142+0000","last_clean":"2024-09-11T23:21:50.308142+0000","last_became_active":"2024-09-11T23:19:59.741810+0000","last_became_peered":"2024-09-11T23:19:59.741810+0000","last_unstale":"2024-09-11T23:21:50.308142+0000","last_undegraded":"2024-09-11T23:21:50.308142+0000","last_fullsized":"2024-09-11T23:21:50.308142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:19:52.720286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455554+0000","last_change":"2024-09-11T23:19:57.616244+0000","last_active":"2024-09-11T23:21:50.455554+0000","last_peered":"2024-09-11T23:21:50.455554+0000","last_clean":"2024-09-11T23:21:50.455554+0000","last_became_active":"2024-09-11T23:19:57.615868+0000","last_became_peered":"2024-09-11T23:19:57.615868+0000","last_unstale":"2024-09-11T23:21:50.455554+0000","last_undegraded":"2024-09-11T23:21:50.455554+0000","last_fullsized":"2024-09-11T23:21:50.455554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:42.510896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311069+0000","last_change":"2024-09-11T23:21:02.434265+0000","last_active":"2024-09-11T23:21:50.311069+0000","last_peered":"2024-09-11T23:21:50.311069+0000","last_clean":"2024-09-11T23:21:50.311069+0000","last_became_active":"2024-09-11T23:21:02.433840+0000","last_became_peered":"2024-09-11T23:21:02.433840+0000","last_unstale":"2024-09-11T23:21:50.311069+0000","last_undegraded":"2024-09-11T23:21:50.311069+0000","last_fullsized":"2024-09-11T23:21:50.311069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:04.619695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297618+0000","last_change":"2024-09-11T23:19:57.824334+0000","last_active":"2024-09-11T23:21:50.297618+0000","last_peered":"2024-09-11T23:21:50.297618+0000","last_clean":"2024-09-11T23:21:50.297618+0000","last_became_active":"2024-09-11T23:19:57.823972+0000","last_became_peered":"2024-09-11T23:19:57.823972+0000","last_unstale":"2024-09-11T23:21:50.297618+0000","last_undegraded":"2024-09-11T23:21:50.297618+0000","last_fullsized":"2024-09-11T23:21:50.297618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:39.183281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311164+0000","last_change":"2024-09-11T23:19:59.728064+0000","last_active":"2024-09-11T23:21:50.311164+0000","last_peered":"2024-09-11T23:21:50.311164+0000","last_clean":"2024-09-11T23:21:50.311164+0000","last_became_active":"2024-09-11T23:19:59.727732+0000","last_became_peered":"2024-09-11T23:19:59.727732+0000","last_unstale":"2024-09-11T23:21:50.311164+0000","last_undegraded":"2024-09-11T23:21:50.311164+0000","last_fullsized":"2024-09-11T23:21:50.311164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:15.383974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299855+0000","last_change":"2024-09-11T23:19:57.824362+0000","last_active":"2024-09-11T23:21:50.299855+0000","last_peered":"2024-09-11T23:21:50.299855+0000","last_clean":"2024-09-11T23:21:50.299855+0000","last_became_active":"2024-09-11T23:19:57.823753+0000","last_became_peered":"2024-09-11T23:19:57.823753+0000","last_unstale":"2024-09-11T23:21:50.299855+0000","last_undegraded":"2024-09-11T23:21:50.299855+0000","last_fullsized":"2024-09-11T23:21:50.299855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:56.357268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210323+0000","last_change":"2024-09-11T23:19:59.730623+0000","last_active":"2024-09-11T23:21:50.210323+0000","last_peered":"2024-09-11T23:21:50.210323+0000","last_clean":"2024-09-11T23:21:50.210323+0000","last_became_active":"2024-09-11T23:19:59.730307+0000","last_became_peered":"2024-09-11T23:19:59.730307+0000","last_unstale":"2024-09-11T23:21:50.210323+0000","last_undegraded":"2024-09-11T23:21:50.210323+0000","last_fullsized":"2024-09-11T23:21:50.210323+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:52.874884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'100","reported_seq":213,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:22:01.200407+0000","last_change":"2024-09-11T23:19:57.611730+0000","last_active":"2024-09-11T23:22:01.200407+0000","last_peered":"2024-09-11T23:22:01.200407+0000","last_clean":"2024-09-11T23:22:01.200407+0000","last_became_active":"2024-09-11T23:19:57.611394+0000","last_became_peered":"2024-09-11T23:19:57.611394+0000","last_unstale":"2024-09-11T23:22:01.200407+0000","last_undegraded":"2024-09-11T23:22:01.200407+0000","last_fullsized":"2024-09-11T23:22:01.200407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:15.518573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":183,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.302035+0000","last_change":"2024-09-11T23:19:59.741396+0000","last_active":"2024-09-11T23:21:50.302035+0000","last_peered":"2024-09-11T23:21:50.302035+0000","last_clean":"2024-09-11T23:21:50.302035+0000","last_became_active":"2024-09-11T23:19:59.741102+0000","last_became_peered":"2024-09-11T23:19:59.741102+0000","last_unstale":"2024-09-11T23:21:50.302035+0000","last_undegraded":"2024-09-11T23:21:50.302035+0000","last_fullsized":"2024-09-11T23:21:50.302035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:08.759512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.453918+0000","last_change":"2024-09-11T23:19:57.824812+0000","last_active":"2024-09-11T23:21:50.453918+0000","last_peered":"2024-09-11T23:21:50.453918+0000","last_clean":"2024-09-11T23:21:50.453918+0000","last_became_active":"2024-09-11T23:19:57.824454+0000","last_became_peered":"2024-09-11T23:19:57.824454+0000","last_unstale":"2024-09-11T23:21:50.453918+0000","last_undegraded":"2024-09-11T23:21:50.453918+0000","last_fullsized":"2024-09-11T23:21:50.453918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:25.275573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455034+0000","last_change":"2024-09-11T23:19:59.735857+0000","last_active":"2024-09-11T23:21:50.455034+0000","last_peered":"2024-09-11T23:21:50.455034+0000","last_clean":"2024-09-11T23:21:50.455034+0000","last_became_active":"2024-09-11T23:19:59.735541+0000","last_became_peered":"2024-09-11T23:19:59.735541+0000","last_unstale":"2024-09-11T23:21:50.455034+0000","last_undegraded":"2024-09-11T23:21:50.455034+0000","last_fullsized":"2024-09-11T23:21:50.455034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:21.600490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211231+0000","last_change":"2024-09-11T23:19:57.823468+0000","last_active":"2024-09-11T23:21:50.211231+0000","last_peered":"2024-09-11T23:21:50.211231+0000","last_clean":"2024-09-11T23:21:50.211231+0000","last_became_active":"2024-09-11T23:19:57.823154+0000","last_became_peered":"2024-09-11T23:19:57.823154+0000","last_unstale":"2024-09-11T23:21:50.211231+0000","last_undegraded":"2024-09-11T23:21:50.211231+0000","last_fullsized":"2024-09-11T23:21:50.211231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:28.770627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299509+0000","last_change":"2024-09-11T23:19:59.741263+0000","last_active":"2024-09-11T23:21:50.299509+0000","last_peered":"2024-09-11T23:21:50.299509+0000","last_clean":"2024-09-11T23:21:50.299509+0000","last_became_active":"2024-09-11T23:19:59.740938+0000","last_became_peered":"2024-09-11T23:19:59.740938+0000","last_unstale":"2024-09-11T23:21:50.299509+0000","last_undegraded":"2024-09-11T23:21:50.299509+0000","last_fullsized":"2024-09-11T23:21:50.299509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:54.768652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.301723+0000","last_change":"2024-09-11T23:19:57.612517+0000","last_active":"2024-09-11T23:21:50.301723+0000","last_peered":"2024-09-11T23:21:50.301723+0000","last_clean":"2024-09-11T23:21:50.301723+0000","last_became_active":"2024-09-11T23:19:57.612188+0000","last_became_peered":"2024-09-11T23:19:57.612188+0000","last_unstale":"2024-09-11T23:21:50.301723+0000","last_undegraded":"2024-09-11T23:21:50.301723+0000","last_fullsized":"2024-09-11T23:21:50.301723+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:52.896413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598315+0000","last_change":"2024-09-11T23:19:59.732520+0000","last_active":"2024-09-11T23:21:50.598315+0000","last_peered":"2024-09-11T23:21:50.598315+0000","last_clean":"2024-09-11T23:21:50.598315+0000","last_became_active":"2024-09-11T23:19:59.732173+0000","last_became_peered":"2024-09-11T23:19:59.732173+0000","last_unstale":"2024-09-11T23:21:50.598315+0000","last_undegraded":"2024-09-11T23:21:50.598315+0000","last_fullsized":"2024-09-11T23:21:50.598315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:04.626312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598177+0000","last_change":"2024-09-11T23:21:02.365533+0000","last_active":"2024-09-11T23:21:50.598177+0000","last_peered":"2024-09-11T23:21:50.598177+0000","last_clean":"2024-09-11T23:21:50.598177+0000","last_became_active":"2024-09-11T23:21:02.359345+0000","last_became_peered":"2024-09-11T23:21:02.359345+0000","last_unstale":"2024-09-11T23:21:50.598177+0000","last_undegraded":"2024-09-11T23:21:50.598177+0000","last_fullsized":"2024-09-11T23:21:50.598177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:27.195666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.301266+0000","last_change":"2024-09-11T23:19:59.734667+0000","last_active":"2024-09-11T23:21:50.301266+0000","last_peered":"2024-09-11T23:21:50.301266+0000","last_clean":"2024-09-11T23:21:50.301266+0000","last_became_active":"2024-09-11T23:19:59.733754+0000","last_became_peered":"2024-09-11T23:19:59.733754+0000","last_unstale":"2024-09-11T23:21:50.301266+0000","last_undegraded":"2024-09-11T23:21:50.301266+0000","last_fullsized":"2024-09-11T23:21:50.301266+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:57.392272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299623+0000","last_change":"2024-09-11T23:19:57.614579+0000","last_active":"2024-09-11T23:21:50.299623+0000","last_peered":"2024-09-11T23:21:50.299623+0000","last_clean":"2024-09-11T23:21:50.299623+0000","last_became_active":"2024-09-11T23:19:57.614227+0000","last_became_peered":"2024-09-11T23:19:57.614227+0000","last_unstale":"2024-09-11T23:21:50.299623+0000","last_undegraded":"2024-09-11T23:21:50.299623+0000","last_fullsized":"2024-09-11T23:21:50.299623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:47.952920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455414+0000","last_change":"2024-09-11T23:19:59.744219+0000","last_active":"2024-09-11T23:21:50.455414+0000","last_peered":"2024-09-11T23:21:50.455414+0000","last_clean":"2024-09-11T23:21:50.455414+0000","last_became_active":"2024-09-11T23:19:59.743914+0000","last_became_peered":"2024-09-11T23:19:59.743914+0000","last_unstale":"2024-09-11T23:21:50.455414+0000","last_undegraded":"2024-09-11T23:21:50.455414+0000","last_fullsized":"2024-09-11T23:21:50.455414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:52.149588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308372+0000","last_change":"2024-09-11T23:21:02.361647+0000","last_active":"2024-09-11T23:21:50.308372+0000","last_peered":"2024-09-11T23:21:50.308372+0000","last_clean":"2024-09-11T23:21:50.308372+0000","last_became_active":"2024-09-11T23:21:02.361319+0000","last_became_peered":"2024-09-11T23:21:02.361319+0000","last_unstale":"2024-09-11T23:21:50.308372+0000","last_undegraded":"2024-09-11T23:21:50.308372+0000","last_fullsized":"2024-09-11T23:21:50.308372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:45.125525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299796+0000","last_change":"2024-09-11T23:19:57.610389+0000","last_active":"2024-09-11T23:21:50.299796+0000","last_peered":"2024-09-11T23:21:50.299796+0000","last_clean":"2024-09-11T23:21:50.299796+0000","last_became_active":"2024-09-11T23:19:57.609911+0000","last_became_peered":"2024-09-11T23:19:57.609911+0000","last_unstale":"2024-09-11T23:21:50.299796+0000","last_undegraded":"2024-09-11T23:21:50.299796+0000","last_fullsized":"2024-09-11T23:21:50.299796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:23.949929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211496+0000","last_change":"2024-09-11T23:19:59.736867+0000","last_active":"2024-09-11T23:21:50.211496+0000","last_peered":"2024-09-11T23:21:50.211496+0000","last_clean":"2024-09-11T23:21:50.211496+0000","last_became_active":"2024-09-11T23:19:59.736341+0000","last_became_peered":"2024-09-11T23:19:59.736341+0000","last_unstale":"2024-09-11T23:21:50.211496+0000","last_undegraded":"2024-09-11T23:21:50.211496+0000","last_fullsized":"2024-09-11T23:21:50.211496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:57:51.015235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598177+0000","last_change":"2024-09-11T23:21:02.368959+0000","last_active":"2024-09-11T23:21:50.598177+0000","last_peered":"2024-09-11T23:21:50.598177+0000","last_clean":"2024-09-11T23:21:50.598177+0000","last_became_active":"2024-09-11T23:21:02.363263+0000","last_became_peered":"2024-09-11T23:21:02.363263+0000","last_unstale":"2024-09-11T23:21:50.598177+0000","last_undegraded":"2024-09-11T23:21:50.598177+0000","last_fullsized":"2024-09-11T23:21:50.598177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:29.827330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455355+0000","last_change":"2024-09-11T23:19:59.742049+0000","last_active":"2024-09-11T23:21:50.455355+0000","last_peered":"2024-09-11T23:21:50.455355+0000","last_clean":"2024-09-11T23:21:50.455355+0000","last_became_active":"2024-09-11T23:19:59.741763+0000","last_became_peered":"2024-09-11T23:19:59.741763+0000","last_unstale":"2024-09-11T23:21:50.455355+0000","last_undegraded":"2024-09-11T23:21:50.455355+0000","last_fullsized":"2024-09-11T23:21:50.455355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:48.698896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.302151+0000","last_change":"2024-09-11T23:19:57.822949+0000","last_active":"2024-09-11T23:21:50.302151+0000","last_peered":"2024-09-11T23:21:50.302151+0000","last_clean":"2024-09-11T23:21:50.302151+0000","last_became_active":"2024-09-11T23:19:57.822564+0000","last_became_peered":"2024-09-11T23:19:57.822564+0000","last_unstale":"2024-09-11T23:21:50.302151+0000","last_undegraded":"2024-09-11T23:21:50.302151+0000","last_fullsized":"2024-09-11T23:21:50.302151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:29.532109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297740+0000","last_change":"2024-09-11T23:21:02.363639+0000","last_active":"2024-09-11T23:21:50.297740+0000","last_peered":"2024-09-11T23:21:50.297740+0000","last_clean":"2024-09-11T23:21:50.297740+0000","last_became_active":"2024-09-11T23:21:02.363247+0000","last_became_peered":"2024-09-11T23:21:02.363247+0000","last_unstale":"2024-09-11T23:21:50.297740+0000","last_undegraded":"2024-09-11T23:21:50.297740+0000","last_fullsized":"2024-09-11T23:21:50.297740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:18:58.906939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297895+0000","last_change":"2024-09-11T23:19:57.613323+0000","last_active":"2024-09-11T23:21:50.297895+0000","last_peered":"2024-09-11T23:21:50.297895+0000","last_clean":"2024-09-11T23:21:50.297895+0000","last_became_active":"2024-09-11T23:19:57.612985+0000","last_became_peered":"2024-09-11T23:19:57.612985+0000","last_unstale":"2024-09-11T23:21:50.297895+0000","last_undegraded":"2024-09-11T23:21:50.297895+0000","last_fullsized":"2024-09-11T23:21:50.297895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:13:09.179547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":797142,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1038,"num_read_kb":732,"num_write":418,"num_write_kb":2357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":2391426,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108948,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514533,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":450800,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743465,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1126400,"data_stored":1442204,"data_compressed":60217,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906992,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":844252,"data_compressed":27988,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168629,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":844172,"data_compressed":27974,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":1044601,"data_compressed":45015,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561087,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":450425,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1044480,"data_stored":1370804,"data_compressed":60217,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":452575,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1011712,"data_stored":1172184,"data_compressed":42814,"data_compressed_allocated":57344,"data_compressed_original":724992,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":521939,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":520196,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":946176,"data_stored":451696,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":330101,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":401501,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:22:05.458 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T23:22:05.459 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T23:22:05.717 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:06.013 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:05 smithi043 ceph-mon[30950]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-09-11T23:22:06.013 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:05 smithi043 ceph-mon[30950]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:22:06.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:05 smithi090 ceph-mon[35130]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-09-11T23:22:06.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:05 smithi090 ceph-mon[35130]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:22:06.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:05 smithi097 ceph-mon[35562]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-09-11T23:22:06.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:05 smithi097 ceph-mon[35562]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:22:06.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.202+0000 7f92b52f3640 1 -- 172.21.15.43:0/1000720805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b0102d40 msgr2=0x7f92b0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:06.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.202+0000 7f92b52f3640 1 --2- 172.21.15.43:0/1000720805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b0102d40 0x7f92b0103120 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009ca0 tx=0x7f92a002a950 comp rx=0 tx=0).stop 2024-09-11T23:22:06.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.202+0000 7f92b52f3640 1 -- 172.21.15.43:0/1000720805 shutdown_connections 2024-09-11T23:22:06.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.202+0000 7f92b52f3640 1 --2- 172.21.15.43:0/1000720805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b010c170 0x7f92b010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:06.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.202+0000 7f92b52f3640 1 --2- 172.21.15.43:0/1000720805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92b0103660 0x7f92b010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:06.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.202+0000 7f92b52f3640 1 --2- 172.21.15.43:0/1000720805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b0102d40 0x7f92b0103120 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:06.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.202+0000 7f92b52f3640 1 -- 172.21.15.43:0/1000720805 >> 172.21.15.43:0/1000720805 conn(0x7f92b00fc7a0 msgr2=0x7f92b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:06.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.202+0000 7f92b52f3640 1 -- 172.21.15.43:0/1000720805 shutdown_connections 2024-09-11T23:22:06.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.202+0000 7f92b52f3640 1 -- 172.21.15.43:0/1000720805 wait complete. 2024-09-11T23:22:06.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.203+0000 7f92b52f3640 1 Processor -- start 2024-09-11T23:22:06.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.203+0000 7f92b52f3640 1 -- start start 2024-09-11T23:22:06.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.203+0000 7f92b52f3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b0102d40 0x7f92b0198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:06.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.203+0000 7f92b52f3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b0103660 0x7f92b0198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:06.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.203+0000 7f92b52f3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92b010c170 0x7f92b019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:06.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.203+0000 7f92b52f3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f92b0111f70 con 0x7f92b0102d40 2024-09-11T23:22:06.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.203+0000 7f92b52f3640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f92b0111df0 con 0x7f92b010c170 2024-09-11T23:22:06.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.203+0000 7f92b52f3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f92b01120f0 con 0x7f92b0103660 2024-09-11T23:22:06.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.204+0000 7f92aeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b0102d40 0x7f92b0198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:06.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.204+0000 7f92aeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b0102d40 0x7f92b0198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49972/0 (socket says 172.21.15.43:49972) 2024-09-11T23:22:06.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.204+0000 7f92aeffd640 1 -- 172.21.15.43:0/3149131947 learned_addr learned my addr 172.21.15.43:0/3149131947 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:06.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.204+0000 7f92af7fe640 1 --2- 172.21.15.43:0/3149131947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92b010c170 0x7f92b019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:06.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.204+0000 7f92ae7fc640 1 --2- 172.21.15.43:0/3149131947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b0103660 0x7f92b0198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:06.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.204+0000 7f92aeffd640 1 -- 172.21.15.43:0/3149131947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b0103660 msgr2=0x7f92b0198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:06.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.204+0000 7f92aeffd640 1 --2- 172.21.15.43:0/3149131947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b0103660 0x7f92b0198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:06.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.204+0000 7f92aeffd640 1 -- 172.21.15.43:0/3149131947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92b010c170 msgr2=0x7f92b019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:06.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.204+0000 7f92aeffd640 1 --2- 172.21.15.43:0/3149131947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92b010c170 0x7f92b019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:06.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.204+0000 7f92aeffd640 1 -- 172.21.15.43:0/3149131947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b019d3d0 con 0x7f92b0102d40 2024-09-11T23:22:06.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.204+0000 7f92af7fe640 1 --2- 172.21.15.43:0/3149131947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92b010c170 0x7f92b019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:06.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.204+0000 7f92aeffd640 1 --2- 172.21.15.43:0/3149131947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b0102d40 0x7f92b0198420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f92a0005cc0 tx=0x7f92a002f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:06.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.204+0000 7f9297fff640 1 -- 172.21.15.43:0/3149131947 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a0034040 con 0x7f92b0102d40 2024-09-11T23:22:06.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.204+0000 7f9297fff640 1 -- 172.21.15.43:0/3149131947 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92a0030920 con 0x7f92b0102d40 2024-09-11T23:22:06.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.204+0000 7f92b52f3640 1 -- 172.21.15.43:0/3149131947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b019d660 con 0x7f92b0102d40 2024-09-11T23:22:06.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.204+0000 7f9297fff640 1 -- 172.21.15.43:0/3149131947 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a0004830 con 0x7f92b0102d40 2024-09-11T23:22:06.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.204+0000 7f92b52f3640 1 -- 172.21.15.43:0/3149131947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b01a4fa0 con 0x7f92b0102d40 2024-09-11T23:22:06.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.205+0000 7f92b52f3640 1 -- 172.21.15.43:0/3149131947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b0103120 con 0x7f92b0102d40 2024-09-11T23:22:06.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.206+0000 7f9297fff640 1 -- 172.21.15.43:0/3149131947 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f92a0037020 con 0x7f92b0102d40 2024-09-11T23:22:06.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.207+0000 7f9297fff640 1 --2- 172.21.15.43:0/3149131947 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f92980a2e90 0x7f92980a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:06.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.207+0000 7f92ae7fc640 1 --2- 172.21.15.43:0/3149131947 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f92980a2e90 0x7f92980a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:06.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.207+0000 7f9297fff640 1 -- 172.21.15.43:0/3149131947 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f92a00f06e0 con 0x7f92b0102d40 2024-09-11T23:22:06.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.208+0000 7f92ae7fc640 1 --2- 172.21.15.43:0/3149131947 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f92980a2e90 0x7f92980a5350 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f92a40058c0 tx=0x7f92a4005850 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:06.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.210+0000 7f9297fff640 1 -- 172.21.15.43:0/3149131947 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f92a00bd1c0 con 0x7f92b0102d40 2024-09-11T23:22:06.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.385+0000 7f92b52f3640 1 -- 172.21.15.43:0/3149131947 --> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f92b0107240 con 0x7f92980a2e90 2024-09-11T23:22:06.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.391+0000 7f9297fff640 1 -- 172.21.15.43:0/3149131947 <== mgr.15126 v2:172.21.15.43:6800/2510386404 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258040 (secure 0 0 0) 0x7f92b0107240 con 0x7f92980a2e90 2024-09-11T23:22:06.393 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:22:06.394 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-11T23:22:06.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.395+0000 7f92b52f3640 1 -- 172.21.15.43:0/3149131947 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f92980a2e90 msgr2=0x7f92980a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:06.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.395+0000 7f92b52f3640 1 --2- 172.21.15.43:0/3149131947 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f92980a2e90 0x7f92980a5350 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f92a40058c0 tx=0x7f92a4005850 comp rx=0 tx=0).stop 2024-09-11T23:22:06.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.395+0000 7f92b52f3640 1 -- 172.21.15.43:0/3149131947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b0102d40 msgr2=0x7f92b0198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:06.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.395+0000 7f92b52f3640 1 --2- 172.21.15.43:0/3149131947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b0102d40 0x7f92b0198420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f92a0005cc0 tx=0x7f92a002f630 comp rx=0 tx=0).stop 2024-09-11T23:22:06.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.395+0000 7f92b52f3640 1 -- 172.21.15.43:0/3149131947 shutdown_connections 2024-09-11T23:22:06.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.395+0000 7f92b52f3640 1 --2- 172.21.15.43:0/3149131947 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f92980a2e90 0x7f92980a5350 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:06.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.395+0000 7f92b52f3640 1 --2- 172.21.15.43:0/3149131947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92b010c170 0x7f92b019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:06.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.395+0000 7f92b52f3640 1 --2- 172.21.15.43:0/3149131947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b0103660 0x7f92b0198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:06.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.395+0000 7f92b52f3640 1 --2- 172.21.15.43:0/3149131947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b0102d40 0x7f92b0198420 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:06.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.395+0000 7f92b52f3640 1 -- 172.21.15.43:0/3149131947 >> 172.21.15.43:0/3149131947 conn(0x7f92b00fc7a0 msgr2=0x7f92b00fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:06.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.395+0000 7f92b52f3640 1 -- 172.21.15.43:0/3149131947 shutdown_connections 2024-09-11T23:22:06.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:06.395+0000 7f92b52f3640 1 -- 172.21.15.43:0/3149131947 wait complete. 2024-09-11T23:22:07.153 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-09-11T23:22:06.226032+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1387744,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1275,"num_read_kb":1062,"num_write":621,"num_write_kb":5297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":958,"ondisk_log_size":958,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369160,"kb_used_data":11556,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572664,"statfs":{"total":1151940427776,"available":1151562407936,"internally_reserved":0,"allocated":11833344,"data_stored":9565848,"data_compressed":316419,"data_compressed_allocated":417792,"data_compressed_original":4165632,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":199938,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":408,"num_read_kb":468,"num_write":147,"num_write_kb":920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002637"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598561+0000","last_change":"2024-09-11T23:19:59.734378+0000","last_active":"2024-09-11T23:21:50.598561+0000","last_peered":"2024-09-11T23:21:50.598561+0000","last_clean":"2024-09-11T23:21:50.598561+0000","last_became_active":"2024-09-11T23:19:59.734067+0000","last_became_peered":"2024-09-11T23:19:59.734067+0000","last_unstale":"2024-09-11T23:21:50.598561+0000","last_undegraded":"2024-09-11T23:21:50.598561+0000","last_fullsized":"2024-09-11T23:21:50.598561+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:28:18.384453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.454939+0000","last_change":"2024-09-11T23:19:59.733892+0000","last_active":"2024-09-11T23:21:50.454939+0000","last_peered":"2024-09-11T23:21:50.454939+0000","last_clean":"2024-09-11T23:21:50.454939+0000","last_became_active":"2024-09-11T23:19:59.733540+0000","last_became_peered":"2024-09-11T23:19:59.733540+0000","last_unstale":"2024-09-11T23:21:50.454939+0000","last_undegraded":"2024-09-11T23:21:50.454939+0000","last_fullsized":"2024-09-11T23:21:50.454939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:42.186458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455001+0000","last_change":"2024-09-11T23:19:59.739443+0000","last_active":"2024-09-11T23:21:50.455001+0000","last_peered":"2024-09-11T23:21:50.455001+0000","last_clean":"2024-09-11T23:21:50.455001+0000","last_became_active":"2024-09-11T23:19:59.739025+0000","last_became_peered":"2024-09-11T23:19:59.739025+0000","last_unstale":"2024-09-11T23:21:50.455001+0000","last_undegraded":"2024-09-11T23:21:50.455001+0000","last_fullsized":"2024-09-11T23:21:50.455001+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:40.819321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297376+0000","last_change":"2024-09-11T23:21:02.363511+0000","last_active":"2024-09-11T23:21:50.297376+0000","last_peered":"2024-09-11T23:21:50.297376+0000","last_clean":"2024-09-11T23:21:50.297376+0000","last_became_active":"2024-09-11T23:21:02.363156+0000","last_became_peered":"2024-09-11T23:21:02.363156+0000","last_unstale":"2024-09-11T23:21:50.297376+0000","last_undegraded":"2024-09-11T23:21:50.297376+0000","last_fullsized":"2024-09-11T23:21:50.297376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:03:17.059509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299585+0000","last_change":"2024-09-11T23:19:59.738662+0000","last_active":"2024-09-11T23:21:50.299585+0000","last_peered":"2024-09-11T23:21:50.299585+0000","last_clean":"2024-09-11T23:21:50.299585+0000","last_became_active":"2024-09-11T23:19:59.738320+0000","last_became_peered":"2024-09-11T23:19:59.738320+0000","last_unstale":"2024-09-11T23:21:50.299585+0000","last_undegraded":"2024-09-11T23:21:50.299585+0000","last_fullsized":"2024-09-11T23:21:50.299585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:15.645179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.310947+0000","last_change":"2024-09-11T23:19:59.731442+0000","last_active":"2024-09-11T23:21:50.310947+0000","last_peered":"2024-09-11T23:21:50.310947+0000","last_clean":"2024-09-11T23:21:50.310947+0000","last_became_active":"2024-09-11T23:19:59.731079+0000","last_became_peered":"2024-09-11T23:19:59.731079+0000","last_unstale":"2024-09-11T23:21:50.310947+0000","last_undegraded":"2024-09-11T23:21:50.310947+0000","last_fullsized":"2024-09-11T23:21:50.310947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:32.764991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455210+0000","last_change":"2024-09-11T23:21:02.359992+0000","last_active":"2024-09-11T23:21:50.455210+0000","last_peered":"2024-09-11T23:21:50.455210+0000","last_clean":"2024-09-11T23:21:50.455210+0000","last_became_active":"2024-09-11T23:21:02.359564+0000","last_became_peered":"2024-09-11T23:21:02.359564+0000","last_unstale":"2024-09-11T23:21:50.455210+0000","last_undegraded":"2024-09-11T23:21:50.455210+0000","last_fullsized":"2024-09-11T23:21:50.455210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:53.633219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300215+0000","last_change":"2024-09-11T23:19:59.740733+0000","last_active":"2024-09-11T23:21:50.300215+0000","last_peered":"2024-09-11T23:21:50.300215+0000","last_clean":"2024-09-11T23:21:50.300215+0000","last_became_active":"2024-09-11T23:19:59.740438+0000","last_became_peered":"2024-09-11T23:19:59.740438+0000","last_unstale":"2024-09-11T23:21:50.300215+0000","last_undegraded":"2024-09-11T23:21:50.300215+0000","last_fullsized":"2024-09-11T23:21:50.300215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:27.063305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.212024+0000","last_change":"2024-09-11T23:19:59.742439+0000","last_active":"2024-09-11T23:21:50.212024+0000","last_peered":"2024-09-11T23:21:50.212024+0000","last_clean":"2024-09-11T23:21:50.212024+0000","last_became_active":"2024-09-11T23:19:59.741983+0000","last_became_peered":"2024-09-11T23:19:59.741983+0000","last_unstale":"2024-09-11T23:21:50.212024+0000","last_undegraded":"2024-09-11T23:21:50.212024+0000","last_fullsized":"2024-09-11T23:21:50.212024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:39.477167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311317+0000","last_change":"2024-09-11T23:19:59.732544+0000","last_active":"2024-09-11T23:21:50.311317+0000","last_peered":"2024-09-11T23:21:50.311317+0000","last_clean":"2024-09-11T23:21:50.311317+0000","last_became_active":"2024-09-11T23:19:59.732205+0000","last_became_peered":"2024-09-11T23:19:59.732205+0000","last_unstale":"2024-09-11T23:21:50.311317+0000","last_undegraded":"2024-09-11T23:21:50.311317+0000","last_fullsized":"2024-09-11T23:21:50.311317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:22.494344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455518+0000","last_change":"2024-09-11T23:19:59.741857+0000","last_active":"2024-09-11T23:21:50.455518+0000","last_peered":"2024-09-11T23:21:50.455518+0000","last_clean":"2024-09-11T23:21:50.455518+0000","last_became_active":"2024-09-11T23:19:59.741484+0000","last_became_peered":"2024-09-11T23:19:59.741484+0000","last_unstale":"2024-09-11T23:21:50.455518+0000","last_undegraded":"2024-09-11T23:21:50.455518+0000","last_fullsized":"2024-09-11T23:21:50.455518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:52.998258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.453834+0000","last_change":"2024-09-11T23:19:59.736042+0000","last_active":"2024-09-11T23:21:50.453834+0000","last_peered":"2024-09-11T23:21:50.453834+0000","last_clean":"2024-09-11T23:21:50.453834+0000","last_became_active":"2024-09-11T23:19:59.735731+0000","last_became_peered":"2024-09-11T23:19:59.735731+0000","last_unstale":"2024-09-11T23:21:50.453834+0000","last_undegraded":"2024-09-11T23:21:50.453834+0000","last_fullsized":"2024-09-11T23:21:50.453834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:19.085804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210129+0000","last_change":"2024-09-11T23:19:59.742049+0000","last_active":"2024-09-11T23:21:50.210129+0000","last_peered":"2024-09-11T23:21:50.210129+0000","last_clean":"2024-09-11T23:21:50.210129+0000","last_became_active":"2024-09-11T23:19:59.740974+0000","last_became_peered":"2024-09-11T23:19:59.740974+0000","last_unstale":"2024-09-11T23:21:50.210129+0000","last_undegraded":"2024-09-11T23:21:50.210129+0000","last_fullsized":"2024-09-11T23:21:50.210129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:10.443113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.454569+0000","last_change":"2024-09-11T23:19:59.734270+0000","last_active":"2024-09-11T23:21:50.454569+0000","last_peered":"2024-09-11T23:21:50.454569+0000","last_clean":"2024-09-11T23:21:50.454569+0000","last_became_active":"2024-09-11T23:19:59.733967+0000","last_became_peered":"2024-09-11T23:19:59.733967+0000","last_unstale":"2024-09-11T23:21:50.454569+0000","last_undegraded":"2024-09-11T23:21:50.454569+0000","last_fullsized":"2024-09-11T23:21:50.454569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:02.185757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.454215+0000","last_change":"2024-09-11T23:19:59.734152+0000","last_active":"2024-09-11T23:21:50.454215+0000","last_peered":"2024-09-11T23:21:50.454215+0000","last_clean":"2024-09-11T23:21:50.454215+0000","last_became_active":"2024-09-11T23:19:59.733163+0000","last_became_peered":"2024-09-11T23:19:59.733163+0000","last_unstale":"2024-09-11T23:21:50.454215+0000","last_undegraded":"2024-09-11T23:21:50.454215+0000","last_fullsized":"2024-09-11T23:21:50.454215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:22.370591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211142+0000","last_change":"2024-09-11T23:19:59.735517+0000","last_active":"2024-09-11T23:21:50.211142+0000","last_peered":"2024-09-11T23:21:50.211142+0000","last_clean":"2024-09-11T23:21:50.211142+0000","last_became_active":"2024-09-11T23:19:59.735182+0000","last_became_peered":"2024-09-11T23:19:59.735182+0000","last_unstale":"2024-09-11T23:21:50.211142+0000","last_undegraded":"2024-09-11T23:21:50.211142+0000","last_fullsized":"2024-09-11T23:21:50.211142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:25.989416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598585+0000","last_change":"2024-09-11T23:19:59.733579+0000","last_active":"2024-09-11T23:21:50.598585+0000","last_peered":"2024-09-11T23:21:50.598585+0000","last_clean":"2024-09-11T23:21:50.598585+0000","last_became_active":"2024-09-11T23:19:59.733245+0000","last_became_peered":"2024-09-11T23:19:59.733245+0000","last_unstale":"2024-09-11T23:21:50.598585+0000","last_undegraded":"2024-09-11T23:21:50.598585+0000","last_fullsized":"2024-09-11T23:21:50.598585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:41:22.361158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297955+0000","last_change":"2024-09-11T23:19:59.734354+0000","last_active":"2024-09-11T23:21:50.297955+0000","last_peered":"2024-09-11T23:21:50.297955+0000","last_clean":"2024-09-11T23:21:50.297955+0000","last_became_active":"2024-09-11T23:19:59.733988+0000","last_became_peered":"2024-09-11T23:19:59.733988+0000","last_unstale":"2024-09-11T23:21:50.297955+0000","last_undegraded":"2024-09-11T23:21:50.297955+0000","last_fullsized":"2024-09-11T23:21:50.297955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:41.401496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.212032+0000","last_change":"2024-09-11T23:19:59.730318+0000","last_active":"2024-09-11T23:21:50.212032+0000","last_peered":"2024-09-11T23:21:50.212032+0000","last_clean":"2024-09-11T23:21:50.212032+0000","last_became_active":"2024-09-11T23:19:59.730009+0000","last_became_peered":"2024-09-11T23:19:59.730009+0000","last_unstale":"2024-09-11T23:21:50.212032+0000","last_undegraded":"2024-09-11T23:21:50.212032+0000","last_fullsized":"2024-09-11T23:21:50.212032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:01:38.531967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598328+0000","last_change":"2024-09-11T23:19:59.737254+0000","last_active":"2024-09-11T23:21:50.598328+0000","last_peered":"2024-09-11T23:21:50.598328+0000","last_clean":"2024-09-11T23:21:50.598328+0000","last_became_active":"2024-09-11T23:19:59.736936+0000","last_became_peered":"2024-09-11T23:19:59.736936+0000","last_unstale":"2024-09-11T23:21:50.598328+0000","last_undegraded":"2024-09-11T23:21:50.598328+0000","last_fullsized":"2024-09-11T23:21:50.598328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:00.361468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210147+0000","last_change":"2024-09-11T23:19:59.741617+0000","last_active":"2024-09-11T23:21:50.210147+0000","last_peered":"2024-09-11T23:21:50.210147+0000","last_clean":"2024-09-11T23:21:50.210147+0000","last_became_active":"2024-09-11T23:19:59.740358+0000","last_became_peered":"2024-09-11T23:19:59.740358+0000","last_unstale":"2024-09-11T23:21:50.210147+0000","last_undegraded":"2024-09-11T23:21:50.210147+0000","last_fullsized":"2024-09-11T23:21:50.210147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:18.541105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211292+0000","last_change":"2024-09-11T23:19:59.736437+0000","last_active":"2024-09-11T23:21:50.211292+0000","last_peered":"2024-09-11T23:21:50.211292+0000","last_clean":"2024-09-11T23:21:50.211292+0000","last_became_active":"2024-09-11T23:19:59.736029+0000","last_became_peered":"2024-09-11T23:19:59.736029+0000","last_unstale":"2024-09-11T23:21:50.211292+0000","last_undegraded":"2024-09-11T23:21:50.211292+0000","last_fullsized":"2024-09-11T23:21:50.211292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:29.226939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300441+0000","last_change":"2024-09-11T23:19:59.736104+0000","last_active":"2024-09-11T23:21:50.300441+0000","last_peered":"2024-09-11T23:21:50.300441+0000","last_clean":"2024-09-11T23:21:50.300441+0000","last_became_active":"2024-09-11T23:19:59.735760+0000","last_became_peered":"2024-09-11T23:19:59.735760+0000","last_unstale":"2024-09-11T23:21:50.300441+0000","last_undegraded":"2024-09-11T23:21:50.300441+0000","last_fullsized":"2024-09-11T23:21:50.300441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:56.554313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308481+0000","last_change":"2024-09-11T23:19:59.735455+0000","last_active":"2024-09-11T23:21:50.308481+0000","last_peered":"2024-09-11T23:21:50.308481+0000","last_clean":"2024-09-11T23:21:50.308481+0000","last_became_active":"2024-09-11T23:19:59.735078+0000","last_became_peered":"2024-09-11T23:19:59.735078+0000","last_unstale":"2024-09-11T23:21:50.308481+0000","last_undegraded":"2024-09-11T23:21:50.308481+0000","last_fullsized":"2024-09-11T23:21:50.308481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:21:16.336006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.301909+0000","last_change":"2024-09-11T23:19:59.745997+0000","last_active":"2024-09-11T23:21:50.301909+0000","last_peered":"2024-09-11T23:21:50.301909+0000","last_clean":"2024-09-11T23:21:50.301909+0000","last_became_active":"2024-09-11T23:19:59.745715+0000","last_became_peered":"2024-09-11T23:19:59.745715+0000","last_unstale":"2024-09-11T23:21:50.301909+0000","last_undegraded":"2024-09-11T23:21:50.301909+0000","last_fullsized":"2024-09-11T23:21:50.301909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:34.776229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.212119+0000","last_change":"2024-09-11T23:19:59.734181+0000","last_active":"2024-09-11T23:21:50.212119+0000","last_peered":"2024-09-11T23:21:50.212119+0000","last_clean":"2024-09-11T23:21:50.212119+0000","last_became_active":"2024-09-11T23:19:59.733793+0000","last_became_peered":"2024-09-11T23:19:59.733793+0000","last_unstale":"2024-09-11T23:21:50.212119+0000","last_undegraded":"2024-09-11T23:21:50.212119+0000","last_fullsized":"2024-09-11T23:21:50.212119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:36.906308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598526+0000","last_change":"2024-09-11T23:19:59.738713+0000","last_active":"2024-09-11T23:21:50.598526+0000","last_peered":"2024-09-11T23:21:50.598526+0000","last_clean":"2024-09-11T23:21:50.598526+0000","last_became_active":"2024-09-11T23:19:59.737579+0000","last_became_peered":"2024-09-11T23:19:59.737579+0000","last_unstale":"2024-09-11T23:21:50.598526+0000","last_undegraded":"2024-09-11T23:21:50.598526+0000","last_fullsized":"2024-09-11T23:21:50.598526+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:47.135775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210205+0000","last_change":"2024-09-11T23:19:59.741381+0000","last_active":"2024-09-11T23:21:50.210205+0000","last_peered":"2024-09-11T23:21:50.210205+0000","last_clean":"2024-09-11T23:21:50.210205+0000","last_became_active":"2024-09-11T23:19:59.740914+0000","last_became_peered":"2024-09-11T23:19:59.740914+0000","last_unstale":"2024-09-11T23:21:50.210205+0000","last_undegraded":"2024-09-11T23:21:50.210205+0000","last_fullsized":"2024-09-11T23:21:50.210205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:24:04.365704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455133+0000","last_change":"2024-09-11T23:19:59.736840+0000","last_active":"2024-09-11T23:21:50.455133+0000","last_peered":"2024-09-11T23:21:50.455133+0000","last_clean":"2024-09-11T23:21:50.455133+0000","last_became_active":"2024-09-11T23:19:59.736554+0000","last_became_peered":"2024-09-11T23:19:59.736554+0000","last_unstale":"2024-09-11T23:21:50.455133+0000","last_undegraded":"2024-09-11T23:21:50.455133+0000","last_fullsized":"2024-09-11T23:21:50.455133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:48.508369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211738+0000","last_change":"2024-09-11T23:19:59.735719+0000","last_active":"2024-09-11T23:21:50.211738+0000","last_peered":"2024-09-11T23:21:50.211738+0000","last_clean":"2024-09-11T23:21:50.211738+0000","last_became_active":"2024-09-11T23:19:59.735397+0000","last_became_peered":"2024-09-11T23:19:59.735397+0000","last_unstale":"2024-09-11T23:21:50.211738+0000","last_undegraded":"2024-09-11T23:21:50.211738+0000","last_fullsized":"2024-09-11T23:21:50.211738+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:26:19.392730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211747+0000","last_change":"2024-09-11T23:19:59.737704+0000","last_active":"2024-09-11T23:21:50.211747+0000","last_peered":"2024-09-11T23:21:50.211747+0000","last_clean":"2024-09-11T23:21:50.211747+0000","last_became_active":"2024-09-11T23:19:59.737176+0000","last_became_peered":"2024-09-11T23:19:59.737176+0000","last_unstale":"2024-09-11T23:21:50.211747+0000","last_undegraded":"2024-09-11T23:21:50.211747+0000","last_fullsized":"2024-09-11T23:21:50.211747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:22.938171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598301+0000","last_change":"2024-09-11T23:19:59.735499+0000","last_active":"2024-09-11T23:21:50.598301+0000","last_peered":"2024-09-11T23:21:50.598301+0000","last_clean":"2024-09-11T23:21:50.598301+0000","last_became_active":"2024-09-11T23:19:59.735088+0000","last_became_peered":"2024-09-11T23:19:59.735088+0000","last_unstale":"2024-09-11T23:21:50.598301+0000","last_undegraded":"2024-09-11T23:21:50.598301+0000","last_fullsized":"2024-09-11T23:21:50.598301+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:56:12.633104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299549+0000","last_change":"2024-09-11T23:19:59.743972+0000","last_active":"2024-09-11T23:21:50.299549+0000","last_peered":"2024-09-11T23:21:50.299549+0000","last_clean":"2024-09-11T23:21:50.299549+0000","last_became_active":"2024-09-11T23:19:59.743675+0000","last_became_peered":"2024-09-11T23:19:59.743675+0000","last_unstale":"2024-09-11T23:21:50.299549+0000","last_undegraded":"2024-09-11T23:21:50.299549+0000","last_fullsized":"2024-09-11T23:21:50.299549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:46.645696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":144,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:59.879602+0000","last_change":"2024-09-11T23:19:57.615472+0000","last_active":"2024-09-11T23:21:59.879602+0000","last_peered":"2024-09-11T23:21:59.879602+0000","last_clean":"2024-09-11T23:21:59.879602+0000","last_became_active":"2024-09-11T23:19:57.615104+0000","last_became_peered":"2024-09-11T23:19:57.615104+0000","last_unstale":"2024-09-11T23:21:59.879602+0000","last_undegraded":"2024-09-11T23:21:59.879602+0000","last_fullsized":"2024-09-11T23:21:59.879602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:34:03.601363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308144+0000","last_change":"2024-09-11T23:19:59.741027+0000","last_active":"2024-09-11T23:21:50.308144+0000","last_peered":"2024-09-11T23:21:50.308144+0000","last_clean":"2024-09-11T23:21:50.308144+0000","last_became_active":"2024-09-11T23:19:59.740681+0000","last_became_peered":"2024-09-11T23:19:59.740681+0000","last_unstale":"2024-09-11T23:21:50.308144+0000","last_undegraded":"2024-09-11T23:21:50.308144+0000","last_fullsized":"2024-09-11T23:21:50.308144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:36.948147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311025+0000","last_change":"2024-09-11T23:19:57.612730+0000","last_active":"2024-09-11T23:21:50.311025+0000","last_peered":"2024-09-11T23:21:50.311025+0000","last_clean":"2024-09-11T23:21:50.311025+0000","last_became_active":"2024-09-11T23:19:57.612369+0000","last_became_peered":"2024-09-11T23:19:57.612369+0000","last_unstale":"2024-09-11T23:21:50.311025+0000","last_undegraded":"2024-09-11T23:21:50.311025+0000","last_fullsized":"2024-09-11T23:21:50.311025+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:08:17.823616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.209881+0000","last_change":"2024-09-11T23:19:57.611470+0000","last_active":"2024-09-11T23:21:50.209881+0000","last_peered":"2024-09-11T23:21:50.209881+0000","last_clean":"2024-09-11T23:21:50.209881+0000","last_became_active":"2024-09-11T23:19:57.611141+0000","last_became_peered":"2024-09-11T23:19:57.611141+0000","last_unstale":"2024-09-11T23:21:50.209881+0000","last_undegraded":"2024-09-11T23:21:50.209881+0000","last_fullsized":"2024-09-11T23:21:50.209881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:43.173955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300408+0000","last_change":"2024-09-11T23:19:59.732375+0000","last_active":"2024-09-11T23:21:50.300408+0000","last_peered":"2024-09-11T23:21:50.300408+0000","last_clean":"2024-09-11T23:21:50.300408+0000","last_became_active":"2024-09-11T23:19:59.731793+0000","last_became_peered":"2024-09-11T23:19:59.731793+0000","last_unstale":"2024-09-11T23:21:50.300408+0000","last_undegraded":"2024-09-11T23:21:50.300408+0000","last_fullsized":"2024-09-11T23:21:50.300408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:02:47.278646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308399+0000","last_change":"2024-09-11T23:19:57.614062+0000","last_active":"2024-09-11T23:21:50.308399+0000","last_peered":"2024-09-11T23:21:50.308399+0000","last_clean":"2024-09-11T23:21:50.308399+0000","last_became_active":"2024-09-11T23:19:57.613749+0000","last_became_peered":"2024-09-11T23:19:57.613749+0000","last_unstale":"2024-09-11T23:21:50.308399+0000","last_undegraded":"2024-09-11T23:21:50.308399+0000","last_fullsized":"2024-09-11T23:21:50.308399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:45:22.299379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297086+0000","last_change":"2024-09-11T23:19:59.730758+0000","last_active":"2024-09-11T23:21:50.297086+0000","last_peered":"2024-09-11T23:21:50.297086+0000","last_clean":"2024-09-11T23:21:50.297086+0000","last_became_active":"2024-09-11T23:19:59.730423+0000","last_became_peered":"2024-09-11T23:19:59.730423+0000","last_unstale":"2024-09-11T23:21:50.297086+0000","last_undegraded":"2024-09-11T23:21:50.297086+0000","last_fullsized":"2024-09-11T23:21:50.297086+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:30.679451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":139,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:59.879280+0000","last_change":"2024-09-11T23:19:57.615574+0000","last_active":"2024-09-11T23:21:59.879280+0000","last_peered":"2024-09-11T23:21:59.879280+0000","last_clean":"2024-09-11T23:21:59.879280+0000","last_became_active":"2024-09-11T23:19:57.615192+0000","last_became_peered":"2024-09-11T23:19:57.615192+0000","last_unstale":"2024-09-11T23:21:59.879280+0000","last_undegraded":"2024-09-11T23:21:59.879280+0000","last_fullsized":"2024-09-11T23:21:59.879280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:29.186899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211749+0000","last_change":"2024-09-11T23:19:59.738433+0000","last_active":"2024-09-11T23:21:50.211749+0000","last_peered":"2024-09-11T23:21:50.211749+0000","last_clean":"2024-09-11T23:21:50.211749+0000","last_became_active":"2024-09-11T23:19:59.738081+0000","last_became_peered":"2024-09-11T23:19:59.738081+0000","last_unstale":"2024-09-11T23:21:50.211749+0000","last_undegraded":"2024-09-11T23:21:50.211749+0000","last_fullsized":"2024-09-11T23:21:50.211749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:10:46.786454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455225+0000","last_change":"2024-09-11T23:21:02.437498+0000","last_active":"2024-09-11T23:21:50.455225+0000","last_peered":"2024-09-11T23:21:50.455225+0000","last_clean":"2024-09-11T23:21:50.455225+0000","last_became_active":"2024-09-11T23:21:02.434895+0000","last_became_peered":"2024-09-11T23:21:02.434895+0000","last_unstale":"2024-09-11T23:21:50.455225+0000","last_undegraded":"2024-09-11T23:21:50.455225+0000","last_fullsized":"2024-09-11T23:21:50.455225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:33.142041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455584+0000","last_change":"2024-09-11T23:19:59.741081+0000","last_active":"2024-09-11T23:21:50.455584+0000","last_peered":"2024-09-11T23:21:50.455584+0000","last_clean":"2024-09-11T23:21:50.455584+0000","last_became_active":"2024-09-11T23:19:59.739984+0000","last_became_peered":"2024-09-11T23:19:59.739984+0000","last_unstale":"2024-09-11T23:21:50.455584+0000","last_undegraded":"2024-09-11T23:21:50.455584+0000","last_fullsized":"2024-09-11T23:21:50.455584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:03.513121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308186+0000","last_change":"2024-09-11T23:19:57.823272+0000","last_active":"2024-09-11T23:21:50.308186+0000","last_peered":"2024-09-11T23:21:50.308186+0000","last_clean":"2024-09-11T23:21:50.308186+0000","last_became_active":"2024-09-11T23:19:57.822957+0000","last_became_peered":"2024-09-11T23:19:57.822957+0000","last_unstale":"2024-09-11T23:21:50.308186+0000","last_undegraded":"2024-09-11T23:21:50.308186+0000","last_fullsized":"2024-09-11T23:21:50.308186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:06.779385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211510+0000","last_change":"2024-09-11T23:19:59.738398+0000","last_active":"2024-09-11T23:21:50.211510+0000","last_peered":"2024-09-11T23:21:50.211510+0000","last_clean":"2024-09-11T23:21:50.211510+0000","last_became_active":"2024-09-11T23:19:59.738042+0000","last_became_peered":"2024-09-11T23:19:59.738042+0000","last_unstale":"2024-09-11T23:21:50.211510+0000","last_undegraded":"2024-09-11T23:21:50.211510+0000","last_fullsized":"2024-09-11T23:21:50.211510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:05.098729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'36","reported_seq":296,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:22:00.880783+0000","last_change":"2024-09-11T23:19:57.615036+0000","last_active":"2024-09-11T23:22:00.880783+0000","last_peered":"2024-09-11T23:22:00.880783+0000","last_clean":"2024-09-11T23:22:00.880783+0000","last_became_active":"2024-09-11T23:19:57.614641+0000","last_became_peered":"2024-09-11T23:19:57.614641+0000","last_unstale":"2024-09-11T23:22:00.880783+0000","last_undegraded":"2024-09-11T23:22:00.880783+0000","last_fullsized":"2024-09-11T23:22:00.880783+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:05.430508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69978,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":236,"num_read_kb":187,"num_write":37,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299857+0000","last_change":"2024-09-11T23:19:59.741597+0000","last_active":"2024-09-11T23:21:50.299857+0000","last_peered":"2024-09-11T23:21:50.299857+0000","last_clean":"2024-09-11T23:21:50.299857+0000","last_became_active":"2024-09-11T23:19:59.741296+0000","last_became_peered":"2024-09-11T23:19:59.741296+0000","last_unstale":"2024-09-11T23:21:50.299857+0000","last_undegraded":"2024-09-11T23:21:50.299857+0000","last_fullsized":"2024-09-11T23:21:50.299857+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:06:20.742916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211837+0000","last_change":"2024-09-11T23:19:57.617550+0000","last_active":"2024-09-11T23:21:50.211837+0000","last_peered":"2024-09-11T23:21:50.211837+0000","last_clean":"2024-09-11T23:21:50.211837+0000","last_became_active":"2024-09-11T23:19:57.617211+0000","last_became_peered":"2024-09-11T23:19:57.617211+0000","last_unstale":"2024-09-11T23:21:50.211837+0000","last_undegraded":"2024-09-11T23:21:50.211837+0000","last_fullsized":"2024-09-11T23:21:50.211837+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:19.707742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210301+0000","last_change":"2024-09-11T23:19:59.742070+0000","last_active":"2024-09-11T23:21:50.210301+0000","last_peered":"2024-09-11T23:21:50.210301+0000","last_clean":"2024-09-11T23:21:50.210301+0000","last_became_active":"2024-09-11T23:19:59.739367+0000","last_became_peered":"2024-09-11T23:19:59.739367+0000","last_unstale":"2024-09-11T23:21:50.210301+0000","last_undegraded":"2024-09-11T23:21:50.210301+0000","last_fullsized":"2024-09-11T23:21:50.210301+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:27.850136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308168+0000","last_change":"2024-09-11T23:19:57.618054+0000","last_active":"2024-09-11T23:21:50.308168+0000","last_peered":"2024-09-11T23:21:50.308168+0000","last_clean":"2024-09-11T23:21:50.308168+0000","last_became_active":"2024-09-11T23:19:57.617702+0000","last_became_peered":"2024-09-11T23:19:57.617702+0000","last_unstale":"2024-09-11T23:21:50.308168+0000","last_undegraded":"2024-09-11T23:21:50.308168+0000","last_fullsized":"2024-09-11T23:21:50.308168+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:40.171851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455548+0000","last_change":"2024-09-11T23:19:59.744000+0000","last_active":"2024-09-11T23:21:50.455548+0000","last_peered":"2024-09-11T23:21:50.455548+0000","last_clean":"2024-09-11T23:21:50.455548+0000","last_became_active":"2024-09-11T23:19:59.743629+0000","last_became_peered":"2024-09-11T23:19:59.743629+0000","last_unstale":"2024-09-11T23:21:50.455548+0000","last_undegraded":"2024-09-11T23:21:50.455548+0000","last_fullsized":"2024-09-11T23:21:50.455548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:20:04.133258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'8","reported_seq":149,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:59.879721+0000","last_change":"2024-09-11T23:19:57.615117+0000","last_active":"2024-09-11T23:21:59.879721+0000","last_peered":"2024-09-11T23:21:59.879721+0000","last_clean":"2024-09-11T23:21:59.879721+0000","last_became_active":"2024-09-11T23:19:57.614758+0000","last_became_peered":"2024-09-11T23:19:57.614758+0000","last_unstale":"2024-09-11T23:21:59.879721+0000","last_undegraded":"2024-09-11T23:21:59.879721+0000","last_fullsized":"2024-09-11T23:21:59.879721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:58.113986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2301,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":7,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455281+0000","last_change":"2024-09-11T23:19:59.739403+0000","last_active":"2024-09-11T23:21:50.455281+0000","last_peered":"2024-09-11T23:21:50.455281+0000","last_clean":"2024-09-11T23:21:50.455281+0000","last_became_active":"2024-09-11T23:19:59.739078+0000","last_became_peered":"2024-09-11T23:19:59.739078+0000","last_unstale":"2024-09-11T23:21:50.455281+0000","last_undegraded":"2024-09-11T23:21:50.455281+0000","last_fullsized":"2024-09-11T23:21:50.455281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:46.168720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299488+0000","last_change":"2024-09-11T23:19:57.823323+0000","last_active":"2024-09-11T23:21:50.299488+0000","last_peered":"2024-09-11T23:21:50.299488+0000","last_clean":"2024-09-11T23:21:50.299488+0000","last_became_active":"2024-09-11T23:19:57.822947+0000","last_became_peered":"2024-09-11T23:19:57.822947+0000","last_unstale":"2024-09-11T23:21:50.299488+0000","last_undegraded":"2024-09-11T23:21:50.299488+0000","last_fullsized":"2024-09-11T23:21:50.299488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:12.542911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308091+0000","last_change":"2024-09-11T23:19:59.741163+0000","last_active":"2024-09-11T23:21:50.308091+0000","last_peered":"2024-09-11T23:21:50.308091+0000","last_clean":"2024-09-11T23:21:50.308091+0000","last_became_active":"2024-09-11T23:19:59.740633+0000","last_became_peered":"2024-09-11T23:19:59.740633+0000","last_unstale":"2024-09-11T23:21:50.308091+0000","last_undegraded":"2024-09-11T23:21:50.308091+0000","last_fullsized":"2024-09-11T23:21:50.308091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:26.301775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299941+0000","last_change":"2024-09-11T23:19:57.615804+0000","last_active":"2024-09-11T23:21:50.299941+0000","last_peered":"2024-09-11T23:21:50.299941+0000","last_clean":"2024-09-11T23:21:50.299941+0000","last_became_active":"2024-09-11T23:19:57.615443+0000","last_became_peered":"2024-09-11T23:19:57.615443+0000","last_unstale":"2024-09-11T23:21:50.299941+0000","last_undegraded":"2024-09-11T23:21:50.299941+0000","last_fullsized":"2024-09-11T23:21:50.299941+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:39.296873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598177+0000","last_change":"2024-09-11T23:21:02.360875+0000","last_active":"2024-09-11T23:21:50.598177+0000","last_peered":"2024-09-11T23:21:50.598177+0000","last_clean":"2024-09-11T23:21:50.598177+0000","last_became_active":"2024-09-11T23:21:02.360480+0000","last_became_peered":"2024-09-11T23:21:02.360480+0000","last_unstale":"2024-09-11T23:21:50.598177+0000","last_undegraded":"2024-09-11T23:21:50.598177+0000","last_fullsized":"2024-09-11T23:21:50.598177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:34:04.141587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210448+0000","last_change":"2024-09-11T23:19:57.612873+0000","last_active":"2024-09-11T23:21:50.210448+0000","last_peered":"2024-09-11T23:21:50.210448+0000","last_clean":"2024-09-11T23:21:50.210448+0000","last_became_active":"2024-09-11T23:19:57.612566+0000","last_became_peered":"2024-09-11T23:19:57.612566+0000","last_unstale":"2024-09-11T23:21:50.210448+0000","last_undegraded":"2024-09-11T23:21:50.210448+0000","last_fullsized":"2024-09-11T23:21:50.210448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:46.218212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455770+0000","last_change":"2024-09-11T23:19:59.736352+0000","last_active":"2024-09-11T23:21:50.455770+0000","last_peered":"2024-09-11T23:21:50.455770+0000","last_clean":"2024-09-11T23:21:50.455770+0000","last_became_active":"2024-09-11T23:19:59.735402+0000","last_became_peered":"2024-09-11T23:19:59.735402+0000","last_unstale":"2024-09-11T23:21:50.455770+0000","last_undegraded":"2024-09-11T23:21:50.455770+0000","last_fullsized":"2024-09-11T23:21:50.455770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:41.854526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'65","reported_seq":119,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:22:00.820163+0000","last_change":"2024-09-11T23:19:57.612439+0000","last_active":"2024-09-11T23:22:00.820163+0000","last_peered":"2024-09-11T23:22:00.820163+0000","last_clean":"2024-09-11T23:22:00.820163+0000","last_became_active":"2024-09-11T23:19:57.612030+0000","last_became_peered":"2024-09-11T23:19:57.612030+0000","last_unstale":"2024-09-11T23:22:00.820163+0000","last_undegraded":"2024-09-11T23:22:00.820163+0000","last_fullsized":"2024-09-11T23:22:00.820163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:36:24.579246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455480+0000","last_change":"2024-09-11T23:19:59.741241+0000","last_active":"2024-09-11T23:21:50.455480+0000","last_peered":"2024-09-11T23:21:50.455480+0000","last_clean":"2024-09-11T23:21:50.455480+0000","last_became_active":"2024-09-11T23:19:59.740265+0000","last_became_peered":"2024-09-11T23:19:59.740265+0000","last_unstale":"2024-09-11T23:21:50.455480+0000","last_undegraded":"2024-09-11T23:21:50.455480+0000","last_fullsized":"2024-09-11T23:21:50.455480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:00.676762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211975+0000","last_change":"2024-09-11T23:19:57.825238+0000","last_active":"2024-09-11T23:21:50.211975+0000","last_peered":"2024-09-11T23:21:50.211975+0000","last_clean":"2024-09-11T23:21:50.211975+0000","last_became_active":"2024-09-11T23:19:57.824732+0000","last_became_peered":"2024-09-11T23:19:57.824732+0000","last_unstale":"2024-09-11T23:21:50.211975+0000","last_undegraded":"2024-09-11T23:21:50.211975+0000","last_fullsized":"2024-09-11T23:21:50.211975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:24.465863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455634+0000","last_change":"2024-09-11T23:19:59.741338+0000","last_active":"2024-09-11T23:21:50.455634+0000","last_peered":"2024-09-11T23:21:50.455634+0000","last_clean":"2024-09-11T23:21:50.455634+0000","last_became_active":"2024-09-11T23:19:59.741019+0000","last_became_peered":"2024-09-11T23:19:59.741019+0000","last_unstale":"2024-09-11T23:21:50.455634+0000","last_undegraded":"2024-09-11T23:21:50.455634+0000","last_fullsized":"2024-09-11T23:21:50.455634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:36:03.480006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"85'148","reported_seq":395,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:52.311759+0000","last_change":"2024-09-11T23:18:45.606646+0000","last_active":"2024-09-11T23:21:52.311759+0000","last_peered":"2024-09-11T23:21:52.311759+0000","last_clean":"2024-09-11T23:21:52.311759+0000","last_became_active":"2024-09-11T23:18:45.606065+0000","last_became_peered":"2024-09-11T23:18:45.606065+0000","last_unstale":"2024-09-11T23:21:52.311759+0000","last_undegraded":"2024-09-11T23:21:52.311759+0000","last_fullsized":"2024-09-11T23:21:52.311759+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:15:17.148562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:15:17.148562+0000","last_clean_scrub_stamp":"2024-09-11T23:15:17.148562+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:07.992971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.453802+0000","last_change":"2024-09-11T23:19:57.612584+0000","last_active":"2024-09-11T23:21:50.453802+0000","last_peered":"2024-09-11T23:21:50.453802+0000","last_clean":"2024-09-11T23:21:50.453802+0000","last_became_active":"2024-09-11T23:19:57.612211+0000","last_became_peered":"2024-09-11T23:19:57.612211+0000","last_unstale":"2024-09-11T23:21:50.453802+0000","last_undegraded":"2024-09-11T23:21:50.453802+0000","last_fullsized":"2024-09-11T23:21:50.453802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:04.026233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.300938+0000","last_change":"2024-09-11T23:21:02.359249+0000","last_active":"2024-09-11T23:21:50.300938+0000","last_peered":"2024-09-11T23:21:50.300938+0000","last_clean":"2024-09-11T23:21:50.300938+0000","last_became_active":"2024-09-11T23:21:02.358906+0000","last_became_peered":"2024-09-11T23:21:02.358906+0000","last_unstale":"2024-09-11T23:21:50.300938+0000","last_undegraded":"2024-09-11T23:21:50.300938+0000","last_fullsized":"2024-09-11T23:21:50.300938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:05.973581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455482+0000","last_change":"2024-09-11T23:19:57.618283+0000","last_active":"2024-09-11T23:21:50.455482+0000","last_peered":"2024-09-11T23:21:50.455482+0000","last_clean":"2024-09-11T23:21:50.455482+0000","last_became_active":"2024-09-11T23:19:57.617942+0000","last_became_peered":"2024-09-11T23:19:57.617942+0000","last_unstale":"2024-09-11T23:21:50.455482+0000","last_undegraded":"2024-09-11T23:21:50.455482+0000","last_fullsized":"2024-09-11T23:21:50.455482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:30.832602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.209864+0000","last_change":"2024-09-11T23:19:59.740730+0000","last_active":"2024-09-11T23:21:50.209864+0000","last_peered":"2024-09-11T23:21:50.209864+0000","last_clean":"2024-09-11T23:21:50.209864+0000","last_became_active":"2024-09-11T23:19:59.740369+0000","last_became_peered":"2024-09-11T23:19:59.740369+0000","last_unstale":"2024-09-11T23:21:50.209864+0000","last_undegraded":"2024-09-11T23:21:50.209864+0000","last_fullsized":"2024-09-11T23:21:50.209864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:14:50.264644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":142,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:22:00.880529+0000","last_change":"2024-09-11T23:19:57.823253+0000","last_active":"2024-09-11T23:22:00.880529+0000","last_peered":"2024-09-11T23:22:00.880529+0000","last_clean":"2024-09-11T23:22:00.880529+0000","last_became_active":"2024-09-11T23:19:57.822895+0000","last_became_peered":"2024-09-11T23:19:57.822895+0000","last_unstale":"2024-09-11T23:22:00.880529+0000","last_undegraded":"2024-09-11T23:22:00.880529+0000","last_fullsized":"2024-09-11T23:22:00.880529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:22.998371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308142+0000","last_change":"2024-09-11T23:19:59.742248+0000","last_active":"2024-09-11T23:21:50.308142+0000","last_peered":"2024-09-11T23:21:50.308142+0000","last_clean":"2024-09-11T23:21:50.308142+0000","last_became_active":"2024-09-11T23:19:59.741810+0000","last_became_peered":"2024-09-11T23:19:59.741810+0000","last_unstale":"2024-09-11T23:21:50.308142+0000","last_undegraded":"2024-09-11T23:21:50.308142+0000","last_fullsized":"2024-09-11T23:21:50.308142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:19:52.720286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455554+0000","last_change":"2024-09-11T23:19:57.616244+0000","last_active":"2024-09-11T23:21:50.455554+0000","last_peered":"2024-09-11T23:21:50.455554+0000","last_clean":"2024-09-11T23:21:50.455554+0000","last_became_active":"2024-09-11T23:19:57.615868+0000","last_became_peered":"2024-09-11T23:19:57.615868+0000","last_unstale":"2024-09-11T23:21:50.455554+0000","last_undegraded":"2024-09-11T23:21:50.455554+0000","last_fullsized":"2024-09-11T23:21:50.455554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:30:42.510896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311069+0000","last_change":"2024-09-11T23:21:02.434265+0000","last_active":"2024-09-11T23:21:50.311069+0000","last_peered":"2024-09-11T23:21:50.311069+0000","last_clean":"2024-09-11T23:21:50.311069+0000","last_became_active":"2024-09-11T23:21:02.433840+0000","last_became_peered":"2024-09-11T23:21:02.433840+0000","last_unstale":"2024-09-11T23:21:50.311069+0000","last_undegraded":"2024-09-11T23:21:50.311069+0000","last_fullsized":"2024-09-11T23:21:50.311069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:04.619695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297618+0000","last_change":"2024-09-11T23:19:57.824334+0000","last_active":"2024-09-11T23:21:50.297618+0000","last_peered":"2024-09-11T23:21:50.297618+0000","last_clean":"2024-09-11T23:21:50.297618+0000","last_became_active":"2024-09-11T23:19:57.823972+0000","last_became_peered":"2024-09-11T23:19:57.823972+0000","last_unstale":"2024-09-11T23:21:50.297618+0000","last_undegraded":"2024-09-11T23:21:50.297618+0000","last_fullsized":"2024-09-11T23:21:50.297618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:00:39.183281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.311164+0000","last_change":"2024-09-11T23:19:59.728064+0000","last_active":"2024-09-11T23:21:50.311164+0000","last_peered":"2024-09-11T23:21:50.311164+0000","last_clean":"2024-09-11T23:21:50.311164+0000","last_became_active":"2024-09-11T23:19:59.727732+0000","last_became_peered":"2024-09-11T23:19:59.727732+0000","last_unstale":"2024-09-11T23:21:50.311164+0000","last_undegraded":"2024-09-11T23:21:50.311164+0000","last_fullsized":"2024-09-11T23:21:50.311164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:15.383974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299855+0000","last_change":"2024-09-11T23:19:57.824362+0000","last_active":"2024-09-11T23:21:50.299855+0000","last_peered":"2024-09-11T23:21:50.299855+0000","last_clean":"2024-09-11T23:21:50.299855+0000","last_became_active":"2024-09-11T23:19:57.823753+0000","last_became_peered":"2024-09-11T23:19:57.823753+0000","last_unstale":"2024-09-11T23:21:50.299855+0000","last_undegraded":"2024-09-11T23:21:50.299855+0000","last_fullsized":"2024-09-11T23:21:50.299855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:56.357268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.210323+0000","last_change":"2024-09-11T23:19:59.730623+0000","last_active":"2024-09-11T23:21:50.210323+0000","last_peered":"2024-09-11T23:21:50.210323+0000","last_clean":"2024-09-11T23:21:50.210323+0000","last_became_active":"2024-09-11T23:19:59.730307+0000","last_became_peered":"2024-09-11T23:19:59.730307+0000","last_unstale":"2024-09-11T23:21:50.210323+0000","last_undegraded":"2024-09-11T23:21:50.210323+0000","last_fullsized":"2024-09-11T23:21:50.210323+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:52.874884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'100","reported_seq":213,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:22:01.200407+0000","last_change":"2024-09-11T23:19:57.611730+0000","last_active":"2024-09-11T23:22:01.200407+0000","last_peered":"2024-09-11T23:22:01.200407+0000","last_clean":"2024-09-11T23:22:01.200407+0000","last_became_active":"2024-09-11T23:19:57.611394+0000","last_became_peered":"2024-09-11T23:19:57.611394+0000","last_unstale":"2024-09-11T23:22:01.200407+0000","last_undegraded":"2024-09-11T23:22:01.200407+0000","last_fullsized":"2024-09-11T23:22:01.200407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:15.518573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":183,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.302035+0000","last_change":"2024-09-11T23:19:59.741396+0000","last_active":"2024-09-11T23:21:50.302035+0000","last_peered":"2024-09-11T23:21:50.302035+0000","last_clean":"2024-09-11T23:21:50.302035+0000","last_became_active":"2024-09-11T23:19:59.741102+0000","last_became_peered":"2024-09-11T23:19:59.741102+0000","last_unstale":"2024-09-11T23:21:50.302035+0000","last_undegraded":"2024-09-11T23:21:50.302035+0000","last_fullsized":"2024-09-11T23:21:50.302035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:08.759512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.453918+0000","last_change":"2024-09-11T23:19:57.824812+0000","last_active":"2024-09-11T23:21:50.453918+0000","last_peered":"2024-09-11T23:21:50.453918+0000","last_clean":"2024-09-11T23:21:50.453918+0000","last_became_active":"2024-09-11T23:19:57.824454+0000","last_became_peered":"2024-09-11T23:19:57.824454+0000","last_unstale":"2024-09-11T23:21:50.453918+0000","last_undegraded":"2024-09-11T23:21:50.453918+0000","last_fullsized":"2024-09-11T23:21:50.453918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:25.275573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455034+0000","last_change":"2024-09-11T23:19:59.735857+0000","last_active":"2024-09-11T23:21:50.455034+0000","last_peered":"2024-09-11T23:21:50.455034+0000","last_clean":"2024-09-11T23:21:50.455034+0000","last_became_active":"2024-09-11T23:19:59.735541+0000","last_became_peered":"2024-09-11T23:19:59.735541+0000","last_unstale":"2024-09-11T23:21:50.455034+0000","last_undegraded":"2024-09-11T23:21:50.455034+0000","last_fullsized":"2024-09-11T23:21:50.455034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:21.600490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211231+0000","last_change":"2024-09-11T23:19:57.823468+0000","last_active":"2024-09-11T23:21:50.211231+0000","last_peered":"2024-09-11T23:21:50.211231+0000","last_clean":"2024-09-11T23:21:50.211231+0000","last_became_active":"2024-09-11T23:19:57.823154+0000","last_became_peered":"2024-09-11T23:19:57.823154+0000","last_unstale":"2024-09-11T23:21:50.211231+0000","last_undegraded":"2024-09-11T23:21:50.211231+0000","last_fullsized":"2024-09-11T23:21:50.211231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:28.770627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299509+0000","last_change":"2024-09-11T23:19:59.741263+0000","last_active":"2024-09-11T23:21:50.299509+0000","last_peered":"2024-09-11T23:21:50.299509+0000","last_clean":"2024-09-11T23:21:50.299509+0000","last_became_active":"2024-09-11T23:19:59.740938+0000","last_became_peered":"2024-09-11T23:19:59.740938+0000","last_unstale":"2024-09-11T23:21:50.299509+0000","last_undegraded":"2024-09-11T23:21:50.299509+0000","last_fullsized":"2024-09-11T23:21:50.299509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:54.768652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.301723+0000","last_change":"2024-09-11T23:19:57.612517+0000","last_active":"2024-09-11T23:21:50.301723+0000","last_peered":"2024-09-11T23:21:50.301723+0000","last_clean":"2024-09-11T23:21:50.301723+0000","last_became_active":"2024-09-11T23:19:57.612188+0000","last_became_peered":"2024-09-11T23:19:57.612188+0000","last_unstale":"2024-09-11T23:21:50.301723+0000","last_undegraded":"2024-09-11T23:21:50.301723+0000","last_fullsized":"2024-09-11T23:21:50.301723+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:52.896413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598315+0000","last_change":"2024-09-11T23:19:59.732520+0000","last_active":"2024-09-11T23:21:50.598315+0000","last_peered":"2024-09-11T23:21:50.598315+0000","last_clean":"2024-09-11T23:21:50.598315+0000","last_became_active":"2024-09-11T23:19:59.732173+0000","last_became_peered":"2024-09-11T23:19:59.732173+0000","last_unstale":"2024-09-11T23:21:50.598315+0000","last_undegraded":"2024-09-11T23:21:50.598315+0000","last_fullsized":"2024-09-11T23:21:50.598315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:04.626312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598177+0000","last_change":"2024-09-11T23:21:02.365533+0000","last_active":"2024-09-11T23:21:50.598177+0000","last_peered":"2024-09-11T23:21:50.598177+0000","last_clean":"2024-09-11T23:21:50.598177+0000","last_became_active":"2024-09-11T23:21:02.359345+0000","last_became_peered":"2024-09-11T23:21:02.359345+0000","last_unstale":"2024-09-11T23:21:50.598177+0000","last_undegraded":"2024-09-11T23:21:50.598177+0000","last_fullsized":"2024-09-11T23:21:50.598177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:27.195666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.301266+0000","last_change":"2024-09-11T23:19:59.734667+0000","last_active":"2024-09-11T23:21:50.301266+0000","last_peered":"2024-09-11T23:21:50.301266+0000","last_clean":"2024-09-11T23:21:50.301266+0000","last_became_active":"2024-09-11T23:19:59.733754+0000","last_became_peered":"2024-09-11T23:19:59.733754+0000","last_unstale":"2024-09-11T23:21:50.301266+0000","last_undegraded":"2024-09-11T23:21:50.301266+0000","last_fullsized":"2024-09-11T23:21:50.301266+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:57.392272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299623+0000","last_change":"2024-09-11T23:19:57.614579+0000","last_active":"2024-09-11T23:21:50.299623+0000","last_peered":"2024-09-11T23:21:50.299623+0000","last_clean":"2024-09-11T23:21:50.299623+0000","last_became_active":"2024-09-11T23:19:57.614227+0000","last_became_peered":"2024-09-11T23:19:57.614227+0000","last_unstale":"2024-09-11T23:21:50.299623+0000","last_undegraded":"2024-09-11T23:21:50.299623+0000","last_fullsized":"2024-09-11T23:21:50.299623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:47.952920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455414+0000","last_change":"2024-09-11T23:19:59.744219+0000","last_active":"2024-09-11T23:21:50.455414+0000","last_peered":"2024-09-11T23:21:50.455414+0000","last_clean":"2024-09-11T23:21:50.455414+0000","last_became_active":"2024-09-11T23:19:59.743914+0000","last_became_peered":"2024-09-11T23:19:59.743914+0000","last_unstale":"2024-09-11T23:21:50.455414+0000","last_undegraded":"2024-09-11T23:21:50.455414+0000","last_fullsized":"2024-09-11T23:21:50.455414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:52.149588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.308372+0000","last_change":"2024-09-11T23:21:02.361647+0000","last_active":"2024-09-11T23:21:50.308372+0000","last_peered":"2024-09-11T23:21:50.308372+0000","last_clean":"2024-09-11T23:21:50.308372+0000","last_became_active":"2024-09-11T23:21:02.361319+0000","last_became_peered":"2024-09-11T23:21:02.361319+0000","last_unstale":"2024-09-11T23:21:50.308372+0000","last_undegraded":"2024-09-11T23:21:50.308372+0000","last_fullsized":"2024-09-11T23:21:50.308372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:48:45.125525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.299796+0000","last_change":"2024-09-11T23:19:57.610389+0000","last_active":"2024-09-11T23:21:50.299796+0000","last_peered":"2024-09-11T23:21:50.299796+0000","last_clean":"2024-09-11T23:21:50.299796+0000","last_became_active":"2024-09-11T23:19:57.609911+0000","last_became_peered":"2024-09-11T23:19:57.609911+0000","last_unstale":"2024-09-11T23:21:50.299796+0000","last_undegraded":"2024-09-11T23:21:50.299796+0000","last_fullsized":"2024-09-11T23:21:50.299796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:48:23.949929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.211496+0000","last_change":"2024-09-11T23:19:59.736867+0000","last_active":"2024-09-11T23:21:50.211496+0000","last_peered":"2024-09-11T23:21:50.211496+0000","last_clean":"2024-09-11T23:21:50.211496+0000","last_became_active":"2024-09-11T23:19:59.736341+0000","last_became_peered":"2024-09-11T23:19:59.736341+0000","last_unstale":"2024-09-11T23:21:50.211496+0000","last_undegraded":"2024-09-11T23:21:50.211496+0000","last_fullsized":"2024-09-11T23:21:50.211496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:57:51.015235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.598177+0000","last_change":"2024-09-11T23:21:02.368959+0000","last_active":"2024-09-11T23:21:50.598177+0000","last_peered":"2024-09-11T23:21:50.598177+0000","last_clean":"2024-09-11T23:21:50.598177+0000","last_became_active":"2024-09-11T23:21:02.363263+0000","last_became_peered":"2024-09-11T23:21:02.363263+0000","last_unstale":"2024-09-11T23:21:50.598177+0000","last_undegraded":"2024-09-11T23:21:50.598177+0000","last_fullsized":"2024-09-11T23:21:50.598177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:29.827330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.455355+0000","last_change":"2024-09-11T23:19:59.742049+0000","last_active":"2024-09-11T23:21:50.455355+0000","last_peered":"2024-09-11T23:21:50.455355+0000","last_clean":"2024-09-11T23:21:50.455355+0000","last_became_active":"2024-09-11T23:19:59.741763+0000","last_became_peered":"2024-09-11T23:19:59.741763+0000","last_unstale":"2024-09-11T23:21:50.455355+0000","last_undegraded":"2024-09-11T23:21:50.455355+0000","last_fullsized":"2024-09-11T23:21:50.455355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:48.698896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.302151+0000","last_change":"2024-09-11T23:19:57.822949+0000","last_active":"2024-09-11T23:21:50.302151+0000","last_peered":"2024-09-11T23:21:50.302151+0000","last_clean":"2024-09-11T23:21:50.302151+0000","last_became_active":"2024-09-11T23:19:57.822564+0000","last_became_peered":"2024-09-11T23:19:57.822564+0000","last_unstale":"2024-09-11T23:21:50.302151+0000","last_undegraded":"2024-09-11T23:21:50.302151+0000","last_fullsized":"2024-09-11T23:21:50.302151+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:03:29.532109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297740+0000","last_change":"2024-09-11T23:21:02.363639+0000","last_active":"2024-09-11T23:21:50.297740+0000","last_peered":"2024-09-11T23:21:50.297740+0000","last_clean":"2024-09-11T23:21:50.297740+0000","last_became_active":"2024-09-11T23:21:02.363247+0000","last_became_peered":"2024-09-11T23:21:02.363247+0000","last_unstale":"2024-09-11T23:21:50.297740+0000","last_undegraded":"2024-09-11T23:21:50.297740+0000","last_fullsized":"2024-09-11T23:21:50.297740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:58.708428+0000","last_clean_scrub_stamp":"2024-09-11T23:19:58.708428+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:18:58.906939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T23:21:50.297895+0000","last_change":"2024-09-11T23:19:57.613323+0000","last_active":"2024-09-11T23:21:50.297895+0000","last_peered":"2024-09-11T23:21:50.297895+0000","last_clean":"2024-09-11T23:21:50.297895+0000","last_became_active":"2024-09-11T23:19:57.612985+0000","last_became_peered":"2024-09-11T23:19:57.612985+0000","last_unstale":"2024-09-11T23:21:50.297895+0000","last_undegraded":"2024-09-11T23:21:50.297895+0000","last_fullsized":"2024-09-11T23:21:50.297895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T23:19:56.586466+0000","last_clean_scrub_stamp":"2024-09-11T23:19:56.586466+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:13:09.179547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":797142,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1038,"num_read_kb":732,"num_write":418,"num_write_kb":2357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":2391426,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108948,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514533,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":450800,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743465,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1126400,"data_stored":1442204,"data_compressed":60217,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906992,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":844252,"data_compressed":27988,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168629,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":844172,"data_compressed":27974,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":1044601,"data_compressed":45015,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561087,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":450425,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1044480,"data_stored":1370804,"data_compressed":60217,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":452575,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1011712,"data_stored":1172184,"data_compressed":42814,"data_compressed_allocated":57344,"data_compressed_original":724992,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":521939,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":520196,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":946176,"data_stored":451696,"data_compressed":8699,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36316,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":330101,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":401501,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T23:22:07.158 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:22:07.415 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:07.838 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:07 smithi043 ceph-mon[30950]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-09-11T23:22:07.838 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:07 smithi043 ceph-mon[30950]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:22:07.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.922+0000 7f5170028640 1 -- 172.21.15.43:0/3874566243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51681009d0 msgr2=0x7f5168100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:07.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.922+0000 7f5170028640 1 --2- 172.21.15.43:0/3874566243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51681009d0 0x7f5168100db0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5164009900 tx=0x7f516402a410 comp rx=0 tx=0).stop 2024-09-11T23:22:07.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.922+0000 7f5170028640 1 -- 172.21.15.43:0/3874566243 shutdown_connections 2024-09-11T23:22:07.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.922+0000 7f5170028640 1 --2- 172.21.15.43:0/3874566243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f516810e4f0 0x7f5168110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.922+0000 7f5170028640 1 --2- 172.21.15.43:0/3874566243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5168101380 0x7f516810dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.922+0000 7f5170028640 1 --2- 172.21.15.43:0/3874566243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51681009d0 0x7f5168100db0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.922+0000 7f5170028640 1 -- 172.21.15.43:0/3874566243 >> 172.21.15.43:0/3874566243 conn(0x7f51680fc7d0 msgr2=0x7f51680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.922+0000 7f5170028640 1 -- 172.21.15.43:0/3874566243 shutdown_connections 2024-09-11T23:22:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.922+0000 7f5170028640 1 -- 172.21.15.43:0/3874566243 wait complete. 2024-09-11T23:22:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.923+0000 7f5170028640 1 Processor -- start 2024-09-11T23:22:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.923+0000 7f5170028640 1 -- start start 2024-09-11T23:22:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.923+0000 7f5170028640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51681009d0 0x7f51681a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.923+0000 7f5170028640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5168101380 0x7f51681a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.923+0000 7f5170028640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f516810e4f0 0x7f516819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.923+0000 7f5170028640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f51681142c0 con 0x7f516810e4f0 2024-09-11T23:22:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.923+0000 7f5170028640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5168114140 con 0x7f5168101380 2024-09-11T23:22:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.923+0000 7f5170028640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5168114440 con 0x7f51681009d0 2024-09-11T23:22:07.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.923+0000 7f516e5c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f516810e4f0 0x7f516819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.923+0000 7f516e5c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f516810e4f0 0x7f516819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50000/0 (socket says 172.21.15.43:50000) 2024-09-11T23:22:07.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.923+0000 7f516e5c5640 1 -- 172.21.15.43:0/2354406283 learned_addr learned my addr 172.21.15.43:0/2354406283 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:07.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.923+0000 7f516ddc4640 1 --2- 172.21.15.43:0/2354406283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51681009d0 0x7f51681a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.923+0000 7f516d5c3640 1 --2- 172.21.15.43:0/2354406283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5168101380 0x7f51681a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.923+0000 7f516e5c5640 1 -- 172.21.15.43:0/2354406283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51681009d0 msgr2=0x7f51681a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:07.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.923+0000 7f516e5c5640 1 --2- 172.21.15.43:0/2354406283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51681009d0 0x7f51681a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.923+0000 7f516e5c5640 1 -- 172.21.15.43:0/2354406283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5168101380 msgr2=0x7f51681a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:07.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.923+0000 7f516e5c5640 1 --2- 172.21.15.43:0/2354406283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5168101380 0x7f51681a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:07.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.923+0000 7f516e5c5640 1 -- 172.21.15.43:0/2354406283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516819ace0 con 0x7f516810e4f0 2024-09-11T23:22:07.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.924+0000 7f516e5c5640 1 --2- 172.21.15.43:0/2354406283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f516810e4f0 0x7f516819a5b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f516000e9d0 tx=0x7f516000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:07.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.924+0000 7f515effd640 1 -- 172.21.15.43:0/2354406283 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516000cd80 con 0x7f516810e4f0 2024-09-11T23:22:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.924+0000 7f5170028640 1 -- 172.21.15.43:0/2354406283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f516819afd0 con 0x7f516810e4f0 2024-09-11T23:22:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.924+0000 7f515effd640 1 -- 172.21.15.43:0/2354406283 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5160005020 con 0x7f516810e4f0 2024-09-11T23:22:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.924+0000 7f515effd640 1 -- 172.21.15.43:0/2354406283 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51600116d0 con 0x7f516810e4f0 2024-09-11T23:22:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.924+0000 7f5170028640 1 -- 172.21.15.43:0/2354406283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51681a72c0 con 0x7f516810e4f0 2024-09-11T23:22:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.925+0000 7f5170028640 1 -- 172.21.15.43:0/2354406283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f516806a7d0 con 0x7f516810e4f0 2024-09-11T23:22:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.927+0000 7f515effd640 1 -- 172.21.15.43:0/2354406283 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f51600051d0 con 0x7f516810e4f0 2024-09-11T23:22:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.927+0000 7f515effd640 1 --2- 172.21.15.43:0/2354406283 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f51540a2e10 0x7f51540a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.928+0000 7f515effd640 1 -- 172.21.15.43:0/2354406283 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f51600cfbb0 con 0x7f516810e4f0 2024-09-11T23:22:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.928+0000 7f516ddc4640 1 --2- 172.21.15.43:0/2354406283 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f51540a2e10 0x7f51540a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.928+0000 7f516ddc4640 1 --2- 172.21.15.43:0/2354406283 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f51540a2e10 0x7f51540a52d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5164000c00 tx=0x7f5164005b50 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:07.931+0000 7f515effd640 1 -- 172.21.15.43:0/2354406283 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5160016030 con 0x7f516810e4f0 2024-09-11T23:22:08.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:07 smithi090 ceph-mon[35130]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-09-11T23:22:08.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:07 smithi090 ceph-mon[35130]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:22:08.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:07 smithi097 ceph-mon[35562]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-09-11T23:22:08.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:07 smithi097 ceph-mon[35562]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T23:22:08.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:08.144+0000 7f5170028640 1 -- 172.21.15.43:0/2354406283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5168109680 con 0x7f516810e4f0 2024-09-11T23:22:08.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:08.145+0000 7f515effd640 1 -- 172.21.15.43:0/2354406283 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11896 (secure 0 0 0) 0x7f516009bed0 con 0x7f516810e4f0 2024-09-11T23:22:08.145 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:22:08.146 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"btime":"2024-09-11T23:21:49:167551+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:21:49.117342+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:22:08.146 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 24 2024-09-11T23:22:08.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:08.147+0000 7f5170028640 1 -- 172.21.15.43:0/2354406283 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f51540a2e10 msgr2=0x7f51540a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:08.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:08.147+0000 7f5170028640 1 --2- 172.21.15.43:0/2354406283 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f51540a2e10 0x7f51540a52d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5164000c00 tx=0x7f5164005b50 comp rx=0 tx=0).stop 2024-09-11T23:22:08.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:08.147+0000 7f5170028640 1 -- 172.21.15.43:0/2354406283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f516810e4f0 msgr2=0x7f516819a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:08.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:08.147+0000 7f5170028640 1 --2- 172.21.15.43:0/2354406283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f516810e4f0 0x7f516819a5b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f516000e9d0 tx=0x7f516000ee90 comp rx=0 tx=0).stop 2024-09-11T23:22:08.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:08.147+0000 7f5170028640 1 -- 172.21.15.43:0/2354406283 shutdown_connections 2024-09-11T23:22:08.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:08.147+0000 7f5170028640 1 --2- 172.21.15.43:0/2354406283 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f51540a2e10 0x7f51540a52d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:08.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:08.147+0000 7f5170028640 1 --2- 172.21.15.43:0/2354406283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f516810e4f0 0x7f516819a5b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:08.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:08.147+0000 7f5170028640 1 --2- 172.21.15.43:0/2354406283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5168101380 0x7f51681a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:08.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:08.147+0000 7f5170028640 1 --2- 172.21.15.43:0/2354406283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51681009d0 0x7f51681a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:08.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:08.147+0000 7f5170028640 1 -- 172.21.15.43:0/2354406283 >> 172.21.15.43:0/2354406283 conn(0x7f51680fc7d0 msgr2=0x7f51680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:08.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:08.147+0000 7f5170028640 1 -- 172.21.15.43:0/2354406283 shutdown_connections 2024-09-11T23:22:08.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:08.147+0000 7f5170028640 1 -- 172.21.15.43:0/2354406283 wait complete. 2024-09-11T23:22:08.860 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:22:08.885 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:08 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2354406283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:09.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:08 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2354406283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:09.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:08 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2354406283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:09.121 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:09.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.620+0000 7f81a5762640 1 -- 172.21.15.43:0/3108270814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a01008d0 msgr2=0x7f81a0100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:09.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.620+0000 7f81a5762640 1 --2- 172.21.15.43:0/3108270814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a01008d0 0x7f81a0100cb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8190009900 tx=0x7f819002a410 comp rx=0 tx=0).stop 2024-09-11T23:22:09.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.620+0000 7f81a5762640 1 -- 172.21.15.43:0/3108270814 shutdown_connections 2024-09-11T23:22:09.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.620+0000 7f81a5762640 1 --2- 172.21.15.43:0/3108270814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a010e480 0x7f81a0110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:09.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.620+0000 7f81a5762640 1 --2- 172.21.15.43:0/3108270814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81a0101280 0x7f81a010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:09.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.620+0000 7f81a5762640 1 --2- 172.21.15.43:0/3108270814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a01008d0 0x7f81a0100cb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:09.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.620+0000 7f81a5762640 1 -- 172.21.15.43:0/3108270814 >> 172.21.15.43:0/3108270814 conn(0x7f81a00fc7a0 msgr2=0x7f81a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:09.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.620+0000 7f81a5762640 1 -- 172.21.15.43:0/3108270814 shutdown_connections 2024-09-11T23:22:09.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.620+0000 7f81a5762640 1 -- 172.21.15.43:0/3108270814 wait complete. 2024-09-11T23:22:09.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.621+0000 7f81a5762640 1 Processor -- start 2024-09-11T23:22:09.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.621+0000 7f81a5762640 1 -- start start 2024-09-11T23:22:09.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.621+0000 7f81a5762640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a01008d0 0x7f81a019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:09.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.621+0000 7f81a5762640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81a0101280 0x7f81a019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:09.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.621+0000 7f81a5762640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a010e480 0x7f81a019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:09.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.621+0000 7f81a5762640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f81a0114190 con 0x7f81a010e480 2024-09-11T23:22:09.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.621+0000 7f81a5762640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f81a0114010 con 0x7f81a01008d0 2024-09-11T23:22:09.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.621+0000 7f81a5762640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f81a0114310 con 0x7f81a0101280 2024-09-11T23:22:09.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.621+0000 7f819f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a010e480 0x7f81a019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:09.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.621+0000 7f819f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a010e480 0x7f81a019ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50018/0 (socket says 172.21.15.43:50018) 2024-09-11T23:22:09.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.621+0000 7f819f7fe640 1 -- 172.21.15.43:0/1845691135 learned_addr learned my addr 172.21.15.43:0/1845691135 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:09.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.621+0000 7f819e7fc640 1 --2- 172.21.15.43:0/1845691135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81a0101280 0x7f81a019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:09.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.622+0000 7f819effd640 1 --2- 172.21.15.43:0/1845691135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a01008d0 0x7f81a019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:09.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.622+0000 7f819f7fe640 1 -- 172.21.15.43:0/1845691135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81a0101280 msgr2=0x7f81a019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:09.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.622+0000 7f819f7fe640 1 --2- 172.21.15.43:0/1845691135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81a0101280 0x7f81a019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:09.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.622+0000 7f819f7fe640 1 -- 172.21.15.43:0/1845691135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a01008d0 msgr2=0x7f81a019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:09.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.622+0000 7f819f7fe640 1 --2- 172.21.15.43:0/1845691135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a01008d0 0x7f81a019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:09.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.622+0000 7f819f7fe640 1 -- 172.21.15.43:0/1845691135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a019f660 con 0x7f81a010e480 2024-09-11T23:22:09.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.622+0000 7f819effd640 1 --2- 172.21.15.43:0/1845691135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a01008d0 0x7f81a019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:09.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.622+0000 7f819f7fe640 1 --2- 172.21.15.43:0/1845691135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a010e480 0x7f81a019ef80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f819800cab0 tx=0x7f819800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:09.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.622+0000 7f8187fff640 1 -- 172.21.15.43:0/1845691135 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198007e10 con 0x7f81a010e480 2024-09-11T23:22:09.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.622+0000 7f8187fff640 1 -- 172.21.15.43:0/1845691135 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f819800f930 con 0x7f81a010e480 2024-09-11T23:22:09.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.622+0000 7f81a5762640 1 -- 172.21.15.43:0/1845691135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a019f950 con 0x7f81a010e480 2024-09-11T23:22:09.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.622+0000 7f8187fff640 1 -- 172.21.15.43:0/1845691135 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198018b80 con 0x7f81a010e480 2024-09-11T23:22:09.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.622+0000 7f81a5762640 1 -- 172.21.15.43:0/1845691135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a01a7230 con 0x7f81a010e480 2024-09-11T23:22:09.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.623+0000 7f81a5762640 1 -- 172.21.15.43:0/1845691135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a0100cb0 con 0x7f81a010e480 2024-09-11T23:22:09.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.624+0000 7f8187fff640 1 -- 172.21.15.43:0/1845691135 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f819800fae0 con 0x7f81a010e480 2024-09-11T23:22:09.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.625+0000 7f8187fff640 1 --2- 172.21.15.43:0/1845691135 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f81880a2e10 0x7f81880a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:09.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.625+0000 7f819effd640 1 --2- 172.21.15.43:0/1845691135 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f81880a2e10 0x7f81880a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:09.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.625+0000 7f8187fff640 1 -- 172.21.15.43:0/1845691135 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f81980cfa00 con 0x7f81a010e480 2024-09-11T23:22:09.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.626+0000 7f819effd640 1 --2- 172.21.15.43:0/1845691135 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f81880a2e10 0x7f81880a52d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8190005ab0 tx=0x7f8190005a20 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:09.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.629+0000 7f8187fff640 1 -- 172.21.15.43:0/1845691135 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8198014030 con 0x7f81a010e480 2024-09-11T23:22:09.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.840+0000 7f81a5762640 1 -- 172.21.15.43:0/1845691135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81a002d7c0 con 0x7f81a010e480 2024-09-11T23:22:09.841 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:09 smithi043 ceph-mon[30950]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-11T23:22:09.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.841+0000 7f8187fff640 1 -- 172.21.15.43:0/1845691135 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11896 (secure 0 0 0) 0x7f819809c3d0 con 0x7f81a010e480 2024-09-11T23:22:09.842 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:22:09.842 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"btime":"2024-09-11T23:21:49:167551+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:21:49.117342+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:22:09.842 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 24 2024-09-11T23:22:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.844+0000 7f81a5762640 1 -- 172.21.15.43:0/1845691135 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f81880a2e10 msgr2=0x7f81880a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.844+0000 7f81a5762640 1 --2- 172.21.15.43:0/1845691135 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f81880a2e10 0x7f81880a52d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8190005ab0 tx=0x7f8190005a20 comp rx=0 tx=0).stop 2024-09-11T23:22:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.844+0000 7f81a5762640 1 -- 172.21.15.43:0/1845691135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a010e480 msgr2=0x7f81a019ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.844+0000 7f81a5762640 1 --2- 172.21.15.43:0/1845691135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a010e480 0x7f81a019ef80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f819800cab0 tx=0x7f819800cf70 comp rx=0 tx=0).stop 2024-09-11T23:22:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.844+0000 7f81a5762640 1 -- 172.21.15.43:0/1845691135 shutdown_connections 2024-09-11T23:22:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.844+0000 7f81a5762640 1 --2- 172.21.15.43:0/1845691135 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f81880a2e10 0x7f81880a52d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.844+0000 7f81a5762640 1 --2- 172.21.15.43:0/1845691135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a010e480 0x7f81a019ef80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.844+0000 7f81a5762640 1 --2- 172.21.15.43:0/1845691135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81a0101280 0x7f81a019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.844+0000 7f81a5762640 1 --2- 172.21.15.43:0/1845691135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a01008d0 0x7f81a019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.844+0000 7f81a5762640 1 -- 172.21.15.43:0/1845691135 >> 172.21.15.43:0/1845691135 conn(0x7f81a00fc7a0 msgr2=0x7f81a010f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.844+0000 7f81a5762640 1 -- 172.21.15.43:0/1845691135 shutdown_connections 2024-09-11T23:22:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:09.844+0000 7f81a5762640 1 -- 172.21.15.43:0/1845691135 wait complete. 2024-09-11T23:22:10.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:09 smithi090 ceph-mon[35130]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-11T23:22:10.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:09 smithi097 ceph-mon[35562]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-11T23:22:10.543 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:22:10.843 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:10.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:10 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1845691135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:11.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:10 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1845691135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:11.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:10 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1845691135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:11.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.336+0000 7f699ba2d640 1 -- 172.21.15.43:0/3183814775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6994106f10 msgr2=0x7f699410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.336+0000 7f699ba2d640 1 --2- 172.21.15.43:0/3183814775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6994106f10 0x7f699410ea90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f698c009ca0 tx=0x7f698c02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:22:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.336+0000 7f699ba2d640 1 -- 172.21.15.43:0/3183814775 shutdown_connections 2024-09-11T23:22:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.336+0000 7f699ba2d640 1 --2- 172.21.15.43:0/3183814775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6994106f10 0x7f699410ea90 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.336+0000 7f699ba2d640 1 --2- 172.21.15.43:0/3183814775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6994102790 0x7f69941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.336+0000 7f699ba2d640 1 --2- 172.21.15.43:0/3183814775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6994101de0 0x7f69941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.336+0000 7f699ba2d640 1 -- 172.21.15.43:0/3183814775 >> 172.21.15.43:0/3183814775 conn(0x7f69940fd8c0 msgr2=0x7f69940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.336+0000 7f699ba2d640 1 -- 172.21.15.43:0/3183814775 shutdown_connections 2024-09-11T23:22:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.336+0000 7f699ba2d640 1 -- 172.21.15.43:0/3183814775 wait complete. 2024-09-11T23:22:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.337+0000 7f699ba2d640 1 Processor -- start 2024-09-11T23:22:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.337+0000 7f699ba2d640 1 -- start start 2024-09-11T23:22:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.337+0000 7f699ba2d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6994101de0 0x7f69940744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.337+0000 7f699ba2d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6994102790 0x7f6994074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:11.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.337+0000 7f699ba2d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6994106f10 0x7f6994074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:11.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.337+0000 7f699ba2d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6994111f70 con 0x7f6994101de0 2024-09-11T23:22:11.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.337+0000 7f699ba2d640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6994111df0 con 0x7f6994102790 2024-09-11T23:22:11.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.337+0000 7f699ba2d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f69941120f0 con 0x7f6994106f10 2024-09-11T23:22:11.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.337+0000 7f69997c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6994101de0 0x7f69940744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:11.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.337+0000 7f69997c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6994101de0 0x7f69940744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36216/0 (socket says 172.21.15.43:36216) 2024-09-11T23:22:11.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.337+0000 7f69997c9640 1 -- 172.21.15.43:0/1540689241 learned_addr learned my addr 172.21.15.43:0/1540689241 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:11.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.337+0000 7f6998fc8640 1 --2- 172.21.15.43:0/1540689241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6994102790 0x7f6994074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:11.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.338+0000 7f6999fca640 1 --2- 172.21.15.43:0/1540689241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6994106f10 0x7f6994074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.338+0000 7f69997c9640 1 -- 172.21.15.43:0/1540689241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6994106f10 msgr2=0x7f6994074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.338+0000 7f69997c9640 1 --2- 172.21.15.43:0/1540689241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6994106f10 0x7f6994074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.338+0000 7f69997c9640 1 -- 172.21.15.43:0/1540689241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6994102790 msgr2=0x7f6994074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.338+0000 7f69997c9640 1 --2- 172.21.15.43:0/1540689241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6994102790 0x7f6994074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.338+0000 7f69997c9640 1 -- 172.21.15.43:0/1540689241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69941a68b0 con 0x7f6994101de0 2024-09-11T23:22:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.338+0000 7f69997c9640 1 --2- 172.21.15.43:0/1540689241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6994101de0 0x7f69940744f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f699000d950 tx=0x7f699000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.338+0000 7f698a7fc640 1 -- 172.21.15.43:0/1540689241 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990017070 con 0x7f6994101de0 2024-09-11T23:22:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.338+0000 7f698a7fc640 1 -- 172.21.15.43:0/1540689241 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6990009c30 con 0x7f6994101de0 2024-09-11T23:22:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.338+0000 7f699ba2d640 1 -- 172.21.15.43:0/1540689241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69941a6ba0 con 0x7f6994101de0 2024-09-11T23:22:11.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.338+0000 7f698a7fc640 1 -- 172.21.15.43:0/1540689241 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699000f6b0 con 0x7f6994101de0 2024-09-11T23:22:11.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.338+0000 7f699ba2d640 1 -- 172.21.15.43:0/1540689241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69941a7130 con 0x7f6994101de0 2024-09-11T23:22:11.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.339+0000 7f699ba2d640 1 -- 172.21.15.43:0/1540689241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69941021c0 con 0x7f6994101de0 2024-09-11T23:22:11.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.340+0000 7f698a7fc640 1 -- 172.21.15.43:0/1540689241 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f69900109d0 con 0x7f6994101de0 2024-09-11T23:22:11.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.341+0000 7f698a7fc640 1 --2- 172.21.15.43:0/1540689241 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f69800a2e90 0x7f69800a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:11.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.341+0000 7f6998fc8640 1 --2- 172.21.15.43:0/1540689241 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f69800a2e90 0x7f69800a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:11.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.342+0000 7f698a7fc640 1 -- 172.21.15.43:0/1540689241 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f69900cf290 con 0x7f6994101de0 2024-09-11T23:22:11.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.342+0000 7f6998fc8640 1 --2- 172.21.15.43:0/1540689241 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f69800a2e90 0x7f69800a5350 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6994075f80 tx=0x7f698400b390 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:11.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.346+0000 7f698a7fc640 1 -- 172.21.15.43:0/1540689241 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6990009de0 con 0x7f6994101de0 2024-09-11T23:22:11.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.524+0000 7f699ba2d640 1 -- 172.21.15.43:0/1540689241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6994109b60 con 0x7f6994101de0 2024-09-11T23:22:11.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.525+0000 7f698a7fc640 1 -- 172.21.15.43:0/1540689241 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+23048 (secure 0 0 0) 0x7f699009bc60 con 0x7f6994101de0 2024-09-11T23:22:11.525 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:22:11.526 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":85,"fsid":"25c0cd84-7093-11ef-bcea-c7b262605968","created":"2024-09-11T23:12:03.504406+0000","modified":"2024-09-11T23:21:50.205389+0000","last_up_change":"2024-09-11T23:19:07.797873+0000","last_in_change":"2024-09-11T23:18:44.595176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:15:17.058713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:19:56.221406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:19:58.705875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24076596-3909-4c87-85f1-cbf168070dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6803","nonce":43839997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6805","nonce":43839997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6809","nonce":43839997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6807","nonce":43839997}]},"public_addr":"172.21.15.43:6803/43839997","cluster_addr":"172.21.15.43:6805/43839997","heartbeat_back_addr":"172.21.15.43:6809/43839997","heartbeat_front_addr":"172.21.15.43:6807/43839997","state":["exists","up"]},{"osd":1,"uuid":"a0472a57-243c-489d-a247-972ee10a71f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6801","nonce":1825149246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6803","nonce":1825149246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6807","nonce":1825149246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6805","nonce":1825149246}]},"public_addr":"172.21.15.90:6801/1825149246","cluster_addr":"172.21.15.90:6803/1825149246","heartbeat_back_addr":"172.21.15.90:6807/1825149246","heartbeat_front_addr":"172.21.15.90:6805/1825149246","state":["exists","up"]},{"osd":2,"uuid":"40366536-0fcf-45a0-b8e3-49178401dc34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6801","nonce":1456846443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6803","nonce":1456846443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6807","nonce":1456846443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6805","nonce":1456846443}]},"public_addr":"172.21.15.97:6801/1456846443","cluster_addr":"172.21.15.97:6803/1456846443","heartbeat_back_addr":"172.21.15.97:6807/1456846443","heartbeat_front_addr":"172.21.15.97:6805/1456846443","state":["exists","up"]},{"osd":3,"uuid":"3062133e-000b-4421-9d53-ac434b9ed9b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6811","nonce":1962837804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6813","nonce":1962837804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6817","nonce":1962837804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6815","nonce":1962837804}]},"public_addr":"172.21.15.43:6811/1962837804","cluster_addr":"172.21.15.43:6813/1962837804","heartbeat_back_addr":"172.21.15.43:6817/1962837804","heartbeat_front_addr":"172.21.15.43:6815/1962837804","state":["exists","up"]},{"osd":4,"uuid":"2e73fa4b-f22c-4319-828d-87b91b9f19f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6809","nonce":181614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6811","nonce":181614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6815","nonce":181614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6813","nonce":181614173}]},"public_addr":"172.21.15.90:6809/181614173","cluster_addr":"172.21.15.90:6811/181614173","heartbeat_back_addr":"172.21.15.90:6815/181614173","heartbeat_front_addr":"172.21.15.90:6813/181614173","state":["exists","up"]},{"osd":5,"uuid":"1e1ad4f9-e724-440f-b515-3ec1ad97d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6809","nonce":1540039543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6811","nonce":1540039543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6815","nonce":1540039543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6813","nonce":1540039543}]},"public_addr":"172.21.15.97:6809/1540039543","cluster_addr":"172.21.15.97:6811/1540039543","heartbeat_back_addr":"172.21.15.97:6815/1540039543","heartbeat_front_addr":"172.21.15.97:6813/1540039543","state":["exists","up"]},{"osd":6,"uuid":"d34df8df-f40d-4159-8b29-5401b1b0e431","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6819","nonce":3785677637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6821","nonce":3785677637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6825","nonce":3785677637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6823","nonce":3785677637}]},"public_addr":"172.21.15.43:6819/3785677637","cluster_addr":"172.21.15.43:6821/3785677637","heartbeat_back_addr":"172.21.15.43:6825/3785677637","heartbeat_front_addr":"172.21.15.43:6823/3785677637","state":["exists","up"]},{"osd":7,"uuid":"b2e01078-528b-49fd-b000-9f584f784b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6817","nonce":236874962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6819","nonce":236874962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6823","nonce":236874962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6821","nonce":236874962}]},"public_addr":"172.21.15.90:6817/236874962","cluster_addr":"172.21.15.90:6819/236874962","heartbeat_back_addr":"172.21.15.90:6823/236874962","heartbeat_front_addr":"172.21.15.90:6821/236874962","state":["exists","up"]},{"osd":8,"uuid":"abef8f27-bf9b-4d63-bfbe-f967fc3d8b17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6817","nonce":1707327301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6819","nonce":1707327301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6823","nonce":1707327301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6821","nonce":1707327301}]},"public_addr":"172.21.15.97:6817/1707327301","cluster_addr":"172.21.15.97:6819/1707327301","heartbeat_back_addr":"172.21.15.97:6823/1707327301","heartbeat_front_addr":"172.21.15.97:6821/1707327301","state":["exists","up"]},{"osd":9,"uuid":"190ab8b2-a835-47e6-9b39-16a58331c58a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6827","nonce":3170306011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6829","nonce":3170306011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6833","nonce":3170306011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6831","nonce":3170306011}]},"public_addr":"172.21.15.43:6827/3170306011","cluster_addr":"172.21.15.43:6829/3170306011","heartbeat_back_addr":"172.21.15.43:6833/3170306011","heartbeat_front_addr":"172.21.15.43:6831/3170306011","state":["exists","up"]},{"osd":10,"uuid":"b32ab5fe-466f-4c8e-8696-cd9f820a3263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6825","nonce":59487041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6827","nonce":59487041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6831","nonce":59487041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6829","nonce":59487041}]},"public_addr":"172.21.15.90:6825/59487041","cluster_addr":"172.21.15.90:6827/59487041","heartbeat_back_addr":"172.21.15.90:6831/59487041","heartbeat_front_addr":"172.21.15.90:6829/59487041","state":["exists","up"]},{"osd":11,"uuid":"d0faa589-21c8-4a9f-8a73-813aafe6d33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6825","nonce":194699315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6827","nonce":194699315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6831","nonce":194699315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6829","nonce":194699315}]},"public_addr":"172.21.15.97:6825/194699315","cluster_addr":"172.21.15.97:6827/194699315","heartbeat_back_addr":"172.21.15.97:6831/194699315","heartbeat_front_addr":"172.21.15.97:6829/194699315","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:23.381247+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:48.843190+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:13.538979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:40.136983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:05.513790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:31.247090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:56.635221+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:22.667327+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:47.910422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:14.353914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:40.584503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:19:06.476766+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/2262294782":"2024-09-12T23:21:50.205362+0000","172.21.15.43:0/2398981254":"2024-09-12T23:21:50.205362+0000","172.21.15.43:0/481605249":"2024-09-12T23:21:50.205362+0000","172.21.15.43:0/1375379108":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1512201905":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/206044413":"2024-09-12T23:12:35.756598+0000","172.21.15.43:6800/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1579053924":"2024-09-12T23:21:50.205362+0000","172.21.15.43:6801/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:6800/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/2370495020":"2024-09-12T23:21:50.205362+0000","172.21.15.43:0/4006124758":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/76575371":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/120495444":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6800/2370495020":"2024-09-12T23:21:50.205362+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:22:11.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.528+0000 7f699ba2d640 1 -- 172.21.15.43:0/1540689241 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f69800a2e90 msgr2=0x7f69800a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:11.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.528+0000 7f699ba2d640 1 --2- 172.21.15.43:0/1540689241 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f69800a2e90 0x7f69800a5350 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6994075f80 tx=0x7f698400b390 comp rx=0 tx=0).stop 2024-09-11T23:22:11.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.528+0000 7f699ba2d640 1 -- 172.21.15.43:0/1540689241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6994101de0 msgr2=0x7f69940744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:11.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.528+0000 7f699ba2d640 1 --2- 172.21.15.43:0/1540689241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6994101de0 0x7f69940744f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f699000d950 tx=0x7f699000de10 comp rx=0 tx=0).stop 2024-09-11T23:22:11.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.528+0000 7f699ba2d640 1 -- 172.21.15.43:0/1540689241 shutdown_connections 2024-09-11T23:22:11.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.528+0000 7f699ba2d640 1 --2- 172.21.15.43:0/1540689241 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f69800a2e90 0x7f69800a5350 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:11.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.528+0000 7f699ba2d640 1 --2- 172.21.15.43:0/1540689241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6994106f10 0x7f6994074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:11.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.528+0000 7f699ba2d640 1 --2- 172.21.15.43:0/1540689241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6994102790 0x7f6994074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:11.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.528+0000 7f699ba2d640 1 --2- 172.21.15.43:0/1540689241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6994101de0 0x7f69940744f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:11.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.528+0000 7f699ba2d640 1 -- 172.21.15.43:0/1540689241 >> 172.21.15.43:0/1540689241 conn(0x7f69940fd8c0 msgr2=0x7f69940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:11.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.528+0000 7f699ba2d640 1 -- 172.21.15.43:0/1540689241 shutdown_connections 2024-09-11T23:22:11.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:11.528+0000 7f699ba2d640 1 -- 172.21.15.43:0/1540689241 wait complete. 2024-09-11T23:22:11.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:11 smithi043 ceph-mon[30950]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-09-11T23:22:11.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:11 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1540689241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:22:12.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:11 smithi090 ceph-mon[35130]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-09-11T23:22:12.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:11 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1540689241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:22:12.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:11 smithi097 ceph-mon[35562]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-09-11T23:22:12.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:11 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1540689241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:22:12.219 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-11T23:22:12.479 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:13.635 INFO:teuthology.orchestra.run.smithi043.stdout:{"run_dir":"/var/run/ceph"} 2024-09-11T23:22:13.635 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-11T23:22:13.636 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 481 2024-09-11T23:22:13.636 DEBUG:tasks.quiescer:Yielding 2024-09-11T23:22:13.636 INFO:teuthology.run_tasks:Running task workunit... 2024-09-11T23:22:13.649 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-11T23:22:13.650 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-11T23:22:13.650 INFO:tasks.workunit:timeout=3h 2024-09-11T23:22:13.650 INFO:tasks.workunit:cleanup=True 2024-09-11T23:22:13.650 DEBUG:teuthology.orchestra.run.smithi043:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-11T23:22:13.651 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:22:13.669 INFO:teuthology.orchestra.run.smithi043.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-11T23:22:13.669 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-11T23:22:13.669 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-11T23:22:13.669 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T23:22:13.669 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-11T23:22:13.669 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-11 23:20:19.714805396 +0000 2024-09-11T23:22:13.669 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-11 23:20:19.714805396 +0000 2024-09-11T23:22:13.669 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-11 23:21:48.429801872 +0000 2024-09-11T23:22:13.669 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-11 23:20:19.714805396 +0000 2024-09-11T23:22:13.669 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-11T23:22:13.670 DEBUG:teuthology.orchestra.run.smithi043:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-11T23:22:13.745 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-11T23:22:13.802 INFO:tasks.workunit.client.0.smithi043.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-11T23:22:13.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:13 smithi043 ceph-mon[30950]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-09-11T23:22:13.921 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:14.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:13 smithi097 ceph-mon[35562]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-09-11T23:22:14.099 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:13 smithi090 ceph-mon[35130]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-09-11T23:22:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.441+0000 7f78f1423640 1 -- 172.21.15.43:0/3378590910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78ec101440 msgr2=0x7f78ec10df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.441+0000 7f78f1423640 1 --2- 172.21.15.43:0/3378590910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78ec101440 0x7f78ec10df30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f78d4009900 tx=0x7f78d402a410 comp rx=0 tx=0).stop 2024-09-11T23:22:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.441+0000 7f78f1423640 1 -- 172.21.15.43:0/3378590910 shutdown_connections 2024-09-11T23:22:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.441+0000 7f78f1423640 1 --2- 172.21.15.43:0/3378590910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78ec10e500 0x7f78ec110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.441+0000 7f78f1423640 1 --2- 172.21.15.43:0/3378590910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78ec101440 0x7f78ec10df30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.441+0000 7f78f1423640 1 --2- 172.21.15.43:0/3378590910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78ec100b20 0x7f78ec100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.441+0000 7f78f1423640 1 -- 172.21.15.43:0/3378590910 >> 172.21.15.43:0/3378590910 conn(0x7f78ec0fc7d0 msgr2=0x7f78ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.441+0000 7f78f1423640 1 -- 172.21.15.43:0/3378590910 shutdown_connections 2024-09-11T23:22:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.441+0000 7f78f1423640 1 -- 172.21.15.43:0/3378590910 wait complete. 2024-09-11T23:22:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.441+0000 7f78f1423640 1 Processor -- start 2024-09-11T23:22:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.441+0000 7f78f1423640 1 -- start start 2024-09-11T23:22:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.442+0000 7f78f1423640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78ec100b20 0x7f78ec1a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.442+0000 7f78f1423640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78ec101440 0x7f78ec1a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.442+0000 7f78f1423640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78ec10e500 0x7f78ec19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.442+0000 7f78f1423640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec114270 con 0x7f78ec101440 2024-09-11T23:22:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.442+0000 7f78f1423640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec1140f0 con 0x7f78ec100b20 2024-09-11T23:22:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.442+0000 7f78f1423640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec1143f0 con 0x7f78ec10e500 2024-09-11T23:22:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.442+0000 7f78ea7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78ec101440 0x7f78ec1a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.442+0000 7f78eaffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78ec100b20 0x7f78ec1a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.442+0000 7f78ea7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78ec101440 0x7f78ec1a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36236/0 (socket says 172.21.15.43:36236) 2024-09-11T23:22:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.442+0000 7f78eb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78ec10e500 0x7f78ec19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.442+0000 7f78ea7fc640 1 -- 172.21.15.43:0/152295099 learned_addr learned my addr 172.21.15.43:0/152295099 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.442+0000 7f78eaffd640 1 -- 172.21.15.43:0/152295099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78ec10e500 msgr2=0x7f78ec19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.442+0000 7f78eaffd640 1 --2- 172.21.15.43:0/152295099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78ec10e500 0x7f78ec19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.442+0000 7f78eaffd640 1 -- 172.21.15.43:0/152295099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78ec101440 msgr2=0x7f78ec1a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.442+0000 7f78eaffd640 1 --2- 172.21.15.43:0/152295099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78ec101440 0x7f78ec1a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.442+0000 7f78eaffd640 1 -- 172.21.15.43:0/152295099 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ec19ad30 con 0x7f78ec100b20 2024-09-11T23:22:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.442+0000 7f78eb7fe640 1 --2- 172.21.15.43:0/152295099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78ec10e500 0x7f78ec19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.442+0000 7f78eaffd640 1 --2- 172.21.15.43:0/152295099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78ec100b20 0x7f78ec1a0440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00e9f0 tx=0x7f78dc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.443+0000 7f78e3fff640 1 -- 172.21.15.43:0/152295099 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc00ccc0 con 0x7f78ec100b20 2024-09-11T23:22:14.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.443+0000 7f78f1423640 1 -- 172.21.15.43:0/152295099 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78ec19b020 con 0x7f78ec100b20 2024-09-11T23:22:14.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.443+0000 7f78e3fff640 1 -- 172.21.15.43:0/152295099 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78dc005020 con 0x7f78ec100b20 2024-09-11T23:22:14.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.443+0000 7f78e3fff640 1 -- 172.21.15.43:0/152295099 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc0049e0 con 0x7f78ec100b20 2024-09-11T23:22:14.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.443+0000 7f78f1423640 1 -- 172.21.15.43:0/152295099 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78ec1a7310 con 0x7f78ec100b20 2024-09-11T23:22:14.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.444+0000 7f78f1423640 1 -- 172.21.15.43:0/152295099 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78ec100f00 con 0x7f78ec100b20 2024-09-11T23:22:14.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.445+0000 7f78e3fff640 1 -- 172.21.15.43:0/152295099 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f78dc005560 con 0x7f78ec100b20 2024-09-11T23:22:14.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.446+0000 7f78e3fff640 1 --2- 172.21.15.43:0/152295099 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f78d80a2e10 0x7f78d80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.446+0000 7f78e3fff640 1 -- 172.21.15.43:0/152295099 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f78dc0cf670 con 0x7f78ec100b20 2024-09-11T23:22:14.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.446+0000 7f78ea7fc640 1 --2- 172.21.15.43:0/152295099 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f78d80a2e10 0x7f78d80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.446+0000 7f78ea7fc640 1 --2- 172.21.15.43:0/152295099 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f78d80a2e10 0x7f78d80a52d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f78ec10ee00 tx=0x7f78d40058d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:14.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.450+0000 7f78e3fff640 1 -- 172.21.15.43:0/152295099 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f78dc0051d0 con 0x7f78ec100b20 2024-09-11T23:22:14.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.662+0000 7f78f1423640 1 -- 172.21.15.43:0/152295099 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78ec109600 con 0x7f78ec100b20 2024-09-11T23:22:14.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.663+0000 7f78e3fff640 1 -- 172.21.15.43:0/152295099 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11896 (secure 0 0 0) 0x7f78dc09c040 con 0x7f78ec100b20 2024-09-11T23:22:14.663 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:22:14.663 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"btime":"2024-09-11T23:21:49:167551+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:21:49.117342+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:22:14.663 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 24 2024-09-11T23:22:14.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.665+0000 7f78f1423640 1 -- 172.21.15.43:0/152295099 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f78d80a2e10 msgr2=0x7f78d80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:14.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.665+0000 7f78f1423640 1 --2- 172.21.15.43:0/152295099 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f78d80a2e10 0x7f78d80a52d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f78ec10ee00 tx=0x7f78d40058d0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.665+0000 7f78f1423640 1 -- 172.21.15.43:0/152295099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78ec100b20 msgr2=0x7f78ec1a0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:14.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.665+0000 7f78f1423640 1 --2- 172.21.15.43:0/152295099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78ec100b20 0x7f78ec1a0440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00e9f0 tx=0x7f78dc00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.665+0000 7f78f1423640 1 -- 172.21.15.43:0/152295099 shutdown_connections 2024-09-11T23:22:14.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.665+0000 7f78f1423640 1 --2- 172.21.15.43:0/152295099 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f78d80a2e10 0x7f78d80a52d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.665+0000 7f78f1423640 1 --2- 172.21.15.43:0/152295099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78ec10e500 0x7f78ec19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.665+0000 7f78f1423640 1 --2- 172.21.15.43:0/152295099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78ec101440 0x7f78ec1a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.665+0000 7f78f1423640 1 --2- 172.21.15.43:0/152295099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78ec100b20 0x7f78ec1a0440 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.665+0000 7f78f1423640 1 -- 172.21.15.43:0/152295099 >> 172.21.15.43:0/152295099 conn(0x7f78ec0fc7d0 msgr2=0x7f78ec10f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:14.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.665+0000 7f78f1423640 1 -- 172.21.15.43:0/152295099 shutdown_connections 2024-09-11T23:22:14.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:14.665+0000 7f78f1423640 1 -- 172.21.15.43:0/152295099 wait complete. 2024-09-11T23:22:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.796+0000 7f3059be4640 1 -- 172.21.15.43:0/255904718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30541009d0 msgr2=0x7f3054100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.796+0000 7f3059be4640 1 --2- 172.21.15.43:0/255904718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30541009d0 0x7f3054100e30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3048009860 tx=0x7f304802a340 comp rx=0 tx=0).stop 2024-09-11T23:22:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.796+0000 7f3059be4640 1 -- 172.21.15.43:0/255904718 shutdown_connections 2024-09-11T23:22:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.796+0000 7f3059be4640 1 --2- 172.21.15.43:0/255904718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3054101370 0x7f305410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.796+0000 7f3059be4640 1 --2- 172.21.15.43:0/255904718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30541009d0 0x7f3054100e30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.796+0000 7f3059be4640 1 --2- 172.21.15.43:0/255904718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30541069e0 0x7f3054106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.796+0000 7f3059be4640 1 -- 172.21.15.43:0/255904718 >> 172.21.15.43:0/255904718 conn(0x7f30540fc6b0 msgr2=0x7f30540fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.796+0000 7f3059be4640 1 -- 172.21.15.43:0/255904718 shutdown_connections 2024-09-11T23:22:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.796+0000 7f3059be4640 1 -- 172.21.15.43:0/255904718 wait complete. 2024-09-11T23:22:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.797+0000 7f3059be4640 1 Processor -- start 2024-09-11T23:22:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.797+0000 7f3059be4640 1 -- start start 2024-09-11T23:22:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.797+0000 7f3059be4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30541009d0 0x7f30541983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.797+0000 7f3059be4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3054101370 0x7f3054198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.797+0000 7f3059be4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30541069e0 0x7f305419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.797+0000 7f3059be4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3054111f10 con 0x7f30541069e0 2024-09-11T23:22:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.797+0000 7f3059be4640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3054111d90 con 0x7f30541009d0 2024-09-11T23:22:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.797+0000 7f3059be4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3054112090 con 0x7f3054101370 2024-09-11T23:22:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.797+0000 7f3053fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30541069e0 0x7f305419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.797+0000 7f3053fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30541069e0 0x7f305419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36246/0 (socket says 172.21.15.43:36246) 2024-09-11T23:22:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.797+0000 7f3053fff640 1 -- 172.21.15.43:0/300677349 learned_addr learned my addr 172.21.15.43:0/300677349 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.797+0000 7f30537fe640 1 --2- 172.21.15.43:0/300677349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30541009d0 0x7f30541983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.797+0000 7f3052ffd640 1 --2- 172.21.15.43:0/300677349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3054101370 0x7f3054198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.798+0000 7f3053fff640 1 -- 172.21.15.43:0/300677349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3054101370 msgr2=0x7f3054198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.798+0000 7f3053fff640 1 --2- 172.21.15.43:0/300677349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3054101370 0x7f3054198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.798+0000 7f3053fff640 1 -- 172.21.15.43:0/300677349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30541009d0 msgr2=0x7f30541983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.798+0000 7f3053fff640 1 --2- 172.21.15.43:0/300677349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30541009d0 0x7f30541983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.798+0000 7f3053fff640 1 -- 172.21.15.43:0/300677349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305419d3d0 con 0x7f30541069e0 2024-09-11T23:22:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.798+0000 7f30537fe640 1 --2- 172.21.15.43:0/300677349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30541009d0 0x7f30541983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.798+0000 7f3053fff640 1 --2- 172.21.15.43:0/300677349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30541069e0 0x7f305419cca0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f304c00eb10 tx=0x7f304c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.798+0000 7f3050ff9640 1 -- 172.21.15.43:0/300677349 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c00ce20 con 0x7f30541069e0 2024-09-11T23:22:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.798+0000 7f3050ff9640 1 -- 172.21.15.43:0/300677349 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f304c005020 con 0x7f30541069e0 2024-09-11T23:22:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.798+0000 7f3059be4640 1 -- 172.21.15.43:0/300677349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305419d6c0 con 0x7f30541069e0 2024-09-11T23:22:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.798+0000 7f3050ff9640 1 -- 172.21.15.43:0/300677349 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c010690 con 0x7f30541069e0 2024-09-11T23:22:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.798+0000 7f3059be4640 1 -- 172.21.15.43:0/300677349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30541a4f50 con 0x7f30541069e0 2024-09-11T23:22:14.808 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.800+0000 7f3050ff9640 1 -- 172.21.15.43:0/300677349 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f304c005540 con 0x7f30541069e0 2024-09-11T23:22:14.808 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.801+0000 7f305ca56740 1 Processor -- start 2024-09-11T23:22:14.808 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.801+0000 7f305ca56740 1 -- start start 2024-09-11T23:22:14.808 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.801+0000 7f305ca56740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fc5ad9b9f0 0x55fc5ad9bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.801+0000 7f305ca56740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5ad9c310 0x55fc5adaab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.801+0000 7f305ca56740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5adab0c0 0x55fc5adad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.801+0000 7f305ca56740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fc5ad002f0 con 0x55fc5adab0c0 2024-09-11T23:22:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.801+0000 7f305ca56740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fc5aca8270 con 0x55fc5ad9b9f0 2024-09-11T23:22:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.801+0000 7f305ca56740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55fc5acfc2b0 con 0x55fc5ad9c310 2024-09-11T23:22:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.801+0000 7f30537fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5ad9c310 0x55fc5adaab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.801+0000 7f3052ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5adab0c0 0x55fc5adad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.801+0000 7f30537fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5ad9c310 0x55fc5adaab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:53584/0 (socket says 172.21.15.43:53584) 2024-09-11T23:22:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.801+0000 7f30537fe640 1 -- 172.21.15.43:0/1516488031 learned_addr learned my addr 172.21.15.43:0/1516488031 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:14.809 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.801+0000 7f3052ffd640 1 --2- 172.21.15.43:0/1516488031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fc5ad9b9f0 0x55fc5ad9bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.801+0000 7f3050ff9640 1 --2- 172.21.15.43:0/300677349 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f303c0a2e10 0x7f303c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.801+0000 7f30537fe640 1 -- 172.21.15.43:0/1516488031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fc5ad9b9f0 msgr2=0x55fc5ad9bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.801+0000 7f30537fe640 1 --2- 172.21.15.43:0/1516488031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fc5ad9b9f0 0x55fc5ad9bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.801+0000 7f30537fe640 1 -- 172.21.15.43:0/1516488031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5adab0c0 msgr2=0x55fc5adad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.801+0000 7f30537fe640 1 --2- 172.21.15.43:0/1516488031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5adab0c0 0x55fc5adad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.801+0000 7f30537fe640 1 -- 172.21.15.43:0/1516488031 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc5aca7340 con 0x55fc5ad9c310 2024-09-11T23:22:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.801+0000 7f3053fff640 1 --2- 172.21.15.43:0/300677349 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f303c0a2e10 0x7f303c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.802+0000 7f3050ff9640 1 -- 172.21.15.43:0/300677349 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f304c0cfde0 con 0x7f30541069e0 2024-09-11T23:22:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.802+0000 7f30537fe640 1 --2- 172.21.15.43:0/1516488031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5ad9c310 0x55fc5adaab80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f30440114f0 tx=0x7f30440119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5821edab566ed713 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.802+0000 7f3039ffb640 1 -- 172.21.15.43:0/1516488031 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044011310 con 0x55fc5ad9c310 2024-09-11T23:22:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.802+0000 7f3039ffb640 1 -- 172.21.15.43:0/1516488031 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30440035f0 con 0x55fc5ad9c310 2024-09-11T23:22:14.810 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.802+0000 7f3039ffb640 1 -- 172.21.15.43:0/1516488031 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304400f7c0 con 0x55fc5ad9c310 2024-09-11T23:22:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.802+0000 7f3053fff640 1 --2- 172.21.15.43:0/300677349 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f303c0a2e10 0x7f303c0a52d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f304c0cd410 tx=0x7f304c06f120 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.802+0000 7f305ca56740 1 -- 172.21.15.43:0/1516488031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5ad9c310 msgr2=0x55fc5adaab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.802+0000 7f305ca56740 1 --2- 172.21.15.43:0/1516488031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5ad9c310 0x55fc5adaab80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f30440114f0 tx=0x7f30440119b0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.802+0000 7f305ca56740 1 -- 172.21.15.43:0/1516488031 shutdown_connections 2024-09-11T23:22:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.802+0000 7f305ca56740 1 --2- 172.21.15.43:0/1516488031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5adab0c0 0x55fc5adad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.802+0000 7f305ca56740 1 --2- 172.21.15.43:0/1516488031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5ad9c310 0x55fc5adaab80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.802+0000 7f305ca56740 1 --2- 172.21.15.43:0/1516488031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fc5ad9b9f0 0x55fc5ad9bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.802+0000 7f305ca56740 1 -- 172.21.15.43:0/1516488031 >> 172.21.15.43:0/1516488031 conn(0x55fc5ad98060 msgr2=0x55fc5ad98440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.803+0000 7f305ca56740 1 -- 172.21.15.43:0/1516488031 shutdown_connections 2024-09-11T23:22:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.803+0000 7f305ca56740 1 -- 172.21.15.43:0/1516488031 wait complete. 2024-09-11T23:22:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.803+0000 7f305ca56740 1 -- start start 2024-09-11T23:22:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.803+0000 7f305ca56740 1 Processor -- start 2024-09-11T23:22:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.803+0000 7f305ca56740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5ad9b9f0 0x55fc5ada7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.803+0000 7f305ca56740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5ad9c310 0x55fc5ada8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.803+0000 7f305ca56740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fc5adab0c0 0x55fc5ada8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.803+0000 7f305ca56740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fc5adadd20 con 0x55fc5ad9b9f0 2024-09-11T23:22:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.803+0000 7f305ca56740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fc5ac89290 con 0x55fc5adab0c0 2024-09-11T23:22:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.803+0000 7f305ca56740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55fc5adadea0 con 0x55fc5ad9c310 2024-09-11T23:22:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.803+0000 7f305ca56740 10 client.0 _ll_register_callbacks cb 0x55fc5acba140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.803+0000 7f3052ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5ad9b9f0 0x55fc5ada7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.803+0000 7f3052ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5ad9b9f0 0x55fc5ada7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36274/0 (socket says 172.21.15.43:36274) 2024-09-11T23:22:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.803+0000 7f3052ffd640 1 -- 172.21.15.43:0/1482176104 learned_addr learned my addr 172.21.15.43:0/1482176104 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.804+0000 7f3052ffd640 1 --2- 172.21.15.43:0/1482176104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fc5adab0c0 0x55fc5ada8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.804+0000 7f30537fe640 1 --2- 172.21.15.43:0/1482176104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5ad9c310 0x55fc5ada8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.804+0000 7f3052ffd640 1 -- 172.21.15.43:0/1482176104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5ad9c310 msgr2=0x55fc5ada8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.804+0000 7f3052ffd640 1 --2- 172.21.15.43:0/1482176104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5ad9c310 0x55fc5ada8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.804+0000 7f3052ffd640 1 -- 172.21.15.43:0/1482176104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fc5adab0c0 msgr2=0x55fc5ada8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.804+0000 7f3052ffd640 1 --2- 172.21.15.43:0/1482176104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fc5adab0c0 0x55fc5ada8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.804+0000 7f3052ffd640 1 -- 172.21.15.43:0/1482176104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc5aa13210 con 0x55fc5ad9b9f0 2024-09-11T23:22:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.804+0000 7f3052ffd640 1 --2- 172.21.15.43:0/1482176104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5ad9b9f0 0x55fc5ada7fd0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f304802a860 tx=0x7f304802a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac59b78cbadeb217 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.804+0000 7f3032ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.804+0000 7f3032ffd640 1 -- 172.21.15.43:0/1482176104 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048037040 con 0x55fc5ad9b9f0 2024-09-11T23:22:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.804+0000 7f305ca56740 1 -- 172.21.15.43:0/1482176104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30440035f0 con 0x55fc5ad9b9f0 2024-09-11T23:22:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.804+0000 7f3032ffd640 1 -- 172.21.15.43:0/1482176104 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f30480035f0 con 0x55fc5ad9b9f0 2024-09-11T23:22:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.804+0000 7f3032ffd640 1 -- 172.21.15.43:0/1482176104 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f30480037f0 con 0x55fc5ad9b9f0 2024-09-11T23:22:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.804+0000 7f305ca56740 10 client.15381 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.804+0000 7f305ca56740 1 -- 172.21.15.43:0/1482176104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fc5aadf620 con 0x55fc5ad9b9f0 2024-09-11T23:22:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.805+0000 7f3032ffd640 1 -- 172.21.15.43:0/1482176104 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30440035f0 con 0x55fc5ad9b9f0 2024-09-11T23:22:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.805+0000 7f3032ffd640 1 -- 172.21.15.43:0/1482176104 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f304804c060 con 0x55fc5ad9b9f0 2024-09-11T23:22:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.805+0000 7f305ca56740 10 client.15381 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.805+0000 7f305ca56740 10 client.15381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.805+0000 7f305ca56740 20 client.15381 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.805+0000 7f305ca56740 1 --2- 172.21.15.43:0/1482176104 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55fc5adbb0c0 0x55fc5adbd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.805+0000 7f305ca56740 4 client.15381 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:22:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.805+0000 7f305ca56740 1 -- 172.21.15.43:0/1482176104 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc5ad0da20 con 0x55fc5adbb0c0 2024-09-11T23:22:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.805+0000 7f30537fe640 1 --2- 172.21.15.43:0/1482176104 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55fc5adbb0c0 0x55fc5adbd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.805+0000 7f30537fe640 1 --2- 172.21.15.43:0/1482176104 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55fc5adbb0c0 0x55fc5adbd4b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c40ceae447f1eb7e server_cookie=91ca3bd821173586 in_seq=0 out_seq=0 2024-09-11T23:22:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.805+0000 7f3032ffd640 10 client.15381 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.806+0000 7f3032ffd640 1 -- 172.21.15.43:0/1482176104 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc5ad0da20 con 0x55fc5adbb0c0 2024-09-11T23:22:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.806+0000 7f3032ffd640 10 client.15381 handle_command_reply: tid=0 2024-09-11T23:22:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.806+0000 7f305ca56740 1 client.15381 shutdown 2024-09-11T23:22:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.806+0000 7f305ca56740 1 -- 172.21.15.43:0/1482176104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5ad9b9f0 msgr2=0x55fc5ada7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.806+0000 7f305ca56740 1 --2- 172.21.15.43:0/1482176104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5ad9b9f0 0x55fc5ada7fd0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f304802a860 tx=0x7f304802a340 comp rx=0 tx=0).stop 2024-09-11T23:22:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.806+0000 7f305ca56740 1 -- 172.21.15.43:0/1482176104 shutdown_connections 2024-09-11T23:22:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.806+0000 7f305ca56740 1 --2- 172.21.15.43:0/1482176104 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55fc5adbb0c0 0x55fc5adbd4b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.806+0000 7f305ca56740 1 --2- 172.21.15.43:0/1482176104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fc5adab0c0 0x55fc5ada8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.806+0000 7f3032ffd640 0 client.15381 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.806+0000 7f305ca56740 1 --2- 172.21.15.43:0/1482176104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5ad9c310 0x55fc5ada8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.806+0000 7f305ca56740 1 --2- 172.21.15.43:0/1482176104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5ad9b9f0 0x55fc5ada7fd0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.806+0000 7f305ca56740 1 -- 172.21.15.43:0/1482176104 >> 172.21.15.43:0/1482176104 conn(0x55fc5ad98060 msgr2=0x55fc5ad9dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.806+0000 7f305ca56740 1 -- 172.21.15.43:0/1482176104 shutdown_connections 2024-09-11T23:22:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.806+0000 7f305ca56740 1 -- 172.21.15.43:0/1482176104 wait complete. 2024-09-11T23:22:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.806+0000 7f305ca56740 20 client.15381 trim_cache size 0 max 16384 2024-09-11T23:22:14.825 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.821+0000 7f305ca56740 1 Processor -- start 2024-09-11T23:22:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.821+0000 7f305ca56740 1 -- start start 2024-09-11T23:22:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.821+0000 7f305ca56740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5ad9b9f0 0x55fc5adc2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.821+0000 7f305ca56740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fc5ad9c310 0x55fc5adc32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.821+0000 7f305ca56740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5adab0c0 0x55fc5ada5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.821+0000 7f305ca56740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f30541ae030 con 0x55fc5adab0c0 2024-09-11T23:22:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.821+0000 7f305ca56740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f30541ab900 con 0x55fc5ad9c310 2024-09-11T23:22:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.821+0000 7f305ca56740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f30541a9750 con 0x55fc5ad9b9f0 2024-09-11T23:22:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.821+0000 7f3052ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5adab0c0 0x55fc5ada5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.821+0000 7f3052ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5adab0c0 0x55fc5ada5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36288/0 (socket says 172.21.15.43:36288) 2024-09-11T23:22:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.821+0000 7f3052ffd640 1 -- 172.21.15.43:0/2745813933 learned_addr learned my addr 172.21.15.43:0/2745813933 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.821+0000 7f30537fe640 1 --2- 172.21.15.43:0/2745813933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fc5ad9c310 0x55fc5adc32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.821+0000 7f3052ffd640 1 --2- 172.21.15.43:0/2745813933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5ad9b9f0 0x55fc5adc2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.821+0000 7f3052ffd640 1 -- 172.21.15.43:0/2745813933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5ad9b9f0 msgr2=0x55fc5adc2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.821+0000 7f3052ffd640 1 --2- 172.21.15.43:0/2745813933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5ad9b9f0 0x55fc5adc2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.821+0000 7f3052ffd640 1 -- 172.21.15.43:0/2745813933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fc5ad9c310 msgr2=0x55fc5adc32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.821+0000 7f3052ffd640 1 --2- 172.21.15.43:0/2745813933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fc5ad9c310 0x55fc5adc32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.821+0000 7f3052ffd640 1 -- 172.21.15.43:0/2745813933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc5ad9c140 con 0x55fc5adab0c0 2024-09-11T23:22:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.821+0000 7f3052ffd640 1 --2- 172.21.15.43:0/2745813933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5adab0c0 0x55fc5ada5210 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3048030970 tx=0x7f304802c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926bcb987c5a4551 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.822+0000 7f30397fa640 1 -- 172.21.15.43:0/2745813933 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048045070 con 0x55fc5adab0c0 2024-09-11T23:22:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.822+0000 7f30397fa640 1 -- 172.21.15.43:0/2745813933 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3048005850 con 0x55fc5adab0c0 2024-09-11T23:22:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.822+0000 7f30397fa640 1 -- 172.21.15.43:0/2745813933 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048037020 con 0x55fc5adab0c0 2024-09-11T23:22:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.822+0000 7f305ca56740 1 -- 172.21.15.43:0/2745813933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5adab0c0 msgr2=0x55fc5ada5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.822+0000 7f305ca56740 1 --2- 172.21.15.43:0/2745813933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5adab0c0 0x55fc5ada5210 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3048030970 tx=0x7f304802c040 comp rx=0 tx=0).stop 2024-09-11T23:22:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.822+0000 7f305ca56740 1 -- 172.21.15.43:0/2745813933 shutdown_connections 2024-09-11T23:22:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.822+0000 7f305ca56740 1 --2- 172.21.15.43:0/2745813933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5adab0c0 0x55fc5ada5210 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.822+0000 7f305ca56740 1 --2- 172.21.15.43:0/2745813933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fc5ad9c310 0x55fc5adc32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.822+0000 7f305ca56740 1 --2- 172.21.15.43:0/2745813933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5ad9b9f0 0x55fc5adc2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.822+0000 7f305ca56740 1 -- 172.21.15.43:0/2745813933 >> 172.21.15.43:0/2745813933 conn(0x55fc5ad98060 msgr2=0x55fc5ad971c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.822+0000 7f305ca56740 1 -- 172.21.15.43:0/2745813933 shutdown_connections 2024-09-11T23:22:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.822+0000 7f305ca56740 1 -- 172.21.15.43:0/2745813933 wait complete. 2024-09-11T23:22:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.823+0000 7f305ca56740 1 -- start start 2024-09-11T23:22:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.823+0000 7f305ca56740 1 Processor -- start 2024-09-11T23:22:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.823+0000 7f305ca56740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5ad9b9f0 0x55fc5adaee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.823+0000 7f305ca56740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fc5ad9c310 0x55fc5adaf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.823+0000 7f305ca56740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5adab0c0 0x55fc5adaf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.823+0000 7f305ca56740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f30541a9dc0 con 0x55fc5ad9b9f0 2024-09-11T23:22:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.823+0000 7f305ca56740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f30541ab230 con 0x55fc5ad9c310 2024-09-11T23:22:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.823+0000 7f305ca56740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f30541aa490 con 0x55fc5adab0c0 2024-09-11T23:22:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.823+0000 7f305ca56740 10 client.0 _ll_register_callbacks cb 0x7f30480096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.823+0000 7f3052ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5ad9b9f0 0x55fc5adaee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.823+0000 7f3052ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5ad9b9f0 0x55fc5adaee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36292/0 (socket says 172.21.15.43:36292) 2024-09-11T23:22:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.823+0000 7f3052ffd640 1 -- 172.21.15.43:0/950027933 learned_addr learned my addr 172.21.15.43:0/950027933 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.823+0000 7f30537fe640 1 --2- 172.21.15.43:0/950027933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fc5ad9c310 0x55fc5adaf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.823+0000 7f3052ffd640 1 --2- 172.21.15.43:0/950027933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5adab0c0 0x55fc5adaf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.824+0000 7f3052ffd640 1 -- 172.21.15.43:0/950027933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5adab0c0 msgr2=0x55fc5adaf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.824+0000 7f3052ffd640 1 --2- 172.21.15.43:0/950027933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5adab0c0 0x55fc5adaf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.824+0000 7f3052ffd640 1 -- 172.21.15.43:0/950027933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fc5ad9c310 msgr2=0x55fc5adaf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.824+0000 7f3052ffd640 1 --2- 172.21.15.43:0/950027933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fc5ad9c310 0x55fc5adaf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.824+0000 7f3052ffd640 1 -- 172.21.15.43:0/950027933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc5ae24d40 con 0x55fc5ad9b9f0 2024-09-11T23:22:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.824+0000 7f3052ffd640 1 --2- 172.21.15.43:0/950027933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5ad9b9f0 0x55fc5adaee10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3048005e00 tx=0x7f3048005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55155b7686222a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.824+0000 7f3032ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.824+0000 7f3032ffd640 1 -- 172.21.15.43:0/950027933 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048045070 con 0x55fc5ad9b9f0 2024-09-11T23:22:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.824+0000 7f305ca56740 1 -- 172.21.15.43:0/950027933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3048005850 con 0x55fc5ad9b9f0 2024-09-11T23:22:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.824+0000 7f3032ffd640 1 -- 172.21.15.43:0/950027933 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f3048032d00 con 0x55fc5ad9b9f0 2024-09-11T23:22:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.824+0000 7f3032ffd640 1 -- 172.21.15.43:0/950027933 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3048037030 con 0x55fc5ad9b9f0 2024-09-11T23:22:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.824+0000 7f305ca56740 10 client.15387 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.824+0000 7f305ca56740 1 -- 172.21.15.43:0/950027933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fc5ae25fd0 con 0x55fc5ad9b9f0 2024-09-11T23:22:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.824+0000 7f3032ffd640 1 -- 172.21.15.43:0/950027933 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3048005850 con 0x55fc5ad9b9f0 2024-09-11T23:22:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.824+0000 7f3032ffd640 1 -- 172.21.15.43:0/950027933 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f304804c040 con 0x55fc5ad9b9f0 2024-09-11T23:22:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.825+0000 7f305ca56740 10 client.15387 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.825+0000 7f305ca56740 10 client.15387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.825+0000 7f305ca56740 20 client.15387 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.825+0000 7f305ca56740 1 --2- 172.21.15.43:0/950027933 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55fc5ae264f0 0x55fc5ae288e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.825+0000 7f305ca56740 4 client.15387 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:22:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.825+0000 7f305ca56740 1 -- 172.21.15.43:0/950027933 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc5ad07d70 con 0x55fc5ae264f0 2024-09-11T23:22:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.825+0000 7f30537fe640 1 --2- 172.21.15.43:0/950027933 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55fc5ae264f0 0x55fc5ae288e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.825+0000 7f3032ffd640 10 client.15387 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.825+0000 7f30537fe640 1 --2- 172.21.15.43:0/950027933 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55fc5ae264f0 0x55fc5ae288e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4da68b28ab96864 server_cookie=bcb597eff9408a22 in_seq=0 out_seq=0 2024-09-11T23:22:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.835+0000 7f3032ffd640 1 -- 172.21.15.43:0/950027933 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fc5ad07d70 con 0x55fc5ae264f0 2024-09-11T23:22:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.835+0000 7f3032ffd640 10 client.15387 handle_command_reply: tid=0 2024-09-11T23:22:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.835+0000 7f305ca56740 1 client.15387 shutdown 2024-09-11T23:22:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.835+0000 7f305ca56740 1 -- 172.21.15.43:0/950027933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5ad9b9f0 msgr2=0x55fc5adaee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.835+0000 7f305ca56740 1 --2- 172.21.15.43:0/950027933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5ad9b9f0 0x55fc5adaee10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3048005e00 tx=0x7f3048005d20 comp rx=0 tx=0).stop 2024-09-11T23:22:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.835+0000 7f305ca56740 1 -- 172.21.15.43:0/950027933 shutdown_connections 2024-09-11T23:22:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.835+0000 7f305ca56740 1 --2- 172.21.15.43:0/950027933 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55fc5ae264f0 0x55fc5ae288e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.835+0000 7f305ca56740 1 --2- 172.21.15.43:0/950027933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc5adab0c0 0x55fc5adaf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.835+0000 7f305ca56740 1 --2- 172.21.15.43:0/950027933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fc5ad9c310 0x55fc5adaf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.835+0000 7f3032ffd640 0 client.15387 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.835+0000 7f305ca56740 1 --2- 172.21.15.43:0/950027933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc5ad9b9f0 0x55fc5adaee10 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.835+0000 7f305ca56740 1 -- 172.21.15.43:0/950027933 >> 172.21.15.43:0/950027933 conn(0x55fc5ad98060 msgr2=0x55fc5ad969a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.835+0000 7f305ca56740 1 -- 172.21.15.43:0/950027933 shutdown_connections 2024-09-11T23:22:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.835+0000 7f305ca56740 1 -- 172.21.15.43:0/950027933 wait complete. 2024-09-11T23:22:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.835+0000 7f305ca56740 20 client.15387 trim_cache size 0 max 16384 2024-09-11T23:22:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:22:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:22:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:22:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:22:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.836+0000 7f3059be4640 1 -- 172.21.15.43:0/300677349 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f303c0a2e10 msgr2=0x7f303c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.836+0000 7f3059be4640 1 --2- 172.21.15.43:0/300677349 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f303c0a2e10 0x7f303c0a52d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f304c0cd410 tx=0x7f304c06f120 comp rx=0 tx=0).stop 2024-09-11T23:22:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.836+0000 7f3059be4640 1 -- 172.21.15.43:0/300677349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30541069e0 msgr2=0x7f305419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.836+0000 7f3059be4640 1 --2- 172.21.15.43:0/300677349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30541069e0 0x7f305419cca0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f304c00eb10 tx=0x7f304c00ef10 comp rx=0 tx=0).stop 2024-09-11T23:22:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.836+0000 7f3059be4640 1 -- 172.21.15.43:0/300677349 shutdown_connections 2024-09-11T23:22:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.836+0000 7f3059be4640 1 --2- 172.21.15.43:0/300677349 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f303c0a2e10 0x7f303c0a52d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.836+0000 7f3059be4640 1 --2- 172.21.15.43:0/300677349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30541069e0 0x7f305419cca0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.836+0000 7f3059be4640 1 --2- 172.21.15.43:0/300677349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3054101370 0x7f3054198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.836+0000 7f3059be4640 1 --2- 172.21.15.43:0/300677349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30541009d0 0x7f30541983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.836+0000 7f3059be4640 1 -- 172.21.15.43:0/300677349 >> 172.21.15.43:0/300677349 conn(0x7f30540fc6b0 msgr2=0x7f30540fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.836+0000 7f3059be4640 1 -- 172.21.15.43:0/300677349 shutdown_connections 2024-09-11T23:22:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:14.836+0000 7f3059be4640 1 -- 172.21.15.43:0/300677349 wait complete. 2024-09-11T23:22:15.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:14 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/152295099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:15.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:14 smithi097 ceph-mon[35562]: scrub summary: idle 2024-09-11T23:22:15.061 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:14 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/152295099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:15.061 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:14 smithi090 ceph-mon[35130]: scrub summary: idle 2024-09-11T23:22:15.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:14 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/152295099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:15.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:14 smithi043 ceph-mon[30950]: scrub summary: idle 2024-09-11T23:22:15.521 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T23:20:00.465830+0000', 'modified': '2024-09-11T23:21:49.117342+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24497, 'mds_1': 24470, 'mds_2': 24457}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24436': {'gid': 24436, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6835/1640511441', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 1640511441}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 1640511441}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6837/191447096', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 191447096}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 191447096}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.90:6835/4233140081', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6834', 'nonce': 4233140081}, {'type': 'v1', 'addr': '172.21.15.90:6835', 'nonce': 4233140081}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.97:6839/1424670024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 1424670024}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 1424670024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.90:6837/2946725567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6836', 'nonce': 2946725567}, {'type': 'v1', 'addr': '172.21.15.90:6837', 'nonce': 2946725567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.90:6839/3234895964', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6838', 'nonce': 3234895964}, {'type': 'v1', 'addr': '172.21.15.90:6839', 'nonce': 3234895964}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24497, 'qdb_cluster': [24497, 24470, 24457]} 2024-09-11T23:22:15.521 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-11T23:22:15.521 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T23:22:15.817 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:16.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:15 smithi097 ceph-mon[35562]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T23:22:16.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:15 smithi043 ceph-mon[30950]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T23:22:16.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:15 smithi090 ceph-mon[35130]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T23:22:16.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.292+0000 7fdc44e3f640 1 -- 172.21.15.43:0/412575051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc401036e0 msgr2=0x7fdc40107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:16.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.292+0000 7fdc44e3f640 1 --2- 172.21.15.43:0/412575051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc401036e0 0x7fdc40107b90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fdc28009930 tx=0x7fdc2802a420 comp rx=0 tx=0).stop 2024-09-11T23:22:16.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.292+0000 7fdc44e3f640 1 -- 172.21.15.43:0/412575051 shutdown_connections 2024-09-11T23:22:16.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.292+0000 7fdc44e3f640 1 --2- 172.21.15.43:0/412575051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc401082a0 0x7fdc4010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.292+0000 7fdc44e3f640 1 --2- 172.21.15.43:0/412575051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc401036e0 0x7fdc40107b90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.292+0000 7fdc44e3f640 1 --2- 172.21.15.43:0/412575051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc40102d30 0x7fdc40103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.292+0000 7fdc44e3f640 1 -- 172.21.15.43:0/412575051 >> 172.21.15.43:0/412575051 conn(0x7fdc400fc7d0 msgr2=0x7fdc400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:16.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.292+0000 7fdc44e3f640 1 -- 172.21.15.43:0/412575051 shutdown_connections 2024-09-11T23:22:16.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.292+0000 7fdc44e3f640 1 -- 172.21.15.43:0/412575051 wait complete. 2024-09-11T23:22:16.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.293+0000 7fdc44e3f640 1 Processor -- start 2024-09-11T23:22:16.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.293+0000 7fdc44e3f640 1 -- start start 2024-09-11T23:22:16.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.293+0000 7fdc44e3f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc40102d30 0x7fdc40198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:16.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.293+0000 7fdc44e3f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc401036e0 0x7fdc40198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:16.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.293+0000 7fdc44e3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc401082a0 0x7fdc4019d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:16.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.293+0000 7fdc44e3f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdc40112140 con 0x7fdc401082a0 2024-09-11T23:22:16.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.293+0000 7fdc44e3f640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdc40111fc0 con 0x7fdc401036e0 2024-09-11T23:22:16.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.293+0000 7fdc44e3f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdc401122c0 con 0x7fdc40102d30 2024-09-11T23:22:16.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.293+0000 7fdc3ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc401082a0 0x7fdc4019d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:16.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.293+0000 7fdc3ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc401082a0 0x7fdc4019d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36314/0 (socket says 172.21.15.43:36314) 2024-09-11T23:22:16.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.293+0000 7fdc3ed9d640 1 -- 172.21.15.43:0/692320889 learned_addr learned my addr 172.21.15.43:0/692320889 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:16.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.293+0000 7fdc3e59c640 1 --2- 172.21.15.43:0/692320889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc40102d30 0x7fdc40198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:16.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.293+0000 7fdc3dd9b640 1 --2- 172.21.15.43:0/692320889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc401036e0 0x7fdc40198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:16.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.294+0000 7fdc3ed9d640 1 -- 172.21.15.43:0/692320889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc40102d30 msgr2=0x7fdc40198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:16.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.294+0000 7fdc3ed9d640 1 --2- 172.21.15.43:0/692320889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc40102d30 0x7fdc40198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.294+0000 7fdc3ed9d640 1 -- 172.21.15.43:0/692320889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc401036e0 msgr2=0x7fdc40198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:16.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.294+0000 7fdc3ed9d640 1 --2- 172.21.15.43:0/692320889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc401036e0 0x7fdc40198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.294+0000 7fdc3ed9d640 1 -- 172.21.15.43:0/692320889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4019d6f0 con 0x7fdc401082a0 2024-09-11T23:22:16.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.294+0000 7fdc3e59c640 1 --2- 172.21.15.43:0/692320889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc40102d30 0x7fdc40198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:16.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.294+0000 7fdc3ed9d640 1 --2- 172.21.15.43:0/692320889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc401082a0 0x7fdc4019d010 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009fd0 tx=0x7fdc3000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:16.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.294+0000 7fdc377fe640 1 -- 172.21.15.43:0/692320889 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30011830 con 0x7fdc401082a0 2024-09-11T23:22:16.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.294+0000 7fdc44e3f640 1 -- 172.21.15.43:0/692320889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc4019d9e0 con 0x7fdc401082a0 2024-09-11T23:22:16.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.294+0000 7fdc377fe640 1 -- 172.21.15.43:0/692320889 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc30016070 con 0x7fdc401082a0 2024-09-11T23:22:16.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.294+0000 7fdc377fe640 1 -- 172.21.15.43:0/692320889 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3000f2e0 con 0x7fdc401082a0 2024-09-11T23:22:16.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.294+0000 7fdc44e3f640 1 -- 172.21.15.43:0/692320889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc401a52c0 con 0x7fdc401082a0 2024-09-11T23:22:16.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.296+0000 7fdc47218740 1 Processor -- start 2024-09-11T23:22:16.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.296+0000 7fdc377fe640 1 -- 172.21.15.43:0/692320889 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fdc300119d0 con 0x7fdc401082a0 2024-09-11T23:22:16.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.297+0000 7fdc47218740 1 -- start start 2024-09-11T23:22:16.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.297+0000 7fdc47218740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597eadfd9d0 0x5597eadfddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:16.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.297+0000 7fdc47218740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfe2f0 0x5597eae08670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:16.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.297+0000 7fdc47218740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eae08bb0 0x5597eae0afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:16.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.297+0000 7fdc47218740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5597ead64090 con 0x5597eadfe2f0 2024-09-11T23:22:16.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.297+0000 7fdc47218740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5597ead0ba70 con 0x5597eadfd9d0 2024-09-11T23:22:16.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.297+0000 7fdc47218740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5597ead60050 con 0x5597eae08bb0 2024-09-11T23:22:16.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.297+0000 7fdc3e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfe2f0 0x5597eae08670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:16.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.297+0000 7fdc3e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfe2f0 0x5597eae08670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36326/0 (socket says 172.21.15.43:36326) 2024-09-11T23:22:16.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.297+0000 7fdc3e59c640 1 -- 172.21.15.43:0/3720017035 learned_addr learned my addr 172.21.15.43:0/3720017035 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:16.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.297+0000 7fdc3dd9b640 1 --2- 172.21.15.43:0/3720017035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597eadfd9d0 0x5597eadfddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:16.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.298+0000 7fdc3dd9b640 1 --2- 172.21.15.43:0/3720017035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eae08bb0 0x5597eae0afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:16.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.298+0000 7fdc3e59c640 1 -- 172.21.15.43:0/3720017035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eae08bb0 msgr2=0x5597eae0afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:16.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.298+0000 7fdc3e59c640 1 --2- 172.21.15.43:0/3720017035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eae08bb0 0x5597eae0afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.298+0000 7fdc3e59c640 1 -- 172.21.15.43:0/3720017035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597eadfd9d0 msgr2=0x5597eadfddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:16.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.298+0000 7fdc3e59c640 1 --2- 172.21.15.43:0/3720017035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597eadfd9d0 0x5597eadfddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.298+0000 7fdc3e59c640 1 -- 172.21.15.43:0/3720017035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597ead18130 con 0x5597eadfe2f0 2024-09-11T23:22:16.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.298+0000 7fdc3dd9b640 1 --2- 172.21.15.43:0/3720017035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eae08bb0 0x5597eae0afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:22:16.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.298+0000 7fdc377fe640 1 --2- 172.21.15.43:0/692320889 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdc2c0a2e10 0x7fdc2c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:16.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.298+0000 7fdc377fe640 1 -- 172.21.15.43:0/692320889 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fdc300ce8d0 con 0x7fdc401082a0 2024-09-11T23:22:16.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.298+0000 7fdc3e59c640 1 --2- 172.21.15.43:0/3720017035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfe2f0 0x5597eae08670 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdc3800c4a0 tx=0x7fdc3800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da91e791d7b016e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:16.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.298+0000 7fdc3dd9b640 1 --2- 172.21.15.43:0/692320889 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdc2c0a2e10 0x7fdc2c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:16.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.298+0000 7fdc34ff9640 1 -- 172.21.15.43:0/3720017035 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3801a070 con 0x5597eadfe2f0 2024-09-11T23:22:16.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.298+0000 7fdc34ff9640 1 -- 172.21.15.43:0/3720017035 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc380158f0 con 0x5597eadfe2f0 2024-09-11T23:22:16.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.298+0000 7fdc34ff9640 1 -- 172.21.15.43:0/3720017035 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3800a960 con 0x5597eadfe2f0 2024-09-11T23:22:16.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.299+0000 7fdc3dd9b640 1 --2- 172.21.15.43:0/692320889 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdc2c0a2e10 0x7fdc2c0a52d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdc28009900 tx=0x7fdc280058b0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:16.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.299+0000 7fdc47218740 1 -- 172.21.15.43:0/3720017035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfe2f0 msgr2=0x5597eae08670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:16.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.299+0000 7fdc47218740 1 --2- 172.21.15.43:0/3720017035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfe2f0 0x5597eae08670 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdc3800c4a0 tx=0x7fdc3800c8a0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.299+0000 7fdc47218740 1 -- 172.21.15.43:0/3720017035 shutdown_connections 2024-09-11T23:22:16.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.299+0000 7fdc47218740 1 --2- 172.21.15.43:0/3720017035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eae08bb0 0x5597eae0afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.299+0000 7fdc47218740 1 --2- 172.21.15.43:0/3720017035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfe2f0 0x5597eae08670 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.299+0000 7fdc47218740 1 --2- 172.21.15.43:0/3720017035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597eadfd9d0 0x5597eadfddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.299+0000 7fdc47218740 1 -- 172.21.15.43:0/3720017035 >> 172.21.15.43:0/3720017035 conn(0x5597eadf9f60 msgr2=0x5597eadfa340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:16.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.299+0000 7fdc47218740 1 -- 172.21.15.43:0/3720017035 shutdown_connections 2024-09-11T23:22:16.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.299+0000 7fdc47218740 1 -- 172.21.15.43:0/3720017035 wait complete. 2024-09-11T23:22:16.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.299+0000 7fdc47218740 1 -- start start 2024-09-11T23:22:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.300+0000 7fdc47218740 1 Processor -- start 2024-09-11T23:22:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.300+0000 7fdc47218740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfd9d0 0x5597eae084d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.300+0000 7fdc47218740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eadfe2f0 0x5597eae05820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.300+0000 7fdc47218740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597eae08bb0 0x5597eae05d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.300+0000 7fdc47218740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5597eae0b780 con 0x5597eadfd9d0 2024-09-11T23:22:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.300+0000 7fdc47218740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5597ead1ef20 con 0x5597eae08bb0 2024-09-11T23:22:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.300+0000 7fdc47218740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5597eae0b900 con 0x5597eadfe2f0 2024-09-11T23:22:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.300+0000 7fdc47218740 10 client.0 _ll_register_callbacks cb 0x5597eace8030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.300+0000 7fdc3e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfd9d0 0x5597eae084d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.300+0000 7fdc3e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfd9d0 0x5597eae084d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36342/0 (socket says 172.21.15.43:36342) 2024-09-11T23:22:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.300+0000 7fdc3e59c640 1 -- 172.21.15.43:0/2127144092 learned_addr learned my addr 172.21.15.43:0/2127144092 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.300+0000 7fdc3dd9b640 1 --2- 172.21.15.43:0/2127144092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eadfe2f0 0x5597eae05820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:16.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.300+0000 7fdc3ed9d640 1 --2- 172.21.15.43:0/2127144092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597eae08bb0 0x5597eae05d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:16.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.300+0000 7fdc3e59c640 1 -- 172.21.15.43:0/2127144092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eadfe2f0 msgr2=0x5597eae05820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:16.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.300+0000 7fdc3e59c640 1 --2- 172.21.15.43:0/2127144092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eadfe2f0 0x5597eae05820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.300+0000 7fdc3e59c640 1 -- 172.21.15.43:0/2127144092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597eae08bb0 msgr2=0x5597eae05d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:16.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.300+0000 7fdc3e59c640 1 --2- 172.21.15.43:0/2127144092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597eae08bb0 0x5597eae05d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.300+0000 7fdc3e59c640 1 -- 172.21.15.43:0/2127144092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597eaad52b0 con 0x5597eadfd9d0 2024-09-11T23:22:16.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.301+0000 7fdc3e59c640 1 --2- 172.21.15.43:0/2127144092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfd9d0 0x5597eae084d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdc3800c470 tx=0x7fdc38007070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f88bbfe95bc43e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:16.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.301+0000 7fdc25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:16.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.301+0000 7fdc25ffb640 1 -- 172.21.15.43:0/2127144092 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3801a040 con 0x5597eadfd9d0 2024-09-11T23:22:16.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.301+0000 7fdc47218740 1 -- 172.21.15.43:0/2127144092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc380158f0 con 0x5597eadfd9d0 2024-09-11T23:22:16.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.301+0000 7fdc25ffb640 1 -- 172.21.15.43:0/2127144092 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fdc3800a6e0 con 0x5597eadfd9d0 2024-09-11T23:22:16.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.301+0000 7fdc25ffb640 1 -- 172.21.15.43:0/2127144092 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdc3802e050 con 0x5597eadfd9d0 2024-09-11T23:22:16.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.301+0000 7fdc47218740 10 client.15399 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:16.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.301+0000 7fdc47218740 1 -- 172.21.15.43:0/2127144092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5597eae17070 con 0x5597eadfd9d0 2024-09-11T23:22:16.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.301+0000 7fdc25ffb640 1 -- 172.21.15.43:0/2127144092 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc380158f0 con 0x5597eadfd9d0 2024-09-11T23:22:16.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.301+0000 7fdc25ffb640 1 -- 172.21.15.43:0/2127144092 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fdc380205d0 con 0x5597eadfd9d0 2024-09-11T23:22:16.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.301+0000 7fdc47218740 10 client.15399 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:16.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.302+0000 7fdc47218740 10 client.15399 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:22:16.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.302+0000 7fdc47218740 20 client.15399 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:16.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.302+0000 7fdc47218740 1 --2- 172.21.15.43:0/2127144092 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5597eae192d0 0x5597eae1a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:16.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.302+0000 7fdc47218740 4 client.15399 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:22:16.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.302+0000 7fdc47218740 1 -- 172.21.15.43:0/2127144092 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597ead71600 con 0x5597eae192d0 2024-09-11T23:22:16.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.302+0000 7fdc3dd9b640 1 --2- 172.21.15.43:0/2127144092 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5597eae192d0 0x5597eae1a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:16.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.302+0000 7fdc25ffb640 10 client.15399 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:22:16.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.302+0000 7fdc3dd9b640 1 --2- 172.21.15.43:0/2127144092 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5597eae192d0 0x5597eae1a740 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7fb2124e08d9416c server_cookie=bfa01359ce2f77c4 in_seq=0 out_seq=0 2024-09-11T23:22:16.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.303+0000 7fdc25ffb640 1 -- 172.21.15.43:0/2127144092 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5597ead71600 con 0x5597eae192d0 2024-09-11T23:22:16.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.303+0000 7fdc25ffb640 10 client.15399 handle_command_reply: tid=0 2024-09-11T23:22:16.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.303+0000 7fdc47218740 1 client.15399 shutdown 2024-09-11T23:22:16.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.303+0000 7fdc47218740 1 -- 172.21.15.43:0/2127144092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfd9d0 msgr2=0x5597eae084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:16.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.303+0000 7fdc47218740 1 --2- 172.21.15.43:0/2127144092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfd9d0 0x5597eae084d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdc3800c470 tx=0x7fdc38007070 comp rx=0 tx=0).stop 2024-09-11T23:22:16.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.303+0000 7fdc47218740 1 -- 172.21.15.43:0/2127144092 shutdown_connections 2024-09-11T23:22:16.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.303+0000 7fdc47218740 1 --2- 172.21.15.43:0/2127144092 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5597eae192d0 0x5597eae1a740 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.303+0000 7fdc47218740 1 --2- 172.21.15.43:0/2127144092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597eae08bb0 0x5597eae05d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.303+0000 7fdc47218740 1 --2- 172.21.15.43:0/2127144092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eadfe2f0 0x5597eae05820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.303+0000 7fdc47218740 1 --2- 172.21.15.43:0/2127144092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfd9d0 0x5597eae084d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.303+0000 7fdc47218740 1 -- 172.21.15.43:0/2127144092 >> 172.21.15.43:0/2127144092 conn(0x5597eadf9f60 msgr2=0x5597eadffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:16.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.303+0000 7fdc25ffb640 0 client.15399 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:22:16.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.303+0000 7fdc47218740 1 -- 172.21.15.43:0/2127144092 shutdown_connections 2024-09-11T23:22:16.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.303+0000 7fdc47218740 1 -- 172.21.15.43:0/2127144092 wait complete. 2024-09-11T23:22:16.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.304+0000 7fdc47218740 20 client.15399 trim_cache size 0 max 16384 2024-09-11T23:22:16.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.321+0000 7fdc47218740 1 Processor -- start 2024-09-11T23:22:16.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.321+0000 7fdc47218740 1 -- start start 2024-09-11T23:22:16.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.322+0000 7fdc47218740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfd9d0 0x5597eae18c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:16.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.322+0000 7fdc47218740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eadfe2f0 0x5597eae1a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:16.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.322+0000 7fdc47218740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597eae08bb0 0x5597eae184d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:16.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.322+0000 7fdc47218740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5597eae18a80 con 0x5597eadfd9d0 2024-09-11T23:22:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.322+0000 7fdc47218740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5597eae19150 con 0x5597eae08bb0 2024-09-11T23:22:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.322+0000 7fdc47218740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5597eae19820 con 0x5597eadfe2f0 2024-09-11T23:22:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.322+0000 7fdc3e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfd9d0 0x5597eae18c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.322+0000 7fdc3e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfd9d0 0x5597eae18c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36356/0 (socket says 172.21.15.43:36356) 2024-09-11T23:22:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.322+0000 7fdc3e59c640 1 -- 172.21.15.43:0/1506855242 learned_addr learned my addr 172.21.15.43:0/1506855242 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.322+0000 7fdc3dd9b640 1 --2- 172.21.15.43:0/1506855242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eadfe2f0 0x5597eae1a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.322+0000 7fdc3ed9d640 1 --2- 172.21.15.43:0/1506855242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597eae08bb0 0x5597eae184d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.322+0000 7fdc3e59c640 1 -- 172.21.15.43:0/1506855242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eadfe2f0 msgr2=0x5597eae1a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.322+0000 7fdc3e59c640 1 --2- 172.21.15.43:0/1506855242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eadfe2f0 0x5597eae1a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.322+0000 7fdc3e59c640 1 -- 172.21.15.43:0/1506855242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597eae08bb0 msgr2=0x5597eae184d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.322+0000 7fdc3e59c640 1 --2- 172.21.15.43:0/1506855242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597eae08bb0 0x5597eae184d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.322+0000 7fdc3e59c640 1 -- 172.21.15.43:0/1506855242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597eae080f0 con 0x5597eadfd9d0 2024-09-11T23:22:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.322+0000 7fdc3ed9d640 1 --2- 172.21.15.43:0/1506855242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597eae08bb0 0x5597eae184d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:22:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.322+0000 7fdc3e59c640 1 --2- 172.21.15.43:0/1506855242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfd9d0 0x5597eae18c00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdc38002230 tx=0x7fdc3800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f482216544cb5ed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.322+0000 7fdc27fff640 1 -- 172.21.15.43:0/1506855242 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3801a030 con 0x5597eadfd9d0 2024-09-11T23:22:16.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.323+0000 7fdc27fff640 1 -- 172.21.15.43:0/1506855242 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc38025070 con 0x5597eadfd9d0 2024-09-11T23:22:16.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.323+0000 7fdc27fff640 1 -- 172.21.15.43:0/1506855242 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3800a910 con 0x5597eadfd9d0 2024-09-11T23:22:16.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.323+0000 7fdc47218740 1 -- 172.21.15.43:0/1506855242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfd9d0 msgr2=0x5597eae18c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:16.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.323+0000 7fdc47218740 1 --2- 172.21.15.43:0/1506855242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfd9d0 0x5597eae18c00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdc38002230 tx=0x7fdc3800c8a0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.323+0000 7fdc47218740 1 -- 172.21.15.43:0/1506855242 shutdown_connections 2024-09-11T23:22:16.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.323+0000 7fdc47218740 1 --2- 172.21.15.43:0/1506855242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597eae08bb0 0x5597eae184d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.323+0000 7fdc47218740 1 --2- 172.21.15.43:0/1506855242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eadfe2f0 0x5597eae1a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.323+0000 7fdc47218740 1 --2- 172.21.15.43:0/1506855242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfd9d0 0x5597eae18c00 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.323+0000 7fdc47218740 1 -- 172.21.15.43:0/1506855242 >> 172.21.15.43:0/1506855242 conn(0x5597eadf9f60 msgr2=0x5597eae06f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:16.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.323+0000 7fdc47218740 1 -- 172.21.15.43:0/1506855242 shutdown_connections 2024-09-11T23:22:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.323+0000 7fdc47218740 1 -- 172.21.15.43:0/1506855242 wait complete. 2024-09-11T23:22:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.324+0000 7fdc47218740 1 -- start start 2024-09-11T23:22:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.324+0000 7fdc47218740 1 Processor -- start 2024-09-11T23:22:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.324+0000 7fdc47218740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfd9d0 0x5597eae1dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.324+0000 7fdc47218740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597eadfe2f0 0x5597eae1e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.324+0000 7fdc47218740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eae08bb0 0x5597eae819d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.324+0000 7fdc47218740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5597eae1a5c0 con 0x5597eadfd9d0 2024-09-11T23:22:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.324+0000 7fdc47218740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5597eae19ef0 con 0x5597eadfe2f0 2024-09-11T23:22:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.324+0000 7fdc47218740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5597eae1ac90 con 0x5597eae08bb0 2024-09-11T23:22:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.324+0000 7fdc47218740 10 client.0 _ll_register_callbacks cb 0x7fdc38018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.324+0000 7fdc3e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfd9d0 0x5597eae1dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.324+0000 7fdc3e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfd9d0 0x5597eae1dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36360/0 (socket says 172.21.15.43:36360) 2024-09-11T23:22:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.324+0000 7fdc3e59c640 1 -- 172.21.15.43:0/2151014931 learned_addr learned my addr 172.21.15.43:0/2151014931 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:16.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.324+0000 7fdc3dd9b640 1 --2- 172.21.15.43:0/2151014931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597eadfe2f0 0x5597eae1e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:16.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.325+0000 7fdc3ed9d640 1 --2- 172.21.15.43:0/2151014931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eae08bb0 0x5597eae819d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:16.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.325+0000 7fdc3e59c640 1 -- 172.21.15.43:0/2151014931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eae08bb0 msgr2=0x5597eae819d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:16.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.325+0000 7fdc3e59c640 1 --2- 172.21.15.43:0/2151014931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eae08bb0 0x5597eae819d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.325+0000 7fdc3e59c640 1 -- 172.21.15.43:0/2151014931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597eadfe2f0 msgr2=0x5597eae1e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:16.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.325+0000 7fdc3e59c640 1 --2- 172.21.15.43:0/2151014931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597eadfe2f0 0x5597eae1e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.325+0000 7fdc3e59c640 1 -- 172.21.15.43:0/2151014931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597eae86350 con 0x5597eadfd9d0 2024-09-11T23:22:16.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.325+0000 7fdc3ed9d640 1 --2- 172.21.15.43:0/2151014931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eae08bb0 0x5597eae819d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:16.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.325+0000 7fdc3e59c640 1 --2- 172.21.15.43:0/2151014931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfd9d0 0x5597eae1dfd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdc38020a20 tx=0x7fdc3800a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2952867cacad432c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:16.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.325+0000 7fdc25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:16.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.325+0000 7fdc25ffb640 1 -- 172.21.15.43:0/2151014931 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38036030 con 0x5597eadfd9d0 2024-09-11T23:22:16.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.325+0000 7fdc25ffb640 1 -- 172.21.15.43:0/2151014931 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fdc38012050 con 0x5597eadfd9d0 2024-09-11T23:22:16.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.325+0000 7fdc47218740 1 -- 172.21.15.43:0/2151014931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc38025070 con 0x5597eadfd9d0 2024-09-11T23:22:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.325+0000 7fdc25ffb640 1 -- 172.21.15.43:0/2151014931 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdc3803f050 con 0x5597eadfd9d0 2024-09-11T23:22:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.325+0000 7fdc47218740 10 client.15405 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.325+0000 7fdc47218740 1 -- 172.21.15.43:0/2151014931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5597eae87540 con 0x5597eadfd9d0 2024-09-11T23:22:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.326+0000 7fdc25ffb640 1 -- 172.21.15.43:0/2151014931 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc38025070 con 0x5597eadfd9d0 2024-09-11T23:22:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.326+0000 7fdc25ffb640 1 -- 172.21.15.43:0/2151014931 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fdc3800ab60 con 0x5597eadfd9d0 2024-09-11T23:22:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.326+0000 7fdc47218740 10 client.15405 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.326+0000 7fdc47218740 10 client.15405 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T23:22:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.326+0000 7fdc47218740 20 client.15405 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.326+0000 7fdc47218740 1 --2- 172.21.15.43:0/2151014931 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5597eae8bf00 0x5597eae8ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.326+0000 7fdc47218740 4 client.15405 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:22:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.326+0000 7fdc47218740 1 -- 172.21.15.43:0/2151014931 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "status"}) -- 0x5597ead6bb10 con 0x5597eae8bf00 2024-09-11T23:22:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.326+0000 7fdc3dd9b640 1 --2- 172.21.15.43:0/2151014931 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5597eae8bf00 0x5597eae8ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:16.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.327+0000 7fdc25ffb640 10 client.15405 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:22:16.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.327+0000 7fdc3dd9b640 1 --2- 172.21.15.43:0/2151014931 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5597eae8bf00 0x5597eae8ac10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=247828388751328a server_cookie=64b09ad1fa47e63b in_seq=0 out_seq=0 2024-09-11T23:22:16.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.327+0000 7fdc25ffb640 1 -- 172.21.15.43:0/2151014931 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5597ead6bb10 con 0x5597eae8bf00 2024-09-11T23:22:16.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.327+0000 7fdc25ffb640 10 client.15405 handle_command_reply: tid=0 2024-09-11T23:22:16.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.327+0000 7fdc47218740 1 client.15405 shutdown 2024-09-11T23:22:16.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.327+0000 7fdc47218740 1 -- 172.21.15.43:0/2151014931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfd9d0 msgr2=0x5597eae1dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:16.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.327+0000 7fdc47218740 1 --2- 172.21.15.43:0/2151014931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfd9d0 0x5597eae1dfd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdc38020a20 tx=0x7fdc3800a480 comp rx=0 tx=0).stop 2024-09-11T23:22:16.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.327+0000 7fdc47218740 1 -- 172.21.15.43:0/2151014931 shutdown_connections 2024-09-11T23:22:16.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.327+0000 7fdc47218740 1 --2- 172.21.15.43:0/2151014931 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5597eae8bf00 0x5597eae8ac10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.327+0000 7fdc47218740 1 --2- 172.21.15.43:0/2151014931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5597eae08bb0 0x5597eae819d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.327+0000 7fdc47218740 1 --2- 172.21.15.43:0/2151014931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597eadfe2f0 0x5597eae1e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.327+0000 7fdc47218740 1 --2- 172.21.15.43:0/2151014931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5597eadfd9d0 0x5597eae1dfd0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.327+0000 7fdc47218740 1 -- 172.21.15.43:0/2151014931 >> 172.21.15.43:0/2151014931 conn(0x5597eadf9f60 msgr2=0x5597eadf9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:16.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.327+0000 7fdc25ffb640 0 client.15405 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:22:16.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.327+0000 7fdc47218740 1 -- 172.21.15.43:0/2151014931 shutdown_connections 2024-09-11T23:22:16.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.327+0000 7fdc47218740 1 -- 172.21.15.43:0/2151014931 wait complete. 2024-09-11T23:22:16.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.327+0000 7fdc47218740 20 client.15405 trim_cache size 0 max 16384 2024-09-11T23:22:16.333 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:22:16.333 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "25c0cd84-7093-11ef-bcea-c7b262605968", 2024-09-11T23:22:16.333 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 2, 2024-09-11T23:22:16.333 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24457, 2024-09-11T23:22:16.333 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-11T23:22:16.333 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-11T23:22:16.334 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-11T23:22:16.334 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 126.60817532, 2024-09-11T23:22:16.334 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 24, 2024-09-11T23:22:16.334 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 85, 2024-09-11T23:22:16.334 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 84, 2024-09-11T23:22:16.334 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 155.85266328099999 2024-09-11T23:22:16.334 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:22:16.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.328+0000 7fdc44e3f640 1 -- 172.21.15.43:0/692320889 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdc2c0a2e10 msgr2=0x7fdc2c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:16.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.328+0000 7fdc44e3f640 1 --2- 172.21.15.43:0/692320889 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdc2c0a2e10 0x7fdc2c0a52d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdc28009900 tx=0x7fdc280058b0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.328+0000 7fdc44e3f640 1 -- 172.21.15.43:0/692320889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc401082a0 msgr2=0x7fdc4019d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:16.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.328+0000 7fdc44e3f640 1 --2- 172.21.15.43:0/692320889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc401082a0 0x7fdc4019d010 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009fd0 tx=0x7fdc3000b8e0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.328+0000 7fdc44e3f640 1 -- 172.21.15.43:0/692320889 shutdown_connections 2024-09-11T23:22:16.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.328+0000 7fdc44e3f640 1 --2- 172.21.15.43:0/692320889 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdc2c0a2e10 0x7fdc2c0a52d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.328+0000 7fdc44e3f640 1 --2- 172.21.15.43:0/692320889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc401082a0 0x7fdc4019d010 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.328+0000 7fdc44e3f640 1 --2- 172.21.15.43:0/692320889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc401036e0 0x7fdc40198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.328+0000 7fdc44e3f640 1 --2- 172.21.15.43:0/692320889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc40102d30 0x7fdc40198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:16.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.328+0000 7fdc44e3f640 1 -- 172.21.15.43:0/692320889 >> 172.21.15.43:0/692320889 conn(0x7fdc400fc7d0 msgr2=0x7fdc40105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:16.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.328+0000 7fdc44e3f640 1 -- 172.21.15.43:0/692320889 shutdown_connections 2024-09-11T23:22:16.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:16.328+0000 7fdc44e3f640 1 -- 172.21.15.43:0/692320889 wait complete. 2024-09-11T23:22:17.075 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T23:22:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:17.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.836+0000 7f2eda43d640 1 -- 172.21.15.43:0/3688727380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed4106ba0 msgr2=0x7f2ed4106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:17.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.836+0000 7f2eda43d640 1 --2- 172.21.15.43:0/3688727380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed4106ba0 0x7f2ed4106f80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009900 tx=0x7f2ec402a410 comp rx=0 tx=0).stop 2024-09-11T23:22:17.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.837+0000 7f2eda43d640 1 -- 172.21.15.43:0/3688727380 shutdown_connections 2024-09-11T23:22:17.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.837+0000 7f2eda43d640 1 --2- 172.21.15.43:0/3688727380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed4100a70 0x7f2ed410ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.837+0000 7f2eda43d640 1 --2- 172.21.15.43:0/3688727380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ed41000d0 0x7f2ed4100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.837+0000 7f2eda43d640 1 --2- 172.21.15.43:0/3688727380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed4106ba0 0x7f2ed4106f80 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.837+0000 7f2eda43d640 1 -- 172.21.15.43:0/3688727380 >> 172.21.15.43:0/3688727380 conn(0x7f2ed4077210 msgr2=0x7f2ed40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:17.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.837+0000 7f2eda43d640 1 -- 172.21.15.43:0/3688727380 shutdown_connections 2024-09-11T23:22:17.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.837+0000 7f2eda43d640 1 -- 172.21.15.43:0/3688727380 wait complete. 2024-09-11T23:22:17.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.837+0000 7f2eda43d640 1 Processor -- start 2024-09-11T23:22:17.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.837+0000 7f2eda43d640 1 -- start start 2024-09-11T23:22:17.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.838+0000 7f2eda43d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ed41000d0 0x7f2ed4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:17.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.838+0000 7f2eda43d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed4100a70 0x7f2ed41989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:17.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.838+0000 7f2eda43d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed4106ba0 0x7f2ed419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:17.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.838+0000 7f2eda43d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed4112050 con 0x7f2ed4106ba0 2024-09-11T23:22:17.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.838+0000 7f2eda43d640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed4111ed0 con 0x7f2ed41000d0 2024-09-11T23:22:17.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.838+0000 7f2eda43d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed41121d0 con 0x7f2ed4100a70 2024-09-11T23:22:17.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.838+0000 7f2ed89da640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed4106ba0 0x7f2ed419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:17.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.838+0000 7f2ed89da640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed4106ba0 0x7f2ed419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36374/0 (socket says 172.21.15.43:36374) 2024-09-11T23:22:17.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.838+0000 7f2ed89da640 1 -- 172.21.15.43:0/306837242 learned_addr learned my addr 172.21.15.43:0/306837242 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:17.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.838+0000 7f2ed37fe640 1 --2- 172.21.15.43:0/306837242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed4100a70 0x7f2ed41989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:17.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.838+0000 7f2ed3fff640 1 --2- 172.21.15.43:0/306837242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ed41000d0 0x7f2ed4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:17.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.838+0000 7f2ed89da640 1 -- 172.21.15.43:0/306837242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed4100a70 msgr2=0x7f2ed41989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:17.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.838+0000 7f2ed89da640 1 --2- 172.21.15.43:0/306837242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed4100a70 0x7f2ed41989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.838+0000 7f2ed89da640 1 -- 172.21.15.43:0/306837242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ed41000d0 msgr2=0x7f2ed4198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:17.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.838+0000 7f2ed89da640 1 --2- 172.21.15.43:0/306837242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ed41000d0 0x7f2ed4198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.838+0000 7f2ed89da640 1 -- 172.21.15.43:0/306837242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed419d420 con 0x7f2ed4106ba0 2024-09-11T23:22:17.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.838+0000 7f2ed37fe640 1 --2- 172.21.15.43:0/306837242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed4100a70 0x7f2ed41989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:17.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.838+0000 7f2ed89da640 1 --2- 172.21.15.43:0/306837242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed4106ba0 0x7f2ed419cd40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00eb10 tx=0x7f2ecc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:17.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.838+0000 7f2ed17fa640 1 -- 172.21.15.43:0/306837242 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc004b20 con 0x7f2ed4106ba0 2024-09-11T23:22:17.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.838+0000 7f2eda43d640 1 -- 172.21.15.43:0/306837242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed419d710 con 0x7f2ed4106ba0 2024-09-11T23:22:17.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.838+0000 7f2ed17fa640 1 -- 172.21.15.43:0/306837242 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ecc004cc0 con 0x7f2ed4106ba0 2024-09-11T23:22:17.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.839+0000 7f2ed17fa640 1 -- 172.21.15.43:0/306837242 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc010760 con 0x7f2ed4106ba0 2024-09-11T23:22:17.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.839+0000 7f2eda43d640 1 -- 172.21.15.43:0/306837242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed41a4ff0 con 0x7f2ed4106ba0 2024-09-11T23:22:17.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.842+0000 7f2ed17fa640 1 -- 172.21.15.43:0/306837242 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f2ecc005020 con 0x7f2ed4106ba0 2024-09-11T23:22:17.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.842+0000 7f2edc816740 1 Processor -- start 2024-09-11T23:22:17.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.842+0000 7f2edc816740 1 -- start start 2024-09-11T23:22:17.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.842+0000 7f2edc816740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb759d0 0x5651edb75db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:17.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.842+0000 7f2edc816740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651edb762f0 0x5651edb80670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:17.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.842+0000 7f2edc816740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651edb80bb0 0x5651edb82fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:17.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.842+0000 7f2edc816740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5651edadc090 con 0x5651edb759d0 2024-09-11T23:22:17.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.842+0000 7f2edc816740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5651eda83a70 con 0x5651edb80bb0 2024-09-11T23:22:17.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.842+0000 7f2edc816740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5651edad8050 con 0x5651edb762f0 2024-09-11T23:22:17.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.842+0000 7f2ed37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb759d0 0x5651edb75db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:17.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.842+0000 7f2ed37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb759d0 0x5651edb75db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36390/0 (socket says 172.21.15.43:36390) 2024-09-11T23:22:17.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.842+0000 7f2ed37fe640 1 -- 172.21.15.43:0/1979214834 learned_addr learned my addr 172.21.15.43:0/1979214834 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:17.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.842+0000 7f2ed37fe640 1 --2- 172.21.15.43:0/1979214834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651edb80bb0 0x5651edb82fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:17.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.842+0000 7f2ed3fff640 1 --2- 172.21.15.43:0/1979214834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651edb762f0 0x5651edb80670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:17.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.842+0000 7f2ed17fa640 1 --2- 172.21.15.43:0/306837242 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f2ebc0a2e10 0x7f2ebc0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:17.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.842+0000 7f2ed37fe640 1 -- 172.21.15.43:0/1979214834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651edb762f0 msgr2=0x5651edb80670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:17.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.843+0000 7f2ed37fe640 1 --2- 172.21.15.43:0/1979214834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651edb762f0 0x5651edb80670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.843+0000 7f2ed37fe640 1 -- 172.21.15.43:0/1979214834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651edb80bb0 msgr2=0x5651edb82fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:17.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.843+0000 7f2ed37fe640 1 --2- 172.21.15.43:0/1979214834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651edb80bb0 0x5651edb82fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.843+0000 7f2ed37fe640 1 -- 172.21.15.43:0/1979214834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651eda90130 con 0x5651edb759d0 2024-09-11T23:22:17.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.843+0000 7f2ed3fff640 1 --2- 172.21.15.43:0/1979214834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651edb762f0 0x5651edb80670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:17.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.843+0000 7f2ed89da640 1 --2- 172.21.15.43:0/306837242 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f2ebc0a2e10 0x7f2ebc0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:17.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.843+0000 7f2ed37fe640 1 --2- 172.21.15.43:0/1979214834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb759d0 0x5651edb75db0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8013970 tx=0x7f2ec800a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd81265d1dfa633a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:17.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.843+0000 7f2ed17fa640 1 -- 172.21.15.43:0/306837242 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f2ecc0cfde0 con 0x7f2ed4106ba0 2024-09-11T23:22:17.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.843+0000 7f2eba7fc640 1 -- 172.21.15.43:0/1979214834 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec800eb00 con 0x5651edb759d0 2024-09-11T23:22:17.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.843+0000 7f2eba7fc640 1 -- 172.21.15.43:0/1979214834 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ec8004c80 con 0x5651edb759d0 2024-09-11T23:22:17.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.843+0000 7f2eba7fc640 1 -- 172.21.15.43:0/1979214834 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8005440 con 0x5651edb759d0 2024-09-11T23:22:17.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.844+0000 7f2edc816740 1 -- 172.21.15.43:0/1979214834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb759d0 msgr2=0x5651edb75db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:17.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.844+0000 7f2ed89da640 1 --2- 172.21.15.43:0/306837242 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f2ebc0a2e10 0x7f2ebc0a52d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0cd410 tx=0x7f2ecc06bf10 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:17.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.844+0000 7f2edc816740 1 --2- 172.21.15.43:0/1979214834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb759d0 0x5651edb75db0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8013970 tx=0x7f2ec800a510 comp rx=0 tx=0).stop 2024-09-11T23:22:17.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.844+0000 7f2edc816740 1 -- 172.21.15.43:0/1979214834 shutdown_connections 2024-09-11T23:22:17.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.844+0000 7f2edc816740 1 --2- 172.21.15.43:0/1979214834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651edb80bb0 0x5651edb82fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.844+0000 7f2edc816740 1 --2- 172.21.15.43:0/1979214834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651edb762f0 0x5651edb80670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.844+0000 7f2edc816740 1 --2- 172.21.15.43:0/1979214834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb759d0 0x5651edb75db0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.844+0000 7f2edc816740 1 -- 172.21.15.43:0/1979214834 >> 172.21.15.43:0/1979214834 conn(0x5651edb71f60 msgr2=0x5651edb72340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.844+0000 7f2edc816740 1 -- 172.21.15.43:0/1979214834 shutdown_connections 2024-09-11T23:22:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.844+0000 7f2edc816740 1 -- 172.21.15.43:0/1979214834 wait complete. 2024-09-11T23:22:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.844+0000 7f2edc816740 1 -- start start 2024-09-11T23:22:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.845+0000 7f2edc816740 1 Processor -- start 2024-09-11T23:22:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.845+0000 7f2edc816740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb759d0 0x5651edb803f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.845+0000 7f2edc816740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651edb762f0 0x5651edb7d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.845+0000 7f2edc816740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651edb80bb0 0x5651edb7dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.845+0000 7f2edc816740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5651edb83780 con 0x5651edb759d0 2024-09-11T23:22:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.845+0000 7f2edc816740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5651eda96f20 con 0x5651edb80bb0 2024-09-11T23:22:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.845+0000 7f2edc816740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5651edb83900 con 0x5651edb762f0 2024-09-11T23:22:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.845+0000 7f2edc816740 10 client.0 _ll_register_callbacks cb 0x5651eda60030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.845+0000 7f2ed37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb759d0 0x5651edb803f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.845+0000 7f2ed37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb759d0 0x5651edb803f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36406/0 (socket says 172.21.15.43:36406) 2024-09-11T23:22:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.845+0000 7f2ed37fe640 1 -- 172.21.15.43:0/1726850176 learned_addr learned my addr 172.21.15.43:0/1726850176 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.845+0000 7f2ed3fff640 1 --2- 172.21.15.43:0/1726850176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651edb762f0 0x5651edb7d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.845+0000 7f2ed37fe640 1 -- 172.21.15.43:0/1726850176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651edb762f0 msgr2=0x5651edb7d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.845+0000 7f2ed37fe640 1 --2- 172.21.15.43:0/1726850176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651edb762f0 0x5651edb7d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.845+0000 7f2ed37fe640 1 -- 172.21.15.43:0/1726850176 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651edb80bb0 msgr2=0x5651edb7dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.845+0000 7f2ed37fe640 1 --2- 172.21.15.43:0/1726850176 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651edb80bb0 0x5651edb7dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.845+0000 7f2ed37fe640 1 -- 172.21.15.43:0/1726850176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651ed84d2b0 con 0x5651edb759d0 2024-09-11T23:22:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.846+0000 7f2ed37fe640 1 --2- 172.21.15.43:0/1726850176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb759d0 0x5651edb803f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800ad70 tx=0x7f2ec8004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2464fb00b1a27846 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.846+0000 7f2eb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.846+0000 7f2eb37fe640 1 -- 172.21.15.43:0/1726850176 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8004ad0 con 0x5651edb759d0 2024-09-11T23:22:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.846+0000 7f2eb37fe640 1 -- 172.21.15.43:0/1726850176 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f2ec800e900 con 0x5651edb759d0 2024-09-11T23:22:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.846+0000 7f2edc816740 1 -- 172.21.15.43:0/1726850176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ec8004c80 con 0x5651edb759d0 2024-09-11T23:22:17.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.846+0000 7f2eb37fe640 1 -- 172.21.15.43:0/1726850176 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2ec800eb00 con 0x5651edb759d0 2024-09-11T23:22:17.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.846+0000 7f2edc816740 10 client.15417 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:17.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.846+0000 7f2edc816740 1 -- 172.21.15.43:0/1726850176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5651edb8f040 con 0x5651edb759d0 2024-09-11T23:22:17.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.846+0000 7f2eb37fe640 1 -- 172.21.15.43:0/1726850176 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ec8004c80 con 0x5651edb759d0 2024-09-11T23:22:17.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.846+0000 7f2eb37fe640 1 -- 172.21.15.43:0/1726850176 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2ec800f040 con 0x5651edb759d0 2024-09-11T23:22:17.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.846+0000 7f2edc816740 10 client.15417 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:17.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.846+0000 7f2edc816740 10 client.15417 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T23:22:17.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.846+0000 7f2edc816740 20 client.15417 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:17.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.846+0000 7f2edc816740 1 --2- 172.21.15.43:0/1726850176 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5651edb90440 0x5651edb92640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:17.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.846+0000 7f2edc816740 4 client.15417 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:22:17.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.846+0000 7f2edc816740 1 -- 172.21.15.43:0/1726850176 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651edae9600 con 0x5651edb90440 2024-09-11T23:22:17.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.847+0000 7f2ed3fff640 1 --2- 172.21.15.43:0/1726850176 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5651edb90440 0x5651edb92640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:17.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.847+0000 7f2eb37fe640 10 client.15417 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:22:17.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.847+0000 7f2ed3fff640 1 --2- 172.21.15.43:0/1726850176 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5651edb90440 0x5651edb92640 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb33a1e488c786c3 server_cookie=22487652fd7f5375 in_seq=0 out_seq=0 2024-09-11T23:22:17.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.848+0000 7f2eb37fe640 1 -- 172.21.15.43:0/1726850176 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5651edae9600 con 0x5651edb90440 2024-09-11T23:22:17.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.848+0000 7f2eb37fe640 10 client.15417 handle_command_reply: tid=0 2024-09-11T23:22:17.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.848+0000 7f2edc816740 1 client.15417 shutdown 2024-09-11T23:22:17.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.848+0000 7f2edc816740 1 -- 172.21.15.43:0/1726850176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb759d0 msgr2=0x5651edb803f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:17.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.848+0000 7f2edc816740 1 --2- 172.21.15.43:0/1726850176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb759d0 0x5651edb803f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800ad70 tx=0x7f2ec8004ec0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.848+0000 7f2edc816740 1 -- 172.21.15.43:0/1726850176 shutdown_connections 2024-09-11T23:22:17.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.848+0000 7f2edc816740 1 --2- 172.21.15.43:0/1726850176 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5651edb90440 0x5651edb92640 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.848+0000 7f2edc816740 1 --2- 172.21.15.43:0/1726850176 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651edb80bb0 0x5651edb7dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.848+0000 7f2edc816740 1 --2- 172.21.15.43:0/1726850176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651edb762f0 0x5651edb7d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.848+0000 7f2edc816740 1 --2- 172.21.15.43:0/1726850176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb759d0 0x5651edb803f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.848+0000 7f2eb37fe640 0 client.15417 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:22:17.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.848+0000 7f2edc816740 1 -- 172.21.15.43:0/1726850176 >> 172.21.15.43:0/1726850176 conn(0x5651edb71f60 msgr2=0x5651edb77c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:17.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.848+0000 7f2edc816740 1 -- 172.21.15.43:0/1726850176 shutdown_connections 2024-09-11T23:22:17.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.848+0000 7f2edc816740 1 -- 172.21.15.43:0/1726850176 wait complete. 2024-09-11T23:22:17.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.848+0000 7f2edc816740 20 client.15417 trim_cache size 0 max 16384 2024-09-11T23:22:17.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.866+0000 7f2edc816740 1 Processor -- start 2024-09-11T23:22:17.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.866+0000 7f2edc816740 1 -- start start 2024-09-11T23:22:17.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.866+0000 7f2edc816740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb759d0 0x5651edb90b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:17.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.866+0000 7f2edc816740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651edb762f0 0x5651edb918a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.866+0000 7f2edc816740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651edb80bb0 0x5651edb7c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.866+0000 7f2edc816740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5651edb90980 con 0x5651edb759d0 2024-09-11T23:22:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.866+0000 7f2edc816740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5651edb91050 con 0x5651edb80bb0 2024-09-11T23:22:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.866+0000 7f2edc816740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5651edb91720 con 0x5651edb762f0 2024-09-11T23:22:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.866+0000 7f2ed37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb759d0 0x5651edb90b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.866+0000 7f2ed37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb759d0 0x5651edb90b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36410/0 (socket says 172.21.15.43:36410) 2024-09-11T23:22:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.866+0000 7f2ed37fe640 1 -- 172.21.15.43:0/660074822 learned_addr learned my addr 172.21.15.43:0/660074822 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.866+0000 7f2ed37fe640 1 -- 172.21.15.43:0/660074822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651edb762f0 msgr2=0x5651edb918a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.866+0000 7f2ed37fe640 1 --2- 172.21.15.43:0/660074822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651edb762f0 0x5651edb918a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.866+0000 7f2ed37fe640 1 -- 172.21.15.43:0/660074822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651edb80bb0 msgr2=0x5651edb7c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.866+0000 7f2ed37fe640 1 --2- 172.21.15.43:0/660074822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651edb80bb0 0x5651edb7c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.866+0000 7f2ed37fe640 1 -- 172.21.15.43:0/660074822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651edb7e850 con 0x5651edb759d0 2024-09-11T23:22:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.867+0000 7f2ed37fe640 1 --2- 172.21.15.43:0/660074822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb759d0 0x5651edb90b00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8005b90 tx=0x7f2ec8013920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a469fed1c73528 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.867+0000 7f2eb9ffb640 1 -- 172.21.15.43:0/660074822 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec80036c0 con 0x5651edb759d0 2024-09-11T23:22:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.867+0000 7f2eb9ffb640 1 -- 172.21.15.43:0/660074822 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ec8025070 con 0x5651edb759d0 2024-09-11T23:22:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.867+0000 7f2eb9ffb640 1 -- 172.21.15.43:0/660074822 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec801e480 con 0x5651edb759d0 2024-09-11T23:22:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.867+0000 7f2edc816740 1 -- 172.21.15.43:0/660074822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb759d0 msgr2=0x5651edb90b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.867+0000 7f2edc816740 1 --2- 172.21.15.43:0/660074822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb759d0 0x5651edb90b00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8005b90 tx=0x7f2ec8013920 comp rx=0 tx=0).stop 2024-09-11T23:22:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.867+0000 7f2edc816740 1 -- 172.21.15.43:0/660074822 shutdown_connections 2024-09-11T23:22:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.867+0000 7f2edc816740 1 --2- 172.21.15.43:0/660074822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651edb80bb0 0x5651edb7c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.867+0000 7f2edc816740 1 --2- 172.21.15.43:0/660074822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651edb762f0 0x5651edb918a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.867+0000 7f2edc816740 1 --2- 172.21.15.43:0/660074822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb759d0 0x5651edb90b00 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.867+0000 7f2edc816740 1 -- 172.21.15.43:0/660074822 >> 172.21.15.43:0/660074822 conn(0x5651edb71f60 msgr2=0x5651edb6cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.868+0000 7f2edc816740 1 -- 172.21.15.43:0/660074822 shutdown_connections 2024-09-11T23:22:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.868+0000 7f2edc816740 1 -- 172.21.15.43:0/660074822 wait complete. 2024-09-11T23:22:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.868+0000 7f2edc816740 1 -- start start 2024-09-11T23:22:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.868+0000 7f2edc816740 1 Processor -- start 2024-09-11T23:22:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.868+0000 7f2edc816740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651edb759d0 0x5651edb957d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.868+0000 7f2edc816740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651edb762f0 0x5651edb95d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.868+0000 7f2edc816740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb80bb0 0x5651edb96250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.868+0000 7f2edc816740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5651edb924c0 con 0x5651edb80bb0 2024-09-11T23:22:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.868+0000 7f2edc816740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5651edb91df0 con 0x5651edb759d0 2024-09-11T23:22:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.868+0000 7f2edc816740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5651edb92b90 con 0x5651edb762f0 2024-09-11T23:22:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.868+0000 7f2edc816740 10 client.0 _ll_register_callbacks cb 0x7f2ec800af50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.868+0000 7f2ed37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb80bb0 0x5651edb96250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.868+0000 7f2ed37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb80bb0 0x5651edb96250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36420/0 (socket says 172.21.15.43:36420) 2024-09-11T23:22:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.869+0000 7f2ed37fe640 1 -- 172.21.15.43:0/83598464 learned_addr learned my addr 172.21.15.43:0/83598464 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.869+0000 7f2ed37fe640 1 --2- 172.21.15.43:0/83598464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651edb759d0 0x5651edb957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.869+0000 7f2ed3fff640 1 --2- 172.21.15.43:0/83598464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651edb762f0 0x5651edb95d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.869+0000 7f2ed37fe640 1 -- 172.21.15.43:0/83598464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651edb762f0 msgr2=0x5651edb95d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.869+0000 7f2ed37fe640 1 --2- 172.21.15.43:0/83598464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651edb762f0 0x5651edb95d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.869+0000 7f2ed37fe640 1 -- 172.21.15.43:0/83598464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651edb759d0 msgr2=0x5651edb957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.869+0000 7f2ed37fe640 1 --2- 172.21.15.43:0/83598464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651edb759d0 0x5651edb957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.869+0000 7f2ed37fe640 1 -- 172.21.15.43:0/83598464 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651edbf9ad0 con 0x5651edb80bb0 2024-09-11T23:22:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.869+0000 7f2ed37fe640 1 --2- 172.21.15.43:0/83598464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb80bb0 0x5651edb96250 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2ec80053d0 tx=0x7f2ec8020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123ef8937f67a981 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.869+0000 7f2eb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.869+0000 7f2eb37fe640 1 -- 172.21.15.43:0/83598464 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8004ad0 con 0x5651edb80bb0 2024-09-11T23:22:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.869+0000 7f2edc816740 1 -- 172.21.15.43:0/83598464 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ec8025070 con 0x5651edb80bb0 2024-09-11T23:22:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.869+0000 7f2eb37fe640 1 -- 172.21.15.43:0/83598464 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f2ec8004c70 con 0x5651edb80bb0 2024-09-11T23:22:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.869+0000 7f2eb37fe640 1 -- 172.21.15.43:0/83598464 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2ec801e660 con 0x5651edb80bb0 2024-09-11T23:22:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.869+0000 7f2edc816740 10 client.15423 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.869+0000 7f2edc816740 1 -- 172.21.15.43:0/83598464 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5651edbfad90 con 0x5651edb80bb0 2024-09-11T23:22:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.870+0000 7f2eb37fe640 1 -- 172.21.15.43:0/83598464 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ec8025070 con 0x5651edb80bb0 2024-09-11T23:22:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.870+0000 7f2eb37fe640 1 -- 172.21.15.43:0/83598464 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2ec8045070 con 0x5651edb80bb0 2024-09-11T23:22:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.870+0000 7f2edc816740 10 client.15423 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.870+0000 7f2edc816740 10 client.15423 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T23:22:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.870+0000 7f2edc816740 20 client.15423 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.870+0000 7f2edc816740 1 --2- 172.21.15.43:0/83598464 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5651edbff360 0x5651edbfe070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.870+0000 7f2edc816740 4 client.15423 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:22:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.870+0000 7f2edc816740 1 -- 172.21.15.43:0/83598464 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "status"}) -- 0x5651edae3b10 con 0x5651edbff360 2024-09-11T23:22:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.870+0000 7f2ed3fff640 1 --2- 172.21.15.43:0/83598464 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5651edbff360 0x5651edbfe070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.870+0000 7f2ed3fff640 1 --2- 172.21.15.43:0/83598464 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5651edbff360 0x5651edbfe070 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da334e1953cf102e server_cookie=aee047d251983aeb in_seq=0 out_seq=0 2024-09-11T23:22:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.870+0000 7f2eb37fe640 10 client.15423 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:22:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.871+0000 7f2eb37fe640 1 -- 172.21.15.43:0/83598464 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5651edae3b10 con 0x5651edbff360 2024-09-11T23:22:17.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.871+0000 7f2eb37fe640 10 client.15423 handle_command_reply: tid=0 2024-09-11T23:22:17.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.871+0000 7f2edc816740 1 client.15423 shutdown 2024-09-11T23:22:17.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.871+0000 7f2edc816740 1 -- 172.21.15.43:0/83598464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb80bb0 msgr2=0x5651edb96250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:17.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.871+0000 7f2edc816740 1 --2- 172.21.15.43:0/83598464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb80bb0 0x5651edb96250 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2ec80053d0 tx=0x7f2ec8020040 comp rx=0 tx=0).stop 2024-09-11T23:22:17.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.871+0000 7f2edc816740 1 -- 172.21.15.43:0/83598464 shutdown_connections 2024-09-11T23:22:17.890 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:22:17.890 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "25c0cd84-7093-11ef-bcea-c7b262605968", 2024-09-11T23:22:17.890 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 1, 2024-09-11T23:22:17.890 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24470, 2024-09-11T23:22:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-11T23:22:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-11T23:22:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-11T23:22:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 129.15833252499999, 2024-09-11T23:22:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 24, 2024-09-11T23:22:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 85, 2024-09-11T23:22:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 84, 2024-09-11T23:22:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 160.02086535999999 2024-09-11T23:22:17.891 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:22:17.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.871+0000 7f2edc816740 1 --2- 172.21.15.43:0/83598464 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5651edbff360 0x5651edbfe070 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.871+0000 7f2eb37fe640 0 client.15423 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:22:17.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.871+0000 7f2edc816740 1 --2- 172.21.15.43:0/83598464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651edb80bb0 0x5651edb96250 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.871+0000 7f2edc816740 1 --2- 172.21.15.43:0/83598464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651edb762f0 0x5651edb95d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.871+0000 7f2edc816740 1 --2- 172.21.15.43:0/83598464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651edb759d0 0x5651edb957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.871+0000 7f2edc816740 1 -- 172.21.15.43:0/83598464 >> 172.21.15.43:0/83598464 conn(0x5651edb71f60 msgr2=0x5651edb7bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:17.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.871+0000 7f2edc816740 1 -- 172.21.15.43:0/83598464 shutdown_connections 2024-09-11T23:22:17.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.871+0000 7f2edc816740 1 -- 172.21.15.43:0/83598464 wait complete. 2024-09-11T23:22:17.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.871+0000 7f2edc816740 20 client.15423 trim_cache size 0 max 16384 2024-09-11T23:22:17.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.872+0000 7f2eda43d640 1 -- 172.21.15.43:0/306837242 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f2ebc0a2e10 msgr2=0x7f2ebc0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:17.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.872+0000 7f2eda43d640 1 --2- 172.21.15.43:0/306837242 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f2ebc0a2e10 0x7f2ebc0a52d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0cd410 tx=0x7f2ecc06bf10 comp rx=0 tx=0).stop 2024-09-11T23:22:17.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.872+0000 7f2eda43d640 1 -- 172.21.15.43:0/306837242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed4106ba0 msgr2=0x7f2ed419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:17.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.872+0000 7f2eda43d640 1 --2- 172.21.15.43:0/306837242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed4106ba0 0x7f2ed419cd40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00eb10 tx=0x7f2ecc00c2d0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.872+0000 7f2eda43d640 1 -- 172.21.15.43:0/306837242 shutdown_connections 2024-09-11T23:22:17.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.872+0000 7f2eda43d640 1 --2- 172.21.15.43:0/306837242 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f2ebc0a2e10 0x7f2ebc0a52d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.872+0000 7f2eda43d640 1 --2- 172.21.15.43:0/306837242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed4106ba0 0x7f2ed419cd40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.872+0000 7f2eda43d640 1 --2- 172.21.15.43:0/306837242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed4100a70 0x7f2ed41989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.872+0000 7f2eda43d640 1 --2- 172.21.15.43:0/306837242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ed41000d0 0x7f2ed4198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:17.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.872+0000 7f2eda43d640 1 -- 172.21.15.43:0/306837242 >> 172.21.15.43:0/306837242 conn(0x7f2ed4077210 msgr2=0x7f2ed4104b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:17.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.872+0000 7f2eda43d640 1 -- 172.21.15.43:0/306837242 shutdown_connections 2024-09-11T23:22:17.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:17.872+0000 7f2eda43d640 1 -- 172.21.15.43:0/306837242 wait complete. 2024-09-11T23:22:17.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:17 smithi043 ceph-mon[30950]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T23:22:18.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:17 smithi097 ceph-mon[35562]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T23:22:18.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:17 smithi090 ceph-mon[35130]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T23:22:18.700 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T23:22:18.959 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:19.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.487+0000 7f9ec3e69640 1 -- 172.21.15.43:0/3221304624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ebc106f10 msgr2=0x7f9ebc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:19.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.487+0000 7f9ec3e69640 1 --2- 172.21.15.43:0/3221304624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ebc106f10 0x7f9ebc10ea90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009ca0 tx=0x7f9eb402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.487+0000 7f9ec3e69640 1 -- 172.21.15.43:0/3221304624 shutdown_connections 2024-09-11T23:22:19.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.487+0000 7f9ec3e69640 1 --2- 172.21.15.43:0/3221304624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ebc106f10 0x7f9ebc10ea90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.487+0000 7f9ec3e69640 1 --2- 172.21.15.43:0/3221304624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ebc102790 0x7f9ebc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.487+0000 7f9ec3e69640 1 --2- 172.21.15.43:0/3221304624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ebc101de0 0x7f9ebc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.487+0000 7f9ec3e69640 1 -- 172.21.15.43:0/3221304624 >> 172.21.15.43:0/3221304624 conn(0x7f9ebc0fd8c0 msgr2=0x7f9ebc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:19.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.487+0000 7f9ec3e69640 1 -- 172.21.15.43:0/3221304624 shutdown_connections 2024-09-11T23:22:19.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.487+0000 7f9ec3e69640 1 -- 172.21.15.43:0/3221304624 wait complete. 2024-09-11T23:22:19.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.488+0000 7f9ec3e69640 1 Processor -- start 2024-09-11T23:22:19.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.488+0000 7f9ec3e69640 1 -- start start 2024-09-11T23:22:19.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.488+0000 7f9ec3e69640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ebc101de0 0x7f9ebc198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:19.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.488+0000 7f9ec3e69640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ebc102790 0x7f9ebc1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:19.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.488+0000 7f9ec3e69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ebc106f10 0x7f9ebc19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:19.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.488+0000 7f9ec3e69640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc111f70 con 0x7f9ebc106f10 2024-09-11T23:22:19.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.488+0000 7f9ec3e69640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc111df0 con 0x7f9ebc101de0 2024-09-11T23:22:19.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.488+0000 7f9ec3e69640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc1120f0 con 0x7f9ebc102790 2024-09-11T23:22:19.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.488+0000 7f9ec2406640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ebc106f10 0x7f9ebc19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:19.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.488+0000 7f9ec2406640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ebc106f10 0x7f9ebc19cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36446/0 (socket says 172.21.15.43:36446) 2024-09-11T23:22:19.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.488+0000 7f9ec2406640 1 -- 172.21.15.43:0/1706518768 learned_addr learned my addr 172.21.15.43:0/1706518768 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:19.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.488+0000 7f9ec1c05640 1 --2- 172.21.15.43:0/1706518768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ebc101de0 0x7f9ebc198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:19.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.488+0000 7f9ec1404640 1 --2- 172.21.15.43:0/1706518768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ebc102790 0x7f9ebc1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:19.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.488+0000 7f9ec2406640 1 -- 172.21.15.43:0/1706518768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ebc102790 msgr2=0x7f9ebc1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:19.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.488+0000 7f9ec2406640 1 --2- 172.21.15.43:0/1706518768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ebc102790 0x7f9ebc1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.488+0000 7f9ec2406640 1 -- 172.21.15.43:0/1706518768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ebc101de0 msgr2=0x7f9ebc198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:19.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.488+0000 7f9ec2406640 1 --2- 172.21.15.43:0/1706518768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ebc101de0 0x7f9ebc198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.488+0000 7f9ec2406640 1 -- 172.21.15.43:0/1706518768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ebc19d340 con 0x7f9ebc106f10 2024-09-11T23:22:19.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.488+0000 7f9ec1404640 1 --2- 172.21.15.43:0/1706518768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ebc102790 0x7f9ebc1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:19.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.489+0000 7f9ec2406640 1 --2- 172.21.15.43:0/1706518768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ebc106f10 0x7f9ebc19cc60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4031a60 tx=0x7f9eb4031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:19.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.489+0000 7f9eb2ffd640 1 -- 172.21.15.43:0/1706518768 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4003b70 con 0x7f9ebc106f10 2024-09-11T23:22:19.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.489+0000 7f9eb2ffd640 1 -- 172.21.15.43:0/1706518768 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eb4003d10 con 0x7f9ebc106f10 2024-09-11T23:22:19.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.489+0000 7f9ec3e69640 1 -- 172.21.15.43:0/1706518768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ebc19d5d0 con 0x7f9ebc106f10 2024-09-11T23:22:19.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.489+0000 7f9eb2ffd640 1 -- 172.21.15.43:0/1706518768 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4004770 con 0x7f9ebc106f10 2024-09-11T23:22:19.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.489+0000 7f9ec3e69640 1 -- 172.21.15.43:0/1706518768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ebc1a4f10 con 0x7f9ebc106f10 2024-09-11T23:22:19.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.491+0000 7f9eb2ffd640 1 -- 172.21.15.43:0/1706518768 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f9eb402f790 con 0x7f9ebc106f10 2024-09-11T23:22:19.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.491+0000 7f9ec6242740 1 Processor -- start 2024-09-11T23:22:19.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.491+0000 7f9ec6242740 1 -- start start 2024-09-11T23:22:19.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.492+0000 7f9ec6242740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9e69d0 0x55ec0e9e6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:19.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.492+0000 7f9ec6242740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec0e9e72f0 0x55ec0e9f1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:19.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.492+0000 7f9ec6242740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec0e9f1bb0 0x55ec0e9f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:19.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.492+0000 7f9ec6242740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ec0e94d090 con 0x55ec0e9e69d0 2024-09-11T23:22:19.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.492+0000 7f9ec6242740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ec0e8f4a70 con 0x55ec0e9f1bb0 2024-09-11T23:22:19.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.492+0000 7f9ec6242740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ec0e949050 con 0x55ec0e9e72f0 2024-09-11T23:22:19.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.492+0000 7f9ec1404640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9e69d0 0x55ec0e9e6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:19.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.492+0000 7f9ec1404640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9e69d0 0x55ec0e9e6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36454/0 (socket says 172.21.15.43:36454) 2024-09-11T23:22:19.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.492+0000 7f9ec1404640 1 -- 172.21.15.43:0/2977031138 learned_addr learned my addr 172.21.15.43:0/2977031138 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:19.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.492+0000 7f9eb2ffd640 1 --2- 172.21.15.43:0/1706518768 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9ea80a2e90 0x7f9ea80a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:19.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.492+0000 7f9ec1c05640 1 --2- 172.21.15.43:0/2977031138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec0e9e72f0 0x55ec0e9f1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:19.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.492+0000 7f9ec2406640 1 --2- 172.21.15.43:0/1706518768 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9ea80a2e90 0x7f9ea80a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:19.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.492+0000 7f9ec1404640 1 --2- 172.21.15.43:0/2977031138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec0e9f1bb0 0x55ec0e9f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:19.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.492+0000 7f9ec1404640 1 -- 172.21.15.43:0/2977031138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec0e9e72f0 msgr2=0x55ec0e9f1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:19.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.492+0000 7f9ec1404640 1 --2- 172.21.15.43:0/2977031138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec0e9e72f0 0x55ec0e9f1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.492+0000 7f9eb2ffd640 1 -- 172.21.15.43:0/1706518768 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f9eb40f08e0 con 0x7f9ebc106f10 2024-09-11T23:22:19.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.492+0000 7f9ec1404640 1 -- 172.21.15.43:0/2977031138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec0e9f1bb0 msgr2=0x55ec0e9f3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:19.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.492+0000 7f9ec1404640 1 --2- 172.21.15.43:0/2977031138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec0e9f1bb0 0x55ec0e9f3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.492+0000 7f9ec1404640 1 -- 172.21.15.43:0/2977031138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec0e901130 con 0x55ec0e9e69d0 2024-09-11T23:22:19.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.493+0000 7f9ec1404640 1 --2- 172.21.15.43:0/2977031138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9e69d0 0x55ec0e9e6db0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9eac005d90 tx=0x7f9eac00c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbf1a59d7bb046a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:19.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.493+0000 7f9ec2406640 1 --2- 172.21.15.43:0/1706518768 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9ea80a2e90 0x7f9ea80a5350 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9eb40ee020 tx=0x7f9eb40b2270 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:19.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.493+0000 7f9ea7fff640 1 -- 172.21.15.43:0/2977031138 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac018030 con 0x55ec0e9e69d0 2024-09-11T23:22:19.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.493+0000 7f9ea7fff640 1 -- 172.21.15.43:0/2977031138 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eac005500 con 0x55ec0e9e69d0 2024-09-11T23:22:19.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.493+0000 7f9ea7fff640 1 -- 172.21.15.43:0/2977031138 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0108a0 con 0x55ec0e9e69d0 2024-09-11T23:22:19.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.493+0000 7f9ec6242740 1 -- 172.21.15.43:0/2977031138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9e69d0 msgr2=0x55ec0e9e6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:19.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.493+0000 7f9ec6242740 1 --2- 172.21.15.43:0/2977031138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9e69d0 0x55ec0e9e6db0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9eac005d90 tx=0x7f9eac00c600 comp rx=0 tx=0).stop 2024-09-11T23:22:19.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.494+0000 7f9ec6242740 1 -- 172.21.15.43:0/2977031138 shutdown_connections 2024-09-11T23:22:19.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.494+0000 7f9ec6242740 1 --2- 172.21.15.43:0/2977031138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec0e9f1bb0 0x55ec0e9f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.494+0000 7f9ec6242740 1 --2- 172.21.15.43:0/2977031138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec0e9e72f0 0x55ec0e9f1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.494+0000 7f9ec6242740 1 --2- 172.21.15.43:0/2977031138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9e69d0 0x55ec0e9e6db0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.494+0000 7f9ec6242740 1 -- 172.21.15.43:0/2977031138 >> 172.21.15.43:0/2977031138 conn(0x55ec0e9e2f60 msgr2=0x55ec0e9e3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:19.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.494+0000 7f9ec6242740 1 -- 172.21.15.43:0/2977031138 shutdown_connections 2024-09-11T23:22:19.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.494+0000 7f9ec6242740 1 -- 172.21.15.43:0/2977031138 wait complete. 2024-09-11T23:22:19.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.494+0000 7f9ec6242740 1 -- start start 2024-09-11T23:22:19.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.494+0000 7f9ec6242740 1 Processor -- start 2024-09-11T23:22:19.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.494+0000 7f9ec6242740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec0e9e69d0 0x55ec0e9f1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:19.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.494+0000 7f9ec6242740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9e72f0 0x55ec0e9ee7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:19.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.494+0000 7f9ec6242740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec0e9f1bb0 0x55ec0e9eece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:19.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ec6242740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ec0e9f4780 con 0x55ec0e9e72f0 2024-09-11T23:22:19.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ec6242740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ec0e907f20 con 0x55ec0e9f1bb0 2024-09-11T23:22:19.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ec6242740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ec0e9f4900 con 0x55ec0e9e69d0 2024-09-11T23:22:19.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ec6242740 10 client.0 _ll_register_callbacks cb 0x55ec0e8d1030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:19.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ec1c05640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9e72f0 0x55ec0e9ee7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:19.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ec1c05640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9e72f0 0x55ec0e9ee7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36460/0 (socket says 172.21.15.43:36460) 2024-09-11T23:22:19.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ec1c05640 1 -- 172.21.15.43:0/1935909021 learned_addr learned my addr 172.21.15.43:0/1935909021 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:19.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ec1404640 1 --2- 172.21.15.43:0/1935909021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec0e9e69d0 0x55ec0e9f1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:19.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ec1404640 1 --2- 172.21.15.43:0/1935909021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec0e9f1bb0 0x55ec0e9eece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:19.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ec1c05640 1 -- 172.21.15.43:0/1935909021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec0e9e69d0 msgr2=0x55ec0e9f1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:19.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ec1c05640 1 --2- 172.21.15.43:0/1935909021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec0e9e69d0 0x55ec0e9f1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ec1c05640 1 -- 172.21.15.43:0/1935909021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec0e9f1bb0 msgr2=0x55ec0e9eece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:19.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ec1c05640 1 --2- 172.21.15.43:0/1935909021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec0e9f1bb0 0x55ec0e9eece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ec1c05640 1 -- 172.21.15.43:0/1935909021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec0e6be2b0 con 0x55ec0e9e72f0 2024-09-11T23:22:19.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ec1404640 1 --2- 172.21.15.43:0/1935909021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec0e9f1bb0 0x55ec0e9eece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:22:19.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ec1c05640 1 --2- 172.21.15.43:0/1935909021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9e72f0 0x55ec0e9ee7a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8010a50 tx=0x7f9eb8010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d231220f66a52f13 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:19.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ea57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:19.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ea57fa640 1 -- 172.21.15.43:0/1935909021 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb801e060 con 0x55ec0e9e72f0 2024-09-11T23:22:19.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ea57fa640 1 -- 172.21.15.43:0/1935909021 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f9eb800fbd0 con 0x55ec0e9e72f0 2024-09-11T23:22:19.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ec6242740 1 -- 172.21.15.43:0/1935909021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9eac005500 con 0x55ec0e9e72f0 2024-09-11T23:22:19.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ea57fa640 1 -- 172.21.15.43:0/1935909021 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9eb800fdd0 con 0x55ec0e9e72f0 2024-09-11T23:22:19.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ec6242740 10 client.15435 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:19.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.495+0000 7f9ec6242740 1 -- 172.21.15.43:0/1935909021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ec0ea00100 con 0x55ec0e9e72f0 2024-09-11T23:22:19.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.496+0000 7f9ea57fa640 1 -- 172.21.15.43:0/1935909021 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eac005500 con 0x55ec0e9e72f0 2024-09-11T23:22:19.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.496+0000 7f9ea57fa640 1 -- 172.21.15.43:0/1935909021 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9eb8021570 con 0x55ec0e9e72f0 2024-09-11T23:22:19.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.496+0000 7f9ec6242740 10 client.15435 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:19.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.496+0000 7f9ec6242740 10 client.15435 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:22:19.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.496+0000 7f9ec6242740 20 client.15435 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:19.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.496+0000 7f9ec6242740 1 --2- 172.21.15.43:0/1935909021 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55ec0ea02750 0x55ec0ea03bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:19.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.496+0000 7f9ec6242740 4 client.15435 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:22:19.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.496+0000 7f9ec6242740 1 -- 172.21.15.43:0/1935909021 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec0e95a600 con 0x55ec0ea02750 2024-09-11T23:22:19.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.496+0000 7f9ec1404640 1 --2- 172.21.15.43:0/1935909021 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55ec0ea02750 0x55ec0ea03bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:19.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.497+0000 7f9ea57fa640 10 client.15435 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:19.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.497+0000 7f9ec1404640 1 --2- 172.21.15.43:0/1935909021 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55ec0ea02750 0x55ec0ea03bc0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2b8eaa28988631c server_cookie=e608ba37f5d43a48 in_seq=0 out_seq=0 2024-09-11T23:22:19.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.497+0000 7f9ea57fa640 1 -- 172.21.15.43:0/1935909021 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec0e95a600 con 0x55ec0ea02750 2024-09-11T23:22:19.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.497+0000 7f9ea57fa640 10 client.15435 handle_command_reply: tid=0 2024-09-11T23:22:19.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.497+0000 7f9ec6242740 1 client.15435 shutdown 2024-09-11T23:22:19.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.498+0000 7f9ec6242740 1 -- 172.21.15.43:0/1935909021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9e72f0 msgr2=0x55ec0e9ee7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:19.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.498+0000 7f9ec6242740 1 --2- 172.21.15.43:0/1935909021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9e72f0 0x55ec0e9ee7a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8010a50 tx=0x7f9eb8010f10 comp rx=0 tx=0).stop 2024-09-11T23:22:19.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.498+0000 7f9ec6242740 1 -- 172.21.15.43:0/1935909021 shutdown_connections 2024-09-11T23:22:19.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.498+0000 7f9ec6242740 1 --2- 172.21.15.43:0/1935909021 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55ec0ea02750 0x55ec0ea03bc0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.498+0000 7f9ec6242740 1 --2- 172.21.15.43:0/1935909021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec0e9f1bb0 0x55ec0e9eece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.498+0000 7f9ec6242740 1 --2- 172.21.15.43:0/1935909021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9e72f0 0x55ec0e9ee7a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.498+0000 7f9ec6242740 1 --2- 172.21.15.43:0/1935909021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec0e9e69d0 0x55ec0e9f1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.498+0000 7f9ea57fa640 0 client.15435 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:19.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.498+0000 7f9ec6242740 1 -- 172.21.15.43:0/1935909021 >> 172.21.15.43:0/1935909021 conn(0x55ec0e9e2f60 msgr2=0x55ec0e9e8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:19.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.498+0000 7f9ec6242740 1 -- 172.21.15.43:0/1935909021 shutdown_connections 2024-09-11T23:22:19.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.498+0000 7f9ec6242740 1 -- 172.21.15.43:0/1935909021 wait complete. 2024-09-11T23:22:19.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.498+0000 7f9ec6242740 20 client.15435 trim_cache size 0 max 16384 2024-09-11T23:22:19.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.516+0000 7f9ec6242740 1 Processor -- start 2024-09-11T23:22:19.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.516+0000 7f9ec6242740 1 -- start start 2024-09-11T23:22:19.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.516+0000 7f9ec6242740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec0e9e69d0 0x55ec0ea03bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:19.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.516+0000 7f9ec6242740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec0e9e72f0 0x55ec0e9eba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.516+0000 7f9ec6242740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9f1bb0 0x55ec0e9ebf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.516+0000 7f9ec6242740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ec0ea01f00 con 0x55ec0e9f1bb0 2024-09-11T23:22:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.516+0000 7f9ec6242740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ec0ea025d0 con 0x55ec0e9e72f0 2024-09-11T23:22:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.516+0000 7f9ec6242740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ec0ea02ca0 con 0x55ec0e9e69d0 2024-09-11T23:22:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.516+0000 7f9ec1404640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9f1bb0 0x55ec0e9ebf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.516+0000 7f9ec1404640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9f1bb0 0x55ec0e9ebf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36462/0 (socket says 172.21.15.43:36462) 2024-09-11T23:22:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.516+0000 7f9ec1404640 1 -- 172.21.15.43:0/2665709687 learned_addr learned my addr 172.21.15.43:0/2665709687 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.516+0000 7f9ec1c05640 1 --2- 172.21.15.43:0/2665709687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec0e9e72f0 0x55ec0e9eba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.516+0000 7f9ec1404640 1 --2- 172.21.15.43:0/2665709687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec0e9e69d0 0x55ec0ea03bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.516+0000 7f9ec1404640 1 -- 172.21.15.43:0/2665709687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec0e9e69d0 msgr2=0x55ec0ea03bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.516+0000 7f9ec1404640 1 --2- 172.21.15.43:0/2665709687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec0e9e69d0 0x55ec0ea03bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.516+0000 7f9ec1404640 1 -- 172.21.15.43:0/2665709687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec0e9e72f0 msgr2=0x55ec0e9eba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.516+0000 7f9ec1404640 1 --2- 172.21.15.43:0/2665709687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec0e9e72f0 0x55ec0e9eba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.516+0000 7f9ec1404640 1 -- 172.21.15.43:0/2665709687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec0e9f1070 con 0x55ec0e9f1bb0 2024-09-11T23:22:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.517+0000 7f9ec1404640 1 --2- 172.21.15.43:0/2665709687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9f1bb0 0x55ec0e9ebf60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9eac012eb0 tx=0x7f9eac00c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185ab2077351a74d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.517+0000 7f9ea77fe640 1 -- 172.21.15.43:0/2665709687 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac018040 con 0x55ec0e9f1bb0 2024-09-11T23:22:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.517+0000 7f9ea77fe640 1 -- 172.21.15.43:0/2665709687 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eac019040 con 0x55ec0e9f1bb0 2024-09-11T23:22:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.517+0000 7f9ea77fe640 1 -- 172.21.15.43:0/2665709687 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0053f0 con 0x55ec0e9f1bb0 2024-09-11T23:22:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.517+0000 7f9ec6242740 1 -- 172.21.15.43:0/2665709687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9f1bb0 msgr2=0x55ec0e9ebf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.517+0000 7f9ec6242740 1 --2- 172.21.15.43:0/2665709687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9f1bb0 0x55ec0e9ebf60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9eac012eb0 tx=0x7f9eac00c600 comp rx=0 tx=0).stop 2024-09-11T23:22:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.517+0000 7f9ec6242740 1 -- 172.21.15.43:0/2665709687 shutdown_connections 2024-09-11T23:22:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.517+0000 7f9ec6242740 1 --2- 172.21.15.43:0/2665709687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9f1bb0 0x55ec0e9ebf60 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.517+0000 7f9ec6242740 1 --2- 172.21.15.43:0/2665709687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec0e9e72f0 0x55ec0e9eba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.517+0000 7f9ec6242740 1 --2- 172.21.15.43:0/2665709687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec0e9e69d0 0x55ec0ea03bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.517+0000 7f9ec6242740 1 -- 172.21.15.43:0/2665709687 >> 172.21.15.43:0/2665709687 conn(0x55ec0e9e2f60 msgr2=0x55ec0e9ed070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.517+0000 7f9ec6242740 1 -- 172.21.15.43:0/2665709687 shutdown_connections 2024-09-11T23:22:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.518+0000 7f9ec6242740 1 -- 172.21.15.43:0/2665709687 wait complete. 2024-09-11T23:22:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.518+0000 7f9ec6242740 1 -- start start 2024-09-11T23:22:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.518+0000 7f9ec6242740 1 Processor -- start 2024-09-11T23:22:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.518+0000 7f9ec6242740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec0e9e69d0 0x55ec0ea0c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.518+0000 7f9ec6242740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec0e9e72f0 0x55ec0ea0c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.518+0000 7f9ec6242740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9f1bb0 0x55ec0ea057a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.518+0000 7f9ec6242740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ec0ea03a40 con 0x55ec0e9f1bb0 2024-09-11T23:22:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.518+0000 7f9ec6242740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ec0ea03370 con 0x55ec0e9e69d0 2024-09-11T23:22:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.518+0000 7f9ec6242740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ec0ea04110 con 0x55ec0e9e72f0 2024-09-11T23:22:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.518+0000 7f9ec6242740 10 client.0 _ll_register_callbacks cb 0x7f9eb800a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.518+0000 7f9ec1404640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9f1bb0 0x55ec0ea057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.519+0000 7f9ec1404640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9f1bb0 0x55ec0ea057a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36466/0 (socket says 172.21.15.43:36466) 2024-09-11T23:22:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.519+0000 7f9ec1404640 1 -- 172.21.15.43:0/1363263260 learned_addr learned my addr 172.21.15.43:0/1363263260 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.519+0000 7f9ec1404640 1 --2- 172.21.15.43:0/1363263260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec0e9e69d0 0x55ec0ea0c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.519+0000 7f9ec1c05640 1 --2- 172.21.15.43:0/1363263260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec0e9e72f0 0x55ec0ea0c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.519+0000 7f9ec1404640 1 -- 172.21.15.43:0/1363263260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec0e9e72f0 msgr2=0x55ec0ea0c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.519+0000 7f9ec1404640 1 --2- 172.21.15.43:0/1363263260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec0e9e72f0 0x55ec0ea0c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.519+0000 7f9ec1404640 1 -- 172.21.15.43:0/1363263260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec0e9e69d0 msgr2=0x55ec0ea0c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.519+0000 7f9ec1404640 1 --2- 172.21.15.43:0/1363263260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec0e9e69d0 0x55ec0ea0c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.519+0000 7f9ec1404640 1 -- 172.21.15.43:0/1363263260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec0ea05fd0 con 0x55ec0e9f1bb0 2024-09-11T23:22:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.519+0000 7f9ec1404640 1 --2- 172.21.15.43:0/1363263260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9f1bb0 0x55ec0ea057a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00cf90 tx=0x7f9eac014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9579812a64e188 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.519+0000 7f9ea57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.519+0000 7f9ea57fa640 1 -- 172.21.15.43:0/1363263260 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac018070 con 0x55ec0e9f1bb0 2024-09-11T23:22:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.519+0000 7f9ea57fa640 1 -- 172.21.15.43:0/1363263260 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f9eac004220 con 0x55ec0e9f1bb0 2024-09-11T23:22:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.519+0000 7f9ec6242740 1 -- 172.21.15.43:0/1363263260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9eac019040 con 0x55ec0e9f1bb0 2024-09-11T23:22:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.519+0000 7f9ea57fa640 1 -- 172.21.15.43:0/1363263260 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9eac004420 con 0x55ec0e9f1bb0 2024-09-11T23:22:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.520+0000 7f9ec6242740 10 client.15441 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.520+0000 7f9ec6242740 1 -- 172.21.15.43:0/1363263260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ec0ea0a9e0 con 0x55ec0e9f1bb0 2024-09-11T23:22:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.520+0000 7f9ea57fa640 1 -- 172.21.15.43:0/1363263260 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eac019040 con 0x55ec0e9f1bb0 2024-09-11T23:22:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.520+0000 7f9ea57fa640 1 -- 172.21.15.43:0/1363263260 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9eac010b40 con 0x55ec0e9f1bb0 2024-09-11T23:22:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.520+0000 7f9ec6242740 10 client.15441 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.520+0000 7f9ec6242740 10 client.15441 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T23:22:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.520+0000 7f9ec6242740 20 client.15441 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.520+0000 7f9ec6242740 1 --2- 172.21.15.43:0/1363263260 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55ec0ea70f30 0x55ec0ea6fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.520+0000 7f9ec6242740 4 client.15441 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T23:22:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.520+0000 7f9ec6242740 1 -- 172.21.15.43:0/1363263260 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "status"}) -- 0x55ec0e954b10 con 0x55ec0ea70f30 2024-09-11T23:22:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.520+0000 7f9ec1c05640 1 --2- 172.21.15.43:0/1363263260 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55ec0ea70f30 0x55ec0ea6fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.521+0000 7f9ec1c05640 1 --2- 172.21.15.43:0/1363263260 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55ec0ea70f30 0x55ec0ea6fc40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e91609fba7c49b server_cookie=fa4e52e56f1339ba in_seq=0 out_seq=0 2024-09-11T23:22:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.521+0000 7f9ea57fa640 10 client.15441 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.521+0000 7f9ea57fa640 1 -- 172.21.15.43:0/1363263260 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55ec0e954b10 con 0x55ec0ea70f30 2024-09-11T23:22:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.521+0000 7f9ea57fa640 10 client.15441 handle_command_reply: tid=0 2024-09-11T23:22:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.521+0000 7f9ec6242740 1 client.15441 shutdown 2024-09-11T23:22:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.521+0000 7f9ec6242740 1 -- 172.21.15.43:0/1363263260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9f1bb0 msgr2=0x55ec0ea057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.521+0000 7f9ec6242740 1 --2- 172.21.15.43:0/1363263260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9f1bb0 0x55ec0ea057a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00cf90 tx=0x7f9eac014040 comp rx=0 tx=0).stop 2024-09-11T23:22:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.521+0000 7f9ec6242740 1 -- 172.21.15.43:0/1363263260 shutdown_connections 2024-09-11T23:22:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.521+0000 7f9ec6242740 1 --2- 172.21.15.43:0/1363263260 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55ec0ea70f30 0x55ec0ea6fc40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.521+0000 7f9ec6242740 1 --2- 172.21.15.43:0/1363263260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ec0e9f1bb0 0x55ec0ea057a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.521+0000 7f9ec6242740 1 --2- 172.21.15.43:0/1363263260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ec0e9e72f0 0x55ec0ea0c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.521+0000 7f9ec6242740 1 --2- 172.21.15.43:0/1363263260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec0e9e69d0 0x55ec0ea0c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.521+0000 7f9ea57fa640 0 client.15441 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.521+0000 7f9ec6242740 1 -- 172.21.15.43:0/1363263260 >> 172.21.15.43:0/1363263260 conn(0x55ec0e9e2f60 msgr2=0x55ec0e9efe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.521+0000 7f9ec6242740 1 -- 172.21.15.43:0/1363263260 shutdown_connections 2024-09-11T23:22:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.521+0000 7f9ec6242740 1 -- 172.21.15.43:0/1363263260 wait complete. 2024-09-11T23:22:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.521+0000 7f9ec6242740 20 client.15441 trim_cache size 0 max 16384 2024-09-11T23:22:19.533 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:22:19.534 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "25c0cd84-7093-11ef-bcea-c7b262605968", 2024-09-11T23:22:19.534 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-09-11T23:22:19.534 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24497, 2024-09-11T23:22:19.534 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-09-11T23:22:19.534 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-09-11T23:22:19.534 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-09-11T23:22:19.535 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 138.785852718, 2024-09-11T23:22:19.535 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 24, 2024-09-11T23:22:19.535 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 85, 2024-09-11T23:22:19.535 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 84, 2024-09-11T23:22:19.535 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 149.806977472 2024-09-11T23:22:19.536 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:22:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.522+0000 7f9ec3e69640 1 -- 172.21.15.43:0/1706518768 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9ea80a2e90 msgr2=0x7f9ea80a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.522+0000 7f9ec3e69640 1 --2- 172.21.15.43:0/1706518768 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9ea80a2e90 0x7f9ea80a5350 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9eb40ee020 tx=0x7f9eb40b2270 comp rx=0 tx=0).stop 2024-09-11T23:22:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.522+0000 7f9ec3e69640 1 -- 172.21.15.43:0/1706518768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ebc106f10 msgr2=0x7f9ebc19cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.522+0000 7f9ec3e69640 1 --2- 172.21.15.43:0/1706518768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ebc106f10 0x7f9ebc19cc60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4031a60 tx=0x7f9eb4031a90 comp rx=0 tx=0).stop 2024-09-11T23:22:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.522+0000 7f9ec3e69640 1 -- 172.21.15.43:0/1706518768 shutdown_connections 2024-09-11T23:22:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.522+0000 7f9ec3e69640 1 --2- 172.21.15.43:0/1706518768 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9ea80a2e90 0x7f9ea80a5350 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.522+0000 7f9ec3e69640 1 --2- 172.21.15.43:0/1706518768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ebc106f10 0x7f9ebc19cc60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.522+0000 7f9ec3e69640 1 --2- 172.21.15.43:0/1706518768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ebc102790 0x7f9ebc1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.522+0000 7f9ec3e69640 1 --2- 172.21.15.43:0/1706518768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ebc101de0 0x7f9ebc198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.522+0000 7f9ec3e69640 1 -- 172.21.15.43:0/1706518768 >> 172.21.15.43:0/1706518768 conn(0x7f9ebc0fd8c0 msgr2=0x7f9ebc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.522+0000 7f9ec3e69640 1 -- 172.21.15.43:0/1706518768 shutdown_connections 2024-09-11T23:22:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:19.522+0000 7f9ec3e69640 1 -- 172.21.15.43:0/1706518768 wait complete. 2024-09-11T23:22:20.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:19 smithi097 ceph-mon[35562]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.9 KiB/s wr, 14 op/s 2024-09-11T23:22:20.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:19 smithi043 ceph-mon[30950]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.9 KiB/s wr, 14 op/s 2024-09-11T23:22:20.200 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-11T23:22:20.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:19 smithi090 ceph-mon[35130]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.9 KiB/s wr, 14 op/s 2024-09-11T23:22:22.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:21 smithi097 ceph-mon[35562]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-11T23:22:22.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:21 smithi043 ceph-mon[30950]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-11T23:22:22.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:21 smithi090 ceph-mon[35130]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-11T23:22:24.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:23 smithi097 ceph-mon[35562]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-09-11T23:22:24.683 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:23 smithi090 ceph-mon[35130]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-09-11T23:22:24.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:23 smithi043 ceph-mon[30950]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-09-11T23:22:26.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:25 smithi097 ceph-mon[35562]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-11T23:22:26.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:25 smithi043 ceph-mon[30950]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-11T23:22:26.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:25 smithi090 ceph-mon[35130]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-11T23:22:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.938+0000 7f26d996d640 1 -- 172.21.15.43:0/85945768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26d410c0a0 msgr2=0x7f26d410e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.938+0000 7f26d996d640 1 --2- 172.21.15.43:0/85945768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26d410c0a0 0x7f26d410e540 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009ca0 tx=0x7f26cc02a950 comp rx=0 tx=0).stop 2024-09-11T23:22:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.938+0000 7f26d996d640 1 -- 172.21.15.43:0/85945768 shutdown_connections 2024-09-11T23:22:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.938+0000 7f26d996d640 1 --2- 172.21.15.43:0/85945768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26d410c0a0 0x7f26d410e540 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.938+0000 7f26d996d640 1 --2- 172.21.15.43:0/85945768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26d4103680 0x7f26d410bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.938+0000 7f26d996d640 1 --2- 172.21.15.43:0/85945768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26d4102d60 0x7f26d4103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.938+0000 7f26d996d640 1 -- 172.21.15.43:0/85945768 >> 172.21.15.43:0/85945768 conn(0x7f26d40779a0 msgr2=0x7f26d40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.938+0000 7f26d996d640 1 -- 172.21.15.43:0/85945768 shutdown_connections 2024-09-11T23:22:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.938+0000 7f26d996d640 1 -- 172.21.15.43:0/85945768 wait complete. 2024-09-11T23:22:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.939+0000 7f26d996d640 1 Processor -- start 2024-09-11T23:22:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.939+0000 7f26d996d640 1 -- start start 2024-09-11T23:22:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.940+0000 7f26d996d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26d4102d60 0x7f26d4198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.940+0000 7f26d996d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26d4103680 0x7f26d41987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.940+0000 7f26d996d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26d410c0a0 0x7f26d419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.940+0000 7f26d996d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26d4111ea0 con 0x7f26d410c0a0 2024-09-11T23:22:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.940+0000 7f26d996d640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f26d4111d20 con 0x7f26d4103680 2024-09-11T23:22:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.940+0000 7f26d996d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f26d4112020 con 0x7f26d4102d60 2024-09-11T23:22:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.940+0000 7f26d37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26d410c0a0 0x7f26d419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.940+0000 7f26d37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26d410c0a0 0x7f26d419cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53968/0 (socket says 172.21.15.43:53968) 2024-09-11T23:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.940+0000 7f26d37fe640 1 -- 172.21.15.43:0/2073039532 learned_addr learned my addr 172.21.15.43:0/2073039532 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.940+0000 7f26d2ffd640 1 --2- 172.21.15.43:0/2073039532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26d4102d60 0x7f26d4198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.940+0000 7f26d27fc640 1 --2- 172.21.15.43:0/2073039532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26d4103680 0x7f26d41987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.940+0000 7f26d37fe640 1 -- 172.21.15.43:0/2073039532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26d4102d60 msgr2=0x7f26d4198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.940+0000 7f26d37fe640 1 --2- 172.21.15.43:0/2073039532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26d4102d60 0x7f26d4198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.940+0000 7f26d37fe640 1 -- 172.21.15.43:0/2073039532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26d4103680 msgr2=0x7f26d41987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.940+0000 7f26d37fe640 1 --2- 172.21.15.43:0/2073039532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26d4103680 0x7f26d41987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.940+0000 7f26d37fe640 1 -- 172.21.15.43:0/2073039532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d419d260 con 0x7f26d410c0a0 2024-09-11T23:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.940+0000 7f26d27fc640 1 --2- 172.21.15.43:0/2073039532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26d4103680 0x7f26d41987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.940+0000 7f26d37fe640 1 --2- 172.21.15.43:0/2073039532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26d410c0a0 0x7f26d419cb30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f26cc02ae70 tx=0x7f26cc02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.941+0000 7f26d896b640 1 -- 172.21.15.43:0/2073039532 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc034040 con 0x7f26d410c0a0 2024-09-11T23:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.941+0000 7f26d996d640 1 -- 172.21.15.43:0/2073039532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d419d4f0 con 0x7f26d410c0a0 2024-09-11T23:22:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.941+0000 7f26d896b640 1 -- 172.21.15.43:0/2073039532 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26cc030920 con 0x7f26d410c0a0 2024-09-11T23:22:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.941+0000 7f26d896b640 1 -- 172.21.15.43:0/2073039532 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc045480 con 0x7f26d410c0a0 2024-09-11T23:22:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.941+0000 7f26d996d640 1 -- 172.21.15.43:0/2073039532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d41a4de0 con 0x7f26d410c0a0 2024-09-11T23:22:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.944+0000 7f26d896b640 1 -- 172.21.15.43:0/2073039532 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f26cc037020 con 0x7f26d410c0a0 2024-09-11T23:22:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.944+0000 7f26dc7e9740 1 Processor -- start 2024-09-11T23:22:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.944+0000 7f26dc7e9740 1 -- start start 2024-09-11T23:22:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.944+0000 7f26dc7e9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e70952af0 0x561e70952ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.944+0000 7f26dc7e9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70953410 0x561e7095d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.944+0000 7f26dc7e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e7095dcb0 0x561e709600a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.944+0000 7f26dc7e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561e708b72f0 con 0x561e7095dcb0 2024-09-11T23:22:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.944+0000 7f26dc7e9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561e7085f270 con 0x561e70953410 2024-09-11T23:22:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.944+0000 7f26dc7e9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561e708b32b0 con 0x561e70952af0 2024-09-11T23:22:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.944+0000 7f26d27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e7095dcb0 0x561e709600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.944+0000 7f26d27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e7095dcb0 0x561e709600a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53984/0 (socket says 172.21.15.43:53984) 2024-09-11T23:22:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.944+0000 7f26d27fc640 1 -- 172.21.15.43:0/1974814214 learned_addr learned my addr 172.21.15.43:0/1974814214 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.945+0000 7f26d2ffd640 1 --2- 172.21.15.43:0/1974814214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70953410 0x561e7095d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.945+0000 7f26d27fc640 1 --2- 172.21.15.43:0/1974814214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e70952af0 0x561e70952ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.945+0000 7f26d896b640 1 --2- 172.21.15.43:0/2073039532 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f26bc0a2e10 0x7f26bc0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.945+0000 7f26d27fc640 1 -- 172.21.15.43:0/1974814214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e70952af0 msgr2=0x561e70952ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.945+0000 7f26d27fc640 1 --2- 172.21.15.43:0/1974814214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e70952af0 0x561e70952ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.945+0000 7f26d27fc640 1 -- 172.21.15.43:0/1974814214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70953410 msgr2=0x561e7095d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.945+0000 7f26d27fc640 1 --2- 172.21.15.43:0/1974814214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70953410 0x561e7095d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.945+0000 7f26d27fc640 1 -- 172.21.15.43:0/1974814214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e7085e340 con 0x561e7095dcb0 2024-09-11T23:22:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.945+0000 7f26d37fe640 1 --2- 172.21.15.43:0/2073039532 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f26bc0a2e10 0x7f26bc0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.945+0000 7f26d27fc640 1 --2- 172.21.15.43:0/1974814214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e7095dcb0 0x561e709600a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f26c800aee0 tx=0x7f26c800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd750201a3fe43c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.945+0000 7f26d896b640 1 -- 172.21.15.43:0/2073039532 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f26cc0f0500 con 0x7f26d410c0a0 2024-09-11T23:22:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.945+0000 7f26b9ffb640 1 -- 172.21.15.43:0/1974814214 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8005220 con 0x561e7095dcb0 2024-09-11T23:22:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.945+0000 7f26b9ffb640 1 -- 172.21.15.43:0/1974814214 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c8016070 con 0x561e7095dcb0 2024-09-11T23:22:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.945+0000 7f26b9ffb640 1 -- 172.21.15.43:0/1974814214 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c80128a0 con 0x561e7095dcb0 2024-09-11T23:22:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.946+0000 7f26dc7e9740 1 -- 172.21.15.43:0/1974814214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e7095dcb0 msgr2=0x561e709600a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.946+0000 7f26d37fe640 1 --2- 172.21.15.43:0/2073039532 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f26bc0a2e10 0x7f26bc0a52d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f26cc0edc40 tx=0x7f26cc0b8540 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.946+0000 7f26dc7e9740 1 --2- 172.21.15.43:0/1974814214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e7095dcb0 0x561e709600a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f26c800aee0 tx=0x7f26c800a650 comp rx=0 tx=0).stop 2024-09-11T23:22:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.946+0000 7f26dc7e9740 1 -- 172.21.15.43:0/1974814214 shutdown_connections 2024-09-11T23:22:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.946+0000 7f26dc7e9740 1 --2- 172.21.15.43:0/1974814214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e7095dcb0 0x561e709600a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.946+0000 7f26dc7e9740 1 --2- 172.21.15.43:0/1974814214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70953410 0x561e7095d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.946+0000 7f26dc7e9740 1 --2- 172.21.15.43:0/1974814214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e70952af0 0x561e70952ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.946+0000 7f26dc7e9740 1 -- 172.21.15.43:0/1974814214 >> 172.21.15.43:0/1974814214 conn(0x561e7094f060 msgr2=0x561e7094f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.946+0000 7f26dc7e9740 1 -- 172.21.15.43:0/1974814214 shutdown_connections 2024-09-11T23:22:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.946+0000 7f26dc7e9740 1 -- 172.21.15.43:0/1974814214 wait complete. 2024-09-11T23:22:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.946+0000 7f26dc7e9740 1 -- start start 2024-09-11T23:22:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.947+0000 7f26dc7e9740 1 Processor -- start 2024-09-11T23:22:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.947+0000 7f26dc7e9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70952af0 0x561e7095abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.947+0000 7f26dc7e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e70953410 0x561e7095b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.947+0000 7f26dc7e9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e7095dcb0 0x561e7095b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.947+0000 7f26dc7e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561e70960880 con 0x561e70953410 2024-09-11T23:22:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.947+0000 7f26dc7e9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561e70840290 con 0x561e70952af0 2024-09-11T23:22:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.947+0000 7f26dc7e9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561e70960a00 con 0x561e7095dcb0 2024-09-11T23:22:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.947+0000 7f26dc7e9740 10 client.0 _ll_register_callbacks cb 0x561e70871140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.947+0000 7f26d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e70953410 0x561e7095b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.947+0000 7f26d2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e70953410 0x561e7095b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53990/0 (socket says 172.21.15.43:53990) 2024-09-11T23:22:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.947+0000 7f26d2ffd640 1 -- 172.21.15.43:0/2385256188 learned_addr learned my addr 172.21.15.43:0/2385256188 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.947+0000 7f26d27fc640 1 --2- 172.21.15.43:0/2385256188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e7095dcb0 0x561e7095b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.947+0000 7f26d27fc640 1 --2- 172.21.15.43:0/2385256188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70952af0 0x561e7095abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.947+0000 7f26d2ffd640 1 -- 172.21.15.43:0/2385256188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e7095dcb0 msgr2=0x561e7095b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.947+0000 7f26d2ffd640 1 --2- 172.21.15.43:0/2385256188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e7095dcb0 0x561e7095b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.947+0000 7f26d2ffd640 1 -- 172.21.15.43:0/2385256188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70952af0 msgr2=0x561e7095abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.947+0000 7f26d2ffd640 1 --2- 172.21.15.43:0/2385256188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70952af0 0x561e7095abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.947+0000 7f26d2ffd640 1 -- 172.21.15.43:0/2385256188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e705ca210 con 0x561e70953410 2024-09-11T23:22:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.947+0000 7f26d27fc640 1 --2- 172.21.15.43:0/2385256188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e7095dcb0 0x561e7095b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.948+0000 7f26d2ffd640 1 --2- 172.21.15.43:0/2385256188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e70953410 0x561e7095b100 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f26c400ba20 tx=0x7f26c400bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ff665743370f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.948+0000 7f26b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.948+0000 7f26b2ffd640 1 -- 172.21.15.43:0/2385256188 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c401b640 con 0x561e70953410 2024-09-11T23:22:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.948+0000 7f26b2ffd640 1 -- 172.21.15.43:0/2385256188 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f26c401b7e0 con 0x561e70953410 2024-09-11T23:22:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.948+0000 7f26dc7e9740 1 -- 172.21.15.43:0/2385256188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26c8016070 con 0x561e70953410 2024-09-11T23:22:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.948+0000 7f26b2ffd640 1 -- 172.21.15.43:0/2385256188 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f26c401b9e0 con 0x561e70953410 2024-09-11T23:22:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.948+0000 7f26dc7e9740 10 client.15453 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.948+0000 7f26dc7e9740 1 -- 172.21.15.43:0/2385256188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561e70696620 con 0x561e70953410 2024-09-11T23:22:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.948+0000 7f26b2ffd640 1 -- 172.21.15.43:0/2385256188 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c8016070 con 0x561e70953410 2024-09-11T23:22:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.948+0000 7f26b2ffd640 1 -- 172.21.15.43:0/2385256188 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f26c4019020 con 0x561e70953410 2024-09-11T23:22:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.948+0000 7f26dc7e9740 10 client.15453 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.948+0000 7f26dc7e9740 10 client.15453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.948+0000 7f26dc7e9740 20 client.15453 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.948+0000 7f26dc7e9740 1 --2- 172.21.15.43:0/2385256188 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561e7096dc60 0x561e70970050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.948+0000 7f26dc7e9740 4 client.15453 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:22:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.948+0000 7f26dc7e9740 1 -- 172.21.15.43:0/2385256188 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e708c4a20 con 0x561e7096dc60 2024-09-11T23:22:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.949+0000 7f26d27fc640 1 --2- 172.21.15.43:0/2385256188 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561e7096dc60 0x561e70970050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.949+0000 7f26d27fc640 1 --2- 172.21.15.43:0/2385256188 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561e7096dc60 0x561e70970050 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ec1560392e8b36c server_cookie=2f39195f5e7de7d8 in_seq=0 out_seq=0 2024-09-11T23:22:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.949+0000 7f26b2ffd640 10 client.15453 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.950+0000 7f26b2ffd640 1 -- 172.21.15.43:0/2385256188 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e708c4a20 con 0x561e7096dc60 2024-09-11T23:22:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.950+0000 7f26b2ffd640 10 client.15453 handle_command_reply: tid=0 2024-09-11T23:22:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.950+0000 7f26dc7e9740 1 client.15453 shutdown 2024-09-11T23:22:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.950+0000 7f26dc7e9740 1 -- 172.21.15.43:0/2385256188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e70953410 msgr2=0x561e7095b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.950+0000 7f26dc7e9740 1 --2- 172.21.15.43:0/2385256188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e70953410 0x561e7095b100 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f26c400ba20 tx=0x7f26c400bee0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.950+0000 7f26dc7e9740 1 -- 172.21.15.43:0/2385256188 shutdown_connections 2024-09-11T23:22:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.950+0000 7f26dc7e9740 1 --2- 172.21.15.43:0/2385256188 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561e7096dc60 0x561e70970050 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.950+0000 7f26dc7e9740 1 --2- 172.21.15.43:0/2385256188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e7095dcb0 0x561e7095b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.950+0000 7f26b2ffd640 0 client.15453 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.950+0000 7f26dc7e9740 1 --2- 172.21.15.43:0/2385256188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e70953410 0x561e7095b100 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.950+0000 7f26dc7e9740 1 --2- 172.21.15.43:0/2385256188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70952af0 0x561e7095abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.950+0000 7f26dc7e9740 1 -- 172.21.15.43:0/2385256188 >> 172.21.15.43:0/2385256188 conn(0x561e7094f060 msgr2=0x561e70954d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.950+0000 7f26dc7e9740 1 -- 172.21.15.43:0/2385256188 shutdown_connections 2024-09-11T23:22:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.950+0000 7f26dc7e9740 1 -- 172.21.15.43:0/2385256188 wait complete. 2024-09-11T23:22:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.950+0000 7f26dc7e9740 20 client.15453 trim_cache size 0 max 16384 2024-09-11T23:22:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.965+0000 7f26dc7e9740 1 Processor -- start 2024-09-11T23:22:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.965+0000 7f26dc7e9740 1 -- start start 2024-09-11T23:22:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.965+0000 7f26dc7e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e70952af0 0x561e7095b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.965+0000 7f26dc7e9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70953410 0x561e70975960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.965+0000 7f26dc7e9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e7095dcb0 0x561e70975ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.965+0000 7f26dc7e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41b0040 con 0x561e70952af0 2024-09-11T23:22:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.965+0000 7f26dc7e9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41ad910 con 0x561e70953410 2024-09-11T23:22:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.965+0000 7f26dc7e9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41ab760 con 0x561e7095dcb0 2024-09-11T23:22:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.965+0000 7f26d27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e70952af0 0x561e7095b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.965+0000 7f26d27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e70952af0 0x561e7095b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53994/0 (socket says 172.21.15.43:53994) 2024-09-11T23:22:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.965+0000 7f26d27fc640 1 -- 172.21.15.43:0/348320759 learned_addr learned my addr 172.21.15.43:0/348320759 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.965+0000 7f26d2ffd640 1 --2- 172.21.15.43:0/348320759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70953410 0x561e70975960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.965+0000 7f26d27fc640 1 -- 172.21.15.43:0/348320759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e7095dcb0 msgr2=0x561e70975ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.965+0000 7f26d27fc640 1 --2- 172.21.15.43:0/348320759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e7095dcb0 0x561e70975ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.965+0000 7f26d27fc640 1 -- 172.21.15.43:0/348320759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70953410 msgr2=0x561e70975960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.965+0000 7f26d27fc640 1 --2- 172.21.15.43:0/348320759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70953410 0x561e70975960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.966+0000 7f26d27fc640 1 -- 172.21.15.43:0/348320759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e7094ee90 con 0x561e70952af0 2024-09-11T23:22:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.966+0000 7f26d2ffd640 1 --2- 172.21.15.43:0/348320759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70953410 0x561e70975960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:22:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.966+0000 7f26d27fc640 1 --2- 172.21.15.43:0/348320759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e70952af0 0x561e7095b470 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f26c80055b0 tx=0x7f26c800ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d29c970b389a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.966+0000 7f26b97fa640 1 -- 172.21.15.43:0/348320759 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c800a5d0 con 0x561e70952af0 2024-09-11T23:22:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.966+0000 7f26b97fa640 1 -- 172.21.15.43:0/348320759 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c8016040 con 0x561e70952af0 2024-09-11T23:22:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.966+0000 7f26b97fa640 1 -- 172.21.15.43:0/348320759 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8012660 con 0x561e70952af0 2024-09-11T23:22:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.966+0000 7f26dc7e9740 1 -- 172.21.15.43:0/348320759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e70952af0 msgr2=0x561e7095b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.966+0000 7f26dc7e9740 1 --2- 172.21.15.43:0/348320759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e70952af0 0x561e7095b470 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f26c80055b0 tx=0x7f26c800ac90 comp rx=0 tx=0).stop 2024-09-11T23:22:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.967+0000 7f26dc7e9740 1 -- 172.21.15.43:0/348320759 shutdown_connections 2024-09-11T23:22:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.967+0000 7f26dc7e9740 1 --2- 172.21.15.43:0/348320759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e7095dcb0 0x561e70975ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.967+0000 7f26dc7e9740 1 --2- 172.21.15.43:0/348320759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70953410 0x561e70975960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.967+0000 7f26dc7e9740 1 --2- 172.21.15.43:0/348320759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e70952af0 0x561e7095b470 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.967+0000 7f26dc7e9740 1 -- 172.21.15.43:0/348320759 >> 172.21.15.43:0/348320759 conn(0x561e7094f060 msgr2=0x561e7095cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.967+0000 7f26dc7e9740 1 -- 172.21.15.43:0/348320759 shutdown_connections 2024-09-11T23:22:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.967+0000 7f26dc7e9740 1 -- 172.21.15.43:0/348320759 wait complete. 2024-09-11T23:22:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.967+0000 7f26dc7e9740 1 -- start start 2024-09-11T23:22:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.967+0000 7f26dc7e9740 1 Processor -- start 2024-09-11T23:22:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.967+0000 7f26dc7e9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e70952af0 0x561e709663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.967+0000 7f26dc7e9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70953410 0x561e709616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.967+0000 7f26dc7e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e7095dcb0 0x561e70961c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.967+0000 7f26dc7e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41abdd0 con 0x561e7095dcb0 2024-09-11T23:22:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.967+0000 7f26dc7e9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41ad240 con 0x561e70953410 2024-09-11T23:22:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.967+0000 7f26dc7e9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f26d41ac4a0 con 0x561e70952af0 2024-09-11T23:22:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.968+0000 7f26dc7e9740 10 client.0 _ll_register_callbacks cb 0x7f26c4004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.968+0000 7f26d27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e7095dcb0 0x561e70961c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.968+0000 7f26d2ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70953410 0x561e709616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.968+0000 7f26d27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e7095dcb0 0x561e70961c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54002/0 (socket says 172.21.15.43:54002) 2024-09-11T23:22:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.968+0000 7f26d2ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70953410 0x561e709616e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:37414/0 (socket says 172.21.15.43:37414) 2024-09-11T23:22:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.968+0000 7f26d27fc640 1 -- 172.21.15.43:0/1236538697 learned_addr learned my addr 172.21.15.43:0/1236538697 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.968+0000 7f26d27fc640 1 --2- 172.21.15.43:0/1236538697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e70952af0 0x561e709663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.968+0000 7f26d27fc640 1 -- 172.21.15.43:0/1236538697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e70952af0 msgr2=0x561e709663c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.968+0000 7f26d27fc640 1 --2- 172.21.15.43:0/1236538697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e70952af0 0x561e709663c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.968+0000 7f26d27fc640 1 -- 172.21.15.43:0/1236538697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70953410 msgr2=0x561e709616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.968+0000 7f26d27fc640 1 --2- 172.21.15.43:0/1236538697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70953410 0x561e709616e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.968+0000 7f26d27fc640 1 -- 172.21.15.43:0/1236538697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e70962400 con 0x561e7095dcb0 2024-09-11T23:22:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.968+0000 7f26d27fc640 1 --2- 172.21.15.43:0/1236538697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e7095dcb0 0x561e70961c20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f26c8011010 tx=0x7f26c802a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8057cd8a07915715 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.968+0000 7f26b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.968+0000 7f26b2ffd640 1 -- 172.21.15.43:0/1236538697 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8004680 con 0x561e7095dcb0 2024-09-11T23:22:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.968+0000 7f26b2ffd640 1 -- 172.21.15.43:0/1236538697 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f26c802d070 con 0x561e7095dcb0 2024-09-11T23:22:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.968+0000 7f26dc7e9740 1 -- 172.21.15.43:0/1236538697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26c8016040 con 0x561e7095dcb0 2024-09-11T23:22:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.969+0000 7f26b2ffd640 1 -- 172.21.15.43:0/1236538697 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f26c8004820 con 0x561e7095dcb0 2024-09-11T23:22:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.969+0000 7f26dc7e9740 10 client.15459 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.969+0000 7f26dc7e9740 1 -- 172.21.15.43:0/1236538697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561e709d8860 con 0x561e7095dcb0 2024-09-11T23:22:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.969+0000 7f26b2ffd640 1 -- 172.21.15.43:0/1236538697 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c8016040 con 0x561e7095dcb0 2024-09-11T23:22:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.969+0000 7f26b2ffd640 1 -- 172.21.15.43:0/1236538697 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f26c800f040 con 0x561e7095dcb0 2024-09-11T23:22:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.969+0000 7f26dc7e9740 10 client.15459 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.969+0000 7f26dc7e9740 10 client.15459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.969+0000 7f26dc7e9740 20 client.15459 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.969+0000 7f26dc7e9740 1 --2- 172.21.15.43:0/1236538697 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561e709d8cd0 0x561e709db0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.969+0000 7f26dc7e9740 4 client.15459 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:22:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.969+0000 7f26dc7e9740 1 -- 172.21.15.43:0/1236538697 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e708bed70 con 0x561e709d8cd0 2024-09-11T23:22:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.969+0000 7f26d2ffd640 1 --2- 172.21.15.43:0/1236538697 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561e709d8cd0 0x561e709db0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.970+0000 7f26d2ffd640 1 --2- 172.21.15.43:0/1236538697 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561e709d8cd0 0x561e709db0c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69771adeef2f95e3 server_cookie=dc1d6f35306df83d in_seq=0 out_seq=0 2024-09-11T23:22:27.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.970+0000 7f26b2ffd640 10 client.15459 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.977+0000 7f26b2ffd640 1 -- 172.21.15.43:0/1236538697 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e708bed70 con 0x561e709d8cd0 2024-09-11T23:22:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.977+0000 7f26b2ffd640 10 client.15459 handle_command_reply: tid=0 2024-09-11T23:22:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.977+0000 7f26dc7e9740 1 client.15459 shutdown 2024-09-11T23:22:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.977+0000 7f26dc7e9740 1 -- 172.21.15.43:0/1236538697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e7095dcb0 msgr2=0x561e70961c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.977+0000 7f26dc7e9740 1 --2- 172.21.15.43:0/1236538697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e7095dcb0 0x561e70961c20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f26c8011010 tx=0x7f26c802a040 comp rx=0 tx=0).stop 2024-09-11T23:22:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.977+0000 7f26dc7e9740 1 -- 172.21.15.43:0/1236538697 shutdown_connections 2024-09-11T23:22:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.977+0000 7f26dc7e9740 1 --2- 172.21.15.43:0/1236538697 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561e709d8cd0 0x561e709db0c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.977+0000 7f26dc7e9740 1 --2- 172.21.15.43:0/1236538697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e7095dcb0 0x561e70961c20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.977+0000 7f26dc7e9740 1 --2- 172.21.15.43:0/1236538697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e70953410 0x561e709616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.977+0000 7f26dc7e9740 1 --2- 172.21.15.43:0/1236538697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e70952af0 0x561e709663c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.977+0000 7f26b2ffd640 0 client.15459 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.977+0000 7f26dc7e9740 1 -- 172.21.15.43:0/1236538697 >> 172.21.15.43:0/1236538697 conn(0x561e7094f060 msgr2=0x561e70954550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.977+0000 7f26dc7e9740 1 -- 172.21.15.43:0/1236538697 shutdown_connections 2024-09-11T23:22:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.977+0000 7f26dc7e9740 1 -- 172.21.15.43:0/1236538697 wait complete. 2024-09-11T23:22:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.977+0000 7f26dc7e9740 20 client.15459 trim_cache size 0 max 16384 2024-09-11T23:22:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:22:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:22:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:22:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.978+0000 7f26d996d640 1 -- 172.21.15.43:0/2073039532 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f26bc0a2e10 msgr2=0x7f26bc0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.978+0000 7f26d996d640 1 --2- 172.21.15.43:0/2073039532 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f26bc0a2e10 0x7f26bc0a52d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f26cc0edc40 tx=0x7f26cc0b8540 comp rx=0 tx=0).stop 2024-09-11T23:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.978+0000 7f26d996d640 1 -- 172.21.15.43:0/2073039532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26d410c0a0 msgr2=0x7f26d419cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.978+0000 7f26d996d640 1 --2- 172.21.15.43:0/2073039532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26d410c0a0 0x7f26d419cb30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f26cc02ae70 tx=0x7f26cc02f5d0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.978+0000 7f26d996d640 1 -- 172.21.15.43:0/2073039532 shutdown_connections 2024-09-11T23:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.978+0000 7f26d996d640 1 --2- 172.21.15.43:0/2073039532 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f26bc0a2e10 0x7f26bc0a52d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.978+0000 7f26d996d640 1 --2- 172.21.15.43:0/2073039532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26d410c0a0 0x7f26d419cb30 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.978+0000 7f26d996d640 1 --2- 172.21.15.43:0/2073039532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26d4103680 0x7f26d41987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.978+0000 7f26d996d640 1 --2- 172.21.15.43:0/2073039532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26d4102d60 0x7f26d4198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.978+0000 7f26d996d640 1 -- 172.21.15.43:0/2073039532 >> 172.21.15.43:0/2073039532 conn(0x7f26d40779a0 msgr2=0x7f26d40fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.978+0000 7f26d996d640 1 -- 172.21.15.43:0/2073039532 shutdown_connections 2024-09-11T23:22:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:27.978+0000 7f26d996d640 1 -- 172.21.15.43:0/2073039532 wait complete. 2024-09-11T23:22:28.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:27 smithi097 ceph-mon[35562]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-11T23:22:28.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:27 smithi043 ceph-mon[30950]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-11T23:22:28.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:27 smithi090 ceph-mon[35130]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-11T23:22:30.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:29 smithi097 ceph-mon[35562]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-09-11T23:22:30.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:29 smithi043 ceph-mon[30950]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-09-11T23:22:30.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:29 smithi090 ceph-mon[35130]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-09-11T23:22:32.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:31 smithi097 ceph-mon[35562]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:22:32.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:31 smithi043 ceph-mon[30950]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:22:32.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:31 smithi090 ceph-mon[35130]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:22:32.344 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:22:32.608 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:33.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.132+0000 7f32560e7640 1 -- 172.21.15.43:0/2871655241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3250103780 msgr2=0x7f3250107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:33.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.132+0000 7f32560e7640 1 --2- 172.21.15.43:0/2871655241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3250103780 0x7f3250107bc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3244009930 tx=0x7f324402a420 comp rx=0 tx=0).stop 2024-09-11T23:22:33.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.132+0000 7f32560e7640 1 -- 172.21.15.43:0/2871655241 shutdown_connections 2024-09-11T23:22:33.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.132+0000 7f32560e7640 1 --2- 172.21.15.43:0/2871655241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3250108300 0x7f325010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.132+0000 7f32560e7640 1 --2- 172.21.15.43:0/2871655241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3250103780 0x7f3250107bc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.132+0000 7f32560e7640 1 --2- 172.21.15.43:0/2871655241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3250102dd0 0x7f32501031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.132+0000 7f32560e7640 1 -- 172.21.15.43:0/2871655241 >> 172.21.15.43:0/2871655241 conn(0x7f32500fc7d0 msgr2=0x7f32500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:33.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.133+0000 7f32560e7640 1 -- 172.21.15.43:0/2871655241 shutdown_connections 2024-09-11T23:22:33.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.133+0000 7f32560e7640 1 -- 172.21.15.43:0/2871655241 wait complete. 2024-09-11T23:22:33.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.133+0000 7f32560e7640 1 Processor -- start 2024-09-11T23:22:33.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.133+0000 7f32560e7640 1 -- start start 2024-09-11T23:22:33.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.134+0000 7f32560e7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3250102dd0 0x7f32501985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.134+0000 7f32560e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3250103780 0x7f3250198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.134+0000 7f32560e7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3250108300 0x7f325019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.134+0000 7f32560e7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3250112070 con 0x7f3250103780 2024-09-11T23:22:33.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.134+0000 7f32560e7640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3250111ef0 con 0x7f3250102dd0 2024-09-11T23:22:33.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.134+0000 7f32560e7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f32501121f0 con 0x7f3250108300 2024-09-11T23:22:33.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.134+0000 7f324effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3250103780 0x7f3250198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:33.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.134+0000 7f324effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3250103780 0x7f3250198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60336/0 (socket says 172.21.15.43:60336) 2024-09-11T23:22:33.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.134+0000 7f324effd640 1 -- 172.21.15.43:0/734535672 learned_addr learned my addr 172.21.15.43:0/734535672 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:33.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.134+0000 7f324ffff640 1 --2- 172.21.15.43:0/734535672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3250108300 0x7f325019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:33.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.134+0000 7f324f7fe640 1 --2- 172.21.15.43:0/734535672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3250102dd0 0x7f32501985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:33.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.134+0000 7f324effd640 1 -- 172.21.15.43:0/734535672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3250108300 msgr2=0x7f325019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:33.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.134+0000 7f324effd640 1 --2- 172.21.15.43:0/734535672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3250108300 0x7f325019cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.134+0000 7f324effd640 1 -- 172.21.15.43:0/734535672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3250102dd0 msgr2=0x7f32501985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:33.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.134+0000 7f324effd640 1 --2- 172.21.15.43:0/734535672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3250102dd0 0x7f32501985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.134+0000 7f324effd640 1 -- 172.21.15.43:0/734535672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f325019d5a0 con 0x7f3250103780 2024-09-11T23:22:33.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.134+0000 7f324effd640 1 --2- 172.21.15.43:0/734535672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3250103780 0x7f3250198b30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3244005940 tx=0x7f32440040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:33.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.135+0000 7f324cff9640 1 -- 172.21.15.43:0/734535672 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324403f030 con 0x7f3250103780 2024-09-11T23:22:33.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.135+0000 7f32560e7640 1 -- 172.21.15.43:0/734535672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f325019d830 con 0x7f3250103780 2024-09-11T23:22:33.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.135+0000 7f324cff9640 1 -- 172.21.15.43:0/734535672 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32440044c0 con 0x7f3250103780 2024-09-11T23:22:33.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.135+0000 7f324cff9640 1 -- 172.21.15.43:0/734535672 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244035070 con 0x7f3250103780 2024-09-11T23:22:33.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.135+0000 7f32560e7640 1 -- 172.21.15.43:0/734535672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3250100960 con 0x7f3250103780 2024-09-11T23:22:33.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.137+0000 7f32584c0740 1 Processor -- start 2024-09-11T23:22:33.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.137+0000 7f324cff9640 1 -- 172.21.15.43:0/734535672 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f3244042030 con 0x7f3250103780 2024-09-11T23:22:33.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.137+0000 7f32584c0740 1 -- start start 2024-09-11T23:22:33.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.138+0000 7f32584c0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce844a0 0x55f58ce84880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.138+0000 7f32584c0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f58ce84dc0 0x55f58ce8f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.138+0000 7f32584c0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce8f680 0x55f58ce91a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.138+0000 7f32584c0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f58cdea2e0 con 0x55f58ce8f680 2024-09-11T23:22:33.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.138+0000 7f32584c0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f58cd91ce0 con 0x55f58ce844a0 2024-09-11T23:22:33.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.138+0000 7f32584c0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f58cde62a0 con 0x55f58ce84dc0 2024-09-11T23:22:33.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.138+0000 7f324effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce8f680 0x55f58ce91a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:33.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.138+0000 7f324effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce8f680 0x55f58ce91a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60352/0 (socket says 172.21.15.43:60352) 2024-09-11T23:22:33.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.138+0000 7f324effd640 1 -- 172.21.15.43:0/4015456414 learned_addr learned my addr 172.21.15.43:0/4015456414 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:33.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.138+0000 7f324ffff640 1 --2- 172.21.15.43:0/4015456414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce844a0 0x55f58ce84880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:33.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.138+0000 7f324f7fe640 1 --2- 172.21.15.43:0/4015456414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f58ce84dc0 0x55f58ce8f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:33.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.138+0000 7f324cff9640 1 --2- 172.21.15.43:0/734535672 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f32380a7220 0x7f32380a96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.138+0000 7f324ffff640 1 -- 172.21.15.43:0/4015456414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f58ce84dc0 msgr2=0x55f58ce8f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:33.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.138+0000 7f324ffff640 1 --2- 172.21.15.43:0/4015456414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f58ce84dc0 0x55f58ce8f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.138+0000 7f324ffff640 1 -- 172.21.15.43:0/4015456414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce8f680 msgr2=0x55f58ce91a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:33.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.138+0000 7f324ffff640 1 --2- 172.21.15.43:0/4015456414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce8f680 0x55f58ce91a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.138+0000 7f324ffff640 1 -- 172.21.15.43:0/4015456414 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f58cd76440 con 0x55f58ce844a0 2024-09-11T23:22:33.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.138+0000 7f324effd640 1 --2- 172.21.15.43:0/4015456414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce8f680 0x55f58ce91a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:33.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.138+0000 7f324cff9640 1 -- 172.21.15.43:0/734535672 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f32440ef650 con 0x7f3250103780 2024-09-11T23:22:33.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.138+0000 7f324ffff640 1 --2- 172.21.15.43:0/734535672 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f32380a7220 0x7f32380a96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:33.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.139+0000 7f324ffff640 1 --2- 172.21.15.43:0/4015456414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce844a0 0x55f58ce84880 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f32480103f0 tx=0x7f32480108b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7055526da1d9979f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:33.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.139+0000 7f324ffff640 1 --2- 172.21.15.43:0/734535672 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f32380a7220 0x7f32380a96e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3248021000 tx=0x7f324801e000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:33.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.139+0000 7f3235ffb640 1 -- 172.21.15.43:0/4015456414 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3248024070 con 0x55f58ce844a0 2024-09-11T23:22:33.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.139+0000 7f3235ffb640 1 -- 172.21.15.43:0/4015456414 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f324801e960 con 0x55f58ce844a0 2024-09-11T23:22:33.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.139+0000 7f3235ffb640 1 -- 172.21.15.43:0/4015456414 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324801c7b0 con 0x55f58ce844a0 2024-09-11T23:22:33.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.140+0000 7f32584c0740 1 -- 172.21.15.43:0/4015456414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce844a0 msgr2=0x55f58ce84880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:33.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.140+0000 7f32584c0740 1 --2- 172.21.15.43:0/4015456414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce844a0 0x55f58ce84880 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f32480103f0 tx=0x7f32480108b0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.140+0000 7f32584c0740 1 -- 172.21.15.43:0/4015456414 shutdown_connections 2024-09-11T23:22:33.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.140+0000 7f32584c0740 1 --2- 172.21.15.43:0/4015456414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce8f680 0x55f58ce91a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.140+0000 7f32584c0740 1 --2- 172.21.15.43:0/4015456414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f58ce84dc0 0x55f58ce8f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.140+0000 7f32584c0740 1 --2- 172.21.15.43:0/4015456414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce844a0 0x55f58ce84880 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.140+0000 7f32584c0740 1 -- 172.21.15.43:0/4015456414 >> 172.21.15.43:0/4015456414 conn(0x55f58ce80a30 msgr2=0x55f58ce80e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:33.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.140+0000 7f32584c0740 1 -- 172.21.15.43:0/4015456414 shutdown_connections 2024-09-11T23:22:33.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.140+0000 7f32584c0740 1 -- 172.21.15.43:0/4015456414 wait complete. 2024-09-11T23:22:33.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.140+0000 7f32584c0740 1 -- start start 2024-09-11T23:22:33.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.141+0000 7f32584c0740 1 Processor -- start 2024-09-11T23:22:33.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.141+0000 7f32584c0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce844a0 0x55f58ce8f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.141+0000 7f32584c0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce84dc0 0x55f58ce8c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.141+0000 7f32584c0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f58ce8f680 0x55f58ce8c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.141+0000 7f32584c0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f58ce92250 con 0x55f58ce844a0 2024-09-11T23:22:33.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.141+0000 7f32584c0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f58cda5050 con 0x55f58ce84dc0 2024-09-11T23:22:33.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.141+0000 7f32584c0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f58ce923d0 con 0x55f58ce8f680 2024-09-11T23:22:33.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.141+0000 7f32584c0740 10 client.0 _ll_register_callbacks cb 0x55f58cde5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:33.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.141+0000 7f324f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce844a0 0x55f58ce8f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:33.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.141+0000 7f324f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce844a0 0x55f58ce8f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60368/0 (socket says 172.21.15.43:60368) 2024-09-11T23:22:33.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.141+0000 7f324f7fe640 1 -- 172.21.15.43:0/367961621 learned_addr learned my addr 172.21.15.43:0/367961621 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:33.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.141+0000 7f324effd640 1 --2- 172.21.15.43:0/367961621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce84dc0 0x55f58ce8c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:33.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.142+0000 7f324f7fe640 1 -- 172.21.15.43:0/367961621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f58ce8f680 msgr2=0x55f58ce8c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:33.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.142+0000 7f324f7fe640 1 --2- 172.21.15.43:0/367961621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f58ce8f680 0x55f58ce8c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.142+0000 7f324f7fe640 1 -- 172.21.15.43:0/367961621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce84dc0 msgr2=0x55f58ce8c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:33.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.142+0000 7f324f7fe640 1 --2- 172.21.15.43:0/367961621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce84dc0 0x55f58ce8c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.142+0000 7f324f7fe640 1 -- 172.21.15.43:0/367961621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f58cb5b3c0 con 0x55f58ce844a0 2024-09-11T23:22:33.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.142+0000 7f324effd640 1 --2- 172.21.15.43:0/367961621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce84dc0 0x55f58ce8c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:33.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.142+0000 7f324f7fe640 1 --2- 172.21.15.43:0/367961621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce844a0 0x55f58ce8f030 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f32400126e0 tx=0x7f3240012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d38cc06a2d405a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:33.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.142+0000 7f322effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:33.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.142+0000 7f322effd640 1 -- 172.21.15.43:0/367961621 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3240012190 con 0x55f58ce844a0 2024-09-11T23:22:33.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.142+0000 7f322effd640 1 -- 172.21.15.43:0/367961621 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f324000f920 con 0x55f58ce844a0 2024-09-11T23:22:33.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.142+0000 7f32584c0740 1 -- 172.21.15.43:0/367961621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f324801e960 con 0x55f58ce844a0 2024-09-11T23:22:33.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.142+0000 7f322effd640 1 -- 172.21.15.43:0/367961621 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f324000fb20 con 0x55f58ce844a0 2024-09-11T23:22:33.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.142+0000 7f32584c0740 10 client.15471 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:33.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.142+0000 7f32584c0740 1 -- 172.21.15.43:0/367961621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f58ce9dcc0 con 0x55f58ce844a0 2024-09-11T23:22:33.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.142+0000 7f322effd640 1 -- 172.21.15.43:0/367961621 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f324801e960 con 0x55f58ce844a0 2024-09-11T23:22:33.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.143+0000 7f322effd640 1 -- 172.21.15.43:0/367961621 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f324001c420 con 0x55f58ce844a0 2024-09-11T23:22:33.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.143+0000 7f32584c0740 10 client.15471 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:33.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.143+0000 7f32584c0740 10 client.15471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:33.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.143+0000 7f32584c0740 20 client.15471 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:33.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.143+0000 7f32584c0740 1 --2- 172.21.15.43:0/367961621 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f58ce9f180 0x55f58cea1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.143+0000 7f32584c0740 4 client.15471 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:22:33.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.143+0000 7f32584c0740 1 -- 172.21.15.43:0/367961621 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f58cdf7850 con 0x55f58ce9f180 2024-09-11T23:22:33.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.143+0000 7f324effd640 1 --2- 172.21.15.43:0/367961621 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f58ce9f180 0x55f58cea1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:33.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.143+0000 7f324effd640 1 --2- 172.21.15.43:0/367961621 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f58ce9f180 0x55f58cea1640 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcf23c7f92fdcc01 server_cookie=b4d2054fbd703fcb in_seq=0 out_seq=0 2024-09-11T23:22:33.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.143+0000 7f322effd640 10 client.15471 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:33.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.144+0000 7f322effd640 1 -- 172.21.15.43:0/367961621 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f58cdf7850 con 0x55f58ce9f180 2024-09-11T23:22:33.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.144+0000 7f322effd640 10 client.15471 handle_command_reply: tid=0 2024-09-11T23:22:33.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.144+0000 7f32584c0740 1 client.15471 shutdown 2024-09-11T23:22:33.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.144+0000 7f32584c0740 1 -- 172.21.15.43:0/367961621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce844a0 msgr2=0x55f58ce8f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:33.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.144+0000 7f32584c0740 1 --2- 172.21.15.43:0/367961621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce844a0 0x55f58ce8f030 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f32400126e0 tx=0x7f3240012ba0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.144+0000 7f32584c0740 1 -- 172.21.15.43:0/367961621 shutdown_connections 2024-09-11T23:22:33.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.145+0000 7f32584c0740 1 --2- 172.21.15.43:0/367961621 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f58ce9f180 0x55f58cea1640 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.145+0000 7f32584c0740 1 --2- 172.21.15.43:0/367961621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f58ce8f680 0x55f58ce8c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.145+0000 7f322effd640 0 client.15471 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:33.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.145+0000 7f32584c0740 1 --2- 172.21.15.43:0/367961621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce84dc0 0x55f58ce8c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.145+0000 7f32584c0740 1 --2- 172.21.15.43:0/367961621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce844a0 0x55f58ce8f030 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.145+0000 7f32584c0740 1 -- 172.21.15.43:0/367961621 >> 172.21.15.43:0/367961621 conn(0x55f58ce80a30 msgr2=0x55f58ce86710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:33.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.145+0000 7f32584c0740 1 -- 172.21.15.43:0/367961621 shutdown_connections 2024-09-11T23:22:33.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.145+0000 7f32584c0740 1 -- 172.21.15.43:0/367961621 wait complete. 2024-09-11T23:22:33.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.145+0000 7f32584c0740 20 client.15471 trim_cache size 0 max 16384 2024-09-11T23:22:33.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.163+0000 7f32584c0740 1 Processor -- start 2024-09-11T23:22:33.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.163+0000 7f32584c0740 1 -- start start 2024-09-11T23:22:33.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.163+0000 7f32584c0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f58ce844a0 0x55f58ce9b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.163+0000 7f32584c0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce84dc0 0x55f58ce8abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.163+0000 7f32584c0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce8f680 0x55f58cea6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.163+0000 7f32584c0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f32501ae050 con 0x55f58ce8f680 2024-09-11T23:22:33.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.163+0000 7f32584c0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f32501ab920 con 0x55f58ce84dc0 2024-09-11T23:22:33.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.163+0000 7f32584c0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f32501a9770 con 0x55f58ce844a0 2024-09-11T23:22:33.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.163+0000 7f324ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce8f680 0x55f58cea6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:33.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.163+0000 7f324ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce8f680 0x55f58cea6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60380/0 (socket says 172.21.15.43:60380) 2024-09-11T23:22:33.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.163+0000 7f324ffff640 1 -- 172.21.15.43:0/3463112006 learned_addr learned my addr 172.21.15.43:0/3463112006 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:33.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.163+0000 7f324effd640 1 --2- 172.21.15.43:0/3463112006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce84dc0 0x55f58ce8abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:33.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.163+0000 7f324f7fe640 1 --2- 172.21.15.43:0/3463112006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f58ce844a0 0x55f58ce9b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:33.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.163+0000 7f324ffff640 1 -- 172.21.15.43:0/3463112006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f58ce844a0 msgr2=0x55f58ce9b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:33.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.163+0000 7f324ffff640 1 --2- 172.21.15.43:0/3463112006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f58ce844a0 0x55f58ce9b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.163+0000 7f324ffff640 1 -- 172.21.15.43:0/3463112006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce84dc0 msgr2=0x55f58ce8abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:33.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.163+0000 7f324ffff640 1 --2- 172.21.15.43:0/3463112006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce84dc0 0x55f58ce8abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.163+0000 7f324ffff640 1 -- 172.21.15.43:0/3463112006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f58ce8b290 con 0x55f58ce8f680 2024-09-11T23:22:33.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.163+0000 7f324effd640 1 --2- 172.21.15.43:0/3463112006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce84dc0 0x55f58ce8abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:33.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.164+0000 7f324ffff640 1 --2- 172.21.15.43:0/3463112006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce8f680 0x55f58cea6f60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f32480039e0 tx=0x7f3248009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32147587f20f47fb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:33.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.164+0000 7f32357fa640 1 -- 172.21.15.43:0/3463112006 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3248024070 con 0x55f58ce8f680 2024-09-11T23:22:33.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.164+0000 7f32357fa640 1 -- 172.21.15.43:0/3463112006 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3248013c60 con 0x55f58ce8f680 2024-09-11T23:22:33.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.164+0000 7f32357fa640 1 -- 172.21.15.43:0/3463112006 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324801eac0 con 0x55f58ce8f680 2024-09-11T23:22:33.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.165+0000 7f32584c0740 1 -- 172.21.15.43:0/3463112006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce8f680 msgr2=0x55f58cea6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:33.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.165+0000 7f32584c0740 1 --2- 172.21.15.43:0/3463112006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce8f680 0x55f58cea6f60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f32480039e0 tx=0x7f3248009f40 comp rx=0 tx=0).stop 2024-09-11T23:22:33.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.165+0000 7f32584c0740 1 -- 172.21.15.43:0/3463112006 shutdown_connections 2024-09-11T23:22:33.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.165+0000 7f32584c0740 1 --2- 172.21.15.43:0/3463112006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce8f680 0x55f58cea6f60 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.165+0000 7f32584c0740 1 --2- 172.21.15.43:0/3463112006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce84dc0 0x55f58ce8abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.165+0000 7f32584c0740 1 --2- 172.21.15.43:0/3463112006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f58ce844a0 0x55f58ce9b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.165+0000 7f32584c0740 1 -- 172.21.15.43:0/3463112006 >> 172.21.15.43:0/3463112006 conn(0x55f58ce80a30 msgr2=0x55f58ce87900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:33.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.165+0000 7f32584c0740 1 -- 172.21.15.43:0/3463112006 shutdown_connections 2024-09-11T23:22:33.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.165+0000 7f32584c0740 1 -- 172.21.15.43:0/3463112006 wait complete. 2024-09-11T23:22:33.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.165+0000 7f32584c0740 1 -- start start 2024-09-11T23:22:33.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.166+0000 7f32584c0740 1 Processor -- start 2024-09-11T23:22:33.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.166+0000 7f32584c0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce844a0 0x55f58ce97620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.166+0000 7f32584c0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f58ce84dc0 0x55f58ce97b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.166+0000 7f32584c0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce8f680 0x55f58ce945f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.166+0000 7f32584c0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f32501a9de0 con 0x55f58ce844a0 2024-09-11T23:22:33.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.166+0000 7f32584c0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f32501ab250 con 0x55f58ce8f680 2024-09-11T23:22:33.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.166+0000 7f32584c0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f32501aa4b0 con 0x55f58ce84dc0 2024-09-11T23:22:33.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.166+0000 7f32584c0740 10 client.0 _ll_register_callbacks cb 0x7f3240009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:33.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.166+0000 7f324f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce844a0 0x55f58ce97620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:33.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.166+0000 7f324f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce844a0 0x55f58ce97620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60396/0 (socket says 172.21.15.43:60396) 2024-09-11T23:22:33.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.166+0000 7f324f7fe640 1 -- 172.21.15.43:0/2295008387 learned_addr learned my addr 172.21.15.43:0/2295008387 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:33.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.166+0000 7f324effd640 1 --2- 172.21.15.43:0/2295008387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f58ce84dc0 0x55f58ce97b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:33.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.166+0000 7f324ffff640 1 --2- 172.21.15.43:0/2295008387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce8f680 0x55f58ce945f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:33.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.166+0000 7f324f7fe640 1 -- 172.21.15.43:0/2295008387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f58ce84dc0 msgr2=0x55f58ce97b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:33.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.166+0000 7f324f7fe640 1 --2- 172.21.15.43:0/2295008387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f58ce84dc0 0x55f58ce97b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.166+0000 7f324f7fe640 1 -- 172.21.15.43:0/2295008387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce8f680 msgr2=0x55f58ce945f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:33.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.166+0000 7f324f7fe640 1 --2- 172.21.15.43:0/2295008387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce8f680 0x55f58ce945f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.166+0000 7f324f7fe640 1 -- 172.21.15.43:0/2295008387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f58ce8cbf0 con 0x55f58ce844a0 2024-09-11T23:22:33.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.166+0000 7f324ffff640 1 --2- 172.21.15.43:0/2295008387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce8f680 0x55f58ce945f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:22:33.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.167+0000 7f324f7fe640 1 --2- 172.21.15.43:0/2295008387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce844a0 0x55f58ce97620 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f32400126e0 tx=0x7f324000f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b95b22b93a049c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:33.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.167+0000 7f322effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:33.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.167+0000 7f322effd640 1 -- 172.21.15.43:0/2295008387 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324001cad0 con 0x55f58ce844a0 2024-09-11T23:22:33.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.167+0000 7f32584c0740 1 -- 172.21.15.43:0/2295008387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3248013c60 con 0x55f58ce844a0 2024-09-11T23:22:33.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.167+0000 7f322effd640 1 -- 172.21.15.43:0/2295008387 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f324001cc70 con 0x55f58ce844a0 2024-09-11T23:22:33.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.167+0000 7f322effd640 1 -- 172.21.15.43:0/2295008387 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3240033050 con 0x55f58ce844a0 2024-09-11T23:22:33.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.167+0000 7f32584c0740 10 client.15477 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:33.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.167+0000 7f32584c0740 1 -- 172.21.15.43:0/2295008387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f58ce8f3a0 con 0x55f58ce844a0 2024-09-11T23:22:33.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.167+0000 7f322effd640 1 -- 172.21.15.43:0/2295008387 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3248013c60 con 0x55f58ce844a0 2024-09-11T23:22:33.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.168+0000 7f322effd640 1 -- 172.21.15.43:0/2295008387 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f324002b020 con 0x55f58ce844a0 2024-09-11T23:22:33.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.168+0000 7f32584c0740 10 client.15477 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:33.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.168+0000 7f32584c0740 10 client.15477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:33.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.168+0000 7f32584c0740 20 client.15477 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:33.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.168+0000 7f32584c0740 1 --2- 172.21.15.43:0/2295008387 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f58ce9f180 0x55f58cf0c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:33.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.168+0000 7f32584c0740 4 client.15477 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:22:33.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.168+0000 7f32584c0740 1 -- 172.21.15.43:0/2295008387 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f58cdf1d60 con 0x55f58ce9f180 2024-09-11T23:22:33.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.168+0000 7f324effd640 1 --2- 172.21.15.43:0/2295008387 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f58ce9f180 0x55f58cf0c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:33.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.168+0000 7f324effd640 1 --2- 172.21.15.43:0/2295008387 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f58ce9f180 0x55f58cf0c910 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a93ecaa68ddc826c server_cookie=b947f09edeefb634 in_seq=0 out_seq=0 2024-09-11T23:22:33.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.168+0000 7f322effd640 10 client.15477 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:33.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.169+0000 7f322effd640 1 -- 172.21.15.43:0/2295008387 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f58cdf1d60 con 0x55f58ce9f180 2024-09-11T23:22:33.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.169+0000 7f322effd640 10 client.15477 handle_command_reply: tid=0 2024-09-11T23:22:33.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.169+0000 7f32584c0740 1 client.15477 shutdown 2024-09-11T23:22:33.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.169+0000 7f32584c0740 1 -- 172.21.15.43:0/2295008387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce844a0 msgr2=0x55f58ce97620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:33.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.169+0000 7f32584c0740 1 --2- 172.21.15.43:0/2295008387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce844a0 0x55f58ce97620 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f32400126e0 tx=0x7f324000f920 comp rx=0 tx=0).stop 2024-09-11T23:22:33.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.169+0000 7f32584c0740 1 -- 172.21.15.43:0/2295008387 shutdown_connections 2024-09-11T23:22:33.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.169+0000 7f32584c0740 1 --2- 172.21.15.43:0/2295008387 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f58ce9f180 0x55f58cf0c910 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.169+0000 7f32584c0740 1 --2- 172.21.15.43:0/2295008387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f58ce8f680 0x55f58ce945f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.169+0000 7f32584c0740 1 --2- 172.21.15.43:0/2295008387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f58ce84dc0 0x55f58ce97b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.169+0000 7f322effd640 0 client.15477 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:33.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.169+0000 7f32584c0740 1 --2- 172.21.15.43:0/2295008387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f58ce844a0 0x55f58ce97620 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.169+0000 7f32584c0740 1 -- 172.21.15.43:0/2295008387 >> 172.21.15.43:0/2295008387 conn(0x55f58ce80a30 msgr2=0x55f58ce878e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.169+0000 7f32584c0740 1 -- 172.21.15.43:0/2295008387 shutdown_connections 2024-09-11T23:22:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.169+0000 7f32584c0740 1 -- 172.21.15.43:0/2295008387 wait complete. 2024-09-11T23:22:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.169+0000 7f32584c0740 20 client.15477 trim_cache size 0 max 16384 2024-09-11T23:22:33.196 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:22:33.196 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-11T23:22:33.197 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-11T23:22:33.197 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:22:33.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.170+0000 7f32560e7640 1 -- 172.21.15.43:0/734535672 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f32380a7220 msgr2=0x7f32380a96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:33.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.170+0000 7f32560e7640 1 --2- 172.21.15.43:0/734535672 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f32380a7220 0x7f32380a96e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3248021000 tx=0x7f324801e000 comp rx=0 tx=0).stop 2024-09-11T23:22:33.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.170+0000 7f32560e7640 1 -- 172.21.15.43:0/734535672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3250103780 msgr2=0x7f3250198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:33.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.170+0000 7f32560e7640 1 --2- 172.21.15.43:0/734535672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3250103780 0x7f3250198b30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3244005940 tx=0x7f32440040c0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.170+0000 7f32560e7640 1 -- 172.21.15.43:0/734535672 shutdown_connections 2024-09-11T23:22:33.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.170+0000 7f32560e7640 1 --2- 172.21.15.43:0/734535672 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f32380a7220 0x7f32380a96e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.170+0000 7f32560e7640 1 --2- 172.21.15.43:0/734535672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3250108300 0x7f325019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.170+0000 7f32560e7640 1 --2- 172.21.15.43:0/734535672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3250103780 0x7f3250198b30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.170+0000 7f32560e7640 1 --2- 172.21.15.43:0/734535672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3250102dd0 0x7f32501985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:33.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.170+0000 7f32560e7640 1 -- 172.21.15.43:0/734535672 >> 172.21.15.43:0/734535672 conn(0x7f32500fc7d0 msgr2=0x7f3250105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:33.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.170+0000 7f32560e7640 1 -- 172.21.15.43:0/734535672 shutdown_connections 2024-09-11T23:22:33.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:33.170+0000 7f32560e7640 1 -- 172.21.15.43:0/734535672 wait complete. 2024-09-11T23:22:33.868 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:22:33.868 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:22:34.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:33 smithi043 ceph-mon[30950]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-11T23:22:34.129 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:34.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:33 smithi090 ceph-mon[35130]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-11T23:22:34.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:33 smithi097 ceph-mon[35562]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-11T23:22:34.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.655+0000 7fdf8b2d3640 1 -- 172.21.15.43:0/3480698761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf84101de0 msgr2=0x7fdf841021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:34.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.655+0000 7fdf8b2d3640 1 --2- 172.21.15.43:0/3480698761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf84101de0 0x7fdf841021c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009900 tx=0x7fdf8002a410 comp rx=0 tx=0).stop 2024-09-11T23:22:34.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.655+0000 7fdf8b2d3640 1 -- 172.21.15.43:0/3480698761 shutdown_connections 2024-09-11T23:22:34.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.655+0000 7fdf8b2d3640 1 --2- 172.21.15.43:0/3480698761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf84106f10 0x7fdf8410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.655+0000 7fdf8b2d3640 1 --2- 172.21.15.43:0/3480698761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf84102790 0x7fdf841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.655+0000 7fdf8b2d3640 1 --2- 172.21.15.43:0/3480698761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf84101de0 0x7fdf841021c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.655+0000 7fdf8b2d3640 1 -- 172.21.15.43:0/3480698761 >> 172.21.15.43:0/3480698761 conn(0x7fdf840fd8e0 msgr2=0x7fdf840ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:34.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.655+0000 7fdf8b2d3640 1 -- 172.21.15.43:0/3480698761 shutdown_connections 2024-09-11T23:22:34.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.655+0000 7fdf8b2d3640 1 -- 172.21.15.43:0/3480698761 wait complete. 2024-09-11T23:22:34.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.656+0000 7fdf8b2d3640 1 Processor -- start 2024-09-11T23:22:34.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.656+0000 7fdf8b2d3640 1 -- start start 2024-09-11T23:22:34.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.656+0000 7fdf8b2d3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf84101de0 0x7fdf841983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.656+0000 7fdf8b2d3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf84102790 0x7fdf841988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.656+0000 7fdf8b2d3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf84106f10 0x7fdf8419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.656+0000 7fdf8b2d3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdf84111f70 con 0x7fdf84101de0 2024-09-11T23:22:34.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.656+0000 7fdf8b2d3640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdf84111df0 con 0x7fdf84102790 2024-09-11T23:22:34.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.656+0000 7fdf8b2d3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdf841120f0 con 0x7fdf84106f10 2024-09-11T23:22:34.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.656+0000 7fdf8906f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf84101de0 0x7fdf841983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:34.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.656+0000 7fdf8906f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf84101de0 0x7fdf841983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60420/0 (socket says 172.21.15.43:60420) 2024-09-11T23:22:34.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.656+0000 7fdf8906f640 1 -- 172.21.15.43:0/3130170904 learned_addr learned my addr 172.21.15.43:0/3130170904 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:34.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.656+0000 7fdf8886e640 1 --2- 172.21.15.43:0/3130170904 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf84102790 0x7fdf841988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:34.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.657+0000 7fdf89870640 1 --2- 172.21.15.43:0/3130170904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf84106f10 0x7fdf8419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:34.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.657+0000 7fdf8906f640 1 -- 172.21.15.43:0/3130170904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf84106f10 msgr2=0x7fdf8419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:34.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.657+0000 7fdf8906f640 1 --2- 172.21.15.43:0/3130170904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf84106f10 0x7fdf8419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.657+0000 7fdf8906f640 1 -- 172.21.15.43:0/3130170904 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf84102790 msgr2=0x7fdf841988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:34.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.657+0000 7fdf8906f640 1 --2- 172.21.15.43:0/3130170904 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf84102790 0x7fdf841988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.657+0000 7fdf8906f640 1 -- 172.21.15.43:0/3130170904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8419d360 con 0x7fdf84101de0 2024-09-11T23:22:34.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.657+0000 7fdf8906f640 1 --2- 172.21.15.43:0/3130170904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf84101de0 0x7fdf841983b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdf80005ab0 tx=0x7fdf800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:34.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.657+0000 7fdf7a7fc640 1 -- 172.21.15.43:0/3130170904 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8003f030 con 0x7fdf84101de0 2024-09-11T23:22:34.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.657+0000 7fdf8b2d3640 1 -- 172.21.15.43:0/3130170904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf8419d5f0 con 0x7fdf84101de0 2024-09-11T23:22:34.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.657+0000 7fdf7a7fc640 1 -- 172.21.15.43:0/3130170904 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf80003ce0 con 0x7fdf84101de0 2024-09-11T23:22:34.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.657+0000 7fdf7a7fc640 1 -- 172.21.15.43:0/3130170904 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf80035070 con 0x7fdf84101de0 2024-09-11T23:22:34.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.657+0000 7fdf8b2d3640 1 -- 172.21.15.43:0/3130170904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf841a4f30 con 0x7fdf84101de0 2024-09-11T23:22:34.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.658+0000 7fdf8b2d3640 1 -- 172.21.15.43:0/3130170904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf841032c0 con 0x7fdf84101de0 2024-09-11T23:22:34.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.660+0000 7fdf7a7fc640 1 -- 172.21.15.43:0/3130170904 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fdf80042030 con 0x7fdf84101de0 2024-09-11T23:22:34.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.661+0000 7fdf7a7fc640 1 --2- 172.21.15.43:0/3130170904 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdf700a2e10 0x7fdf700a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:34.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.661+0000 7fdf8886e640 1 --2- 172.21.15.43:0/3130170904 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdf700a2e10 0x7fdf700a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:34.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.662+0000 7fdf7a7fc640 1 -- 172.21.15.43:0/3130170904 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fdf800f01f0 con 0x7fdf84101de0 2024-09-11T23:22:34.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.662+0000 7fdf8886e640 1 --2- 172.21.15.43:0/3130170904 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdf700a2e10 0x7fdf700a52d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdf840fdd70 tx=0x7fdf7400a2d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:34.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.664+0000 7fdf7a7fc640 1 -- 172.21.15.43:0/3130170904 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf800bcc90 con 0x7fdf84101de0 2024-09-11T23:22:34.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.885+0000 7fdf8b2d3640 1 -- 172.21.15.43:0/3130170904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf8410a0f0 con 0x7fdf84101de0 2024-09-11T23:22:34.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.886+0000 7fdf7a7fc640 1 -- 172.21.15.43:0/3130170904 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11896 (secure 0 0 0) 0x7fdf800bc590 con 0x7fdf84101de0 2024-09-11T23:22:34.886 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:22:34.886 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"btime":"2024-09-11T23:21:49:167551+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:21:49.117342+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:22:34.886 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 24 2024-09-11T23:22:34.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.888+0000 7fdf8b2d3640 1 -- 172.21.15.43:0/3130170904 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdf700a2e10 msgr2=0x7fdf700a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:34.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.888+0000 7fdf8b2d3640 1 --2- 172.21.15.43:0/3130170904 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdf700a2e10 0x7fdf700a52d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdf840fdd70 tx=0x7fdf7400a2d0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.888+0000 7fdf8b2d3640 1 -- 172.21.15.43:0/3130170904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf84101de0 msgr2=0x7fdf841983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:34.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.888+0000 7fdf8b2d3640 1 --2- 172.21.15.43:0/3130170904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf84101de0 0x7fdf841983b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdf80005ab0 tx=0x7fdf800042f0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.888+0000 7fdf8b2d3640 1 -- 172.21.15.43:0/3130170904 shutdown_connections 2024-09-11T23:22:34.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.888+0000 7fdf8b2d3640 1 --2- 172.21.15.43:0/3130170904 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdf700a2e10 0x7fdf700a52d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.888+0000 7fdf8b2d3640 1 --2- 172.21.15.43:0/3130170904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf84106f10 0x7fdf8419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.888+0000 7fdf8b2d3640 1 --2- 172.21.15.43:0/3130170904 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf84102790 0x7fdf841988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.888+0000 7fdf8b2d3640 1 --2- 172.21.15.43:0/3130170904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf84101de0 0x7fdf841983b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:34.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.888+0000 7fdf8b2d3640 1 -- 172.21.15.43:0/3130170904 >> 172.21.15.43:0/3130170904 conn(0x7fdf840fd8e0 msgr2=0x7fdf840feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:34.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.888+0000 7fdf8b2d3640 1 -- 172.21.15.43:0/3130170904 shutdown_connections 2024-09-11T23:22:34.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:34.888+0000 7fdf8b2d3640 1 -- 172.21.15.43:0/3130170904 wait complete. 2024-09-11T23:22:35.201 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-11T23:22:35.202 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:22:35.566 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:22:35.621 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:36.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:35 smithi043 ceph-mon[30950]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-11T23:22:36.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:35 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3130170904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:36.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.129+0000 7f678e5e7640 1 -- 172.21.15.43:0/4164665284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6788102570 msgr2=0x7f67881029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:36.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.129+0000 7f678e5e7640 1 --2- 172.21.15.43:0/4164665284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6788102570 0x7f67881029b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f677c009900 tx=0x7f677c02a480 comp rx=0 tx=0).stop 2024-09-11T23:22:36.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.129+0000 7f678e5e7640 1 -- 172.21.15.43:0/4164665284 shutdown_connections 2024-09-11T23:22:36.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.129+0000 7f678e5e7640 1 --2- 172.21.15.43:0/4164665284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6788102ef0 0x7f678810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:36.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.129+0000 7f678e5e7640 1 --2- 172.21.15.43:0/4164665284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6788102570 0x7f67881029b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:36.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.129+0000 7f678e5e7640 1 --2- 172.21.15.43:0/4164665284 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6788100e20 0x7f6788101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:36.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.129+0000 7f678e5e7640 1 -- 172.21.15.43:0/4164665284 >> 172.21.15.43:0/4164665284 conn(0x7f67880fc7d0 msgr2=0x7f67880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:36.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.129+0000 7f678e5e7640 1 -- 172.21.15.43:0/4164665284 shutdown_connections 2024-09-11T23:22:36.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.129+0000 7f678e5e7640 1 -- 172.21.15.43:0/4164665284 wait complete. 2024-09-11T23:22:36.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.129+0000 7f678e5e7640 1 Processor -- start 2024-09-11T23:22:36.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.130+0000 7f678e5e7640 1 -- start start 2024-09-11T23:22:36.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.130+0000 7f678e5e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6788100e20 0x7f67881971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:36.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.130+0000 7f678e5e7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6788102570 0x7f6788197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:36.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.130+0000 7f678e5e7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6788102ef0 0x7f6788197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:36.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.130+0000 7f678e5e7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6788110fa0 con 0x7f6788100e20 2024-09-11T23:22:36.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.130+0000 7f678e5e7640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6788110e20 con 0x7f6788102570 2024-09-11T23:22:36.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.130+0000 7f678e5e7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6788111120 con 0x7f6788102ef0 2024-09-11T23:22:36.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.130+0000 7f6787fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6788100e20 0x7f67881971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:36.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.130+0000 7f6787fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6788100e20 0x7f67881971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60426/0 (socket says 172.21.15.43:60426) 2024-09-11T23:22:36.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.130+0000 7f6787fff640 1 -- 172.21.15.43:0/3091584064 learned_addr learned my addr 172.21.15.43:0/3091584064 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:36.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.130+0000 7f67877fe640 1 --2- 172.21.15.43:0/3091584064 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6788102570 0x7f6788197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:36.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.130+0000 7f678cb84640 1 --2- 172.21.15.43:0/3091584064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6788102ef0 0x7f6788197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:36.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.130+0000 7f6787fff640 1 -- 172.21.15.43:0/3091584064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6788102ef0 msgr2=0x7f6788197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:36.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.130+0000 7f6787fff640 1 --2- 172.21.15.43:0/3091584064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6788102ef0 0x7f6788197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:36.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.130+0000 7f6787fff640 1 -- 172.21.15.43:0/3091584064 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6788102570 msgr2=0x7f6788197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:36.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.131+0000 7f6787fff640 1 --2- 172.21.15.43:0/3091584064 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6788102570 0x7f6788197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:36.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.131+0000 7f6787fff640 1 -- 172.21.15.43:0/3091584064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67881a6a30 con 0x7f6788100e20 2024-09-11T23:22:36.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.131+0000 7f678cb84640 1 --2- 172.21.15.43:0/3091584064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6788102ef0 0x7f6788197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:22:36.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.131+0000 7f6787fff640 1 --2- 172.21.15.43:0/3091584064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6788100e20 0x7f67881971c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f677800d950 tx=0x7f677800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:36.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.131+0000 7f67857fa640 1 -- 172.21.15.43:0/3091584064 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778017070 con 0x7f6788100e20 2024-09-11T23:22:36.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.131+0000 7f678e5e7640 1 -- 172.21.15.43:0/3091584064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67881a6d20 con 0x7f6788100e20 2024-09-11T23:22:36.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.131+0000 7f67857fa640 1 -- 172.21.15.43:0/3091584064 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6778009c30 con 0x7f6788100e20 2024-09-11T23:22:36.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.131+0000 7f67857fa640 1 -- 172.21.15.43:0/3091584064 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677800f6b0 con 0x7f6788100e20 2024-09-11T23:22:36.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.131+0000 7f678e5e7640 1 -- 172.21.15.43:0/3091584064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67881a7250 con 0x7f6788100e20 2024-09-11T23:22:36.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.132+0000 7f678e5e7640 1 -- 172.21.15.43:0/3091584064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6788101200 con 0x7f6788100e20 2024-09-11T23:22:36.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.134+0000 7f67857fa640 1 -- 172.21.15.43:0/3091584064 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f67780109d0 con 0x7f6788100e20 2024-09-11T23:22:36.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.135+0000 7f67857fa640 1 --2- 172.21.15.43:0/3091584064 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f67700a2e10 0x7f67700a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:36.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.135+0000 7f67857fa640 1 -- 172.21.15.43:0/3091584064 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f67780cf310 con 0x7f6788100e20 2024-09-11T23:22:36.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.135+0000 7f67877fe640 1 --2- 172.21.15.43:0/3091584064 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f67700a2e10 0x7f67700a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:36.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.136+0000 7f67877fe640 1 --2- 172.21.15.43:0/3091584064 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f67700a2e10 0x7f67700a52d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f677c009230 tx=0x7f677c0058d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:36.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.138+0000 7f67857fa640 1 -- 172.21.15.43:0/3091584064 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6778009de0 con 0x7f6788100e20 2024-09-11T23:22:36.196 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:36.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:35 smithi090 ceph-mon[35130]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-11T23:22:36.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:35 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3130170904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:36.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:35 smithi097 ceph-mon[35562]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-11T23:22:36.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:35 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3130170904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:36.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.377+0000 7f678e5e7640 1 -- 172.21.15.43:0/3091584064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f678802d7c0 con 0x7f6788100e20 2024-09-11T23:22:36.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.378+0000 7f67857fa640 1 -- 172.21.15.43:0/3091584064 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11896 (secure 0 0 0) 0x7f677809bce0 con 0x7f6788100e20 2024-09-11T23:22:36.378 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:22:36.378 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"btime":"2024-09-11T23:21:49:167551+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:21:49.117342+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:22:36.379 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 24 2024-09-11T23:22:36.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.381+0000 7f678e5e7640 1 -- 172.21.15.43:0/3091584064 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f67700a2e10 msgr2=0x7f67700a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:36.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.381+0000 7f678e5e7640 1 --2- 172.21.15.43:0/3091584064 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f67700a2e10 0x7f67700a52d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f677c009230 tx=0x7f677c0058d0 comp rx=0 tx=0).stop 2024-09-11T23:22:36.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.381+0000 7f678e5e7640 1 -- 172.21.15.43:0/3091584064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6788100e20 msgr2=0x7f67881971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:36.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.381+0000 7f678e5e7640 1 --2- 172.21.15.43:0/3091584064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6788100e20 0x7f67881971c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f677800d950 tx=0x7f677800de10 comp rx=0 tx=0).stop 2024-09-11T23:22:36.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.381+0000 7f678e5e7640 1 -- 172.21.15.43:0/3091584064 shutdown_connections 2024-09-11T23:22:36.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.381+0000 7f678e5e7640 1 --2- 172.21.15.43:0/3091584064 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f67700a2e10 0x7f67700a52d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:36.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.381+0000 7f678e5e7640 1 --2- 172.21.15.43:0/3091584064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6788102ef0 0x7f6788197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:36.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.381+0000 7f678e5e7640 1 --2- 172.21.15.43:0/3091584064 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6788102570 0x7f6788197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:36.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.381+0000 7f678e5e7640 1 --2- 172.21.15.43:0/3091584064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6788100e20 0x7f67881971c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:36.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.381+0000 7f678e5e7640 1 -- 172.21.15.43:0/3091584064 >> 172.21.15.43:0/3091584064 conn(0x7f67880fc7d0 msgr2=0x7f67880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:36.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.381+0000 7f678e5e7640 1 -- 172.21.15.43:0/3091584064 shutdown_connections 2024-09-11T23:22:36.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.381+0000 7f678e5e7640 1 -- 172.21.15.43:0/3091584064 wait complete. 2024-09-11T23:22:37.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:36 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3091584064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:37.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:36 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3091584064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:37.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:36 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3091584064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:22:37.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.874+0000 7f4eaef9b640 1 -- 172.21.15.43:0/119613458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ea8100e20 msgr2=0x7f4ea8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:37.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.874+0000 7f4eaef9b640 1 --2- 172.21.15.43:0/119613458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ea8100e20 0x7f4ea8101200 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009900 tx=0x7f4e9c02a480 comp rx=0 tx=0).stop 2024-09-11T23:22:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.874+0000 7f4eaef9b640 1 -- 172.21.15.43:0/119613458 shutdown_connections 2024-09-11T23:22:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.874+0000 7f4eaef9b640 1 --2- 172.21.15.43:0/119613458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ea8102ef0 0x7f4ea810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.874+0000 7f4eaef9b640 1 --2- 172.21.15.43:0/119613458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8102570 0x7f4ea81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.874+0000 7f4eaef9b640 1 --2- 172.21.15.43:0/119613458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ea8100e20 0x7f4ea8101200 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.874+0000 7f4eaef9b640 1 -- 172.21.15.43:0/119613458 >> 172.21.15.43:0/119613458 conn(0x7f4ea80fc7d0 msgr2=0x7f4ea80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.874+0000 7f4eaef9b640 1 -- 172.21.15.43:0/119613458 shutdown_connections 2024-09-11T23:22:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.874+0000 7f4eaef9b640 1 -- 172.21.15.43:0/119613458 wait complete. 2024-09-11T23:22:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.875+0000 7f4eaef9b640 1 Processor -- start 2024-09-11T23:22:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.875+0000 7f4eaef9b640 1 -- start start 2024-09-11T23:22:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.875+0000 7f4eaef9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100e20 0x7f4ea8198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.875+0000 7f4eaef9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ea8102570 0x7f4ea8198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.875+0000 7f4eaef9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ea8102ef0 0x7f4ea819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:37.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.875+0000 7f4eaef9b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea8110f80 con 0x7f4ea8102570 2024-09-11T23:22:37.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.875+0000 7f4eaef9b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea8110e00 con 0x7f4ea8102ef0 2024-09-11T23:22:37.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.875+0000 7f4eaef9b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea8111100 con 0x7f4ea8100e20 2024-09-11T23:22:37.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.875+0000 7f4ea7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ea8102570 0x7f4ea8198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:37.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.875+0000 7f4ea7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ea8102570 0x7f4ea8198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60438/0 (socket says 172.21.15.43:60438) 2024-09-11T23:22:37.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.875+0000 7f4ea7fff640 1 -- 172.21.15.43:0/2240274906 learned_addr learned my addr 172.21.15.43:0/2240274906 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:37.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.876+0000 7f4eacd37640 1 --2- 172.21.15.43:0/2240274906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100e20 0x7f4ea8198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:37.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.876+0000 7f4ead538640 1 --2- 172.21.15.43:0/2240274906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ea8102ef0 0x7f4ea819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:37.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.876+0000 7f4ea7fff640 1 -- 172.21.15.43:0/2240274906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100e20 msgr2=0x7f4ea8198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:37.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.876+0000 7f4ea7fff640 1 --2- 172.21.15.43:0/2240274906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100e20 0x7f4ea8198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.876+0000 7f4ea7fff640 1 -- 172.21.15.43:0/2240274906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ea8102ef0 msgr2=0x7f4ea819ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:37.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.876+0000 7f4ea7fff640 1 --2- 172.21.15.43:0/2240274906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ea8102ef0 0x7f4ea819ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.876+0000 7f4ea7fff640 1 -- 172.21.15.43:0/2240274906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea819d520 con 0x7f4ea8102570 2024-09-11T23:22:37.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.876+0000 7f4ead538640 1 --2- 172.21.15.43:0/2240274906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ea8102ef0 0x7f4ea819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:37.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.876+0000 7f4ea7fff640 1 --2- 172.21.15.43:0/2240274906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ea8102570 0x7f4ea8198ab0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400c970 tx=0x7f4e9400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:37.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.876+0000 7f4ea5ffb640 1 -- 172.21.15.43:0/2240274906 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e94007bb0 con 0x7f4ea8102570 2024-09-11T23:22:37.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.876+0000 7f4ea5ffb640 1 -- 172.21.15.43:0/2240274906 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e94007d50 con 0x7f4ea8102570 2024-09-11T23:22:37.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.876+0000 7f4eaef9b640 1 -- 172.21.15.43:0/2240274906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea819d810 con 0x7f4ea8102570 2024-09-11T23:22:37.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.876+0000 7f4ea5ffb640 1 -- 172.21.15.43:0/2240274906 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9400f450 con 0x7f4ea8102570 2024-09-11T23:22:37.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.877+0000 7f4eaef9b640 1 -- 172.21.15.43:0/2240274906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea81a50f0 con 0x7f4ea8102570 2024-09-11T23:22:37.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.879+0000 7f4ea5ffb640 1 -- 172.21.15.43:0/2240274906 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f4e94005020 con 0x7f4ea8102570 2024-09-11T23:22:37.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.879+0000 7f4eb1374740 1 Processor -- start 2024-09-11T23:22:37.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.879+0000 7f4eb1374740 1 -- start start 2024-09-11T23:22:37.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.879+0000 7f4eb1374740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049c8b40 0x55f9049c7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:37.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.879+0000 7f4eb1374740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9049c75d0 0x55f9049c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:37.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.879+0000 7f4eb1374740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049d2370 0x55f9049d4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:37.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.879+0000 7f4eb1374740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f90492d2f0 con 0x55f9049d2370 2024-09-11T23:22:37.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.879+0000 7f4eb1374740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f9048d4cf0 con 0x55f9049c8b40 2024-09-11T23:22:37.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.879+0000 7f4eb1374740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f9049292b0 con 0x55f9049c75d0 2024-09-11T23:22:37.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.880+0000 7f4ea7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049d2370 0x55f9049d4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:37.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.880+0000 7f4ea7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049d2370 0x55f9049d4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60446/0 (socket says 172.21.15.43:60446) 2024-09-11T23:22:37.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.880+0000 7f4ea7fff640 1 -- 172.21.15.43:0/2557192830 learned_addr learned my addr 172.21.15.43:0/2557192830 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:37.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.880+0000 7f4ead538640 1 --2- 172.21.15.43:0/2557192830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049c8b40 0x55f9049c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:37.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.880+0000 7f4eacd37640 1 --2- 172.21.15.43:0/2557192830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9049c75d0 0x55f9049c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:37.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.880+0000 7f4ea5ffb640 1 --2- 172.21.15.43:0/2240274906 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f4e980a2e10 0x7f4e980a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:37.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.880+0000 7f4ead538640 1 -- 172.21.15.43:0/2557192830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9049c75d0 msgr2=0x55f9049c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:37.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.880+0000 7f4ead538640 1 --2- 172.21.15.43:0/2557192830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9049c75d0 0x55f9049c7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.880+0000 7f4ead538640 1 -- 172.21.15.43:0/2557192830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049d2370 msgr2=0x55f9049d4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:37.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.880+0000 7f4ea5ffb640 1 -- 172.21.15.43:0/2240274906 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f4e940cf590 con 0x7f4ea8102570 2024-09-11T23:22:37.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.880+0000 7f4ead538640 1 --2- 172.21.15.43:0/2557192830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049d2370 0x55f9049d4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.880+0000 7f4ead538640 1 -- 172.21.15.43:0/2557192830 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9048b93a0 con 0x55f9049c8b40 2024-09-11T23:22:37.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.880+0000 7f4ea7fff640 1 --2- 172.21.15.43:0/2557192830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049d2370 0x55f9049d4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:37.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.880+0000 7f4ead538640 1 --2- 172.21.15.43:0/2240274906 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f4e980a2e10 0x7f4e980a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:37.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.880+0000 7f4ead538640 1 --2- 172.21.15.43:0/2557192830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049c8b40 0x55f9049c7090 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000af80 tx=0x7f4ea000e7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf3620937853b676 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:37.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.881+0000 7f4e92ffd640 1 -- 172.21.15.43:0/2557192830 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0022070 con 0x55f9049c8b40 2024-09-11T23:22:37.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.881+0000 7f4e92ffd640 1 -- 172.21.15.43:0/2557192830 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ea0018070 con 0x55f9049c8b40 2024-09-11T23:22:37.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.881+0000 7f4e92ffd640 1 -- 172.21.15.43:0/2557192830 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea00055b0 con 0x55f9049c8b40 2024-09-11T23:22:37.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.881+0000 7f4ead538640 1 --2- 172.21.15.43:0/2240274906 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f4e980a2e10 0x7f4e980a52d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4ea001f000 tx=0x7f4ea0020000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:37.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.881+0000 7f4eb1374740 1 -- 172.21.15.43:0/2557192830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049c8b40 msgr2=0x55f9049c7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.881+0000 7f4eb1374740 1 --2- 172.21.15.43:0/2557192830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049c8b40 0x55f9049c7090 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000af80 tx=0x7f4ea000e7e0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.881+0000 7f4eb1374740 1 -- 172.21.15.43:0/2557192830 shutdown_connections 2024-09-11T23:22:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.881+0000 7f4eb1374740 1 --2- 172.21.15.43:0/2557192830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049d2370 0x55f9049d4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.881+0000 7f4eb1374740 1 --2- 172.21.15.43:0/2557192830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9049c75d0 0x55f9049c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.881+0000 7f4eb1374740 1 --2- 172.21.15.43:0/2557192830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049c8b40 0x55f9049c7090 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.881+0000 7f4eb1374740 1 -- 172.21.15.43:0/2557192830 >> 172.21.15.43:0/2557192830 conn(0x55f904990310 msgr2=0x55f9049c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.881+0000 7f4eb1374740 1 -- 172.21.15.43:0/2557192830 shutdown_connections 2024-09-11T23:22:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.882+0000 7f4eb1374740 1 -- 172.21.15.43:0/2557192830 wait complete. 2024-09-11T23:22:37.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.882+0000 7f4eb1374740 1 -- start start 2024-09-11T23:22:37.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.882+0000 7f4eb1374740 1 Processor -- start 2024-09-11T23:22:37.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.882+0000 7f4eb1374740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9049c75d0 0x55f9049d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:37.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.882+0000 7f4eb1374740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049c8b40 0x55f9049cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:37.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.882+0000 7f4eb1374740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049d2370 0x55f9049cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:37.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.882+0000 7f4eb1374740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f9049d9480 con 0x55f9049c8b40 2024-09-11T23:22:37.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.882+0000 7f4eb1374740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f9048e8060 con 0x55f9049d2370 2024-09-11T23:22:37.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.882+0000 7f4eb1374740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f9049d9600 con 0x55f9049c75d0 2024-09-11T23:22:37.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.882+0000 7f4eb1374740 10 client.0 _ll_register_callbacks cb 0x55f90463b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.882+0000 7f4ea7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049c8b40 0x55f9049cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.882+0000 7f4ea7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049c8b40 0x55f9049cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60460/0 (socket says 172.21.15.43:60460) 2024-09-11T23:22:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.883+0000 7f4ea7fff640 1 -- 172.21.15.43:0/2013784711 learned_addr learned my addr 172.21.15.43:0/2013784711 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.883+0000 7f4ead538640 1 --2- 172.21.15.43:0/2013784711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049d2370 0x55f9049cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.883+0000 7f4eacd37640 1 --2- 172.21.15.43:0/2013784711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9049c75d0 0x55f9049d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.883+0000 7f4ead538640 1 -- 172.21.15.43:0/2013784711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9049c75d0 msgr2=0x55f9049d1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.883+0000 7f4ead538640 1 --2- 172.21.15.43:0/2013784711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9049c75d0 0x55f9049d1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.883+0000 7f4ead538640 1 -- 172.21.15.43:0/2013784711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049c8b40 msgr2=0x55f9049cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.883+0000 7f4ead538640 1 --2- 172.21.15.43:0/2013784711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049c8b40 0x55f9049cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.883+0000 7f4ead538640 1 -- 172.21.15.43:0/2013784711 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f90469e3c0 con 0x55f9049d2370 2024-09-11T23:22:37.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.883+0000 7f4ea7fff640 1 --2- 172.21.15.43:0/2013784711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049c8b40 0x55f9049cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:37.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.883+0000 7f4ead538640 1 --2- 172.21.15.43:0/2013784711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049d2370 0x55f9049cf650 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000ece0 tx=0x7f4ea0005ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9758dd24391885e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:37.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.883+0000 7f4e87fff640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:22:37.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.883+0000 7f4e87fff640 1 -- 172.21.15.43:0/2013784711 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0033060 con 0x55f9049d2370 2024-09-11T23:22:37.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.883+0000 7f4e87fff640 1 -- 172.21.15.43:0/2013784711 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f4ea0011b80 con 0x55f9049d2370 2024-09-11T23:22:37.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.883+0000 7f4eb1374740 1 -- 172.21.15.43:0/2013784711 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ea0018070 con 0x55f9049d2370 2024-09-11T23:22:37.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.884+0000 7f4e87fff640 1 -- 172.21.15.43:0/2013784711 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4ea0022040 con 0x55f9049d2370 2024-09-11T23:22:37.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.884+0000 7f4eb1374740 10 client.25115 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:37.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.884+0000 7f4eb1374740 1 -- 172.21.15.43:0/2013784711 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f9049e4f50 con 0x55f9049d2370 2024-09-11T23:22:37.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.884+0000 7f4e87fff640 1 -- 172.21.15.43:0/2013784711 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ea0018070 con 0x55f9049d2370 2024-09-11T23:22:37.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.884+0000 7f4e87fff640 1 -- 172.21.15.43:0/2013784711 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f4ea001c430 con 0x55f9049d2370 2024-09-11T23:22:37.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.884+0000 7f4eb1374740 10 client.25115 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.884+0000 7f4eb1374740 10 client.25115 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T23:22:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.884+0000 7f4eb1374740 20 client.25115 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.884+0000 7f4eb1374740 1 --2- 172.21.15.43:0/2013784711 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55f9049e62a0 0x55f9049e8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.884+0000 7f4eb1374740 4 client.25115 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:22:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.884+0000 7f4eb1374740 1 -- 172.21.15.43:0/2013784711 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f90493a860 con 0x55f9049e62a0 2024-09-11T23:22:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.884+0000 7f4eacd37640 1 --2- 172.21.15.43:0/2013784711 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55f9049e62a0 0x55f9049e8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.885+0000 7f4eacd37640 1 --2- 172.21.15.43:0/2013784711 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55f9049e62a0 0x55f9049e8690 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fbe315c803cd7852 server_cookie=6bb674b28076c9d4 in_seq=0 out_seq=0 2024-09-11T23:22:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.885+0000 7f4e87fff640 10 client.25115 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:22:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.885+0000 7f4e87fff640 1 -- 172.21.15.43:0/2013784711 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f90493a860 con 0x55f9049e62a0 2024-09-11T23:22:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.885+0000 7f4e87fff640 10 client.25115 handle_command_reply: tid=0 2024-09-11T23:22:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.886+0000 7f4eb1374740 1 client.25115 shutdown 2024-09-11T23:22:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.886+0000 7f4eb1374740 1 -- 172.21.15.43:0/2013784711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049d2370 msgr2=0x55f9049cf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.886+0000 7f4eb1374740 1 --2- 172.21.15.43:0/2013784711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049d2370 0x55f9049cf650 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000ece0 tx=0x7f4ea0005ba0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.886+0000 7f4eb1374740 1 -- 172.21.15.43:0/2013784711 shutdown_connections 2024-09-11T23:22:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.886+0000 7f4eb1374740 1 --2- 172.21.15.43:0/2013784711 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55f9049e62a0 0x55f9049e8690 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.886+0000 7f4eb1374740 1 --2- 172.21.15.43:0/2013784711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049d2370 0x55f9049cf650 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.886+0000 7f4eb1374740 1 --2- 172.21.15.43:0/2013784711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049c8b40 0x55f9049cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.886+0000 7f4eb1374740 1 --2- 172.21.15.43:0/2013784711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9049c75d0 0x55f9049d1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.886+0000 7f4e87fff640 0 client.25115 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:22:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.886+0000 7f4eb1374740 1 -- 172.21.15.43:0/2013784711 >> 172.21.15.43:0/2013784711 conn(0x55f904990310 msgr2=0x55f9049c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.886+0000 7f4eb1374740 1 -- 172.21.15.43:0/2013784711 shutdown_connections 2024-09-11T23:22:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.886+0000 7f4eb1374740 1 -- 172.21.15.43:0/2013784711 wait complete. 2024-09-11T23:22:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.886+0000 7f4eb1374740 20 client.25115 trim_cache size 0 max 16384 2024-09-11T23:22:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.899+0000 7f4eb1374740 1 Processor -- start 2024-09-11T23:22:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.899+0000 7f4eb1374740 1 -- start start 2024-09-11T23:22:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.900+0000 7f4eb1374740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9049c75d0 0x55f9049e28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.900+0000 7f4eb1374740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049c8b40 0x55f9049d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.900+0000 7f4eb1374740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049d2370 0x55f9049ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.900+0000 7f4eb1374740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea81adfa0 con 0x55f9049d2370 2024-09-11T23:22:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.900+0000 7f4eb1374740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea81ab870 con 0x55f9049c8b40 2024-09-11T23:22:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.900+0000 7f4eb1374740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea81a96c0 con 0x55f9049c75d0 2024-09-11T23:22:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.900+0000 7f4ead538640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049d2370 0x55f9049ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.900+0000 7f4ead538640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049d2370 0x55f9049ca0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60462/0 (socket says 172.21.15.43:60462) 2024-09-11T23:22:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.900+0000 7f4ead538640 1 -- 172.21.15.43:0/1639801898 learned_addr learned my addr 172.21.15.43:0/1639801898 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.900+0000 7f4eacd37640 1 --2- 172.21.15.43:0/1639801898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9049c75d0 0x55f9049e28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.900+0000 7f4ea7fff640 1 --2- 172.21.15.43:0/1639801898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049c8b40 0x55f9049d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.900+0000 7f4ead538640 1 -- 172.21.15.43:0/1639801898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9049c75d0 msgr2=0x55f9049e28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.900+0000 7f4ead538640 1 --2- 172.21.15.43:0/1639801898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9049c75d0 0x55f9049e28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.900+0000 7f4ead538640 1 -- 172.21.15.43:0/1639801898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049c8b40 msgr2=0x55f9049d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:37.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.900+0000 7f4ead538640 1 --2- 172.21.15.43:0/1639801898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049c8b40 0x55f9049d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.900+0000 7f4ead538640 1 -- 172.21.15.43:0/1639801898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9049d1750 con 0x55f9049d2370 2024-09-11T23:22:37.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.900+0000 7f4ead538640 1 --2- 172.21.15.43:0/1639801898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049d2370 0x55f9049ca0f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4ea001c800 tx=0x7f4ea0012650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284fd3ab00645d14 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:37.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.900+0000 7f4e927fc640 1 -- 172.21.15.43:0/1639801898 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0033070 con 0x55f9049d2370 2024-09-11T23:22:37.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.901+0000 7f4e927fc640 1 -- 172.21.15.43:0/1639801898 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ea0018040 con 0x55f9049d2370 2024-09-11T23:22:37.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.901+0000 7f4e927fc640 1 -- 172.21.15.43:0/1639801898 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea003d080 con 0x55f9049d2370 2024-09-11T23:22:37.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.901+0000 7f4eb1374740 1 -- 172.21.15.43:0/1639801898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049d2370 msgr2=0x55f9049ca0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:37.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.901+0000 7f4eb1374740 1 --2- 172.21.15.43:0/1639801898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049d2370 0x55f9049ca0f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4ea001c800 tx=0x7f4ea0012650 comp rx=0 tx=0).stop 2024-09-11T23:22:37.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.901+0000 7f4eb1374740 1 -- 172.21.15.43:0/1639801898 shutdown_connections 2024-09-11T23:22:37.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.901+0000 7f4eb1374740 1 --2- 172.21.15.43:0/1639801898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049d2370 0x55f9049ca0f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.901+0000 7f4eb1374740 1 --2- 172.21.15.43:0/1639801898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049c8b40 0x55f9049d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.901+0000 7f4eb1374740 1 --2- 172.21.15.43:0/1639801898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9049c75d0 0x55f9049e28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.901+0000 7f4eb1374740 1 -- 172.21.15.43:0/1639801898 >> 172.21.15.43:0/1639801898 conn(0x55f904990310 msgr2=0x55f9049c2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:37.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.901+0000 7f4eb1374740 1 -- 172.21.15.43:0/1639801898 shutdown_connections 2024-09-11T23:22:37.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.901+0000 7f4eb1374740 1 -- 172.21.15.43:0/1639801898 wait complete. 2024-09-11T23:22:37.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.902+0000 7f4eb1374740 1 -- start start 2024-09-11T23:22:37.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.902+0000 7f4eb1374740 1 Processor -- start 2024-09-11T23:22:37.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.902+0000 7f4eb1374740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049c75d0 0x55f9049ebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:37.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.902+0000 7f4eb1374740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049c8b40 0x55f9049e8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:37.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.902+0000 7f4eb1374740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9049d2370 0x55f9049e9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:37.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.902+0000 7f4eb1374740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea81a9d30 con 0x55f9049c8b40 2024-09-11T23:22:37.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.902+0000 7f4eb1374740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea81ab1a0 con 0x55f9049c75d0 2024-09-11T23:22:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.902+0000 7f4eb1374740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea81aa400 con 0x55f9049d2370 2024-09-11T23:22:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.902+0000 7f4eb1374740 10 client.0 _ll_register_callbacks cb 0x7f4ea0020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.902+0000 7f4ea7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049c8b40 0x55f9049e8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.902+0000 7f4ea7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049c8b40 0x55f9049e8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60466/0 (socket says 172.21.15.43:60466) 2024-09-11T23:22:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.902+0000 7f4ea7fff640 1 -- 172.21.15.43:0/2496790055 learned_addr learned my addr 172.21.15.43:0/2496790055 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.902+0000 7f4ead538640 1 --2- 172.21.15.43:0/2496790055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9049d2370 0x55f9049e9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.902+0000 7f4eacd37640 1 --2- 172.21.15.43:0/2496790055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049c75d0 0x55f9049ebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.903+0000 7f4ea7fff640 1 -- 172.21.15.43:0/2496790055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9049d2370 msgr2=0x55f9049e9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.903+0000 7f4ea7fff640 1 --2- 172.21.15.43:0/2496790055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9049d2370 0x55f9049e9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.903+0000 7f4ea7fff640 1 -- 172.21.15.43:0/2496790055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049c75d0 msgr2=0x55f9049ebaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.903+0000 7f4ea7fff640 1 --2- 172.21.15.43:0/2496790055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049c75d0 0x55f9049ebaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.903+0000 7f4ea7fff640 1 -- 172.21.15.43:0/2496790055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9049e98c0 con 0x55f9049c8b40 2024-09-11T23:22:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.903+0000 7f4ea7fff640 1 --2- 172.21.15.43:0/2496790055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049c8b40 0x55f9049e8e40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4e940ccc70 tx=0x7f4e9400a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29679ea6fe23ff0b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:37.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.903+0000 7f4e87fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:37.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.903+0000 7f4e87fff640 1 -- 172.21.15.43:0/2496790055 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9401d040 con 0x55f9049c8b40 2024-09-11T23:22:37.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.903+0000 7f4e87fff640 1 -- 172.21.15.43:0/2496790055 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f4e940cce40 con 0x55f9049c8b40 2024-09-11T23:22:37.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.903+0000 7f4eb1374740 1 -- 172.21.15.43:0/2496790055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ea0018040 con 0x55f9049c8b40 2024-09-11T23:22:37.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.903+0000 7f4e87fff640 1 -- 172.21.15.43:0/2496790055 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4e9409b1c0 con 0x55f9049c8b40 2024-09-11T23:22:37.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.903+0000 7f4eb1374740 10 client.15507 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:37.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.903+0000 7f4eb1374740 1 -- 172.21.15.43:0/2496790055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f904a509b0 con 0x55f9049c8b40 2024-09-11T23:22:37.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.903+0000 7f4e87fff640 1 -- 172.21.15.43:0/2496790055 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ea0018040 con 0x55f9049c8b40 2024-09-11T23:22:37.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.903+0000 7f4e87fff640 1 -- 172.21.15.43:0/2496790055 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f4e9409cb90 con 0x55f9049c8b40 2024-09-11T23:22:37.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.903+0000 7f4eb1374740 10 client.15507 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:37.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.903+0000 7f4eb1374740 10 client.15507 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T23:22:37.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.903+0000 7f4eb1374740 20 client.15507 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:37.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.903+0000 7f4eb1374740 1 --2- 172.21.15.43:0/2496790055 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55f904a510e0 0x55f904a53500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:37.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.903+0000 7f4eb1374740 4 client.15507 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:22:37.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.904+0000 7f4eb1374740 1 -- 172.21.15.43:0/2496790055 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f904934d70 con 0x55f904a510e0 2024-09-11T23:22:37.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.904+0000 7f4eacd37640 1 --2- 172.21.15.43:0/2496790055 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55f904a510e0 0x55f904a53500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:37.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.904+0000 7f4eacd37640 1 --2- 172.21.15.43:0/2496790055 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55f904a510e0 0x55f904a53500 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1df2317d975628cb server_cookie=c2d8f13dd62b8ca1 in_seq=0 out_seq=0 2024-09-11T23:22:37.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.904+0000 7f4e87fff640 10 client.15507 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:22:37.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.905+0000 7f4e87fff640 1 -- 172.21.15.43:0/2496790055 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f904934d70 con 0x55f904a510e0 2024-09-11T23:22:37.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.905+0000 7f4e87fff640 10 client.15507 handle_command_reply: tid=0 2024-09-11T23:22:37.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.905+0000 7f4eb1374740 1 client.15507 shutdown 2024-09-11T23:22:37.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.905+0000 7f4eb1374740 1 -- 172.21.15.43:0/2496790055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049c8b40 msgr2=0x55f9049e8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:37.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.905+0000 7f4eb1374740 1 --2- 172.21.15.43:0/2496790055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049c8b40 0x55f9049e8e40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4e940ccc70 tx=0x7f4e9400a570 comp rx=0 tx=0).stop 2024-09-11T23:22:37.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.905+0000 7f4eb1374740 1 -- 172.21.15.43:0/2496790055 shutdown_connections 2024-09-11T23:22:37.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.905+0000 7f4eb1374740 1 --2- 172.21.15.43:0/2496790055 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55f904a510e0 0x55f904a53500 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.905+0000 7f4eb1374740 1 --2- 172.21.15.43:0/2496790055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9049d2370 0x55f9049e9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.905+0000 7f4eb1374740 1 --2- 172.21.15.43:0/2496790055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9049c8b40 0x55f9049e8e40 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.905+0000 7f4eb1374740 1 --2- 172.21.15.43:0/2496790055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9049c75d0 0x55f9049ebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.905+0000 7f4e87fff640 0 client.15507 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:22:37.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.905+0000 7f4eb1374740 1 -- 172.21.15.43:0/2496790055 >> 172.21.15.43:0/2496790055 conn(0x55f904990310 msgr2=0x55f9049cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:37.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.905+0000 7f4eb1374740 1 -- 172.21.15.43:0/2496790055 shutdown_connections 2024-09-11T23:22:37.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.905+0000 7f4eb1374740 1 -- 172.21.15.43:0/2496790055 wait complete. 2024-09-11T23:22:37.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.905+0000 7f4eb1374740 20 client.15507 trim_cache size 0 max 16384 2024-09-11T23:22:37.705 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:22:37.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.905+0000 7f4eaef9b640 1 -- 172.21.15.43:0/2240274906 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f4e980a2e10 msgr2=0x7f4e980a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:37.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.905+0000 7f4eaef9b640 1 --2- 172.21.15.43:0/2240274906 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f4e980a2e10 0x7f4e980a52d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4ea001f000 tx=0x7f4ea0020000 comp rx=0 tx=0).stop 2024-09-11T23:22:37.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.905+0000 7f4eaef9b640 1 -- 172.21.15.43:0/2240274906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ea8102570 msgr2=0x7f4ea8198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:37.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.905+0000 7f4eaef9b640 1 --2- 172.21.15.43:0/2240274906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ea8102570 0x7f4ea8198ab0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400c970 tx=0x7f4e9400ce30 comp rx=0 tx=0).stop 2024-09-11T23:22:37.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.906+0000 7f4eaef9b640 1 -- 172.21.15.43:0/2240274906 shutdown_connections 2024-09-11T23:22:37.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.906+0000 7f4eaef9b640 1 --2- 172.21.15.43:0/2240274906 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f4e980a2e10 0x7f4e980a52d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.906+0000 7f4eaef9b640 1 --2- 172.21.15.43:0/2240274906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ea8102ef0 0x7f4ea819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.906+0000 7f4eaef9b640 1 --2- 172.21.15.43:0/2240274906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ea8102570 0x7f4ea8198ab0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.906+0000 7f4eaef9b640 1 --2- 172.21.15.43:0/2240274906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ea8100e20 0x7f4ea8198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:37.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.906+0000 7f4eaef9b640 1 -- 172.21.15.43:0/2240274906 >> 172.21.15.43:0/2240274906 conn(0x7f4ea80fc7d0 msgr2=0x7f4ea806a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:37.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.906+0000 7f4eaef9b640 1 -- 172.21.15.43:0/2240274906 shutdown_connections 2024-09-11T23:22:37.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:36.906+0000 7f4eaef9b640 1 -- 172.21.15.43:0/2240274906 wait complete. 2024-09-11T23:22:38.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:37 smithi043 ceph-mon[30950]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-11T23:22:38.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:37 smithi090 ceph-mon[35130]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-11T23:22:38.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:37 smithi097 ceph-mon[35562]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-11T23:22:40.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:39 smithi043 ceph-mon[30950]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-11T23:22:40.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:39 smithi090 ceph-mon[35130]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-11T23:22:40.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:39 smithi097 ceph-mon[35562]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-11T23:22:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.071+0000 7fa5438d0640 1 -- 172.21.15.43:0/2106488114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa53c103620 msgr2=0x7fa53c10bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.071+0000 7fa5438d0640 1 --2- 172.21.15.43:0/2106488114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa53c103620 0x7fa53c10bb00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009900 tx=0x7fa52c02a480 comp rx=0 tx=0).stop 2024-09-11T23:22:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.071+0000 7fa5438d0640 1 -- 172.21.15.43:0/2106488114 shutdown_connections 2024-09-11T23:22:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.071+0000 7fa5438d0640 1 --2- 172.21.15.43:0/2106488114 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa53c10c0d0 0x7fa53c10e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.071+0000 7fa5438d0640 1 --2- 172.21.15.43:0/2106488114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa53c103620 0x7fa53c10bb00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.071+0000 7fa5438d0640 1 --2- 172.21.15.43:0/2106488114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa53c102d00 0x7fa53c1030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.071+0000 7fa5438d0640 1 -- 172.21.15.43:0/2106488114 >> 172.21.15.43:0/2106488114 conn(0x7fa53c0779a0 msgr2=0x7fa53c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.071+0000 7fa5438d0640 1 -- 172.21.15.43:0/2106488114 shutdown_connections 2024-09-11T23:22:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.071+0000 7fa5438d0640 1 -- 172.21.15.43:0/2106488114 wait complete. 2024-09-11T23:22:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.072+0000 7fa5438d0640 1 Processor -- start 2024-09-11T23:22:41.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.072+0000 7fa5438d0640 1 -- start start 2024-09-11T23:22:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.072+0000 7fa5438d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa53c102d00 0x7fa53c100950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.072+0000 7fa5438d0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa53c103620 0x7fa53c100e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.072+0000 7fa5438d0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa53c10c0d0 0x7fa53c1013d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.072+0000 7fa5438d0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c111ea0 con 0x7fa53c102d00 2024-09-11T23:22:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.072+0000 7fa5438d0640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c111d20 con 0x7fa53c103620 2024-09-11T23:22:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.072+0000 7fa5438d0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c112020 con 0x7fa53c10c0d0 2024-09-11T23:22:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.072+0000 7fa54166c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa53c102d00 0x7fa53c100950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.072+0000 7fa54166c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa53c102d00 0x7fa53c100950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53834/0 (socket says 172.21.15.43:53834) 2024-09-11T23:22:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.072+0000 7fa54166c640 1 -- 172.21.15.43:0/594661783 learned_addr learned my addr 172.21.15.43:0/594661783 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.072+0000 7fa541e6d640 1 --2- 172.21.15.43:0/594661783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa53c10c0d0 0x7fa53c1013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.072+0000 7fa540e6b640 1 --2- 172.21.15.43:0/594661783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa53c103620 0x7fa53c100e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.073+0000 7fa54166c640 1 -- 172.21.15.43:0/594661783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa53c10c0d0 msgr2=0x7fa53c1013d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.073+0000 7fa54166c640 1 --2- 172.21.15.43:0/594661783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa53c10c0d0 0x7fa53c1013d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.073+0000 7fa54166c640 1 -- 172.21.15.43:0/594661783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa53c103620 msgr2=0x7fa53c100e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.073+0000 7fa54166c640 1 --2- 172.21.15.43:0/594661783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa53c103620 0x7fa53c100e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.073+0000 7fa54166c640 1 -- 172.21.15.43:0/594661783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53c1a6910 con 0x7fa53c102d00 2024-09-11T23:22:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.073+0000 7fa540e6b640 1 --2- 172.21.15.43:0/594661783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa53c103620 0x7fa53c100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:22:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.073+0000 7fa541e6d640 1 --2- 172.21.15.43:0/594661783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa53c10c0d0 0x7fa53c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.073+0000 7fa54166c640 1 --2- 172.21.15.43:0/594661783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa53c102d00 0x7fa53c100950 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa53800d950 tx=0x7fa53800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.073+0000 7fa5327fc640 1 -- 172.21.15.43:0/594661783 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa538017030 con 0x7fa53c102d00 2024-09-11T23:22:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.073+0000 7fa5327fc640 1 -- 172.21.15.43:0/594661783 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa538004020 con 0x7fa53c102d00 2024-09-11T23:22:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.073+0000 7fa5438d0640 1 -- 172.21.15.43:0/594661783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa53c1a6c00 con 0x7fa53c102d00 2024-09-11T23:22:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.073+0000 7fa5327fc640 1 -- 172.21.15.43:0/594661783 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5380047a0 con 0x7fa53c102d00 2024-09-11T23:22:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.073+0000 7fa5438d0640 1 -- 172.21.15.43:0/594661783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa53c1a71a0 con 0x7fa53c102d00 2024-09-11T23:22:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.075+0000 7fa5327fc640 1 -- 172.21.15.43:0/594661783 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fa5380041d0 con 0x7fa53c102d00 2024-09-11T23:22:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.076+0000 7fa5327fc640 1 --2- 172.21.15.43:0/594661783 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa5280a2e90 0x7fa5280a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.076+0000 7fa5327fc640 1 -- 172.21.15.43:0/594661783 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fa5380cf130 con 0x7fa53c102d00 2024-09-11T23:22:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.076+0000 7fa540e6b640 1 --2- 172.21.15.43:0/594661783 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa5280a2e90 0x7fa5280a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.077+0000 7fa546666740 1 Processor -- start 2024-09-11T23:22:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.077+0000 7fa546666740 1 -- start start 2024-09-11T23:22:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.077+0000 7fa540e6b640 1 --2- 172.21.15.43:0/594661783 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa5280a2e90 0x7fa5280a5350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa52c000c00 tx=0x7fa52c032c70 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.077+0000 7fa546666740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f8daf0 0x55bf73f8ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.077+0000 7fa546666740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8e410 0x55bf73f98770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.077+0000 7fa546666740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f98cb0 0x55bf73f9b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.077+0000 7fa546666740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bf73ef22f0 con 0x55bf73f98cb0 2024-09-11T23:22:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.077+0000 7fa546666740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bf73e9a270 con 0x55bf73f8e410 2024-09-11T23:22:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.077+0000 7fa546666740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bf73eee2b0 con 0x55bf73f8daf0 2024-09-11T23:22:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.077+0000 7fa541e6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f98cb0 0x55bf73f9b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.077+0000 7fa541e6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f98cb0 0x55bf73f9b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53850/0 (socket says 172.21.15.43:53850) 2024-09-11T23:22:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.077+0000 7fa541e6d640 1 -- 172.21.15.43:0/1406574110 learned_addr learned my addr 172.21.15.43:0/1406574110 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.077+0000 7fa54166c640 1 --2- 172.21.15.43:0/1406574110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f8daf0 0x55bf73f8ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.077+0000 7fa540e6b640 1 --2- 172.21.15.43:0/1406574110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8e410 0x55bf73f98770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.077+0000 7fa541e6d640 1 -- 172.21.15.43:0/1406574110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f8daf0 msgr2=0x55bf73f8ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.078+0000 7fa541e6d640 1 --2- 172.21.15.43:0/1406574110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f8daf0 0x55bf73f8ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.078+0000 7fa541e6d640 1 -- 172.21.15.43:0/1406574110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8e410 msgr2=0x55bf73f98770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.078+0000 7fa541e6d640 1 --2- 172.21.15.43:0/1406574110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8e410 0x55bf73f98770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.078+0000 7fa541e6d640 1 -- 172.21.15.43:0/1406574110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf73e99340 con 0x55bf73f98cb0 2024-09-11T23:22:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.078+0000 7fa540e6b640 1 --2- 172.21.15.43:0/1406574110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8e410 0x55bf73f98770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:22:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.078+0000 7fa54166c640 1 --2- 172.21.15.43:0/1406574110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f8daf0 0x55bf73f8ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.078+0000 7fa541e6d640 1 --2- 172.21.15.43:0/1406574110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f98cb0 0x55bf73f9b0a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa53400ae80 tx=0x7fa53400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21403d737c04834b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.078+0000 7fa5277fe640 1 -- 172.21.15.43:0/1406574110 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa534003b70 con 0x55bf73f98cb0 2024-09-11T23:22:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.078+0000 7fa5277fe640 1 -- 172.21.15.43:0/1406574110 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa534010b20 con 0x55bf73f98cb0 2024-09-11T23:22:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.078+0000 7fa5277fe640 1 -- 172.21.15.43:0/1406574110 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53400fb30 con 0x55bf73f98cb0 2024-09-11T23:22:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.079+0000 7fa546666740 1 -- 172.21.15.43:0/1406574110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f98cb0 msgr2=0x55bf73f9b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.079+0000 7fa546666740 1 --2- 172.21.15.43:0/1406574110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f98cb0 0x55bf73f9b0a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa53400ae80 tx=0x7fa53400e720 comp rx=0 tx=0).stop 2024-09-11T23:22:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.079+0000 7fa546666740 1 -- 172.21.15.43:0/1406574110 shutdown_connections 2024-09-11T23:22:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.079+0000 7fa546666740 1 --2- 172.21.15.43:0/1406574110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f98cb0 0x55bf73f9b0a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.079+0000 7fa546666740 1 --2- 172.21.15.43:0/1406574110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8e410 0x55bf73f98770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.079+0000 7fa546666740 1 --2- 172.21.15.43:0/1406574110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f8daf0 0x55bf73f8ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.079+0000 7fa546666740 1 -- 172.21.15.43:0/1406574110 >> 172.21.15.43:0/1406574110 conn(0x55bf73f8a060 msgr2=0x55bf73f8a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.079+0000 7fa546666740 1 -- 172.21.15.43:0/1406574110 shutdown_connections 2024-09-11T23:22:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.079+0000 7fa546666740 1 -- 172.21.15.43:0/1406574110 wait complete. 2024-09-11T23:22:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.079+0000 7fa546666740 1 -- start start 2024-09-11T23:22:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.079+0000 7fa546666740 1 Processor -- start 2024-09-11T23:22:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.080+0000 7fa546666740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f8daf0 0x55bf73f95b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.080+0000 7fa546666740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8e410 0x55bf73f96090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.080+0000 7fa546666740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f98cb0 0x55bf73f965d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.080+0000 7fa546666740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bf73f9b880 con 0x55bf73f8daf0 2024-09-11T23:22:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.080+0000 7fa546666740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bf73e7b290 con 0x55bf73f8e410 2024-09-11T23:22:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.080+0000 7fa546666740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bf73f9ba00 con 0x55bf73f98cb0 2024-09-11T23:22:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.080+0000 7fa546666740 10 client.0 _ll_register_callbacks cb 0x55bf73eac140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.080+0000 7fa54166c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f8daf0 0x55bf73f95b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.080+0000 7fa54166c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f8daf0 0x55bf73f95b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53852/0 (socket says 172.21.15.43:53852) 2024-09-11T23:22:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.080+0000 7fa54166c640 1 -- 172.21.15.43:0/667708814 learned_addr learned my addr 172.21.15.43:0/667708814 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.080+0000 7fa541e6d640 1 --2- 172.21.15.43:0/667708814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f98cb0 0x55bf73f965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.080+0000 7fa540e6b640 1 --2- 172.21.15.43:0/667708814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8e410 0x55bf73f96090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.080+0000 7fa54166c640 1 -- 172.21.15.43:0/667708814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f98cb0 msgr2=0x55bf73f965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.080+0000 7fa54166c640 1 --2- 172.21.15.43:0/667708814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f98cb0 0x55bf73f965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.080+0000 7fa54166c640 1 -- 172.21.15.43:0/667708814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8e410 msgr2=0x55bf73f96090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.080+0000 7fa54166c640 1 --2- 172.21.15.43:0/667708814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8e410 0x55bf73f96090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.080+0000 7fa54166c640 1 -- 172.21.15.43:0/667708814 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf73c05210 con 0x55bf73f8daf0 2024-09-11T23:22:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.080+0000 7fa54166c640 1 --2- 172.21.15.43:0/667708814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f8daf0 0x55bf73f95b50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa538097860 tx=0x7fa5380977c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bc0eb0d9e44d47 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.080+0000 7fa524ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.081+0000 7fa524ff9640 1 -- 172.21.15.43:0/667708814 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5380ccbe0 con 0x55bf73f8daf0 2024-09-11T23:22:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.081+0000 7fa524ff9640 1 -- 172.21.15.43:0/667708814 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fa53809a850 con 0x55bf73f8daf0 2024-09-11T23:22:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.081+0000 7fa546666740 1 -- 172.21.15.43:0/667708814 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa534010b20 con 0x55bf73f8daf0 2024-09-11T23:22:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.081+0000 7fa524ff9640 1 -- 172.21.15.43:0/667708814 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa53809b400 con 0x55bf73f8daf0 2024-09-11T23:22:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.081+0000 7fa546666740 10 client.15519 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.081+0000 7fa546666740 1 -- 172.21.15.43:0/667708814 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bf73cd1620 con 0x55bf73f8daf0 2024-09-11T23:22:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.081+0000 7fa524ff9640 1 -- 172.21.15.43:0/667708814 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa534010b20 con 0x55bf73f8daf0 2024-09-11T23:22:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.081+0000 7fa524ff9640 1 -- 172.21.15.43:0/667708814 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fa53809a2c0 con 0x55bf73f8daf0 2024-09-11T23:22:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.081+0000 7fa546666740 10 client.15519 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.081+0000 7fa546666740 10 client.15519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.081+0000 7fa546666740 20 client.15519 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.081+0000 7fa546666740 1 --2- 172.21.15.43:0/667708814 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bf73fa8b60 0x55bf73faaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.081+0000 7fa546666740 4 client.15519 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:22:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.081+0000 7fa546666740 1 -- 172.21.15.43:0/667708814 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf73effa20 con 0x55bf73fa8b60 2024-09-11T23:22:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.082+0000 7fa540e6b640 1 --2- 172.21.15.43:0/667708814 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bf73fa8b60 0x55bf73faaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.082+0000 7fa524ff9640 10 client.15519 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.082+0000 7fa540e6b640 1 --2- 172.21.15.43:0/667708814 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bf73fa8b60 0x55bf73faaf50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58968deda6273244 server_cookie=d42d51ac85e84b7e in_seq=0 out_seq=0 2024-09-11T23:22:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.083+0000 7fa524ff9640 1 -- 172.21.15.43:0/667708814 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf73effa20 con 0x55bf73fa8b60 2024-09-11T23:22:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.083+0000 7fa524ff9640 10 client.15519 handle_command_reply: tid=0 2024-09-11T23:22:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.083+0000 7fa546666740 1 client.15519 shutdown 2024-09-11T23:22:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.083+0000 7fa546666740 1 -- 172.21.15.43:0/667708814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f8daf0 msgr2=0x55bf73f95b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.083+0000 7fa546666740 1 --2- 172.21.15.43:0/667708814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f8daf0 0x55bf73f95b50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa538097860 tx=0x7fa5380977c0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.083+0000 7fa546666740 1 -- 172.21.15.43:0/667708814 shutdown_connections 2024-09-11T23:22:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.083+0000 7fa546666740 1 --2- 172.21.15.43:0/667708814 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bf73fa8b60 0x55bf73faaf50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.083+0000 7fa546666740 1 --2- 172.21.15.43:0/667708814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f98cb0 0x55bf73f965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.083+0000 7fa524ff9640 0 client.15519 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.083+0000 7fa546666740 1 --2- 172.21.15.43:0/667708814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8e410 0x55bf73f96090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.083+0000 7fa546666740 1 --2- 172.21.15.43:0/667708814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f8daf0 0x55bf73f95b50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.083+0000 7fa546666740 1 -- 172.21.15.43:0/667708814 >> 172.21.15.43:0/667708814 conn(0x55bf73f8a060 msgr2=0x55bf73f8fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.083+0000 7fa546666740 1 -- 172.21.15.43:0/667708814 shutdown_connections 2024-09-11T23:22:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.083+0000 7fa546666740 1 -- 172.21.15.43:0/667708814 wait complete. 2024-09-11T23:22:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.083+0000 7fa546666740 20 client.15519 trim_cache size 0 max 16384 2024-09-11T23:22:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.104+0000 7fa546666740 1 Processor -- start 2024-09-11T23:22:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.104+0000 7fa546666740 1 -- start start 2024-09-11T23:22:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.104+0000 7fa546666740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8daf0 0x55bf73f96080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.104+0000 7fa546666740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f8e410 0x55bf73f965c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.104+0000 7fa546666740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f98cb0 0x55bf73fb0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.104+0000 7fa546666740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c1b2490 con 0x55bf73f98cb0 2024-09-11T23:22:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.104+0000 7fa546666740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c1afd60 con 0x55bf73f8daf0 2024-09-11T23:22:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.104+0000 7fa546666740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c1adbb0 con 0x55bf73f8e410 2024-09-11T23:22:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.104+0000 7fa541e6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f98cb0 0x55bf73fb0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.104+0000 7fa541e6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f98cb0 0x55bf73fb0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53866/0 (socket says 172.21.15.43:53866) 2024-09-11T23:22:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.104+0000 7fa541e6d640 1 -- 172.21.15.43:0/2102129424 learned_addr learned my addr 172.21.15.43:0/2102129424 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.105+0000 7fa54166c640 1 --2- 172.21.15.43:0/2102129424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8daf0 0x55bf73f96080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.105+0000 7fa540e6b640 1 --2- 172.21.15.43:0/2102129424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f8e410 0x55bf73f965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.105+0000 7fa541e6d640 1 -- 172.21.15.43:0/2102129424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f8e410 msgr2=0x55bf73f965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.105+0000 7fa541e6d640 1 --2- 172.21.15.43:0/2102129424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f8e410 0x55bf73f965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.105+0000 7fa541e6d640 1 -- 172.21.15.43:0/2102129424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8daf0 msgr2=0x55bf73f96080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.105+0000 7fa541e6d640 1 --2- 172.21.15.43:0/2102129424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8daf0 0x55bf73f96080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.105+0000 7fa541e6d640 1 -- 172.21.15.43:0/2102129424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf73f986a0 con 0x55bf73f98cb0 2024-09-11T23:22:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.105+0000 7fa540e6b640 1 --2- 172.21.15.43:0/2102129424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f8e410 0x55bf73f965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:22:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.105+0000 7fa54166c640 1 --2- 172.21.15.43:0/2102129424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8daf0 0x55bf73f96080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.105+0000 7fa541e6d640 1 --2- 172.21.15.43:0/2102129424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f98cb0 0x55bf73fb0860 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa534018000 tx=0x7fa53400abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce444e7d13ec0b33 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.105+0000 7fa526ffd640 1 -- 172.21.15.43:0/2102129424 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa534004c00 con 0x55bf73f98cb0 2024-09-11T23:22:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.105+0000 7fa526ffd640 1 -- 172.21.15.43:0/2102129424 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa534003e50 con 0x55bf73f98cb0 2024-09-11T23:22:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.105+0000 7fa526ffd640 1 -- 172.21.15.43:0/2102129424 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa534010640 con 0x55bf73f98cb0 2024-09-11T23:22:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.106+0000 7fa546666740 1 -- 172.21.15.43:0/2102129424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f98cb0 msgr2=0x55bf73fb0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.106+0000 7fa546666740 1 --2- 172.21.15.43:0/2102129424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f98cb0 0x55bf73fb0860 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa534018000 tx=0x7fa53400abd0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.106+0000 7fa546666740 1 -- 172.21.15.43:0/2102129424 shutdown_connections 2024-09-11T23:22:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.106+0000 7fa546666740 1 --2- 172.21.15.43:0/2102129424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f98cb0 0x55bf73fb0860 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.106+0000 7fa546666740 1 --2- 172.21.15.43:0/2102129424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f8e410 0x55bf73f965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.106+0000 7fa546666740 1 --2- 172.21.15.43:0/2102129424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8daf0 0x55bf73f96080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.106+0000 7fa546666740 1 -- 172.21.15.43:0/2102129424 >> 172.21.15.43:0/2102129424 conn(0x55bf73f8a060 msgr2=0x55bf73f97bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.106+0000 7fa546666740 1 -- 172.21.15.43:0/2102129424 shutdown_connections 2024-09-11T23:22:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.106+0000 7fa546666740 1 -- 172.21.15.43:0/2102129424 wait complete. 2024-09-11T23:22:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.106+0000 7fa546666740 1 -- start start 2024-09-11T23:22:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.106+0000 7fa546666740 1 Processor -- start 2024-09-11T23:22:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.107+0000 7fa546666740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8daf0 0x55bf73fa13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.107+0000 7fa546666740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f8e410 0x55bf73f92880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.107+0000 7fa546666740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f98cb0 0x55bf73f92dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.107+0000 7fa546666740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c1ae220 con 0x55bf73f8e410 2024-09-11T23:22:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.107+0000 7fa546666740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c1af690 con 0x55bf73f8daf0 2024-09-11T23:22:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.107+0000 7fa546666740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c1ae8f0 con 0x55bf73f98cb0 2024-09-11T23:22:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.107+0000 7fa546666740 10 client.0 _ll_register_callbacks cb 0x7fa53809b1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.107+0000 7fa540e6b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f8e410 0x55bf73f92880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.107+0000 7fa540e6b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f8e410 0x55bf73f92880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53882/0 (socket says 172.21.15.43:53882) 2024-09-11T23:22:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.107+0000 7fa540e6b640 1 -- 172.21.15.43:0/918999217 learned_addr learned my addr 172.21.15.43:0/918999217 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.107+0000 7fa54166c640 1 --2- 172.21.15.43:0/918999217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8daf0 0x55bf73fa13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.107+0000 7fa541e6d640 1 --2- 172.21.15.43:0/918999217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f98cb0 0x55bf73f92dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.107+0000 7fa540e6b640 1 -- 172.21.15.43:0/918999217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f98cb0 msgr2=0x55bf73f92dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.107+0000 7fa540e6b640 1 --2- 172.21.15.43:0/918999217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f98cb0 0x55bf73f92dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.107+0000 7fa540e6b640 1 -- 172.21.15.43:0/918999217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8daf0 msgr2=0x55bf73fa13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.107+0000 7fa540e6b640 1 --2- 172.21.15.43:0/918999217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8daf0 0x55bf73fa13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.107+0000 7fa540e6b640 1 -- 172.21.15.43:0/918999217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf73f935f0 con 0x55bf73f8e410 2024-09-11T23:22:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.107+0000 7fa540e6b640 1 --2- 172.21.15.43:0/918999217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f8e410 0x55bf73f92880 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009900 tx=0x7fa52c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00decfec450649d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.108+0000 7fa524ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.108+0000 7fa524ff9640 1 -- 172.21.15.43:0/918999217 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c035050 con 0x55bf73f8e410 2024-09-11T23:22:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.108+0000 7fa524ff9640 1 -- 172.21.15.43:0/918999217 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fa52c005250 con 0x55bf73f8e410 2024-09-11T23:22:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.108+0000 7fa546666740 1 -- 172.21.15.43:0/918999217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa534003e50 con 0x55bf73f8e410 2024-09-11T23:22:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.108+0000 7fa524ff9640 1 -- 172.21.15.43:0/918999217 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa52c005450 con 0x55bf73f8e410 2024-09-11T23:22:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.108+0000 7fa546666740 10 client.15525 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.108+0000 7fa546666740 1 -- 172.21.15.43:0/918999217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bf740137a0 con 0x55bf73f8e410 2024-09-11T23:22:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.108+0000 7fa524ff9640 1 -- 172.21.15.43:0/918999217 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa534003e50 con 0x55bf73f8e410 2024-09-11T23:22:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.108+0000 7fa524ff9640 1 -- 172.21.15.43:0/918999217 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fa52c0035f0 con 0x55bf73f8e410 2024-09-11T23:22:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.108+0000 7fa546666740 10 client.15525 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.108+0000 7fa546666740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.108+0000 7fa546666740 20 client.15525 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.108+0000 7fa546666740 1 --2- 172.21.15.43:0/918999217 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bf73fa8b60 0x55bf74015f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.108+0000 7fa546666740 4 client.15525 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:22:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.108+0000 7fa546666740 1 -- 172.21.15.43:0/918999217 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf73ef9d70 con 0x55bf73fa8b60 2024-09-11T23:22:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.109+0000 7fa54166c640 1 --2- 172.21.15.43:0/918999217 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bf73fa8b60 0x55bf74015f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.109+0000 7fa524ff9640 10 client.15525 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.109+0000 7fa54166c640 1 --2- 172.21.15.43:0/918999217 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bf73fa8b60 0x55bf74015f60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f730a2bd5759d2f8 server_cookie=2382760a061de49d in_seq=0 out_seq=0 2024-09-11T23:22:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.115+0000 7fa524ff9640 1 -- 172.21.15.43:0/918999217 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf73ef9d70 con 0x55bf73fa8b60 2024-09-11T23:22:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.115+0000 7fa524ff9640 10 client.15525 handle_command_reply: tid=0 2024-09-11T23:22:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.115+0000 7fa546666740 1 client.15525 shutdown 2024-09-11T23:22:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.115+0000 7fa546666740 1 -- 172.21.15.43:0/918999217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f8e410 msgr2=0x55bf73f92880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.115+0000 7fa546666740 1 --2- 172.21.15.43:0/918999217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f8e410 0x55bf73f92880 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009900 tx=0x7fa52c009260 comp rx=0 tx=0).stop 2024-09-11T23:22:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.115+0000 7fa546666740 1 -- 172.21.15.43:0/918999217 shutdown_connections 2024-09-11T23:22:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.115+0000 7fa546666740 1 --2- 172.21.15.43:0/918999217 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bf73fa8b60 0x55bf74015f60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.115+0000 7fa546666740 1 --2- 172.21.15.43:0/918999217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf73f98cb0 0x55bf73f92dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.115+0000 7fa546666740 1 --2- 172.21.15.43:0/918999217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf73f8e410 0x55bf73f92880 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.115+0000 7fa524ff9640 0 client.15525 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.115+0000 7fa546666740 1 --2- 172.21.15.43:0/918999217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bf73f8daf0 0x55bf73fa13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.115+0000 7fa546666740 1 -- 172.21.15.43:0/918999217 >> 172.21.15.43:0/918999217 conn(0x55bf73f8a060 msgr2=0x55bf73f889a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.115+0000 7fa546666740 1 -- 172.21.15.43:0/918999217 shutdown_connections 2024-09-11T23:22:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.115+0000 7fa546666740 1 -- 172.21.15.43:0/918999217 wait complete. 2024-09-11T23:22:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.115+0000 7fa546666740 20 client.15525 trim_cache size 0 max 16384 2024-09-11T23:22:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:22:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:22:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:22:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:22:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.116+0000 7fa5438d0640 1 -- 172.21.15.43:0/594661783 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa5280a2e90 msgr2=0x7fa5280a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.116+0000 7fa5438d0640 1 --2- 172.21.15.43:0/594661783 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa5280a2e90 0x7fa5280a5350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa52c000c00 tx=0x7fa52c032c70 comp rx=0 tx=0).stop 2024-09-11T23:22:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.116+0000 7fa5438d0640 1 -- 172.21.15.43:0/594661783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa53c102d00 msgr2=0x7fa53c100950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.116+0000 7fa5438d0640 1 --2- 172.21.15.43:0/594661783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa53c102d00 0x7fa53c100950 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa53800d950 tx=0x7fa53800de10 comp rx=0 tx=0).stop 2024-09-11T23:22:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.116+0000 7fa5438d0640 1 -- 172.21.15.43:0/594661783 shutdown_connections 2024-09-11T23:22:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.116+0000 7fa5438d0640 1 --2- 172.21.15.43:0/594661783 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa5280a2e90 0x7fa5280a5350 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.116+0000 7fa5438d0640 1 --2- 172.21.15.43:0/594661783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa53c10c0d0 0x7fa53c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.116+0000 7fa5438d0640 1 --2- 172.21.15.43:0/594661783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa53c103620 0x7fa53c100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.116+0000 7fa5438d0640 1 --2- 172.21.15.43:0/594661783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa53c102d00 0x7fa53c100950 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.116+0000 7fa5438d0640 1 -- 172.21.15.43:0/594661783 >> 172.21.15.43:0/594661783 conn(0x7fa53c0779a0 msgr2=0x7fa53c0fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.116+0000 7fa5438d0640 1 -- 172.21.15.43:0/594661783 shutdown_connections 2024-09-11T23:22:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:41.116+0000 7fa5438d0640 1 -- 172.21.15.43:0/594661783 wait complete. 2024-09-11T23:22:41.207 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-11T23:22:41.207 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-11T23:22:41.646 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:22:41.719 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:41.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:41 smithi043 ceph-mon[30950]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-11T23:22:42.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:41 smithi090 ceph-mon[35130]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-11T23:22:42.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.242+0000 7f8865655640 1 -- 172.21.15.43:0/1237725964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8860102570 msgr2=0x7f88601029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:42.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.243+0000 7f8865655640 1 --2- 172.21.15.43:0/1237725964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8860102570 0x7f88601029b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8848009900 tx=0x7f884802a480 comp rx=0 tx=0).stop 2024-09-11T23:22:42.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.243+0000 7f8865655640 1 -- 172.21.15.43:0/1237725964 shutdown_connections 2024-09-11T23:22:42.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.243+0000 7f8865655640 1 --2- 172.21.15.43:0/1237725964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8860102ef0 0x7f886010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.243+0000 7f8865655640 1 --2- 172.21.15.43:0/1237725964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8860102570 0x7f88601029b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.243+0000 7f8865655640 1 --2- 172.21.15.43:0/1237725964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8860100e20 0x7f8860101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.243+0000 7f8865655640 1 -- 172.21.15.43:0/1237725964 >> 172.21.15.43:0/1237725964 conn(0x7f88600fc7d0 msgr2=0x7f88600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:42.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.243+0000 7f8865655640 1 -- 172.21.15.43:0/1237725964 shutdown_connections 2024-09-11T23:22:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.243+0000 7f8865655640 1 -- 172.21.15.43:0/1237725964 wait complete. 2024-09-11T23:22:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.243+0000 7f8865655640 1 Processor -- start 2024-09-11T23:22:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.243+0000 7f8865655640 1 -- start start 2024-09-11T23:22:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.244+0000 7f8865655640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8860100e20 0x7f886019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.244+0000 7f8865655640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8860102570 0x7f886019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.244+0000 7f8865655640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8860102ef0 0x7f886019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.244+0000 7f8865655640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8860110fa0 con 0x7f8860102570 2024-09-11T23:22:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.244+0000 7f8865655640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8860110e20 con 0x7f8860100e20 2024-09-11T23:22:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.244+0000 7f8865655640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8860111120 con 0x7f8860102ef0 2024-09-11T23:22:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.244+0000 7f8857fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8860102570 0x7f886019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.244+0000 7f8857fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8860102570 0x7f886019ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53908/0 (socket says 172.21.15.43:53908) 2024-09-11T23:22:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.244+0000 7f8857fff640 1 -- 172.21.15.43:0/513041555 learned_addr learned my addr 172.21.15.43:0/513041555 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.244+0000 7f885effd640 1 --2- 172.21.15.43:0/513041555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8860100e20 0x7f886019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.244+0000 7f885f7fe640 1 --2- 172.21.15.43:0/513041555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8860102ef0 0x7f886019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.244+0000 7f8857fff640 1 -- 172.21.15.43:0/513041555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8860102ef0 msgr2=0x7f886019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.244+0000 7f8857fff640 1 --2- 172.21.15.43:0/513041555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8860102ef0 0x7f886019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.244+0000 7f8857fff640 1 -- 172.21.15.43:0/513041555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8860100e20 msgr2=0x7f886019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.244+0000 7f8857fff640 1 --2- 172.21.15.43:0/513041555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8860100e20 0x7f886019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.244+0000 7f8857fff640 1 -- 172.21.15.43:0/513041555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f886019f670 con 0x7f8860102570 2024-09-11T23:22:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.245+0000 7f8857fff640 1 --2- 172.21.15.43:0/513041555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8860102570 0x7f886019ac00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8848006010 tx=0x7f8848003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.245+0000 7f885cff9640 1 -- 172.21.15.43:0/513041555 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884803f070 con 0x7f8860102570 2024-09-11T23:22:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.245+0000 7f8865655640 1 -- 172.21.15.43:0/513041555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f886019f900 con 0x7f8860102570 2024-09-11T23:22:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.245+0000 7f885cff9640 1 -- 172.21.15.43:0/513041555 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8848003e30 con 0x7f8860102570 2024-09-11T23:22:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.245+0000 7f885cff9640 1 -- 172.21.15.43:0/513041555 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848035030 con 0x7f8860102570 2024-09-11T23:22:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.245+0000 7f8865655640 1 -- 172.21.15.43:0/513041555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88601a7240 con 0x7f8860102570 2024-09-11T23:22:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.247+0000 7f8867a2e740 1 Processor -- start 2024-09-11T23:22:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.248+0000 7f8867a2e740 1 -- start start 2024-09-11T23:22:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.248+0000 7f885cff9640 1 -- 172.21.15.43:0/513041555 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f8848042070 con 0x7f8860102570 2024-09-11T23:22:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.248+0000 7f8867a2e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2e95760 0x55f1c2e95b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.248+0000 7f8867a2e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2ea0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.248+0000 7f8867a2e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2ea0940 0x55f1c2ea2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.248+0000 7f8867a2e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f1c2dfb2a0 con 0x55f1c2e96080 2024-09-11T23:22:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.248+0000 7f8867a2e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f1c2da2b10 con 0x55f1c2e95760 2024-09-11T23:22:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.248+0000 7f8867a2e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f1c2df7260 con 0x55f1c2ea0940 2024-09-11T23:22:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.248+0000 7f885effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2ea0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.248+0000 7f885effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2ea0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53922/0 (socket says 172.21.15.43:53922) 2024-09-11T23:22:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.248+0000 7f885effd640 1 -- 172.21.15.43:0/493185367 learned_addr learned my addr 172.21.15.43:0/493185367 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.248+0000 7f885f7fe640 1 --2- 172.21.15.43:0/493185367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2e95760 0x55f1c2e95b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:42.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.248+0000 7f8857fff640 1 --2- 172.21.15.43:0/493185367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2ea0940 0x55f1c2ea2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:42.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.249+0000 7f885effd640 1 -- 172.21.15.43:0/493185367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2ea0940 msgr2=0x55f1c2ea2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:42.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.249+0000 7f885effd640 1 --2- 172.21.15.43:0/493185367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2ea0940 0x55f1c2ea2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.249+0000 7f885effd640 1 -- 172.21.15.43:0/493185367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2e95760 msgr2=0x55f1c2e95b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:42.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.249+0000 7f885effd640 1 --2- 172.21.15.43:0/493185367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2e95760 0x55f1c2e95b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.249+0000 7f885effd640 1 -- 172.21.15.43:0/493185367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1c2b6cb90 con 0x55f1c2e96080 2024-09-11T23:22:42.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.249+0000 7f8857fff640 1 --2- 172.21.15.43:0/493185367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2ea0940 0x55f1c2ea2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:22:42.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.249+0000 7f885f7fe640 1 --2- 172.21.15.43:0/493185367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2e95760 0x55f1c2e95b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:42.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.249+0000 7f885cff9640 1 --2- 172.21.15.43:0/513041555 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f884c0a2e10 0x7f884c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:42.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.249+0000 7f885effd640 1 --2- 172.21.15.43:0/493185367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2ea0400 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f885000ad00 tx=0x7f885000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a4af4b915116dc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:42.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.249+0000 7f885f7fe640 1 --2- 172.21.15.43:0/513041555 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f884c0a2e10 0x7f884c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:42.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.249+0000 7f885cff9640 1 -- 172.21.15.43:0/513041555 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f88480eff60 con 0x7f8860102570 2024-09-11T23:22:42.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.249+0000 7f88557fa640 1 -- 172.21.15.43:0/493185367 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885001d070 con 0x55f1c2e96080 2024-09-11T23:22:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.249+0000 7f88557fa640 1 -- 172.21.15.43:0/493185367 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8850018b20 con 0x55f1c2e96080 2024-09-11T23:22:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.249+0000 7f88557fa640 1 -- 172.21.15.43:0/493185367 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850010c20 con 0x55f1c2e96080 2024-09-11T23:22:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.250+0000 7f8867a2e740 1 -- 172.21.15.43:0/493185367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 msgr2=0x55f1c2ea0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.250+0000 7f8867a2e740 1 --2- 172.21.15.43:0/493185367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2ea0400 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f885000ad00 tx=0x7f885000f790 comp rx=0 tx=0).stop 2024-09-11T23:22:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.250+0000 7f885f7fe640 1 --2- 172.21.15.43:0/513041555 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f884c0a2e10 0x7f884c0a52d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f885800af80 tx=0x7f885800a4d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.250+0000 7f8867a2e740 1 -- 172.21.15.43:0/493185367 shutdown_connections 2024-09-11T23:22:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.250+0000 7f8867a2e740 1 --2- 172.21.15.43:0/493185367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2ea0940 0x55f1c2ea2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.250+0000 7f8867a2e740 1 --2- 172.21.15.43:0/493185367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2ea0400 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.250+0000 7f8867a2e740 1 --2- 172.21.15.43:0/493185367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2e95760 0x55f1c2e95b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.250+0000 7f8867a2e740 1 -- 172.21.15.43:0/493185367 >> 172.21.15.43:0/493185367 conn(0x55f1c2e91cf0 msgr2=0x55f1c2e920d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.250+0000 7f8867a2e740 1 -- 172.21.15.43:0/493185367 shutdown_connections 2024-09-11T23:22:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.250+0000 7f8867a2e740 1 -- 172.21.15.43:0/493185367 wait complete. 2024-09-11T23:22:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.251+0000 7f8867a2e740 1 -- start start 2024-09-11T23:22:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.251+0000 7f8867a2e740 1 Processor -- start 2024-09-11T23:22:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.251+0000 7f8867a2e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2e95760 0x55f1c2e9d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.251+0000 7f8867a2e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2e9dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.251+0000 7f8867a2e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2ea0940 0x55f1c2e9e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.251+0000 7f8867a2e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f1c2ea3510 con 0x55f1c2e96080 2024-09-11T23:22:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.251+0000 7f8867a2e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f1c2d836e0 con 0x55f1c2ea0940 2024-09-11T23:22:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.251+0000 7f8867a2e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f1c2ea3690 con 0x55f1c2e95760 2024-09-11T23:22:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.251+0000 7f8867a2e740 10 client.0 _ll_register_callbacks cb 0x55f1c2da7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.251+0000 7f8857fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2e9dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.251+0000 7f8857fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2e9dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53934/0 (socket says 172.21.15.43:53934) 2024-09-11T23:22:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.251+0000 7f8857fff640 1 -- 172.21.15.43:0/1862071779 learned_addr learned my addr 172.21.15.43:0/1862071779 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.251+0000 7f885effd640 1 --2- 172.21.15.43:0/1862071779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2e95760 0x55f1c2e9d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.251+0000 7f885f7fe640 1 --2- 172.21.15.43:0/1862071779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2ea0940 0x55f1c2e9e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.252+0000 7f8857fff640 1 -- 172.21.15.43:0/1862071779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2e95760 msgr2=0x55f1c2e9d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.252+0000 7f8857fff640 1 --2- 172.21.15.43:0/1862071779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2e95760 0x55f1c2e9d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.252+0000 7f8857fff640 1 -- 172.21.15.43:0/1862071779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2ea0940 msgr2=0x55f1c2e9e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.252+0000 7f8857fff640 1 --2- 172.21.15.43:0/1862071779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2ea0940 0x55f1c2e9e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.252+0000 7f8857fff640 1 -- 172.21.15.43:0/1862071779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1c2eaca00 con 0x55f1c2e96080 2024-09-11T23:22:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.252+0000 7f885effd640 1 --2- 172.21.15.43:0/1862071779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2e95760 0x55f1c2e9d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.252+0000 7f885f7fe640 1 --2- 172.21.15.43:0/1862071779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2ea0940 0x55f1c2e9e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.252+0000 7f8857fff640 1 --2- 172.21.15.43:0/1862071779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2e9dd50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8848003c80 tx=0x7f88480bc200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65eb6fe2f07e953 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.252+0000 7f88467fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.252+0000 7f88467fc640 1 -- 172.21.15.43:0/1862071779 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848049070 con 0x55f1c2e96080 2024-09-11T23:22:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.252+0000 7f88467fc640 1 -- 172.21.15.43:0/1862071779 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f88480b8970 con 0x55f1c2e96080 2024-09-11T23:22:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.252+0000 7f8867a2e740 1 -- 172.21.15.43:0/1862071779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8850018b20 con 0x55f1c2e96080 2024-09-11T23:22:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.252+0000 7f88467fc640 1 -- 172.21.15.43:0/1862071779 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f88480bca00 con 0x55f1c2e96080 2024-09-11T23:22:42.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.252+0000 7f8867a2e740 10 client.15537 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:42.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.252+0000 7f8867a2e740 1 -- 172.21.15.43:0/1862071779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f1c2eaf2e0 con 0x55f1c2e96080 2024-09-11T23:22:42.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.253+0000 7f88467fc640 1 -- 172.21.15.43:0/1862071779 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8850018b20 con 0x55f1c2e96080 2024-09-11T23:22:42.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.253+0000 7f88467fc640 1 -- 172.21.15.43:0/1862071779 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f88480d5120 con 0x55f1c2e96080 2024-09-11T23:22:42.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.253+0000 7f8867a2e740 10 client.15537 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:42.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.253+0000 7f8867a2e740 10 client.15537 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:22:42.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.253+0000 7f8867a2e740 20 client.15537 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:42.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.253+0000 7f8867a2e740 1 --2- 172.21.15.43:0/1862071779 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f1c2eb0740 0x55f1c2eb2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:42.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.253+0000 7f8867a2e740 4 client.15537 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:22:42.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.253+0000 7f8867a2e740 1 -- 172.21.15.43:0/1862071779 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1c2e089d0 con 0x55f1c2eb0740 2024-09-11T23:22:42.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.253+0000 7f885effd640 1 --2- 172.21.15.43:0/1862071779 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f1c2eb0740 0x55f1c2eb2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:42.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.254+0000 7f88467fc640 10 client.15537 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:42.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.254+0000 7f885effd640 1 --2- 172.21.15.43:0/1862071779 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f1c2eb0740 0x55f1c2eb2ba0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3381df0e58d6da1 server_cookie=9d97404ef3a37107 in_seq=0 out_seq=0 2024-09-11T23:22:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.254+0000 7f88467fc640 1 -- 172.21.15.43:0/1862071779 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f1c2e089d0 con 0x55f1c2eb0740 2024-09-11T23:22:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.254+0000 7f88467fc640 10 client.15537 handle_command_reply: tid=0 2024-09-11T23:22:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.254+0000 7f8867a2e740 1 client.15537 shutdown 2024-09-11T23:22:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.254+0000 7f8867a2e740 1 -- 172.21.15.43:0/1862071779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 msgr2=0x55f1c2e9dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.254+0000 7f8867a2e740 1 --2- 172.21.15.43:0/1862071779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2e9dd50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8848003c80 tx=0x7f88480bc200 comp rx=0 tx=0).stop 2024-09-11T23:22:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.255+0000 7f8867a2e740 1 -- 172.21.15.43:0/1862071779 shutdown_connections 2024-09-11T23:22:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.255+0000 7f8867a2e740 1 --2- 172.21.15.43:0/1862071779 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f1c2eb0740 0x55f1c2eb2ba0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.255+0000 7f88467fc640 0 client.15537 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.255+0000 7f8867a2e740 1 --2- 172.21.15.43:0/1862071779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2ea0940 0x55f1c2e9e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.255+0000 7f8867a2e740 1 --2- 172.21.15.43:0/1862071779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2e9dd50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.255+0000 7f8867a2e740 1 --2- 172.21.15.43:0/1862071779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2e95760 0x55f1c2e9d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.255+0000 7f8867a2e740 1 -- 172.21.15.43:0/1862071779 >> 172.21.15.43:0/1862071779 conn(0x55f1c2e91cf0 msgr2=0x55f1c2e97a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.255+0000 7f8867a2e740 1 -- 172.21.15.43:0/1862071779 shutdown_connections 2024-09-11T23:22:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.255+0000 7f8867a2e740 1 -- 172.21.15.43:0/1862071779 wait complete. 2024-09-11T23:22:42.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.255+0000 7f8867a2e740 20 client.15537 trim_cache size 0 max 16384 2024-09-11T23:22:42.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.272+0000 7f8867a2e740 1 Processor -- start 2024-09-11T23:22:42.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.272+0000 7f8867a2e740 1 -- start start 2024-09-11T23:22:42.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.272+0000 7f8867a2e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2e95760 0x55f1c2e981c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:42.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.272+0000 7f8867a2e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2eb84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:42.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.272+0000 7f8867a2e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2ea0940 0x55f1c2eb89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:42.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.273+0000 7f8867a2e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f88601b2390 con 0x55f1c2e96080 2024-09-11T23:22:42.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.273+0000 7f8867a2e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f88601afc60 con 0x55f1c2e95760 2024-09-11T23:22:42.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.273+0000 7f8867a2e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f88601adab0 con 0x55f1c2ea0940 2024-09-11T23:22:42.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.273+0000 7f8857fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2eb84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:42.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.273+0000 7f8857fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2eb84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53942/0 (socket says 172.21.15.43:53942) 2024-09-11T23:22:42.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.273+0000 7f8857fff640 1 -- 172.21.15.43:0/22254512 learned_addr learned my addr 172.21.15.43:0/22254512 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:42.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.273+0000 7f885f7fe640 1 --2- 172.21.15.43:0/22254512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2ea0940 0x55f1c2eb89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:42.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.273+0000 7f885effd640 1 --2- 172.21.15.43:0/22254512 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2e95760 0x55f1c2e981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:42.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.273+0000 7f885f7fe640 1 -- 172.21.15.43:0/22254512 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2e95760 msgr2=0x55f1c2e981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:42.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.273+0000 7f885f7fe640 1 --2- 172.21.15.43:0/22254512 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2e95760 0x55f1c2e981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.273+0000 7f885f7fe640 1 -- 172.21.15.43:0/22254512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 msgr2=0x55f1c2eb84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:42.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.273+0000 7f885f7fe640 1 --2- 172.21.15.43:0/22254512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2eb84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.273+0000 7f885f7fe640 1 -- 172.21.15.43:0/22254512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1c2e91b20 con 0x55f1c2ea0940 2024-09-11T23:22:42.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.273+0000 7f8857fff640 1 --2- 172.21.15.43:0/22254512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2eb84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:42.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.274+0000 7f885f7fe640 1 --2- 172.21.15.43:0/22254512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2ea0940 0x55f1c2eb89f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f885800f5b0 tx=0x7f8858014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35d1393e1ec6006c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.274+0000 7f8854ff9640 1 -- 172.21.15.43:0/22254512 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8858027640 con 0x55f1c2ea0940 2024-09-11T23:22:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.274+0000 7f8854ff9640 1 -- 172.21.15.43:0/22254512 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f885801d8f0 con 0x55f1c2ea0940 2024-09-11T23:22:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.274+0000 7f8854ff9640 1 -- 172.21.15.43:0/22254512 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8858026840 con 0x55f1c2ea0940 2024-09-11T23:22:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.275+0000 7f8867a2e740 1 -- 172.21.15.43:0/22254512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2ea0940 msgr2=0x55f1c2eb89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.275+0000 7f8867a2e740 1 --2- 172.21.15.43:0/22254512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2ea0940 0x55f1c2eb89f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f885800f5b0 tx=0x7f8858014000 comp rx=0 tx=0).stop 2024-09-11T23:22:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.276+0000 7f8867a2e740 1 -- 172.21.15.43:0/22254512 shutdown_connections 2024-09-11T23:22:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.276+0000 7f8867a2e740 1 --2- 172.21.15.43:0/22254512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2ea0940 0x55f1c2eb89f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.276+0000 7f8867a2e740 1 --2- 172.21.15.43:0/22254512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2eb84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.276+0000 7f8867a2e740 1 --2- 172.21.15.43:0/22254512 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2e95760 0x55f1c2e981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.276+0000 7f8867a2e740 1 -- 172.21.15.43:0/22254512 >> 172.21.15.43:0/22254512 conn(0x55f1c2e91cf0 msgr2=0x55f1c2e9f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.276+0000 7f8867a2e740 1 -- 172.21.15.43:0/22254512 shutdown_connections 2024-09-11T23:22:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.276+0000 7f8867a2e740 1 -- 172.21.15.43:0/22254512 wait complete. 2024-09-11T23:22:42.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.276+0000 7f8867a2e740 1 -- start start 2024-09-11T23:22:42.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.277+0000 7f8867a2e740 1 Processor -- start 2024-09-11T23:22:42.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.277+0000 7f8867a2e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2e95760 0x55f1c2ea4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:42.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.277+0000 7f8867a2e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2ea53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:42.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.277+0000 7f8867a2e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2ea0940 0x55f1c2f1a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:42.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.277+0000 7f8867a2e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f88601ae120 con 0x55f1c2e96080 2024-09-11T23:22:42.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.277+0000 7f8867a2e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f88601af590 con 0x55f1c2ea0940 2024-09-11T23:22:42.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.277+0000 7f8867a2e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f88601ae7f0 con 0x55f1c2e95760 2024-09-11T23:22:42.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.277+0000 7f8867a2e740 10 client.0 _ll_register_callbacks cb 0x7f88480095e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:42.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.277+0000 7f885effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2e95760 0x55f1c2ea4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:42.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.278+0000 7f885effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2e95760 0x55f1c2ea4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:53982/0 (socket says 172.21.15.43:53982) 2024-09-11T23:22:42.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.278+0000 7f885effd640 1 -- 172.21.15.43:0/1463708858 learned_addr learned my addr 172.21.15.43:0/1463708858 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:42.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.278+0000 7f8857fff640 1 --2- 172.21.15.43:0/1463708858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2ea53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:42.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.278+0000 7f885f7fe640 1 --2- 172.21.15.43:0/1463708858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2ea0940 0x55f1c2f1a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:42.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.278+0000 7f885effd640 1 -- 172.21.15.43:0/1463708858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2ea0940 msgr2=0x55f1c2f1a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:42.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.278+0000 7f885effd640 1 --2- 172.21.15.43:0/1463708858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2ea0940 0x55f1c2f1a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.278+0000 7f885effd640 1 -- 172.21.15.43:0/1463708858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 msgr2=0x55f1c2ea53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:42.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.278+0000 7f885effd640 1 --2- 172.21.15.43:0/1463708858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2ea53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.278+0000 7f885effd640 1 -- 172.21.15.43:0/1463708858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1c2e92980 con 0x55f1c2e95760 2024-09-11T23:22:42.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.278+0000 7f885f7fe640 1 --2- 172.21.15.43:0/1463708858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2ea0940 0x55f1c2f1a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:42.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.278+0000 7f885effd640 1 --2- 172.21.15.43:0/1463708858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2e95760 0x55f1c2ea4e60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f885001b040 tx=0x7f8850010430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=625076b39802de2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:42.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.278+0000 7f88467fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-11T23:22:42.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.279+0000 7f88467fc640 1 -- 172.21.15.43:0/1463708858 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850031060 con 0x55f1c2e95760 2024-09-11T23:22:42.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.279+0000 7f88467fc640 1 -- 172.21.15.43:0/1463708858 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f8850010b60 con 0x55f1c2e95760 2024-09-11T23:22:42.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.279+0000 7f8867a2e740 1 -- 172.21.15.43:0/1463708858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f885801d8f0 con 0x55f1c2e95760 2024-09-11T23:22:42.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.279+0000 7f88467fc640 1 -- 172.21.15.43:0/1463708858 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f885001d040 con 0x55f1c2e95760 2024-09-11T23:22:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.279+0000 7f8867a2e740 10 client.25156 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.279+0000 7f8867a2e740 1 -- 172.21.15.43:0/1463708858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f1c2e8d260 con 0x55f1c2e95760 2024-09-11T23:22:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.279+0000 7f88467fc640 1 -- 172.21.15.43:0/1463708858 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f885801d8f0 con 0x55f1c2e95760 2024-09-11T23:22:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.280+0000 7f88467fc640 1 -- 172.21.15.43:0/1463708858 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f88500185b0 con 0x55f1c2e95760 2024-09-11T23:22:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.280+0000 7f8867a2e740 10 client.25156 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.280+0000 7f8867a2e740 10 client.25156 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:22:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.280+0000 7f8867a2e740 20 client.25156 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.280+0000 7f8867a2e740 1 --2- 172.21.15.43:0/1463708858 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f1c2f1c120 0x55f1c2f1e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.280+0000 7f8867a2e740 4 client.25156 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-11T23:22:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.280+0000 7f8867a2e740 1 -- 172.21.15.43:0/1463708858 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x55f1c2e02d20 con 0x55f1c2f1c120 2024-09-11T23:22:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.281+0000 7f8857fff640 1 --2- 172.21.15.43:0/1463708858 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f1c2f1c120 0x55f1c2f1e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.281+0000 7f88467fc640 10 client.25156 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.281+0000 7f8857fff640 1 --2- 172.21.15.43:0/1463708858 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f1c2f1c120 0x55f1c2f1e540 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68a4f940ab3dc1ba server_cookie=967a0734f0f89a37 in_seq=0 out_seq=0 2024-09-11T23:22:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.292+0000 7f88467fc640 1 -- 172.21.15.43:0/1463708858 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55f1c2e02d20 con 0x55f1c2f1c120 2024-09-11T23:22:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.292+0000 7f88467fc640 10 client.25156 handle_command_reply: tid=0 2024-09-11T23:22:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.292+0000 7f8867a2e740 1 client.25156 shutdown 2024-09-11T23:22:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.292+0000 7f8867a2e740 1 -- 172.21.15.43:0/1463708858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2e95760 msgr2=0x55f1c2ea4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.292+0000 7f8867a2e740 1 --2- 172.21.15.43:0/1463708858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2e95760 0x55f1c2ea4e60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f885001b040 tx=0x7f8850010430 comp rx=0 tx=0).stop 2024-09-11T23:22:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.293+0000 7f8867a2e740 1 -- 172.21.15.43:0/1463708858 shutdown_connections 2024-09-11T23:22:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.293+0000 7f8867a2e740 1 --2- 172.21.15.43:0/1463708858 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f1c2f1c120 0x55f1c2f1e540 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.293+0000 7f8867a2e740 1 --2- 172.21.15.43:0/1463708858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1c2ea0940 0x55f1c2f1a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.293+0000 7f8867a2e740 1 --2- 172.21.15.43:0/1463708858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1c2e96080 0x55f1c2ea53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.293+0000 7f8867a2e740 1 --2- 172.21.15.43:0/1463708858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1c2e95760 0x55f1c2ea4e60 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f885001b040 tx=0x7f8850010430 comp rx=0 tx=0).stop 2024-09-11T23:22:42.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.293+0000 7f8867a2e740 1 -- 172.21.15.43:0/1463708858 >> 172.21.15.43:0/1463708858 conn(0x55f1c2e91cf0 msgr2=0x55f1c2e9f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:42.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.293+0000 7f88467fc640 0 client.25156 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:42.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.293+0000 7f8867a2e740 1 -- 172.21.15.43:0/1463708858 shutdown_connections 2024-09-11T23:22:42.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.293+0000 7f8867a2e740 1 -- 172.21.15.43:0/1463708858 wait complete. 2024-09-11T23:22:42.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.293+0000 7f8867a2e740 20 client.25156 trim_cache size 0 max 16384 2024-09-11T23:22:42.294 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:22:42.294 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 24, 2024-09-11T23:22:42.294 INFO:teuthology.orchestra.run.smithi043.stdout: "leader": 24497, 2024-09-11T23:22:42.294 INFO:teuthology.orchestra.run.smithi043.stdout: "set_version": 2, 2024-09-11T23:22:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: "sets": { 2024-09-11T23:22:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: "433305f6": { 2024-09-11T23:22:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 2, 2024-09-11T23:22:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: "age_ref": 0.0, 2024-09-11T23:22:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:22:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "QUIESCED", 2024-09-11T23:22:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-09-11T23:22:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:22:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: "timeout": 90.0, 2024-09-11T23:22:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: "expiration": 144.5, 2024-09-11T23:22:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: "members": { 2024-09-11T23:22:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: "file:/": { 2024-09-11T23:22:42.295 INFO:teuthology.orchestra.run.smithi043.stdout: "excluded": false, 2024-09-11T23:22:42.296 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:22:42.296 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "QUIESCED", 2024-09-11T23:22:42.296 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-09-11T23:22:42.296 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:22:42.296 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:22:42.296 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:22:42.296 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:22:42.296 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:22:42.296 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:22:42.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.294+0000 7f8865655640 1 -- 172.21.15.43:0/513041555 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f884c0a2e10 msgr2=0x7f884c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:42.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.294+0000 7f8865655640 1 --2- 172.21.15.43:0/513041555 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f884c0a2e10 0x7f884c0a52d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f885800af80 tx=0x7f885800a4d0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.294+0000 7f8865655640 1 -- 172.21.15.43:0/513041555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8860102570 msgr2=0x7f886019ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:42.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.294+0000 7f8865655640 1 --2- 172.21.15.43:0/513041555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8860102570 0x7f886019ac00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8848006010 tx=0x7f8848003c10 comp rx=0 tx=0).stop 2024-09-11T23:22:42.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.294+0000 7f8865655640 1 -- 172.21.15.43:0/513041555 shutdown_connections 2024-09-11T23:22:42.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.294+0000 7f8865655640 1 --2- 172.21.15.43:0/513041555 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f884c0a2e10 0x7f884c0a52d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.294+0000 7f8865655640 1 --2- 172.21.15.43:0/513041555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8860102ef0 0x7f886019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.294+0000 7f8865655640 1 --2- 172.21.15.43:0/513041555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8860102570 0x7f886019ac00 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.294+0000 7f8865655640 1 --2- 172.21.15.43:0/513041555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8860100e20 0x7f886019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.294+0000 7f8865655640 1 -- 172.21.15.43:0/513041555 >> 172.21.15.43:0/513041555 conn(0x7f88600fc7d0 msgr2=0x7f8860069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.294+0000 7f8865655640 1 -- 172.21.15.43:0/513041555 shutdown_connections 2024-09-11T23:22:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:42.294+0000 7f8865655640 1 -- 172.21.15.43:0/513041555 wait complete. 2024-09-11T23:22:42.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:41 smithi097 ceph-mon[35562]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-11T23:22:42.544 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:43.026 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '433305f6', quiesce took 7.8 seconds. Will release after: 16.7 2024-09-11T23:22:43.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.497+0000 7f56f8392640 1 -- 172.21.15.43:0/3521612046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f010c150 msgr2=0x7f56f010e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:43.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.497+0000 7f56f8392640 1 --2- 172.21.15.43:0/3521612046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f010c150 0x7f56f010e5f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009ca0 tx=0x7f56e802a950 comp rx=0 tx=0).stop 2024-09-11T23:22:43.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.498+0000 7f56f8392640 1 -- 172.21.15.43:0/3521612046 shutdown_connections 2024-09-11T23:22:43.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.498+0000 7f56f8392640 1 --2- 172.21.15.43:0/3521612046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f010c150 0x7f56f010e5f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.498+0000 7f56f8392640 1 --2- 172.21.15.43:0/3521612046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56f0103640 0x7f56f010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.498+0000 7f56f8392640 1 --2- 172.21.15.43:0/3521612046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56f0102d20 0x7f56f0103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.498+0000 7f56f8392640 1 -- 172.21.15.43:0/3521612046 >> 172.21.15.43:0/3521612046 conn(0x7f56f00fc780 msgr2=0x7f56f00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:43.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.498+0000 7f56f8392640 1 -- 172.21.15.43:0/3521612046 shutdown_connections 2024-09-11T23:22:43.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.498+0000 7f56f8392640 1 -- 172.21.15.43:0/3521612046 wait complete. 2024-09-11T23:22:43.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.498+0000 7f56f8392640 1 Processor -- start 2024-09-11T23:22:43.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.499+0000 7f56f8392640 1 -- start start 2024-09-11T23:22:43.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.499+0000 7f56f8392640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f0102d20 0x7f56f0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.499+0000 7f56f8392640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56f0103640 0x7f56f0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.499+0000 7f56f8392640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56f010c150 0x7f56f019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.499+0000 7f56f8392640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f56f0111f10 con 0x7f56f0102d20 2024-09-11T23:22:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.499+0000 7f56f8392640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f56f0111d90 con 0x7f56f0103640 2024-09-11T23:22:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.499+0000 7f56f8392640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f56f0112090 con 0x7f56f010c150 2024-09-11T23:22:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.499+0000 7f56f612e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f0102d20 0x7f56f0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.499+0000 7f56f612e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f0102d20 0x7f56f0198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53972/0 (socket says 172.21.15.43:53972) 2024-09-11T23:22:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.499+0000 7f56f612e640 1 -- 172.21.15.43:0/804238901 learned_addr learned my addr 172.21.15.43:0/804238901 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.499+0000 7f56f592d640 1 --2- 172.21.15.43:0/804238901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56f0103640 0x7f56f0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.499+0000 7f56f692f640 1 --2- 172.21.15.43:0/804238901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56f010c150 0x7f56f019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.499+0000 7f56f612e640 1 -- 172.21.15.43:0/804238901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56f010c150 msgr2=0x7f56f019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.499+0000 7f56f612e640 1 --2- 172.21.15.43:0/804238901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56f010c150 0x7f56f019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.499+0000 7f56f612e640 1 -- 172.21.15.43:0/804238901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56f0103640 msgr2=0x7f56f0198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:43.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.499+0000 7f56f612e640 1 --2- 172.21.15.43:0/804238901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56f0103640 0x7f56f0198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.500+0000 7f56f612e640 1 -- 172.21.15.43:0/804238901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f019d3e0 con 0x7f56f0102d20 2024-09-11T23:22:43.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.500+0000 7f56f692f640 1 --2- 172.21.15.43:0/804238901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56f010c150 0x7f56f019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:43.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.500+0000 7f56f612e640 1 --2- 172.21.15.43:0/804238901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f0102d20 0x7f56f0198430 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00c890 tx=0x7f56ec00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:43.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.500+0000 7f56e77fe640 1 -- 172.21.15.43:0/804238901 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec004220 con 0x7f56f0102d20 2024-09-11T23:22:43.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.500+0000 7f56e77fe640 1 -- 172.21.15.43:0/804238901 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56ec0043c0 con 0x7f56f0102d20 2024-09-11T23:22:43.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.500+0000 7f56f8392640 1 -- 172.21.15.43:0/804238901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f019d6d0 con 0x7f56f0102d20 2024-09-11T23:22:43.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.500+0000 7f56f8392640 1 -- 172.21.15.43:0/804238901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f01a4fb0 con 0x7f56f0102d20 2024-09-11T23:22:43.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.500+0000 7f56e77fe640 1 -- 172.21.15.43:0/804238901 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec0049e0 con 0x7f56f0102d20 2024-09-11T23:22:43.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.503+0000 7f56fa76b740 1 Processor -- start 2024-09-11T23:22:43.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.503+0000 7f56fa76b740 1 -- start start 2024-09-11T23:22:43.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.504+0000 7f56fa76b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fc0b40 0x55dfd3fbf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.504+0000 7f56fa76b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fbf5d0 0x55dfd3fbfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.504+0000 7f56fa76b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfd3fce780 0x55dfd3fd0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.504+0000 7f56fa76b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dfd3f252f0 con 0x55dfd3fc0b40 2024-09-11T23:22:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.504+0000 7f56fa76b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dfd3ecccf0 con 0x55dfd3fbf5d0 2024-09-11T23:22:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.504+0000 7f56fa76b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dfd3f212b0 con 0x55dfd3fce780 2024-09-11T23:22:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.504+0000 7f56f692f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fc0b40 0x55dfd3fbf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.504+0000 7f56f692f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fc0b40 0x55dfd3fbf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53986/0 (socket says 172.21.15.43:53986) 2024-09-11T23:22:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.504+0000 7f56f692f640 1 -- 172.21.15.43:0/2646793600 learned_addr learned my addr 172.21.15.43:0/2646793600 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.504+0000 7f56f592d640 1 --2- 172.21.15.43:0/2646793600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfd3fce780 0x55dfd3fd0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.504+0000 7f56e77fe640 1 -- 172.21.15.43:0/804238901 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f56ec0079b0 con 0x7f56f0102d20 2024-09-11T23:22:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.504+0000 7f56f692f640 1 -- 172.21.15.43:0/2646793600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfd3fce780 msgr2=0x55dfd3fd0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.504+0000 7f56f692f640 1 --2- 172.21.15.43:0/2646793600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfd3fce780 0x55dfd3fd0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.504+0000 7f56f612e640 1 --2- 172.21.15.43:0/2646793600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fbf5d0 0x55dfd3fbfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:43.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.504+0000 7f56f692f640 1 -- 172.21.15.43:0/2646793600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fbf5d0 msgr2=0x55dfd3fbfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:43.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.504+0000 7f56f692f640 1 --2- 172.21.15.43:0/2646793600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fbf5d0 0x55dfd3fbfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.504+0000 7f56f692f640 1 -- 172.21.15.43:0/2646793600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfd3eb13a0 con 0x55dfd3fc0b40 2024-09-11T23:22:43.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.504+0000 7f56f612e640 1 --2- 172.21.15.43:0/2646793600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fbf5d0 0x55dfd3fbfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:22:43.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.505+0000 7f56f692f640 1 --2- 172.21.15.43:0/2646793600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fc0b40 0x55dfd3fbf090 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009a50 tx=0x7f56e802f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27dbea9b0e9fcdfd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:43.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.505+0000 7f56e77fe640 1 --2- 172.21.15.43:0/804238901 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56dc0a2e90 0x7f56dc0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:43.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.505+0000 7f56e4dea640 1 -- 172.21.15.43:0/2646793600 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8034040 con 0x55dfd3fc0b40 2024-09-11T23:22:43.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.505+0000 7f56e4dea640 1 -- 172.21.15.43:0/2646793600 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56e8003970 con 0x55dfd3fc0b40 2024-09-11T23:22:43.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.505+0000 7f56e4dea640 1 -- 172.21.15.43:0/2646793600 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e80046f0 con 0x55dfd3fc0b40 2024-09-11T23:22:43.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.505+0000 7f56e77fe640 1 -- 172.21.15.43:0/804238901 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f56ec0cf6e0 con 0x7f56f0102d20 2024-09-11T23:22:43.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.505+0000 7f56f612e640 1 --2- 172.21.15.43:0/804238901 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56dc0a2e90 0x7f56dc0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:43.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.506+0000 7f56fa76b740 1 -- 172.21.15.43:0/2646793600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fc0b40 msgr2=0x55dfd3fbf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:43.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.506+0000 7f56f612e640 1 --2- 172.21.15.43:0/804238901 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56dc0a2e90 0x7f56dc0a5350 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f56ec0cccb0 tx=0x7f56ec000bc0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:43.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.506+0000 7f56fa76b740 1 --2- 172.21.15.43:0/2646793600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fc0b40 0x55dfd3fbf090 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009a50 tx=0x7f56e802f940 comp rx=0 tx=0).stop 2024-09-11T23:22:43.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.506+0000 7f56fa76b740 1 -- 172.21.15.43:0/2646793600 shutdown_connections 2024-09-11T23:22:43.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.506+0000 7f56fa76b740 1 --2- 172.21.15.43:0/2646793600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfd3fce780 0x55dfd3fd0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.506+0000 7f56fa76b740 1 --2- 172.21.15.43:0/2646793600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fbf5d0 0x55dfd3fbfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.506+0000 7f56fa76b740 1 --2- 172.21.15.43:0/2646793600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fc0b40 0x55dfd3fbf090 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.506+0000 7f56fa76b740 1 -- 172.21.15.43:0/2646793600 >> 172.21.15.43:0/2646793600 conn(0x55dfd3f88310 msgr2=0x55dfd3fbbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:43.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.507+0000 7f56fa76b740 1 -- 172.21.15.43:0/2646793600 shutdown_connections 2024-09-11T23:22:43.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.507+0000 7f56fa76b740 1 -- 172.21.15.43:0/2646793600 wait complete. 2024-09-11T23:22:43.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.507+0000 7f56fa76b740 1 -- start start 2024-09-11T23:22:43.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.507+0000 7f56fa76b740 1 Processor -- start 2024-09-11T23:22:43.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.508+0000 7f56fa76b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfd3fbf5d0 0x55dfd3fce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:43.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.508+0000 7f56fa76b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fc0b40 0x55dfd3fcb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:43.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.508+0000 7f56fa76b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fce780 0x55dfd3fcb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:43.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.508+0000 7f56fa76b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dfd3fd1420 con 0x55dfd3fce780 2024-09-11T23:22:43.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.508+0000 7f56fa76b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dfd3ee0060 con 0x55dfd3fc0b40 2024-09-11T23:22:43.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.508+0000 7f56fa76b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dfd3fd15a0 con 0x55dfd3fbf5d0 2024-09-11T23:22:43.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.508+0000 7f56fa76b740 10 client.0 _ll_register_callbacks cb 0x55dfd3c338c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:43.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.508+0000 7f56f692f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fce780 0x55dfd3fcb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:43.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.508+0000 7f56f592d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fc0b40 0x55dfd3fcb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:43.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.508+0000 7f56f692f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fce780 0x55dfd3fcb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53992/0 (socket says 172.21.15.43:53992) 2024-09-11T23:22:43.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.508+0000 7f56f692f640 1 -- 172.21.15.43:0/1240541313 learned_addr learned my addr 172.21.15.43:0/1240541313 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:43.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.508+0000 7f56f692f640 1 --2- 172.21.15.43:0/1240541313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfd3fbf5d0 0x55dfd3fce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:43.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.508+0000 7f56f692f640 1 -- 172.21.15.43:0/1240541313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfd3fbf5d0 msgr2=0x55dfd3fce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.508+0000 7f56f692f640 1 --2- 172.21.15.43:0/1240541313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfd3fbf5d0 0x55dfd3fce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.508+0000 7f56f692f640 1 -- 172.21.15.43:0/1240541313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fc0b40 msgr2=0x55dfd3fcb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.508+0000 7f56f692f640 1 --2- 172.21.15.43:0/1240541313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fc0b40 0x55dfd3fcb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.508+0000 7f56f692f640 1 -- 172.21.15.43:0/1240541313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfd3c963c0 con 0x55dfd3fce780 2024-09-11T23:22:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.508+0000 7f56f692f640 1 --2- 172.21.15.43:0/1240541313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fce780 0x55dfd3fcb940 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009960 tx=0x7f56e80042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929125bc24467e3a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.508+0000 7f56d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.509+0000 7f56d9ffb640 1 -- 172.21.15.43:0/1240541313 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e802f3a0 con 0x55dfd3fce780 2024-09-11T23:22:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.509+0000 7f56d9ffb640 1 -- 172.21.15.43:0/1240541313 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f56e802f540 con 0x55dfd3fce780 2024-09-11T23:22:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.509+0000 7f56fa76b740 1 -- 172.21.15.43:0/1240541313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56e8003970 con 0x55dfd3fce780 2024-09-11T23:22:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.509+0000 7f56d9ffb640 1 -- 172.21.15.43:0/1240541313 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f56e802f740 con 0x55dfd3fce780 2024-09-11T23:22:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.509+0000 7f56fa76b740 10 client.15555 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.509+0000 7f56fa76b740 1 -- 172.21.15.43:0/1240541313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dfd3fd1420 con 0x55dfd3fce780 2024-09-11T23:22:43.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.509+0000 7f56d9ffb640 1 -- 172.21.15.43:0/1240541313 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56e8003970 con 0x55dfd3fce780 2024-09-11T23:22:43.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.509+0000 7f56d9ffb640 1 -- 172.21.15.43:0/1240541313 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f56e8042020 con 0x55dfd3fce780 2024-09-11T23:22:43.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.509+0000 7f56fa76b740 10 client.15555 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:43.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.509+0000 7f56fa76b740 10 client.15555 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T23:22:43.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.509+0000 7f56fa76b740 20 client.15555 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:43.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.509+0000 7f56fa76b740 1 --2- 172.21.15.43:0/1240541313 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55dfd3fc3470 0x55dfd3fc58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:43.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.509+0000 7f56fa76b740 4 client.15555 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:22:43.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.509+0000 7f56fa76b740 1 -- 172.21.15.43:0/1240541313 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfd3f32860 con 0x55dfd3fc3470 2024-09-11T23:22:43.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.510+0000 7f56f592d640 1 --2- 172.21.15.43:0/1240541313 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55dfd3fc3470 0x55dfd3fc58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:43.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.510+0000 7f56f592d640 1 --2- 172.21.15.43:0/1240541313 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55dfd3fc3470 0x55dfd3fc58b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e00211d54fbcde7b server_cookie=f8786321ed8aded5 in_seq=0 out_seq=0 2024-09-11T23:22:43.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.510+0000 7f56d9ffb640 10 client.15555 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:22:43.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.511+0000 7f56d9ffb640 1 -- 172.21.15.43:0/1240541313 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dfd3f32860 con 0x55dfd3fc3470 2024-09-11T23:22:43.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.511+0000 7f56d9ffb640 10 client.15555 handle_command_reply: tid=0 2024-09-11T23:22:43.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.511+0000 7f56fa76b740 1 client.15555 shutdown 2024-09-11T23:22:43.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.511+0000 7f56fa76b740 1 -- 172.21.15.43:0/1240541313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fce780 msgr2=0x55dfd3fcb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:43.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.511+0000 7f56fa76b740 1 --2- 172.21.15.43:0/1240541313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fce780 0x55dfd3fcb940 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009960 tx=0x7f56e80042d0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.511+0000 7f56fa76b740 1 -- 172.21.15.43:0/1240541313 shutdown_connections 2024-09-11T23:22:43.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.511+0000 7f56fa76b740 1 --2- 172.21.15.43:0/1240541313 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55dfd3fc3470 0x55dfd3fc58b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.511+0000 7f56fa76b740 1 --2- 172.21.15.43:0/1240541313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fce780 0x55dfd3fcb940 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.511+0000 7f56fa76b740 1 --2- 172.21.15.43:0/1240541313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fc0b40 0x55dfd3fcb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.511+0000 7f56fa76b740 1 --2- 172.21.15.43:0/1240541313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfd3fbf5d0 0x55dfd3fce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.511+0000 7f56fa76b740 1 -- 172.21.15.43:0/1240541313 >> 172.21.15.43:0/1240541313 conn(0x55dfd3f88310 msgr2=0x55dfd3fc1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:43.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.511+0000 7f56d9ffb640 0 client.15555 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:22:43.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.511+0000 7f56fa76b740 1 -- 172.21.15.43:0/1240541313 shutdown_connections 2024-09-11T23:22:43.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.511+0000 7f56fa76b740 1 -- 172.21.15.43:0/1240541313 wait complete. 2024-09-11T23:22:43.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.511+0000 7f56fa76b740 20 client.15555 trim_cache size 0 max 16384 2024-09-11T23:22:43.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.524+0000 7f56fa76b740 1 Processor -- start 2024-09-11T23:22:43.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.524+0000 7f56fa76b740 1 -- start start 2024-09-11T23:22:43.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.524+0000 7f56fa76b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fbf5d0 0x55dfd3fcd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:43.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.524+0000 7f56fa76b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fc0b40 0x55dfd3fc8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:43.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.525+0000 7f56fa76b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfd3fce780 0x55dfd3fc8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:43.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.525+0000 7f56fa76b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f56d4011090 con 0x55dfd3fbf5d0 2024-09-11T23:22:43.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.525+0000 7f56fa76b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f56d400e960 con 0x55dfd3fc0b40 2024-09-11T23:22:43.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.525+0000 7f56fa76b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f56d400c7b0 con 0x55dfd3fce780 2024-09-11T23:22:43.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.525+0000 7f56f692f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fbf5d0 0x55dfd3fcd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:43.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.525+0000 7f56f692f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fbf5d0 0x55dfd3fcd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53996/0 (socket says 172.21.15.43:53996) 2024-09-11T23:22:43.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.525+0000 7f56f692f640 1 -- 172.21.15.43:0/1610632582 learned_addr learned my addr 172.21.15.43:0/1610632582 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:43.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.525+0000 7f56f592d640 1 --2- 172.21.15.43:0/1610632582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fc0b40 0x55dfd3fc8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:43.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.525+0000 7f56f692f640 1 -- 172.21.15.43:0/1610632582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfd3fce780 msgr2=0x55dfd3fc8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:43.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.525+0000 7f56f692f640 1 --2- 172.21.15.43:0/1610632582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfd3fce780 0x55dfd3fc8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.525+0000 7f56f692f640 1 -- 172.21.15.43:0/1610632582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fc0b40 msgr2=0x55dfd3fc8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:43.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.525+0000 7f56f692f640 1 --2- 172.21.15.43:0/1610632582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fc0b40 0x55dfd3fc8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.525+0000 7f56f692f640 1 -- 172.21.15.43:0/1610632582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfd3fbfd50 con 0x55dfd3fbf5d0 2024-09-11T23:22:43.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.525+0000 7f56f692f640 1 --2- 172.21.15.43:0/1610632582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fbf5d0 0x55dfd3fcd330 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f56e802fd90 tx=0x7f56e80509b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32477f691f12969 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:43.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.526+0000 7f56dbfff640 1 -- 172.21.15.43:0/1610632582 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e806c070 con 0x55dfd3fbf5d0 2024-09-11T23:22:43.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.526+0000 7f56dbfff640 1 -- 172.21.15.43:0/1610632582 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56e8003970 con 0x55dfd3fbf5d0 2024-09-11T23:22:43.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.526+0000 7f56dbfff640 1 -- 172.21.15.43:0/1610632582 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e803f020 con 0x55dfd3fbf5d0 2024-09-11T23:22:43.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.526+0000 7f56fa76b740 1 -- 172.21.15.43:0/1610632582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fbf5d0 msgr2=0x55dfd3fcd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:43.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.526+0000 7f56fa76b740 1 --2- 172.21.15.43:0/1610632582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fbf5d0 0x55dfd3fcd330 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f56e802fd90 tx=0x7f56e80509b0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.527+0000 7f56fa76b740 1 -- 172.21.15.43:0/1610632582 shutdown_connections 2024-09-11T23:22:43.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.527+0000 7f56fa76b740 1 --2- 172.21.15.43:0/1610632582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfd3fce780 0x55dfd3fc8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.527+0000 7f56fa76b740 1 --2- 172.21.15.43:0/1610632582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fc0b40 0x55dfd3fc8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.527+0000 7f56fa76b740 1 --2- 172.21.15.43:0/1610632582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fbf5d0 0x55dfd3fcd330 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.527+0000 7f56fa76b740 1 -- 172.21.15.43:0/1610632582 >> 172.21.15.43:0/1610632582 conn(0x55dfd3f88310 msgr2=0x55dfd3fccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:43.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.527+0000 7f56fa76b740 1 -- 172.21.15.43:0/1610632582 shutdown_connections 2024-09-11T23:22:43.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.527+0000 7f56fa76b740 1 -- 172.21.15.43:0/1610632582 wait complete. 2024-09-11T23:22:43.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.527+0000 7f56fa76b740 1 -- start start 2024-09-11T23:22:43.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.527+0000 7f56fa76b740 1 Processor -- start 2024-09-11T23:22:43.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.527+0000 7f56fa76b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fbf5d0 0x55dfd3fe1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:43.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.527+0000 7f56fa76b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfd3fc0b40 0x55dfd3fe2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:43.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.527+0000 7f56fa76b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fce780 0x55dfd3fdf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:43.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.528+0000 7f56f692f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fbf5d0 0x55dfd3fe1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:43.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.528+0000 7f56fa76b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f56d400ce20 con 0x55dfd3fce780 2024-09-11T23:22:43.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.528+0000 7f56fa76b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f56d400e290 con 0x55dfd3fbf5d0 2024-09-11T23:22:43.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.528+0000 7f56fa76b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f56d400d4f0 con 0x55dfd3fc0b40 2024-09-11T23:22:43.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.528+0000 7f56f692f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fbf5d0 0x55dfd3fe1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:44488/0 (socket says 172.21.15.43:44488) 2024-09-11T23:22:43.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.528+0000 7f56f692f640 1 -- 172.21.15.43:0/1147471224 learned_addr learned my addr 172.21.15.43:0/1147471224 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:43.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.528+0000 7f56f692f640 1 --2- 172.21.15.43:0/1147471224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fce780 0x55dfd3fdf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:43.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.528+0000 7f56fa76b740 10 client.0 _ll_register_callbacks cb 0x7f56f006b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:43.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.528+0000 7f56f592d640 1 --2- 172.21.15.43:0/1147471224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfd3fc0b40 0x55dfd3fe2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.528+0000 7f56f692f640 1 -- 172.21.15.43:0/1147471224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfd3fc0b40 msgr2=0x55dfd3fe2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.528+0000 7f56f692f640 1 --2- 172.21.15.43:0/1147471224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfd3fc0b40 0x55dfd3fe2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.528+0000 7f56f692f640 1 -- 172.21.15.43:0/1147471224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fbf5d0 msgr2=0x55dfd3fe1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.528+0000 7f56f692f640 1 --2- 172.21.15.43:0/1147471224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fbf5d0 0x55dfd3fe1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.528+0000 7f56f692f640 1 -- 172.21.15.43:0/1147471224 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfd3fdfc00 con 0x55dfd3fce780 2024-09-11T23:22:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.528+0000 7f56f692f640 1 --2- 172.21.15.43:0/1147471224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fce780 0x55dfd3fdf400 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f56e802f8e0 tx=0x7f56e8000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4954f851af8424 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.528+0000 7f56d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.529+0000 7f56d9ffb640 1 -- 172.21.15.43:0/1147471224 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e806c070 con 0x55dfd3fce780 2024-09-11T23:22:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.529+0000 7f56d9ffb640 1 -- 172.21.15.43:0/1147471224 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f56e8050b70 con 0x55dfd3fce780 2024-09-11T23:22:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.529+0000 7f56fa76b740 1 -- 172.21.15.43:0/1147471224 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56e8003970 con 0x55dfd3fce780 2024-09-11T23:22:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.529+0000 7f56d9ffb640 1 -- 172.21.15.43:0/1147471224 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f56e803f030 con 0x55dfd3fce780 2024-09-11T23:22:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.529+0000 7f56fa76b740 10 client.15561 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.529+0000 7f56fa76b740 1 -- 172.21.15.43:0/1147471224 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55dfd4044910 con 0x55dfd3fce780 2024-09-11T23:22:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.529+0000 7f56d9ffb640 1 -- 172.21.15.43:0/1147471224 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56e8003970 con 0x55dfd3fce780 2024-09-11T23:22:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.530+0000 7f56d9ffb640 1 -- 172.21.15.43:0/1147471224 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f56e8042040 con 0x55dfd3fce780 2024-09-11T23:22:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.530+0000 7f56fa76b740 10 client.15561 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.530+0000 7f56fa76b740 10 client.15561 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T23:22:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.530+0000 7f56fa76b740 20 client.15561 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.530+0000 7f56fa76b740 1 --2- 172.21.15.43:0/1147471224 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55dfd4044e40 0x55dfd4047260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.530+0000 7f56fa76b740 4 client.15561 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:22:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.530+0000 7f56fa76b740 1 -- 172.21.15.43:0/1147471224 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dfd3fc5d90 con 0x55dfd4044e40 2024-09-11T23:22:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.530+0000 7f56f592d640 1 --2- 172.21.15.43:0/1147471224 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55dfd4044e40 0x55dfd4047260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.531+0000 7f56f592d640 1 --2- 172.21.15.43:0/1147471224 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55dfd4044e40 0x55dfd4047260 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c6fd769038f3523a server_cookie=eda5ec7819fb8897 in_seq=0 out_seq=0 2024-09-11T23:22:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.531+0000 7f56d9ffb640 10 client.15561 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:22:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.531+0000 7f56d9ffb640 1 -- 172.21.15.43:0/1147471224 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dfd3fc5d90 con 0x55dfd4044e40 2024-09-11T23:22:43.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.531+0000 7f56d9ffb640 10 client.15561 handle_command_reply: tid=0 2024-09-11T23:22:43.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.531+0000 7f56fa76b740 1 client.15561 shutdown 2024-09-11T23:22:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.531+0000 7f56fa76b740 1 -- 172.21.15.43:0/1147471224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fce780 msgr2=0x55dfd3fdf400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.531+0000 7f56fa76b740 1 --2- 172.21.15.43:0/1147471224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fce780 0x55dfd3fdf400 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f56e802f8e0 tx=0x7f56e8000f40 comp rx=0 tx=0).stop 2024-09-11T23:22:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.531+0000 7f56fa76b740 1 -- 172.21.15.43:0/1147471224 shutdown_connections 2024-09-11T23:22:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.531+0000 7f56fa76b740 1 --2- 172.21.15.43:0/1147471224 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55dfd4044e40 0x55dfd4047260 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.531+0000 7f56fa76b740 1 --2- 172.21.15.43:0/1147471224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dfd3fce780 0x55dfd3fdf400 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.531+0000 7f56fa76b740 1 --2- 172.21.15.43:0/1147471224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfd3fc0b40 0x55dfd3fe2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.531+0000 7f56fa76b740 1 --2- 172.21.15.43:0/1147471224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfd3fbf5d0 0x55dfd3fe1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.531+0000 7f56fa76b740 1 -- 172.21.15.43:0/1147471224 >> 172.21.15.43:0/1147471224 conn(0x55dfd3f88310 msgr2=0x55dfd3fccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.531+0000 7f56d9ffb640 0 client.15561 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:22:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.531+0000 7f56fa76b740 1 -- 172.21.15.43:0/1147471224 shutdown_connections 2024-09-11T23:22:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.532+0000 7f56fa76b740 1 -- 172.21.15.43:0/1147471224 wait complete. 2024-09-11T23:22:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.532+0000 7f56fa76b740 20 client.15561 trim_cache size 0 max 16384 2024-09-11T23:22:43.549 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:22:43.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.532+0000 7f56f8392640 1 -- 172.21.15.43:0/804238901 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56dc0a2e90 msgr2=0x7f56dc0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:43.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.532+0000 7f56f8392640 1 --2- 172.21.15.43:0/804238901 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56dc0a2e90 0x7f56dc0a5350 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f56ec0cccb0 tx=0x7f56ec000bc0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.532+0000 7f56f8392640 1 -- 172.21.15.43:0/804238901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f0102d20 msgr2=0x7f56f0198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:43.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.532+0000 7f56f8392640 1 --2- 172.21.15.43:0/804238901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f0102d20 0x7f56f0198430 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00c890 tx=0x7f56ec00cd50 comp rx=0 tx=0).stop 2024-09-11T23:22:43.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.533+0000 7f56f8392640 1 -- 172.21.15.43:0/804238901 shutdown_connections 2024-09-11T23:22:43.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.533+0000 7f56f8392640 1 --2- 172.21.15.43:0/804238901 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56dc0a2e90 0x7f56dc0a5350 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.533+0000 7f56f8392640 1 --2- 172.21.15.43:0/804238901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56f010c150 0x7f56f019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.533+0000 7f56f8392640 1 --2- 172.21.15.43:0/804238901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56f0103640 0x7f56f0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.533+0000 7f56f8392640 1 --2- 172.21.15.43:0/804238901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f0102d20 0x7f56f0198430 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:43.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.533+0000 7f56f8392640 1 -- 172.21.15.43:0/804238901 >> 172.21.15.43:0/804238901 conn(0x7f56f00fc780 msgr2=0x7f56f00fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:43.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.533+0000 7f56f8392640 1 -- 172.21.15.43:0/804238901 shutdown_connections 2024-09-11T23:22:43.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:43.533+0000 7f56f8392640 1 -- 172.21.15.43:0/804238901 wait complete. 2024-09-11T23:22:44.072 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:43 smithi090 ceph-mon[35130]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-11T23:22:44.103 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:43 smithi097 ceph-mon[35562]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-11T23:22:44.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:43 smithi043 ceph-mon[30950]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-11T23:22:44.295 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:22:44.649 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:45.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.171+0000 7f78f60c6640 1 -- 172.21.15.43:0/2040762833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78f0102ef0 msgr2=0x7f78f010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:45.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.171+0000 7f78f60c6640 1 --2- 172.21.15.43:0/2040762833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78f0102ef0 0x7f78f010da40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009ca0 tx=0x7f78e802a950 comp rx=0 tx=0).stop 2024-09-11T23:22:45.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.171+0000 7f78f60c6640 1 -- 172.21.15.43:0/2040762833 shutdown_connections 2024-09-11T23:22:45.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.171+0000 7f78f60c6640 1 --2- 172.21.15.43:0/2040762833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78f0102ef0 0x7f78f010da40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.171+0000 7f78f60c6640 1 --2- 172.21.15.43:0/2040762833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78f0102570 0x7f78f01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.171+0000 7f78f60c6640 1 --2- 172.21.15.43:0/2040762833 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f0100e20 0x7f78f0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.171+0000 7f78f60c6640 1 -- 172.21.15.43:0/2040762833 >> 172.21.15.43:0/2040762833 conn(0x7f78f00fc7d0 msgr2=0x7f78f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:45.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.171+0000 7f78f60c6640 1 -- 172.21.15.43:0/2040762833 shutdown_connections 2024-09-11T23:22:45.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.171+0000 7f78f60c6640 1 -- 172.21.15.43:0/2040762833 wait complete. 2024-09-11T23:22:45.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.172+0000 7f78f60c6640 1 Processor -- start 2024-09-11T23:22:45.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.172+0000 7f78f60c6640 1 -- start start 2024-09-11T23:22:45.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.173+0000 7f78f60c6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f0100e20 0x7f78f01984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:45.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.173+0000 7f78f60c6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78f0102570 0x7f78f0198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:45.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.173+0000 7f78f60c6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78f0102ef0 0x7f78f019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:45.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.173+0000 7f78f60c6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78f0110f80 con 0x7f78f0102ef0 2024-09-11T23:22:45.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.173+0000 7f78f60c6640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f78f0110e00 con 0x7f78f0100e20 2024-09-11T23:22:45.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.173+0000 7f78f60c6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f78f0111100 con 0x7f78f0102570 2024-09-11T23:22:45.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.173+0000 7f78ef7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f0100e20 0x7f78f01984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:45.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.173+0000 7f78ef7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f0100e20 0x7f78f01984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:44512/0 (socket says 172.21.15.43:44512) 2024-09-11T23:22:45.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.173+0000 7f78ef7fe640 1 -- 172.21.15.43:0/1471686818 learned_addr learned my addr 172.21.15.43:0/1471686818 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:45.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.173+0000 7f78effff640 1 --2- 172.21.15.43:0/1471686818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78f0102ef0 0x7f78f019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:45.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.173+0000 7f78eeffd640 1 --2- 172.21.15.43:0/1471686818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78f0102570 0x7f78f0198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:45.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.173+0000 7f78effff640 1 -- 172.21.15.43:0/1471686818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78f0102570 msgr2=0x7f78f0198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:45.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.173+0000 7f78effff640 1 --2- 172.21.15.43:0/1471686818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78f0102570 0x7f78f0198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.173+0000 7f78effff640 1 -- 172.21.15.43:0/1471686818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f0100e20 msgr2=0x7f78f01984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:45.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.173+0000 7f78effff640 1 --2- 172.21.15.43:0/1471686818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f0100e20 0x7f78f01984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.173+0000 7f78effff640 1 -- 172.21.15.43:0/1471686818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f019d4a0 con 0x7f78f0102ef0 2024-09-11T23:22:45.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.173+0000 7f78ef7fe640 1 --2- 172.21.15.43:0/1471686818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f0100e20 0x7f78f01984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:45.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.174+0000 7f78effff640 1 --2- 172.21.15.43:0/1471686818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78f0102ef0 0x7f78f019cdc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f78e8005cc0 tx=0x7f78e802f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:45.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.174+0000 7f78ecff9640 1 -- 172.21.15.43:0/1471686818 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8034040 con 0x7f78f0102ef0 2024-09-11T23:22:45.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.174+0000 7f78ecff9640 1 -- 172.21.15.43:0/1471686818 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78e8030920 con 0x7f78f0102ef0 2024-09-11T23:22:45.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.174+0000 7f78f60c6640 1 -- 172.21.15.43:0/1471686818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f019d730 con 0x7f78f0102ef0 2024-09-11T23:22:45.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.174+0000 7f78f60c6640 1 -- 172.21.15.43:0/1471686818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f019dc00 con 0x7f78f0102ef0 2024-09-11T23:22:45.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.175+0000 7f78ecff9640 1 -- 172.21.15.43:0/1471686818 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e803cd30 con 0x7f78f0102ef0 2024-09-11T23:22:45.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.177+0000 7f78f849f740 1 Processor -- start 2024-09-11T23:22:45.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.178+0000 7f78ecff9640 1 -- 172.21.15.43:0/1471686818 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f78e804e060 con 0x7f78f0102ef0 2024-09-11T23:22:45.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.178+0000 7f78f849f740 1 -- start start 2024-09-11T23:22:45.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.178+0000 7f78f849f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f257b40 0x56291f256090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:45.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.178+0000 7f78f849f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f2565d0 0x56291f256a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:45.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.178+0000 7f78f849f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f261370 0x56291f263760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:45.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.178+0000 7f78f849f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56291f1bc2f0 con 0x56291f261370 2024-09-11T23:22:45.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.178+0000 7f78f849f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56291f163cf0 con 0x56291f257b40 2024-09-11T23:22:45.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.178+0000 7f78f849f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56291f1b82b0 con 0x56291f2565d0 2024-09-11T23:22:45.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.178+0000 7f78eeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f261370 0x56291f263760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:45.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.178+0000 7f78eeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f261370 0x56291f263760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54034/0 (socket says 172.21.15.43:54034) 2024-09-11T23:22:45.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.178+0000 7f78eeffd640 1 -- 172.21.15.43:0/614751548 learned_addr learned my addr 172.21.15.43:0/614751548 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:45.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.178+0000 7f78eeffd640 1 --2- 172.21.15.43:0/614751548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f257b40 0x56291f256090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:45.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.179+0000 7f78ef7fe640 1 --2- 172.21.15.43:0/614751548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f2565d0 0x56291f256a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:45.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.179+0000 7f78eeffd640 1 -- 172.21.15.43:0/614751548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f2565d0 msgr2=0x56291f256a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:45.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.179+0000 7f78eeffd640 1 --2- 172.21.15.43:0/614751548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f2565d0 0x56291f256a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.179+0000 7f78eeffd640 1 -- 172.21.15.43:0/614751548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f257b40 msgr2=0x56291f256090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:45.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.179+0000 7f78eeffd640 1 --2- 172.21.15.43:0/614751548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f257b40 0x56291f256090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.179+0000 7f78eeffd640 1 -- 172.21.15.43:0/614751548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56291f1483a0 con 0x56291f261370 2024-09-11T23:22:45.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.179+0000 7f78eeffd640 1 --2- 172.21.15.43:0/614751548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f261370 0x56291f263760 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f78e40058c0 tx=0x7f78e4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9082b091a623de55 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:45.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.179+0000 7f78d5ffb640 1 -- 172.21.15.43:0/614751548 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e400f7c0 con 0x56291f261370 2024-09-11T23:22:45.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.179+0000 7f78d5ffb640 1 -- 172.21.15.43:0/614751548 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78e400fe40 con 0x56291f261370 2024-09-11T23:22:45.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.179+0000 7f78d5ffb640 1 -- 172.21.15.43:0/614751548 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e400e3b0 con 0x56291f261370 2024-09-11T23:22:45.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.180+0000 7f78ecff9640 1 --2- 172.21.15.43:0/1471686818 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f78d80a2e90 0x7f78d80a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:45.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.180+0000 7f78ecff9640 1 -- 172.21.15.43:0/1471686818 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f78e80f3e00 con 0x7f78f0102ef0 2024-09-11T23:22:45.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.180+0000 7f78ef7fe640 1 --2- 172.21.15.43:0/1471686818 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f78d80a2e90 0x7f78d80a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:45.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.180+0000 7f78f849f740 1 -- 172.21.15.43:0/614751548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f261370 msgr2=0x56291f263760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:45.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.180+0000 7f78f849f740 1 --2- 172.21.15.43:0/614751548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f261370 0x56291f263760 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f78e40058c0 tx=0x7f78e4005850 comp rx=0 tx=0).stop 2024-09-11T23:22:45.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.180+0000 7f78f849f740 1 -- 172.21.15.43:0/614751548 shutdown_connections 2024-09-11T23:22:45.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.180+0000 7f78f849f740 1 --2- 172.21.15.43:0/614751548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f261370 0x56291f263760 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.180+0000 7f78f849f740 1 --2- 172.21.15.43:0/614751548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f2565d0 0x56291f256a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.180+0000 7f78f849f740 1 --2- 172.21.15.43:0/614751548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f257b40 0x56291f256090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.180+0000 7f78f849f740 1 -- 172.21.15.43:0/614751548 >> 172.21.15.43:0/614751548 conn(0x56291f21f310 msgr2=0x56291f252c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:45.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.181+0000 7f78ef7fe640 1 --2- 172.21.15.43:0/1471686818 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f78d80a2e90 0x7f78d80a5350 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f78e0009fd0 tx=0x7f78e0005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:45.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.181+0000 7f78f849f740 1 -- 172.21.15.43:0/614751548 shutdown_connections 2024-09-11T23:22:45.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.181+0000 7f78f849f740 1 -- 172.21.15.43:0/614751548 wait complete. 2024-09-11T23:22:45.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.181+0000 7f78f849f740 1 -- start start 2024-09-11T23:22:45.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.181+0000 7f78f849f740 1 Processor -- start 2024-09-11T23:22:45.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.182+0000 7f78f849f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f2565d0 0x56291f25e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.182+0000 7f78f849f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f257b40 0x56291f25e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.182+0000 7f78f849f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f261370 0x56291f25eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.182+0000 7f78ef7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f2565d0 0x56291f25e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.182+0000 7f78ef7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f2565d0 0x56291f25e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:44532/0 (socket says 172.21.15.43:44532) 2024-09-11T23:22:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.182+0000 7f78ef7fe640 1 -- 172.21.15.43:0/1446472504 learned_addr learned my addr 172.21.15.43:0/1446472504 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.182+0000 7f78eeffd640 1 --2- 172.21.15.43:0/1446472504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f257b40 0x56291f25e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.182+0000 7f78f849f740 1 -- 172.21.15.43:0/1446472504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56291f264070 con 0x56291f257b40 2024-09-11T23:22:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.182+0000 7f78f849f740 1 -- 172.21.15.43:0/1446472504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56291f177060 con 0x56291f2565d0 2024-09-11T23:22:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.182+0000 7f78f849f740 1 -- 172.21.15.43:0/1446472504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56291f2641f0 con 0x56291f261370 2024-09-11T23:22:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.182+0000 7f78f849f740 10 client.0 _ll_register_callbacks cb 0x56291eeca8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.182+0000 7f78effff640 1 --2- 172.21.15.43:0/1446472504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f261370 0x56291f25eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.182+0000 7f78ef7fe640 1 -- 172.21.15.43:0/1446472504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f261370 msgr2=0x56291f25eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.182+0000 7f78ef7fe640 1 --2- 172.21.15.43:0/1446472504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f261370 0x56291f25eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.182+0000 7f78ef7fe640 1 -- 172.21.15.43:0/1446472504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f257b40 msgr2=0x56291f25e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:45.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.182+0000 7f78ef7fe640 1 --2- 172.21.15.43:0/1446472504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f257b40 0x56291f25e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.182+0000 7f78ef7fe640 1 -- 172.21.15.43:0/1446472504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56291ef2d3c0 con 0x56291f2565d0 2024-09-11T23:22:45.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.183+0000 7f78ef7fe640 1 --2- 172.21.15.43:0/1446472504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f2565d0 0x56291f25e0f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f78e0012f70 tx=0x7f78e0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=136e91d7b2980606 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:45.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.183+0000 7f78ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:22:45.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.183+0000 7f78ceffd640 1 -- 172.21.15.43:0/1446472504 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e000fe60 con 0x56291f2565d0 2024-09-11T23:22:45.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.183+0000 7f78ceffd640 1 -- 172.21.15.43:0/1446472504 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f78e0011b50 con 0x56291f2565d0 2024-09-11T23:22:45.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.183+0000 7f78f849f740 1 -- 172.21.15.43:0/1446472504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78e400fe40 con 0x56291f2565d0 2024-09-11T23:22:45.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.183+0000 7f78ceffd640 1 -- 172.21.15.43:0/1446472504 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f78e0011d50 con 0x56291f2565d0 2024-09-11T23:22:45.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.183+0000 7f78f849f740 10 client.25172 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:45.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.183+0000 7f78f849f740 1 -- 172.21.15.43:0/1446472504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56291f26fca0 con 0x56291f2565d0 2024-09-11T23:22:45.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.184+0000 7f78ceffd640 1 -- 172.21.15.43:0/1446472504 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78e400fe40 con 0x56291f2565d0 2024-09-11T23:22:45.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.184+0000 7f78ceffd640 1 -- 172.21.15.43:0/1446472504 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f78e0027070 con 0x56291f2565d0 2024-09-11T23:22:45.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.184+0000 7f78f849f740 10 client.25172 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:45.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.184+0000 7f78f849f740 10 client.25172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:45.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.184+0000 7f78f849f740 20 client.25172 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:45.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.184+0000 7f78f849f740 1 --2- 172.21.15.43:0/1446472504 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56291f2715d0 0x56291f273a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:45.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.184+0000 7f78f849f740 4 client.25172 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:22:45.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.184+0000 7f78f849f740 1 -- 172.21.15.43:0/1446472504 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56291f1c9860 con 0x56291f2715d0 2024-09-11T23:22:45.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.184+0000 7f78eeffd640 1 --2- 172.21.15.43:0/1446472504 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56291f2715d0 0x56291f273a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:45.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.185+0000 7f78ceffd640 10 client.25172 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:45.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.185+0000 7f78eeffd640 1 --2- 172.21.15.43:0/1446472504 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56291f2715d0 0x56291f273a70 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f43d48dbf6289f5 server_cookie=591cd477030fb167 in_seq=0 out_seq=0 2024-09-11T23:22:45.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.185+0000 7f78ceffd640 1 -- 172.21.15.43:0/1446472504 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f78e402ae80 con 0x56291f2715d0 2024-09-11T23:22:45.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.185+0000 7f78ceffd640 10 client.25172 handle_command_reply: tid=0 2024-09-11T23:22:45.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.185+0000 7f78f849f740 1 client.25172 shutdown 2024-09-11T23:22:45.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.186+0000 7f78f849f740 1 -- 172.21.15.43:0/1446472504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f2565d0 msgr2=0x56291f25e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:45.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.186+0000 7f78f849f740 1 --2- 172.21.15.43:0/1446472504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f2565d0 0x56291f25e0f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f78e0012f70 tx=0x7f78e0004730 comp rx=0 tx=0).stop 2024-09-11T23:22:45.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.186+0000 7f78f849f740 1 -- 172.21.15.43:0/1446472504 shutdown_connections 2024-09-11T23:22:45.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.186+0000 7f78f849f740 1 --2- 172.21.15.43:0/1446472504 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56291f2715d0 0x56291f273a70 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.186+0000 7f78f849f740 1 --2- 172.21.15.43:0/1446472504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f261370 0x56291f25eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.186+0000 7f78f849f740 1 --2- 172.21.15.43:0/1446472504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f257b40 0x56291f25e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.186+0000 7f78ceffd640 0 client.25172 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:45.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.186+0000 7f78f849f740 1 --2- 172.21.15.43:0/1446472504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f2565d0 0x56291f25e0f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.186+0000 7f78f849f740 1 -- 172.21.15.43:0/1446472504 >> 172.21.15.43:0/1446472504 conn(0x56291f21f310 msgr2=0x56291f258960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:45.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.186+0000 7f78f849f740 1 -- 172.21.15.43:0/1446472504 shutdown_connections 2024-09-11T23:22:45.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.186+0000 7f78f849f740 1 -- 172.21.15.43:0/1446472504 wait complete. 2024-09-11T23:22:45.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.186+0000 7f78f849f740 20 client.25172 trim_cache size 0 max 16384 2024-09-11T23:22:45.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.199+0000 7f78f849f740 1 Processor -- start 2024-09-11T23:22:45.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.200+0000 7f78f849f740 1 -- start start 2024-09-11T23:22:45.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.200+0000 7f78f849f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f2565d0 0x56291f25d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:45.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.200+0000 7f78f849f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f257b40 0x56291f262760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:45.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.200+0000 7f78f849f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f261370 0x56291f262ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:45.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.200+0000 7f78f849f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78f01adf50 con 0x56291f261370 2024-09-11T23:22:45.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.200+0000 7f78f849f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f78f01ab820 con 0x56291f2565d0 2024-09-11T23:22:45.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.200+0000 7f78f849f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f78f01a9670 con 0x56291f257b40 2024-09-11T23:22:45.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.200+0000 7f78effff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f261370 0x56291f262ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:45.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.200+0000 7f78effff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f261370 0x56291f262ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54052/0 (socket says 172.21.15.43:54052) 2024-09-11T23:22:45.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.200+0000 7f78effff640 1 -- 172.21.15.43:0/1927462265 learned_addr learned my addr 172.21.15.43:0/1927462265 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:45.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.200+0000 7f78ef7fe640 1 --2- 172.21.15.43:0/1927462265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f2565d0 0x56291f25d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:45.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.200+0000 7f78eeffd640 1 --2- 172.21.15.43:0/1927462265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f257b40 0x56291f262760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:45.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.200+0000 7f78effff640 1 -- 172.21.15.43:0/1927462265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f257b40 msgr2=0x56291f262760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:45.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.200+0000 7f78effff640 1 --2- 172.21.15.43:0/1927462265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f257b40 0x56291f262760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.200+0000 7f78effff640 1 -- 172.21.15.43:0/1927462265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f2565d0 msgr2=0x56291f25d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:45.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.200+0000 7f78effff640 1 --2- 172.21.15.43:0/1927462265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f2565d0 0x56291f25d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.200+0000 7f78effff640 1 -- 172.21.15.43:0/1927462265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56291f24df90 con 0x56291f261370 2024-09-11T23:22:45.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.200+0000 7f78ef7fe640 1 --2- 172.21.15.43:0/1927462265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f2565d0 0x56291f25d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:45.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.201+0000 7f78effff640 1 --2- 172.21.15.43:0/1927462265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f261370 0x56291f262ca0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f78e80bfdd0 tx=0x7f78e80bfaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd9cd70b517acf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:45.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.201+0000 7f78d57fa640 1 -- 172.21.15.43:0/1927462265 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e80c00d0 con 0x56291f261370 2024-09-11T23:22:45.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.201+0000 7f78d57fa640 1 -- 172.21.15.43:0/1927462265 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78e80c4480 con 0x56291f261370 2024-09-11T23:22:45.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.201+0000 7f78d57fa640 1 -- 172.21.15.43:0/1927462265 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e80c0330 con 0x56291f261370 2024-09-11T23:22:45.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.202+0000 7f78f849f740 1 -- 172.21.15.43:0/1927462265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f261370 msgr2=0x56291f262ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:45.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.202+0000 7f78f849f740 1 --2- 172.21.15.43:0/1927462265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f261370 0x56291f262ca0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f78e80bfdd0 tx=0x7f78e80bfaf0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.202+0000 7f78f849f740 1 -- 172.21.15.43:0/1927462265 shutdown_connections 2024-09-11T23:22:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.202+0000 7f78f849f740 1 --2- 172.21.15.43:0/1927462265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f261370 0x56291f262ca0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.202+0000 7f78f849f740 1 --2- 172.21.15.43:0/1927462265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f257b40 0x56291f262760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.202+0000 7f78f849f740 1 --2- 172.21.15.43:0/1927462265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f2565d0 0x56291f25d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.202+0000 7f78f849f740 1 -- 172.21.15.43:0/1927462265 >> 172.21.15.43:0/1927462265 conn(0x56291f21f310 msgr2=0x56291f26cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.202+0000 7f78f849f740 1 -- 172.21.15.43:0/1927462265 shutdown_connections 2024-09-11T23:22:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.202+0000 7f78f849f740 1 -- 172.21.15.43:0/1927462265 wait complete. 2024-09-11T23:22:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.203+0000 7f78f849f740 1 -- start start 2024-09-11T23:22:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.203+0000 7f78f849f740 1 Processor -- start 2024-09-11T23:22:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.203+0000 7f78f849f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f2565d0 0x56291f2698a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.203+0000 7f78f849f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f257b40 0x56291f269de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.203+0000 7f78f849f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f261370 0x56291f25b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.204+0000 7f78f849f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f78f01a9ce0 con 0x56291f261370 2024-09-11T23:22:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.204+0000 7f78f849f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f78f01ab150 con 0x56291f257b40 2024-09-11T23:22:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.204+0000 7f78eeffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f257b40 0x56291f269de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.204+0000 7f78ef7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f2565d0 0x56291f2698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.204+0000 7f78ef7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f2565d0 0x56291f2698a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:54102/0 (socket says 172.21.15.43:54102) 2024-09-11T23:22:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.204+0000 7f78ef7fe640 1 -- 172.21.15.43:0/446763134 learned_addr learned my addr 172.21.15.43:0/446763134 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.204+0000 7f78f849f740 1 -- 172.21.15.43:0/446763134 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f78f01aa3b0 con 0x56291f2565d0 2024-09-11T23:22:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.204+0000 7f78effff640 1 --2- 172.21.15.43:0/446763134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f261370 0x56291f25b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.204+0000 7f78f849f740 10 client.0 _ll_register_callbacks cb 0x7f78e0012e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.204+0000 7f78ef7fe640 1 -- 172.21.15.43:0/446763134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f257b40 msgr2=0x56291f269de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.204+0000 7f78ef7fe640 1 --2- 172.21.15.43:0/446763134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f257b40 0x56291f269de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.204+0000 7f78ef7fe640 1 -- 172.21.15.43:0/446763134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f261370 msgr2=0x56291f25b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.204+0000 7f78ef7fe640 1 --2- 172.21.15.43:0/446763134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f261370 0x56291f25b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.204+0000 7f78ef7fe640 1 -- 172.21.15.43:0/446763134 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56291f25bf10 con 0x56291f2565d0 2024-09-11T23:22:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.204+0000 7f78eeffd640 1 --2- 172.21.15.43:0/446763134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f257b40 0x56291f269de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.204+0000 7f78effff640 1 --2- 172.21.15.43:0/446763134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f261370 0x56291f25b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.204+0000 7f78ef7fe640 1 --2- 172.21.15.43:0/446763134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f2565d0 0x56291f2698a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f78e0012f70 tx=0x7f78e0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db6ba036001b632b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.204+0000 7f78ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-11T23:22:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.205+0000 7f78ceffd640 1 -- 172.21.15.43:0/446763134 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e0034070 con 0x56291f2565d0 2024-09-11T23:22:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.205+0000 7f78ceffd640 1 -- 172.21.15.43:0/446763134 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f78e00043f0 con 0x56291f2565d0 2024-09-11T23:22:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.205+0000 7f78f849f740 1 -- 172.21.15.43:0/446763134 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78e80c4480 con 0x56291f2565d0 2024-09-11T23:22:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.205+0000 7f78ceffd640 1 -- 172.21.15.43:0/446763134 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f78e001f8d0 con 0x56291f2565d0 2024-09-11T23:22:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.205+0000 7f78f849f740 10 client.25186 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.205+0000 7f78f849f740 1 -- 172.21.15.43:0/446763134 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56291f2dc350 con 0x56291f2565d0 2024-09-11T23:22:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.205+0000 7f78ceffd640 1 -- 172.21.15.43:0/446763134 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78e80c4480 con 0x56291f2565d0 2024-09-11T23:22:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.205+0000 7f78ceffd640 1 -- 172.21.15.43:0/446763134 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f78e0027070 con 0x56291f2565d0 2024-09-11T23:22:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.205+0000 7f78f849f740 10 client.25186 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:45.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.205+0000 7f78f849f740 10 client.25186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:45.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.205+0000 7f78f849f740 20 client.25186 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:45.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.205+0000 7f78f849f740 1 --2- 172.21.15.43:0/446763134 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56291f2dc860 0x56291f2dec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:45.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.205+0000 7f78f849f740 4 client.25186 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:22:45.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.205+0000 7f78f849f740 1 -- 172.21.15.43:0/446763134 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56291f1c3d70 con 0x56291f2dc860 2024-09-11T23:22:45.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.206+0000 7f78eeffd640 1 --2- 172.21.15.43:0/446763134 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56291f2dc860 0x56291f2dec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:45.213 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:22:45.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.206+0000 7f78ceffd640 10 client.25186 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:45.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.206+0000 7f78eeffd640 1 --2- 172.21.15.43:0/446763134 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56291f2dc860 0x56291f2dec50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b61400f55e48af6c server_cookie=ae05e5745221b0df in_seq=0 out_seq=0 2024-09-11T23:22:45.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.206+0000 7f78ceffd640 1 -- 172.21.15.43:0/446763134 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56291f1c3d70 con 0x56291f2dc860 2024-09-11T23:22:45.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.206+0000 7f78ceffd640 10 client.25186 handle_command_reply: tid=0 2024-09-11T23:22:45.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.206+0000 7f78f849f740 1 client.25186 shutdown 2024-09-11T23:22:45.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.207+0000 7f78f849f740 1 -- 172.21.15.43:0/446763134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f2565d0 msgr2=0x56291f2698a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:45.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.207+0000 7f78f849f740 1 --2- 172.21.15.43:0/446763134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f2565d0 0x56291f2698a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f78e0012f70 tx=0x7f78e0011810 comp rx=0 tx=0).stop 2024-09-11T23:22:45.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.207+0000 7f78f849f740 1 -- 172.21.15.43:0/446763134 shutdown_connections 2024-09-11T23:22:45.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.207+0000 7f78f849f740 1 --2- 172.21.15.43:0/446763134 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56291f2dc860 0x56291f2dec50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.207+0000 7f78ceffd640 0 client.25186 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:45.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.207+0000 7f78f849f740 1 --2- 172.21.15.43:0/446763134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56291f261370 0x56291f25b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.207+0000 7f78f849f740 1 --2- 172.21.15.43:0/446763134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56291f257b40 0x56291f269de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.207+0000 7f78f849f740 1 --2- 172.21.15.43:0/446763134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56291f2565d0 0x56291f2698a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.207+0000 7f78f849f740 1 -- 172.21.15.43:0/446763134 >> 172.21.15.43:0/446763134 conn(0x56291f21f310 msgr2=0x56291f25f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:45.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.207+0000 7f78f849f740 1 -- 172.21.15.43:0/446763134 shutdown_connections 2024-09-11T23:22:45.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.207+0000 7f78f849f740 1 -- 172.21.15.43:0/446763134 wait complete. 2024-09-11T23:22:45.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.207+0000 7f78f849f740 20 client.25186 trim_cache size 0 max 16384 2024-09-11T23:22:45.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.208+0000 7f78f60c6640 1 -- 172.21.15.43:0/1471686818 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f78d80a2e90 msgr2=0x7f78d80a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:45.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.208+0000 7f78f60c6640 1 --2- 172.21.15.43:0/1471686818 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f78d80a2e90 0x7f78d80a5350 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f78e0009fd0 tx=0x7f78e0005f90 comp rx=0 tx=0).stop 2024-09-11T23:22:45.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.208+0000 7f78f60c6640 1 -- 172.21.15.43:0/1471686818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78f0102ef0 msgr2=0x7f78f019cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:45.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.208+0000 7f78f60c6640 1 --2- 172.21.15.43:0/1471686818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78f0102ef0 0x7f78f019cdc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f78e8005cc0 tx=0x7f78e802f3a0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.208+0000 7f78f60c6640 1 -- 172.21.15.43:0/1471686818 shutdown_connections 2024-09-11T23:22:45.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.208+0000 7f78f60c6640 1 --2- 172.21.15.43:0/1471686818 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f78d80a2e90 0x7f78d80a5350 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.208+0000 7f78f60c6640 1 --2- 172.21.15.43:0/1471686818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78f0102ef0 0x7f78f019cdc0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.208+0000 7f78f60c6640 1 --2- 172.21.15.43:0/1471686818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78f0102570 0x7f78f0198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.208+0000 7f78f60c6640 1 --2- 172.21.15.43:0/1471686818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f0100e20 0x7f78f01984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:45.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.208+0000 7f78f60c6640 1 -- 172.21.15.43:0/1471686818 >> 172.21.15.43:0/1471686818 conn(0x7f78f00fc7d0 msgr2=0x7f78f006a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:45.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.208+0000 7f78f60c6640 1 -- 172.21.15.43:0/1471686818 shutdown_connections 2024-09-11T23:22:45.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:45.208+0000 7f78f60c6640 1 -- 172.21.15.43:0/1471686818 wait complete. 2024-09-11T23:22:46.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:45 smithi043 ceph-mon[30950]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-11T23:22:46.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:45 smithi090 ceph-mon[35130]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-11T23:22:46.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:45 smithi097 ceph-mon[35562]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-11T23:22:46.947 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:22:46.948 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:22:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.764+0000 7f12717ac640 1 -- 172.21.15.43:0/169198524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f126c102ef0 msgr2=0x7f126c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:47.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.764+0000 7f12717ac640 1 --2- 172.21.15.43:0/169198524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f126c102ef0 0x7f126c10da40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1264009ca0 tx=0x7f126402a950 comp rx=0 tx=0).stop 2024-09-11T23:22:47.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.764+0000 7f12717ac640 1 -- 172.21.15.43:0/169198524 shutdown_connections 2024-09-11T23:22:47.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.764+0000 7f12717ac640 1 --2- 172.21.15.43:0/169198524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f126c102ef0 0x7f126c10da40 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.764+0000 7f12717ac640 1 --2- 172.21.15.43:0/169198524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f126c102570 0x7f126c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.764+0000 7f12717ac640 1 --2- 172.21.15.43:0/169198524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f126c100e20 0x7f126c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.764+0000 7f12717ac640 1 -- 172.21.15.43:0/169198524 >> 172.21.15.43:0/169198524 conn(0x7f126c0fc7d0 msgr2=0x7f126c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:47.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.764+0000 7f12717ac640 1 -- 172.21.15.43:0/169198524 shutdown_connections 2024-09-11T23:22:47.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.764+0000 7f12717ac640 1 -- 172.21.15.43:0/169198524 wait complete. 2024-09-11T23:22:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.765+0000 7f12717ac640 1 Processor -- start 2024-09-11T23:22:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.765+0000 7f12717ac640 1 -- start start 2024-09-11T23:22:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.765+0000 7f12717ac640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f126c100e20 0x7f126c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.765+0000 7f12717ac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f126c102570 0x7f126c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.766+0000 7f12717ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f126c102ef0 0x7f126c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.766+0000 7f12717ac640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f126c110f80 con 0x7f126c102ef0 2024-09-11T23:22:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.766+0000 7f126a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f126c102570 0x7f126c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.766+0000 7f126b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f126c102ef0 0x7f126c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.766+0000 7f126b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f126c102ef0 0x7f126c19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54080/0 (socket says 172.21.15.43:54080) 2024-09-11T23:22:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.766+0000 7f126b7fe640 1 -- 172.21.15.43:0/3047440099 learned_addr learned my addr 172.21.15.43:0/3047440099 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.766+0000 7f126affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f126c100e20 0x7f126c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.766+0000 7f12717ac640 1 -- 172.21.15.43:0/3047440099 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f126c110e00 con 0x7f126c100e20 2024-09-11T23:22:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.766+0000 7f12717ac640 1 -- 172.21.15.43:0/3047440099 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f126c111100 con 0x7f126c102570 2024-09-11T23:22:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.766+0000 7f126b7fe640 1 -- 172.21.15.43:0/3047440099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f126c102570 msgr2=0x7f126c19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.766+0000 7f126b7fe640 1 --2- 172.21.15.43:0/3047440099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f126c102570 0x7f126c19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.766+0000 7f126b7fe640 1 -- 172.21.15.43:0/3047440099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f126c100e20 msgr2=0x7f126c19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.766+0000 7f126b7fe640 1 --2- 172.21.15.43:0/3047440099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f126c100e20 0x7f126c19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.766+0000 7f126b7fe640 1 -- 172.21.15.43:0/3047440099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126c19f5f0 con 0x7f126c102ef0 2024-09-11T23:22:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.767+0000 7f126b7fe640 1 --2- 172.21.15.43:0/3047440099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f126c102ef0 0x7f126c19ef10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1264004af0 tx=0x7f126402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.767+0000 7f1253fff640 1 -- 172.21.15.43:0/3047440099 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264034040 con 0x7f126c102ef0 2024-09-11T23:22:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.767+0000 7f1253fff640 1 -- 172.21.15.43:0/3047440099 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1264030920 con 0x7f126c102ef0 2024-09-11T23:22:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.767+0000 7f1253fff640 1 -- 172.21.15.43:0/3047440099 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264004630 con 0x7f126c102ef0 2024-09-11T23:22:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.767+0000 7f12717ac640 1 -- 172.21.15.43:0/3047440099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126c19f880 con 0x7f126c102ef0 2024-09-11T23:22:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.767+0000 7f12717ac640 1 -- 172.21.15.43:0/3047440099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f126c19fd10 con 0x7f126c102ef0 2024-09-11T23:22:47.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.770+0000 7f1253fff640 1 -- 172.21.15.43:0/3047440099 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f1264037020 con 0x7f126c102ef0 2024-09-11T23:22:47.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.770+0000 7f1273b85740 1 Processor -- start 2024-09-11T23:22:47.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.770+0000 7f1273b85740 1 -- start start 2024-09-11T23:22:47.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.771+0000 7f1273b85740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dace2b60 0x55c4dace2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:47.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.771+0000 7f1273b85740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4dace3480 0x55c4daced800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:47.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.771+0000 7f1273b85740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4dacedd40 0x55c4dacf0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:47.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.771+0000 7f1273b85740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c4dac48280 con 0x55c4dace2b60 2024-09-11T23:22:47.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.771+0000 7f1273b85740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c4dabefb10 con 0x55c4dacedd40 2024-09-11T23:22:47.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.771+0000 7f1273b85740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c4dac44240 con 0x55c4dace3480 2024-09-11T23:22:47.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.771+0000 7f1253fff640 1 --2- 172.21.15.43:0/3047440099 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f12540a2e90 0x7f12540a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:47.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.771+0000 7f126a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dace2b60 0x55c4dace2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:47.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.771+0000 7f126a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dace2b60 0x55c4dace2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54082/0 (socket says 172.21.15.43:54082) 2024-09-11T23:22:47.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.771+0000 7f126a7fc640 1 -- 172.21.15.43:0/682297964 learned_addr learned my addr 172.21.15.43:0/682297964 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.771+0000 7f126affd640 1 --2- 172.21.15.43:0/682297964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4dace3480 0x55c4daced800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.771+0000 7f126a7fc640 1 --2- 172.21.15.43:0/682297964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4dacedd40 0x55c4dacf0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.771+0000 7f1253fff640 1 -- 172.21.15.43:0/3047440099 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f12640f0350 con 0x7f126c102ef0 2024-09-11T23:22:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.771+0000 7f126b7fe640 1 --2- 172.21.15.43:0/3047440099 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f12540a2e90 0x7f12540a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.772+0000 7f126affd640 1 -- 172.21.15.43:0/682297964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4dacedd40 msgr2=0x55c4dacf0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.772+0000 7f126affd640 1 --2- 172.21.15.43:0/682297964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4dacedd40 0x55c4dacf0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.772+0000 7f126affd640 1 -- 172.21.15.43:0/682297964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dace2b60 msgr2=0x55c4dace2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.772+0000 7f126affd640 1 --2- 172.21.15.43:0/682297964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dace2b60 0x55c4dace2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.772+0000 7f126b7fe640 1 --2- 172.21.15.43:0/3047440099 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f12540a2e90 0x7f12540a5350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f12640eda90 tx=0x7f126403c040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.772+0000 7f126affd640 1 -- 172.21.15.43:0/682297964 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4dabfc1f0 con 0x55c4dace3480 2024-09-11T23:22:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.772+0000 7f126a7fc640 1 --2- 172.21.15.43:0/682297964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dace2b60 0x55c4dace2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.772+0000 7f126affd640 1 --2- 172.21.15.43:0/682297964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4dace3480 0x55c4daced800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f125c009c20 tx=0x7f125c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=940384d9960216bd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.772+0000 7f12517fa640 1 -- 172.21.15.43:0/682297964 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c01e070 con 0x55c4dace3480 2024-09-11T23:22:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.772+0000 7f12517fa640 1 -- 172.21.15.43:0/682297964 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f125c009dd0 con 0x55c4dace3480 2024-09-11T23:22:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.772+0000 7f12517fa640 1 -- 172.21.15.43:0/682297964 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c0187b0 con 0x55c4dace3480 2024-09-11T23:22:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.773+0000 7f1273b85740 1 -- 172.21.15.43:0/682297964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4dace3480 msgr2=0x55c4daced800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.773+0000 7f1273b85740 1 --2- 172.21.15.43:0/682297964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4dace3480 0x55c4daced800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f125c009c20 tx=0x7f125c011810 comp rx=0 tx=0).stop 2024-09-11T23:22:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.773+0000 7f1273b85740 1 -- 172.21.15.43:0/682297964 shutdown_connections 2024-09-11T23:22:47.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.773+0000 7f1273b85740 1 --2- 172.21.15.43:0/682297964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4dacedd40 0x55c4dacf0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.773+0000 7f1273b85740 1 --2- 172.21.15.43:0/682297964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4dace3480 0x55c4daced800 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.773+0000 7f1273b85740 1 --2- 172.21.15.43:0/682297964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dace2b60 0x55c4dace2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.773+0000 7f1273b85740 1 -- 172.21.15.43:0/682297964 >> 172.21.15.43:0/682297964 conn(0x55c4dacdf0f0 msgr2=0x55c4dacdf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:47.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.773+0000 7f1273b85740 1 -- 172.21.15.43:0/682297964 shutdown_connections 2024-09-11T23:22:47.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.773+0000 7f1273b85740 1 -- 172.21.15.43:0/682297964 wait complete. 2024-09-11T23:22:47.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.774+0000 7f1273b85740 1 -- start start 2024-09-11T23:22:47.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.774+0000 7f1273b85740 1 Processor -- start 2024-09-11T23:22:47.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.774+0000 7f1273b85740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dace2b60 0x55c4daceabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:47.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.774+0000 7f1273b85740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4dace3480 0x55c4daceb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:47.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.774+0000 7f1273b85740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4dacedd40 0x55c4daceb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:47.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.774+0000 7f1273b85740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c4dacf0910 con 0x55c4dace2b60 2024-09-11T23:22:47.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.774+0000 7f1273b85740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c4dac02f10 con 0x55c4dace3480 2024-09-11T23:22:47.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.774+0000 7f1273b85740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c4dacf0a90 con 0x55c4dacedd40 2024-09-11T23:22:47.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.774+0000 7f1273b85740 10 client.0 _ll_register_callbacks cb 0x55c4dabcba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:47.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.775+0000 7f126a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dace2b60 0x55c4daceabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:47.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.775+0000 7f126a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dace2b60 0x55c4daceabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54084/0 (socket says 172.21.15.43:54084) 2024-09-11T23:22:47.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.775+0000 7f126a7fc640 1 -- 172.21.15.43:0/2883556586 learned_addr learned my addr 172.21.15.43:0/2883556586 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:47.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.775+0000 7f126affd640 1 --2- 172.21.15.43:0/2883556586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4dace3480 0x55c4daceb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:47.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.775+0000 7f126affd640 1 -- 172.21.15.43:0/2883556586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4dacedd40 msgr2=0x55c4daceb630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:22:47.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.775+0000 7f126affd640 1 --2- 172.21.15.43:0/2883556586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4dacedd40 0x55c4daceb630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.775+0000 7f126affd640 1 -- 172.21.15.43:0/2883556586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dace2b60 msgr2=0x55c4daceabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:47.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.775+0000 7f126affd640 1 --2- 172.21.15.43:0/2883556586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dace2b60 0x55c4daceabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.775+0000 7f126affd640 1 -- 172.21.15.43:0/2883556586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4da9b96a0 con 0x55c4dace3480 2024-09-11T23:22:47.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.775+0000 7f126a7fc640 1 --2- 172.21.15.43:0/2883556586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dace2b60 0x55c4daceabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:47.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.775+0000 7f126affd640 1 --2- 172.21.15.43:0/2883556586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4dace3480 0x55c4daceb0f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f125c011d10 tx=0x7f125c019da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b7ab0db99f9cc90 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:47.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.775+0000 7f124eff5640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:22:47.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.776+0000 7f124eff5640 1 -- 172.21.15.43:0/2883556586 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c01e040 con 0x55c4dace3480 2024-09-11T23:22:47.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.776+0000 7f124eff5640 1 -- 172.21.15.43:0/2883556586 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f125c005030 con 0x55c4dace3480 2024-09-11T23:22:47.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.776+0000 7f1273b85740 1 -- 172.21.15.43:0/2883556586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f125c009dd0 con 0x55c4dace3480 2024-09-11T23:22:47.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.776+0000 7f124eff5640 1 -- 172.21.15.43:0/2883556586 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f125c031050 con 0x55c4dace3480 2024-09-11T23:22:47.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.776+0000 7f1273b85740 10 client.25190 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:47.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.776+0000 7f1273b85740 1 -- 172.21.15.43:0/2883556586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c4dacfc4d0 con 0x55c4dace3480 2024-09-11T23:22:47.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.776+0000 7f124eff5640 1 -- 172.21.15.43:0/2883556586 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f125c009dd0 con 0x55c4dace3480 2024-09-11T23:22:47.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.776+0000 7f124eff5640 1 -- 172.21.15.43:0/2883556586 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f125c02c030 con 0x55c4dace3480 2024-09-11T23:22:47.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.776+0000 7f1273b85740 10 client.25190 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:47.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.776+0000 7f1273b85740 10 client.25190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:47.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.777+0000 7f1273b85740 20 client.25190 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:47.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.777+0000 7f1273b85740 1 --2- 172.21.15.43:0/2883556586 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c4dacfd990 0x55c4dacffd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:47.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.777+0000 7f1273b85740 4 client.25190 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:22:47.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.777+0000 7f1273b85740 1 -- 172.21.15.43:0/2883556586 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4dac557f0 con 0x55c4dacfd990 2024-09-11T23:22:47.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.777+0000 7f126a7fc640 1 --2- 172.21.15.43:0/2883556586 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c4dacfd990 0x55c4dacffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:47.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.777+0000 7f124eff5640 10 client.25190 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:47.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.777+0000 7f126a7fc640 1 --2- 172.21.15.43:0/2883556586 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c4dacfd990 0x55c4dacffd80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d47f08546dff1062 server_cookie=b5ce1ca78640c698 in_seq=0 out_seq=0 2024-09-11T23:22:47.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.778+0000 7f124eff5640 1 -- 172.21.15.43:0/2883556586 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c4dac557f0 con 0x55c4dacfd990 2024-09-11T23:22:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.778+0000 7f124eff5640 10 client.25190 handle_command_reply: tid=0 2024-09-11T23:22:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.778+0000 7f1273b85740 1 client.25190 shutdown 2024-09-11T23:22:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.778+0000 7f1273b85740 1 -- 172.21.15.43:0/2883556586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4dace3480 msgr2=0x55c4daceb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.778+0000 7f1273b85740 1 --2- 172.21.15.43:0/2883556586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4dace3480 0x55c4daceb0f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f125c011d10 tx=0x7f125c019da0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.778+0000 7f1273b85740 1 -- 172.21.15.43:0/2883556586 shutdown_connections 2024-09-11T23:22:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.778+0000 7f1273b85740 1 --2- 172.21.15.43:0/2883556586 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c4dacfd990 0x55c4dacffd80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.778+0000 7f124eff5640 0 client.25190 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.778+0000 7f1273b85740 1 --2- 172.21.15.43:0/2883556586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4dacedd40 0x55c4daceb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.778+0000 7f1273b85740 1 --2- 172.21.15.43:0/2883556586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4dace3480 0x55c4daceb0f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.778+0000 7f1273b85740 1 --2- 172.21.15.43:0/2883556586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dace2b60 0x55c4daceabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.778+0000 7f1273b85740 1 -- 172.21.15.43:0/2883556586 >> 172.21.15.43:0/2883556586 conn(0x55c4dacdf0f0 msgr2=0x55c4dacde270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:47.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.778+0000 7f1273b85740 1 -- 172.21.15.43:0/2883556586 shutdown_connections 2024-09-11T23:22:47.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.778+0000 7f1273b85740 1 -- 172.21.15.43:0/2883556586 wait complete. 2024-09-11T23:22:47.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.779+0000 7f1273b85740 20 client.25190 trim_cache size 0 max 16384 2024-09-11T23:22:47.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.796+0000 7f1273b85740 1 Processor -- start 2024-09-11T23:22:47.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.796+0000 7f1273b85740 1 -- start start 2024-09-11T23:22:47.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.796+0000 7f1273b85740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dace2b60 0x55c4dad05560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:47.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.796+0000 7f1273b85740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4dace3480 0x55c4dad05aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:47.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.796+0000 7f1273b85740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4dacedd40 0x55c4dace9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:47.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.796+0000 7f1273b85740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1adf40 con 0x55c4dace2b60 2024-09-11T23:22:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.796+0000 7f1273b85740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1ab810 con 0x55c4dacedd40 2024-09-11T23:22:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.796+0000 7f1273b85740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1a9680 con 0x55c4dace3480 2024-09-11T23:22:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.796+0000 7f126a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dace2b60 0x55c4dad05560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.796+0000 7f126a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dace2b60 0x55c4dad05560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54092/0 (socket says 172.21.15.43:54092) 2024-09-11T23:22:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.796+0000 7f126a7fc640 1 -- 172.21.15.43:0/3536013162 learned_addr learned my addr 172.21.15.43:0/3536013162 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.796+0000 7f126affd640 1 --2- 172.21.15.43:0/3536013162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4dace3480 0x55c4dad05aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.797+0000 7f126a7fc640 1 --2- 172.21.15.43:0/3536013162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4dacedd40 0x55c4dace9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.797+0000 7f126a7fc640 1 -- 172.21.15.43:0/3536013162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4dace3480 msgr2=0x55c4dad05aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.797+0000 7f126a7fc640 1 --2- 172.21.15.43:0/3536013162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4dace3480 0x55c4dad05aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.797+0000 7f126a7fc640 1 -- 172.21.15.43:0/3536013162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4dacedd40 msgr2=0x55c4dace9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.797+0000 7f126a7fc640 1 --2- 172.21.15.43:0/3536013162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4dacedd40 0x55c4dace9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.797+0000 7f126a7fc640 1 -- 172.21.15.43:0/3536013162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4dacda830 con 0x55c4dace2b60 2024-09-11T23:22:47.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.797+0000 7f126a7fc640 1 --2- 172.21.15.43:0/3536013162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dace2b60 0x55c4dad05560 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1260004640 tx=0x7f1260004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1afae7e7b1331e64 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:47.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.797+0000 7f1250ff9640 1 -- 172.21.15.43:0/3536013162 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260003e40 con 0x55c4dace2b60 2024-09-11T23:22:47.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.797+0000 7f1250ff9640 1 -- 172.21.15.43:0/3536013162 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1260003fe0 con 0x55c4dace2b60 2024-09-11T23:22:47.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.797+0000 7f1250ff9640 1 -- 172.21.15.43:0/3536013162 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260011750 con 0x55c4dace2b60 2024-09-11T23:22:47.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.798+0000 7f1273b85740 1 -- 172.21.15.43:0/3536013162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dace2b60 msgr2=0x55c4dad05560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:47.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.798+0000 7f1273b85740 1 --2- 172.21.15.43:0/3536013162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dace2b60 0x55c4dad05560 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1260004640 tx=0x7f1260004730 comp rx=0 tx=0).stop 2024-09-11T23:22:47.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.798+0000 7f1273b85740 1 -- 172.21.15.43:0/3536013162 shutdown_connections 2024-09-11T23:22:47.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.798+0000 7f1273b85740 1 --2- 172.21.15.43:0/3536013162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4dacedd40 0x55c4dace9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.798+0000 7f1273b85740 1 --2- 172.21.15.43:0/3536013162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4dace3480 0x55c4dad05aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.798+0000 7f1273b85740 1 --2- 172.21.15.43:0/3536013162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dace2b60 0x55c4dad05560 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.798+0000 7f1273b85740 1 -- 172.21.15.43:0/3536013162 >> 172.21.15.43:0/3536013162 conn(0x55c4dacdf0f0 msgr2=0x55c4dacecb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:47.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.798+0000 7f1273b85740 1 -- 172.21.15.43:0/3536013162 shutdown_connections 2024-09-11T23:22:47.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.798+0000 7f1273b85740 1 -- 172.21.15.43:0/3536013162 wait complete. 2024-09-11T23:22:47.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.799+0000 7f1273b85740 1 -- start start 2024-09-11T23:22:47.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.799+0000 7f1273b85740 1 Processor -- start 2024-09-11T23:22:47.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.799+0000 7f1273b85740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4dace2b60 0x55c4dacf1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:47.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.799+0000 7f1273b85740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4dace3480 0x55c4dacf17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:47.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.799+0000 7f1273b85740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dacedd40 0x55c4dad66520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:47.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.799+0000 7f1273b85740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1a9cd0 con 0x55c4dacedd40 2024-09-11T23:22:47.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.799+0000 7f1273b85740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1ab140 con 0x55c4dace3480 2024-09-11T23:22:47.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.799+0000 7f1273b85740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1aa3a0 con 0x55c4dace2b60 2024-09-11T23:22:47.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.799+0000 7f1273b85740 10 client.0 _ll_register_callbacks cb 0x7f125c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:47.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.799+0000 7f126a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dacedd40 0x55c4dad66520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:47.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.799+0000 7f126a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dacedd40 0x55c4dad66520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54094/0 (socket says 172.21.15.43:54094) 2024-09-11T23:22:47.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.799+0000 7f126a7fc640 1 -- 172.21.15.43:0/2774704051 learned_addr learned my addr 172.21.15.43:0/2774704051 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:47.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.800+0000 7f126a7fc640 1 --2- 172.21.15.43:0/2774704051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4dace2b60 0x55c4dacf1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:47.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.800+0000 7f126affd640 1 --2- 172.21.15.43:0/2774704051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4dace3480 0x55c4dacf17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:47.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.800+0000 7f126a7fc640 1 -- 172.21.15.43:0/2774704051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4dace2b60 msgr2=0x55c4dacf1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:47.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.800+0000 7f126a7fc640 1 --2- 172.21.15.43:0/2774704051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4dace2b60 0x55c4dacf1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.800+0000 7f126a7fc640 1 -- 172.21.15.43:0/2774704051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4dace3480 msgr2=0x55c4dacf17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:47.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.800+0000 7f126a7fc640 1 --2- 172.21.15.43:0/2774704051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4dace3480 0x55c4dacf17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.800+0000 7f126a7fc640 1 -- 172.21.15.43:0/2774704051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4dace32b0 con 0x55c4dacedd40 2024-09-11T23:22:47.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.800+0000 7f126a7fc640 1 --2- 172.21.15.43:0/2774704051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dacedd40 0x55c4dad66520 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1260004d00 tx=0x7f1260022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377f8e1b5097e2ad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:47.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.800+0000 7f124eff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:47.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.800+0000 7f124eff5640 1 -- 172.21.15.43:0/2774704051 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260037060 con 0x55c4dacedd40 2024-09-11T23:22:47.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.800+0000 7f124eff5640 1 -- 172.21.15.43:0/2774704051 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f1260011b30 con 0x55c4dacedd40 2024-09-11T23:22:47.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.800+0000 7f1273b85740 1 -- 172.21.15.43:0/2774704051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1260003fe0 con 0x55c4dacedd40 2024-09-11T23:22:47.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.801+0000 7f124eff5640 1 -- 172.21.15.43:0/2774704051 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1260004da0 con 0x55c4dacedd40 2024-09-11T23:22:47.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.801+0000 7f1273b85740 10 client.15597 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:47.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.801+0000 7f1273b85740 1 -- 172.21.15.43:0/2774704051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c4dad67c60 con 0x55c4dacedd40 2024-09-11T23:22:47.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.801+0000 7f124eff5640 1 -- 172.21.15.43:0/2774704051 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1260005420 con 0x55c4dacedd40 2024-09-11T23:22:47.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.801+0000 7f124eff5640 1 -- 172.21.15.43:0/2774704051 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f126002f070 con 0x55c4dacedd40 2024-09-11T23:22:47.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.801+0000 7f1273b85740 10 client.15597 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:47.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.801+0000 7f1273b85740 10 client.15597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:47.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.801+0000 7f1273b85740 20 client.15597 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:47.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.801+0000 7f1273b85740 1 --2- 172.21.15.43:0/2774704051 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c4dacfd990 0x55c4dad6a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:47.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.801+0000 7f1273b85740 4 client.15597 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:22:47.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.802+0000 7f1273b85740 1 -- 172.21.15.43:0/2774704051 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f1260003fe0 con 0x55c4dacfd990 2024-09-11T23:22:47.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.802+0000 7f126affd640 1 --2- 172.21.15.43:0/2774704051 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c4dacfd990 0x55c4dad6a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:47.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.802+0000 7f126affd640 1 --2- 172.21.15.43:0/2774704051 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c4dacfd990 0x55c4dad6a340 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c1c11115cda383a server_cookie=aef17f0a25d39787 in_seq=0 out_seq=0 2024-09-11T23:22:47.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.802+0000 7f124eff5640 10 client.15597 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:47.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.803+0000 7f124eff5640 1 -- 172.21.15.43:0/2774704051 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f1260003fe0 con 0x55c4dacfd990 2024-09-11T23:22:47.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.803+0000 7f124eff5640 10 client.15597 handle_command_reply: tid=0 2024-09-11T23:22:47.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.803+0000 7f1273b85740 1 client.15597 shutdown 2024-09-11T23:22:47.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.803+0000 7f1273b85740 1 -- 172.21.15.43:0/2774704051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dacedd40 msgr2=0x55c4dad66520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:47.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.803+0000 7f1273b85740 1 --2- 172.21.15.43:0/2774704051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dacedd40 0x55c4dad66520 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1260004d00 tx=0x7f1260022040 comp rx=0 tx=0).stop 2024-09-11T23:22:47.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.804+0000 7f1273b85740 1 -- 172.21.15.43:0/2774704051 shutdown_connections 2024-09-11T23:22:47.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.804+0000 7f1273b85740 1 --2- 172.21.15.43:0/2774704051 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c4dacfd990 0x55c4dad6a340 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.804+0000 7f124eff5640 0 client.15597 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:47.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.804+0000 7f1273b85740 1 --2- 172.21.15.43:0/2774704051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4dacedd40 0x55c4dad66520 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.804+0000 7f1273b85740 1 --2- 172.21.15.43:0/2774704051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c4dace3480 0x55c4dacf17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.804+0000 7f1273b85740 1 --2- 172.21.15.43:0/2774704051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4dace2b60 0x55c4dacf1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.804+0000 7f1273b85740 1 -- 172.21.15.43:0/2774704051 >> 172.21.15.43:0/2774704051 conn(0x55c4dacdf0f0 msgr2=0x55c4dace45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:47.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.804+0000 7f1273b85740 1 -- 172.21.15.43:0/2774704051 shutdown_connections 2024-09-11T23:22:47.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.804+0000 7f1273b85740 1 -- 172.21.15.43:0/2774704051 wait complete. 2024-09-11T23:22:47.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.804+0000 7f1273b85740 20 client.15597 trim_cache size 0 max 16384 2024-09-11T23:22:47.808 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:22:47.808 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-11T23:22:47.808 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "c4de2c2c-6e2a-4b5d-8461-42fe147e3305", 2024-09-11T23:22:47.808 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-11T23:22:47.808 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:22:47.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.805+0000 7f12717ac640 1 -- 172.21.15.43:0/3047440099 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f12540a2e90 msgr2=0x7f12540a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:47.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.805+0000 7f12717ac640 1 --2- 172.21.15.43:0/3047440099 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f12540a2e90 0x7f12540a5350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f12640eda90 tx=0x7f126403c040 comp rx=0 tx=0).stop 2024-09-11T23:22:47.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.805+0000 7f12717ac640 1 -- 172.21.15.43:0/3047440099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f126c102ef0 msgr2=0x7f126c19ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:47.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.805+0000 7f12717ac640 1 --2- 172.21.15.43:0/3047440099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f126c102ef0 0x7f126c19ef10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1264004af0 tx=0x7f126402f3a0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.805+0000 7f12717ac640 1 -- 172.21.15.43:0/3047440099 shutdown_connections 2024-09-11T23:22:47.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.805+0000 7f12717ac640 1 --2- 172.21.15.43:0/3047440099 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f12540a2e90 0x7f12540a5350 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.805+0000 7f12717ac640 1 --2- 172.21.15.43:0/3047440099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f126c102ef0 0x7f126c19ef10 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.805+0000 7f12717ac640 1 --2- 172.21.15.43:0/3047440099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f126c102570 0x7f126c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.805+0000 7f12717ac640 1 --2- 172.21.15.43:0/3047440099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f126c100e20 0x7f126c19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:47.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.805+0000 7f12717ac640 1 -- 172.21.15.43:0/3047440099 >> 172.21.15.43:0/3047440099 conn(0x7f126c0fc7d0 msgr2=0x7f126c06a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:47.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.805+0000 7f12717ac640 1 -- 172.21.15.43:0/3047440099 shutdown_connections 2024-09-11T23:22:47.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:47.805+0000 7f12717ac640 1 -- 172.21.15.43:0/3047440099 wait complete. 2024-09-11T23:22:48.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:47 smithi043 ceph-mon[30950]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-11T23:22:48.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:47 smithi090 ceph-mon[35130]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-11T23:22:48.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:47 smithi097 ceph-mon[35562]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-11T23:22:48.715 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:22:49.054 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:22:49.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:48 smithi043 ceph-mon[30950]: scrub queued for path: / 2024-09-11T23:22:49.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:48 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:22:49.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:48 smithi043 ceph-mon[30950]: scrub summary: active paths [/] 2024-09-11T23:22:49.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:48 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:22:49.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:48 smithi090 ceph-mon[35130]: scrub queued for path: / 2024-09-11T23:22:49.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:48 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:22:49.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:48 smithi090 ceph-mon[35130]: scrub summary: active paths [/] 2024-09-11T23:22:49.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:48 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:22:49.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:48 smithi097 ceph-mon[35562]: scrub queued for path: / 2024-09-11T23:22:49.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:48 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:22:49.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:48 smithi097 ceph-mon[35562]: scrub summary: active paths [/] 2024-09-11T23:22:49.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:48 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:22:49.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.648+0000 7fde707f6640 1 -- 172.21.15.43:0/2402277431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde68100e20 msgr2=0x7fde68101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:49.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.648+0000 7fde707f6640 1 --2- 172.21.15.43:0/2402277431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde68100e20 0x7fde68101200 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fde64009900 tx=0x7fde6402a480 comp rx=0 tx=0).stop 2024-09-11T23:22:49.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.648+0000 7fde707f6640 1 -- 172.21.15.43:0/2402277431 shutdown_connections 2024-09-11T23:22:49.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.648+0000 7fde707f6640 1 --2- 172.21.15.43:0/2402277431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde68102ef0 0x7fde6810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.648+0000 7fde707f6640 1 --2- 172.21.15.43:0/2402277431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde68102570 0x7fde681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.648+0000 7fde707f6640 1 --2- 172.21.15.43:0/2402277431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde68100e20 0x7fde68101200 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.648+0000 7fde707f6640 1 -- 172.21.15.43:0/2402277431 >> 172.21.15.43:0/2402277431 conn(0x7fde680fc7d0 msgr2=0x7fde680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:49.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.648+0000 7fde707f6640 1 -- 172.21.15.43:0/2402277431 shutdown_connections 2024-09-11T23:22:49.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.648+0000 7fde707f6640 1 -- 172.21.15.43:0/2402277431 wait complete. 2024-09-11T23:22:49.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.649+0000 7fde707f6640 1 Processor -- start 2024-09-11T23:22:49.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.649+0000 7fde707f6640 1 -- start start 2024-09-11T23:22:49.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.650+0000 7fde707f6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde68100e20 0x7fde680fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:49.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.650+0000 7fde707f6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde68102570 0x7fde680fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:49.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.650+0000 7fde707f6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde68102ef0 0x7fde680fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:49.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.650+0000 7fde707f6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fde68110fa0 con 0x7fde68102570 2024-09-11T23:22:49.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.650+0000 7fde707f6640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fde68110e20 con 0x7fde68100e20 2024-09-11T23:22:49.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.650+0000 7fde707f6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fde68111120 con 0x7fde68102ef0 2024-09-11T23:22:49.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.650+0000 7fde6e592640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde68100e20 0x7fde680fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:49.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.650+0000 7fde6e592640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde68100e20 0x7fde680fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:44616/0 (socket says 172.21.15.43:44616) 2024-09-11T23:22:49.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.650+0000 7fde6e592640 1 -- 172.21.15.43:0/1447698608 learned_addr learned my addr 172.21.15.43:0/1447698608 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:49.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.650+0000 7fde6dd91640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde68102570 0x7fde680fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:49.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.650+0000 7fde6ed93640 1 --2- 172.21.15.43:0/1447698608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde68102ef0 0x7fde680fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:49.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.650+0000 7fde6e592640 1 -- 172.21.15.43:0/1447698608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde68102ef0 msgr2=0x7fde680fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:49.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.650+0000 7fde6e592640 1 --2- 172.21.15.43:0/1447698608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde68102ef0 0x7fde680fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.650+0000 7fde6e592640 1 -- 172.21.15.43:0/1447698608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde68102570 msgr2=0x7fde680fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:49.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.650+0000 7fde6e592640 1 --2- 172.21.15.43:0/1447698608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde68102570 0x7fde680fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.650+0000 7fde6e592640 1 -- 172.21.15.43:0/1447698608 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde681aaea0 con 0x7fde68100e20 2024-09-11T23:22:49.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.651+0000 7fde6e592640 1 --2- 172.21.15.43:0/1447698608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde68100e20 0x7fde680fcc80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fde6402aa70 tx=0x7fde64003b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:49.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.651+0000 7fde5f7fe640 1 -- 172.21.15.43:0/1447698608 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6403f070 con 0x7fde68100e20 2024-09-11T23:22:49.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.651+0000 7fde707f6640 1 -- 172.21.15.43:0/1447698608 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde681ab070 con 0x7fde68100e20 2024-09-11T23:22:49.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.651+0000 7fde5f7fe640 1 -- 172.21.15.43:0/1447698608 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde64031c20 con 0x7fde68100e20 2024-09-11T23:22:49.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.651+0000 7fde5f7fe640 1 -- 172.21.15.43:0/1447698608 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde64035030 con 0x7fde68100e20 2024-09-11T23:22:49.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.651+0000 7fde707f6640 1 -- 172.21.15.43:0/1447698608 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde681ab540 con 0x7fde68100e20 2024-09-11T23:22:49.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.654+0000 7fde5f7fe640 1 -- 172.21.15.43:0/1447698608 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fde64042070 con 0x7fde68100e20 2024-09-11T23:22:49.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.654+0000 7fde72bcf740 1 Processor -- start 2024-09-11T23:22:49.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.655+0000 7fde72bcf740 1 -- start start 2024-09-11T23:22:49.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.655+0000 7fde72bcf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d80e490 0x564e4d80e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:49.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.655+0000 7fde72bcf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e4d80edb0 0x564e4d819130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:49.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.655+0000 7fde72bcf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d819670 0x564e4d81ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:49.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.655+0000 7fde72bcf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564e4d7742e0 con 0x564e4d819670 2024-09-11T23:22:49.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.655+0000 7fde72bcf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564e4d71bce0 con 0x564e4d80e490 2024-09-11T23:22:49.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.655+0000 7fde72bcf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564e4d7702a0 con 0x564e4d80edb0 2024-09-11T23:22:49.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.656+0000 7fde6e592640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e4d80edb0 0x564e4d819130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:49.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.656+0000 7fde6ed93640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d80e490 0x564e4d80e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:49.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.656+0000 7fde6ed93640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d80e490 0x564e4d80e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:44620/0 (socket says 172.21.15.43:44620) 2024-09-11T23:22:49.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.656+0000 7fde6ed93640 1 -- 172.21.15.43:0/2412571193 learned_addr learned my addr 172.21.15.43:0/2412571193 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:49.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.656+0000 7fde6dd91640 1 --2- 172.21.15.43:0/2412571193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d819670 0x564e4d81ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:49.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.656+0000 7fde6ed93640 1 -- 172.21.15.43:0/2412571193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e4d80edb0 msgr2=0x564e4d819130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:49.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.656+0000 7fde6ed93640 1 --2- 172.21.15.43:0/2412571193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e4d80edb0 0x564e4d819130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.656+0000 7fde6ed93640 1 -- 172.21.15.43:0/2412571193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d819670 msgr2=0x564e4d81ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:49.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.656+0000 7fde6ed93640 1 --2- 172.21.15.43:0/2412571193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d819670 0x564e4d81ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.656+0000 7fde6ed93640 1 -- 172.21.15.43:0/2412571193 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e4d700440 con 0x564e4d80e490 2024-09-11T23:22:49.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.656+0000 7fde6dd91640 1 --2- 172.21.15.43:0/2412571193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d819670 0x564e4d81ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:49.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.656+0000 7fde5f7fe640 1 --2- 172.21.15.43:0/1447698608 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fde540a2e90 0x7fde540a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:49.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.657+0000 7fde6ed93640 1 --2- 172.21.15.43:0/2412571193 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d80e490 0x564e4d80e870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fde60009f70 tx=0x7fde60010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85dee37f40da1492 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:49.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.657+0000 7fde5cff9640 1 -- 172.21.15.43:0/2412571193 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60016070 con 0x564e4d80e490 2024-09-11T23:22:49.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.657+0000 7fde5cff9640 1 -- 172.21.15.43:0/2412571193 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde60004d20 con 0x564e4d80e490 2024-09-11T23:22:49.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.657+0000 7fde6e592640 1 --2- 172.21.15.43:0/1447698608 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fde540a2e90 0x7fde540a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:49.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.658+0000 7fde5cff9640 1 -- 172.21.15.43:0/2412571193 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60012950 con 0x564e4d80e490 2024-09-11T23:22:49.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.658+0000 7fde72bcf740 1 -- 172.21.15.43:0/2412571193 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d80e490 msgr2=0x564e4d80e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:49.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.658+0000 7fde72bcf740 1 --2- 172.21.15.43:0/2412571193 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d80e490 0x564e4d80e870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fde60009f70 tx=0x7fde60010800 comp rx=0 tx=0).stop 2024-09-11T23:22:49.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.658+0000 7fde5f7fe640 1 -- 172.21.15.43:0/1447698608 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fde640efcf0 con 0x7fde68100e20 2024-09-11T23:22:49.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.658+0000 7fde6e592640 1 --2- 172.21.15.43:0/1447698608 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fde540a2e90 0x7fde540a5350 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fde640ed320 tx=0x7fde640f3530 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:49.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.658+0000 7fde72bcf740 1 -- 172.21.15.43:0/2412571193 shutdown_connections 2024-09-11T23:22:49.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.658+0000 7fde72bcf740 1 --2- 172.21.15.43:0/2412571193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d819670 0x564e4d81ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.658+0000 7fde72bcf740 1 --2- 172.21.15.43:0/2412571193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e4d80edb0 0x564e4d819130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.658+0000 7fde72bcf740 1 --2- 172.21.15.43:0/2412571193 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d80e490 0x564e4d80e870 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.658+0000 7fde72bcf740 1 -- 172.21.15.43:0/2412571193 >> 172.21.15.43:0/2412571193 conn(0x564e4d80aa20 msgr2=0x564e4d80ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:49.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.658+0000 7fde72bcf740 1 -- 172.21.15.43:0/2412571193 shutdown_connections 2024-09-11T23:22:49.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.658+0000 7fde72bcf740 1 -- 172.21.15.43:0/2412571193 wait complete. 2024-09-11T23:22:49.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.659+0000 7fde72bcf740 1 -- start start 2024-09-11T23:22:49.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.659+0000 7fde72bcf740 1 Processor -- start 2024-09-11T23:22:49.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.659+0000 7fde72bcf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d80e490 0x564e4d818ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:49.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.660+0000 7fde72bcf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d80edb0 0x564e4d8161a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:49.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.660+0000 7fde6ed93640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d80e490 0x564e4d818ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:49.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.660+0000 7fde6ed93640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d80e490 0x564e4d818ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:44632/0 (socket says 172.21.15.43:44632) 2024-09-11T23:22:49.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.660+0000 7fde6ed93640 1 -- 172.21.15.43:0/487734256 learned_addr learned my addr 172.21.15.43:0/487734256 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:49.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.660+0000 7fde72bcf740 1 --2- 172.21.15.43:0/487734256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e4d819670 0x564e4d8166e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:49.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.660+0000 7fde6dd91640 1 --2- 172.21.15.43:0/487734256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d80edb0 0x564e4d8161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:49.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.660+0000 7fde72bcf740 1 -- 172.21.15.43:0/487734256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564e4d81c2e0 con 0x564e4d80edb0 2024-09-11T23:22:49.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.660+0000 7fde72bcf740 1 -- 172.21.15.43:0/487734256 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564e4d72f050 con 0x564e4d80e490 2024-09-11T23:22:49.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.660+0000 7fde72bcf740 1 -- 172.21.15.43:0/487734256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564e4d81c460 con 0x564e4d819670 2024-09-11T23:22:49.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.660+0000 7fde72bcf740 10 client.0 _ll_register_callbacks cb 0x564e4d76f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:49.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.660+0000 7fde6ed93640 1 -- 172.21.15.43:0/487734256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e4d819670 msgr2=0x564e4d8166e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:49.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.660+0000 7fde6ed93640 1 --2- 172.21.15.43:0/487734256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e4d819670 0x564e4d8166e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.660+0000 7fde6ed93640 1 -- 172.21.15.43:0/487734256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d80edb0 msgr2=0x564e4d8161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:49.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.660+0000 7fde6ed93640 1 --2- 172.21.15.43:0/487734256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d80edb0 0x564e4d8161a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.660+0000 7fde6ed93640 1 -- 172.21.15.43:0/487734256 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e4d4e53c0 con 0x564e4d80e490 2024-09-11T23:22:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.660+0000 7fde6dd91640 1 --2- 172.21.15.43:0/487734256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d80edb0 0x564e4d8161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.661+0000 7fde6ed93640 1 --2- 172.21.15.43:0/487734256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d80e490 0x564e4d818ea0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fde60005100 tx=0x7fde60005130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c3dcbf9493f07d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.661+0000 7fde51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:22:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.661+0000 7fde51ffb640 1 -- 172.21.15.43:0/487734256 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60016030 con 0x564e4d80e490 2024-09-11T23:22:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.661+0000 7fde72bcf740 1 -- 172.21.15.43:0/487734256 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde60004d20 con 0x564e4d80e490 2024-09-11T23:22:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.661+0000 7fde51ffb640 1 -- 172.21.15.43:0/487734256 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fde600054e0 con 0x564e4d80e490 2024-09-11T23:22:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.661+0000 7fde51ffb640 1 -- 172.21.15.43:0/487734256 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fde60009b70 con 0x564e4d80e490 2024-09-11T23:22:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.661+0000 7fde72bcf740 10 client.25208 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:49.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.662+0000 7fde72bcf740 1 -- 172.21.15.43:0/487734256 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564e4d827c60 con 0x564e4d80e490 2024-09-11T23:22:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.662+0000 7fde51ffb640 1 -- 172.21.15.43:0/487734256 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde60004d20 con 0x564e4d80e490 2024-09-11T23:22:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.663+0000 7fde51ffb640 1 -- 172.21.15.43:0/487734256 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fde6000f040 con 0x564e4d80e490 2024-09-11T23:22:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.663+0000 7fde72bcf740 10 client.25208 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.663+0000 7fde72bcf740 10 client.25208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.663+0000 7fde72bcf740 20 client.25208 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.663+0000 7fde72bcf740 1 --2- 172.21.15.43:0/487734256 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564e4d8294e0 0x564e4d82b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.663+0000 7fde72bcf740 4 client.25208 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:22:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.663+0000 7fde72bcf740 1 -- 172.21.15.43:0/487734256 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e4d781850 con 0x564e4d8294e0 2024-09-11T23:22:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.664+0000 7fde6dd91640 1 --2- 172.21.15.43:0/487734256 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564e4d8294e0 0x564e4d82b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.664+0000 7fde51ffb640 10 client.25208 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:49.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.664+0000 7fde6dd91640 1 --2- 172.21.15.43:0/487734256 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564e4d8294e0 0x564e4d82b960 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3261c69018dfcc9a server_cookie=634443a7205bb157 in_seq=0 out_seq=0 2024-09-11T23:22:49.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.665+0000 7fde51ffb640 1 -- 172.21.15.43:0/487734256 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e4d781850 con 0x564e4d8294e0 2024-09-11T23:22:49.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.665+0000 7fde51ffb640 10 client.25208 handle_command_reply: tid=0 2024-09-11T23:22:49.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.665+0000 7fde72bcf740 1 client.25208 shutdown 2024-09-11T23:22:49.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.665+0000 7fde72bcf740 1 -- 172.21.15.43:0/487734256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d80e490 msgr2=0x564e4d818ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:49.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.665+0000 7fde72bcf740 1 --2- 172.21.15.43:0/487734256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d80e490 0x564e4d818ea0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fde60005100 tx=0x7fde60005130 comp rx=0 tx=0).stop 2024-09-11T23:22:49.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.665+0000 7fde72bcf740 1 -- 172.21.15.43:0/487734256 shutdown_connections 2024-09-11T23:22:49.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.665+0000 7fde72bcf740 1 --2- 172.21.15.43:0/487734256 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564e4d8294e0 0x564e4d82b960 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.665+0000 7fde72bcf740 1 --2- 172.21.15.43:0/487734256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e4d819670 0x564e4d8166e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.665+0000 7fde72bcf740 1 --2- 172.21.15.43:0/487734256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d80edb0 0x564e4d8161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.665+0000 7fde72bcf740 1 --2- 172.21.15.43:0/487734256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d80e490 0x564e4d818ea0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.665+0000 7fde72bcf740 1 -- 172.21.15.43:0/487734256 >> 172.21.15.43:0/487734256 conn(0x564e4d80aa20 msgr2=0x564e4d809ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:49.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.665+0000 7fde51ffb640 0 client.25208 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:49.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.666+0000 7fde72bcf740 1 -- 172.21.15.43:0/487734256 shutdown_connections 2024-09-11T23:22:49.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.666+0000 7fde72bcf740 1 -- 172.21.15.43:0/487734256 wait complete. 2024-09-11T23:22:49.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.666+0000 7fde72bcf740 20 client.25208 trim_cache size 0 max 16384 2024-09-11T23:22:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.700+0000 7fde72bcf740 1 Processor -- start 2024-09-11T23:22:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.700+0000 7fde72bcf740 1 -- start start 2024-09-11T23:22:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.700+0000 7fde72bcf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e4d80e490 0x564e4d831230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.700+0000 7fde72bcf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d80edb0 0x564e4d831770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.700+0000 7fde72bcf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d819670 0x564e4d821720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.700+0000 7fde72bcf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fde681b6660 con 0x564e4d819670 2024-09-11T23:22:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.700+0000 7fde72bcf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fde681b3f30 con 0x564e4d80edb0 2024-09-11T23:22:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.700+0000 7fde72bcf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fde681b1e30 con 0x564e4d80e490 2024-09-11T23:22:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.701+0000 7fde6ed93640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d819670 0x564e4d821720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.701+0000 7fde6ed93640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d819670 0x564e4d821720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54140/0 (socket says 172.21.15.43:54140) 2024-09-11T23:22:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.701+0000 7fde6ed93640 1 -- 172.21.15.43:0/154533597 learned_addr learned my addr 172.21.15.43:0/154533597 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.701+0000 7fde6ed93640 1 --2- 172.21.15.43:0/154533597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e4d80e490 0x564e4d831230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.701+0000 7fde6ed93640 1 -- 172.21.15.43:0/154533597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e4d80e490 msgr2=0x564e4d831230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.701+0000 7fde6ed93640 1 --2- 172.21.15.43:0/154533597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e4d80e490 0x564e4d831230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.701+0000 7fde6ed93640 1 -- 172.21.15.43:0/154533597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d80edb0 msgr2=0x564e4d831770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:22:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.701+0000 7fde6ed93640 1 --2- 172.21.15.43:0/154533597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d80edb0 0x564e4d831770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.701+0000 7fde6ed93640 1 -- 172.21.15.43:0/154533597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e4d80a850 con 0x564e4d819670 2024-09-11T23:22:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.701+0000 7fde6ed93640 1 --2- 172.21.15.43:0/154533597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d819670 0x564e4d821720 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fde60009a80 tx=0x7fde600021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9802d3bb632c8e2b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.702+0000 7fde53fff640 1 -- 172.21.15.43:0/154533597 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60016040 con 0x564e4d819670 2024-09-11T23:22:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.702+0000 7fde53fff640 1 -- 172.21.15.43:0/154533597 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde60012610 con 0x564e4d819670 2024-09-11T23:22:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.702+0000 7fde53fff640 1 -- 172.21.15.43:0/154533597 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde600258c0 con 0x564e4d819670 2024-09-11T23:22:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.703+0000 7fde72bcf740 1 -- 172.21.15.43:0/154533597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d819670 msgr2=0x564e4d821720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.703+0000 7fde72bcf740 1 --2- 172.21.15.43:0/154533597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d819670 0x564e4d821720 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fde60009a80 tx=0x7fde600021f0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.703+0000 7fde72bcf740 1 -- 172.21.15.43:0/154533597 shutdown_connections 2024-09-11T23:22:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.703+0000 7fde72bcf740 1 --2- 172.21.15.43:0/154533597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d819670 0x564e4d821720 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.703+0000 7fde72bcf740 1 --2- 172.21.15.43:0/154533597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d80edb0 0x564e4d831770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.703+0000 7fde72bcf740 1 --2- 172.21.15.43:0/154533597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e4d80e490 0x564e4d831230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.703+0000 7fde72bcf740 1 -- 172.21.15.43:0/154533597 >> 172.21.15.43:0/154533597 conn(0x564e4d80aa20 msgr2=0x564e4d809ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.703+0000 7fde72bcf740 1 -- 172.21.15.43:0/154533597 shutdown_connections 2024-09-11T23:22:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.703+0000 7fde72bcf740 1 -- 172.21.15.43:0/154533597 wait complete. 2024-09-11T23:22:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.704+0000 7fde72bcf740 1 -- start start 2024-09-11T23:22:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.704+0000 7fde72bcf740 1 Processor -- start 2024-09-11T23:22:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.704+0000 7fde72bcf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e4d80e490 0x564e4d81e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.704+0000 7fde72bcf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d80edb0 0x564e4d81eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.705+0000 7fde72bcf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d819670 0x564e4d81eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.705+0000 7fde72bcf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fde681b23f0 con 0x564e4d80edb0 2024-09-11T23:22:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.705+0000 7fde72bcf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fde681b3860 con 0x564e4d819670 2024-09-11T23:22:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.705+0000 7fde72bcf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fde681b2ac0 con 0x564e4d80e490 2024-09-11T23:22:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.705+0000 7fde72bcf740 10 client.0 _ll_register_callbacks cb 0x7fde5800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.705+0000 7fde6ed93640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e4d80e490 0x564e4d81e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.705+0000 7fde6ed93640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e4d80e490 0x564e4d81e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:54198/0 (socket says 172.21.15.43:54198) 2024-09-11T23:22:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.705+0000 7fde6ed93640 1 -- 172.21.15.43:0/3806775354 learned_addr learned my addr 172.21.15.43:0/3806775354 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.705+0000 7fde6ed93640 1 --2- 172.21.15.43:0/3806775354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d819670 0x564e4d81eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.705+0000 7fde6dd91640 1 --2- 172.21.15.43:0/3806775354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d80edb0 0x564e4d81eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.705+0000 7fde6ed93640 1 -- 172.21.15.43:0/3806775354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d819670 msgr2=0x564e4d81eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.705+0000 7fde6ed93640 1 --2- 172.21.15.43:0/3806775354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d819670 0x564e4d81eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.705+0000 7fde6ed93640 1 -- 172.21.15.43:0/3806775354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d80edb0 msgr2=0x564e4d81eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.705+0000 7fde6ed93640 1 --2- 172.21.15.43:0/3806775354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d80edb0 0x564e4d81eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.705+0000 7fde6ed93640 1 -- 172.21.15.43:0/3806775354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e4d8922d0 con 0x564e4d80e490 2024-09-11T23:22:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.706+0000 7fde6ed93640 1 --2- 172.21.15.43:0/3806775354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e4d80e490 0x564e4d81e570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fde60000f80 tx=0x7fde60025420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd2d2fb98e7f6d21 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.706+0000 7fde51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-11T23:22:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.706+0000 7fde51ffb640 1 -- 172.21.15.43:0/3806775354 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60016040 con 0x564e4d80e490 2024-09-11T23:22:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.706+0000 7fde51ffb640 1 -- 172.21.15.43:0/3806775354 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fde60014e30 con 0x564e4d80e490 2024-09-11T23:22:49.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.706+0000 7fde72bcf740 1 -- 172.21.15.43:0/3806775354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde60012610 con 0x564e4d80e490 2024-09-11T23:22:49.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.706+0000 7fde51ffb640 1 -- 172.21.15.43:0/3806775354 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fde60005750 con 0x564e4d80e490 2024-09-11T23:22:49.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.706+0000 7fde72bcf740 10 client.25216 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:49.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.706+0000 7fde72bcf740 1 -- 172.21.15.43:0/3806775354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564e4d8935c0 con 0x564e4d80e490 2024-09-11T23:22:49.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.707+0000 7fde51ffb640 1 -- 172.21.15.43:0/3806775354 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde60012610 con 0x564e4d80e490 2024-09-11T23:22:49.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.707+0000 7fde51ffb640 1 -- 172.21.15.43:0/3806775354 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fde60036040 con 0x564e4d80e490 2024-09-11T23:22:49.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.707+0000 7fde72bcf740 10 client.25216 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:49.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.707+0000 7fde72bcf740 10 client.25216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:49.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.707+0000 7fde72bcf740 20 client.25216 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:49.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.707+0000 7fde72bcf740 1 --2- 172.21.15.43:0/3806775354 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564e4d893b20 0x564e4d895f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:49.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.707+0000 7fde72bcf740 4 client.25216 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:22:49.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.707+0000 7fde72bcf740 1 -- 172.21.15.43:0/3806775354 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564e4d77bd60 con 0x564e4d893b20 2024-09-11T23:22:49.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.707+0000 7fde6dd91640 1 --2- 172.21.15.43:0/3806775354 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564e4d893b20 0x564e4d895f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:49.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.708+0000 7fde6dd91640 1 --2- 172.21.15.43:0/3806775354 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564e4d893b20 0x564e4d895f40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de035da1e5e66b0c server_cookie=bb7f610c617e40a1 in_seq=0 out_seq=0 2024-09-11T23:22:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.708+0000 7fde51ffb640 10 client.25216 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.708+0000 7fde51ffb640 1 -- 172.21.15.43:0/3806775354 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564e4d77bd60 con 0x564e4d893b20 2024-09-11T23:22:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.708+0000 7fde51ffb640 10 client.25216 handle_command_reply: tid=0 2024-09-11T23:22:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.708+0000 7fde72bcf740 1 client.25216 shutdown 2024-09-11T23:22:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.708+0000 7fde72bcf740 1 -- 172.21.15.43:0/3806775354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e4d80e490 msgr2=0x564e4d81e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.708+0000 7fde72bcf740 1 --2- 172.21.15.43:0/3806775354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e4d80e490 0x564e4d81e570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fde60000f80 tx=0x7fde60025420 comp rx=0 tx=0).stop 2024-09-11T23:22:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.708+0000 7fde72bcf740 1 -- 172.21.15.43:0/3806775354 shutdown_connections 2024-09-11T23:22:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.708+0000 7fde72bcf740 1 --2- 172.21.15.43:0/3806775354 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564e4d893b20 0x564e4d895f40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.709+0000 7fde51ffb640 0 client.25216 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.708+0000 7fde72bcf740 1 --2- 172.21.15.43:0/3806775354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564e4d819670 0x564e4d81eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.709+0000 7fde72bcf740 1 --2- 172.21.15.43:0/3806775354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564e4d80edb0 0x564e4d81eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.709+0000 7fde72bcf740 1 --2- 172.21.15.43:0/3806775354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e4d80e490 0x564e4d81e570 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.709+0000 7fde72bcf740 1 -- 172.21.15.43:0/3806775354 >> 172.21.15.43:0/3806775354 conn(0x564e4d80aa20 msgr2=0x564e4d815a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.709+0000 7fde72bcf740 1 -- 172.21.15.43:0/3806775354 shutdown_connections 2024-09-11T23:22:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.709+0000 7fde72bcf740 1 -- 172.21.15.43:0/3806775354 wait complete. 2024-09-11T23:22:49.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.709+0000 7fde72bcf740 20 client.25216 trim_cache size 0 max 16384 2024-09-11T23:22:49.713 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:22:49.713 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:22:49.713 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-09-11T23:22:49.714 INFO:teuthology.orchestra.run.smithi043.stdout: "c4de2c2c-6e2a-4b5d-8461-42fe147e3305": { 2024-09-11T23:22:49.714 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-09-11T23:22:49.714 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "c4de2c2c-6e2a-4b5d-8461-42fe147e3305", 2024-09-11T23:22:49.714 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-09-11T23:22:49.714 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:22:49.714 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:22:49.714 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:22:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.710+0000 7fde707f6640 1 -- 172.21.15.43:0/1447698608 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fde540a2e90 msgr2=0x7fde540a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.710+0000 7fde707f6640 1 --2- 172.21.15.43:0/1447698608 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fde540a2e90 0x7fde540a5350 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fde640ed320 tx=0x7fde640f3530 comp rx=0 tx=0).stop 2024-09-11T23:22:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.710+0000 7fde707f6640 1 -- 172.21.15.43:0/1447698608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde68100e20 msgr2=0x7fde680fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.710+0000 7fde707f6640 1 --2- 172.21.15.43:0/1447698608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde68100e20 0x7fde680fcc80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fde6402aa70 tx=0x7fde64003b60 comp rx=0 tx=0).stop 2024-09-11T23:22:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.710+0000 7fde707f6640 1 -- 172.21.15.43:0/1447698608 shutdown_connections 2024-09-11T23:22:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.710+0000 7fde707f6640 1 --2- 172.21.15.43:0/1447698608 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fde540a2e90 0x7fde540a5350 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.710+0000 7fde707f6640 1 --2- 172.21.15.43:0/1447698608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde68102ef0 0x7fde680fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.710+0000 7fde707f6640 1 --2- 172.21.15.43:0/1447698608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde68102570 0x7fde680fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.710+0000 7fde707f6640 1 --2- 172.21.15.43:0/1447698608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde68100e20 0x7fde680fcc80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.710+0000 7fde707f6640 1 -- 172.21.15.43:0/1447698608 >> 172.21.15.43:0/1447698608 conn(0x7fde680fc7d0 msgr2=0x7fde6806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.710+0000 7fde707f6640 1 -- 172.21.15.43:0/1447698608 shutdown_connections 2024-09-11T23:22:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:22:49.710+0000 7fde707f6640 1 -- 172.21.15.43:0/1447698608 wait complete. 2024-09-11T23:22:49.835 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:49 smithi043 ceph-mon[30950]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-11T23:22:49.835 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:49 smithi043 ceph-mon[30950]: scrub summary: idle 2024-09-11T23:22:50.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:49 smithi090 ceph-mon[35130]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-11T23:22:50.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:49 smithi090 ceph-mon[35130]: scrub summary: idle 2024-09-11T23:22:50.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:49 smithi097 ceph-mon[35562]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-11T23:22:50.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:49 smithi097 ceph-mon[35562]: scrub summary: idle 2024-09-11T23:22:50.558 INFO:tasks.cephfs.filesystem:scrub status for tag:c4de2c2c-6e2a-4b5d-8461-42fe147e3305 - {'path': '/', 'tag': 'c4de2c2c-6e2a-4b5d-8461-42fe147e3305', 'options': 'recursive,force'} 2024-09-11T23:22:52.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:51 smithi043 ceph-mon[30950]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-11T23:22:52.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:51 smithi090 ceph-mon[35130]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-11T23:22:52.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:51 smithi097 ceph-mon[35562]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-11T23:22:54.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:53 smithi043 ceph-mon[30950]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 9 op/s 2024-09-11T23:22:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.252+0000 7f6de70dd640 1 -- 172.21.15.43:0/3530028968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de01009a0 msgr2=0x7f6de0100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.252+0000 7f6de70dd640 1 --2- 172.21.15.43:0/3530028968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de01009a0 0x7f6de0100e00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009930 tx=0x7f6dcc02e830 comp rx=0 tx=0).stop 2024-09-11T23:22:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.253+0000 7f6de70dd640 1 -- 172.21.15.43:0/3530028968 shutdown_connections 2024-09-11T23:22:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.253+0000 7f6de70dd640 1 --2- 172.21.15.43:0/3530028968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6de0101340 0x7f6de010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.253+0000 7f6de70dd640 1 --2- 172.21.15.43:0/3530028968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de01009a0 0x7f6de0100e00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.253+0000 7f6de70dd640 1 --2- 172.21.15.43:0/3530028968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6de01069b0 0x7f6de0106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.253+0000 7f6de70dd640 1 -- 172.21.15.43:0/3530028968 >> 172.21.15.43:0/3530028968 conn(0x7f6de00779a0 msgr2=0x7f6de00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.253+0000 7f6de70dd640 1 -- 172.21.15.43:0/3530028968 shutdown_connections 2024-09-11T23:22:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.253+0000 7f6de70dd640 1 -- 172.21.15.43:0/3530028968 wait complete. 2024-09-11T23:22:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.253+0000 7f6de70dd640 1 Processor -- start 2024-09-11T23:22:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.254+0000 7f6de70dd640 1 -- start start 2024-09-11T23:22:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.254+0000 7f6de70dd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6de01009a0 0x7f6de0196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.254+0000 7f6de70dd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6de0101340 0x7f6de01967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.254+0000 7f6de70dd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de01069b0 0x7f6de019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.254+0000 7f6de70dd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6de010fd20 con 0x7f6de01069b0 2024-09-11T23:22:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.254+0000 7f6de70dd640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6de010fba0 con 0x7f6de01009a0 2024-09-11T23:22:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.254+0000 7f6de70dd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6de010fea0 con 0x7f6de0101340 2024-09-11T23:22:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.254+0000 7f6de567a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de01069b0 0x7f6de019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.254+0000 7f6de567a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de01069b0 0x7f6de019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36770/0 (socket says 172.21.15.43:36770) 2024-09-11T23:22:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.254+0000 7f6de567a640 1 -- 172.21.15.43:0/884295314 learned_addr learned my addr 172.21.15.43:0/884295314 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.254+0000 7f6de4e79640 1 --2- 172.21.15.43:0/884295314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6de01009a0 0x7f6de0196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.254+0000 7f6ddffff640 1 --2- 172.21.15.43:0/884295314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6de0101340 0x7f6de01967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.254+0000 7f6de567a640 1 -- 172.21.15.43:0/884295314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6de0101340 msgr2=0x7f6de01967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.255+0000 7f6de567a640 1 --2- 172.21.15.43:0/884295314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6de0101340 0x7f6de01967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.255+0000 7f6de567a640 1 -- 172.21.15.43:0/884295314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6de01009a0 msgr2=0x7f6de0196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.255+0000 7f6de567a640 1 --2- 172.21.15.43:0/884295314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6de01009a0 0x7f6de0196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.255+0000 7f6de567a640 1 -- 172.21.15.43:0/884295314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de019b280 con 0x7f6de01069b0 2024-09-11T23:22:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.255+0000 7f6ddffff640 1 --2- 172.21.15.43:0/884295314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6de0101340 0x7f6de01967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:22:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.255+0000 7f6de567a640 1 --2- 172.21.15.43:0/884295314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de01069b0 0x7f6de019ab50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800eb10 tx=0x7f6dd800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.255+0000 7f6dddffb640 1 -- 172.21.15.43:0/884295314 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800ce20 con 0x7f6de01069b0 2024-09-11T23:22:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.255+0000 7f6dddffb640 1 -- 172.21.15.43:0/884295314 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd8005020 con 0x7f6de01069b0 2024-09-11T23:22:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.255+0000 7f6de70dd640 1 -- 172.21.15.43:0/884295314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de019b570 con 0x7f6de01069b0 2024-09-11T23:22:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.255+0000 7f6dddffb640 1 -- 172.21.15.43:0/884295314 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8010720 con 0x7f6de01069b0 2024-09-11T23:22:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.256+0000 7f6de70dd640 1 -- 172.21.15.43:0/884295314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de01a2e00 con 0x7f6de01069b0 2024-09-11T23:22:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.259+0000 7f6dddffb640 1 -- 172.21.15.43:0/884295314 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f6dd8005540 con 0x7f6de01069b0 2024-09-11T23:22:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.259+0000 7f6de9e7b740 1 Processor -- start 2024-09-11T23:22:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.260+0000 7f6de9e7b740 1 -- start start 2024-09-11T23:22:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.260+0000 7f6de9e7b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd0aaf0 0x55c07dd0aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.260+0000 7f6de9e7b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c07dd0b410 0x55c07dd15770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.260+0000 7f6de9e7b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c07dd15cb0 0x55c07dd180a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.260+0000 7f6de9e7b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c07dc6f2f0 con 0x55c07dd0aaf0 2024-09-11T23:22:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.260+0000 7f6de9e7b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c07dc17270 con 0x55c07dd0b410 2024-09-11T23:22:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.260+0000 7f6de9e7b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c07dc6b2b0 con 0x55c07dd15cb0 2024-09-11T23:22:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.260+0000 7f6ddffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd0aaf0 0x55c07dd0aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.260+0000 7f6ddffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd0aaf0 0x55c07dd0aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36786/0 (socket says 172.21.15.43:36786) 2024-09-11T23:22:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.260+0000 7f6ddffff640 1 -- 172.21.15.43:0/2495953470 learned_addr learned my addr 172.21.15.43:0/2495953470 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.260+0000 7f6dddffb640 1 --2- 172.21.15.43:0/884295314 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6dd00a2e10 0x7f6dd00a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.260+0000 7f6de4e79640 1 --2- 172.21.15.43:0/2495953470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c07dd0b410 0x55c07dd15770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.260+0000 7f6de567a640 1 --2- 172.21.15.43:0/884295314 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6dd00a2e10 0x7f6dd00a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.260+0000 7f6ddffff640 1 -- 172.21.15.43:0/2495953470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c07dd15cb0 msgr2=0x55c07dd180a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.260+0000 7f6ddffff640 1 --2- 172.21.15.43:0/2495953470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c07dd15cb0 0x55c07dd180a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.260+0000 7f6ddffff640 1 -- 172.21.15.43:0/2495953470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c07dd0b410 msgr2=0x55c07dd15770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.260+0000 7f6ddffff640 1 --2- 172.21.15.43:0/2495953470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c07dd0b410 0x55c07dd15770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.260+0000 7f6dddffb640 1 -- 172.21.15.43:0/884295314 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f6dd80cfdc0 con 0x7f6de01069b0 2024-09-11T23:22:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.260+0000 7f6ddffff640 1 -- 172.21.15.43:0/2495953470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c07dc16340 con 0x55c07dd0aaf0 2024-09-11T23:22:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.261+0000 7f6de4e79640 1 --2- 172.21.15.43:0/2495953470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c07dd0b410 0x55c07dd15770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:22:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.261+0000 7f6ddffff640 1 --2- 172.21.15.43:0/2495953470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd0aaf0 0x55c07dd0aed0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc005ec0 tx=0x7f6dcc037510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8eb5ec5c58b300c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.261+0000 7f6de567a640 1 --2- 172.21.15.43:0/884295314 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6dd00a2e10 0x7f6dd00a52d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80cd3f0 tx=0x7f6dd8014d30 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.261+0000 7f6dcaffd640 1 -- 172.21.15.43:0/2495953470 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc046070 con 0x55c07dd0aaf0 2024-09-11T23:22:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.261+0000 7f6dcaffd640 1 -- 172.21.15.43:0/2495953470 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dcc004240 con 0x55c07dd0aaf0 2024-09-11T23:22:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.261+0000 7f6dcaffd640 1 -- 172.21.15.43:0/2495953470 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc004920 con 0x55c07dd0aaf0 2024-09-11T23:22:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.262+0000 7f6de9e7b740 1 -- 172.21.15.43:0/2495953470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd0aaf0 msgr2=0x55c07dd0aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.262+0000 7f6de9e7b740 1 --2- 172.21.15.43:0/2495953470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd0aaf0 0x55c07dd0aed0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc005ec0 tx=0x7f6dcc037510 comp rx=0 tx=0).stop 2024-09-11T23:22:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.262+0000 7f6de9e7b740 1 -- 172.21.15.43:0/2495953470 shutdown_connections 2024-09-11T23:22:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.262+0000 7f6de9e7b740 1 --2- 172.21.15.43:0/2495953470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c07dd15cb0 0x55c07dd180a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.262+0000 7f6de9e7b740 1 --2- 172.21.15.43:0/2495953470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c07dd0b410 0x55c07dd15770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.262+0000 7f6de9e7b740 1 --2- 172.21.15.43:0/2495953470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd0aaf0 0x55c07dd0aed0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.262+0000 7f6de9e7b740 1 -- 172.21.15.43:0/2495953470 >> 172.21.15.43:0/2495953470 conn(0x55c07dd07060 msgr2=0x55c07dd07440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.262+0000 7f6de9e7b740 1 -- 172.21.15.43:0/2495953470 shutdown_connections 2024-09-11T23:22:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.262+0000 7f6de9e7b740 1 -- 172.21.15.43:0/2495953470 wait complete. 2024-09-11T23:22:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.263+0000 7f6de9e7b740 1 -- start start 2024-09-11T23:22:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.263+0000 7f6de9e7b740 1 Processor -- start 2024-09-11T23:22:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.263+0000 7f6de9e7b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c07dd0aaf0 0x55c07dd12b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.263+0000 7f6de9e7b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c07dd0b410 0x55c07dd130d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.263+0000 7f6de9e7b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd15cb0 0x55c07dd13610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.264+0000 7f6de9e7b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c07dd18880 con 0x55c07dd15cb0 2024-09-11T23:22:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.264+0000 7f6de9e7b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c07dbf8290 con 0x55c07dd0aaf0 2024-09-11T23:22:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.264+0000 7f6de9e7b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c07dd18a00 con 0x55c07dd0b410 2024-09-11T23:22:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.264+0000 7f6de9e7b740 10 client.0 _ll_register_callbacks cb 0x55c07dc29140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.264+0000 7f6ddffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd15cb0 0x55c07dd13610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.264+0000 7f6ddffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd15cb0 0x55c07dd13610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36794/0 (socket says 172.21.15.43:36794) 2024-09-11T23:22:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.264+0000 7f6ddffff640 1 -- 172.21.15.43:0/3471074963 learned_addr learned my addr 172.21.15.43:0/3471074963 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.264+0000 7f6de4e79640 1 --2- 172.21.15.43:0/3471074963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c07dd0b410 0x55c07dd130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.264+0000 7f6ddffff640 1 -- 172.21.15.43:0/3471074963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c07dd0b410 msgr2=0x55c07dd130d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.264+0000 7f6ddffff640 1 --2- 172.21.15.43:0/3471074963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c07dd0b410 0x55c07dd130d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.264+0000 7f6ddffff640 1 -- 172.21.15.43:0/3471074963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c07dd0aaf0 msgr2=0x55c07dd12b90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:22:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.264+0000 7f6ddffff640 1 --2- 172.21.15.43:0/3471074963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c07dd0aaf0 0x55c07dd12b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.264+0000 7f6ddffff640 1 -- 172.21.15.43:0/3471074963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c07d982210 con 0x55c07dd15cb0 2024-09-11T23:22:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.265+0000 7f6ddffff640 1 --2- 172.21.15.43:0/3471074963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd15cb0 0x55c07dd13610 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc02ed50 tx=0x7f6dcc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4f2ce726de86b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.265+0000 7f6dbffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.265+0000 7f6dbffff640 1 -- 172.21.15.43:0/3471074963 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc046040 con 0x55c07dd15cb0 2024-09-11T23:22:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.265+0000 7f6dbffff640 1 -- 172.21.15.43:0/3471074963 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f6dcc037730 con 0x55c07dd15cb0 2024-09-11T23:22:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.265+0000 7f6de9e7b740 1 -- 172.21.15.43:0/3471074963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dcc004240 con 0x55c07dd15cb0 2024-09-11T23:22:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.265+0000 7f6dbffff640 1 -- 172.21.15.43:0/3471074963 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6dcc037930 con 0x55c07dd15cb0 2024-09-11T23:22:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.265+0000 7f6de9e7b740 10 client.15627 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.265+0000 7f6de9e7b740 1 -- 172.21.15.43:0/3471074963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c07da4e620 con 0x55c07dd15cb0 2024-09-11T23:22:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.266+0000 7f6dbffff640 1 -- 172.21.15.43:0/3471074963 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dcc004240 con 0x55c07dd15cb0 2024-09-11T23:22:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.266+0000 7f6dbffff640 1 -- 172.21.15.43:0/3471074963 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f6dcc003790 con 0x55c07dd15cb0 2024-09-11T23:22:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.266+0000 7f6de9e7b740 10 client.15627 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.266+0000 7f6de9e7b740 10 client.15627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.266+0000 7f6de9e7b740 20 client.15627 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.266+0000 7f6de9e7b740 1 --2- 172.21.15.43:0/3471074963 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c07dd25a40 0x55c07dd27e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.266+0000 7f6de9e7b740 4 client.15627 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:22:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.266+0000 7f6de9e7b740 1 -- 172.21.15.43:0/3471074963 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c07dc7ca20 con 0x55c07dd25a40 2024-09-11T23:22:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.266+0000 7f6de4e79640 1 --2- 172.21.15.43:0/3471074963 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c07dd25a40 0x55c07dd27e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.267+0000 7f6de4e79640 1 --2- 172.21.15.43:0/3471074963 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c07dd25a40 0x55c07dd27e30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d0b20dcd70a5f64 server_cookie=a4a87312cb9670ef in_seq=0 out_seq=0 2024-09-11T23:22:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.267+0000 7f6dbffff640 10 client.15627 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.268+0000 7f6dbffff640 1 -- 172.21.15.43:0/3471074963 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c07dc7ca20 con 0x55c07dd25a40 2024-09-11T23:22:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.268+0000 7f6dbffff640 10 client.15627 handle_command_reply: tid=0 2024-09-11T23:22:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.268+0000 7f6de9e7b740 1 client.15627 shutdown 2024-09-11T23:22:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.268+0000 7f6de9e7b740 1 -- 172.21.15.43:0/3471074963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd15cb0 msgr2=0x55c07dd13610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.268+0000 7f6de9e7b740 1 --2- 172.21.15.43:0/3471074963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd15cb0 0x55c07dd13610 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc02ed50 tx=0x7f6dcc004600 comp rx=0 tx=0).stop 2024-09-11T23:22:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.268+0000 7f6de9e7b740 1 -- 172.21.15.43:0/3471074963 shutdown_connections 2024-09-11T23:22:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.268+0000 7f6de9e7b740 1 --2- 172.21.15.43:0/3471074963 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c07dd25a40 0x55c07dd27e30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.268+0000 7f6dbffff640 0 client.15627 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.268+0000 7f6de9e7b740 1 --2- 172.21.15.43:0/3471074963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd15cb0 0x55c07dd13610 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.268+0000 7f6de9e7b740 1 --2- 172.21.15.43:0/3471074963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c07dd0b410 0x55c07dd130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.268+0000 7f6de9e7b740 1 --2- 172.21.15.43:0/3471074963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c07dd0aaf0 0x55c07dd12b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.268+0000 7f6de9e7b740 1 -- 172.21.15.43:0/3471074963 >> 172.21.15.43:0/3471074963 conn(0x55c07dd07060 msgr2=0x55c07dd0cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.268+0000 7f6de9e7b740 1 -- 172.21.15.43:0/3471074963 shutdown_connections 2024-09-11T23:22:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.268+0000 7f6de9e7b740 1 -- 172.21.15.43:0/3471074963 wait complete. 2024-09-11T23:22:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.268+0000 7f6de9e7b740 20 client.15627 trim_cache size 0 max 16384 2024-09-11T23:22:54.272 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:53 smithi090 ceph-mon[35130]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 9 op/s 2024-09-11T23:22:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.296+0000 7f6de9e7b740 1 Processor -- start 2024-09-11T23:22:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.296+0000 7f6de9e7b740 1 -- start start 2024-09-11T23:22:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.297+0000 7f6de9e7b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c07dd0aaf0 0x55c07dd13440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.297+0000 7f6de9e7b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd0b410 0x55c07dd2ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.297+0000 7f6de9e7b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c07dd25a40 0x55c07dd2b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.297+0000 7f6de9e7b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01ae020 con 0x55c07dd0b410 2024-09-11T23:22:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.297+0000 7f6de9e7b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01ab8f0 con 0x55c07dd0aaf0 2024-09-11T23:22:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.297+0000 7f6de9e7b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01a9740 con 0x55c07dd25a40 2024-09-11T23:22:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.297+0000 7f6de4e79640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd0b410 0x55c07dd2ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.297+0000 7f6de4e79640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd0b410 0x55c07dd2ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36802/0 (socket says 172.21.15.43:36802) 2024-09-11T23:22:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.297+0000 7f6de4e79640 1 -- 172.21.15.43:0/530634851 learned_addr learned my addr 172.21.15.43:0/530634851 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.297+0000 7f6ddffff640 1 --2- 172.21.15.43:0/530634851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c07dd0aaf0 0x55c07dd13440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.297+0000 7f6ddffff640 1 --2- 172.21.15.43:0/530634851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c07dd25a40 0x55c07dd2b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.297+0000 7f6de4e79640 1 -- 172.21.15.43:0/530634851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c07dd25a40 msgr2=0x55c07dd2b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.297+0000 7f6de4e79640 1 --2- 172.21.15.43:0/530634851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c07dd25a40 0x55c07dd2b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.297+0000 7f6de4e79640 1 -- 172.21.15.43:0/530634851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c07dd0aaf0 msgr2=0x55c07dd13440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.297+0000 7f6de4e79640 1 --2- 172.21.15.43:0/530634851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c07dd0aaf0 0x55c07dd13440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.297+0000 7f6de4e79640 1 -- 172.21.15.43:0/530634851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c07dd183e0 con 0x55c07dd0b410 2024-09-11T23:22:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.297+0000 7f6ddffff640 1 --2- 172.21.15.43:0/530634851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c07dd0aaf0 0x55c07dd13440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:22:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.297+0000 7f6de4e79640 1 --2- 172.21.15.43:0/530634851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd0b410 0x55c07dd2ae50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x55c07dd12400 tx=0x7f6dd4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4e23073485c60a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.298+0000 7f6dca7fc640 1 -- 172.21.15.43:0/530634851 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd400f040 con 0x55c07dd0b410 2024-09-11T23:22:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.298+0000 7f6dca7fc640 1 -- 172.21.15.43:0/530634851 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd4021030 con 0x55c07dd0b410 2024-09-11T23:22:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.298+0000 7f6dca7fc640 1 -- 172.21.15.43:0/530634851 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4025e60 con 0x55c07dd0b410 2024-09-11T23:22:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.299+0000 7f6de9e7b740 1 -- 172.21.15.43:0/530634851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd0b410 msgr2=0x55c07dd2ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.299+0000 7f6de9e7b740 1 --2- 172.21.15.43:0/530634851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd0b410 0x55c07dd2ae50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x55c07dd12400 tx=0x7f6dd4008040 comp rx=0 tx=0).stop 2024-09-11T23:22:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.299+0000 7f6de9e7b740 1 -- 172.21.15.43:0/530634851 shutdown_connections 2024-09-11T23:22:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.299+0000 7f6de9e7b740 1 --2- 172.21.15.43:0/530634851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c07dd25a40 0x55c07dd2b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.299+0000 7f6de9e7b740 1 --2- 172.21.15.43:0/530634851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd0b410 0x55c07dd2ae50 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.299+0000 7f6de9e7b740 1 --2- 172.21.15.43:0/530634851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c07dd0aaf0 0x55c07dd13440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.299+0000 7f6de9e7b740 1 -- 172.21.15.43:0/530634851 >> 172.21.15.43:0/530634851 conn(0x55c07dd07060 msgr2=0x55c07dd14c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.299+0000 7f6de9e7b740 1 -- 172.21.15.43:0/530634851 shutdown_connections 2024-09-11T23:22:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.299+0000 7f6de9e7b740 1 -- 172.21.15.43:0/530634851 wait complete. 2024-09-11T23:22:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.299+0000 7f6de9e7b740 1 -- start start 2024-09-11T23:22:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.299+0000 7f6de9e7b740 1 Processor -- start 2024-09-11T23:22:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.300+0000 7f6de9e7b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c07dd0aaf0 0x55c07dd28d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.300+0000 7f6de9e7b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd0b410 0x55c07dd29240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.300+0000 7f6de4e79640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd0b410 0x55c07dd29240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.300+0000 7f6de4e79640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd0b410 0x55c07dd29240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36818/0 (socket says 172.21.15.43:36818) 2024-09-11T23:22:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.300+0000 7f6de4e79640 1 -- 172.21.15.43:0/576666358 learned_addr learned my addr 172.21.15.43:0/576666358 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:22:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.300+0000 7f6de9e7b740 1 --2- 172.21.15.43:0/576666358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c07dd25a40 0x55c07dd29780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.300+0000 7f6de9e7b740 1 -- 172.21.15.43:0/576666358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01a9db0 con 0x55c07dd0b410 2024-09-11T23:22:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.300+0000 7f6ddffff640 1 --2- 172.21.15.43:0/576666358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c07dd0aaf0 0x55c07dd28d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.300+0000 7f6de9e7b740 1 -- 172.21.15.43:0/576666358 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01ab220 con 0x55c07dd25a40 2024-09-11T23:22:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.300+0000 7f6de9e7b740 1 -- 172.21.15.43:0/576666358 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01aa480 con 0x55c07dd0aaf0 2024-09-11T23:22:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.301+0000 7f6de9e7b740 10 client.0 _ll_register_callbacks cb 0x7f6dd80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:22:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.301+0000 7f6de4e79640 1 -- 172.21.15.43:0/576666358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c07dd0aaf0 msgr2=0x55c07dd28d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.301+0000 7f6de4e79640 1 --2- 172.21.15.43:0/576666358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c07dd0aaf0 0x55c07dd28d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.301+0000 7f6de4e79640 1 -- 172.21.15.43:0/576666358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c07dd25a40 msgr2=0x55c07dd29780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:22:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.301+0000 7f6de4e79640 1 --2- 172.21.15.43:0/576666358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c07dd25a40 0x55c07dd29780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.301+0000 7f6de4e79640 1 -- 172.21.15.43:0/576666358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c07dd8f4e0 con 0x55c07dd0b410 2024-09-11T23:22:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.301+0000 7f6ddffff640 1 --2- 172.21.15.43:0/576666358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c07dd0aaf0 0x55c07dd28d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:22:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.301+0000 7f6de4e79640 1 --2- 172.21.15.43:0/576666358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd0b410 0x55c07dd29240 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4025e60 tx=0x7f6dd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9b66dbea910dd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:22:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.301+0000 7f6dbffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:22:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.301+0000 7f6dbffff640 1 -- 172.21.15.43:0/576666358 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd40255d0 con 0x55c07dd0b410 2024-09-11T23:22:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.301+0000 7f6dbffff640 1 -- 172.21.15.43:0/576666358 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f6dd4025770 con 0x55c07dd0b410 2024-09-11T23:22:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.301+0000 7f6de9e7b740 1 -- 172.21.15.43:0/576666358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dd4021030 con 0x55c07dd0b410 2024-09-11T23:22:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.301+0000 7f6dbffff640 1 -- 172.21.15.43:0/576666358 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6dd4026420 con 0x55c07dd0b410 2024-09-11T23:22:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.301+0000 7f6de9e7b740 10 client.15633 fetch_fsmap learned FSMap version 24 2024-09-11T23:22:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.301+0000 7f6de9e7b740 1 -- 172.21.15.43:0/576666358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c07dd907a0 con 0x55c07dd0b410 2024-09-11T23:22:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.302+0000 7f6dbffff640 1 -- 172.21.15.43:0/576666358 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd4026aa0 con 0x55c07dd0b410 2024-09-11T23:22:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.302+0000 7f6dbffff640 1 -- 172.21.15.43:0/576666358 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f6dd402e850 con 0x55c07dd0b410 2024-09-11T23:22:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.302+0000 7f6de9e7b740 10 client.15633 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:22:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.302+0000 7f6de9e7b740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:22:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.302+0000 7f6de9e7b740 20 client.15633 populate_metadata read hostname 'smithi043' 2024-09-11T23:22:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.303+0000 7f6de9e7b740 1 --2- 172.21.15.43:0/576666358 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c07dd90e20 0x55c07dd93240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:22:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.303+0000 7f6de9e7b740 4 client.15633 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:22:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.303+0000 7f6de9e7b740 1 -- 172.21.15.43:0/576666358 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f6dd4021030 con 0x55c07dd90e20 2024-09-11T23:22:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.303+0000 7f6ddffff640 1 --2- 172.21.15.43:0/576666358 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c07dd90e20 0x55c07dd93240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:22:54.307 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:53 smithi097 ceph-mon[35562]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 9 op/s 2024-09-11T23:22:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.303+0000 7f6ddffff640 1 --2- 172.21.15.43:0/576666358 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c07dd90e20 0x55c07dd93240 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34b50368dd1270b7 server_cookie=b19cc98d12782d22 in_seq=0 out_seq=0 2024-09-11T23:22:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.303+0000 7f6dbffff640 10 client.15633 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.310+0000 7f6dbffff640 1 -- 172.21.15.43:0/576666358 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f6dd4021030 con 0x55c07dd90e20 2024-09-11T23:22:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.310+0000 7f6dbffff640 10 client.15633 handle_command_reply: tid=0 2024-09-11T23:22:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.310+0000 7f6de9e7b740 1 client.15633 shutdown 2024-09-11T23:22:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.310+0000 7f6de9e7b740 1 -- 172.21.15.43:0/576666358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd0b410 msgr2=0x55c07dd29240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.310+0000 7f6de9e7b740 1 --2- 172.21.15.43:0/576666358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd0b410 0x55c07dd29240 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4025e60 tx=0x7f6dd40046c0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.310+0000 7f6de9e7b740 1 -- 172.21.15.43:0/576666358 shutdown_connections 2024-09-11T23:22:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.310+0000 7f6de9e7b740 1 --2- 172.21.15.43:0/576666358 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c07dd90e20 0x55c07dd93240 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.311+0000 7f6de9e7b740 1 --2- 172.21.15.43:0/576666358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c07dd25a40 0x55c07dd29780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.311+0000 7f6de9e7b740 1 --2- 172.21.15.43:0/576666358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c07dd0b410 0x55c07dd29240 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.311+0000 7f6dbffff640 0 client.15633 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:22:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.311+0000 7f6de9e7b740 1 --2- 172.21.15.43:0/576666358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c07dd0aaf0 0x55c07dd28d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.311+0000 7f6de9e7b740 1 -- 172.21.15.43:0/576666358 >> 172.21.15.43:0/576666358 conn(0x55c07dd07060 msgr2=0x55c07dd0c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.311+0000 7f6de9e7b740 1 -- 172.21.15.43:0/576666358 shutdown_connections 2024-09-11T23:22:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.311+0000 7f6de9e7b740 1 -- 172.21.15.43:0/576666358 wait complete. 2024-09-11T23:22:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.311+0000 7f6de9e7b740 20 client.15633 trim_cache size 0 max 16384 2024-09-11T23:22:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:22:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:22:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:22:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:22:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.312+0000 7f6de70dd640 1 -- 172.21.15.43:0/884295314 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6dd00a2e10 msgr2=0x7f6dd00a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.312+0000 7f6de70dd640 1 --2- 172.21.15.43:0/884295314 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6dd00a2e10 0x7f6dd00a52d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80cd3f0 tx=0x7f6dd8014d30 comp rx=0 tx=0).stop 2024-09-11T23:22:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.312+0000 7f6de70dd640 1 -- 172.21.15.43:0/884295314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de01069b0 msgr2=0x7f6de019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:22:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.312+0000 7f6de70dd640 1 --2- 172.21.15.43:0/884295314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de01069b0 0x7f6de019ab50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800eb10 tx=0x7f6dd800ef10 comp rx=0 tx=0).stop 2024-09-11T23:22:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.312+0000 7f6de70dd640 1 -- 172.21.15.43:0/884295314 shutdown_connections 2024-09-11T23:22:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.312+0000 7f6de70dd640 1 --2- 172.21.15.43:0/884295314 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6dd00a2e10 0x7f6dd00a52d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.312+0000 7f6de70dd640 1 --2- 172.21.15.43:0/884295314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de01069b0 0x7f6de019ab50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.312+0000 7f6de70dd640 1 --2- 172.21.15.43:0/884295314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6de0101340 0x7f6de01967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.312+0000 7f6de70dd640 1 --2- 172.21.15.43:0/884295314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6de01009a0 0x7f6de0196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:22:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.312+0000 7f6de70dd640 1 -- 172.21.15.43:0/884295314 >> 172.21.15.43:0/884295314 conn(0x7f6de00779a0 msgr2=0x7f6de01037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:22:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.312+0000 7f6de70dd640 1 -- 172.21.15.43:0/884295314 shutdown_connections 2024-09-11T23:22:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:22:54.312+0000 7f6de70dd640 1 -- 172.21.15.43:0/884295314 wait complete. 2024-09-11T23:22:56.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:55 smithi043 ceph-mon[30950]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-11T23:22:56.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:55 smithi090 ceph-mon[35130]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-11T23:22:56.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:55 smithi097 ceph-mon[35562]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-11T23:22:57.841 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:57 smithi043 ceph-mon[30950]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-11T23:22:58.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:57 smithi090 ceph-mon[35130]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-11T23:22:58.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:57 smithi097 ceph-mon[35562]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-11T23:22:59.727 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:23:00.008 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:23:00.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:22:59 smithi043 ceph-mon[30950]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-11T23:23:00.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:22:59 smithi090 ceph-mon[35130]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-11T23:23:00.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:22:59 smithi097 ceph-mon[35562]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-11T23:23:00.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.516+0000 7f824ecef640 1 -- 172.21.15.43:0/3283691772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8248106f00 msgr2=0x7f824810ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:00.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.516+0000 7f824ecef640 1 --2- 172.21.15.43:0/3283691772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8248106f00 0x7f824810ea80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8240009ca0 tx=0x7f824002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:23:00.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.516+0000 7f824ecef640 1 -- 172.21.15.43:0/3283691772 shutdown_connections 2024-09-11T23:23:00.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.516+0000 7f824ecef640 1 --2- 172.21.15.43:0/3283691772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8248106f00 0x7f824810ea80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:00.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.516+0000 7f824ecef640 1 --2- 172.21.15.43:0/3283691772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8248102780 0x7f82481067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:00.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.516+0000 7f824ecef640 1 --2- 172.21.15.43:0/3283691772 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8248101dd0 0x7f82481021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:00.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.516+0000 7f824ecef640 1 -- 172.21.15.43:0/3283691772 >> 172.21.15.43:0/3283691772 conn(0x7f82480fd890 msgr2=0x7f82480ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:00.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.516+0000 7f824ecef640 1 -- 172.21.15.43:0/3283691772 shutdown_connections 2024-09-11T23:23:00.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.516+0000 7f824ecef640 1 -- 172.21.15.43:0/3283691772 wait complete. 2024-09-11T23:23:00.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.517+0000 7f824ecef640 1 Processor -- start 2024-09-11T23:23:00.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.517+0000 7f824ecef640 1 -- start start 2024-09-11T23:23:00.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.517+0000 7f824ecef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8248101dd0 0x7f82481981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:00.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.518+0000 7f824ecef640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8248102780 0x7f8248198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:00.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.518+0000 7f824ecef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8248106f00 0x7f824819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:00.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.518+0000 7f824ecef640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8248111f60 con 0x7f8248101dd0 2024-09-11T23:23:00.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.518+0000 7f8247fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8248102780 0x7f8248198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:00.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.518+0000 7f8247fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8248102780 0x7f8248198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:39802/0 (socket says 172.21.15.43:39802) 2024-09-11T23:23:00.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.518+0000 7f8247fff640 1 -- 172.21.15.43:0/1827709521 learned_addr learned my addr 172.21.15.43:0/1827709521 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:00.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.518+0000 7f824ecef640 1 -- 172.21.15.43:0/1827709521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8248111de0 con 0x7f8248102780 2024-09-11T23:23:00.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.518+0000 7f824d28c640 1 --2- 172.21.15.43:0/1827709521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8248106f00 0x7f824819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:00.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.518+0000 7f824ecef640 1 -- 172.21.15.43:0/1827709521 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f82481120e0 con 0x7f8248106f00 2024-09-11T23:23:00.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.518+0000 7f824ca8b640 1 --2- 172.21.15.43:0/1827709521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8248101dd0 0x7f82481981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:00.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.519+0000 7f824ca8b640 1 -- 172.21.15.43:0/1827709521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8248106f00 msgr2=0x7f824819ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:00.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.519+0000 7f824ca8b640 1 --2- 172.21.15.43:0/1827709521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8248106f00 0x7f824819ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:00.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.519+0000 7f824ca8b640 1 -- 172.21.15.43:0/1827709521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8248102780 msgr2=0x7f8248198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.519+0000 7f824ca8b640 1 --2- 172.21.15.43:0/1827709521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8248102780 0x7f8248198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.519+0000 7f824ca8b640 1 -- 172.21.15.43:0/1827709521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824819d170 con 0x7f8248101dd0 2024-09-11T23:23:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.519+0000 7f8247fff640 1 --2- 172.21.15.43:0/1827709521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8248102780 0x7f8248198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:23:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.519+0000 7f824d28c640 1 --2- 172.21.15.43:0/1827709521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8248106f00 0x7f824819ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:23:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.519+0000 7f824ca8b640 1 --2- 172.21.15.43:0/1827709521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8248101dd0 0x7f82481981c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f823c00e9f0 tx=0x7f823c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.519+0000 7f8245ffb640 1 -- 172.21.15.43:0/1827709521 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c00ccc0 con 0x7f8248101dd0 2024-09-11T23:23:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.519+0000 7f8245ffb640 1 -- 172.21.15.43:0/1827709521 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f823c010910 con 0x7f8248101dd0 2024-09-11T23:23:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.519+0000 7f8245ffb640 1 -- 172.21.15.43:0/1827709521 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c018960 con 0x7f8248101dd0 2024-09-11T23:23:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.519+0000 7f824ecef640 1 -- 172.21.15.43:0/1827709521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824819d460 con 0x7f8248101dd0 2024-09-11T23:23:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.520+0000 7f824ecef640 1 -- 172.21.15.43:0/1827709521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8248074420 con 0x7f8248101dd0 2024-09-11T23:23:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.521+0000 7f824ecef640 1 -- 172.21.15.43:0/1827709521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82481032b0 con 0x7f8248101dd0 2024-09-11T23:23:00.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.523+0000 7f8245ffb640 1 -- 172.21.15.43:0/1827709521 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7f823c010ac0 con 0x7f8248101dd0 2024-09-11T23:23:00.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.524+0000 7f8245ffb640 1 --2- 172.21.15.43:0/1827709521 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f82380a2e90 0x7f82380a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:00.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.524+0000 7f8245ffb640 1 -- 172.21.15.43:0/1827709521 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f823c0cf9f0 con 0x7f8248101dd0 2024-09-11T23:23:00.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.525+0000 7f8247fff640 1 --2- 172.21.15.43:0/1827709521 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f82380a2e90 0x7f82380a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:00.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.525+0000 7f8247fff640 1 --2- 172.21.15.43:0/1827709521 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f82380a2e90 0x7f82380a5350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f82480fe140 tx=0x7f8234005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:00.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.532+0000 7f8245ffb640 1 -- 172.21.15.43:0/1827709521 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f823c09bd70 con 0x7f8248101dd0 2024-09-11T23:23:00.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.819+0000 7f824ecef640 1 -- 172.21.15.43:0/1827709521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8248199520 con 0x7f8248101dd0 2024-09-11T23:23:00.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.821+0000 7f8245ffb640 1 -- 172.21.15.43:0/1827709521 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11896 (secure 0 0 0) 0x7f823c0a0070 con 0x7f8248101dd0 2024-09-11T23:23:00.821 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:23:00.821 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"btime":"2024-09-11T23:21:49:167551+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:21:49.117342+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:23:00.822 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 24 2024-09-11T23:23:00.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.826+0000 7f824ecef640 1 -- 172.21.15.43:0/1827709521 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f82380a2e90 msgr2=0x7f82380a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:00.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.826+0000 7f824ecef640 1 --2- 172.21.15.43:0/1827709521 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f82380a2e90 0x7f82380a5350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f82480fe140 tx=0x7f8234005f90 comp rx=0 tx=0).stop 2024-09-11T23:23:00.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.826+0000 7f824ecef640 1 -- 172.21.15.43:0/1827709521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8248101dd0 msgr2=0x7f82481981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:00.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.826+0000 7f824ecef640 1 --2- 172.21.15.43:0/1827709521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8248101dd0 0x7f82481981c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f823c00e9f0 tx=0x7f823c00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:23:00.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.826+0000 7f824ecef640 1 -- 172.21.15.43:0/1827709521 shutdown_connections 2024-09-11T23:23:00.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.826+0000 7f824ecef640 1 --2- 172.21.15.43:0/1827709521 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f82380a2e90 0x7f82380a5350 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:00.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.826+0000 7f824ecef640 1 --2- 172.21.15.43:0/1827709521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8248106f00 0x7f824819ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:00.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.827+0000 7f824ecef640 1 --2- 172.21.15.43:0/1827709521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8248102780 0x7f8248198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:00.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.827+0000 7f824ecef640 1 --2- 172.21.15.43:0/1827709521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8248101dd0 0x7f82481981c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:00.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.827+0000 7f824ecef640 1 -- 172.21.15.43:0/1827709521 >> 172.21.15.43:0/1827709521 conn(0x7f82480fd890 msgr2=0x7f82480fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:00.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.827+0000 7f824ecef640 1 -- 172.21.15.43:0/1827709521 shutdown_connections 2024-09-11T23:23:00.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:00.827+0000 7f824ecef640 1 -- 172.21.15.43:0/1827709521 wait complete. 2024-09-11T23:23:01.113 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:23:00 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:23:00.839+0000 7fe55716c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:23:01.113 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:23:00 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:23:00.840+0000 7fe55716c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:23:01.113 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:23:00 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:23:00.840+0000 7fe55716c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:23:01.113 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:23:00 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:23:00.840+0000 7fe55716c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:23:01.113 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:23:00 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x[31178]: 2024-09-11T23:23:00.840+0000 7fe55716c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T23:23:01.114 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:00 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:23:01.114 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:00 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:23:01.114 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:00 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:23:01.114 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:00 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1827709521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:23:01.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:00 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:23:01.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:00 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:23:01.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:00 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:23:01.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:00 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1827709521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:23:01.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:00 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:23:01.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:00 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:23:01.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:00 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:23:01.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:00 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1827709521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:23:01.635 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 433305f6 --release --await' 2024-09-11T23:23:01.636 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 433305f6 --release --await 2024-09-11T23:23:01.918 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:23:02.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:01 smithi043 ceph-mon[30950]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-11T23:23:02.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:01 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:23:02.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:01 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:23:02.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:01 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:23:02.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:01 smithi090 ceph-mon[35130]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-11T23:23:02.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:01 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:23:02.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:01 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:23:02.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:01 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:23:02.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:01 smithi097 ceph-mon[35562]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-11T23:23:02.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:01 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:23:02.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:01 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:23:02.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:01 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:23:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.486+0000 7fb569237640 1 -- 172.21.15.43:0/2171461388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5641036e0 msgr2=0x7fb564107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.486+0000 7fb569237640 1 --2- 172.21.15.43:0/2171461388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5641036e0 0x7fb564107b90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb558009890 tx=0x7fb55802a360 comp rx=0 tx=0).stop 2024-09-11T23:23:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.486+0000 7fb569237640 1 -- 172.21.15.43:0/2171461388 shutdown_connections 2024-09-11T23:23:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.486+0000 7fb569237640 1 --2- 172.21.15.43:0/2171461388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5641082a0 0x7fb56410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.486+0000 7fb569237640 1 --2- 172.21.15.43:0/2171461388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5641036e0 0x7fb564107b90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.486+0000 7fb569237640 1 --2- 172.21.15.43:0/2171461388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb564102d30 0x7fb564103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.486+0000 7fb569237640 1 -- 172.21.15.43:0/2171461388 >> 172.21.15.43:0/2171461388 conn(0x7fb5640fc7d0 msgr2=0x7fb5640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.486+0000 7fb569237640 1 -- 172.21.15.43:0/2171461388 shutdown_connections 2024-09-11T23:23:02.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.486+0000 7fb569237640 1 -- 172.21.15.43:0/2171461388 wait complete. 2024-09-11T23:23:02.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.487+0000 7fb569237640 1 Processor -- start 2024-09-11T23:23:02.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.487+0000 7fb569237640 1 -- start start 2024-09-11T23:23:02.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.487+0000 7fb569237640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb564102d30 0x7fb5641985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:02.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.488+0000 7fb569237640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5641036e0 0x7fb564198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:02.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.488+0000 7fb562d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb564102d30 0x7fb5641985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:02.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.488+0000 7fb562d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb564102d30 0x7fb5641985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41082/0 (socket says 172.21.15.43:41082) 2024-09-11T23:23:02.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.488+0000 7fb562d9d640 1 -- 172.21.15.43:0/435087521 learned_addr learned my addr 172.21.15.43:0/435087521 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.488+0000 7fb569237640 1 --2- 172.21.15.43:0/435087521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5641082a0 0x7fb56419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.488+0000 7fb569237640 1 -- 172.21.15.43:0/435087521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb5641120a0 con 0x7fb564102d30 2024-09-11T23:23:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.488+0000 7fb569237640 1 -- 172.21.15.43:0/435087521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb564111f20 con 0x7fb5641036e0 2024-09-11T23:23:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.488+0000 7fb569237640 1 -- 172.21.15.43:0/435087521 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb564112220 con 0x7fb5641082a0 2024-09-11T23:23:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.488+0000 7fb56259c640 1 --2- 172.21.15.43:0/435087521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5641036e0 0x7fb564198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.488+0000 7fb56359e640 1 --2- 172.21.15.43:0/435087521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5641082a0 0x7fb56419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.488+0000 7fb562d9d640 1 -- 172.21.15.43:0/435087521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5641082a0 msgr2=0x7fb56419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.488+0000 7fb562d9d640 1 --2- 172.21.15.43:0/435087521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5641082a0 0x7fb56419ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.488+0000 7fb562d9d640 1 -- 172.21.15.43:0/435087521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5641036e0 msgr2=0x7fb564198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.488+0000 7fb562d9d640 1 --2- 172.21.15.43:0/435087521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5641036e0 0x7fb564198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.488+0000 7fb562d9d640 1 -- 172.21.15.43:0/435087521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb56419d570 con 0x7fb564102d30 2024-09-11T23:23:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.488+0000 7fb56359e640 1 --2- 172.21.15.43:0/435087521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5641082a0 0x7fb56419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:23:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.489+0000 7fb562d9d640 1 --2- 172.21.15.43:0/435087521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb564102d30 0x7fb5641985c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb55400ebc0 tx=0x7fb55400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.489+0000 7fb54bfff640 1 -- 172.21.15.43:0/435087521 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb554010630 con 0x7fb564102d30 2024-09-11T23:23:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.489+0000 7fb54bfff640 1 -- 172.21.15.43:0/435087521 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb554010cb0 con 0x7fb564102d30 2024-09-11T23:23:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.489+0000 7fb54bfff640 1 -- 172.21.15.43:0/435087521 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb554018bd0 con 0x7fb564102d30 2024-09-11T23:23:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.489+0000 7fb569237640 1 -- 172.21.15.43:0/435087521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb56419d860 con 0x7fb564102d30 2024-09-11T23:23:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.489+0000 7fb569237640 1 -- 172.21.15.43:0/435087521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5641a5140 con 0x7fb564102d30 2024-09-11T23:23:02.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.493+0000 7fb56b610740 1 Processor -- start 2024-09-11T23:23:02.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.493+0000 7fb56b610740 1 -- start start 2024-09-11T23:23:02.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.493+0000 7fb56b610740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562cdc72d2a0 0x562cdc72d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:02.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.493+0000 7fb56b610740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562cdc72dbc0 0x562cdc727bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:02.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.493+0000 7fb56b610740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc728260 0x562cdc7286e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:02.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.493+0000 7fb56b610740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562cdc683620 con 0x562cdc728260 2024-09-11T23:23:02.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.493+0000 7fb56b610740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562cdc62ae50 con 0x562cdc72dbc0 2024-09-11T23:23:02.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.493+0000 7fb56b610740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562cdc67f5e0 con 0x562cdc72d2a0 2024-09-11T23:23:02.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.494+0000 7fb56259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc728260 0x562cdc7286e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:02.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.494+0000 7fb56259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc728260 0x562cdc7286e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41086/0 (socket says 172.21.15.43:41086) 2024-09-11T23:23:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.494+0000 7fb56259c640 1 -- 172.21.15.43:0/3295987994 learned_addr learned my addr 172.21.15.43:0/3295987994 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.494+0000 7fb54bfff640 1 -- 172.21.15.43:0/435087521 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) ==== 138060+0+0 (secure 0 0 0) 0x7fb5540107d0 con 0x7fb564102d30 2024-09-11T23:23:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.494+0000 7fb56359e640 1 --2- 172.21.15.43:0/3295987994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562cdc72d2a0 0x562cdc72d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.494+0000 7fb562d9d640 1 --2- 172.21.15.43:0/3295987994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562cdc72dbc0 0x562cdc727bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.494+0000 7fb56259c640 1 -- 172.21.15.43:0/3295987994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562cdc72d2a0 msgr2=0x562cdc72d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.494+0000 7fb56259c640 1 --2- 172.21.15.43:0/3295987994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562cdc72d2a0 0x562cdc72d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.494+0000 7fb56259c640 1 -- 172.21.15.43:0/3295987994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562cdc72dbc0 msgr2=0x562cdc727bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.494+0000 7fb56259c640 1 --2- 172.21.15.43:0/3295987994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562cdc72dbc0 0x562cdc727bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.494+0000 7fb56259c640 1 -- 172.21.15.43:0/3295987994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cdc61bd40 con 0x562cdc728260 2024-09-11T23:23:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.495+0000 7fb56259c640 1 --2- 172.21.15.43:0/3295987994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc728260 0x562cdc7286e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb558005940 tx=0x7fb558005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0c9342bf015f09 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:02.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.495+0000 7fb54bfff640 1 --2- 172.21.15.43:0/435087521 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb54c0a2e90 0x7fb54c0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:02.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.495+0000 7fb5497fa640 1 -- 172.21.15.43:0/3295987994 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558037040 con 0x562cdc728260 2024-09-11T23:23:02.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.495+0000 7fb5497fa640 1 -- 172.21.15.43:0/3295987994 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb558030e80 con 0x562cdc728260 2024-09-11T23:23:02.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.495+0000 7fb5497fa640 1 -- 172.21.15.43:0/3295987994 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558003970 con 0x562cdc728260 2024-09-11T23:23:02.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.495+0000 7fb54bfff640 1 -- 172.21.15.43:0/435087521 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb5540cff60 con 0x7fb564102d30 2024-09-11T23:23:02.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.496+0000 7fb56359e640 1 --2- 172.21.15.43:0/435087521 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb54c0a2e90 0x7fb54c0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:02.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.496+0000 7fb56359e640 1 --2- 172.21.15.43:0/435087521 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb54c0a2e90 0x7fb54c0a5350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb55c005fd0 tx=0x7fb55c009260 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:02.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.496+0000 7fb56b610740 1 -- 172.21.15.43:0/3295987994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc728260 msgr2=0x562cdc7286e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:02.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.496+0000 7fb56b610740 1 --2- 172.21.15.43:0/3295987994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc728260 0x562cdc7286e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb558005940 tx=0x7fb558005b20 comp rx=0 tx=0).stop 2024-09-11T23:23:02.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.497+0000 7fb56b610740 1 -- 172.21.15.43:0/3295987994 shutdown_connections 2024-09-11T23:23:02.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.497+0000 7fb56b610740 1 --2- 172.21.15.43:0/3295987994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc728260 0x562cdc7286e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.497+0000 7fb56b610740 1 --2- 172.21.15.43:0/3295987994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562cdc72dbc0 0x562cdc727bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.497+0000 7fb56b610740 1 --2- 172.21.15.43:0/3295987994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562cdc72d2a0 0x562cdc72d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.497+0000 7fb56b610740 1 -- 172.21.15.43:0/3295987994 >> 172.21.15.43:0/3295987994 conn(0x562cdc71a0a0 msgr2=0x562cdc71a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:02.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.497+0000 7fb56b610740 1 -- 172.21.15.43:0/3295987994 shutdown_connections 2024-09-11T23:23:02.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.497+0000 7fb56b610740 1 -- 172.21.15.43:0/3295987994 wait complete. 2024-09-11T23:23:02.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.497+0000 7fb56b610740 1 -- start start 2024-09-11T23:23:02.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.497+0000 7fb56b610740 1 Processor -- start 2024-09-11T23:23:02.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.498+0000 7fb56b610740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc728260 0x562cdc731070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:02.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.498+0000 7fb56b610740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562cdc72d2a0 0x562cdc7315b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:02.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.498+0000 7fb56b610740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562cdc72dbc0 0x562cdc731af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:02.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.498+0000 7fb56b610740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562cdc734580 con 0x562cdc728260 2024-09-11T23:23:02.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.498+0000 7fb562d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc728260 0x562cdc731070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:02.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.498+0000 7fb562d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc728260 0x562cdc731070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41094/0 (socket says 172.21.15.43:41094) 2024-09-11T23:23:02.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.498+0000 7fb562d9d640 1 -- 172.21.15.43:0/1912367004 learned_addr learned my addr 172.21.15.43:0/1912367004 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:02.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.498+0000 7fb56b610740 1 -- 172.21.15.43:0/1912367004 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562cdc60bf10 con 0x562cdc72d2a0 2024-09-11T23:23:02.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.498+0000 7fb56b610740 1 -- 172.21.15.43:0/1912367004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562cdc734700 con 0x562cdc72dbc0 2024-09-11T23:23:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.498+0000 7fb56b610740 10 client.0 _ll_register_callbacks cb 0x562cdc62fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.498+0000 7fb56359e640 1 --2- 172.21.15.43:0/1912367004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562cdc72dbc0 0x562cdc731af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.498+0000 7fb56259c640 1 --2- 172.21.15.43:0/1912367004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562cdc72d2a0 0x562cdc7315b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.499+0000 7fb562d9d640 1 -- 172.21.15.43:0/1912367004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562cdc72dbc0 msgr2=0x562cdc731af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.499+0000 7fb562d9d640 1 --2- 172.21.15.43:0/1912367004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562cdc72dbc0 0x562cdc731af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.499+0000 7fb562d9d640 1 -- 172.21.15.43:0/1912367004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562cdc72d2a0 msgr2=0x562cdc7315b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.499+0000 7fb562d9d640 1 --2- 172.21.15.43:0/1912367004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562cdc72d2a0 0x562cdc7315b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.499+0000 7fb562d9d640 1 -- 172.21.15.43:0/1912367004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cdc392580 con 0x562cdc728260 2024-09-11T23:23:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.499+0000 7fb56359e640 1 --2- 172.21.15.43:0/1912367004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562cdc72dbc0 0x562cdc731af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:23:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.499+0000 7fb56259c640 1 --2- 172.21.15.43:0/1912367004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562cdc72d2a0 0x562cdc7315b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:23:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.499+0000 7fb562d9d640 1 --2- 172.21.15.43:0/1912367004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc728260 0x562cdc731070 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb55409bed0 tx=0x7fb554098530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd061cff049e51ea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.499+0000 7fb546ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:23:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.499+0000 7fb546ff5640 1 -- 172.21.15.43:0/1912367004 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb554010e60 con 0x562cdc728260 2024-09-11T23:23:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.499+0000 7fb546ff5640 1 -- 172.21.15.43:0/1912367004 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb5540a0150 con 0x562cdc728260 2024-09-11T23:23:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.499+0000 7fb56b610740 1 -- 172.21.15.43:0/1912367004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb558030e80 con 0x562cdc728260 2024-09-11T23:23:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.500+0000 7fb546ff5640 1 -- 172.21.15.43:0/1912367004 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb5540bc2b0 con 0x562cdc728260 2024-09-11T23:23:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.500+0000 7fb56b610740 10 client.15654 fetch_fsmap learned FSMap version 24 2024-09-11T23:23:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.500+0000 7fb56b610740 1 -- 172.21.15.43:0/1912367004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562cdc45c960 con 0x562cdc728260 2024-09-11T23:23:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.500+0000 7fb546ff5640 1 -- 172.21.15.43:0/1912367004 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb558030e80 con 0x562cdc728260 2024-09-11T23:23:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.500+0000 7fb546ff5640 1 -- 172.21.15.43:0/1912367004 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb5540cd7b0 con 0x562cdc728260 2024-09-11T23:23:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.500+0000 7fb56b610740 10 client.15654 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:23:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.500+0000 7fb56b610740 10 client.15654 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:23:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.500+0000 7fb56b610740 20 client.15654 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.501+0000 7fb56b610740 1 --2- 172.21.15.43:0/1912367004 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562cdc741510 0x562cdc743980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.501+0000 7fb56b610740 4 client.15654 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:23:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.501+0000 7fb56b610740 1 -- 172.21.15.43:0/1912367004 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cdc690b90 con 0x562cdc741510 2024-09-11T23:23:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.501+0000 7fb56259c640 1 --2- 172.21.15.43:0/1912367004 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562cdc741510 0x562cdc743980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.501+0000 7fb546ff5640 10 client.15654 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.501+0000 7fb56259c640 1 --2- 172.21.15.43:0/1912367004 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562cdc741510 0x562cdc743980 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7f43de6a84aea40 server_cookie=3b307b30eb0e6f2d in_seq=0 out_seq=0 2024-09-11T23:23:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.502+0000 7fb546ff5640 1 -- 172.21.15.43:0/1912367004 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562cdc690b90 con 0x562cdc741510 2024-09-11T23:23:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.502+0000 7fb546ff5640 10 client.15654 handle_command_reply: tid=0 2024-09-11T23:23:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.502+0000 7fb56b610740 1 client.15654 shutdown 2024-09-11T23:23:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.502+0000 7fb56b610740 1 -- 172.21.15.43:0/1912367004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc728260 msgr2=0x562cdc731070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.502+0000 7fb56b610740 1 --2- 172.21.15.43:0/1912367004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc728260 0x562cdc731070 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb55409bed0 tx=0x7fb554098530 comp rx=0 tx=0).stop 2024-09-11T23:23:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.502+0000 7fb56b610740 1 -- 172.21.15.43:0/1912367004 shutdown_connections 2024-09-11T23:23:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.502+0000 7fb56b610740 1 --2- 172.21.15.43:0/1912367004 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562cdc741510 0x562cdc743980 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.503+0000 7fb546ff5640 0 client.15654 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.503+0000 7fb56b610740 1 --2- 172.21.15.43:0/1912367004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562cdc72dbc0 0x562cdc731af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.503+0000 7fb56b610740 1 --2- 172.21.15.43:0/1912367004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562cdc72d2a0 0x562cdc7315b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.503+0000 7fb56b610740 1 --2- 172.21.15.43:0/1912367004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc728260 0x562cdc731070 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.503+0000 7fb56b610740 1 -- 172.21.15.43:0/1912367004 >> 172.21.15.43:0/1912367004 conn(0x562cdc71a0a0 msgr2=0x562cdc719750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:02.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.503+0000 7fb56b610740 1 -- 172.21.15.43:0/1912367004 shutdown_connections 2024-09-11T23:23:02.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.503+0000 7fb56b610740 1 -- 172.21.15.43:0/1912367004 wait complete. 2024-09-11T23:23:02.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.503+0000 7fb56b610740 20 client.15654 trim_cache size 0 max 16384 2024-09-11T23:23:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.520+0000 7fb56b610740 1 Processor -- start 2024-09-11T23:23:02.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.520+0000 7fb56b610740 1 -- start start 2024-09-11T23:23:02.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.520+0000 7fb56b610740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562cdc728260 0x562cdc727bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:02.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.520+0000 7fb56b610740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc72d2a0 0x562cdc731740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:02.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.521+0000 7fb56b610740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562cdc72dbc0 0x562cdc749250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:02.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.521+0000 7fb56b610740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb5641ae040 con 0x562cdc72d2a0 2024-09-11T23:23:02.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.521+0000 7fb56259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc72d2a0 0x562cdc731740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:02.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.521+0000 7fb56b610740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb5641ab910 con 0x562cdc728260 2024-09-11T23:23:02.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.521+0000 7fb56b610740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb5641a9760 con 0x562cdc72dbc0 2024-09-11T23:23:02.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.521+0000 7fb56259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc72d2a0 0x562cdc731740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41106/0 (socket says 172.21.15.43:41106) 2024-09-11T23:23:02.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.521+0000 7fb56259c640 1 -- 172.21.15.43:0/1409859901 learned_addr learned my addr 172.21.15.43:0/1409859901 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:02.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.521+0000 7fb562d9d640 1 --2- 172.21.15.43:0/1409859901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562cdc728260 0x562cdc727bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:02.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.521+0000 7fb56259c640 1 -- 172.21.15.43:0/1409859901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562cdc72dbc0 msgr2=0x562cdc749250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:23:02.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.521+0000 7fb56259c640 1 --2- 172.21.15.43:0/1409859901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562cdc72dbc0 0x562cdc749250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.521+0000 7fb56259c640 1 -- 172.21.15.43:0/1409859901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562cdc728260 msgr2=0x562cdc727bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:02.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.521+0000 7fb56259c640 1 --2- 172.21.15.43:0/1409859901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562cdc728260 0x562cdc727bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.521+0000 7fb56259c640 1 -- 172.21.15.43:0/1409859901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cdc72f5e0 con 0x562cdc72d2a0 2024-09-11T23:23:02.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.521+0000 7fb56259c640 1 --2- 172.21.15.43:0/1409859901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc72d2a0 0x562cdc731740 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb55802a950 tx=0x7fb558005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb617a8b0be6612b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.522+0000 7fb548ff9640 1 -- 172.21.15.43:0/1409859901 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558037040 con 0x562cdc72d2a0 2024-09-11T23:23:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.522+0000 7fb548ff9640 1 -- 172.21.15.43:0/1409859901 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb558030720 con 0x562cdc72d2a0 2024-09-11T23:23:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.522+0000 7fb548ff9640 1 -- 172.21.15.43:0/1409859901 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558003b90 con 0x562cdc72d2a0 2024-09-11T23:23:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.522+0000 7fb56b610740 1 -- 172.21.15.43:0/1409859901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc72d2a0 msgr2=0x562cdc731740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.522+0000 7fb56b610740 1 --2- 172.21.15.43:0/1409859901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc72d2a0 0x562cdc731740 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb55802a950 tx=0x7fb558005dc0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.523+0000 7fb56b610740 1 -- 172.21.15.43:0/1409859901 shutdown_connections 2024-09-11T23:23:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.523+0000 7fb56b610740 1 --2- 172.21.15.43:0/1409859901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562cdc72dbc0 0x562cdc749250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.523+0000 7fb56b610740 1 --2- 172.21.15.43:0/1409859901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc72d2a0 0x562cdc731740 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.523+0000 7fb56b610740 1 --2- 172.21.15.43:0/1409859901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562cdc728260 0x562cdc727bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.523+0000 7fb56b610740 1 -- 172.21.15.43:0/1409859901 >> 172.21.15.43:0/1409859901 conn(0x562cdc71a0a0 msgr2=0x562cdc731030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:02.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.523+0000 7fb56b610740 1 -- 172.21.15.43:0/1409859901 shutdown_connections 2024-09-11T23:23:02.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.523+0000 7fb56b610740 1 -- 172.21.15.43:0/1409859901 wait complete. 2024-09-11T23:23:02.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.523+0000 7fb56b610740 1 -- start start 2024-09-11T23:23:02.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.523+0000 7fb56b610740 1 Processor -- start 2024-09-11T23:23:02.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.524+0000 7fb56b610740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc728260 0x562cdc73a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:02.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.524+0000 7fb56b610740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562cdc72d2a0 0x562cdc736990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:02.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.524+0000 7fb56b610740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562cdc72dbc0 0x562cdc737120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:02.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.524+0000 7fb56b610740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb5641a9dd0 con 0x562cdc728260 2024-09-11T23:23:02.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.524+0000 7fb56b610740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb5641ab240 con 0x562cdc72dbc0 2024-09-11T23:23:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.524+0000 7fb56b610740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb5641aa4a0 con 0x562cdc72d2a0 2024-09-11T23:23:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.524+0000 7fb56b610740 10 client.0 _ll_register_callbacks cb 0x7fb5540d34b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.524+0000 7fb562d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc728260 0x562cdc73a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.524+0000 7fb562d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc728260 0x562cdc73a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41112/0 (socket says 172.21.15.43:41112) 2024-09-11T23:23:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.524+0000 7fb562d9d640 1 -- 172.21.15.43:0/4223962986 learned_addr learned my addr 172.21.15.43:0/4223962986 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.524+0000 7fb56259c640 1 --2- 172.21.15.43:0/4223962986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562cdc72d2a0 0x562cdc736990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.524+0000 7fb562d9d640 1 -- 172.21.15.43:0/4223962986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562cdc72d2a0 msgr2=0x562cdc736990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.524+0000 7fb562d9d640 1 --2- 172.21.15.43:0/4223962986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562cdc72d2a0 0x562cdc736990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.524+0000 7fb562d9d640 1 -- 172.21.15.43:0/4223962986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562cdc72dbc0 msgr2=0x562cdc737120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.524+0000 7fb562d9d640 1 --2- 172.21.15.43:0/4223962986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562cdc72dbc0 0x562cdc737120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.524+0000 7fb562d9d640 1 -- 172.21.15.43:0/4223962986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cdc7ac230 con 0x562cdc728260 2024-09-11T23:23:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.525+0000 7fb562d9d640 1 --2- 172.21.15.43:0/4223962986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc728260 0x562cdc73a280 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb5540186b0 tx=0x7fb55400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0dd5abd656df5e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.525+0000 7fb546ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:23:02.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.525+0000 7fb546ff5640 1 -- 172.21.15.43:0/4223962986 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb554010e60 con 0x562cdc728260 2024-09-11T23:23:02.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.525+0000 7fb546ff5640 1 -- 172.21.15.43:0/4223962986 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb55409b830 con 0x562cdc728260 2024-09-11T23:23:02.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.525+0000 7fb56b610740 1 -- 172.21.15.43:0/4223962986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb558030720 con 0x562cdc728260 2024-09-11T23:23:02.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.525+0000 7fb546ff5640 1 -- 172.21.15.43:0/4223962986 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb55409bd10 con 0x562cdc728260 2024-09-11T23:23:02.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.525+0000 7fb56b610740 10 client.15660 fetch_fsmap learned FSMap version 24 2024-09-11T23:23:02.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.525+0000 7fb56b610740 1 -- 172.21.15.43:0/4223962986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562cdc7ad460 con 0x562cdc728260 2024-09-11T23:23:02.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.526+0000 7fb546ff5640 1 -- 172.21.15.43:0/4223962986 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb558030720 con 0x562cdc728260 2024-09-11T23:23:02.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.526+0000 7fb546ff5640 1 -- 172.21.15.43:0/4223962986 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb5540d30d0 con 0x562cdc728260 2024-09-11T23:23:02.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.526+0000 7fb56b610740 10 client.15660 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:23:02.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.526+0000 7fb56b610740 10 client.15660 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:23:02.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.526+0000 7fb56b610740 20 client.15660 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:02.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.526+0000 7fb56b610740 1 --2- 172.21.15.43:0/4223962986 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562cdc741510 0x562cdc7afc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:02.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.526+0000 7fb56b610740 4 client.15660 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "433305f6", "release": true, "await": true}] 2024-09-11T23:23:02.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.526+0000 7fb56b610740 1 -- 172.21.15.43:0/4223962986 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "quiesce db", "set_id": "433305f6", "release": true, "await": true}) -- 0x562cdc68b0a0 con 0x562cdc741510 2024-09-11T23:23:02.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.526+0000 7fb56259c640 1 --2- 172.21.15.43:0/4223962986 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562cdc741510 0x562cdc7afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:02.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.527+0000 7fb546ff5640 10 client.15660 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:02.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.527+0000 7fb56259c640 1 --2- 172.21.15.43:0/4223962986 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562cdc741510 0x562cdc7afc00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f374f37d388fd0c server_cookie=7c555101991f385c in_seq=0 out_seq=0 2024-09-11T23:23:02.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.529+0000 7fb546ff5640 1 -- 172.21.15.43:0/4223962986 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x562cdc68b0a0 con 0x562cdc741510 2024-09-11T23:23:02.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.529+0000 7fb546ff5640 10 client.15660 handle_command_reply: tid=0 2024-09-11T23:23:02.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.530+0000 7fb56b610740 1 client.15660 shutdown 2024-09-11T23:23:02.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.530+0000 7fb56b610740 1 -- 172.21.15.43:0/4223962986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc728260 msgr2=0x562cdc73a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:02.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.530+0000 7fb56b610740 1 --2- 172.21.15.43:0/4223962986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc728260 0x562cdc73a280 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb5540186b0 tx=0x7fb55400cf90 comp rx=0 tx=0).stop 2024-09-11T23:23:02.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.530+0000 7fb56b610740 1 -- 172.21.15.43:0/4223962986 shutdown_connections 2024-09-11T23:23:02.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.530+0000 7fb56b610740 1 --2- 172.21.15.43:0/4223962986 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562cdc741510 0x562cdc7afc00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.530+0000 7fb56b610740 1 --2- 172.21.15.43:0/4223962986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562cdc72dbc0 0x562cdc737120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.530+0000 7fb56b610740 1 --2- 172.21.15.43:0/4223962986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562cdc72d2a0 0x562cdc736990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.530+0000 7fb56b610740 1 --2- 172.21.15.43:0/4223962986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562cdc728260 0x562cdc73a280 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.530+0000 7fb56b610740 1 -- 172.21.15.43:0/4223962986 >> 172.21.15.43:0/4223962986 conn(0x562cdc71a0a0 msgr2=0x562cdc731830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:02.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.530+0000 7fb546ff5640 0 client.15660 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:02.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.530+0000 7fb56b610740 1 -- 172.21.15.43:0/4223962986 shutdown_connections 2024-09-11T23:23:02.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.530+0000 7fb56b610740 1 -- 172.21.15.43:0/4223962986 wait complete. 2024-09-11T23:23:02.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.530+0000 7fb56b610740 20 client.15660 trim_cache size 0 max 16384 2024-09-11T23:23:02.533 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:23:02.533 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 24, 2024-09-11T23:23:02.533 INFO:teuthology.orchestra.run.smithi043.stdout: "leader": 24497, 2024-09-11T23:23:02.533 INFO:teuthology.orchestra.run.smithi043.stdout: "set_version": 4, 2024-09-11T23:23:02.533 INFO:teuthology.orchestra.run.smithi043.stdout: "sets": { 2024-09-11T23:23:02.533 INFO:teuthology.orchestra.run.smithi043.stdout: "433305f6": { 2024-09-11T23:23:02.533 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 4, 2024-09-11T23:23:02.533 INFO:teuthology.orchestra.run.smithi043.stdout: "age_ref": 0.0, 2024-09-11T23:23:02.534 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:23:02.534 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RELEASED", 2024-09-11T23:23:02.534 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-09-11T23:23:02.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:23:02.534 INFO:teuthology.orchestra.run.smithi043.stdout: "timeout": 90.0, 2024-09-11T23:23:02.534 INFO:teuthology.orchestra.run.smithi043.stdout: "expiration": 144.5, 2024-09-11T23:23:02.534 INFO:teuthology.orchestra.run.smithi043.stdout: "members": { 2024-09-11T23:23:02.534 INFO:teuthology.orchestra.run.smithi043.stdout: "file:/": { 2024-09-11T23:23:02.534 INFO:teuthology.orchestra.run.smithi043.stdout: "excluded": false, 2024-09-11T23:23:02.534 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:23:02.534 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RELEASED", 2024-09-11T23:23:02.534 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-09-11T23:23:02.534 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:23:02.534 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:23:02.535 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:23:02.535 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:23:02.535 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:23:02.535 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:23:02.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.532+0000 7fb569237640 1 -- 172.21.15.43:0/435087521 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb54c0a2e90 msgr2=0x7fb54c0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:02.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.532+0000 7fb569237640 1 --2- 172.21.15.43:0/435087521 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb54c0a2e90 0x7fb54c0a5350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb55c005fd0 tx=0x7fb55c009260 comp rx=0 tx=0).stop 2024-09-11T23:23:02.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.532+0000 7fb569237640 1 -- 172.21.15.43:0/435087521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb564102d30 msgr2=0x7fb5641985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:02.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.532+0000 7fb569237640 1 --2- 172.21.15.43:0/435087521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb564102d30 0x7fb5641985c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb55400ebc0 tx=0x7fb55400c6a0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.532+0000 7fb569237640 1 -- 172.21.15.43:0/435087521 shutdown_connections 2024-09-11T23:23:02.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.532+0000 7fb569237640 1 --2- 172.21.15.43:0/435087521 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb54c0a2e90 0x7fb54c0a5350 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.532+0000 7fb569237640 1 --2- 172.21.15.43:0/435087521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5641082a0 0x7fb56419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.532+0000 7fb569237640 1 --2- 172.21.15.43:0/435087521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5641036e0 0x7fb564198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.532+0000 7fb569237640 1 --2- 172.21.15.43:0/435087521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb564102d30 0x7fb5641985c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:02.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.532+0000 7fb569237640 1 -- 172.21.15.43:0/435087521 >> 172.21.15.43:0/435087521 conn(0x7fb5640fc7d0 msgr2=0x7fb564105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:02.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.533+0000 7fb569237640 1 -- 172.21.15.43:0/435087521 shutdown_connections 2024-09-11T23:23:02.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:02.533+0000 7fb569237640 1 -- 172.21.15.43:0/435087521 wait complete. 2024-09-11T23:23:03.428 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '433305f6', total seconds elapsed: 28.2 2024-09-11T23:23:04.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:03 smithi043 ceph-mon[30950]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-11T23:23:04.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:03 smithi043 ceph-mon[30950]: mgrmap e25: x(active, since 72s), standbys: z, y 2024-09-11T23:23:04.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:03 smithi090 ceph-mon[35130]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-11T23:23:04.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:03 smithi090 ceph-mon[35130]: mgrmap e25: x(active, since 72s), standbys: z, y 2024-09-11T23:23:04.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:03 smithi097 ceph-mon[35562]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-11T23:23:04.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:03 smithi097 ceph-mon[35562]: mgrmap e25: x(active, since 72s), standbys: z, y 2024-09-11T23:23:06.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:05 smithi090 ceph-mon[35130]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-11T23:23:06.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:05 smithi097 ceph-mon[35562]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-11T23:23:06.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:05 smithi043 ceph-mon[30950]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-11T23:23:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.449+0000 7fc7386d0640 1 -- 172.21.15.43:0/1271949122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7301009a0 msgr2=0x7fc730100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.449+0000 7fc7386d0640 1 --2- 172.21.15.43:0/1271949122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7301009a0 0x7fc730100e00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc720009930 tx=0x7fc72002a420 comp rx=0 tx=0).stop 2024-09-11T23:23:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.449+0000 7fc7386d0640 1 -- 172.21.15.43:0/1271949122 shutdown_connections 2024-09-11T23:23:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.449+0000 7fc7386d0640 1 --2- 172.21.15.43:0/1271949122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc730101340 0x7fc73010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.449+0000 7fc7386d0640 1 --2- 172.21.15.43:0/1271949122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7301009a0 0x7fc730100e00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.449+0000 7fc7386d0640 1 --2- 172.21.15.43:0/1271949122 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7301069b0 0x7fc730106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.449+0000 7fc7386d0640 1 -- 172.21.15.43:0/1271949122 >> 172.21.15.43:0/1271949122 conn(0x7fc7300779a0 msgr2=0x7fc7300fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.449+0000 7fc7386d0640 1 -- 172.21.15.43:0/1271949122 shutdown_connections 2024-09-11T23:23:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.449+0000 7fc7386d0640 1 -- 172.21.15.43:0/1271949122 wait complete. 2024-09-11T23:23:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.450+0000 7fc7386d0640 1 Processor -- start 2024-09-11T23:23:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.450+0000 7fc7386d0640 1 -- start start 2024-09-11T23:23:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.450+0000 7fc7386d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7301009a0 0x7fc730199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.450+0000 7fc7386d0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc730101340 0x7fc730199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.450+0000 7fc7386d0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7301069b0 0x7fc7301a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.450+0000 7fc7386d0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc730110e40 con 0x7fc7301009a0 2024-09-11T23:23:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.450+0000 7fc7386d0640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc730110cc0 con 0x7fc7301069b0 2024-09-11T23:23:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.450+0000 7fc7386d0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc730110fc0 con 0x7fc730101340 2024-09-11T23:23:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.450+0000 7fc73646c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7301009a0 0x7fc730199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.450+0000 7fc73646c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7301009a0 0x7fc730199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41132/0 (socket says 172.21.15.43:41132) 2024-09-11T23:23:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.450+0000 7fc73646c640 1 -- 172.21.15.43:0/3507461391 learned_addr learned my addr 172.21.15.43:0/3507461391 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.450+0000 7fc736c6d640 1 --2- 172.21.15.43:0/3507461391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7301069b0 0x7fc7301a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.451+0000 7fc73646c640 1 -- 172.21.15.43:0/3507461391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc730101340 msgr2=0x7fc730199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.451+0000 7fc73646c640 1 --2- 172.21.15.43:0/3507461391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc730101340 0x7fc730199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.451+0000 7fc73646c640 1 -- 172.21.15.43:0/3507461391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7301069b0 msgr2=0x7fc7301a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.451+0000 7fc73646c640 1 --2- 172.21.15.43:0/3507461391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7301069b0 0x7fc7301a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.451+0000 7fc73646c640 1 -- 172.21.15.43:0/3507461391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7301a48d0 con 0x7fc7301009a0 2024-09-11T23:23:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.451+0000 7fc73646c640 1 --2- 172.21.15.43:0/3507461391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7301009a0 0x7fc730199310 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00ea60 tx=0x7fc72c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.451+0000 7fc7277fe640 1 -- 172.21.15.43:0/3507461391 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c00cde0 con 0x7fc7301009a0 2024-09-11T23:23:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.451+0000 7fc7277fe640 1 -- 172.21.15.43:0/3507461391 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72c0035f0 con 0x7fc7301009a0 2024-09-11T23:23:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.451+0000 7fc7386d0640 1 -- 172.21.15.43:0/3507461391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7301a4aa0 con 0x7fc7301009a0 2024-09-11T23:23:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.452+0000 7fc7386d0640 1 -- 172.21.15.43:0/3507461391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7301a4fe0 con 0x7fc7301009a0 2024-09-11T23:23:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.454+0000 7fc7277fe640 1 -- 172.21.15.43:0/3507461391 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c010670 con 0x7fc7301009a0 2024-09-11T23:23:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.455+0000 7fc7277fe640 1 -- 172.21.15.43:0/3507461391 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7fc72c010890 con 0x7fc7301009a0 2024-09-11T23:23:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.455+0000 7fc73b466740 1 Processor -- start 2024-09-11T23:23:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.456+0000 7fc73b466740 1 -- start start 2024-09-11T23:23:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.456+0000 7fc73b466740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61d8af0 0x5630c61d8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.456+0000 7fc73b466740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61d9410 0x5630c61e3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.456+0000 7fc73b466740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61e3cb0 0x5630c61e60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.456+0000 7fc73b466740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5630c613d2f0 con 0x5630c61d9410 2024-09-11T23:23:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.456+0000 7fc73b466740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5630c60e5270 con 0x5630c61d8af0 2024-09-11T23:23:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.456+0000 7fc73b466740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5630c61392b0 con 0x5630c61e3cb0 2024-09-11T23:23:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.456+0000 7fc73646c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61d9410 0x5630c61e3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.456+0000 7fc736c6d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61d8af0 0x5630c61d8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.456+0000 7fc73646c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61d9410 0x5630c61e3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41146/0 (socket says 172.21.15.43:41146) 2024-09-11T23:23:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.456+0000 7fc736c6d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61d8af0 0x5630c61d8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:39890/0 (socket says 172.21.15.43:39890) 2024-09-11T23:23:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.456+0000 7fc73646c640 1 -- 172.21.15.43:0/2572100924 learned_addr learned my addr 172.21.15.43:0/2572100924 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.456+0000 7fc735c6b640 1 --2- 172.21.15.43:0/2572100924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61e3cb0 0x5630c61e60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.456+0000 7fc7277fe640 1 --2- 172.21.15.43:0/3507461391 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc71c0a2f20 0x7fc71c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.457+0000 7fc735c6b640 1 -- 172.21.15.43:0/2572100924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61d8af0 msgr2=0x5630c61d8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.457+0000 7fc735c6b640 1 --2- 172.21.15.43:0/2572100924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61d8af0 0x5630c61d8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.457+0000 7fc735c6b640 1 -- 172.21.15.43:0/2572100924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61d9410 msgr2=0x5630c61e3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.457+0000 7fc735c6b640 1 --2- 172.21.15.43:0/2572100924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61d9410 0x5630c61e3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.457+0000 7fc735c6b640 1 -- 172.21.15.43:0/2572100924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630c60e4340 con 0x5630c61e3cb0 2024-09-11T23:23:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.457+0000 7fc736c6d640 1 --2- 172.21.15.43:0/3507461391 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc71c0a2f20 0x7fc71c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.457+0000 7fc73646c640 1 --2- 172.21.15.43:0/2572100924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61d9410 0x5630c61e3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:23:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.457+0000 7fc7277fe640 1 -- 172.21.15.43:0/3507461391 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fc72c0cfa70 con 0x7fc7301009a0 2024-09-11T23:23:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.457+0000 7fc735c6b640 1 --2- 172.21.15.43:0/2572100924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61e3cb0 0x5630c61e60a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc72002a940 tx=0x7fc7200314d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ce5ac2a4180e68 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.457+0000 7fc724ff9640 1 -- 172.21.15.43:0/2572100924 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72003e030 con 0x5630c61e3cb0 2024-09-11T23:23:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.458+0000 7fc724ff9640 1 -- 172.21.15.43:0/2572100924 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7200035f0 con 0x5630c61e3cb0 2024-09-11T23:23:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.458+0000 7fc724ff9640 1 -- 172.21.15.43:0/2572100924 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc720035040 con 0x5630c61e3cb0 2024-09-11T23:23:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.458+0000 7fc736c6d640 1 --2- 172.21.15.43:0/3507461391 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc71c0a2f20 0x7fc71c0a53e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc728005fd0 tx=0x7fc728009b80 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.458+0000 7fc73b466740 1 -- 172.21.15.43:0/2572100924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61e3cb0 msgr2=0x5630c61e60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.458+0000 7fc73b466740 1 --2- 172.21.15.43:0/2572100924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61e3cb0 0x5630c61e60a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc72002a940 tx=0x7fc7200314d0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.459+0000 7fc73b466740 1 -- 172.21.15.43:0/2572100924 shutdown_connections 2024-09-11T23:23:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.459+0000 7fc73b466740 1 --2- 172.21.15.43:0/2572100924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61e3cb0 0x5630c61e60a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.459+0000 7fc73b466740 1 --2- 172.21.15.43:0/2572100924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61d9410 0x5630c61e3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.459+0000 7fc73b466740 1 --2- 172.21.15.43:0/2572100924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61d8af0 0x5630c61d8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.459+0000 7fc73b466740 1 -- 172.21.15.43:0/2572100924 >> 172.21.15.43:0/2572100924 conn(0x5630c61d5060 msgr2=0x5630c61d5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.459+0000 7fc73b466740 1 -- 172.21.15.43:0/2572100924 shutdown_connections 2024-09-11T23:23:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.459+0000 7fc73b466740 1 -- 172.21.15.43:0/2572100924 wait complete. 2024-09-11T23:23:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.459+0000 7fc73b466740 1 -- start start 2024-09-11T23:23:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.459+0000 7fc73b466740 1 Processor -- start 2024-09-11T23:23:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.460+0000 7fc73b466740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61d8af0 0x5630c61e0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.460+0000 7fc73b466740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61d9410 0x5630c61e1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.460+0000 7fc73b466740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61e3cb0 0x5630c61e15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.460+0000 7fc73b466740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5630c61eac90 con 0x5630c61e3cb0 2024-09-11T23:23:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.460+0000 7fc73b466740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5630c60c6290 con 0x5630c61d8af0 2024-09-11T23:23:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.460+0000 7fc73b466740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5630c61eae10 con 0x5630c61d9410 2024-09-11T23:23:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.460+0000 7fc73b466740 10 client.0 _ll_register_callbacks cb 0x5630c60f7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.460+0000 7fc736c6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61e3cb0 0x5630c61e15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.460+0000 7fc735c6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61d9410 0x5630c61e1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.460+0000 7fc736c6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61e3cb0 0x5630c61e15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41148/0 (socket says 172.21.15.43:41148) 2024-09-11T23:23:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.460+0000 7fc736c6d640 1 -- 172.21.15.43:0/183618729 learned_addr learned my addr 172.21.15.43:0/183618729 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.460+0000 7fc735c6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61d9410 0x5630c61e1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:60280/0 (socket says 172.21.15.43:60280) 2024-09-11T23:23:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.460+0000 7fc73646c640 1 --2- 172.21.15.43:0/183618729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61d8af0 0x5630c61e0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.460+0000 7fc735c6b640 1 -- 172.21.15.43:0/183618729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61d8af0 msgr2=0x5630c61e0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.460+0000 7fc735c6b640 1 --2- 172.21.15.43:0/183618729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61d8af0 0x5630c61e0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.460+0000 7fc735c6b640 1 -- 172.21.15.43:0/183618729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61e3cb0 msgr2=0x5630c61e15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.460+0000 7fc735c6b640 1 --2- 172.21.15.43:0/183618729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61e3cb0 0x5630c61e15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.460+0000 7fc735c6b640 1 -- 172.21.15.43:0/183618729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630c5e50210 con 0x5630c61d9410 2024-09-11T23:23:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.461+0000 7fc735c6b640 1 --2- 172.21.15.43:0/183618729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61d9410 0x5630c61e1070 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5630c61d63e0 tx=0x7fc720005900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d10092e5b5594db6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.461+0000 7fc719ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-11T23:23:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.461+0000 7fc719ffb640 1 -- 172.21.15.43:0/183618729 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc720035050 con 0x5630c61d9410 2024-09-11T23:23:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.461+0000 7fc73b466740 1 -- 172.21.15.43:0/183618729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7200035f0 con 0x5630c61d9410 2024-09-11T23:23:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.461+0000 7fc719ffb640 1 -- 172.21.15.43:0/183618729 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fc720004300 con 0x5630c61d9410 2024-09-11T23:23:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.461+0000 7fc719ffb640 1 -- 172.21.15.43:0/183618729 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc72003e040 con 0x5630c61d9410 2024-09-11T23:23:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.461+0000 7fc73b466740 10 client.25258 fetch_fsmap learned FSMap version 24 2024-09-11T23:23:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.461+0000 7fc73b466740 1 -- 172.21.15.43:0/183618729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5630c5f1c620 con 0x5630c61d9410 2024-09-11T23:23:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.461+0000 7fc719ffb640 1 -- 172.21.15.43:0/183618729 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7200035f0 con 0x5630c61d9410 2024-09-11T23:23:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.461+0000 7fc719ffb640 1 -- 172.21.15.43:0/183618729 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc720043410 con 0x5630c61d9410 2024-09-11T23:23:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.461+0000 7fc73b466740 10 client.25258 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:23:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.461+0000 7fc73b466740 10 client.25258 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.462+0000 7fc73b466740 20 client.25258 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.462+0000 7fc73b466740 1 --2- 172.21.15.43:0/183618729 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5630c61f8010 0x5630c61fa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.462+0000 7fc73b466740 4 client.25258 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:23:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.462+0000 7fc73b466740 1 -- 172.21.15.43:0/183618729 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630c614aa20 con 0x5630c61f8010 2024-09-11T23:23:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.462+0000 7fc73646c640 1 --2- 172.21.15.43:0/183618729 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5630c61f8010 0x5630c61fa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.462+0000 7fc73646c640 1 --2- 172.21.15.43:0/183618729 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5630c61f8010 0x5630c61fa4b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43d31926a0e544cc server_cookie=eeea996c9c56a611 in_seq=0 out_seq=0 2024-09-11T23:23:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.462+0000 7fc719ffb640 10 client.25258 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.463+0000 7fc719ffb640 1 -- 172.21.15.43:0/183618729 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630c614aa20 con 0x5630c61f8010 2024-09-11T23:23:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.463+0000 7fc719ffb640 10 client.25258 handle_command_reply: tid=0 2024-09-11T23:23:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.463+0000 7fc73b466740 1 client.25258 shutdown 2024-09-11T23:23:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.463+0000 7fc73b466740 1 -- 172.21.15.43:0/183618729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61d9410 msgr2=0x5630c61e1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.463+0000 7fc73b466740 1 --2- 172.21.15.43:0/183618729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61d9410 0x5630c61e1070 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5630c61d63e0 tx=0x7fc720005900 comp rx=0 tx=0).stop 2024-09-11T23:23:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.463+0000 7fc73b466740 1 -- 172.21.15.43:0/183618729 shutdown_connections 2024-09-11T23:23:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.463+0000 7fc73b466740 1 --2- 172.21.15.43:0/183618729 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5630c61f8010 0x5630c61fa4b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.463+0000 7fc73b466740 1 --2- 172.21.15.43:0/183618729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61e3cb0 0x5630c61e15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.463+0000 7fc73b466740 1 --2- 172.21.15.43:0/183618729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61d9410 0x5630c61e1070 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.463+0000 7fc73b466740 1 --2- 172.21.15.43:0/183618729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61d8af0 0x5630c61e0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.463+0000 7fc73b466740 1 -- 172.21.15.43:0/183618729 >> 172.21.15.43:0/183618729 conn(0x5630c61d5060 msgr2=0x5630c61dadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.463+0000 7fc719ffb640 0 client.25258 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.463+0000 7fc73b466740 1 -- 172.21.15.43:0/183618729 shutdown_connections 2024-09-11T23:23:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.463+0000 7fc73b466740 1 -- 172.21.15.43:0/183618729 wait complete. 2024-09-11T23:23:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.464+0000 7fc73b466740 20 client.25258 trim_cache size 0 max 16384 2024-09-11T23:23:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.492+0000 7fc73b466740 1 Processor -- start 2024-09-11T23:23:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.492+0000 7fc73b466740 1 -- start start 2024-09-11T23:23:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.493+0000 7fc73b466740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61d8af0 0x5630c61ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.493+0000 7fc73b466740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61d9410 0x5630c6200300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.493+0000 7fc73b466740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61e3cb0 0x5630c61d39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.493+0000 7fc73b466740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc714011090 con 0x5630c61d8af0 2024-09-11T23:23:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.493+0000 7fc73646c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61d8af0 0x5630c61ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.493+0000 7fc73b466740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc71400e960 con 0x5630c61e3cb0 2024-09-11T23:23:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.493+0000 7fc73b466740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc71400c7b0 con 0x5630c61d9410 2024-09-11T23:23:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.493+0000 7fc73646c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61d8af0 0x5630c61ffdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41152/0 (socket says 172.21.15.43:41152) 2024-09-11T23:23:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.493+0000 7fc73646c640 1 -- 172.21.15.43:0/129900597 learned_addr learned my addr 172.21.15.43:0/129900597 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.493+0000 7fc736c6d640 1 --2- 172.21.15.43:0/129900597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61e3cb0 0x5630c61d39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.493+0000 7fc735c6b640 1 --2- 172.21.15.43:0/129900597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61d9410 0x5630c6200300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.494+0000 7fc73646c640 1 -- 172.21.15.43:0/129900597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61d9410 msgr2=0x5630c6200300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.494+0000 7fc73646c640 1 --2- 172.21.15.43:0/129900597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61d9410 0x5630c6200300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.494+0000 7fc73646c640 1 -- 172.21.15.43:0/129900597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61e3cb0 msgr2=0x5630c61d39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.494+0000 7fc73646c640 1 --2- 172.21.15.43:0/129900597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61e3cb0 0x5630c61d39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.494+0000 7fc73646c640 1 -- 172.21.15.43:0/129900597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630c61d4e90 con 0x5630c61d8af0 2024-09-11T23:23:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.494+0000 7fc736c6d640 1 --2- 172.21.15.43:0/129900597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61e3cb0 0x5630c61d39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:23:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.494+0000 7fc735c6b640 1 --2- 172.21.15.43:0/129900597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61d9410 0x5630c6200300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:23:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.494+0000 7fc73646c640 1 --2- 172.21.15.43:0/129900597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61d8af0 0x5630c61ffdc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc72c09ccb0 tx=0x7fc72c0d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5424f3f4bb8640bb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.494+0000 7fc71bfff640 1 -- 172.21.15.43:0/129900597 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c09d420 con 0x5630c61d8af0 2024-09-11T23:23:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.494+0000 7fc71bfff640 1 -- 172.21.15.43:0/129900597 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72c0152d0 con 0x5630c61d8af0 2024-09-11T23:23:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.494+0000 7fc71bfff640 1 -- 172.21.15.43:0/129900597 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c098a70 con 0x5630c61d8af0 2024-09-11T23:23:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.495+0000 7fc73b466740 1 -- 172.21.15.43:0/129900597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61d8af0 msgr2=0x5630c61ffdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.495+0000 7fc73b466740 1 --2- 172.21.15.43:0/129900597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61d8af0 0x5630c61ffdc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc72c09ccb0 tx=0x7fc72c0d1040 comp rx=0 tx=0).stop 2024-09-11T23:23:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.495+0000 7fc73b466740 1 -- 172.21.15.43:0/129900597 shutdown_connections 2024-09-11T23:23:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.495+0000 7fc73b466740 1 --2- 172.21.15.43:0/129900597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61e3cb0 0x5630c61d39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.495+0000 7fc73b466740 1 --2- 172.21.15.43:0/129900597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61d9410 0x5630c6200300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.495+0000 7fc73b466740 1 --2- 172.21.15.43:0/129900597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61d8af0 0x5630c61ffdc0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.495+0000 7fc73b466740 1 -- 172.21.15.43:0/129900597 >> 172.21.15.43:0/129900597 conn(0x5630c61d5060 msgr2=0x5630c61dbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.495+0000 7fc73b466740 1 -- 172.21.15.43:0/129900597 shutdown_connections 2024-09-11T23:23:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.495+0000 7fc73b466740 1 -- 172.21.15.43:0/129900597 wait complete. 2024-09-11T23:23:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.496+0000 7fc73b466740 1 -- start start 2024-09-11T23:23:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.496+0000 7fc73b466740 1 Processor -- start 2024-09-11T23:23:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.496+0000 7fc73b466740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61d8af0 0x5630c61dd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.496+0000 7fc73b466740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61d9410 0x5630c61dda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.496+0000 7fc73b466740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61e3cb0 0x5630c61ef100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.496+0000 7fc73b466740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc71400ce20 con 0x5630c61d9410 2024-09-11T23:23:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.496+0000 7fc73b466740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc71400e290 con 0x5630c61d8af0 2024-09-11T23:23:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.496+0000 7fc73b466740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc71400d4f0 con 0x5630c61e3cb0 2024-09-11T23:23:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.496+0000 7fc73b466740 10 client.0 _ll_register_callbacks cb 0x7fc72c098df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.496+0000 7fc735c6b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61d9410 0x5630c61dda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.496+0000 7fc735c6b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61d9410 0x5630c61dda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41154/0 (socket says 172.21.15.43:41154) 2024-09-11T23:23:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.496+0000 7fc735c6b640 1 -- 172.21.15.43:0/873975131 learned_addr learned my addr 172.21.15.43:0/873975131 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.497+0000 7fc736c6d640 1 --2- 172.21.15.43:0/873975131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61e3cb0 0x5630c61ef100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.497+0000 7fc73646c640 1 --2- 172.21.15.43:0/873975131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61d8af0 0x5630c61dd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.497+0000 7fc735c6b640 1 -- 172.21.15.43:0/873975131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61e3cb0 msgr2=0x5630c61ef100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.497+0000 7fc735c6b640 1 --2- 172.21.15.43:0/873975131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61e3cb0 0x5630c61ef100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.497+0000 7fc735c6b640 1 -- 172.21.15.43:0/873975131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61d8af0 msgr2=0x5630c61dd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.497+0000 7fc735c6b640 1 --2- 172.21.15.43:0/873975131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61d8af0 0x5630c61dd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.497+0000 7fc735c6b640 1 -- 172.21.15.43:0/873975131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630c61ef640 con 0x5630c61d9410 2024-09-11T23:23:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.497+0000 7fc736c6d640 1 --2- 172.21.15.43:0/873975131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61e3cb0 0x5630c61ef100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:23:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.497+0000 7fc735c6b640 1 --2- 172.21.15.43:0/873975131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61d9410 0x5630c61dda70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc720006010 tx=0x7fc720043d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6462e1361dd56ad3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.497+0000 7fc719ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:23:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.497+0000 7fc719ffb640 1 -- 172.21.15.43:0/873975131 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc720058020 con 0x5630c61d9410 2024-09-11T23:23:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.497+0000 7fc719ffb640 1 -- 172.21.15.43:0/873975131 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fc720004300 con 0x5630c61d9410 2024-09-11T23:23:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.498+0000 7fc73b466740 1 -- 172.21.15.43:0/873975131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc72c0152d0 con 0x5630c61d9410 2024-09-11T23:23:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.498+0000 7fc719ffb640 1 -- 172.21.15.43:0/873975131 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc720063050 con 0x5630c61d9410 2024-09-11T23:23:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.498+0000 7fc73b466740 10 client.15678 fetch_fsmap learned FSMap version 24 2024-09-11T23:23:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.498+0000 7fc73b466740 1 -- 172.21.15.43:0/873975131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5630c61f0890 con 0x5630c61d9410 2024-09-11T23:23:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.498+0000 7fc719ffb640 1 -- 172.21.15.43:0/873975131 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc720009520 con 0x5630c61d9410 2024-09-11T23:23:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.499+0000 7fc719ffb640 1 -- 172.21.15.43:0/873975131 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc7200046c0 con 0x5630c61d9410 2024-09-11T23:23:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.499+0000 7fc73b466740 10 client.15678 fetch_fsmap finished waiting for FSMap version 24 2024-09-11T23:23:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.499+0000 7fc73b466740 10 client.15678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.499+0000 7fc73b466740 20 client.15678 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.499+0000 7fc73b466740 1 --2- 172.21.15.43:0/873975131 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5630c6262ea0 0x5630c62652c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.499+0000 7fc73b466740 4 client.15678 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:23:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.499+0000 7fc73b466740 1 -- 172.21.15.43:0/873975131 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fc72c0152d0 con 0x5630c6262ea0 2024-09-11T23:23:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.499+0000 7fc73646c640 1 --2- 172.21.15.43:0/873975131 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5630c6262ea0 0x5630c62652c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.500+0000 7fc73646c640 1 --2- 172.21.15.43:0/873975131 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5630c6262ea0 0x5630c62652c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82d585df953c62da server_cookie=48f972be3dd3b65f in_seq=0 out_seq=0 2024-09-11T23:23:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.500+0000 7fc719ffb640 10 client.15678 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.506+0000 7fc719ffb640 1 -- 172.21.15.43:0/873975131 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fc72c0152d0 con 0x5630c6262ea0 2024-09-11T23:23:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.506+0000 7fc719ffb640 10 client.15678 handle_command_reply: tid=0 2024-09-11T23:23:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.506+0000 7fc73b466740 1 client.15678 shutdown 2024-09-11T23:23:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.506+0000 7fc73b466740 1 -- 172.21.15.43:0/873975131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61d9410 msgr2=0x5630c61dda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.506+0000 7fc73b466740 1 --2- 172.21.15.43:0/873975131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61d9410 0x5630c61dda70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc720006010 tx=0x7fc720043d50 comp rx=0 tx=0).stop 2024-09-11T23:23:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.506+0000 7fc73b466740 1 -- 172.21.15.43:0/873975131 shutdown_connections 2024-09-11T23:23:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.506+0000 7fc73b466740 1 --2- 172.21.15.43:0/873975131 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5630c6262ea0 0x5630c62652c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.506+0000 7fc73b466740 1 --2- 172.21.15.43:0/873975131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5630c61e3cb0 0x5630c61ef100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.506+0000 7fc73b466740 1 --2- 172.21.15.43:0/873975131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5630c61d9410 0x5630c61dda70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.506+0000 7fc73b466740 1 --2- 172.21.15.43:0/873975131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630c61d8af0 0x5630c61dd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.506+0000 7fc73b466740 1 -- 172.21.15.43:0/873975131 >> 172.21.15.43:0/873975131 conn(0x5630c61d5060 msgr2=0x5630c61e0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.506+0000 7fc719ffb640 0 client.15678 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.506+0000 7fc73b466740 1 -- 172.21.15.43:0/873975131 shutdown_connections 2024-09-11T23:23:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.506+0000 7fc73b466740 1 -- 172.21.15.43:0/873975131 wait complete. 2024-09-11T23:23:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.506+0000 7fc73b466740 20 client.15678 trim_cache size 0 max 16384 2024-09-11T23:23:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:23:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:23:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:23:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:23:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.507+0000 7fc7386d0640 1 -- 172.21.15.43:0/3507461391 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc71c0a2f20 msgr2=0x7fc71c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.507+0000 7fc7386d0640 1 --2- 172.21.15.43:0/3507461391 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc71c0a2f20 0x7fc71c0a53e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc728005fd0 tx=0x7fc728009b80 comp rx=0 tx=0).stop 2024-09-11T23:23:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.507+0000 7fc7386d0640 1 -- 172.21.15.43:0/3507461391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7301009a0 msgr2=0x7fc730199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.507+0000 7fc7386d0640 1 --2- 172.21.15.43:0/3507461391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7301009a0 0x7fc730199310 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00ea60 tx=0x7fc72c00ef20 comp rx=0 tx=0).stop 2024-09-11T23:23:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.507+0000 7fc7386d0640 1 -- 172.21.15.43:0/3507461391 shutdown_connections 2024-09-11T23:23:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.507+0000 7fc7386d0640 1 --2- 172.21.15.43:0/3507461391 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc71c0a2f20 0x7fc71c0a53e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.507+0000 7fc7386d0640 1 --2- 172.21.15.43:0/3507461391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7301069b0 0x7fc7301a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.507+0000 7fc7386d0640 1 --2- 172.21.15.43:0/3507461391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc730101340 0x7fc730199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.507+0000 7fc7386d0640 1 --2- 172.21.15.43:0/3507461391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7301009a0 0x7fc730199310 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.507+0000 7fc7386d0640 1 -- 172.21.15.43:0/3507461391 >> 172.21.15.43:0/3507461391 conn(0x7fc7300779a0 msgr2=0x7fc7301037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.507+0000 7fc7386d0640 1 -- 172.21.15.43:0/3507461391 shutdown_connections 2024-09-11T23:23:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:07.508+0000 7fc7386d0640 1 -- 172.21.15.43:0/3507461391 wait complete. 2024-09-11T23:23:08.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:07 smithi043 ceph-mon[30950]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-11T23:23:08.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:07 smithi090 ceph-mon[35130]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-11T23:23:08.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:07 smithi097 ceph-mon[35562]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-11T23:23:10.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:09 smithi090 ceph-mon[35130]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-11T23:23:10.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:09 smithi097 ceph-mon[35562]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-11T23:23:10.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:09 smithi043 ceph-mon[30950]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-11T23:23:12.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:11 smithi090 ceph-mon[35130]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-11T23:23:12.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:11 smithi097 ceph-mon[35562]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-11T23:23:12.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:11 smithi043 ceph-mon[30950]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-11T23:23:12.589 INFO:tasks.workunit.client.0.smithi043.stderr:Updating files: 91% (11757/12903) Updating files: 92% (11871/12903) Updating files: 93% (12000/12903) Updating files: 94% (12129/12903) Updating files: 95% (12258/12903) Updating files: 96% (12387/12903) Updating files: 97% (12516/12903) Updating files: 98% (12645/12903) Updating files: 99% (12774/12903) Updating files: 100% (12903/12903) Updating files: 100% (12903/12903), done. 2024-09-11T23:23:13.387 INFO:tasks.workunit.client.0.smithi043.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-11T23:23:13.387 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-11T23:23:13.388 INFO:tasks.workunit.client.0.smithi043.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-11T23:23:13.388 INFO:tasks.workunit.client.0.smithi043.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-11T23:23:13.388 INFO:tasks.workunit.client.0.smithi043.stderr:state without impacting any branches by switching back to a branch. 2024-09-11T23:23:13.388 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-11T23:23:13.388 INFO:tasks.workunit.client.0.smithi043.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-11T23:23:13.388 INFO:tasks.workunit.client.0.smithi043.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-11T23:23:13.388 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-11T23:23:13.388 INFO:tasks.workunit.client.0.smithi043.stderr: git switch -c 2024-09-11T23:23:13.388 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-11T23:23:13.388 INFO:tasks.workunit.client.0.smithi043.stderr:Or undo this operation with: 2024-09-11T23:23:13.388 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-11T23:23:13.388 INFO:tasks.workunit.client.0.smithi043.stderr: git switch - 2024-09-11T23:23:13.389 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-11T23:23:13.389 INFO:tasks.workunit.client.0.smithi043.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-11T23:23:13.389 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-09-11T23:23:13.389 INFO:tasks.workunit.client.0.smithi043.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-11T23:23:13.393 DEBUG:teuthology.orchestra.run.smithi043:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-11T23:23:13.463 INFO:tasks.workunit.client.0.smithi043.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-11T23:23:13.465 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T23:23:13.465 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-11T23:23:14.187 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-11T23:23:14.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:13 smithi090 ceph-mon[35130]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-11T23:23:14.289 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-11T23:23:14.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:13 smithi097 ceph-mon[35562]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-11T23:23:14.332 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T23:23:14.333 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T23:23:14.333 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-11T23:23:14.359 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:13 smithi043 ceph-mon[30950]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-11T23:23:14.373 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T23:23:14.377 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T23:23:14.377 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-11T23:23:14.434 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-09-11T23:23:14.435 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-09-11T23:23:14.435 DEBUG:teuthology.orchestra.run.smithi043:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-09-11T23:23:14.497 INFO:tasks.workunit.client.0.smithi043.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-09-11T23:23:14.498 INFO:tasks.workunit.client.0.smithi043.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-09-11T23:23:14.498 INFO:tasks.workunit.client.0.smithi043.stderr:+ trap -- retry EXIT 2024-09-11T23:23:14.498 INFO:tasks.workunit.client.0.smithi043.stderr:+ rm -rf ffsb 2024-09-11T23:23:14.498 INFO:tasks.workunit.client.0.smithi043.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-09-11T23:23:14.501 INFO:tasks.workunit.client.0.smithi043.stderr:Cloning into 'ffsb'... 2024-09-11T23:23:15.350 INFO:tasks.workunit.client.0.smithi043.stderr:+ trap - EXIT 2024-09-11T23:23:15.350 INFO:tasks.workunit.client.0.smithi043.stderr:+ cd ffsb 2024-09-11T23:23:15.350 INFO:tasks.workunit.client.0.smithi043.stderr:+ ./configure 2024-09-11T23:23:15.479 INFO:tasks.workunit.client.0.smithi043.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-11T23:23:16.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:15 smithi090 ceph-mon[35130]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-11T23:23:16.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:15 smithi090 ceph-mon[35130]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:23:16.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:15 smithi097 ceph-mon[35562]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-11T23:23:16.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:15 smithi097 ceph-mon[35562]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:23:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:15 smithi043 ceph-mon[30950]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-11T23:23:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:15 smithi043 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:23:16.488 INFO:tasks.workunit.client.0.smithi043.stdout:checking whether build environment is sane... yes 2024-09-11T23:23:16.498 INFO:tasks.workunit.client.0.smithi043.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-09-11T23:23:16.498 INFO:tasks.workunit.client.0.smithi043.stdout:checking for gawk... gawk 2024-09-11T23:23:16.514 INFO:tasks.workunit.client.0.smithi043.stdout:checking whether make sets $(MAKE)... yes 2024-09-11T23:23:16.548 INFO:tasks.workunit.client.0.smithi043.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-11T23:23:16.548 INFO:tasks.workunit.client.0.smithi043.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-11T23:23:16.559 INFO:tasks.workunit.client.0.smithi043.stdout:checking for style of include used by make... GNU 2024-09-11T23:23:16.564 INFO:tasks.workunit.client.0.smithi043.stdout:checking for gcc... gcc 2024-09-11T23:23:16.619 INFO:tasks.workunit.client.0.smithi043.stdout:checking for C compiler default output file name... a.out 2024-09-11T23:23:16.620 INFO:tasks.workunit.client.0.smithi043.stdout:checking whether the C compiler works... yes 2024-09-11T23:23:16.623 INFO:tasks.workunit.client.0.smithi043.stdout:checking whether we are cross compiling... no 2024-09-11T23:23:16.659 INFO:tasks.workunit.client.0.smithi043.stdout:checking for suffix of executables... 2024-09-11T23:23:16.692 INFO:tasks.workunit.client.0.smithi043.stdout:checking for suffix of object files... o 2024-09-11T23:23:16.722 INFO:tasks.workunit.client.0.smithi043.stdout:checking whether we are using the GNU C compiler... yes 2024-09-11T23:23:16.752 INFO:tasks.workunit.client.0.smithi043.stdout:checking whether gcc accepts -g... yes 2024-09-11T23:23:16.828 INFO:tasks.workunit.client.0.smithi043.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-11T23:23:16.927 INFO:tasks.workunit.client.0.smithi043.stdout:checking dependency style of gcc... gcc3 2024-09-11T23:23:16.990 INFO:tasks.workunit.client.0.smithi043.stdout:checking how to run the C preprocessor... gcc -E 2024-09-11T23:23:17.032 INFO:tasks.workunit.client.0.smithi043.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-11T23:23:17.033 INFO:tasks.workunit.client.0.smithi043.stdout:checking for egrep... /usr/bin/grep -E 2024-09-11T23:23:17.042 INFO:tasks.workunit.client.0.smithi043.stdout:checking for AIX... no 2024-09-11T23:23:17.062 INFO:tasks.workunit.client.0.smithi043.stdout:checking for gcc... (cached) gcc 2024-09-11T23:23:17.069 INFO:tasks.workunit.client.0.smithi043.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-11T23:23:17.071 INFO:tasks.workunit.client.0.smithi043.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-11T23:23:17.071 INFO:tasks.workunit.client.0.smithi043.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-11T23:23:17.071 INFO:tasks.workunit.client.0.smithi043.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-11T23:23:17.139 INFO:tasks.workunit.client.0.smithi043.stdout:checking for main in -lm... yes 2024-09-11T23:23:17.188 INFO:tasks.workunit.client.0.smithi043.stdout:checking for main in -lpthread... yes 2024-09-11T23:23:17.344 INFO:tasks.workunit.client.0.smithi043.stdout:checking for ANSI C header files... yes 2024-09-11T23:23:17.413 INFO:tasks.workunit.client.0.smithi043.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-11T23:23:17.458 INFO:tasks.workunit.client.0.smithi043.stdout:checking for sys/types.h... yes 2024-09-11T23:23:17.504 INFO:tasks.workunit.client.0.smithi043.stdout:checking for sys/stat.h... yes 2024-09-11T23:23:17.550 INFO:tasks.workunit.client.0.smithi043.stdout:checking for stdlib.h... yes 2024-09-11T23:23:17.598 INFO:tasks.workunit.client.0.smithi043.stdout:checking for string.h... yes 2024-09-11T23:23:17.646 INFO:tasks.workunit.client.0.smithi043.stdout:checking for memory.h... yes 2024-09-11T23:23:17.694 INFO:tasks.workunit.client.0.smithi043.stdout:checking for strings.h... yes 2024-09-11T23:23:17.742 INFO:tasks.workunit.client.0.smithi043.stdout:checking for inttypes.h... yes 2024-09-11T23:23:17.789 INFO:tasks.workunit.client.0.smithi043.stdout:checking for stdint.h... yes 2024-09-11T23:23:17.839 INFO:tasks.workunit.client.0.smithi043.stdout:checking for unistd.h... yes 2024-09-11T23:23:17.903 INFO:tasks.workunit.client.0.smithi043.stdout:checking pthread.h usability... yes 2024-09-11T23:23:17.926 INFO:tasks.workunit.client.0.smithi043.stdout:checking pthread.h presence... yes 2024-09-11T23:23:17.926 INFO:tasks.workunit.client.0.smithi043.stdout:checking for pthread.h... yes 2024-09-11T23:23:17.978 INFO:tasks.workunit.client.0.smithi043.stdout:checking fcntl.h usability... yes 2024-09-11T23:23:17.999 INFO:tasks.workunit.client.0.smithi043.stdout:checking fcntl.h presence... yes 2024-09-11T23:23:17.999 INFO:tasks.workunit.client.0.smithi043.stdout:checking for fcntl.h... yes 2024-09-11T23:23:18.051 INFO:tasks.workunit.client.0.smithi043.stdout:checking limits.h usability... yes 2024-09-11T23:23:18.070 INFO:tasks.workunit.client.0.smithi043.stdout:checking limits.h presence... yes 2024-09-11T23:23:18.070 INFO:tasks.workunit.client.0.smithi043.stdout:checking for limits.h... yes 2024-09-11T23:23:18.085 INFO:tasks.workunit.client.0.smithi043.stdout:checking for stdint.h... (cached) yes 2024-09-11T23:23:18.137 INFO:tasks.workunit.client.0.smithi043.stdout:checking sys/time.h usability... yes 2024-09-11T23:23:18.157 INFO:tasks.workunit.client.0.smithi043.stdout:checking sys/time.h presence... yes 2024-09-11T23:23:18.157 INFO:tasks.workunit.client.0.smithi043.stdout:checking for sys/time.h... yes 2024-09-11T23:23:18.171 INFO:tasks.workunit.client.0.smithi043.stdout:checking for unistd.h... (cached) yes 2024-09-11T23:23:18.224 INFO:tasks.workunit.client.0.smithi043.stdout:checking sys/vfs.h usability... yes 2024-09-11T23:23:18.244 INFO:tasks.workunit.client.0.smithi043.stdout:checking sys/vfs.h presence... yes 2024-09-11T23:23:18.244 INFO:tasks.workunit.client.0.smithi043.stdout:checking for sys/vfs.h... yes 2024-09-11T23:23:18.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:17 smithi090 ceph-mon[35130]: pgmap v46: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 126 KiB/s wr, 17 op/s 2024-09-11T23:23:18.291 INFO:tasks.workunit.client.0.smithi043.stdout:checking sys/limits.h usability... no 2024-09-11T23:23:18.300 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:17 smithi097 ceph-mon[35562]: pgmap v46: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 126 KiB/s wr, 17 op/s 2024-09-11T23:23:18.312 INFO:tasks.workunit.client.0.smithi043.stdout:checking sys/limits.h presence... no 2024-09-11T23:23:18.312 INFO:tasks.workunit.client.0.smithi043.stdout:checking for sys/limits.h... no 2024-09-11T23:23:18.346 INFO:tasks.workunit.client.0.smithi043.stdout:checking for an ANSI C-conforming const... yes 2024-09-11T23:23:18.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:17 smithi043 ceph-mon[30950]: pgmap v46: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 126 KiB/s wr, 17 op/s 2024-09-11T23:23:18.380 INFO:tasks.workunit.client.0.smithi043.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-11T23:23:18.381 INFO:tasks.workunit.client.0.smithi043.stdout:checking for function prototypes... yes 2024-09-11T23:23:18.408 INFO:tasks.workunit.client.0.smithi043.stdout:checking whether setvbuf arguments are reversed... no 2024-09-11T23:23:18.481 INFO:tasks.workunit.client.0.smithi043.stdout:checking for system... yes 2024-09-11T23:23:18.544 INFO:tasks.workunit.client.0.smithi043.stdout:checking for gettimeofday... yes 2024-09-11T23:23:18.607 INFO:tasks.workunit.client.0.smithi043.stdout:checking for mkdir... yes 2024-09-11T23:23:18.670 INFO:tasks.workunit.client.0.smithi043.stdout:checking for strerror... yes 2024-09-11T23:23:18.733 INFO:tasks.workunit.client.0.smithi043.stdout:checking for open64... yes 2024-09-11T23:23:18.795 INFO:tasks.workunit.client.0.smithi043.stdout:checking for stat64... yes 2024-09-11T23:23:18.859 INFO:tasks.workunit.client.0.smithi043.stdout:checking for fseeko64... yes 2024-09-11T23:23:18.920 INFO:tasks.workunit.client.0.smithi043.stdout:checking for lrand48_r... yes 2024-09-11T23:23:18.982 INFO:tasks.workunit.client.0.smithi043.stdout:checking for srand48_r... yes 2024-09-11T23:23:19.006 INFO:tasks.workunit.client.0.smithi043.stdout:configure: creating ./config.status 2024-09-11T23:23:19.107 INFO:tasks.workunit.client.0.smithi043.stdout:config.status: creating Makefile 2024-09-11T23:23:19.121 INFO:tasks.workunit.client.0.smithi043.stdout:config.status: creating config.h 2024-09-11T23:23:19.138 INFO:tasks.workunit.client.0.smithi043.stdout:config.status: executing depfiles commands 2024-09-11T23:23:19.184 INFO:tasks.workunit.client.0.smithi043.stdout:config.status: executing default commands 2024-09-11T23:23:19.230 INFO:tasks.workunit.client.0.smithi043.stderr:+ make 2024-09-11T23:23:19.236 INFO:tasks.workunit.client.0.smithi043.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-09-11T23:23:19.238 INFO:tasks.workunit.client.0.smithi043.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-09-11T23:23:19.239 INFO:tasks.workunit.client.0.smithi043.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-09-11T23:23:19.239 INFO:tasks.workunit.client.0.smithi043.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-09-11T23:23:19.239 INFO:tasks.workunit.client.0.smithi043.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-09-11T23:23:19.239 INFO:tasks.workunit.client.0.smithi043.stderr: any GNU archive site. 2024-09-11T23:23:19.241 INFO:tasks.workunit.client.0.smithi043.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-09-11T23:23:19.242 INFO:tasks.workunit.client.0.smithi043.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-09-11T23:23:19.243 INFO:tasks.workunit.client.0.smithi043.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-09-11T23:23:19.243 INFO:tasks.workunit.client.0.smithi043.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-09-11T23:23:19.243 INFO:tasks.workunit.client.0.smithi043.stderr: You might want to install the `Automake' and `Perl' packages. 2024-09-11T23:23:19.243 INFO:tasks.workunit.client.0.smithi043.stderr: Grab them from any GNU archive site. 2024-09-11T23:23:19.299 INFO:tasks.workunit.client.0.smithi043.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-09-11T23:23:19.602 INFO:tasks.workunit.client.0.smithi043.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-09-11T23:23:19.602 INFO:tasks.workunit.client.0.smithi043.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-09-11T23:23:19.602 INFO:tasks.workunit.client.0.smithi043.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-09-11T23:23:19.602 INFO:tasks.workunit.client.0.smithi043.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-09-11T23:23:19.719 INFO:tasks.workunit.client.0.smithi043.stdout:/bin/sh ./config.status --recheck 2024-09-11T23:23:19.781 INFO:tasks.workunit.client.0.smithi043.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-09-11T23:23:20.044 INFO:tasks.workunit.client.0.smithi043.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-11T23:23:20.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:19 smithi090 ceph-mon[35130]: pgmap v47: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 126 KiB/s wr, 14 op/s 2024-09-11T23:23:20.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:19 smithi097 ceph-mon[35562]: pgmap v47: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 126 KiB/s wr, 14 op/s 2024-09-11T23:23:20.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:19 smithi043 ceph-mon[30950]: pgmap v47: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 126 KiB/s wr, 14 op/s 2024-09-11T23:23:20.559 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:23:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.603+0000 7f9e84503640 1 -- 172.21.15.43:0/2340495003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e7c1009a0 msgr2=0x7f9e7c100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.603+0000 7f9e84503640 1 --2- 172.21.15.43:0/2340495003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e7c1009a0 0x7f9e7c100e00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9e70009930 tx=0x7f9e7002a420 comp rx=0 tx=0).stop 2024-09-11T23:23:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.603+0000 7f9e84503640 1 -- 172.21.15.43:0/2340495003 shutdown_connections 2024-09-11T23:23:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.603+0000 7f9e84503640 1 --2- 172.21.15.43:0/2340495003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e7c101340 0x7f9e7c10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.603+0000 7f9e84503640 1 --2- 172.21.15.43:0/2340495003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e7c1009a0 0x7f9e7c100e00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.603+0000 7f9e84503640 1 --2- 172.21.15.43:0/2340495003 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e7c1069b0 0x7f9e7c106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.603+0000 7f9e84503640 1 -- 172.21.15.43:0/2340495003 >> 172.21.15.43:0/2340495003 conn(0x7f9e7c0779a0 msgr2=0x7f9e7c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.603+0000 7f9e84503640 1 -- 172.21.15.43:0/2340495003 shutdown_connections 2024-09-11T23:23:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.603+0000 7f9e84503640 1 -- 172.21.15.43:0/2340495003 wait complete. 2024-09-11T23:23:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.604+0000 7f9e84503640 1 Processor -- start 2024-09-11T23:23:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.604+0000 7f9e84503640 1 -- start start 2024-09-11T23:23:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.604+0000 7f9e84503640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e7c1009a0 0x7f9e7c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.604+0000 7f9e84503640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e7c101340 0x7f9e7c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.604+0000 7f9e84503640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e7c1069b0 0x7f9e7c1a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.604+0000 7f9e84503640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c110dc0 con 0x7f9e7c1009a0 2024-09-11T23:23:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.604+0000 7f9e84503640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c110c40 con 0x7f9e7c1069b0 2024-09-11T23:23:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.604+0000 7f9e84503640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c110f40 con 0x7f9e7c101340 2024-09-11T23:23:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.604+0000 7f9e8229f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e7c1009a0 0x7f9e7c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.604+0000 7f9e8229f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e7c1009a0 0x7f9e7c199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35188/0 (socket says 172.21.15.43:35188) 2024-09-11T23:23:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.604+0000 7f9e8229f640 1 -- 172.21.15.43:0/2704123768 learned_addr learned my addr 172.21.15.43:0/2704123768 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.604+0000 7f9e82aa0640 1 --2- 172.21.15.43:0/2704123768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e7c1069b0 0x7f9e7c1a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.605+0000 7f9e81a9e640 1 --2- 172.21.15.43:0/2704123768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e7c101340 0x7f9e7c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.605+0000 7f9e8229f640 1 -- 172.21.15.43:0/2704123768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e7c101340 msgr2=0x7f9e7c199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.605+0000 7f9e8229f640 1 --2- 172.21.15.43:0/2704123768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e7c101340 0x7f9e7c199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.605+0000 7f9e8229f640 1 -- 172.21.15.43:0/2704123768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e7c1069b0 msgr2=0x7f9e7c1a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.605+0000 7f9e8229f640 1 --2- 172.21.15.43:0/2704123768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e7c1069b0 0x7f9e7c1a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.605+0000 7f9e8229f640 1 -- 172.21.15.43:0/2704123768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7c1a47d0 con 0x7f9e7c1009a0 2024-09-11T23:23:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.605+0000 7f9e8229f640 1 --2- 172.21.15.43:0/2704123768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e7c1009a0 0x7f9e7c199210 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00db80 tx=0x7f9e6c00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.605+0000 7f9e7b7fe640 1 -- 172.21.15.43:0/2704123768 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c017070 con 0x7f9e7c1009a0 2024-09-11T23:23:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.605+0000 7f9e84503640 1 -- 172.21.15.43:0/2704123768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7c1a49a0 con 0x7f9e7c1009a0 2024-09-11T23:23:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.605+0000 7f9e7b7fe640 1 -- 172.21.15.43:0/2704123768 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e6c003b70 con 0x7f9e7c1009a0 2024-09-11T23:23:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.605+0000 7f9e7b7fe640 1 -- 172.21.15.43:0/2704123768 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c0106c0 con 0x7f9e7c1009a0 2024-09-11T23:23:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.605+0000 7f9e84503640 1 -- 172.21.15.43:0/2704123768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e7c1a4f10 con 0x7f9e7c1009a0 2024-09-11T23:23:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.608+0000 7f9e8729c740 1 Processor -- start 2024-09-11T23:23:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.608+0000 7f9e7b7fe640 1 -- 172.21.15.43:0/2704123768 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7f9e6c004020 con 0x7f9e7c1009a0 2024-09-11T23:23:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.608+0000 7f9e8729c740 1 -- start start 2024-09-11T23:23:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.608+0000 7f9e8729c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd4e4df9b0 0x55dd4e4dfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.608+0000 7f9e8729c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4e02d0 0x55dd4e4eeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.608+0000 7f9e8729c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4ef0c0 0x55dd4e4f14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.608+0000 7f9e8729c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dd4e4442f0 con 0x55dd4e4ef0c0 2024-09-11T23:23:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.608+0000 7f9e8729c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dd4e3ec270 con 0x55dd4e4e02d0 2024-09-11T23:23:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.608+0000 7f9e8729c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dd4e4402b0 con 0x55dd4e4df9b0 2024-09-11T23:23:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.608+0000 7f9e81a9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4ef0c0 0x55dd4e4f14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.608+0000 7f9e8229f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4e02d0 0x55dd4e4eeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.608+0000 7f9e82aa0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd4e4df9b0 0x55dd4e4dfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.608+0000 7f9e81a9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4ef0c0 0x55dd4e4f14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35198/0 (socket says 172.21.15.43:35198) 2024-09-11T23:23:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.608+0000 7f9e81a9e640 1 -- 172.21.15.43:0/1416297698 learned_addr learned my addr 172.21.15.43:0/1416297698 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.609+0000 7f9e8229f640 1 -- 172.21.15.43:0/1416297698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd4e4df9b0 msgr2=0x55dd4e4dfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.609+0000 7f9e8229f640 1 --2- 172.21.15.43:0/1416297698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd4e4df9b0 0x55dd4e4dfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.609+0000 7f9e8229f640 1 -- 172.21.15.43:0/1416297698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4ef0c0 msgr2=0x55dd4e4f14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.609+0000 7f9e8229f640 1 --2- 172.21.15.43:0/1416297698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4ef0c0 0x55dd4e4f14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.609+0000 7f9e8229f640 1 -- 172.21.15.43:0/1416297698 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd4e3eb340 con 0x55dd4e4e02d0 2024-09-11T23:23:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.609+0000 7f9e7b7fe640 1 --2- 172.21.15.43:0/2704123768 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e640a2ea0 0x7f9e640a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.609+0000 7f9e82aa0640 1 --2- 172.21.15.43:0/2704123768 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e640a2ea0 0x7f9e640a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.609+0000 7f9e7b7fe640 1 -- 172.21.15.43:0/2704123768 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f9e6c0cf4b0 con 0x7f9e7c1009a0 2024-09-11T23:23:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.609+0000 7f9e8229f640 1 --2- 172.21.15.43:0/1416297698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4e02d0 0x55dd4e4eeb80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0cc9c0 tx=0x7f9e6c0d5aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcc2d55c03833387 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.609+0000 7f9e78bfe640 1 -- 172.21.15.43:0/1416297698 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c09b560 con 0x55dd4e4e02d0 2024-09-11T23:23:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.609+0000 7f9e78bfe640 1 -- 172.21.15.43:0/1416297698 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e6c09be90 con 0x55dd4e4e02d0 2024-09-11T23:23:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.609+0000 7f9e78bfe640 1 -- 172.21.15.43:0/1416297698 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c09b0f0 con 0x55dd4e4e02d0 2024-09-11T23:23:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.610+0000 7f9e8729c740 1 -- 172.21.15.43:0/1416297698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4e02d0 msgr2=0x55dd4e4eeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.610+0000 7f9e8729c740 1 --2- 172.21.15.43:0/1416297698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4e02d0 0x55dd4e4eeb80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0cc9c0 tx=0x7f9e6c0d5aa0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.610+0000 7f9e82aa0640 1 --2- 172.21.15.43:0/2704123768 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e640a2ea0 0x7f9e640a5360 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400b7f0 tx=0x7f9e7400b2e0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.610+0000 7f9e8729c740 1 -- 172.21.15.43:0/1416297698 shutdown_connections 2024-09-11T23:23:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.610+0000 7f9e8729c740 1 --2- 172.21.15.43:0/1416297698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4ef0c0 0x55dd4e4f14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.610+0000 7f9e8729c740 1 --2- 172.21.15.43:0/1416297698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4e02d0 0x55dd4e4eeb80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.610+0000 7f9e8729c740 1 --2- 172.21.15.43:0/1416297698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd4e4df9b0 0x55dd4e4dfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.610+0000 7f9e8729c740 1 -- 172.21.15.43:0/1416297698 >> 172.21.15.43:0/1416297698 conn(0x55dd4e4dc060 msgr2=0x55dd4e4dc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.610+0000 7f9e8729c740 1 -- 172.21.15.43:0/1416297698 shutdown_connections 2024-09-11T23:23:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.610+0000 7f9e8729c740 1 -- 172.21.15.43:0/1416297698 wait complete. 2024-09-11T23:23:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.611+0000 7f9e8729c740 1 -- start start 2024-09-11T23:23:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.611+0000 7f9e8729c740 1 Processor -- start 2024-09-11T23:23:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.611+0000 7f9e8729c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4df9b0 0x55dd4e4ec0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.611+0000 7f9e8729c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4e02d0 0x55dd4e4ec610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.611+0000 7f9e8729c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd4e4ef0c0 0x55dd4e4ecb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.611+0000 7f9e8729c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dd4e4f1b10 con 0x55dd4e4e02d0 2024-09-11T23:23:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.611+0000 7f9e8729c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dd4e3cd290 con 0x55dd4e4df9b0 2024-09-11T23:23:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.611+0000 7f9e8729c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dd4e4f1c90 con 0x55dd4e4ef0c0 2024-09-11T23:23:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.611+0000 7f9e8729c740 10 client.0 _ll_register_callbacks cb 0x55dd4e3fe140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.611+0000 7f9e81a9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4e02d0 0x55dd4e4ec610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.611+0000 7f9e81a9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4e02d0 0x55dd4e4ec610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35200/0 (socket says 172.21.15.43:35200) 2024-09-11T23:23:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.611+0000 7f9e81a9e640 1 -- 172.21.15.43:0/3480912015 learned_addr learned my addr 172.21.15.43:0/3480912015 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.611+0000 7f9e8229f640 1 --2- 172.21.15.43:0/3480912015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4df9b0 0x55dd4e4ec0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.611+0000 7f9e82aa0640 1 --2- 172.21.15.43:0/3480912015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd4e4ef0c0 0x55dd4e4ecb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.611+0000 7f9e81a9e640 1 -- 172.21.15.43:0/3480912015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd4e4ef0c0 msgr2=0x55dd4e4ecb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.611+0000 7f9e81a9e640 1 --2- 172.21.15.43:0/3480912015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd4e4ef0c0 0x55dd4e4ecb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.611+0000 7f9e81a9e640 1 -- 172.21.15.43:0/3480912015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4df9b0 msgr2=0x55dd4e4ec0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.611+0000 7f9e81a9e640 1 --2- 172.21.15.43:0/3480912015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4df9b0 0x55dd4e4ec0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.611+0000 7f9e81a9e640 1 -- 172.21.15.43:0/3480912015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd4e157210 con 0x55dd4e4e02d0 2024-09-11T23:23:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.611+0000 7f9e8229f640 1 --2- 172.21.15.43:0/3480912015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4df9b0 0x55dd4e4ec0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:23:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.612+0000 7f9e81a9e640 1 --2- 172.21.15.43:0/3480912015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4e02d0 0x55dd4e4ec610 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9e70006010 tx=0x7f9e70031830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5256e89e487222 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.612+0000 7f9e61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:23:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.612+0000 7f9e61ffb640 1 -- 172.21.15.43:0/3480912015 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e70035040 con 0x55dd4e4e02d0 2024-09-11T23:23:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.612+0000 7f9e8729c740 1 -- 172.21.15.43:0/3480912015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e6c09be90 con 0x55dd4e4e02d0 2024-09-11T23:23:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.612+0000 7f9e61ffb640 1 -- 172.21.15.43:0/3480912015 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f9e70004240 con 0x55dd4e4e02d0 2024-09-11T23:23:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.612+0000 7f9e61ffb640 1 -- 172.21.15.43:0/3480912015 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9e70004440 con 0x55dd4e4e02d0 2024-09-11T23:23:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.612+0000 7f9e8729c740 10 client.15690 fetch_fsmap learned FSMap version 26 2024-09-11T23:23:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.612+0000 7f9e8729c740 1 -- 172.21.15.43:0/3480912015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dd4e223620 con 0x55dd4e4e02d0 2024-09-11T23:23:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.612+0000 7f9e61ffb640 1 -- 172.21.15.43:0/3480912015 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e6c09be90 con 0x55dd4e4e02d0 2024-09-11T23:23:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.612+0000 7f9e61ffb640 1 -- 172.21.15.43:0/3480912015 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9e70004c90 con 0x55dd4e4e02d0 2024-09-11T23:23:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.613+0000 7f9e8729c740 10 client.15690 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:23:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.613+0000 7f9e8729c740 10 client.15690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.613+0000 7f9e8729c740 20 client.15690 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.613+0000 7f9e8729c740 1 --2- 172.21.15.43:0/3480912015 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55dd4e4ff020 0x55dd4e501480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.613+0000 7f9e8729c740 4 client.15690 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:23:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.613+0000 7f9e8729c740 1 -- 172.21.15.43:0/3480912015 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd4e451a20 con 0x55dd4e4ff020 2024-09-11T23:23:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.613+0000 7f9e8229f640 1 --2- 172.21.15.43:0/3480912015 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55dd4e4ff020 0x55dd4e501480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.613+0000 7f9e8229f640 1 --2- 172.21.15.43:0/3480912015 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55dd4e4ff020 0x55dd4e501480 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17a9f3eb4ef25e80 server_cookie=f9ee3ea2a8ef569f in_seq=0 out_seq=0 2024-09-11T23:23:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.613+0000 7f9e61ffb640 10 client.15690 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.614+0000 7f9e61ffb640 1 -- 172.21.15.43:0/3480912015 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dd4e451a20 con 0x55dd4e4ff020 2024-09-11T23:23:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.614+0000 7f9e61ffb640 10 client.15690 handle_command_reply: tid=0 2024-09-11T23:23:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.614+0000 7f9e8729c740 1 client.15690 shutdown 2024-09-11T23:23:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.614+0000 7f9e8729c740 1 -- 172.21.15.43:0/3480912015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4e02d0 msgr2=0x55dd4e4ec610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.614+0000 7f9e8729c740 1 --2- 172.21.15.43:0/3480912015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4e02d0 0x55dd4e4ec610 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9e70006010 tx=0x7f9e70031830 comp rx=0 tx=0).stop 2024-09-11T23:23:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.614+0000 7f9e8729c740 1 -- 172.21.15.43:0/3480912015 shutdown_connections 2024-09-11T23:23:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.614+0000 7f9e8729c740 1 --2- 172.21.15.43:0/3480912015 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55dd4e4ff020 0x55dd4e501480 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.614+0000 7f9e8729c740 1 --2- 172.21.15.43:0/3480912015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd4e4ef0c0 0x55dd4e4ecb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.614+0000 7f9e8729c740 1 --2- 172.21.15.43:0/3480912015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4e02d0 0x55dd4e4ec610 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.614+0000 7f9e61ffb640 0 client.15690 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.614+0000 7f9e8729c740 1 --2- 172.21.15.43:0/3480912015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4df9b0 0x55dd4e4ec0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.614+0000 7f9e8729c740 1 -- 172.21.15.43:0/3480912015 >> 172.21.15.43:0/3480912015 conn(0x55dd4e4dc060 msgr2=0x55dd4e4e1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.614+0000 7f9e8729c740 1 -- 172.21.15.43:0/3480912015 shutdown_connections 2024-09-11T23:23:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.614+0000 7f9e8729c740 1 -- 172.21.15.43:0/3480912015 wait complete. 2024-09-11T23:23:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.614+0000 7f9e8729c740 20 client.15690 trim_cache size 0 max 16384 2024-09-11T23:23:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.628+0000 7f9e8729c740 1 Processor -- start 2024-09-11T23:23:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.629+0000 7f9e8729c740 1 -- start start 2024-09-11T23:23:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.629+0000 7f9e8729c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4df9b0 0x55dd4e506d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.629+0000 7f9e8729c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4e02d0 0x55dd4e5072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.629+0000 7f9e8729c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd4e4ef0c0 0x55dd4e4e9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.629+0000 7f9e8729c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c1adf20 con 0x55dd4e4df9b0 2024-09-11T23:23:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.629+0000 7f9e8729c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c1ab7f0 con 0x55dd4e4e02d0 2024-09-11T23:23:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.629+0000 7f9e8729c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c1a9640 con 0x55dd4e4ef0c0 2024-09-11T23:23:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.629+0000 7f9e8229f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4df9b0 0x55dd4e506d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.629+0000 7f9e8229f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4df9b0 0x55dd4e506d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35210/0 (socket says 172.21.15.43:35210) 2024-09-11T23:23:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.629+0000 7f9e8229f640 1 -- 172.21.15.43:0/1383493693 learned_addr learned my addr 172.21.15.43:0/1383493693 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.629+0000 7f9e82aa0640 1 --2- 172.21.15.43:0/1383493693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd4e4ef0c0 0x55dd4e4e9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.629+0000 7f9e81a9e640 1 --2- 172.21.15.43:0/1383493693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4e02d0 0x55dd4e5072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.629+0000 7f9e8229f640 1 -- 172.21.15.43:0/1383493693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd4e4ef0c0 msgr2=0x55dd4e4e9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.629+0000 7f9e8229f640 1 --2- 172.21.15.43:0/1383493693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd4e4ef0c0 0x55dd4e4e9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.629+0000 7f9e8229f640 1 -- 172.21.15.43:0/1383493693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4e02d0 msgr2=0x55dd4e5072d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.629+0000 7f9e8229f640 1 --2- 172.21.15.43:0/1383493693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4e02d0 0x55dd4e5072d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.629+0000 7f9e8229f640 1 -- 172.21.15.43:0/1383493693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd4e4e0100 con 0x55dd4e4df9b0 2024-09-11T23:23:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.629+0000 7f9e81a9e640 1 --2- 172.21.15.43:0/1383493693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4e02d0 0x55dd4e5072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:23:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.629+0000 7f9e8229f640 1 --2- 172.21.15.43:0/1383493693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4df9b0 0x55dd4e506d90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c099a40 tx=0x7f9e6c0d4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c333fa1f060782ae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.630+0000 7f9e63fff640 1 -- 172.21.15.43:0/1383493693 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c09b2d0 con 0x55dd4e4df9b0 2024-09-11T23:23:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.630+0000 7f9e63fff640 1 -- 172.21.15.43:0/1383493693 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e6c0d2dd0 con 0x55dd4e4df9b0 2024-09-11T23:23:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.630+0000 7f9e63fff640 1 -- 172.21.15.43:0/1383493693 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c09fb40 con 0x55dd4e4df9b0 2024-09-11T23:23:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.630+0000 7f9e8729c740 1 -- 172.21.15.43:0/1383493693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4df9b0 msgr2=0x55dd4e506d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.630+0000 7f9e8729c740 1 --2- 172.21.15.43:0/1383493693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4df9b0 0x55dd4e506d90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c099a40 tx=0x7f9e6c0d4040 comp rx=0 tx=0).stop 2024-09-11T23:23:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.630+0000 7f9e8729c740 1 -- 172.21.15.43:0/1383493693 shutdown_connections 2024-09-11T23:23:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.630+0000 7f9e8729c740 1 --2- 172.21.15.43:0/1383493693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd4e4ef0c0 0x55dd4e4e9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.630+0000 7f9e8729c740 1 --2- 172.21.15.43:0/1383493693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4e02d0 0x55dd4e5072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.630+0000 7f9e8729c740 1 --2- 172.21.15.43:0/1383493693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4df9b0 0x55dd4e506d90 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.630+0000 7f9e8729c740 1 -- 172.21.15.43:0/1383493693 >> 172.21.15.43:0/1383493693 conn(0x55dd4e4dc060 msgr2=0x55dd4e4ee150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.630+0000 7f9e8729c740 1 -- 172.21.15.43:0/1383493693 shutdown_connections 2024-09-11T23:23:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.630+0000 7f9e8729c740 1 -- 172.21.15.43:0/1383493693 wait complete. 2024-09-11T23:23:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.631+0000 7f9e8729c740 1 -- start start 2024-09-11T23:23:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.631+0000 7f9e8729c740 1 Processor -- start 2024-09-11T23:23:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.631+0000 7f9e8729c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4df9b0 0x55dd4e4f3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.631+0000 7f9e8729c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4e02d0 0x55dd4e4f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.631+0000 7f9e8729c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd4e4ef0c0 0x55dd4e568d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.631+0000 7f9e8729c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c1a9cb0 con 0x55dd4e4e02d0 2024-09-11T23:23:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.631+0000 7f9e8729c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c1ab120 con 0x55dd4e4df9b0 2024-09-11T23:23:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.631+0000 7f9e8729c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c1aa380 con 0x55dd4e4ef0c0 2024-09-11T23:23:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.631+0000 7f9e8729c740 10 client.0 _ll_register_callbacks cb 0x7f9e700314d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.631+0000 7f9e81a9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4e02d0 0x55dd4e4f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.631+0000 7f9e81a9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4e02d0 0x55dd4e4f38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35220/0 (socket says 172.21.15.43:35220) 2024-09-11T23:23:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.631+0000 7f9e81a9e640 1 -- 172.21.15.43:0/2309805228 learned_addr learned my addr 172.21.15.43:0/2309805228 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.631+0000 7f9e8229f640 1 --2- 172.21.15.43:0/2309805228 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4df9b0 0x55dd4e4f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.631+0000 7f9e82aa0640 1 --2- 172.21.15.43:0/2309805228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd4e4ef0c0 0x55dd4e568d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.631+0000 7f9e81a9e640 1 -- 172.21.15.43:0/2309805228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd4e4ef0c0 msgr2=0x55dd4e568d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.631+0000 7f9e81a9e640 1 --2- 172.21.15.43:0/2309805228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd4e4ef0c0 0x55dd4e568d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.631+0000 7f9e81a9e640 1 -- 172.21.15.43:0/2309805228 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4df9b0 msgr2=0x55dd4e4f3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.631+0000 7f9e81a9e640 1 --2- 172.21.15.43:0/2309805228 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4df9b0 0x55dd4e4f3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.631+0000 7f9e81a9e640 1 -- 172.21.15.43:0/2309805228 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd4e5692a0 con 0x55dd4e4e02d0 2024-09-11T23:23:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.632+0000 7f9e81a9e640 1 --2- 172.21.15.43:0/2309805228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4e02d0 0x55dd4e4f38d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9e70000c00 tx=0x7f9e700055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c35ea6cbeae87b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.632+0000 7f9e61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:23:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.632+0000 7f9e61ffb640 1 -- 172.21.15.43:0/2309805228 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e70052030 con 0x55dd4e4e02d0 2024-09-11T23:23:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.632+0000 7f9e8729c740 1 -- 172.21.15.43:0/2309805228 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e6c0d2dd0 con 0x55dd4e4e02d0 2024-09-11T23:23:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.632+0000 7f9e61ffb640 1 -- 172.21.15.43:0/2309805228 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f9e70003e40 con 0x55dd4e4e02d0 2024-09-11T23:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.632+0000 7f9e61ffb640 1 -- 172.21.15.43:0/2309805228 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9e70035040 con 0x55dd4e4e02d0 2024-09-11T23:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.632+0000 7f9e8729c740 10 client.15696 fetch_fsmap learned FSMap version 26 2024-09-11T23:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.632+0000 7f9e8729c740 1 -- 172.21.15.43:0/2309805228 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dd4e56a480 con 0x55dd4e4e02d0 2024-09-11T23:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.632+0000 7f9e61ffb640 1 -- 172.21.15.43:0/2309805228 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e6c0d2dd0 con 0x55dd4e4e02d0 2024-09-11T23:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.632+0000 7f9e61ffb640 1 -- 172.21.15.43:0/2309805228 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9e700040d0 con 0x55dd4e4e02d0 2024-09-11T23:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.632+0000 7f9e8729c740 10 client.15696 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.632+0000 7f9e8729c740 10 client.15696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.632+0000 7f9e8729c740 20 client.15696 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.632+0000 7f9e8729c740 1 --2- 172.21.15.43:0/2309805228 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55dd4e56ac00 0x55dd4e56d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.632+0000 7f9e8729c740 4 client.15696 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.632+0000 7f9e8729c740 1 -- 172.21.15.43:0/2309805228 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dd4e44bd70 con 0x55dd4e56ac00 2024-09-11T23:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.633+0000 7f9e8229f640 1 --2- 172.21.15.43:0/2309805228 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55dd4e56ac00 0x55dd4e56d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.633+0000 7f9e8229f640 1 --2- 172.21.15.43:0/2309805228 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55dd4e56ac00 0x55dd4e56d020 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=611830e972070a30 server_cookie=645925eea02fb19e in_seq=0 out_seq=0 2024-09-11T23:23:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.633+0000 7f9e61ffb640 10 client.15696 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.696+0000 7f9e61ffb640 1 -- 172.21.15.43:0/2309805228 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dd4e44bd70 con 0x55dd4e56ac00 2024-09-11T23:23:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.696+0000 7f9e61ffb640 10 client.15696 handle_command_reply: tid=0 2024-09-11T23:23:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.697+0000 7f9e8729c740 1 client.15696 shutdown 2024-09-11T23:23:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.697+0000 7f9e8729c740 1 -- 172.21.15.43:0/2309805228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4e02d0 msgr2=0x55dd4e4f38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.697+0000 7f9e8729c740 1 --2- 172.21.15.43:0/2309805228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4e02d0 0x55dd4e4f38d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9e70000c00 tx=0x7f9e700055a0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.697+0000 7f9e8729c740 1 -- 172.21.15.43:0/2309805228 shutdown_connections 2024-09-11T23:23:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.697+0000 7f9e8729c740 1 --2- 172.21.15.43:0/2309805228 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55dd4e56ac00 0x55dd4e56d020 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.697+0000 7f9e8729c740 1 --2- 172.21.15.43:0/2309805228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd4e4ef0c0 0x55dd4e568d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.697+0000 7f9e8729c740 1 --2- 172.21.15.43:0/2309805228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd4e4e02d0 0x55dd4e4f38d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.697+0000 7f9e61ffb640 0 client.15696 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.697+0000 7f9e8729c740 1 --2- 172.21.15.43:0/2309805228 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dd4e4df9b0 0x55dd4e4f3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.697+0000 7f9e8729c740 1 -- 172.21.15.43:0/2309805228 >> 172.21.15.43:0/2309805228 conn(0x55dd4e4dc060 msgr2=0x55dd4e4da9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.697+0000 7f9e8729c740 1 -- 172.21.15.43:0/2309805228 shutdown_connections 2024-09-11T23:23:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.697+0000 7f9e8729c740 1 -- 172.21.15.43:0/2309805228 wait complete. 2024-09-11T23:23:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.697+0000 7f9e8729c740 20 client.15696 trim_cache size 0 max 16384 2024-09-11T23:23:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:23:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:23:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:23:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:23:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.697+0000 7f9e84503640 1 -- 172.21.15.43:0/2704123768 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e640a2ea0 msgr2=0x7f9e640a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.697+0000 7f9e84503640 1 --2- 172.21.15.43:0/2704123768 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e640a2ea0 0x7f9e640a5360 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400b7f0 tx=0x7f9e7400b2e0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.698+0000 7f9e84503640 1 -- 172.21.15.43:0/2704123768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e7c1009a0 msgr2=0x7f9e7c199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.698+0000 7f9e84503640 1 --2- 172.21.15.43:0/2704123768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e7c1009a0 0x7f9e7c199210 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00db80 tx=0x7f9e6c00b630 comp rx=0 tx=0).stop 2024-09-11T23:23:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.698+0000 7f9e84503640 1 -- 172.21.15.43:0/2704123768 shutdown_connections 2024-09-11T23:23:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.698+0000 7f9e84503640 1 --2- 172.21.15.43:0/2704123768 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e640a2ea0 0x7f9e640a5360 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.698+0000 7f9e84503640 1 --2- 172.21.15.43:0/2704123768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e7c1069b0 0x7f9e7c1a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.698+0000 7f9e84503640 1 --2- 172.21.15.43:0/2704123768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e7c101340 0x7f9e7c199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.698+0000 7f9e84503640 1 --2- 172.21.15.43:0/2704123768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e7c1009a0 0x7f9e7c199210 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.698+0000 7f9e84503640 1 -- 172.21.15.43:0/2704123768 >> 172.21.15.43:0/2704123768 conn(0x7f9e7c0779a0 msgr2=0x7f9e7c103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.698+0000 7f9e84503640 1 -- 172.21.15.43:0/2704123768 shutdown_connections 2024-09-11T23:23:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:20.698+0000 7f9e84503640 1 -- 172.21.15.43:0/2704123768 wait complete. 2024-09-11T23:23:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:23:21.052 INFO:tasks.workunit.client.0.smithi043.stdout:checking whether build environment is sane... yes 2024-09-11T23:23:21.058 INFO:tasks.workunit.client.0.smithi043.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-09-11T23:23:21.058 INFO:tasks.workunit.client.0.smithi043.stdout:checking for gawk... gawk 2024-09-11T23:23:21.073 INFO:tasks.workunit.client.0.smithi043.stdout:checking whether make sets $(MAKE)... yes 2024-09-11T23:23:21.102 INFO:tasks.workunit.client.0.smithi043.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-11T23:23:21.103 INFO:tasks.workunit.client.0.smithi043.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-11T23:23:21.116 INFO:tasks.workunit.client.0.smithi043.stdout:checking for style of include used by make... GNU 2024-09-11T23:23:21.120 INFO:tasks.workunit.client.0.smithi043.stdout:checking for gcc... gcc 2024-09-11T23:23:21.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:20 smithi043 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:23:21.220 INFO:tasks.workunit.client.0.smithi043.stdout:checking whether the C compiler works... yes 2024-09-11T23:23:21.220 INFO:tasks.workunit.client.0.smithi043.stdout:checking for C compiler default output file name... a.out 2024-09-11T23:23:21.259 INFO:tasks.workunit.client.0.smithi043.stdout:checking for suffix of executables... 2024-09-11T23:23:21.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:20 smithi090 ceph-mon[35130]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:23:21.300 INFO:tasks.workunit.client.0.smithi043.stdout:checking whether we are cross compiling... no 2024-09-11T23:23:21.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:20 smithi097 ceph-mon[35562]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:23:21.337 INFO:tasks.workunit.client.0.smithi043.stdout:checking for suffix of object files... o 2024-09-11T23:23:21.360 INFO:tasks.workunit.client.0.smithi043.stdout:checking whether we are using the GNU C compiler... yes 2024-09-11T23:23:21.395 INFO:tasks.workunit.client.0.smithi043.stdout:checking whether gcc accepts -g... yes 2024-09-11T23:23:21.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.365+0000 7ff173ed1640 1 -- 172.21.15.43:0/3244002014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16c1082a0 msgr2=0x7ff16c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:21.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.365+0000 7ff173ed1640 1 --2- 172.21.15.43:0/3244002014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16c1082a0 0x7ff16c10ec40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff164009cd0 tx=0x7ff16402a8e0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.366+0000 7ff173ed1640 1 -- 172.21.15.43:0/3244002014 shutdown_connections 2024-09-11T23:23:21.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.366+0000 7ff173ed1640 1 --2- 172.21.15.43:0/3244002014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16c1082a0 0x7ff16c10ec40 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.366+0000 7ff173ed1640 1 --2- 172.21.15.43:0/3244002014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff16c1036e0 0x7ff16c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.366+0000 7ff173ed1640 1 --2- 172.21.15.43:0/3244002014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff16c102d30 0x7ff16c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.366+0000 7ff173ed1640 1 -- 172.21.15.43:0/3244002014 >> 172.21.15.43:0/3244002014 conn(0x7ff16c0fc7d0 msgr2=0x7ff16c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:21.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.366+0000 7ff173ed1640 1 -- 172.21.15.43:0/3244002014 shutdown_connections 2024-09-11T23:23:21.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.366+0000 7ff173ed1640 1 -- 172.21.15.43:0/3244002014 wait complete. 2024-09-11T23:23:21.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.366+0000 7ff173ed1640 1 Processor -- start 2024-09-11T23:23:21.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.367+0000 7ff173ed1640 1 -- start start 2024-09-11T23:23:21.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.367+0000 7ff173ed1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff16c102d30 0x7ff16c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:21.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.367+0000 7ff173ed1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16c1036e0 0x7ff16c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:21.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.367+0000 7ff173ed1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff16c1082a0 0x7ff16c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:21.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.367+0000 7ff173ed1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c1120a0 con 0x7ff16c1036e0 2024-09-11T23:23:21.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.367+0000 7ff173ed1640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c111f20 con 0x7ff16c102d30 2024-09-11T23:23:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.367+0000 7ff173ed1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c112220 con 0x7ff16c1082a0 2024-09-11T23:23:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.367+0000 7ff17146c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16c1036e0 0x7ff16c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.367+0000 7ff17146c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16c1036e0 0x7ff16c198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35228/0 (socket says 172.21.15.43:35228) 2024-09-11T23:23:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.367+0000 7ff17146c640 1 -- 172.21.15.43:0/4080479935 learned_addr learned my addr 172.21.15.43:0/4080479935 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.367+0000 7ff171c6d640 1 --2- 172.21.15.43:0/4080479935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff16c102d30 0x7ff16c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.367+0000 7ff17246e640 1 --2- 172.21.15.43:0/4080479935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff16c1082a0 0x7ff16c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.367+0000 7ff171c6d640 1 -- 172.21.15.43:0/4080479935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff16c1082a0 msgr2=0x7ff16c19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.367+0000 7ff171c6d640 1 --2- 172.21.15.43:0/4080479935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff16c1082a0 0x7ff16c19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.368+0000 7ff171c6d640 1 -- 172.21.15.43:0/4080479935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16c1036e0 msgr2=0x7ff16c198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.368+0000 7ff171c6d640 1 --2- 172.21.15.43:0/4080479935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16c1036e0 0x7ff16c198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.368+0000 7ff171c6d640 1 -- 172.21.15.43:0/4080479935 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16c19d470 con 0x7ff16c102d30 2024-09-11T23:23:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.368+0000 7ff171c6d640 1 --2- 172.21.15.43:0/4080479935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff16c102d30 0x7ff16c1984c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff168005fd0 tx=0x7ff16800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.368+0000 7ff162ffd640 1 -- 172.21.15.43:0/4080479935 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff168009c30 con 0x7ff16c102d30 2024-09-11T23:23:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.368+0000 7ff173ed1640 1 -- 172.21.15.43:0/4080479935 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16c19d760 con 0x7ff16c102d30 2024-09-11T23:23:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.368+0000 7ff162ffd640 1 -- 172.21.15.43:0/4080479935 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff168009dd0 con 0x7ff16c102d30 2024-09-11T23:23:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.368+0000 7ff162ffd640 1 -- 172.21.15.43:0/4080479935 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff168010460 con 0x7ff16c102d30 2024-09-11T23:23:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.368+0000 7ff173ed1640 1 -- 172.21.15.43:0/4080479935 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff16c1a5040 con 0x7ff16c102d30 2024-09-11T23:23:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.370+0000 7ff162ffd640 1 -- 172.21.15.43:0/4080479935 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 25) ==== 138117+0+0 (secure 0 0 0) 0x7ff168003f50 con 0x7ff16c102d30 2024-09-11T23:23:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.370+0000 7ff1762aa740 1 Processor -- start 2024-09-11T23:23:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.371+0000 7ff1762aa740 1 -- start start 2024-09-11T23:23:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.371+0000 7ff1762aa740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af639824a0 0x55af63982880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.371+0000 7ff1762aa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af63982dc0 0x55af6398d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.371+0000 7ff1762aa740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af6398d680 0x55af6398fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.371+0000 7ff1762aa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55af638e82e0 con 0x55af63982dc0 2024-09-11T23:23:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.371+0000 7ff1762aa740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55af6388fce0 con 0x55af6398d680 2024-09-11T23:23:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.371+0000 7ff1762aa740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55af638e42a0 con 0x55af639824a0 2024-09-11T23:23:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.371+0000 7ff171c6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af63982dc0 0x55af6398d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.371+0000 7ff17146c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af6398d680 0x55af6398fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.371+0000 7ff17146c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af6398d680 0x55af6398fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:45252/0 (socket says 172.21.15.43:45252) 2024-09-11T23:23:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.371+0000 7ff171c6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af63982dc0 0x55af6398d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35230/0 (socket says 172.21.15.43:35230) 2024-09-11T23:23:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.371+0000 7ff17146c640 1 -- 172.21.15.43:0/4168769853 learned_addr learned my addr 172.21.15.43:0/4168769853 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.371+0000 7ff17246e640 1 --2- 172.21.15.43:0/4168769853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af639824a0 0x55af63982880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.372+0000 7ff162ffd640 1 --2- 172.21.15.43:0/4080479935 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff1580a2ea0 0x7ff1580a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.372+0000 7ff17146c640 1 -- 172.21.15.43:0/4168769853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af639824a0 msgr2=0x55af63982880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.372+0000 7ff17146c640 1 --2- 172.21.15.43:0/4168769853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af639824a0 0x55af63982880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.372+0000 7ff17146c640 1 -- 172.21.15.43:0/4168769853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af63982dc0 msgr2=0x55af6398d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.372+0000 7ff17146c640 1 --2- 172.21.15.43:0/4168769853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af63982dc0 0x55af6398d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.372+0000 7ff17146c640 1 -- 172.21.15.43:0/4168769853 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af63874440 con 0x55af6398d680 2024-09-11T23:23:21.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.372+0000 7ff17246e640 1 --2- 172.21.15.43:0/4080479935 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff1580a2ea0 0x7ff1580a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:21.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.372+0000 7ff162ffd640 1 -- 172.21.15.43:0/4080479935 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff1680cf310 con 0x7ff16c102d30 2024-09-11T23:23:21.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.372+0000 7ff17146c640 1 --2- 172.21.15.43:0/4168769853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af6398d680 0x55af6398fa70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00ac70 tx=0x7ff15c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=337ee957d418338f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:21.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.372+0000 7ff157fff640 1 -- 172.21.15.43:0/4168769853 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c01d070 con 0x55af6398d680 2024-09-11T23:23:21.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.372+0000 7ff157fff640 1 -- 172.21.15.43:0/4168769853 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff15c018b20 con 0x55af6398d680 2024-09-11T23:23:21.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.372+0000 7ff157fff640 1 -- 172.21.15.43:0/4168769853 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c010b70 con 0x55af6398d680 2024-09-11T23:23:21.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.373+0000 7ff1762aa740 1 -- 172.21.15.43:0/4168769853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af6398d680 msgr2=0x55af6398fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:21.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.373+0000 7ff1762aa740 1 --2- 172.21.15.43:0/4168769853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af6398d680 0x55af6398fa70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00ac70 tx=0x7ff15c00f790 comp rx=0 tx=0).stop 2024-09-11T23:23:21.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.373+0000 7ff17246e640 1 --2- 172.21.15.43:0/4080479935 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff1580a2ea0 0x7ff1580a5360 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff164006010 tx=0x7ff16403e040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:21.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.373+0000 7ff1762aa740 1 -- 172.21.15.43:0/4168769853 shutdown_connections 2024-09-11T23:23:21.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.373+0000 7ff1762aa740 1 --2- 172.21.15.43:0/4168769853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af6398d680 0x55af6398fa70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.373+0000 7ff1762aa740 1 --2- 172.21.15.43:0/4168769853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af63982dc0 0x55af6398d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.373+0000 7ff1762aa740 1 --2- 172.21.15.43:0/4168769853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af639824a0 0x55af63982880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.373+0000 7ff1762aa740 1 -- 172.21.15.43:0/4168769853 >> 172.21.15.43:0/4168769853 conn(0x55af6397ea30 msgr2=0x55af6397ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:21.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.373+0000 7ff1762aa740 1 -- 172.21.15.43:0/4168769853 shutdown_connections 2024-09-11T23:23:21.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.373+0000 7ff1762aa740 1 -- 172.21.15.43:0/4168769853 wait complete. 2024-09-11T23:23:21.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.373+0000 7ff1762aa740 1 -- start start 2024-09-11T23:23:21.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.373+0000 7ff1762aa740 1 Processor -- start 2024-09-11T23:23:21.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.374+0000 7ff1762aa740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af639824a0 0x55af6398cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:21.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.374+0000 7ff1762aa740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af63982dc0 0x55af6398a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:21.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.374+0000 7ff1762aa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af6398d680 0x55af6398a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:21.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.374+0000 7ff1762aa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55af63994660 con 0x55af6398d680 2024-09-11T23:23:21.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.374+0000 7ff1762aa740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55af638a3050 con 0x55af63982dc0 2024-09-11T23:23:21.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.374+0000 7ff1762aa740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55af639947e0 con 0x55af639824a0 2024-09-11T23:23:21.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.374+0000 7ff1762aa740 10 client.0 _ll_register_callbacks cb 0x55af638e3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:21.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.374+0000 7ff17246e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af6398d680 0x55af6398a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:21.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.374+0000 7ff17246e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af6398d680 0x55af6398a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35242/0 (socket says 172.21.15.43:35242) 2024-09-11T23:23:21.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.374+0000 7ff17246e640 1 -- 172.21.15.43:0/1955466444 learned_addr learned my addr 172.21.15.43:0/1955466444 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:21.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.374+0000 7ff171c6d640 1 --2- 172.21.15.43:0/1955466444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af639824a0 0x55af6398cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:21.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.374+0000 7ff17146c640 1 --2- 172.21.15.43:0/1955466444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af63982dc0 0x55af6398a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:21.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.374+0000 7ff17246e640 1 -- 172.21.15.43:0/1955466444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af639824a0 msgr2=0x55af6398cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:21.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.374+0000 7ff17246e640 1 --2- 172.21.15.43:0/1955466444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af639824a0 0x55af6398cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.374+0000 7ff17246e640 1 -- 172.21.15.43:0/1955466444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af63982dc0 msgr2=0x55af6398a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:21.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.374+0000 7ff17246e640 1 --2- 172.21.15.43:0/1955466444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af63982dc0 0x55af6398a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.374+0000 7ff17246e640 1 -- 172.21.15.43:0/1955466444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af636593c0 con 0x55af6398d680 2024-09-11T23:23:21.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.374+0000 7ff17146c640 1 --2- 172.21.15.43:0/1955466444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af63982dc0 0x55af6398a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:23:21.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.375+0000 7ff17246e640 1 --2- 172.21.15.43:0/1955466444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af6398d680 0x55af6398a760 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff164003c20 tx=0x7ff164005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dceed022118d4496 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:21.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.375+0000 7ff1557fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:23:21.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.375+0000 7ff1557fa640 1 -- 172.21.15.43:0/1955466444 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff164004240 con 0x55af6398d680 2024-09-11T23:23:21.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.375+0000 7ff1557fa640 1 -- 172.21.15.43:0/1955466444 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff1640043e0 con 0x55af6398d680 2024-09-11T23:23:21.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.375+0000 7ff1762aa740 1 -- 172.21.15.43:0/1955466444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff15c018b20 con 0x55af6398d680 2024-09-11T23:23:21.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.375+0000 7ff1557fa640 1 -- 172.21.15.43:0/1955466444 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff1640316e0 con 0x55af6398d680 2024-09-11T23:23:21.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.375+0000 7ff1762aa740 10 client.15708 fetch_fsmap learned FSMap version 26 2024-09-11T23:23:21.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.375+0000 7ff1762aa740 1 -- 172.21.15.43:0/1955466444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55af6399ffd0 con 0x55af6398d680 2024-09-11T23:23:21.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.375+0000 7ff1557fa640 1 -- 172.21.15.43:0/1955466444 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff15c018b20 con 0x55af6398d680 2024-09-11T23:23:21.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.375+0000 7ff1557fa640 1 -- 172.21.15.43:0/1955466444 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff164030070 con 0x55af6398d680 2024-09-11T23:23:21.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.376+0000 7ff1762aa740 10 client.15708 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:23:21.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.376+0000 7ff1762aa740 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:21.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.376+0000 7ff1762aa740 20 client.15708 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:21.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.376+0000 7ff1762aa740 1 --2- 172.21.15.43:0/1955466444 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55af639a13e0 0x55af639a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:21.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.376+0000 7ff1762aa740 4 client.15708 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:23:21.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.376+0000 7ff1762aa740 1 -- 172.21.15.43:0/1955466444 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af638f5850 con 0x55af639a13e0 2024-09-11T23:23:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.376+0000 7ff171c6d640 1 --2- 172.21.15.43:0/1955466444 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55af639a13e0 0x55af639a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.376+0000 7ff171c6d640 1 --2- 172.21.15.43:0/1955466444 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55af639a13e0 0x55af639a3880 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1864b43c15572804 server_cookie=86b13a92a0363b8d in_seq=0 out_seq=0 2024-09-11T23:23:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.376+0000 7ff1557fa640 10 client.15708 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.377+0000 7ff1557fa640 1 -- 172.21.15.43:0/1955466444 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af638f5850 con 0x55af639a13e0 2024-09-11T23:23:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.377+0000 7ff1557fa640 10 client.15708 handle_command_reply: tid=0 2024-09-11T23:23:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.377+0000 7ff1762aa740 1 client.15708 shutdown 2024-09-11T23:23:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.377+0000 7ff1762aa740 1 -- 172.21.15.43:0/1955466444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af6398d680 msgr2=0x55af6398a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.377+0000 7ff1762aa740 1 --2- 172.21.15.43:0/1955466444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af6398d680 0x55af6398a760 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff164003c20 tx=0x7ff164005c50 comp rx=0 tx=0).stop 2024-09-11T23:23:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.377+0000 7ff1762aa740 1 -- 172.21.15.43:0/1955466444 shutdown_connections 2024-09-11T23:23:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.377+0000 7ff1762aa740 1 --2- 172.21.15.43:0/1955466444 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55af639a13e0 0x55af639a3880 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.377+0000 7ff1762aa740 1 --2- 172.21.15.43:0/1955466444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af6398d680 0x55af6398a760 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.377+0000 7ff1762aa740 1 --2- 172.21.15.43:0/1955466444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af63982dc0 0x55af6398a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.377+0000 7ff1557fa640 0 client.15708 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.377+0000 7ff1762aa740 1 --2- 172.21.15.43:0/1955466444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af639824a0 0x55af6398cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.377+0000 7ff1762aa740 1 -- 172.21.15.43:0/1955466444 >> 172.21.15.43:0/1955466444 conn(0x55af6397ea30 msgr2=0x55af63984760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.377+0000 7ff1762aa740 1 -- 172.21.15.43:0/1955466444 shutdown_connections 2024-09-11T23:23:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.377+0000 7ff1762aa740 1 -- 172.21.15.43:0/1955466444 wait complete. 2024-09-11T23:23:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.377+0000 7ff1762aa740 20 client.15708 trim_cache size 0 max 16384 2024-09-11T23:23:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.396+0000 7ff1762aa740 1 Processor -- start 2024-09-11T23:23:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.396+0000 7ff1762aa740 1 -- start start 2024-09-11T23:23:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.396+0000 7ff1762aa740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af639824a0 0x55af6399d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.396+0000 7ff1762aa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af63982dc0 0x55af639a9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.396+0000 7ff1762aa740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af6398d680 0x55af639a9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.396+0000 7ff1762aa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c1ae010 con 0x55af63982dc0 2024-09-11T23:23:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.396+0000 7ff1762aa740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c1ab8e0 con 0x55af6398d680 2024-09-11T23:23:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.396+0000 7ff1762aa740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c1a9730 con 0x55af639824a0 2024-09-11T23:23:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.396+0000 7ff17146c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af63982dc0 0x55af639a9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.396+0000 7ff17146c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af63982dc0 0x55af639a9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35244/0 (socket says 172.21.15.43:35244) 2024-09-11T23:23:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.396+0000 7ff17246e640 1 --2- 172.21.15.43:0/2418315843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af6398d680 0x55af639a9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.396+0000 7ff17146c640 1 -- 172.21.15.43:0/2418315843 learned_addr learned my addr 172.21.15.43:0/2418315843 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.396+0000 7ff171c6d640 1 --2- 172.21.15.43:0/2418315843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af639824a0 0x55af6399d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.396+0000 7ff17246e640 1 -- 172.21.15.43:0/2418315843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af639824a0 msgr2=0x55af6399d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.396+0000 7ff17246e640 1 --2- 172.21.15.43:0/2418315843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af639824a0 0x55af6399d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.396+0000 7ff17246e640 1 -- 172.21.15.43:0/2418315843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af63982dc0 msgr2=0x55af639a9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.396+0000 7ff17246e640 1 --2- 172.21.15.43:0/2418315843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af63982dc0 0x55af639a9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.396+0000 7ff17246e640 1 -- 172.21.15.43:0/2418315843 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af63983640 con 0x55af6398d680 2024-09-11T23:23:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.396+0000 7ff17146c640 1 --2- 172.21.15.43:0/2418315843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af63982dc0 0x55af639a9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:23:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.397+0000 7ff17246e640 1 --2- 172.21.15.43:0/2418315843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af6398d680 0x55af639a9690 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff16402ae00 tx=0x7ff164009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e895330514400f27 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.397+0000 7ff1577fe640 1 -- 172.21.15.43:0/2418315843 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff164004b40 con 0x55af6398d680 2024-09-11T23:23:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.397+0000 7ff1577fe640 1 -- 172.21.15.43:0/2418315843 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff16404b070 con 0x55af6398d680 2024-09-11T23:23:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.397+0000 7ff1577fe640 1 -- 172.21.15.43:0/2418315843 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1640309f0 con 0x55af6398d680 2024-09-11T23:23:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.398+0000 7ff1762aa740 1 -- 172.21.15.43:0/2418315843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af6398d680 msgr2=0x55af639a9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.398+0000 7ff1762aa740 1 --2- 172.21.15.43:0/2418315843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af6398d680 0x55af639a9690 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff16402ae00 tx=0x7ff164009560 comp rx=0 tx=0).stop 2024-09-11T23:23:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.398+0000 7ff1762aa740 1 -- 172.21.15.43:0/2418315843 shutdown_connections 2024-09-11T23:23:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.398+0000 7ff1762aa740 1 --2- 172.21.15.43:0/2418315843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af6398d680 0x55af639a9690 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.398+0000 7ff1762aa740 1 --2- 172.21.15.43:0/2418315843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af63982dc0 0x55af639a9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.398+0000 7ff1762aa740 1 --2- 172.21.15.43:0/2418315843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af639824a0 0x55af6399d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.398+0000 7ff1762aa740 1 -- 172.21.15.43:0/2418315843 >> 172.21.15.43:0/2418315843 conn(0x55af6397ea30 msgr2=0x55af63988af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.398+0000 7ff1762aa740 1 -- 172.21.15.43:0/2418315843 shutdown_connections 2024-09-11T23:23:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.398+0000 7ff1762aa740 1 -- 172.21.15.43:0/2418315843 wait complete. 2024-09-11T23:23:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.398+0000 7ff1762aa740 1 -- start start 2024-09-11T23:23:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.398+0000 7ff1762aa740 1 Processor -- start 2024-09-11T23:23:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.399+0000 7ff1762aa740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af639824a0 0x55af63995870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.399+0000 7ff1762aa740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af63982dc0 0x55af63995db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.399+0000 7ff1762aa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af6398d680 0x55af639962f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.399+0000 7ff1762aa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c1a9da0 con 0x55af6398d680 2024-09-11T23:23:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.399+0000 7ff1762aa740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c1ab210 con 0x55af63982dc0 2024-09-11T23:23:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.399+0000 7ff1762aa740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c1aa470 con 0x55af639824a0 2024-09-11T23:23:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.399+0000 7ff1762aa740 10 client.0 _ll_register_callbacks cb 0x7ff164009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.399+0000 7ff17246e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af6398d680 0x55af639962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.399+0000 7ff17246e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af6398d680 0x55af639962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35256/0 (socket says 172.21.15.43:35256) 2024-09-11T23:23:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.399+0000 7ff17246e640 1 -- 172.21.15.43:0/2546349652 learned_addr learned my addr 172.21.15.43:0/2546349652 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.399+0000 7ff17146c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af63982dc0 0x55af63995db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.399+0000 7ff171c6d640 1 --2- 172.21.15.43:0/2546349652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af639824a0 0x55af63995870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.399+0000 7ff17246e640 1 -- 172.21.15.43:0/2546349652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af639824a0 msgr2=0x55af63995870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.399+0000 7ff17246e640 1 --2- 172.21.15.43:0/2546349652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af639824a0 0x55af63995870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.399+0000 7ff17246e640 1 -- 172.21.15.43:0/2546349652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af63982dc0 msgr2=0x55af63995db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.399+0000 7ff17246e640 1 --2- 172.21.15.43:0/2546349652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af63982dc0 0x55af63995db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.399+0000 7ff17246e640 1 -- 172.21.15.43:0/2546349652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af63a0b220 con 0x55af6398d680 2024-09-11T23:23:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.399+0000 7ff17146c640 1 --2- 172.21.15.43:0/2546349652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af63982dc0 0x55af63995db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:23:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.399+0000 7ff17246e640 1 --2- 172.21.15.43:0/2546349652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af6398d680 0x55af639962f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff164004af0 tx=0x7ff1640305a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101cee4bf64e1826 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.399+0000 7ff1557fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:23:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.400+0000 7ff1557fa640 1 -- 172.21.15.43:0/2546349652 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff164066060 con 0x55af6398d680 2024-09-11T23:23:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.400+0000 7ff1557fa640 1 -- 172.21.15.43:0/2546349652 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff1640306f0 con 0x55af6398d680 2024-09-11T23:23:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.400+0000 7ff1762aa740 1 -- 172.21.15.43:0/2546349652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff16404b070 con 0x55af6398d680 2024-09-11T23:23:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.400+0000 7ff1557fa640 1 -- 172.21.15.43:0/2546349652 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff1640308f0 con 0x55af6398d680 2024-09-11T23:23:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.400+0000 7ff1762aa740 10 client.15714 fetch_fsmap learned FSMap version 26 2024-09-11T23:23:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.400+0000 7ff1762aa740 1 -- 172.21.15.43:0/2546349652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55af63a0c4f0 con 0x55af6398d680 2024-09-11T23:23:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.400+0000 7ff1557fa640 1 -- 172.21.15.43:0/2546349652 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff16404b070 con 0x55af6398d680 2024-09-11T23:23:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.400+0000 7ff1557fa640 1 -- 172.21.15.43:0/2546349652 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff164059cb0 con 0x55af6398d680 2024-09-11T23:23:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.400+0000 7ff1762aa740 10 client.15714 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:23:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.400+0000 7ff1762aa740 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.400+0000 7ff1762aa740 20 client.15714 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.400+0000 7ff1762aa740 1 --2- 172.21.15.43:0/2546349652 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55af63a0cb80 0x55af63a0efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.400+0000 7ff1762aa740 4 client.15714 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:23:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.400+0000 7ff1762aa740 1 -- 172.21.15.43:0/2546349652 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55af6398aaf0 con 0x55af63a0cb80 2024-09-11T23:23:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.401+0000 7ff171c6d640 1 --2- 172.21.15.43:0/2546349652 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55af63a0cb80 0x55af63a0efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.401+0000 7ff171c6d640 1 --2- 172.21.15.43:0/2546349652 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55af63a0cb80 0x55af63a0efa0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61dccb4bc5dc12f3 server_cookie=8697a45700f05bc0 in_seq=0 out_seq=0 2024-09-11T23:23:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.401+0000 7ff1557fa640 10 client.15714 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.401+0000 7ff1557fa640 1 -- 172.21.15.43:0/2546349652 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55af6398aaf0 con 0x55af63a0cb80 2024-09-11T23:23:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.401+0000 7ff1557fa640 10 client.15714 handle_command_reply: tid=0 2024-09-11T23:23:21.422 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:23:21.422 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-11T23:23:21.422 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-11T23:23:21.422 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:23:21.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.401+0000 7ff1762aa740 1 client.15714 shutdown 2024-09-11T23:23:21.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.401+0000 7ff1762aa740 1 -- 172.21.15.43:0/2546349652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af6398d680 msgr2=0x55af639962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:21.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.401+0000 7ff1762aa740 1 --2- 172.21.15.43:0/2546349652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af6398d680 0x55af639962f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff164004af0 tx=0x7ff1640305a0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.401+0000 7ff1762aa740 1 -- 172.21.15.43:0/2546349652 shutdown_connections 2024-09-11T23:23:21.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.401+0000 7ff1762aa740 1 --2- 172.21.15.43:0/2546349652 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55af63a0cb80 0x55af63a0efa0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.401+0000 7ff1762aa740 1 --2- 172.21.15.43:0/2546349652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55af6398d680 0x55af639962f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.401+0000 7ff1762aa740 1 --2- 172.21.15.43:0/2546349652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55af63982dc0 0x55af63995db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.401+0000 7ff1557fa640 0 client.15714 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:21.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.401+0000 7ff1762aa740 1 --2- 172.21.15.43:0/2546349652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af639824a0 0x55af63995870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.401+0000 7ff1762aa740 1 -- 172.21.15.43:0/2546349652 >> 172.21.15.43:0/2546349652 conn(0x55af6397ea30 msgr2=0x55af6397db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.401+0000 7ff1762aa740 1 -- 172.21.15.43:0/2546349652 shutdown_connections 2024-09-11T23:23:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.401+0000 7ff1762aa740 1 -- 172.21.15.43:0/2546349652 wait complete. 2024-09-11T23:23:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.401+0000 7ff1762aa740 20 client.15714 trim_cache size 0 max 16384 2024-09-11T23:23:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.402+0000 7ff173ed1640 1 -- 172.21.15.43:0/4080479935 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff1580a2ea0 msgr2=0x7ff1580a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.402+0000 7ff173ed1640 1 --2- 172.21.15.43:0/4080479935 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff1580a2ea0 0x7ff1580a5360 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff164006010 tx=0x7ff16403e040 comp rx=0 tx=0).stop 2024-09-11T23:23:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.402+0000 7ff173ed1640 1 -- 172.21.15.43:0/4080479935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff16c102d30 msgr2=0x7ff16c1984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.402+0000 7ff173ed1640 1 --2- 172.21.15.43:0/4080479935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff16c102d30 0x7ff16c1984c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff168005fd0 tx=0x7ff16800dde0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.402+0000 7ff173ed1640 1 -- 172.21.15.43:0/4080479935 shutdown_connections 2024-09-11T23:23:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.402+0000 7ff173ed1640 1 --2- 172.21.15.43:0/4080479935 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff1580a2ea0 0x7ff1580a5360 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.402+0000 7ff173ed1640 1 --2- 172.21.15.43:0/4080479935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff16c1082a0 0x7ff16c19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.402+0000 7ff173ed1640 1 --2- 172.21.15.43:0/4080479935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16c1036e0 0x7ff16c198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.402+0000 7ff173ed1640 1 --2- 172.21.15.43:0/4080479935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff16c102d30 0x7ff16c1984c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.402+0000 7ff173ed1640 1 -- 172.21.15.43:0/4080479935 >> 172.21.15.43:0/4080479935 conn(0x7ff16c0fc7d0 msgr2=0x7ff16c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.402+0000 7ff173ed1640 1 -- 172.21.15.43:0/4080479935 shutdown_connections 2024-09-11T23:23:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:21.402+0000 7ff173ed1640 1 -- 172.21.15.43:0/4080479935 wait complete. 2024-09-11T23:23:21.443 INFO:tasks.workunit.client.0.smithi043.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-11T23:23:21.546 INFO:tasks.workunit.client.0.smithi043.stdout:checking dependency style of gcc... gcc3 2024-09-11T23:23:21.590 INFO:tasks.workunit.client.0.smithi043.stdout:checking how to run the C preprocessor... gcc -E 2024-09-11T23:23:21.638 INFO:tasks.workunit.client.0.smithi043.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-11T23:23:21.639 INFO:tasks.workunit.client.0.smithi043.stdout:checking for egrep... /usr/bin/grep -E 2024-09-11T23:23:21.774 INFO:tasks.workunit.client.0.smithi043.stdout:checking for ANSI C header files... yes 2024-09-11T23:23:21.813 INFO:tasks.workunit.client.0.smithi043.stdout:checking for sys/types.h... yes 2024-09-11T23:23:21.853 INFO:tasks.workunit.client.0.smithi043.stdout:checking for sys/stat.h... yes 2024-09-11T23:23:21.891 INFO:tasks.workunit.client.0.smithi043.stdout:checking for stdlib.h... yes 2024-09-11T23:23:21.931 INFO:tasks.workunit.client.0.smithi043.stdout:checking for string.h... yes 2024-09-11T23:23:21.980 INFO:tasks.workunit.client.0.smithi043.stdout:checking for memory.h... yes 2024-09-11T23:23:22.003 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:21 smithi043 ceph-mon[30950]: pgmap v48: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 126 KiB/s wr, 14 op/s 2024-09-11T23:23:22.024 INFO:tasks.workunit.client.0.smithi043.stdout:checking for strings.h... yes 2024-09-11T23:23:22.068 INFO:tasks.workunit.client.0.smithi043.stdout:checking for inttypes.h... yes 2024-09-11T23:23:22.111 INFO:tasks.workunit.client.0.smithi043.stdout:checking for stdint.h... yes 2024-09-11T23:23:22.155 INFO:tasks.workunit.client.0.smithi043.stdout:checking for unistd.h... yes 2024-09-11T23:23:22.210 INFO:tasks.workunit.client.0.smithi043.stdout:checking minix/config.h usability... no 2024-09-11T23:23:22.240 INFO:tasks.workunit.client.0.smithi043.stdout:checking minix/config.h presence... no 2024-09-11T23:23:22.240 INFO:tasks.workunit.client.0.smithi043.stdout:checking for minix/config.h... no 2024-09-11T23:23:22.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:21 smithi090 ceph-mon[35130]: pgmap v48: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 126 KiB/s wr, 14 op/s 2024-09-11T23:23:22.275 INFO:tasks.workunit.client.0.smithi043.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-09-11T23:23:22.275 INFO:tasks.workunit.client.0.smithi043.stdout:checking for gcc... (cached) gcc 2024-09-11T23:23:22.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:21 smithi097 ceph-mon[35562]: pgmap v48: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 126 KiB/s wr, 14 op/s 2024-09-11T23:23:22.308 INFO:tasks.workunit.client.0.smithi043.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-11T23:23:22.308 INFO:tasks.workunit.client.0.smithi043.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-11T23:23:22.309 INFO:tasks.workunit.client.0.smithi043.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-11T23:23:22.309 INFO:tasks.workunit.client.0.smithi043.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-11T23:23:22.359 INFO:tasks.workunit.client.0.smithi043.stdout:checking for main in -lm... yes 2024-09-11T23:23:22.403 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:23:22.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:23:22.405 INFO:tasks.workunit.client.0.smithi043.stdout:checking for main in -lpthread... yes 2024-09-11T23:23:22.407 INFO:tasks.workunit.client.0.smithi043.stdout:checking for ANSI C header files... (cached) yes 2024-09-11T23:23:22.441 INFO:tasks.workunit.client.0.smithi043.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-11T23:23:22.502 INFO:tasks.workunit.client.0.smithi043.stdout:checking pthread.h usability... yes 2024-09-11T23:23:22.520 INFO:tasks.workunit.client.0.smithi043.stdout:checking pthread.h presence... yes 2024-09-11T23:23:22.520 INFO:tasks.workunit.client.0.smithi043.stdout:checking for pthread.h... yes 2024-09-11T23:23:22.566 INFO:tasks.workunit.client.0.smithi043.stdout:checking fcntl.h usability... yes 2024-09-11T23:23:22.583 INFO:tasks.workunit.client.0.smithi043.stdout:checking fcntl.h presence... yes 2024-09-11T23:23:22.583 INFO:tasks.workunit.client.0.smithi043.stdout:checking for fcntl.h... yes 2024-09-11T23:23:22.633 INFO:tasks.workunit.client.0.smithi043.stdout:checking limits.h usability... yes 2024-09-11T23:23:22.650 INFO:tasks.workunit.client.0.smithi043.stdout:checking limits.h presence... yes 2024-09-11T23:23:22.650 INFO:tasks.workunit.client.0.smithi043.stdout:checking for limits.h... yes 2024-09-11T23:23:22.664 INFO:tasks.workunit.client.0.smithi043.stdout:checking for stdint.h... (cached) yes 2024-09-11T23:23:22.681 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:23:22.703 INFO:tasks.workunit.client.0.smithi043.stdout:checking sys/time.h usability... yes 2024-09-11T23:23:22.720 INFO:tasks.workunit.client.0.smithi043.stdout:checking sys/time.h presence... yes 2024-09-11T23:23:22.720 INFO:tasks.workunit.client.0.smithi043.stdout:checking for sys/time.h... yes 2024-09-11T23:23:22.734 INFO:tasks.workunit.client.0.smithi043.stdout:checking for unistd.h... (cached) yes 2024-09-11T23:23:22.786 INFO:tasks.workunit.client.0.smithi043.stdout:checking sys/vfs.h usability... yes 2024-09-11T23:23:22.802 INFO:tasks.workunit.client.0.smithi043.stdout:checking sys/vfs.h presence... yes 2024-09-11T23:23:22.802 INFO:tasks.workunit.client.0.smithi043.stdout:checking for sys/vfs.h... yes 2024-09-11T23:23:22.866 INFO:tasks.workunit.client.0.smithi043.stdout:checking sys/limits.h usability... no 2024-09-11T23:23:22.898 INFO:tasks.workunit.client.0.smithi043.stdout:checking sys/limits.h presence... no 2024-09-11T23:23:22.898 INFO:tasks.workunit.client.0.smithi043.stdout:checking for sys/limits.h... no 2024-09-11T23:23:22.939 INFO:tasks.workunit.client.0.smithi043.stdout:checking for an ANSI C-conforming const... yes 2024-09-11T23:23:22.970 INFO:tasks.workunit.client.0.smithi043.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-11T23:23:23.028 INFO:tasks.workunit.client.0.smithi043.stdout:checking for system... yes 2024-09-11T23:23:23.084 INFO:tasks.workunit.client.0.smithi043.stdout:checking for gettimeofday... yes 2024-09-11T23:23:23.140 INFO:tasks.workunit.client.0.smithi043.stdout:checking for mkdir... yes 2024-09-11T23:23:23.195 INFO:tasks.workunit.client.0.smithi043.stdout:checking for strerror... yes 2024-09-11T23:23:23.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.205+0000 7ff4ccf07640 1 -- 172.21.15.43:0/1291432983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4c8114ab0 msgr2=0x7ff4c8116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:23.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.205+0000 7ff4ccf07640 1 --2- 172.21.15.43:0/1291432983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4c8114ab0 0x7ff4c8116ea0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0009ca0 tx=0x7ff4c002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:23:23.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.205+0000 7ff4ccf07640 1 -- 172.21.15.43:0/1291432983 shutdown_connections 2024-09-11T23:23:23.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.205+0000 7ff4ccf07640 1 --2- 172.21.15.43:0/1291432983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4c8114ab0 0x7ff4c8116ea0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:23.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.205+0000 7ff4ccf07640 1 --2- 172.21.15.43:0/1291432983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4c8109ba0 0x7ff4c8114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:23.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.205+0000 7ff4ccf07640 1 --2- 172.21.15.43:0/1291432983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4c81091f0 0x7ff4c81095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:23.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.205+0000 7ff4ccf07640 1 -- 172.21.15.43:0/1291432983 >> 172.21.15.43:0/1291432983 conn(0x7ff4c806e2d0 msgr2=0x7ff4c806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:23.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.205+0000 7ff4ccf07640 1 -- 172.21.15.43:0/1291432983 shutdown_connections 2024-09-11T23:23:23.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.205+0000 7ff4ccf07640 1 -- 172.21.15.43:0/1291432983 wait complete. 2024-09-11T23:23:23.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.206+0000 7ff4ccf07640 1 Processor -- start 2024-09-11T23:23:23.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.206+0000 7ff4ccf07640 1 -- start start 2024-09-11T23:23:23.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.206+0000 7ff4ccf07640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4c81091f0 0x7ff4c810a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:23.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.206+0000 7ff4ccf07640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4c8109ba0 0x7ff4c810a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:23.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.206+0000 7ff4ccf07640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4c8114ab0 0x7ff4c810ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:23.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.206+0000 7ff4ccf07640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c811a760 con 0x7ff4c8114ab0 2024-09-11T23:23:23.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.206+0000 7ff4ccf07640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c811a5e0 con 0x7ff4c81091f0 2024-09-11T23:23:23.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.206+0000 7ff4ccf07640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c811a8e0 con 0x7ff4c8109ba0 2024-09-11T23:23:23.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.207+0000 7ff4c659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4c81091f0 0x7ff4c810a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:23.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.207+0000 7ff4c6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4c8114ab0 0x7ff4c810ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:23.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.207+0000 7ff4c659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4c81091f0 0x7ff4c810a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:45298/0 (socket says 172.21.15.43:45298) 2024-09-11T23:23:23.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.207+0000 7ff4c6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4c8114ab0 0x7ff4c810ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35288/0 (socket says 172.21.15.43:35288) 2024-09-11T23:23:23.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.207+0000 7ff4c659c640 1 -- 172.21.15.43:0/2985300097 learned_addr learned my addr 172.21.15.43:0/2985300097 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:23.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.207+0000 7ff4bffff640 1 --2- 172.21.15.43:0/2985300097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4c8109ba0 0x7ff4c810a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:23.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.207+0000 7ff4c659c640 1 -- 172.21.15.43:0/2985300097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4c8109ba0 msgr2=0x7ff4c810a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:23.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.207+0000 7ff4c659c640 1 --2- 172.21.15.43:0/2985300097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4c8109ba0 0x7ff4c810a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:23.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.207+0000 7ff4c659c640 1 -- 172.21.15.43:0/2985300097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4c8114ab0 msgr2=0x7ff4c810ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:23.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.207+0000 7ff4c659c640 1 --2- 172.21.15.43:0/2985300097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4c8114ab0 0x7ff4c810ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:23.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.207+0000 7ff4c659c640 1 -- 172.21.15.43:0/2985300097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c81af0a0 con 0x7ff4c81091f0 2024-09-11T23:23:23.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.207+0000 7ff4c659c640 1 --2- 172.21.15.43:0/2985300097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4c81091f0 0x7ff4c810a3d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800d770 tx=0x7ff4b800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:23.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.207+0000 7ff4bf7fe640 1 -- 172.21.15.43:0/2985300097 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8017070 con 0x7ff4c81091f0 2024-09-11T23:23:23.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.207+0000 7ff4bf7fe640 1 -- 172.21.15.43:0/2985300097 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4b8005500 con 0x7ff4c81091f0 2024-09-11T23:23:23.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.207+0000 7ff4ccf07640 1 -- 172.21.15.43:0/2985300097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c81af390 con 0x7ff4c81091f0 2024-09-11T23:23:23.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.207+0000 7ff4bf7fe640 1 -- 172.21.15.43:0/2985300097 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8004e00 con 0x7ff4c81091f0 2024-09-11T23:23:23.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.207+0000 7ff4ccf07640 1 -- 172.21.15.43:0/2985300097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c81af880 con 0x7ff4c81091f0 2024-09-11T23:23:23.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.208+0000 7ff4ccf07640 1 -- 172.21.15.43:0/2985300097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c81095d0 con 0x7ff4c81091f0 2024-09-11T23:23:23.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.209+0000 7ff4bf7fe640 1 -- 172.21.15.43:0/2985300097 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff4b8004fa0 con 0x7ff4c81091f0 2024-09-11T23:23:23.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.210+0000 7ff4bf7fe640 1 --2- 172.21.15.43:0/2985300097 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff4b40a2cf0 0x7ff4b40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:23.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.210+0000 7ff4bffff640 1 --2- 172.21.15.43:0/2985300097 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff4b40a2cf0 0x7ff4b40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:23.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.210+0000 7ff4bf7fe640 1 -- 172.21.15.43:0/2985300097 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff4b80cee80 con 0x7ff4c81091f0 2024-09-11T23:23:23.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.211+0000 7ff4bffff640 1 --2- 172.21.15.43:0/2985300097 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff4b40a2cf0 0x7ff4b40a51b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff4c810be60 tx=0x7ff4b0005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:23.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.214+0000 7ff4bf7fe640 1 -- 172.21.15.43:0/2985300097 <== mon.2 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4b809b3f0 con 0x7ff4c81091f0 2024-09-11T23:23:23.250 INFO:tasks.workunit.client.0.smithi043.stdout:checking for open64... yes 2024-09-11T23:23:23.305 INFO:tasks.workunit.client.0.smithi043.stdout:checking for stat64... yes 2024-09-11T23:23:23.359 INFO:tasks.workunit.client.0.smithi043.stdout:checking for fseeko64... yes 2024-09-11T23:23:23.412 INFO:tasks.workunit.client.0.smithi043.stdout:checking for lrand48_r... yes 2024-09-11T23:23:23.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.430+0000 7ff4ccf07640 1 -- 172.21.15.43:0/2985300097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4c81afbf0 con 0x7ff4c81091f0 2024-09-11T23:23:23.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.430+0000 7ff4bf7fe640 1 -- 172.21.15.43:0/2985300097 <== mon.2 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11899 (secure 0 0 0) 0x7ff4b809f6f0 con 0x7ff4c81091f0 2024-09-11T23:23:23.431 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:23:23.431 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"btime":"2024-09-11T23:23:19:881806+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:23:19.722053+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:23:23.431 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-09-11T23:23:23.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.433+0000 7ff4ccf07640 1 -- 172.21.15.43:0/2985300097 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff4b40a2cf0 msgr2=0x7ff4b40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:23.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.433+0000 7ff4ccf07640 1 --2- 172.21.15.43:0/2985300097 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff4b40a2cf0 0x7ff4b40a51b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff4c810be60 tx=0x7ff4b0005f90 comp rx=0 tx=0).stop 2024-09-11T23:23:23.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.433+0000 7ff4ccf07640 1 -- 172.21.15.43:0/2985300097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4c81091f0 msgr2=0x7ff4c810a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.433+0000 7ff4ccf07640 1 --2- 172.21.15.43:0/2985300097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4c81091f0 0x7ff4c810a3d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800d770 tx=0x7ff4b800dc30 comp rx=0 tx=0).stop 2024-09-11T23:23:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.433+0000 7ff4ccf07640 1 -- 172.21.15.43:0/2985300097 shutdown_connections 2024-09-11T23:23:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.433+0000 7ff4ccf07640 1 --2- 172.21.15.43:0/2985300097 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff4b40a2cf0 0x7ff4b40a51b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.433+0000 7ff4ccf07640 1 --2- 172.21.15.43:0/2985300097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4c8114ab0 0x7ff4c810ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.433+0000 7ff4ccf07640 1 --2- 172.21.15.43:0/2985300097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4c8109ba0 0x7ff4c810a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.433+0000 7ff4ccf07640 1 --2- 172.21.15.43:0/2985300097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4c81091f0 0x7ff4c810a3d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.433+0000 7ff4ccf07640 1 -- 172.21.15.43:0/2985300097 >> 172.21.15.43:0/2985300097 conn(0x7ff4c806e2d0 msgr2=0x7ff4c8070930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.433+0000 7ff4ccf07640 1 -- 172.21.15.43:0/2985300097 shutdown_connections 2024-09-11T23:23:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:23.433+0000 7ff4ccf07640 1 -- 172.21.15.43:0/2985300097 wait complete. 2024-09-11T23:23:23.472 INFO:tasks.workunit.client.0.smithi043.stdout:checking for srand48_r... yes 2024-09-11T23:23:23.494 INFO:tasks.workunit.client.0.smithi043.stdout:configure: creating ./config.status 2024-09-11T23:23:23.554 INFO:tasks.workunit.client.0.smithi043.stdout: /bin/sh ./config.status 2024-09-11T23:23:23.600 INFO:tasks.workunit.client.0.smithi043.stdout:config.status: creating Makefile 2024-09-11T23:23:23.612 INFO:tasks.workunit.client.0.smithi043.stdout:config.status: creating config.h 2024-09-11T23:23:23.632 INFO:tasks.workunit.client.0.smithi043.stdout:config.status: executing depfiles commands 2024-09-11T23:23:23.651 INFO:tasks.workunit.client.0.smithi043.stdout:config.status: executing default commands 2024-09-11T23:23:23.680 INFO:tasks.workunit.client.0.smithi043.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-09-11T23:23:23.890 INFO:tasks.workunit.client.0.smithi043.stdout:rm -f stamp-h1 2024-09-11T23:23:23.893 INFO:tasks.workunit.client.0.smithi043.stdout:touch config.h.in 2024-09-11T23:23:23.895 INFO:tasks.workunit.client.0.smithi043.stdout:cd . && /bin/sh ./config.status config.h 2024-09-11T23:23:23.936 INFO:tasks.workunit.client.0.smithi043.stdout:config.status: creating config.h 2024-09-11T23:23:23.962 INFO:tasks.workunit.client.0.smithi043.stdout:make all-am 2024-09-11T23:23:23.969 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-11T23:23:23.969 INFO:tasks.workunit.client.0.smithi043.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-09-11T23:23:24.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:23 smithi043 ceph-mon[30950]: pgmap v49: 97 pgs: 97 active+clean; 6.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 491 KiB/s wr, 123 op/s 2024-09-11T23:23:24.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:23 smithi043 ceph-mon[30950]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-11T23:23:24.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:23 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2985300097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:23:24.206 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:23:24.225 INFO:tasks.workunit.client.0.smithi043.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-09-11T23:23:24.230 INFO:tasks.workunit.client.0.smithi043.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-09-11T23:23:24.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:23 smithi090 ceph-mon[35130]: pgmap v49: 97 pgs: 97 active+clean; 6.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 491 KiB/s wr, 123 op/s 2024-09-11T23:23:24.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:23 smithi090 ceph-mon[35130]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-11T23:23:24.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:23 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2985300097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:23:24.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:23 smithi097 ceph-mon[35562]: pgmap v49: 97 pgs: 97 active+clean; 6.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 491 KiB/s wr, 123 op/s 2024-09-11T23:23:24.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:23 smithi097 ceph-mon[35562]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-11T23:23:24.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:23 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2985300097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:23:24.327 INFO:tasks.workunit.client.0.smithi043.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-09-11T23:23:24.336 INFO:tasks.workunit.client.0.smithi043.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-09-11T23:23:24.424 INFO:tasks.workunit.client.0.smithi043.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-09-11T23:23:24.432 INFO:tasks.workunit.client.0.smithi043.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-09-11T23:23:24.489 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:23:24.529 INFO:tasks.workunit.client.0.smithi043.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-09-11T23:23:24.537 INFO:tasks.workunit.client.0.smithi043.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-09-11T23:23:24.689 INFO:tasks.workunit.client.0.smithi043.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-09-11T23:23:24.697 INFO:tasks.workunit.client.0.smithi043.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-09-11T23:23:24.747 INFO:tasks.workunit.client.0.smithi043.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-09-11T23:23:24.754 INFO:tasks.workunit.client.0.smithi043.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-09-11T23:23:24.799 INFO:tasks.workunit.client.0.smithi043.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-09-11T23:23:24.806 INFO:tasks.workunit.client.0.smithi043.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-09-11T23:23:24.854 INFO:tasks.workunit.client.0.smithi043.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-09-11T23:23:24.861 INFO:tasks.workunit.client.0.smithi043.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-09-11T23:23:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:24.998+0000 7fea38770640 1 -- 172.21.15.43:0/1952705906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea3010e4f0 msgr2=0x7fea30110900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:24.998+0000 7fea38770640 1 --2- 172.21.15.43:0/1952705906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea3010e4f0 0x7fea30110900 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fea28009ca0 tx=0x7fea2802a8c0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:24.999+0000 7fea38770640 1 -- 172.21.15.43:0/1952705906 shutdown_connections 2024-09-11T23:23:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:24.999+0000 7fea38770640 1 --2- 172.21.15.43:0/1952705906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea3010e4f0 0x7fea30110900 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:24.999+0000 7fea38770640 1 --2- 172.21.15.43:0/1952705906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea30101380 0x7fea3010dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:24.999+0000 7fea38770640 1 --2- 172.21.15.43:0/1952705906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea301009d0 0x7fea30100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:24.999+0000 7fea38770640 1 -- 172.21.15.43:0/1952705906 >> 172.21.15.43:0/1952705906 conn(0x7fea300fc7d0 msgr2=0x7fea300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:24.999+0000 7fea38770640 1 -- 172.21.15.43:0/1952705906 shutdown_connections 2024-09-11T23:23:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:24.999+0000 7fea38770640 1 -- 172.21.15.43:0/1952705906 wait complete. 2024-09-11T23:23:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:24.999+0000 7fea38770640 1 Processor -- start 2024-09-11T23:23:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:24.999+0000 7fea38770640 1 -- start start 2024-09-11T23:23:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.000+0000 7fea38770640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea301009d0 0x7fea3019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.000+0000 7fea38770640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea30101380 0x7fea3019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.000+0000 7fea38770640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea3010e4f0 0x7fea3019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.000+0000 7fea38770640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fea301142a0 con 0x7fea3010e4f0 2024-09-11T23:23:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.000+0000 7fea38770640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fea30114120 con 0x7fea301009d0 2024-09-11T23:23:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.000+0000 7fea38770640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fea30114420 con 0x7fea30101380 2024-09-11T23:23:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.000+0000 7fea36d0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea3010e4f0 0x7fea3019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.000+0000 7fea36d0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea3010e4f0 0x7fea3019f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35300/0 (socket says 172.21.15.43:35300) 2024-09-11T23:23:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.000+0000 7fea36d0d640 1 -- 172.21.15.43:0/1284663614 learned_addr learned my addr 172.21.15.43:0/1284663614 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.000+0000 7fea35d0b640 1 --2- 172.21.15.43:0/1284663614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea30101380 0x7fea3019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.000+0000 7fea3650c640 1 --2- 172.21.15.43:0/1284663614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea301009d0 0x7fea3019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.000+0000 7fea36d0d640 1 -- 172.21.15.43:0/1284663614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea30101380 msgr2=0x7fea3019ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.000+0000 7fea36d0d640 1 --2- 172.21.15.43:0/1284663614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea30101380 0x7fea3019ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.000+0000 7fea36d0d640 1 -- 172.21.15.43:0/1284663614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea301009d0 msgr2=0x7fea3019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.000+0000 7fea36d0d640 1 --2- 172.21.15.43:0/1284663614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea301009d0 0x7fea3019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.000+0000 7fea36d0d640 1 -- 172.21.15.43:0/1284663614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3019f6e0 con 0x7fea3010e4f0 2024-09-11T23:23:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.001+0000 7fea36d0d640 1 --2- 172.21.15.43:0/1284663614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea3010e4f0 0x7fea3019f000 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fea28031f80 tx=0x7fea28031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.001+0000 7fea277fe640 1 -- 172.21.15.43:0/1284663614 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28003b70 con 0x7fea3010e4f0 2024-09-11T23:23:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.001+0000 7fea277fe640 1 -- 172.21.15.43:0/1284663614 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea28003d10 con 0x7fea3010e4f0 2024-09-11T23:23:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.001+0000 7fea38770640 1 -- 172.21.15.43:0/1284663614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3019f970 con 0x7fea3010e4f0 2024-09-11T23:23:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.001+0000 7fea277fe640 1 -- 172.21.15.43:0/1284663614 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28004760 con 0x7fea3010e4f0 2024-09-11T23:23:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.001+0000 7fea38770640 1 -- 172.21.15.43:0/1284663614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea301a72b0 con 0x7fea3010e4f0 2024-09-11T23:23:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.003+0000 7fea277fe640 1 -- 172.21.15.43:0/1284663614 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fea2802f710 con 0x7fea3010e4f0 2024-09-11T23:23:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.004+0000 7fea277fe640 1 --2- 172.21.15.43:0/1284663614 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fea1c0a2cf0 0x7fea1c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.004+0000 7fea3650c640 1 --2- 172.21.15.43:0/1284663614 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fea1c0a2cf0 0x7fea1c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.004+0000 7fea277fe640 1 -- 172.21.15.43:0/1284663614 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fea280f05a0 con 0x7fea3010e4f0 2024-09-11T23:23:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.004+0000 7fea3650c640 1 --2- 172.21.15.43:0/1284663614 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fea1c0a2cf0 0x7fea1c0a51b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0058c0 tx=0x7fea2c00a2d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.004+0000 7fea3ab49740 1 Processor -- start 2024-09-11T23:23:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.005+0000 7fea3ab49740 1 -- start start 2024-09-11T23:23:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.005+0000 7fea3ab49740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 0x55778b8ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.005+0000 7fea3ab49740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 0x55778b8caa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.005+0000 7fea3ab49740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 0x55778b8d7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.005+0000 7fea3ab49740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55778b8302f0 con 0x55778b8d5370 2024-09-11T23:23:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.005+0000 7fea3ab49740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55778b7d7cf0 con 0x55778b8ca5d0 2024-09-11T23:23:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.005+0000 7fea3ab49740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55778b82c2b0 con 0x55778b8cbb40 2024-09-11T23:23:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.005+0000 7fea3650c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 0x55778b8ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.005+0000 7fea36d0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 0x55778b8d7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.005+0000 7fea3650c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 0x55778b8ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:54844/0 (socket says 172.21.15.43:54844) 2024-09-11T23:23:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.005+0000 7fea3650c640 1 -- 172.21.15.43:0/1117650501 learned_addr learned my addr 172.21.15.43:0/1117650501 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.005+0000 7fea35d0b640 1 --2- 172.21.15.43:0/1117650501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 0x55778b8caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.006+0000 7fea35d0b640 1 -- 172.21.15.43:0/1117650501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 msgr2=0x55778b8ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.006+0000 7fea35d0b640 1 --2- 172.21.15.43:0/1117650501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 0x55778b8ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.006+0000 7fea35d0b640 1 -- 172.21.15.43:0/1117650501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 msgr2=0x55778b8d7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.006+0000 7fea35d0b640 1 --2- 172.21.15.43:0/1117650501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 0x55778b8d7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.006+0000 7fea35d0b640 1 -- 172.21.15.43:0/1117650501 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55778b7bc3a0 con 0x55778b8ca5d0 2024-09-11T23:23:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.006+0000 7fea36d0d640 1 --2- 172.21.15.43:0/1117650501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 0x55778b8d7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:23:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.006+0000 7fea35d0b640 1 --2- 172.21.15.43:0/1117650501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 0x55778b8caa10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fea20009970 tx=0x7fea20009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8556e0c200229265 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.006+0000 7fea24ff9640 1 -- 172.21.15.43:0/1117650501 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20017070 con 0x55778b8ca5d0 2024-09-11T23:23:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.006+0000 7fea24ff9640 1 -- 172.21.15.43:0/1117650501 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea20013470 con 0x55778b8ca5d0 2024-09-11T23:23:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.006+0000 7fea24ff9640 1 -- 172.21.15.43:0/1117650501 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20012630 con 0x55778b8ca5d0 2024-09-11T23:23:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.007+0000 7fea3ab49740 1 -- 172.21.15.43:0/1117650501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 msgr2=0x55778b8caa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.007+0000 7fea3ab49740 1 --2- 172.21.15.43:0/1117650501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 0x55778b8caa10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fea20009970 tx=0x7fea20009e30 comp rx=0 tx=0).stop 2024-09-11T23:23:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.007+0000 7fea3ab49740 1 -- 172.21.15.43:0/1117650501 shutdown_connections 2024-09-11T23:23:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.007+0000 7fea3ab49740 1 --2- 172.21.15.43:0/1117650501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 0x55778b8d7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.007+0000 7fea3ab49740 1 --2- 172.21.15.43:0/1117650501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 0x55778b8caa10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.007+0000 7fea3ab49740 1 --2- 172.21.15.43:0/1117650501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 0x55778b8ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.007+0000 7fea3ab49740 1 -- 172.21.15.43:0/1117650501 >> 172.21.15.43:0/1117650501 conn(0x55778b893310 msgr2=0x55778b8c6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.007+0000 7fea3ab49740 1 -- 172.21.15.43:0/1117650501 shutdown_connections 2024-09-11T23:23:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.007+0000 7fea3ab49740 1 -- 172.21.15.43:0/1117650501 wait complete. 2024-09-11T23:23:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.008+0000 7fea3ab49740 1 -- start start 2024-09-11T23:23:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.008+0000 7fea3ab49740 1 Processor -- start 2024-09-11T23:23:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.008+0000 7fea3ab49740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 0x55778b8d4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.008+0000 7fea3ab49740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 0x55778b8d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.008+0000 7fea3ab49740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 0x55778b8d2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.008+0000 7fea3ab49740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55778b8d8070 con 0x55778b8d5370 2024-09-11T23:23:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.008+0000 7fea3ab49740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55778b7eb060 con 0x55778b8ca5d0 2024-09-11T23:23:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.008+0000 7fea3ab49740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55778b8d81f0 con 0x55778b8cbb40 2024-09-11T23:23:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.008+0000 7fea3ab49740 10 client.0 _ll_register_callbacks cb 0x55778b53e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.008+0000 7fea36d0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 0x55778b8d2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.008+0000 7fea36d0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 0x55778b8d2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35322/0 (socket says 172.21.15.43:35322) 2024-09-11T23:23:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.008+0000 7fea36d0d640 1 -- 172.21.15.43:0/1235802707 learned_addr learned my addr 172.21.15.43:0/1235802707 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:25.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.008+0000 7fea3650c640 1 --2- 172.21.15.43:0/1235802707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 0x55778b8d4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.008+0000 7fea35d0b640 1 --2- 172.21.15.43:0/1235802707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 0x55778b8d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.009+0000 7fea36d0d640 1 -- 172.21.15.43:0/1235802707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 msgr2=0x55778b8d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:25.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.009+0000 7fea36d0d640 1 --2- 172.21.15.43:0/1235802707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 0x55778b8d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.009+0000 7fea36d0d640 1 -- 172.21.15.43:0/1235802707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 msgr2=0x55778b8d4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:25.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.009+0000 7fea36d0d640 1 --2- 172.21.15.43:0/1235802707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 0x55778b8d4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.009+0000 7fea36d0d640 1 -- 172.21.15.43:0/1235802707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55778b5a13c0 con 0x55778b8d5370 2024-09-11T23:23:25.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.009+0000 7fea35d0b640 1 --2- 172.21.15.43:0/1235802707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 0x55778b8d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:23:25.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.009+0000 7fea36d0d640 1 --2- 172.21.15.43:0/1235802707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 0x55778b8d2530 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fea28006010 tx=0x7fea280bcb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d852e76df4679e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:25.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.009+0000 7fea19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:23:25.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.009+0000 7fea19ffb640 1 -- 172.21.15.43:0/1235802707 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2802fe80 con 0x55778b8d5370 2024-09-11T23:23:25.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.009+0000 7fea19ffb640 1 -- 172.21.15.43:0/1235802707 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fea280c5470 con 0x55778b8d5370 2024-09-11T23:23:25.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.009+0000 7fea3ab49740 1 -- 172.21.15.43:0/1235802707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea20013470 con 0x55778b8d5370 2024-09-11T23:23:25.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.009+0000 7fea19ffb640 1 -- 172.21.15.43:0/1235802707 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fea280098d0 con 0x55778b8d5370 2024-09-11T23:23:25.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.009+0000 7fea3ab49740 10 client.15732 fetch_fsmap learned FSMap version 26 2024-09-11T23:23:25.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.009+0000 7fea3ab49740 1 -- 172.21.15.43:0/1235802707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55778b8d8070 con 0x55778b8d5370 2024-09-11T23:23:25.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.009+0000 7fea19ffb640 1 -- 172.21.15.43:0/1235802707 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea28036040 con 0x55778b8d5370 2024-09-11T23:23:25.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.010+0000 7fea19ffb640 1 -- 172.21.15.43:0/1235802707 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fea2809bbd0 con 0x55778b8d5370 2024-09-11T23:23:25.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.010+0000 7fea3ab49740 10 client.15732 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:23:25.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.010+0000 7fea3ab49740 10 client.15732 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T23:23:25.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.010+0000 7fea3ab49740 20 client.15732 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:25.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.010+0000 7fea3ab49740 1 --2- 172.21.15.43:0/1235802707 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55778b8e4f60 0x55778b8e7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.010+0000 7fea3ab49740 4 client.15732 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:23:25.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.010+0000 7fea3ab49740 1 -- 172.21.15.43:0/1235802707 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fea20013470 con 0x55778b8e4f60 2024-09-11T23:23:25.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.010+0000 7fea3650c640 1 --2- 172.21.15.43:0/1235802707 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55778b8e4f60 0x55778b8e7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.010+0000 7fea19ffb640 10 client.15732 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:23:25.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.010+0000 7fea3650c640 1 --2- 172.21.15.43:0/1235802707 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55778b8e4f60 0x55778b8e7370 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a980a67dd46513d5 server_cookie=e01b8812de9a84dc in_seq=0 out_seq=0 2024-09-11T23:23:25.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.011+0000 7fea19ffb640 1 -- 172.21.15.43:0/1235802707 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fea20013470 con 0x55778b8e4f60 2024-09-11T23:23:25.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.011+0000 7fea19ffb640 10 client.15732 handle_command_reply: tid=0 2024-09-11T23:23:25.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.011+0000 7fea3ab49740 1 client.15732 shutdown 2024-09-11T23:23:25.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.011+0000 7fea3ab49740 1 -- 172.21.15.43:0/1235802707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 msgr2=0x55778b8d2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:25.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.011+0000 7fea3ab49740 1 --2- 172.21.15.43:0/1235802707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 0x55778b8d2530 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fea28006010 tx=0x7fea280bcb90 comp rx=0 tx=0).stop 2024-09-11T23:23:25.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.011+0000 7fea3ab49740 1 -- 172.21.15.43:0/1235802707 shutdown_connections 2024-09-11T23:23:25.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.012+0000 7fea3ab49740 1 --2- 172.21.15.43:0/1235802707 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55778b8e4f60 0x55778b8e7370 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.012+0000 7fea3ab49740 1 --2- 172.21.15.43:0/1235802707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 0x55778b8d2530 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.012+0000 7fea3ab49740 1 --2- 172.21.15.43:0/1235802707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 0x55778b8d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.012+0000 7fea3ab49740 1 --2- 172.21.15.43:0/1235802707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 0x55778b8d4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.012+0000 7fea3ab49740 1 -- 172.21.15.43:0/1235802707 >> 172.21.15.43:0/1235802707 conn(0x55778b893310 msgr2=0x55778b8cc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:25.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.012+0000 7fea19ffb640 0 client.15732 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:23:25.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.012+0000 7fea3ab49740 1 -- 172.21.15.43:0/1235802707 shutdown_connections 2024-09-11T23:23:25.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.012+0000 7fea3ab49740 1 -- 172.21.15.43:0/1235802707 wait complete. 2024-09-11T23:23:25.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.012+0000 7fea3ab49740 20 client.15732 trim_cache size 0 max 16384 2024-09-11T23:23:25.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.025+0000 7fea3ab49740 1 Processor -- start 2024-09-11T23:23:25.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.025+0000 7fea3ab49740 1 -- start start 2024-09-11T23:23:25.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.025+0000 7fea3ab49740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 0x55778b8d3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.025+0000 7fea3ab49740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 0x55778b8d6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.025+0000 7fea3ab49740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 0x55778b8d6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.025+0000 7fea3ab49740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fea301b2360 con 0x55778b8d5370 2024-09-11T23:23:25.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.025+0000 7fea3ab49740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fea301afc30 con 0x55778b8ca5d0 2024-09-11T23:23:25.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.025+0000 7fea3ab49740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fea301ada80 con 0x55778b8cbb40 2024-09-11T23:23:25.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.025+0000 7fea36d0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 0x55778b8d6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.025+0000 7fea3650c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 0x55778b8d3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.025+0000 7fea3650c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 0x55778b8d3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:45338/0 (socket says 172.21.15.43:45338) 2024-09-11T23:23:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.025+0000 7fea3650c640 1 -- 172.21.15.43:0/29538078 learned_addr learned my addr 172.21.15.43:0/29538078 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.025+0000 7fea35d0b640 1 --2- 172.21.15.43:0/29538078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 0x55778b8d6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.026+0000 7fea3650c640 1 -- 172.21.15.43:0/29538078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 msgr2=0x55778b8d6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.026+0000 7fea3650c640 1 --2- 172.21.15.43:0/29538078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 0x55778b8d6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.026+0000 7fea3650c640 1 -- 172.21.15.43:0/29538078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 msgr2=0x55778b8d6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.026+0000 7fea3650c640 1 --2- 172.21.15.43:0/29538078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 0x55778b8d6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.026+0000 7fea3650c640 1 -- 172.21.15.43:0/29538078 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55778b8e71a0 con 0x55778b8ca5d0 2024-09-11T23:23:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.026+0000 7fea3650c640 1 --2- 172.21.15.43:0/29538078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 0x55778b8d3f20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00f790 tx=0x7fea2c012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f072945123081b30 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.026+0000 7fea1bfff640 1 -- 172.21.15.43:0/29538078 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c016070 con 0x55778b8ca5d0 2024-09-11T23:23:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.026+0000 7fea1bfff640 1 -- 172.21.15.43:0/29538078 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea2c005500 con 0x55778b8ca5d0 2024-09-11T23:23:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.026+0000 7fea1bfff640 1 -- 172.21.15.43:0/29538078 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0316c0 con 0x55778b8ca5d0 2024-09-11T23:23:25.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.027+0000 7fea3ab49740 1 -- 172.21.15.43:0/29538078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 msgr2=0x55778b8d3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:25.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.027+0000 7fea3ab49740 1 --2- 172.21.15.43:0/29538078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 0x55778b8d3f20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00f790 tx=0x7fea2c012000 comp rx=0 tx=0).stop 2024-09-11T23:23:25.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.027+0000 7fea3ab49740 1 -- 172.21.15.43:0/29538078 shutdown_connections 2024-09-11T23:23:25.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.027+0000 7fea3ab49740 1 --2- 172.21.15.43:0/29538078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 0x55778b8d6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.027+0000 7fea3ab49740 1 --2- 172.21.15.43:0/29538078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 0x55778b8d6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.027+0000 7fea3ab49740 1 --2- 172.21.15.43:0/29538078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 0x55778b8d3f20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.027+0000 7fea3ab49740 1 -- 172.21.15.43:0/29538078 >> 172.21.15.43:0/29538078 conn(0x55778b893310 msgr2=0x55778b8d3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:25.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.027+0000 7fea3ab49740 1 -- 172.21.15.43:0/29538078 shutdown_connections 2024-09-11T23:23:25.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.027+0000 7fea3ab49740 1 -- 172.21.15.43:0/29538078 wait complete. 2024-09-11T23:23:25.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.028+0000 7fea3ab49740 1 -- start start 2024-09-11T23:23:25.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.028+0000 7fea3ab49740 1 Processor -- start 2024-09-11T23:23:25.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.028+0000 7fea3ab49740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 0x55778b8ec9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.028+0000 7fea3ab49740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 0x55778b8ecef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.028+0000 7fea3ab49740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 0x55778b8ed430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.028+0000 7fea3ab49740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fea301ae0f0 con 0x55778b8d5370 2024-09-11T23:23:25.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.028+0000 7fea3ab49740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fea301af560 con 0x55778b8ca5d0 2024-09-11T23:23:25.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.028+0000 7fea3ab49740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fea301ae7c0 con 0x55778b8cbb40 2024-09-11T23:23:25.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.028+0000 7fea36d0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 0x55778b8ed430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.028+0000 7fea3ab49740 10 client.0 _ll_register_callbacks cb 0x7fea28009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:25.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.028+0000 7fea36d0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 0x55778b8ed430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35350/0 (socket says 172.21.15.43:35350) 2024-09-11T23:23:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.028+0000 7fea36d0d640 1 -- 172.21.15.43:0/2408065410 learned_addr learned my addr 172.21.15.43:0/2408065410 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.028+0000 7fea35d0b640 1 --2- 172.21.15.43:0/2408065410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 0x55778b8ecef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.028+0000 7fea3650c640 1 --2- 172.21.15.43:0/2408065410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 0x55778b8ec9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.029+0000 7fea36d0d640 1 -- 172.21.15.43:0/2408065410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 msgr2=0x55778b8ecef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.029+0000 7fea36d0d640 1 --2- 172.21.15.43:0/2408065410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 0x55778b8ecef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.029+0000 7fea36d0d640 1 -- 172.21.15.43:0/2408065410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 msgr2=0x55778b8ec9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.029+0000 7fea36d0d640 1 --2- 172.21.15.43:0/2408065410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 0x55778b8ec9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.029+0000 7fea36d0d640 1 -- 172.21.15.43:0/2408065410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55778b8dbd40 con 0x55778b8d5370 2024-09-11T23:23:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.029+0000 7fea36d0d640 1 --2- 172.21.15.43:0/2408065410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 0x55778b8ed430 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fea28009430 tx=0x7fea280edea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c8b6f6bfa75e9a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.029+0000 7fea19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:23:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.029+0000 7fea19ffb640 1 -- 172.21.15.43:0/2408065410 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2809f060 con 0x55778b8d5370 2024-09-11T23:23:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.029+0000 7fea19ffb640 1 -- 172.21.15.43:0/2408065410 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fea280bc0e0 con 0x55778b8d5370 2024-09-11T23:23:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.029+0000 7fea3ab49740 1 -- 172.21.15.43:0/2408065410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea2c005500 con 0x55778b8d5370 2024-09-11T23:23:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.029+0000 7fea19ffb640 1 -- 172.21.15.43:0/2408065410 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fea280bc7d0 con 0x55778b8d5370 2024-09-11T23:23:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.029+0000 7fea3ab49740 10 client.15738 fetch_fsmap learned FSMap version 26 2024-09-11T23:23:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.029+0000 7fea3ab49740 1 -- 172.21.15.43:0/2408065410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55778b950100 con 0x55778b8d5370 2024-09-11T23:23:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.029+0000 7fea19ffb640 1 -- 172.21.15.43:0/2408065410 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea2c005500 con 0x55778b8d5370 2024-09-11T23:23:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.029+0000 7fea19ffb640 1 -- 172.21.15.43:0/2408065410 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fea280c5b50 con 0x55778b8d5370 2024-09-11T23:23:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.030+0000 7fea3ab49740 10 client.15738 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:23:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.030+0000 7fea3ab49740 10 client.15738 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T23:23:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.030+0000 7fea3ab49740 20 client.15738 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.030+0000 7fea3ab49740 1 --2- 172.21.15.43:0/2408065410 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55778b950630 0x55778b952a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.030+0000 7fea3ab49740 4 client.15738 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:23:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.030+0000 7fea3ab49740 1 -- 172.21.15.43:0/2408065410 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55778b83d860 con 0x55778b950630 2024-09-11T23:23:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.030+0000 7fea3650c640 1 --2- 172.21.15.43:0/2408065410 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55778b950630 0x55778b952a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.030+0000 7fea19ffb640 10 client.15738 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:23:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.030+0000 7fea3650c640 1 --2- 172.21.15.43:0/2408065410 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55778b950630 0x55778b952a20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=90827d5c8be0f203 server_cookie=6944b865983af210 in_seq=0 out_seq=0 2024-09-11T23:23:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.031+0000 7fea19ffb640 1 -- 172.21.15.43:0/2408065410 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55778b83d860 con 0x55778b950630 2024-09-11T23:23:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.031+0000 7fea19ffb640 10 client.15738 handle_command_reply: tid=0 2024-09-11T23:23:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.031+0000 7fea3ab49740 1 client.15738 shutdown 2024-09-11T23:23:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.031+0000 7fea3ab49740 1 -- 172.21.15.43:0/2408065410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 msgr2=0x55778b8ed430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.031+0000 7fea3ab49740 1 --2- 172.21.15.43:0/2408065410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 0x55778b8ed430 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fea28009430 tx=0x7fea280edea0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.031+0000 7fea3ab49740 1 -- 172.21.15.43:0/2408065410 shutdown_connections 2024-09-11T23:23:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.031+0000 7fea3ab49740 1 --2- 172.21.15.43:0/2408065410 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55778b950630 0x55778b952a20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.031+0000 7fea3ab49740 1 --2- 172.21.15.43:0/2408065410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55778b8d5370 0x55778b8ed430 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.031+0000 7fea3ab49740 1 --2- 172.21.15.43:0/2408065410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55778b8cbb40 0x55778b8ecef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.031+0000 7fea3ab49740 1 --2- 172.21.15.43:0/2408065410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55778b8ca5d0 0x55778b8ec9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.031+0000 7fea3ab49740 1 -- 172.21.15.43:0/2408065410 >> 172.21.15.43:0/2408065410 conn(0x55778b893310 msgr2=0x55778b8c5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:25.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.031+0000 7fea19ffb640 0 client.15738 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:23:25.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.031+0000 7fea3ab49740 1 -- 172.21.15.43:0/2408065410 shutdown_connections 2024-09-11T23:23:25.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.031+0000 7fea3ab49740 1 -- 172.21.15.43:0/2408065410 wait complete. 2024-09-11T23:23:25.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.031+0000 7fea3ab49740 20 client.15738 trim_cache size 0 max 16384 2024-09-11T23:23:25.036 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:23:25.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.032+0000 7fea38770640 1 -- 172.21.15.43:0/1284663614 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fea1c0a2cf0 msgr2=0x7fea1c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:25.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.032+0000 7fea38770640 1 --2- 172.21.15.43:0/1284663614 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fea1c0a2cf0 0x7fea1c0a51b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0058c0 tx=0x7fea2c00a2d0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.032+0000 7fea38770640 1 -- 172.21.15.43:0/1284663614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea3010e4f0 msgr2=0x7fea3019f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:25.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.032+0000 7fea38770640 1 --2- 172.21.15.43:0/1284663614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea3010e4f0 0x7fea3019f000 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fea28031f80 tx=0x7fea28031a60 comp rx=0 tx=0).stop 2024-09-11T23:23:25.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.032+0000 7fea38770640 1 -- 172.21.15.43:0/1284663614 shutdown_connections 2024-09-11T23:23:25.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.032+0000 7fea38770640 1 --2- 172.21.15.43:0/1284663614 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fea1c0a2cf0 0x7fea1c0a51b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.032+0000 7fea38770640 1 --2- 172.21.15.43:0/1284663614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea3010e4f0 0x7fea3019f000 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.032+0000 7fea38770640 1 --2- 172.21.15.43:0/1284663614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea30101380 0x7fea3019ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.032+0000 7fea38770640 1 --2- 172.21.15.43:0/1284663614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea301009d0 0x7fea3019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:25.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.032+0000 7fea38770640 1 -- 172.21.15.43:0/1284663614 >> 172.21.15.43:0/1284663614 conn(0x7fea300fc7d0 msgr2=0x7fea300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:25.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.032+0000 7fea38770640 1 -- 172.21.15.43:0/1284663614 shutdown_connections 2024-09-11T23:23:25.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:25.032+0000 7fea38770640 1 -- 172.21.15.43:0/1284663614 wait complete. 2024-09-11T23:23:25.080 INFO:tasks.workunit.client.0.smithi043.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-09-11T23:23:25.089 INFO:tasks.workunit.client.0.smithi043.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-09-11T23:23:25.223 INFO:tasks.workunit.client.0.smithi043.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-09-11T23:23:25.232 INFO:tasks.workunit.client.0.smithi043.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-09-11T23:23:25.385 INFO:tasks.workunit.client.0.smithi043.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-09-11T23:23:25.394 INFO:tasks.workunit.client.0.smithi043.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-09-11T23:23:25.463 INFO:tasks.workunit.client.0.smithi043.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-09-11T23:23:25.471 INFO:tasks.workunit.client.0.smithi043.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-09-11T23:23:25.551 INFO:tasks.workunit.client.0.smithi043.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-09-11T23:23:25.559 INFO:tasks.workunit.client.0.smithi043.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-09-11T23:23:25.668 INFO:tasks.workunit.client.0.smithi043.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-09-11T23:23:25.677 INFO:tasks.workunit.client.0.smithi043.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-09-11T23:23:25.704 INFO:tasks.workunit.client.0.smithi043.stderr:In file included from parser.h:22, 2024-09-11T23:23:25.705 INFO:tasks.workunit.client.0.smithi043.stderr: from parser.c:25: 2024-09-11T23:23:25.705 INFO:tasks.workunit.client.0.smithi043.stderr:parser.c: In function ‘init_filesys’: 2024-09-11T23:23:25.705 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-11T23:23:25.705 INFO:tasks.workunit.client.0.smithi043.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-11T23:23:25.705 INFO:tasks.workunit.client.0.smithi043.stderr: | ^ 2024-09-11T23:23:25.705 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-11T23:23:25.705 INFO:tasks.workunit.client.0.smithi043.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-11T23:23:25.705 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~ 2024-09-11T23:23:25.705 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-11T23:23:25.705 INFO:tasks.workunit.client.0.smithi043.stderr: 36 | container_of(ptr, type, member) 2024-09-11T23:23:25.705 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~~~ 2024-09-11T23:23:25.706 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-11T23:23:25.706 INFO:tasks.workunit.client.0.smithi043.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-11T23:23:25.706 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~ 2024-09-11T23:23:25.706 INFO:tasks.workunit.client.0.smithi043.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-11T23:23:25.706 INFO:tasks.workunit.client.0.smithi043.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-11T23:23:25.706 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-11T23:23:25.706 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-11T23:23:25.706 INFO:tasks.workunit.client.0.smithi043.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-11T23:23:25.706 INFO:tasks.workunit.client.0.smithi043.stderr: | ^ 2024-09-11T23:23:25.706 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-11T23:23:25.706 INFO:tasks.workunit.client.0.smithi043.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-11T23:23:25.706 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~ 2024-09-11T23:23:25.706 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-11T23:23:25.706 INFO:tasks.workunit.client.0.smithi043.stderr: 36 | container_of(ptr, type, member) 2024-09-11T23:23:25.707 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~~~ 2024-09-11T23:23:25.707 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-11T23:23:25.707 INFO:tasks.workunit.client.0.smithi043.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-11T23:23:25.707 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~ 2024-09-11T23:23:25.707 INFO:tasks.workunit.client.0.smithi043.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-11T23:23:25.707 INFO:tasks.workunit.client.0.smithi043.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-11T23:23:25.707 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-11T23:23:25.707 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-11T23:23:25.707 INFO:tasks.workunit.client.0.smithi043.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-11T23:23:25.707 INFO:tasks.workunit.client.0.smithi043.stderr: | ^ 2024-09-11T23:23:25.707 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-11T23:23:25.707 INFO:tasks.workunit.client.0.smithi043.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-11T23:23:25.707 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~ 2024-09-11T23:23:25.708 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-11T23:23:25.708 INFO:tasks.workunit.client.0.smithi043.stderr: 36 | container_of(ptr, type, member) 2024-09-11T23:23:25.708 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~~~ 2024-09-11T23:23:25.708 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-11T23:23:25.708 INFO:tasks.workunit.client.0.smithi043.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-11T23:23:25.708 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~ 2024-09-11T23:23:25.708 INFO:tasks.workunit.client.0.smithi043.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-11T23:23:25.708 INFO:tasks.workunit.client.0.smithi043.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-11T23:23:25.708 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-11T23:23:25.708 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-11T23:23:25.708 INFO:tasks.workunit.client.0.smithi043.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-11T23:23:25.708 INFO:tasks.workunit.client.0.smithi043.stderr: | ^ 2024-09-11T23:23:25.708 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-11T23:23:25.709 INFO:tasks.workunit.client.0.smithi043.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-11T23:23:25.709 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~ 2024-09-11T23:23:25.709 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-11T23:23:25.709 INFO:tasks.workunit.client.0.smithi043.stderr: 36 | container_of(ptr, type, member) 2024-09-11T23:23:25.709 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~~~ 2024-09-11T23:23:25.709 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-11T23:23:25.709 INFO:tasks.workunit.client.0.smithi043.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-11T23:23:25.709 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~ 2024-09-11T23:23:25.709 INFO:tasks.workunit.client.0.smithi043.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-11T23:23:25.709 INFO:tasks.workunit.client.0.smithi043.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-11T23:23:25.709 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-11T23:23:25.709 INFO:tasks.workunit.client.0.smithi043.stderr:parser.c: In function ‘init_tg_stats’: 2024-09-11T23:23:25.709 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-11T23:23:25.710 INFO:tasks.workunit.client.0.smithi043.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-11T23:23:25.710 INFO:tasks.workunit.client.0.smithi043.stderr: | ^ 2024-09-11T23:23:25.710 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-11T23:23:25.710 INFO:tasks.workunit.client.0.smithi043.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-11T23:23:25.710 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~ 2024-09-11T23:23:25.710 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-11T23:23:25.710 INFO:tasks.workunit.client.0.smithi043.stderr: 36 | container_of(ptr, type, member) 2024-09-11T23:23:25.710 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~~~ 2024-09-11T23:23:25.710 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-11T23:23:25.710 INFO:tasks.workunit.client.0.smithi043.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-11T23:23:25.710 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~ 2024-09-11T23:23:25.710 INFO:tasks.workunit.client.0.smithi043.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-11T23:23:25.710 INFO:tasks.workunit.client.0.smithi043.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-11T23:23:25.711 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-11T23:23:25.711 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-11T23:23:25.711 INFO:tasks.workunit.client.0.smithi043.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-11T23:23:25.711 INFO:tasks.workunit.client.0.smithi043.stderr: | ^ 2024-09-11T23:23:25.711 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-11T23:23:25.711 INFO:tasks.workunit.client.0.smithi043.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-11T23:23:25.711 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~ 2024-09-11T23:23:25.711 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-11T23:23:25.711 INFO:tasks.workunit.client.0.smithi043.stderr: 36 | container_of(ptr, type, member) 2024-09-11T23:23:25.711 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~~~ 2024-09-11T23:23:25.711 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-11T23:23:25.711 INFO:tasks.workunit.client.0.smithi043.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-11T23:23:25.711 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~ 2024-09-11T23:23:25.712 INFO:tasks.workunit.client.0.smithi043.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-11T23:23:25.712 INFO:tasks.workunit.client.0.smithi043.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-11T23:23:25.712 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-11T23:23:25.712 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-11T23:23:25.712 INFO:tasks.workunit.client.0.smithi043.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-11T23:23:25.712 INFO:tasks.workunit.client.0.smithi043.stderr: | ^ 2024-09-11T23:23:25.712 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-11T23:23:25.712 INFO:tasks.workunit.client.0.smithi043.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-11T23:23:25.712 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~ 2024-09-11T23:23:25.712 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-11T23:23:25.712 INFO:tasks.workunit.client.0.smithi043.stderr: 36 | container_of(ptr, type, member) 2024-09-11T23:23:25.712 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~~~ 2024-09-11T23:23:25.712 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-11T23:23:25.713 INFO:tasks.workunit.client.0.smithi043.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-11T23:23:25.713 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~ 2024-09-11T23:23:25.713 INFO:tasks.workunit.client.0.smithi043.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-11T23:23:25.713 INFO:tasks.workunit.client.0.smithi043.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-11T23:23:25.713 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-11T23:23:25.713 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-11T23:23:25.713 INFO:tasks.workunit.client.0.smithi043.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-11T23:23:25.713 INFO:tasks.workunit.client.0.smithi043.stderr: | ^ 2024-09-11T23:23:25.713 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-11T23:23:25.713 INFO:tasks.workunit.client.0.smithi043.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-11T23:23:25.713 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~ 2024-09-11T23:23:25.713 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-11T23:23:25.713 INFO:tasks.workunit.client.0.smithi043.stderr: 36 | container_of(ptr, type, member) 2024-09-11T23:23:25.713 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~~~ 2024-09-11T23:23:25.714 INFO:tasks.workunit.client.0.smithi043.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-11T23:23:25.714 INFO:tasks.workunit.client.0.smithi043.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-11T23:23:25.714 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~ 2024-09-11T23:23:25.714 INFO:tasks.workunit.client.0.smithi043.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-11T23:23:25.714 INFO:tasks.workunit.client.0.smithi043.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-11T23:23:25.714 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-11T23:23:25.753 INFO:tasks.workunit.client.0.smithi043.stderr:parser.c: In function ‘get_optstr’: 2024-09-11T23:23:25.753 INFO:tasks.workunit.client.0.smithi043.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-09-11T23:23:25.753 INFO:tasks.workunit.client.0.smithi043.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-09-11T23:23:25.753 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-11T23:23:25.753 INFO:tasks.workunit.client.0.smithi043.stderr:parser.c:200:14: note: source object declared here 2024-09-11T23:23:25.753 INFO:tasks.workunit.client.0.smithi043.stderr: 200 | char temp[BUFSIZE]; 2024-09-11T23:23:25.753 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~ 2024-09-11T23:23:25.765 INFO:tasks.workunit.client.0.smithi043.stderr:parser.c: In function ‘size64_convert’: 2024-09-11T23:23:25.765 INFO:tasks.workunit.client.0.smithi043.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-09-11T23:23:25.765 INFO:tasks.workunit.client.0.smithi043.stderr: 119 | memcpy(unit, "\0", 3); 2024-09-11T23:23:25.765 INFO:tasks.workunit.client.0.smithi043.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-09-11T23:23:25.842 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:23:26.066 INFO:tasks.workunit.client.0.smithi043.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-09-11T23:23:26.074 INFO:tasks.workunit.client.0.smithi043.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-09-11T23:23:26.142 INFO:tasks.workunit.client.0.smithi043.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-09-11T23:23:26.200 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:23:26.200 INFO:tasks.workunit.client.0.smithi043.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-09-11T23:23:26.252 INFO:tasks.workunit.client.0.smithi043.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-09-11T23:23:26.261 INFO:tasks.workunit.client.0.smithi043.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-09-11T23:23:26.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:25 smithi090 ceph-mon[35130]: pgmap v50: 97 pgs: 97 active+clean; 6.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 490 KiB/s wr, 120 op/s 2024-09-11T23:23:26.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:25 smithi043 ceph-mon[30950]: pgmap v50: 97 pgs: 97 active+clean; 6.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 490 KiB/s wr, 120 op/s 2024-09-11T23:23:26.291 INFO:tasks.workunit.client.0.smithi043.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-09-11T23:23:26.301 INFO:tasks.workunit.client.0.smithi043.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-09-11T23:23:26.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:25 smithi097 ceph-mon[35562]: pgmap v50: 97 pgs: 97 active+clean; 6.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 490 KiB/s wr, 120 op/s 2024-09-11T23:23:26.333 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-11T23:23:26.333 INFO:tasks.workunit.client.0.smithi043.stderr:+ cd .. 2024-09-11T23:23:26.333 INFO:tasks.workunit.client.0.smithi043.stderr:+ mkdir tmp 2024-09-11T23:23:26.335 INFO:tasks.workunit.client.0.smithi043.stderr:+ cd tmp 2024-09-11T23:23:26.335 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in $mydir/*.ffsb 2024-09-11T23:23:26.335 INFO:tasks.workunit.client.0.smithi043.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-09-11T23:23:26.336 INFO:tasks.workunit.client.0.smithi043.stdout:FFSB version 6.0-RC2 started 2024-09-11T23:23:26.336 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-11T23:23:26.339 INFO:tasks.workunit.client.0.smithi043.stdout:benchmark time = 300 2024-09-11T23:23:26.339 INFO:tasks.workunit.client.0.smithi043.stdout:ThreadGroup 0 2024-09-11T23:23:26.339 INFO:tasks.workunit.client.0.smithi043.stdout:================ 2024-09-11T23:23:26.339 INFO:tasks.workunit.client.0.smithi043.stdout: num_threads = 32 2024-09-11T23:23:26.339 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-11T23:23:26.339 INFO:tasks.workunit.client.0.smithi043.stdout: read_random = off 2024-09-11T23:23:26.339 INFO:tasks.workunit.client.0.smithi043.stdout: read_size = 0 (0B) 2024-09-11T23:23:26.339 INFO:tasks.workunit.client.0.smithi043.stdout: read_blocksize = 0 (0B) 2024-09-11T23:23:26.339 INFO:tasks.workunit.client.0.smithi043.stdout: read_skip = off 2024-09-11T23:23:26.340 INFO:tasks.workunit.client.0.smithi043.stdout: read_skipsize = 0 (0B) 2024-09-11T23:23:26.340 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-11T23:23:26.340 INFO:tasks.workunit.client.0.smithi043.stdout: write_random = on 2024-09-11T23:23:26.340 INFO:tasks.workunit.client.0.smithi043.stdout: write_size = 5242880 (5MB) 2024-09-11T23:23:26.340 INFO:tasks.workunit.client.0.smithi043.stdout: fsync_file = 0 2024-09-11T23:23:26.340 INFO:tasks.workunit.client.0.smithi043.stdout: write_blocksize = 4096 (4KB) 2024-09-11T23:23:26.340 INFO:tasks.workunit.client.0.smithi043.stdout: wait time = 0 2024-09-11T23:23:26.340 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-11T23:23:26.340 INFO:tasks.workunit.client.0.smithi043.stdout: op weights 2024-09-11T23:23:26.340 INFO:tasks.workunit.client.0.smithi043.stdout: read = 0 (0.00%) 2024-09-11T23:23:26.340 INFO:tasks.workunit.client.0.smithi043.stdout: readall = 0 (0.00%) 2024-09-11T23:23:26.340 INFO:tasks.workunit.client.0.smithi043.stdout: write = 1 (100.00%) 2024-09-11T23:23:26.340 INFO:tasks.workunit.client.0.smithi043.stdout: create = 0 (0.00%) 2024-09-11T23:23:26.340 INFO:tasks.workunit.client.0.smithi043.stdout: append = 0 (0.00%) 2024-09-11T23:23:26.341 INFO:tasks.workunit.client.0.smithi043.stdout: delete = 0 (0.00%) 2024-09-11T23:23:26.341 INFO:tasks.workunit.client.0.smithi043.stdout: metaop = 0 (0.00%) 2024-09-11T23:23:26.341 INFO:tasks.workunit.client.0.smithi043.stdout: createdir = 0 (0.00%) 2024-09-11T23:23:26.341 INFO:tasks.workunit.client.0.smithi043.stdout: stat = 0 (0.00%) 2024-09-11T23:23:26.341 INFO:tasks.workunit.client.0.smithi043.stdout: writeall = 0 (0.00%) 2024-09-11T23:23:26.341 INFO:tasks.workunit.client.0.smithi043.stdout: writeall_fsync = 0 (0.00%) 2024-09-11T23:23:26.341 INFO:tasks.workunit.client.0.smithi043.stdout: open_close = 0 (0.00%) 2024-09-11T23:23:26.341 INFO:tasks.workunit.client.0.smithi043.stdout: write_fsync = 0 (0.00%) 2024-09-11T23:23:26.341 INFO:tasks.workunit.client.0.smithi043.stdout: create_fsync = 0 (0.00%) 2024-09-11T23:23:26.341 INFO:tasks.workunit.client.0.smithi043.stdout: append_fsync = 0 (0.00%) 2024-09-11T23:23:26.341 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-11T23:23:26.341 INFO:tasks.workunit.client.0.smithi043.stdout:FileSystem . 2024-09-11T23:23:26.341 INFO:tasks.workunit.client.0.smithi043.stdout:========== 2024-09-11T23:23:26.342 INFO:tasks.workunit.client.0.smithi043.stdout: num_dirs = 0 2024-09-11T23:23:26.342 INFO:tasks.workunit.client.0.smithi043.stdout: starting files = 128 2024-09-11T23:23:26.342 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-11T23:23:26.342 INFO:tasks.workunit.client.0.smithi043.stdout: min file size = 104857600 (100MB) 2024-09-11T23:23:26.342 INFO:tasks.workunit.client.0.smithi043.stdout: max file size = 104857600 (100MB) 2024-09-11T23:23:26.342 INFO:tasks.workunit.client.0.smithi043.stdout: directio = off 2024-09-11T23:23:26.342 INFO:tasks.workunit.client.0.smithi043.stdout: alignedio = on 2024-09-11T23:23:26.342 INFO:tasks.workunit.client.0.smithi043.stdout: bufferedio = off 2024-09-11T23:23:26.342 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-11T23:23:26.342 INFO:tasks.workunit.client.0.smithi043.stdout: aging is off 2024-09-11T23:23:26.342 INFO:tasks.workunit.client.0.smithi043.stdout: current utilization = 0.00% 2024-09-11T23:23:26.343 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-11T23:23:26.343 INFO:tasks.workunit.client.0.smithi043.stdout:checking existing fs: . 2024-09-11T23:23:26.343 INFO:tasks.workunit.client.0.smithi043.stdout:recreating new fileset 2024-09-11T23:23:26.343 INFO:tasks.workunit.client.0.smithi043.stderr:opendir: No such file or directory 2024-09-11T23:23:26.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.695+0000 7fab44518640 1 -- 172.21.15.43:0/402116233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab3c1082a0 msgr2=0x7fab3c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:26.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.695+0000 7fab44518640 1 --2- 172.21.15.43:0/402116233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab3c1082a0 0x7fab3c10ec40 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fab34009c10 tx=0x7fab3402a6e0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.695+0000 7fab44518640 1 -- 172.21.15.43:0/402116233 shutdown_connections 2024-09-11T23:23:26.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.695+0000 7fab44518640 1 --2- 172.21.15.43:0/402116233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab3c1082a0 0x7fab3c10ec40 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.695+0000 7fab44518640 1 --2- 172.21.15.43:0/402116233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab3c1036e0 0x7fab3c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.695+0000 7fab44518640 1 --2- 172.21.15.43:0/402116233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3c102d30 0x7fab3c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.695+0000 7fab44518640 1 -- 172.21.15.43:0/402116233 >> 172.21.15.43:0/402116233 conn(0x7fab3c0fc7d0 msgr2=0x7fab3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:26.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.695+0000 7fab44518640 1 -- 172.21.15.43:0/402116233 shutdown_connections 2024-09-11T23:23:26.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.695+0000 7fab44518640 1 -- 172.21.15.43:0/402116233 wait complete. 2024-09-11T23:23:26.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.696+0000 7fab44518640 1 Processor -- start 2024-09-11T23:23:26.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.696+0000 7fab44518640 1 -- start start 2024-09-11T23:23:26.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.696+0000 7fab44518640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab3c102d30 0x7fab3c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:26.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.696+0000 7fab44518640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3c1036e0 0x7fab3c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:26.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.696+0000 7fab44518640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab3c1082a0 0x7fab3c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:26.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.696+0000 7fab44518640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c112050 con 0x7fab3c102d30 2024-09-11T23:23:26.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.696+0000 7fab44518640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c111ed0 con 0x7fab3c1082a0 2024-09-11T23:23:26.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.696+0000 7fab44518640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c1121d0 con 0x7fab3c1036e0 2024-09-11T23:23:26.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.697+0000 7fab41ab3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3c1036e0 0x7fab3c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:26.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.697+0000 7fab41ab3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3c1036e0 0x7fab3c100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:54888/0 (socket says 172.21.15.43:54888) 2024-09-11T23:23:26.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.697+0000 7fab41ab3640 1 -- 172.21.15.43:0/2137541137 learned_addr learned my addr 172.21.15.43:0/2137541137 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:26.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.697+0000 7fab422b4640 1 --2- 172.21.15.43:0/2137541137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab3c102d30 0x7fab3c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:26.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.697+0000 7fab42ab5640 1 --2- 172.21.15.43:0/2137541137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab3c1082a0 0x7fab3c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.697+0000 7fab41ab3640 1 -- 172.21.15.43:0/2137541137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab3c1082a0 msgr2=0x7fab3c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.697+0000 7fab41ab3640 1 --2- 172.21.15.43:0/2137541137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab3c1082a0 0x7fab3c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.697+0000 7fab41ab3640 1 -- 172.21.15.43:0/2137541137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab3c102d30 msgr2=0x7fab3c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.697+0000 7fab41ab3640 1 --2- 172.21.15.43:0/2137541137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab3c102d30 0x7fab3c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.697+0000 7fab41ab3640 1 -- 172.21.15.43:0/2137541137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3c1a6a70 con 0x7fab3c1036e0 2024-09-11T23:23:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.697+0000 7fab41ab3640 1 --2- 172.21.15.43:0/2137541137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3c1036e0 0x7fab3c100ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00e9f0 tx=0x7fab2c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.697+0000 7fab337fe640 1 -- 172.21.15.43:0/2137541137 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c00ccc0 con 0x7fab3c1036e0 2024-09-11T23:23:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.698+0000 7fab44518640 1 -- 172.21.15.43:0/2137541137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3c1a6d60 con 0x7fab3c1036e0 2024-09-11T23:23:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.698+0000 7fab337fe640 1 -- 172.21.15.43:0/2137541137 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab2c010910 con 0x7fab3c1036e0 2024-09-11T23:23:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.698+0000 7fab337fe640 1 -- 172.21.15.43:0/2137541137 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c0188a0 con 0x7fab3c1036e0 2024-09-11T23:23:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.698+0000 7fab44518640 1 -- 172.21.15.43:0/2137541137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab3c1a7260 con 0x7fab3c1036e0 2024-09-11T23:23:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.700+0000 7fab337fe640 1 -- 172.21.15.43:0/2137541137 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fab2c010ac0 con 0x7fab3c1036e0 2024-09-11T23:23:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.700+0000 7fab468f1740 1 Processor -- start 2024-09-11T23:23:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.700+0000 7fab468f1740 1 -- start start 2024-09-11T23:23:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.701+0000 7fab468f1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6b1b40 0x55cfdd6b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.701+0000 7fab468f1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 0x55cfdd6b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.701+0000 7fab468f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6bb370 0x55cfdd6bd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.701+0000 7fab468f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cfdd6162f0 con 0x55cfdd6bb370 2024-09-11T23:23:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.701+0000 7fab468f1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55cfdd5bdcf0 con 0x55cfdd6b1b40 2024-09-11T23:23:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.701+0000 7fab468f1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cfdd6122b0 con 0x55cfdd6b05d0 2024-09-11T23:23:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.701+0000 7fab42ab5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6b1b40 0x55cfdd6b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.701+0000 7fab422b4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 0x55cfdd6b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.701+0000 7fab42ab5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6b1b40 0x55cfdd6b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:45384/0 (socket says 172.21.15.43:45384) 2024-09-11T23:23:26.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.701+0000 7fab42ab5640 1 -- 172.21.15.43:0/3542940017 learned_addr learned my addr 172.21.15.43:0/3542940017 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:26.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.701+0000 7fab41ab3640 1 --2- 172.21.15.43:0/3542940017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6bb370 0x55cfdd6bd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:26.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.701+0000 7fab337fe640 1 --2- 172.21.15.43:0/2137541137 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fab280a2d70 0x7fab280a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:26.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.701+0000 7fab422b4640 1 -- 172.21.15.43:0/3542940017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6b1b40 msgr2=0x55cfdd6b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:26.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.701+0000 7fab422b4640 1 --2- 172.21.15.43:0/3542940017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6b1b40 0x55cfdd6b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.701+0000 7fab422b4640 1 -- 172.21.15.43:0/3542940017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6bb370 msgr2=0x55cfdd6bd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:26.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.701+0000 7fab422b4640 1 --2- 172.21.15.43:0/3542940017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6bb370 0x55cfdd6bd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.701+0000 7fab42ab5640 1 --2- 172.21.15.43:0/2137541137 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fab280a2d70 0x7fab280a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:26.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.701+0000 7fab422b4640 1 -- 172.21.15.43:0/3542940017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfdd5a23a0 con 0x55cfdd6b05d0 2024-09-11T23:23:26.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.701+0000 7fab41ab3640 1 --2- 172.21.15.43:0/3542940017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6bb370 0x55cfdd6bd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:23:26.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.702+0000 7fab337fe640 1 -- 172.21.15.43:0/2137541137 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fab2c0cf7a0 con 0x7fab3c1036e0 2024-09-11T23:23:26.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.702+0000 7fab422b4640 1 --2- 172.21.15.43:0/3542940017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 0x55cfdd6b0a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fab38009c30 tx=0x7fab38011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=161b081db8f80b00 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:26.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.702+0000 7fab30ff9640 1 -- 172.21.15.43:0/3542940017 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3801e070 con 0x55cfdd6b05d0 2024-09-11T23:23:26.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.702+0000 7fab30ff9640 1 -- 172.21.15.43:0/3542940017 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab380035f0 con 0x55cfdd6b05d0 2024-09-11T23:23:26.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.702+0000 7fab30ff9640 1 -- 172.21.15.43:0/3542940017 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab38019690 con 0x55cfdd6b05d0 2024-09-11T23:23:26.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.703+0000 7fab468f1740 1 -- 172.21.15.43:0/3542940017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 msgr2=0x55cfdd6b0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:26.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.703+0000 7fab42ab5640 1 --2- 172.21.15.43:0/2137541137 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fab280a2d70 0x7fab280a5230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fab340099a0 tx=0x7fab3402a6e0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:26.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.703+0000 7fab468f1740 1 --2- 172.21.15.43:0/3542940017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 0x55cfdd6b0a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fab38009c30 tx=0x7fab38011810 comp rx=0 tx=0).stop 2024-09-11T23:23:26.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.703+0000 7fab468f1740 1 -- 172.21.15.43:0/3542940017 shutdown_connections 2024-09-11T23:23:26.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.703+0000 7fab468f1740 1 --2- 172.21.15.43:0/3542940017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6bb370 0x55cfdd6bd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.703+0000 7fab468f1740 1 --2- 172.21.15.43:0/3542940017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 0x55cfdd6b0a10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.703+0000 7fab468f1740 1 --2- 172.21.15.43:0/3542940017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6b1b40 0x55cfdd6b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.703+0000 7fab468f1740 1 -- 172.21.15.43:0/3542940017 >> 172.21.15.43:0/3542940017 conn(0x55cfdd679310 msgr2=0x55cfdd6acc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:26.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.703+0000 7fab468f1740 1 -- 172.21.15.43:0/3542940017 shutdown_connections 2024-09-11T23:23:26.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.703+0000 7fab468f1740 1 -- 172.21.15.43:0/3542940017 wait complete. 2024-09-11T23:23:26.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.703+0000 7fab468f1740 1 -- start start 2024-09-11T23:23:26.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.704+0000 7fab468f1740 1 Processor -- start 2024-09-11T23:23:26.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.704+0000 7fab468f1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 0x55cfdd6badb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:26.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.704+0000 7fab468f1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6b1b40 0x55cfdd6b8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:26.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.704+0000 7fab468f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6bb370 0x55cfdd6b87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:26.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.704+0000 7fab468f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cfdd6be070 con 0x55cfdd6bb370 2024-09-11T23:23:26.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.704+0000 7fab422b4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 0x55cfdd6badb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:26.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.704+0000 7fab468f1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55cfdd5d1060 con 0x55cfdd6b1b40 2024-09-11T23:23:26.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.704+0000 7fab468f1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cfdd6be1f0 con 0x55cfdd6b05d0 2024-09-11T23:23:26.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.704+0000 7fab42ab5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6bb370 0x55cfdd6b87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:26.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.704+0000 7fab422b4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 0x55cfdd6badb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:54916/0 (socket says 172.21.15.43:54916) 2024-09-11T23:23:26.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.704+0000 7fab422b4640 1 -- 172.21.15.43:0/1136726590 learned_addr learned my addr 172.21.15.43:0/1136726590 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:26.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.704+0000 7fab41ab3640 1 --2- 172.21.15.43:0/1136726590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6b1b40 0x55cfdd6b8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:26.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.704+0000 7fab468f1740 10 client.0 _ll_register_callbacks cb 0x55cfdd3248c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:26.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.704+0000 7fab41ab3640 1 -- 172.21.15.43:0/1136726590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 msgr2=0x55cfdd6badb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:26.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.704+0000 7fab41ab3640 1 --2- 172.21.15.43:0/1136726590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 0x55cfdd6badb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.704+0000 7fab41ab3640 1 -- 172.21.15.43:0/1136726590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6bb370 msgr2=0x55cfdd6b87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:26.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.704+0000 7fab41ab3640 1 --2- 172.21.15.43:0/1136726590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6bb370 0x55cfdd6b87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.705+0000 7fab41ab3640 1 -- 172.21.15.43:0/1136726590 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfdd3873c0 con 0x55cfdd6b1b40 2024-09-11T23:23:26.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.705+0000 7fab42ab5640 1 --2- 172.21.15.43:0/1136726590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6bb370 0x55cfdd6b87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:23:26.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.705+0000 7fab41ab3640 1 --2- 172.21.15.43:0/1136726590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6b1b40 0x55cfdd6b8060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0ccef0 tx=0x7fab2c09b620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bde37f0fcdcaba98 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:26.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.705+0000 7fab25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:23:26.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.705+0000 7fab25ffb640 1 -- 172.21.15.43:0/1136726590 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c0a4930 con 0x55cfdd6b1b40 2024-09-11T23:23:26.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.705+0000 7fab25ffb640 1 -- 172.21.15.43:0/1136726590 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fab2c098330 con 0x55cfdd6b1b40 2024-09-11T23:23:26.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.705+0000 7fab468f1740 1 -- 172.21.15.43:0/1136726590 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab380035f0 con 0x55cfdd6b1b40 2024-09-11T23:23:26.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.705+0000 7fab25ffb640 1 -- 172.21.15.43:0/1136726590 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fab2c09bcc0 con 0x55cfdd6b1b40 2024-09-11T23:23:26.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.705+0000 7fab468f1740 10 client.25325 fetch_fsmap learned FSMap version 26 2024-09-11T23:23:26.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.705+0000 7fab468f1740 1 -- 172.21.15.43:0/1136726590 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cfdd6c9b90 con 0x55cfdd6b1b40 2024-09-11T23:23:26.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.705+0000 7fab25ffb640 1 -- 172.21.15.43:0/1136726590 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab380035f0 con 0x55cfdd6b1b40 2024-09-11T23:23:26.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.706+0000 7fab25ffb640 1 -- 172.21.15.43:0/1136726590 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fab2c097ba0 con 0x55cfdd6b1b40 2024-09-11T23:23:26.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.706+0000 7fab468f1740 10 client.25325 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:23:26.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.706+0000 7fab468f1740 10 client.25325 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T23:23:26.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.706+0000 7fab468f1740 20 client.25325 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:26.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.706+0000 7fab468f1740 1 --2- 172.21.15.43:0/1136726590 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55cfdd6cb050 0x55cfdd6cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:26.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.706+0000 7fab468f1740 4 client.25325 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:23:26.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.706+0000 7fab468f1740 1 -- 172.21.15.43:0/1136726590 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfdd623860 con 0x55cfdd6cb050 2024-09-11T23:23:26.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.706+0000 7fab422b4640 1 --2- 172.21.15.43:0/1136726590 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55cfdd6cb050 0x55cfdd6cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:26.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.706+0000 7fab25ffb640 10 client.25325 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:23:26.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.706+0000 7fab422b4640 1 --2- 172.21.15.43:0/1136726590 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55cfdd6cb050 0x55cfdd6cd460 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=159669a6b58b5712 server_cookie=809052e104236775 in_seq=0 out_seq=0 2024-09-11T23:23:26.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.707+0000 7fab25ffb640 1 -- 172.21.15.43:0/1136726590 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cfdd623860 con 0x55cfdd6cb050 2024-09-11T23:23:26.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.707+0000 7fab25ffb640 10 client.25325 handle_command_reply: tid=0 2024-09-11T23:23:26.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.707+0000 7fab468f1740 1 client.25325 shutdown 2024-09-11T23:23:26.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.707+0000 7fab468f1740 1 -- 172.21.15.43:0/1136726590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6b1b40 msgr2=0x55cfdd6b8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:26.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.707+0000 7fab468f1740 1 --2- 172.21.15.43:0/1136726590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6b1b40 0x55cfdd6b8060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0ccef0 tx=0x7fab2c09b620 comp rx=0 tx=0).stop 2024-09-11T23:23:26.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.707+0000 7fab468f1740 1 -- 172.21.15.43:0/1136726590 shutdown_connections 2024-09-11T23:23:26.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.707+0000 7fab468f1740 1 --2- 172.21.15.43:0/1136726590 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55cfdd6cb050 0x55cfdd6cd460 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.707+0000 7fab25ffb640 0 client.25325 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:23:26.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.707+0000 7fab468f1740 1 --2- 172.21.15.43:0/1136726590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6bb370 0x55cfdd6b87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.707+0000 7fab468f1740 1 --2- 172.21.15.43:0/1136726590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6b1b40 0x55cfdd6b8060 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.707+0000 7fab468f1740 1 --2- 172.21.15.43:0/1136726590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 0x55cfdd6badb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.707+0000 7fab468f1740 1 -- 172.21.15.43:0/1136726590 >> 172.21.15.43:0/1136726590 conn(0x55cfdd679310 msgr2=0x55cfdd6b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:26.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.707+0000 7fab468f1740 1 -- 172.21.15.43:0/1136726590 shutdown_connections 2024-09-11T23:23:26.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.707+0000 7fab468f1740 1 -- 172.21.15.43:0/1136726590 wait complete. 2024-09-11T23:23:26.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.708+0000 7fab468f1740 20 client.25325 trim_cache size 0 max 16384 2024-09-11T23:23:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.735+0000 7fab468f1740 1 Processor -- start 2024-09-11T23:23:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.735+0000 7fab468f1740 1 -- start start 2024-09-11T23:23:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.735+0000 7fab468f1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 0x55cfdd6b7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.735+0000 7fab468f1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6b1b40 0x55cfdd6abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.735+0000 7fab468f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6bb370 0x55cfdd6b3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.735+0000 7fab468f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c1b2420 con 0x55cfdd6bb370 2024-09-11T23:23:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.735+0000 7fab468f1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c1afcf0 con 0x55cfdd6b1b40 2024-09-11T23:23:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.735+0000 7fab468f1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c1adb40 con 0x55cfdd6b05d0 2024-09-11T23:23:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.735+0000 7fab42ab5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6bb370 0x55cfdd6b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.735+0000 7fab42ab5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6bb370 0x55cfdd6b3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35386/0 (socket says 172.21.15.43:35386) 2024-09-11T23:23:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.735+0000 7fab42ab5640 1 -- 172.21.15.43:0/38215491 learned_addr learned my addr 172.21.15.43:0/38215491 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.735+0000 7fab41ab3640 1 --2- 172.21.15.43:0/38215491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6b1b40 0x55cfdd6abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.735+0000 7fab422b4640 1 --2- 172.21.15.43:0/38215491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 0x55cfdd6b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.735+0000 7fab41ab3640 1 -- 172.21.15.43:0/38215491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 msgr2=0x55cfdd6b7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.735+0000 7fab41ab3640 1 --2- 172.21.15.43:0/38215491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 0x55cfdd6b7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.735+0000 7fab41ab3640 1 -- 172.21.15.43:0/38215491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6bb370 msgr2=0x55cfdd6b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.735+0000 7fab41ab3640 1 --2- 172.21.15.43:0/38215491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6bb370 0x55cfdd6b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.735+0000 7fab41ab3640 1 -- 172.21.15.43:0/38215491 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfdd6addd0 con 0x55cfdd6b1b40 2024-09-11T23:23:26.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.735+0000 7fab42ab5640 1 --2- 172.21.15.43:0/38215491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6bb370 0x55cfdd6b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:23:26.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.736+0000 7fab41ab3640 1 --2- 172.21.15.43:0/38215491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6b1b40 0x55cfdd6abbf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00e9c0 tx=0x7fab2c0a01f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67fe3ea3a98d78ad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:26.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.736+0000 7fab27fff640 1 -- 172.21.15.43:0/38215491 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c010430 con 0x55cfdd6b1b40 2024-09-11T23:23:26.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.736+0000 7fab27fff640 1 -- 172.21.15.43:0/38215491 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab2c0105d0 con 0x55cfdd6b1b40 2024-09-11T23:23:26.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.736+0000 7fab27fff640 1 -- 172.21.15.43:0/38215491 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c0a1580 con 0x55cfdd6b1b40 2024-09-11T23:23:26.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.737+0000 7fab468f1740 1 -- 172.21.15.43:0/38215491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6b1b40 msgr2=0x55cfdd6abbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:26.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.737+0000 7fab468f1740 1 --2- 172.21.15.43:0/38215491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6b1b40 0x55cfdd6abbf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00e9c0 tx=0x7fab2c0a01f0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.737+0000 7fab468f1740 1 -- 172.21.15.43:0/38215491 shutdown_connections 2024-09-11T23:23:26.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.737+0000 7fab468f1740 1 --2- 172.21.15.43:0/38215491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6bb370 0x55cfdd6b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.737+0000 7fab468f1740 1 --2- 172.21.15.43:0/38215491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6b1b40 0x55cfdd6abbf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.737+0000 7fab468f1740 1 --2- 172.21.15.43:0/38215491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 0x55cfdd6b7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.737+0000 7fab468f1740 1 -- 172.21.15.43:0/38215491 >> 172.21.15.43:0/38215491 conn(0x55cfdd679310 msgr2=0x55cfdd6ab4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:26.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.737+0000 7fab468f1740 1 -- 172.21.15.43:0/38215491 shutdown_connections 2024-09-11T23:23:26.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.737+0000 7fab468f1740 1 -- 172.21.15.43:0/38215491 wait complete. 2024-09-11T23:23:26.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.737+0000 7fab468f1740 1 -- start start 2024-09-11T23:23:26.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.737+0000 7fab468f1740 1 Processor -- start 2024-09-11T23:23:26.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.737+0000 7fab468f1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 0x55cfdd6c3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:26.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.737+0000 7fab468f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6b1b40 0x55cfdd6b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:26.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.737+0000 7fab468f1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6bb370 0x55cfdd6b5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:26.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.738+0000 7fab468f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c1ae1b0 con 0x55cfdd6b1b40 2024-09-11T23:23:26.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.738+0000 7fab468f1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c1af620 con 0x55cfdd6bb370 2024-09-11T23:23:26.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.738+0000 7fab468f1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c1ae880 con 0x55cfdd6b05d0 2024-09-11T23:23:26.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.738+0000 7fab468f1740 10 client.0 _ll_register_callbacks cb 0x7fab2c09bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:26.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.738+0000 7fab42ab5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6bb370 0x55cfdd6b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:26.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.738+0000 7fab41ab3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6b1b40 0x55cfdd6b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:26.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.738+0000 7fab42ab5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6bb370 0x55cfdd6b5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:45416/0 (socket says 172.21.15.43:45416) 2024-09-11T23:23:26.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.738+0000 7fab42ab5640 1 -- 172.21.15.43:0/1217681658 learned_addr learned my addr 172.21.15.43:0/1217681658 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:26.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.738+0000 7fab422b4640 1 --2- 172.21.15.43:0/1217681658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 0x55cfdd6c3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:26.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.738+0000 7fab41ab3640 1 -- 172.21.15.43:0/1217681658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 msgr2=0x55cfdd6c3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:26.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.738+0000 7fab41ab3640 1 --2- 172.21.15.43:0/1217681658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 0x55cfdd6c3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.738+0000 7fab41ab3640 1 -- 172.21.15.43:0/1217681658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6bb370 msgr2=0x55cfdd6b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:26.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.738+0000 7fab41ab3640 1 --2- 172.21.15.43:0/1217681658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6bb370 0x55cfdd6b5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.738+0000 7fab41ab3640 1 -- 172.21.15.43:0/1217681658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfdd6b5c80 con 0x55cfdd6b1b40 2024-09-11T23:23:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.738+0000 7fab41ab3640 1 --2- 172.21.15.43:0/1217681658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6b1b40 0x55cfdd6b4e90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fab2c098690 tx=0x7fab2c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a37d097440c9c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.738+0000 7fab25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:23:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.738+0000 7fab25ffb640 1 -- 172.21.15.43:0/1217681658 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c010430 con 0x55cfdd6b1b40 2024-09-11T23:23:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.738+0000 7fab25ffb640 1 -- 172.21.15.43:0/1217681658 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fab2c0a4b80 con 0x55cfdd6b1b40 2024-09-11T23:23:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.738+0000 7fab468f1740 1 -- 172.21.15.43:0/1217681658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab2c0105d0 con 0x55cfdd6b1b40 2024-09-11T23:23:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.739+0000 7fab25ffb640 1 -- 172.21.15.43:0/1217681658 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fab2c0a4930 con 0x55cfdd6b1b40 2024-09-11T23:23:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.739+0000 7fab468f1740 10 client.15756 fetch_fsmap learned FSMap version 26 2024-09-11T23:23:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.739+0000 7fab468f1740 1 -- 172.21.15.43:0/1217681658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cfdd735de0 con 0x55cfdd6b1b40 2024-09-11T23:23:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.739+0000 7fab25ffb640 1 -- 172.21.15.43:0/1217681658 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab2c0105d0 con 0x55cfdd6b1b40 2024-09-11T23:23:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.739+0000 7fab25ffb640 1 -- 172.21.15.43:0/1217681658 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fab2c0ba2b0 con 0x55cfdd6b1b40 2024-09-11T23:23:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.739+0000 7fab468f1740 10 client.15756 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:23:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.739+0000 7fab468f1740 10 client.15756 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T23:23:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.739+0000 7fab468f1740 20 client.15756 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.739+0000 7fab468f1740 1 --2- 172.21.15.43:0/1217681658 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55cfdd6cb050 0x55cfdd7382c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.739+0000 7fab468f1740 4 client.15756 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:23:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.739+0000 7fab468f1740 1 -- 172.21.15.43:0/1217681658 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cfdd61dd70 con 0x55cfdd6cb050 2024-09-11T23:23:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.739+0000 7fab422b4640 1 --2- 172.21.15.43:0/1217681658 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55cfdd6cb050 0x55cfdd7382c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.740+0000 7fab25ffb640 10 client.15756 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:23:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.740+0000 7fab422b4640 1 --2- 172.21.15.43:0/1217681658 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55cfdd6cb050 0x55cfdd7382c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7079507f1743fd53 server_cookie=848403cadc1c0b81 in_seq=0 out_seq=0 2024-09-11T23:23:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.740+0000 7fab25ffb640 1 -- 172.21.15.43:0/1217681658 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cfdd61dd70 con 0x55cfdd6cb050 2024-09-11T23:23:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.740+0000 7fab25ffb640 10 client.15756 handle_command_reply: tid=0 2024-09-11T23:23:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.740+0000 7fab468f1740 1 client.15756 shutdown 2024-09-11T23:23:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.740+0000 7fab468f1740 1 -- 172.21.15.43:0/1217681658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6b1b40 msgr2=0x55cfdd6b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.740+0000 7fab468f1740 1 --2- 172.21.15.43:0/1217681658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6b1b40 0x55cfdd6b4e90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fab2c098690 tx=0x7fab2c0096a0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.740+0000 7fab468f1740 1 -- 172.21.15.43:0/1217681658 shutdown_connections 2024-09-11T23:23:26.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.740+0000 7fab468f1740 1 --2- 172.21.15.43:0/1217681658 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55cfdd6cb050 0x55cfdd7382c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.740+0000 7fab25ffb640 0 client.15756 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:23:26.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.740+0000 7fab468f1740 1 --2- 172.21.15.43:0/1217681658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfdd6bb370 0x55cfdd6b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.740+0000 7fab468f1740 1 --2- 172.21.15.43:0/1217681658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cfdd6b1b40 0x55cfdd6b4e90 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.740+0000 7fab468f1740 1 --2- 172.21.15.43:0/1217681658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cfdd6b05d0 0x55cfdd6c3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.740+0000 7fab468f1740 1 -- 172.21.15.43:0/1217681658 >> 172.21.15.43:0/1217681658 conn(0x55cfdd679310 msgr2=0x55cfdd6ab4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:26.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.740+0000 7fab468f1740 1 -- 172.21.15.43:0/1217681658 shutdown_connections 2024-09-11T23:23:26.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.740+0000 7fab468f1740 1 -- 172.21.15.43:0/1217681658 wait complete. 2024-09-11T23:23:26.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.740+0000 7fab468f1740 20 client.15756 trim_cache size 0 max 16384 2024-09-11T23:23:26.746 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:23:26.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.741+0000 7fab44518640 1 -- 172.21.15.43:0/2137541137 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fab280a2d70 msgr2=0x7fab280a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:26.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.741+0000 7fab44518640 1 --2- 172.21.15.43:0/2137541137 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fab280a2d70 0x7fab280a5230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fab340099a0 tx=0x7fab3402a6e0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.741+0000 7fab44518640 1 -- 172.21.15.43:0/2137541137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3c1036e0 msgr2=0x7fab3c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:26.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.741+0000 7fab44518640 1 --2- 172.21.15.43:0/2137541137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3c1036e0 0x7fab3c100ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00e9f0 tx=0x7fab2c00edf0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.741+0000 7fab44518640 1 -- 172.21.15.43:0/2137541137 shutdown_connections 2024-09-11T23:23:26.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.741+0000 7fab44518640 1 --2- 172.21.15.43:0/2137541137 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fab280a2d70 0x7fab280a5230 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.741+0000 7fab44518640 1 --2- 172.21.15.43:0/2137541137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab3c1082a0 0x7fab3c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.741+0000 7fab44518640 1 --2- 172.21.15.43:0/2137541137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3c1036e0 0x7fab3c100ed0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.741+0000 7fab44518640 1 --2- 172.21.15.43:0/2137541137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab3c102d30 0x7fab3c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:26.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.741+0000 7fab44518640 1 -- 172.21.15.43:0/2137541137 >> 172.21.15.43:0/2137541137 conn(0x7fab3c0fc7d0 msgr2=0x7fab3c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:26.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.741+0000 7fab44518640 1 -- 172.21.15.43:0/2137541137 shutdown_connections 2024-09-11T23:23:26.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:26.741+0000 7fab44518640 1 -- 172.21.15.43:0/2137541137 wait complete. 2024-09-11T23:23:27.452 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:23:27.729 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:23:28.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:27 smithi043 ceph-mon[30950]: pgmap v51: 97 pgs: 97 active+clean; 7.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 630 KiB/s wr, 133 op/s 2024-09-11T23:23:28.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:27 smithi090 ceph-mon[35130]: pgmap v51: 97 pgs: 97 active+clean; 7.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 630 KiB/s wr, 133 op/s 2024-09-11T23:23:28.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.284+0000 7f7dfcfa6640 1 -- 172.21.15.43:0/2550682572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8102db0 msgr2=0x7f7df8103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:28.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.284+0000 7f7dfcfa6640 1 --2- 172.21.15.43:0/2550682572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8102db0 0x7f7df8103190 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7de8009930 tx=0x7f7de802a420 comp rx=0 tx=0).stop 2024-09-11T23:23:28.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.285+0000 7f7dfcfa6640 1 -- 172.21.15.43:0/2550682572 shutdown_connections 2024-09-11T23:23:28.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.285+0000 7f7dfcfa6640 1 --2- 172.21.15.43:0/2550682572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7df81082e0 0x7f7df810ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.285+0000 7f7dfcfa6640 1 --2- 172.21.15.43:0/2550682572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7df8103760 0x7f7df8107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.285+0000 7f7dfcfa6640 1 --2- 172.21.15.43:0/2550682572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8102db0 0x7f7df8103190 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.285+0000 7f7dfcfa6640 1 -- 172.21.15.43:0/2550682572 >> 172.21.15.43:0/2550682572 conn(0x7f7df80fc7d0 msgr2=0x7f7df80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:28.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.285+0000 7f7dfcfa6640 1 -- 172.21.15.43:0/2550682572 shutdown_connections 2024-09-11T23:23:28.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.285+0000 7f7dfcfa6640 1 -- 172.21.15.43:0/2550682572 wait complete. 2024-09-11T23:23:28.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.286+0000 7f7dfcfa6640 1 Processor -- start 2024-09-11T23:23:28.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.286+0000 7f7dfcfa6640 1 -- start start 2024-09-11T23:23:28.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.287+0000 7f7dfcfa6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8102db0 0x7f7df81986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:28.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.287+0000 7f7dfcfa6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7df8103760 0x7f7df8198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:28.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.287+0000 7f7dfcfa6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7df81082e0 0x7f7df819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:28.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.287+0000 7f7dfcfa6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7df81120b0 con 0x7f7df8102db0 2024-09-11T23:23:28.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.287+0000 7f7dfcfa6640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7df8111f30 con 0x7f7df8103760 2024-09-11T23:23:28.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.287+0000 7f7dfcfa6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7df8112230 con 0x7f7df81082e0 2024-09-11T23:23:28.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.287+0000 7f7df659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8102db0 0x7f7df81986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:28.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.287+0000 7f7df659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8102db0 0x7f7df81986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35418/0 (socket says 172.21.15.43:35418) 2024-09-11T23:23:28.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.287+0000 7f7df659c640 1 -- 172.21.15.43:0/787052914 learned_addr learned my addr 172.21.15.43:0/787052914 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.287+0000 7f7df6d9d640 1 --2- 172.21.15.43:0/787052914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7df81082e0 0x7f7df819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.287+0000 7f7df659c640 1 -- 172.21.15.43:0/787052914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7df81082e0 msgr2=0x7f7df819cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.287+0000 7f7df659c640 1 --2- 172.21.15.43:0/787052914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7df81082e0 0x7f7df819cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.287+0000 7f7df659c640 1 -- 172.21.15.43:0/787052914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7df8103760 msgr2=0x7f7df8198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.287+0000 7f7df659c640 1 --2- 172.21.15.43:0/787052914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7df8103760 0x7f7df8198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.287+0000 7f7df659c640 1 -- 172.21.15.43:0/787052914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df819d680 con 0x7f7df8102db0 2024-09-11T23:23:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.287+0000 7f7df659c640 1 --2- 172.21.15.43:0/787052914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8102db0 0x7f7df81986d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7de8004770 tx=0x7f7de8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.288+0000 7f7deffff640 1 -- 172.21.15.43:0/787052914 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de803f070 con 0x7f7df8102db0 2024-09-11T23:23:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.288+0000 7f7deffff640 1 -- 172.21.15.43:0/787052914 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de8031c20 con 0x7f7df8102db0 2024-09-11T23:23:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.288+0000 7f7deffff640 1 -- 172.21.15.43:0/787052914 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8035030 con 0x7f7df8102db0 2024-09-11T23:23:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.288+0000 7f7dfcfa6640 1 -- 172.21.15.43:0/787052914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df819d910 con 0x7f7df8102db0 2024-09-11T23:23:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.288+0000 7f7dfcfa6640 1 -- 172.21.15.43:0/787052914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df819dda0 con 0x7f7df8102db0 2024-09-11T23:23:28.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.291+0000 7f7deffff640 1 -- 172.21.15.43:0/787052914 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7de8042070 con 0x7f7df8102db0 2024-09-11T23:23:28.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.291+0000 7f7dff37f740 1 Processor -- start 2024-09-11T23:23:28.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.291+0000 7f7dff37f740 1 -- start start 2024-09-11T23:23:28.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.292+0000 7f7dff37f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dc7d9b40 0x5629dc7d8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:28.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.292+0000 7f7dff37f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7d85d0 0x5629dc7d8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:28.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.292+0000 7f7dff37f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7e7780 0x5629dc7e9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:28.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.292+0000 7f7dff37f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5629dc73e2f0 con 0x5629dc7e7780 2024-09-11T23:23:28.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.292+0000 7f7dff37f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5629dc6e5cf0 con 0x5629dc7d85d0 2024-09-11T23:23:28.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.292+0000 7f7dff37f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5629dc73a2b0 con 0x5629dc7d9b40 2024-09-11T23:23:28.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.292+0000 7f7dedd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7e7780 0x5629dc7e9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:28.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.292+0000 7f7dedd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7e7780 0x5629dc7e9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35432/0 (socket says 172.21.15.43:35432) 2024-09-11T23:23:28.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.292+0000 7f7dedd9b640 1 -- 172.21.15.43:0/2952317153 learned_addr learned my addr 172.21.15.43:0/2952317153 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:28.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.292+0000 7f7df659c640 1 --2- 172.21.15.43:0/2952317153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7d85d0 0x5629dc7d8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:28.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.292+0000 7f7deffff640 1 --2- 172.21.15.43:0/787052914 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7de40a2d70 0x7f7de40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:28.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.292+0000 7f7df6d9d640 1 --2- 172.21.15.43:0/2952317153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dc7d9b40 0x5629dc7d8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:28.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.292+0000 7f7df6d9d640 1 --2- 172.21.15.43:0/787052914 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7de40a2d70 0x7f7de40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:28.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.292+0000 7f7deffff640 1 -- 172.21.15.43:0/787052914 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f7de80efb60 con 0x7f7df8102db0 2024-09-11T23:23:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.292+0000 7f7dedd9b640 1 -- 172.21.15.43:0/2952317153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dc7d9b40 msgr2=0x5629dc7d8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.292+0000 7f7dedd9b640 1 --2- 172.21.15.43:0/2952317153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dc7d9b40 0x5629dc7d8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.292+0000 7f7dedd9b640 1 -- 172.21.15.43:0/2952317153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7d85d0 msgr2=0x5629dc7d8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.292+0000 7f7dedd9b640 1 --2- 172.21.15.43:0/2952317153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7d85d0 0x5629dc7d8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.292+0000 7f7dedd9b640 1 -- 172.21.15.43:0/2952317153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629dc6ca3a0 con 0x5629dc7e7780 2024-09-11T23:23:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.293+0000 7f7dedd9b640 1 --2- 172.21.15.43:0/2952317153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7e7780 0x5629dc7e9b70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009b80 tx=0x7f7de000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19bd34d5c2bb23ea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.293+0000 7f7df6d9d640 1 --2- 172.21.15.43:0/787052914 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7de40a2d70 0x7f7de40a5230 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7df0005fd0 tx=0x7f7df0004ab0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.293+0000 7f7decd99640 1 -- 172.21.15.43:0/2952317153 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0004020 con 0x5629dc7e7780 2024-09-11T23:23:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.293+0000 7f7decd99640 1 -- 172.21.15.43:0/2952317153 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de00041c0 con 0x5629dc7e7780 2024-09-11T23:23:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.293+0000 7f7decd99640 1 -- 172.21.15.43:0/2952317153 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0010670 con 0x5629dc7e7780 2024-09-11T23:23:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.294+0000 7f7dff37f740 1 -- 172.21.15.43:0/2952317153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7e7780 msgr2=0x5629dc7e9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.294+0000 7f7dff37f740 1 --2- 172.21.15.43:0/2952317153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7e7780 0x5629dc7e9b70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009b80 tx=0x7f7de000e720 comp rx=0 tx=0).stop 2024-09-11T23:23:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.294+0000 7f7dff37f740 1 -- 172.21.15.43:0/2952317153 shutdown_connections 2024-09-11T23:23:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.294+0000 7f7dff37f740 1 --2- 172.21.15.43:0/2952317153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7e7780 0x5629dc7e9b70 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.294+0000 7f7dff37f740 1 --2- 172.21.15.43:0/2952317153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7d85d0 0x5629dc7d8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.294+0000 7f7dff37f740 1 --2- 172.21.15.43:0/2952317153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dc7d9b40 0x5629dc7d8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.294+0000 7f7dff37f740 1 -- 172.21.15.43:0/2952317153 >> 172.21.15.43:0/2952317153 conn(0x5629dc7a1310 msgr2=0x5629dc7d4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.294+0000 7f7dff37f740 1 -- 172.21.15.43:0/2952317153 shutdown_connections 2024-09-11T23:23:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.294+0000 7f7dff37f740 1 -- 172.21.15.43:0/2952317153 wait complete. 2024-09-11T23:23:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.294+0000 7f7dff37f740 1 -- start start 2024-09-11T23:23:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.294+0000 7f7dff37f740 1 Processor -- start 2024-09-11T23:23:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.295+0000 7f7dff37f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7d85d0 0x5629dc7e70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.295+0000 7f7dff37f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7d9b40 0x5629dc7e43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.295+0000 7f7dff37f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dc7e7780 0x5629dc7e4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.295+0000 7f7dff37f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5629dc7ea480 con 0x5629dc7d85d0 2024-09-11T23:23:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.295+0000 7f7dff37f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5629dc6f9060 con 0x5629dc7d9b40 2024-09-11T23:23:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.295+0000 7f7dff37f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5629dc7ea600 con 0x5629dc7e7780 2024-09-11T23:23:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.295+0000 7f7dff37f740 10 client.0 _ll_register_callbacks cb 0x5629dc44c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.295+0000 7f7df659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7d85d0 0x5629dc7e70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.295+0000 7f7df659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7d85d0 0x5629dc7e70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35442/0 (socket says 172.21.15.43:35442) 2024-09-11T23:23:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.295+0000 7f7df659c640 1 -- 172.21.15.43:0/2664378988 learned_addr learned my addr 172.21.15.43:0/2664378988 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.295+0000 7f7dedd9b640 1 --2- 172.21.15.43:0/2664378988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7d9b40 0x5629dc7e43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.295+0000 7f7df6d9d640 1 --2- 172.21.15.43:0/2664378988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dc7e7780 0x5629dc7e4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.295+0000 7f7df659c640 1 -- 172.21.15.43:0/2664378988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dc7e7780 msgr2=0x5629dc7e4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.295+0000 7f7df659c640 1 --2- 172.21.15.43:0/2664378988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dc7e7780 0x5629dc7e4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.295+0000 7f7df659c640 1 -- 172.21.15.43:0/2664378988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7d9b40 msgr2=0x5629dc7e43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.295+0000 7f7df659c640 1 --2- 172.21.15.43:0/2664378988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7d9b40 0x5629dc7e43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.295+0000 7f7df659c640 1 -- 172.21.15.43:0/2664378988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629dc4af3c0 con 0x5629dc7d85d0 2024-09-11T23:23:28.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.295+0000 7f7df659c640 1 --2- 172.21.15.43:0/2664378988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7d85d0 0x5629dc7e70f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7de80ed310 tx=0x7f7de80092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130d9b32785a094b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:28.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.295+0000 7f7dddffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:23:28.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.296+0000 7f7dddffb640 1 -- 172.21.15.43:0/2664378988 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8049070 con 0x5629dc7d85d0 2024-09-11T23:23:28.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.296+0000 7f7dddffb640 1 -- 172.21.15.43:0/2664378988 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f7de80c00d0 con 0x5629dc7d85d0 2024-09-11T23:23:28.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.296+0000 7f7dff37f740 1 -- 172.21.15.43:0/2664378988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7de00041c0 con 0x5629dc7d85d0 2024-09-11T23:23:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.296+0000 7f7dddffb640 1 -- 172.21.15.43:0/2664378988 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7de80bc780 con 0x5629dc7d85d0 2024-09-11T23:23:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.296+0000 7f7dff37f740 10 client.15768 fetch_fsmap learned FSMap version 26 2024-09-11T23:23:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.296+0000 7f7dff37f740 1 -- 172.21.15.43:0/2664378988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5629dc7f5e90 con 0x5629dc7d85d0 2024-09-11T23:23:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.296+0000 7f7dddffb640 1 -- 172.21.15.43:0/2664378988 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de00041c0 con 0x5629dc7d85d0 2024-09-11T23:23:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.296+0000 7f7dddffb640 1 -- 172.21.15.43:0/2664378988 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7de803bce0 con 0x5629dc7d85d0 2024-09-11T23:23:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.296+0000 7f7dff37f740 10 client.15768 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:23:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.296+0000 7f7dff37f740 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.296+0000 7f7dff37f740 20 client.15768 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.296+0000 7f7dff37f740 1 --2- 172.21.15.43:0/2664378988 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5629dc7f7230 0x5629dc7f9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.296+0000 7f7dff37f740 4 client.15768 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:23:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.297+0000 7f7dff37f740 1 -- 172.21.15.43:0/2664378988 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629dc74b860 con 0x5629dc7f7230 2024-09-11T23:23:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.297+0000 7f7dedd9b640 1 --2- 172.21.15.43:0/2664378988 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5629dc7f7230 0x5629dc7f9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.297+0000 7f7dedd9b640 1 --2- 172.21.15.43:0/2664378988 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5629dc7f7230 0x5629dc7f9680 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9aa3eaf494dbf76 server_cookie=cb5b7c81992ba304 in_seq=0 out_seq=0 2024-09-11T23:23:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.297+0000 7f7dddffb640 10 client.15768 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.298+0000 7f7dddffb640 1 -- 172.21.15.43:0/2664378988 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5629dc74b860 con 0x5629dc7f7230 2024-09-11T23:23:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.298+0000 7f7dddffb640 10 client.15768 handle_command_reply: tid=0 2024-09-11T23:23:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.298+0000 7f7dff37f740 1 client.15768 shutdown 2024-09-11T23:23:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.298+0000 7f7dff37f740 1 -- 172.21.15.43:0/2664378988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7d85d0 msgr2=0x5629dc7e70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.298+0000 7f7dff37f740 1 --2- 172.21.15.43:0/2664378988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7d85d0 0x5629dc7e70f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7de80ed310 tx=0x7f7de80092f0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.298+0000 7f7dff37f740 1 -- 172.21.15.43:0/2664378988 shutdown_connections 2024-09-11T23:23:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.298+0000 7f7dff37f740 1 --2- 172.21.15.43:0/2664378988 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5629dc7f7230 0x5629dc7f9680 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.298+0000 7f7dff37f740 1 --2- 172.21.15.43:0/2664378988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dc7e7780 0x5629dc7e4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.298+0000 7f7dff37f740 1 --2- 172.21.15.43:0/2664378988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7d9b40 0x5629dc7e43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.298+0000 7f7dff37f740 1 --2- 172.21.15.43:0/2664378988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7d85d0 0x5629dc7e70f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.298+0000 7f7dff37f740 1 -- 172.21.15.43:0/2664378988 >> 172.21.15.43:0/2664378988 conn(0x5629dc7a1310 msgr2=0x5629dc7da960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:28.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.298+0000 7f7dddffb640 0 client.15768 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:28.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.298+0000 7f7dff37f740 1 -- 172.21.15.43:0/2664378988 shutdown_connections 2024-09-11T23:23:28.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.298+0000 7f7dff37f740 1 -- 172.21.15.43:0/2664378988 wait complete. 2024-09-11T23:23:28.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.298+0000 7f7dff37f740 20 client.15768 trim_cache size 0 max 16384 2024-09-11T23:23:28.303 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:27 smithi097 ceph-mon[35562]: pgmap v51: 97 pgs: 97 active+clean; 7.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 630 KiB/s wr, 133 op/s 2024-09-11T23:23:28.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.312+0000 7f7dff37f740 1 Processor -- start 2024-09-11T23:23:28.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.312+0000 7f7dff37f740 1 -- start start 2024-09-11T23:23:28.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.312+0000 7f7dff37f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dc7d85d0 0x5629dc7f3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:28.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.312+0000 7f7dff37f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7d9b40 0x5629dc7d4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:28.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.312+0000 7f7dff37f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7e7780 0x5629dc7fef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:28.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.312+0000 7f7dff37f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7df81adf60 con 0x5629dc7d9b40 2024-09-11T23:23:28.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.312+0000 7f7dff37f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7df81ab830 con 0x5629dc7e7780 2024-09-11T23:23:28.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.312+0000 7f7dff37f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7df81a9680 con 0x5629dc7d85d0 2024-09-11T23:23:28.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.312+0000 7f7dedd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7d9b40 0x5629dc7d4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:28.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.312+0000 7f7df659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dc7d85d0 0x5629dc7f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:28.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.313+0000 7f7df6d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7e7780 0x5629dc7fef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:28.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.313+0000 7f7dedd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7d9b40 0x5629dc7d4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35450/0 (socket says 172.21.15.43:35450) 2024-09-11T23:23:28.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.313+0000 7f7dedd9b640 1 -- 172.21.15.43:0/2981092378 learned_addr learned my addr 172.21.15.43:0/2981092378 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:28.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.313+0000 7f7df6d9d640 1 -- 172.21.15.43:0/2981092378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dc7d85d0 msgr2=0x5629dc7f3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:28.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.313+0000 7f7df6d9d640 1 --2- 172.21.15.43:0/2981092378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dc7d85d0 0x5629dc7f3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.313+0000 7f7df6d9d640 1 -- 172.21.15.43:0/2981092378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7d9b40 msgr2=0x5629dc7d4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:28.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.313+0000 7f7df6d9d640 1 --2- 172.21.15.43:0/2981092378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7d9b40 0x5629dc7d4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.313+0000 7f7df6d9d640 1 -- 172.21.15.43:0/2981092378 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629dc7d5960 con 0x5629dc7e7780 2024-09-11T23:23:28.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.313+0000 7f7dedd9b640 1 --2- 172.21.15.43:0/2981092378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7d9b40 0x5629dc7d4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:23:28.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.313+0000 7f7df659c640 1 --2- 172.21.15.43:0/2981092378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dc7d85d0 0x5629dc7f3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:23:28.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.313+0000 7f7df6d9d640 1 --2- 172.21.15.43:0/2981092378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7e7780 0x5629dc7fef90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7df0005c30 tx=0x7f7df0009ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9fa0a7df5a8f280 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:28.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.314+0000 7f7ddffff640 1 -- 172.21.15.43:0/2981092378 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df00119b0 con 0x5629dc7e7780 2024-09-11T23:23:28.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.314+0000 7f7ddffff640 1 -- 172.21.15.43:0/2981092378 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7df0025070 con 0x5629dc7e7780 2024-09-11T23:23:28.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.314+0000 7f7ddffff640 1 -- 172.21.15.43:0/2981092378 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df0021db0 con 0x5629dc7e7780 2024-09-11T23:23:28.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.315+0000 7f7dff37f740 1 -- 172.21.15.43:0/2981092378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7e7780 msgr2=0x5629dc7fef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:28.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.315+0000 7f7dff37f740 1 --2- 172.21.15.43:0/2981092378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7e7780 0x5629dc7fef90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7df0005c30 tx=0x7f7df0009ab0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.315+0000 7f7dff37f740 1 -- 172.21.15.43:0/2981092378 shutdown_connections 2024-09-11T23:23:28.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.315+0000 7f7dff37f740 1 --2- 172.21.15.43:0/2981092378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7e7780 0x5629dc7fef90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.315+0000 7f7dff37f740 1 --2- 172.21.15.43:0/2981092378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7d9b40 0x5629dc7d4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.315+0000 7f7dff37f740 1 --2- 172.21.15.43:0/2981092378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dc7d85d0 0x5629dc7f3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.315+0000 7f7dff37f740 1 -- 172.21.15.43:0/2981092378 >> 172.21.15.43:0/2981092378 conn(0x5629dc7a1310 msgr2=0x5629dc7e2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:28.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.315+0000 7f7dff37f740 1 -- 172.21.15.43:0/2981092378 shutdown_connections 2024-09-11T23:23:28.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.317+0000 7f7dff37f740 1 -- 172.21.15.43:0/2981092378 wait complete. 2024-09-11T23:23:28.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.318+0000 7f7dff37f740 1 -- start start 2024-09-11T23:23:28.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.318+0000 7f7dff37f740 1 Processor -- start 2024-09-11T23:23:28.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.319+0000 7f7dff37f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dc7d85d0 0x5629dc7dee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:28.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.319+0000 7f7dff37f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7d9b40 0x5629dc7ef890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:28.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.319+0000 7f7dff37f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7e7780 0x5629dc7efdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:28.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.319+0000 7f7dedd9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7d9b40 0x5629dc7ef890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:28.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.319+0000 7f7dedd9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7d9b40 0x5629dc7ef890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:45452/0 (socket says 172.21.15.43:45452) 2024-09-11T23:23:28.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.319+0000 7f7df6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7e7780 0x5629dc7efdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:28.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.319+0000 7f7dedd9b640 1 -- 172.21.15.43:0/1609381155 learned_addr learned my addr 172.21.15.43:0/1609381155 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:28.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.319+0000 7f7dff37f740 1 -- 172.21.15.43:0/1609381155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7df81a9cf0 con 0x5629dc7e7780 2024-09-11T23:23:28.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.319+0000 7f7dff37f740 1 -- 172.21.15.43:0/1609381155 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7df81ab160 con 0x5629dc7d9b40 2024-09-11T23:23:28.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.319+0000 7f7dff37f740 1 -- 172.21.15.43:0/1609381155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7df81aa3c0 con 0x5629dc7d85d0 2024-09-11T23:23:28.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.319+0000 7f7dff37f740 10 client.0 _ll_register_callbacks cb 0x7f7de8005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:28.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.320+0000 7f7df659c640 1 --2- 172.21.15.43:0/1609381155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dc7d85d0 0x5629dc7dee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:28.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.320+0000 7f7dedd9b640 1 -- 172.21.15.43:0/1609381155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dc7d85d0 msgr2=0x5629dc7dee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:28.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.320+0000 7f7dedd9b640 1 --2- 172.21.15.43:0/1609381155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dc7d85d0 0x5629dc7dee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.320+0000 7f7dedd9b640 1 -- 172.21.15.43:0/1609381155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7e7780 msgr2=0x5629dc7efdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:28.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.320+0000 7f7dedd9b640 1 --2- 172.21.15.43:0/1609381155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7e7780 0x5629dc7efdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.320+0000 7f7dedd9b640 1 -- 172.21.15.43:0/1609381155 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629dc7db120 con 0x5629dc7d9b40 2024-09-11T23:23:28.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.320+0000 7f7df6d9d640 1 --2- 172.21.15.43:0/1609381155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7e7780 0x5629dc7efdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:23:28.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.320+0000 7f7dedd9b640 1 --2- 172.21.15.43:0/1609381155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7d9b40 0x5629dc7ef890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009770 tx=0x7f7de00095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aba8ac78ef1876b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:28.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.320+0000 7f7dddffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:23:28.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.321+0000 7f7dddffb640 1 -- 172.21.15.43:0/1609381155 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0030020 con 0x5629dc7d9b40 2024-09-11T23:23:28.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.321+0000 7f7dddffb640 1 -- 172.21.15.43:0/1609381155 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f7de0004200 con 0x5629dc7d9b40 2024-09-11T23:23:28.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.321+0000 7f7dff37f740 1 -- 172.21.15.43:0/1609381155 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7df0025070 con 0x5629dc7d9b40 2024-09-11T23:23:28.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.321+0000 7f7dddffb640 1 -- 172.21.15.43:0/1609381155 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7de0009e80 con 0x5629dc7d9b40 2024-09-11T23:23:28.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.321+0000 7f7dff37f740 10 client.25346 fetch_fsmap learned FSMap version 26 2024-09-11T23:23:28.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.321+0000 7f7dff37f740 1 -- 172.21.15.43:0/1609381155 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5629dc7e22b0 con 0x5629dc7d9b40 2024-09-11T23:23:28.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.321+0000 7f7dddffb640 1 -- 172.21.15.43:0/1609381155 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7df0025070 con 0x5629dc7d9b40 2024-09-11T23:23:28.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.322+0000 7f7dddffb640 1 -- 172.21.15.43:0/1609381155 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7de002b050 con 0x5629dc7d9b40 2024-09-11T23:23:28.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.323+0000 7f7dff37f740 10 client.25346 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:23:28.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.323+0000 7f7dff37f740 10 client.25346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:28.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.323+0000 7f7dff37f740 20 client.25346 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:28.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.323+0000 7f7dff37f740 1 --2- 172.21.15.43:0/1609381155 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5629dc866bb0 0x5629dc868fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:28.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.323+0000 7f7dff37f740 4 client.25346 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:23:28.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.323+0000 7f7dff37f740 1 -- 172.21.15.43:0/1609381155 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629dc745d70 con 0x5629dc866bb0 2024-09-11T23:23:28.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.324+0000 7f7df659c640 1 --2- 172.21.15.43:0/1609381155 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5629dc866bb0 0x5629dc868fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:28.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.325+0000 7f7dddffb640 10 client.25346 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:28.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.325+0000 7f7df659c640 1 --2- 172.21.15.43:0/1609381155 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5629dc866bb0 0x5629dc868fd0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a58303afe2c49631 server_cookie=2604d8cef8594322 in_seq=0 out_seq=0 2024-09-11T23:23:28.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.326+0000 7f7dddffb640 1 -- 172.21.15.43:0/1609381155 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629dc745d70 con 0x5629dc866bb0 2024-09-11T23:23:28.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.326+0000 7f7dddffb640 10 client.25346 handle_command_reply: tid=0 2024-09-11T23:23:28.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.326+0000 7f7dff37f740 1 client.25346 shutdown 2024-09-11T23:23:28.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.326+0000 7f7dff37f740 1 -- 172.21.15.43:0/1609381155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7d9b40 msgr2=0x5629dc7ef890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:28.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.326+0000 7f7dff37f740 1 --2- 172.21.15.43:0/1609381155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7d9b40 0x5629dc7ef890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009770 tx=0x7f7de00095b0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.326+0000 7f7dff37f740 1 -- 172.21.15.43:0/1609381155 shutdown_connections 2024-09-11T23:23:28.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.326+0000 7f7dff37f740 1 --2- 172.21.15.43:0/1609381155 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5629dc866bb0 0x5629dc868fd0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.326+0000 7f7dff37f740 1 --2- 172.21.15.43:0/1609381155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629dc7e7780 0x5629dc7efdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.326+0000 7f7dff37f740 1 --2- 172.21.15.43:0/1609381155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dc7d9b40 0x5629dc7ef890 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.326+0000 7f7dff37f740 1 --2- 172.21.15.43:0/1609381155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629dc7d85d0 0x5629dc7dee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.326+0000 7f7dddffb640 0 client.25346 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:28.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.326+0000 7f7dff37f740 1 -- 172.21.15.43:0/1609381155 >> 172.21.15.43:0/1609381155 conn(0x5629dc7a1310 msgr2=0x5629dc7e2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:28.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.327+0000 7f7dff37f740 1 -- 172.21.15.43:0/1609381155 shutdown_connections 2024-09-11T23:23:28.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.327+0000 7f7dff37f740 1 -- 172.21.15.43:0/1609381155 wait complete. 2024-09-11T23:23:28.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.327+0000 7f7dff37f740 20 client.25346 trim_cache size 0 max 16384 2024-09-11T23:23:28.328 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:23:28.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.328+0000 7f7dfcfa6640 1 -- 172.21.15.43:0/787052914 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7de40a2d70 msgr2=0x7f7de40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:28.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.328+0000 7f7dfcfa6640 1 --2- 172.21.15.43:0/787052914 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7de40a2d70 0x7f7de40a5230 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7df0005fd0 tx=0x7f7df0004ab0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.328+0000 7f7dfcfa6640 1 -- 172.21.15.43:0/787052914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8102db0 msgr2=0x7f7df81986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:28.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.328+0000 7f7dfcfa6640 1 --2- 172.21.15.43:0/787052914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8102db0 0x7f7df81986d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7de8004770 tx=0x7f7de8003b90 comp rx=0 tx=0).stop 2024-09-11T23:23:28.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.328+0000 7f7dfcfa6640 1 -- 172.21.15.43:0/787052914 shutdown_connections 2024-09-11T23:23:28.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.328+0000 7f7dfcfa6640 1 --2- 172.21.15.43:0/787052914 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7de40a2d70 0x7f7de40a5230 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.328+0000 7f7dfcfa6640 1 --2- 172.21.15.43:0/787052914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7df81082e0 0x7f7df819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.328+0000 7f7dfcfa6640 1 --2- 172.21.15.43:0/787052914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7df8103760 0x7f7df8198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.328+0000 7f7dfcfa6640 1 --2- 172.21.15.43:0/787052914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7df8102db0 0x7f7df81986d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:28.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.328+0000 7f7dfcfa6640 1 -- 172.21.15.43:0/787052914 >> 172.21.15.43:0/787052914 conn(0x7f7df80fc7d0 msgr2=0x7f7df8105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:28.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.328+0000 7f7dfcfa6640 1 -- 172.21.15.43:0/787052914 shutdown_connections 2024-09-11T23:23:28.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:28.328+0000 7f7dfcfa6640 1 -- 172.21.15.43:0/787052914 wait complete. 2024-09-11T23:23:30.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:29 smithi090 ceph-mon[35130]: pgmap v52: 97 pgs: 97 active+clean; 7.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 504 KiB/s wr, 121 op/s 2024-09-11T23:23:30.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:29 smithi097 ceph-mon[35562]: pgmap v52: 97 pgs: 97 active+clean; 7.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 504 KiB/s wr, 121 op/s 2024-09-11T23:23:30.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:29 smithi043 ceph-mon[30950]: pgmap v52: 97 pgs: 97 active+clean; 7.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 504 KiB/s wr, 121 op/s 2024-09-11T23:23:30.400 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:23:30.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:23:30.886 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:23:32.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:31 smithi090 ceph-mon[35130]: pgmap v53: 97 pgs: 97 active+clean; 7.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 504 KiB/s wr, 121 op/s 2024-09-11T23:23:32.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:31 smithi097 ceph-mon[35562]: pgmap v53: 97 pgs: 97 active+clean; 7.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 504 KiB/s wr, 121 op/s 2024-09-11T23:23:32.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:31 smithi043 ceph-mon[30950]: pgmap v53: 97 pgs: 97 active+clean; 7.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 504 KiB/s wr, 121 op/s 2024-09-11T23:23:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.916+0000 7f6511247640 1 -- 172.21.15.43:0/2870301101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f650c116ea0 msgr2=0x7f650c119290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.917+0000 7f6511247640 1 --2- 172.21.15.43:0/2870301101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f650c116ea0 0x7f650c119290 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6504009c10 tx=0x7f650402a6f0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.921+0000 7f6511247640 1 -- 172.21.15.43:0/2870301101 shutdown_connections 2024-09-11T23:23:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.921+0000 7f6511247640 1 --2- 172.21.15.43:0/2870301101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f650c116ea0 0x7f650c119290 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.921+0000 7f6511247640 1 --2- 172.21.15.43:0/2870301101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f650c109b10 0x7f650c116960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.921+0000 7f6511247640 1 --2- 172.21.15.43:0/2870301101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f650c1091f0 0x7f650c1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.921+0000 7f6511247640 1 -- 172.21.15.43:0/2870301101 >> 172.21.15.43:0/2870301101 conn(0x7f650c06d200 msgr2=0x7f650c06d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.925+0000 7f6511247640 1 -- 172.21.15.43:0/2870301101 shutdown_connections 2024-09-11T23:23:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.925+0000 7f6511247640 1 -- 172.21.15.43:0/2870301101 wait complete. 2024-09-11T23:23:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.928+0000 7f6511247640 1 Processor -- start 2024-09-11T23:23:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.928+0000 7f6511247640 1 -- start start 2024-09-11T23:23:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.928+0000 7f6511247640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f650c1091f0 0x7f650c1a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.929+0000 7f6511247640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f650c109b10 0x7f650c1a3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.929+0000 7f6511247640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f650c116ea0 0x7f650c1a7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.929+0000 7f6511247640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f650c11c880 con 0x7f650c1091f0 2024-09-11T23:23:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.929+0000 7f6511247640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f650c11c700 con 0x7f650c116ea0 2024-09-11T23:23:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.929+0000 7f6511247640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f650c11ca00 con 0x7f650c109b10 2024-09-11T23:23:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.930+0000 7f650ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f650c1091f0 0x7f650c1a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.931+0000 7f650ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f650c1091f0 0x7f650c1a2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53930/0 (socket says 172.21.15.43:53930) 2024-09-11T23:23:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.931+0000 7f650ad9d640 1 -- 172.21.15.43:0/371559841 learned_addr learned my addr 172.21.15.43:0/371559841 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.931+0000 7f650b59e640 1 --2- 172.21.15.43:0/371559841 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f650c116ea0 0x7f650c1a7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.932+0000 7f650a59c640 1 --2- 172.21.15.43:0/371559841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f650c109b10 0x7f650c1a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.933+0000 7f650ad9d640 1 -- 172.21.15.43:0/371559841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f650c109b10 msgr2=0x7f650c1a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.933+0000 7f650ad9d640 1 --2- 172.21.15.43:0/371559841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f650c109b10 0x7f650c1a3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.933+0000 7f650ad9d640 1 -- 172.21.15.43:0/371559841 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f650c116ea0 msgr2=0x7f650c1a7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.933+0000 7f650ad9d640 1 --2- 172.21.15.43:0/371559841 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f650c116ea0 0x7f650c1a7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.933+0000 7f650ad9d640 1 -- 172.21.15.43:0/371559841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650c1a7dc0 con 0x7f650c1091f0 2024-09-11T23:23:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.935+0000 7f650ad9d640 1 --2- 172.21.15.43:0/371559841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f650c1091f0 0x7f650c1a2dd0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f64f400e9f0 tx=0x7f64f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.936+0000 7f6503fff640 1 -- 172.21.15.43:0/371559841 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f400cd80 con 0x7f650c1091f0 2024-09-11T23:23:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.936+0000 7f6503fff640 1 -- 172.21.15.43:0/371559841 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64f4010910 con 0x7f650c1091f0 2024-09-11T23:23:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.937+0000 7f6503fff640 1 -- 172.21.15.43:0/371559841 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f40188f0 con 0x7f650c1091f0 2024-09-11T23:23:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.937+0000 7f6511247640 1 -- 172.21.15.43:0/371559841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f650c1a80b0 con 0x7f650c1091f0 2024-09-11T23:23:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.937+0000 7f6511247640 1 -- 172.21.15.43:0/371559841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f650c1af940 con 0x7f650c1091f0 2024-09-11T23:23:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.941+0000 7f6513fdf740 1 Processor -- start 2024-09-11T23:23:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.942+0000 7f6513fdf740 1 -- start start 2024-09-11T23:23:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.942+0000 7f6513fdf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56196b7ddaf0 0x56196b7dded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.942+0000 7f6513fdf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7de410 0x56196b7e8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.942+0000 7f6513fdf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56196b7e8cb0 0x56196b7eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.942+0000 7f6513fdf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56196b7422f0 con 0x56196b7de410 2024-09-11T23:23:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.942+0000 7f6513fdf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56196b6ea270 con 0x56196b7e8cb0 2024-09-11T23:23:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.942+0000 7f6513fdf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56196b73e2b0 con 0x56196b7ddaf0 2024-09-11T23:23:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.944+0000 7f6503fff640 1 -- 172.21.15.43:0/371559841 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f64f4010ac0 con 0x7f650c1091f0 2024-09-11T23:23:33.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.702+0000 7f14f82ac640 1 -- 172.21.15.43:0/1187119112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14f0109120 msgr2=0x7f14f0110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:33.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.702+0000 7f14f82ac640 1 --2- 172.21.15.43:0/1187119112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14f0109120 0x7f14f0110ca0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009ca0 tx=0x7f14e802a8f0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.704+0000 7f14f82ac640 1 -- 172.21.15.43:0/1187119112 shutdown_connections 2024-09-11T23:23:33.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.704+0000 7f14f82ac640 1 --2- 172.21.15.43:0/1187119112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14f0109120 0x7f14f0110ca0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.704+0000 7f14f82ac640 1 --2- 172.21.15.43:0/1187119112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14f01049a0 0x7f14f01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.704+0000 7f14f82ac640 1 --2- 172.21.15.43:0/1187119112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14f0103ff0 0x7f14f01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.704+0000 7f14f82ac640 1 -- 172.21.15.43:0/1187119112 >> 172.21.15.43:0/1187119112 conn(0x7f14f00ffab0 msgr2=0x7f14f0101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:33.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.706+0000 7f14f82ac640 1 -- 172.21.15.43:0/1187119112 shutdown_connections 2024-09-11T23:23:33.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.707+0000 7f14f82ac640 1 -- 172.21.15.43:0/1187119112 wait complete. 2024-09-11T23:23:33.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.708+0000 7f14f82ac640 1 Processor -- start 2024-09-11T23:23:33.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.708+0000 7f14f82ac640 1 -- start start 2024-09-11T23:23:33.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.709+0000 7f14f82ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14f0103ff0 0x7f14f019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.709+0000 7f14f82ac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14f01049a0 0x7f14f019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.709+0000 7f14f6048640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14f0103ff0 0x7f14f019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.710+0000 7f14f6048640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14f0103ff0 0x7f14f019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53888/0 (socket says 172.21.15.43:53888) 2024-09-11T23:23:33.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.710+0000 7f14f6048640 1 -- 172.21.15.43:0/1134230480 learned_addr learned my addr 172.21.15.43:0/1134230480 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:33.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.710+0000 7f14f82ac640 1 --2- 172.21.15.43:0/1134230480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14f0109120 0x7f14f019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.710+0000 7f14f82ac640 1 -- 172.21.15.43:0/1134230480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14f01142b0 con 0x7f14f0103ff0 2024-09-11T23:23:33.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.710+0000 7f14f82ac640 1 -- 172.21.15.43:0/1134230480 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f14f0114130 con 0x7f14f0109120 2024-09-11T23:23:33.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.710+0000 7f14f82ac640 1 -- 172.21.15.43:0/1134230480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f14f0114430 con 0x7f14f01049a0 2024-09-11T23:23:33.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.711+0000 7f14f6048640 1 -- 172.21.15.43:0/1134230480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14f01049a0 msgr2=0x7f14f019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:33.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.711+0000 7f14f6048640 1 --2- 172.21.15.43:0/1134230480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14f01049a0 0x7f14f019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.711+0000 7f14f6048640 1 -- 172.21.15.43:0/1134230480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14f0109120 msgr2=0x7f14f019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:33.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.711+0000 7f14f6048640 1 --2- 172.21.15.43:0/1134230480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14f0109120 0x7f14f019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.711+0000 7f14f6048640 1 -- 172.21.15.43:0/1134230480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f019f540 con 0x7f14f0103ff0 2024-09-11T23:23:33.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.712+0000 7f14f6048640 1 --2- 172.21.15.43:0/1134230480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14f0103ff0 0x7f14f019a590 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00b450 tx=0x7f14ec00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:33.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.713+0000 7f14e77fe640 1 -- 172.21.15.43:0/1134230480 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec015020 con 0x7f14f0103ff0 2024-09-11T23:23:33.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.714+0000 7f14e77fe640 1 -- 172.21.15.43:0/1134230480 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14ec011d70 con 0x7f14f0103ff0 2024-09-11T23:23:33.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.714+0000 7f14e77fe640 1 -- 172.21.15.43:0/1134230480 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec00f340 con 0x7f14f0103ff0 2024-09-11T23:23:33.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.715+0000 7f14f82ac640 1 -- 172.21.15.43:0/1134230480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f019f830 con 0x7f14f0103ff0 2024-09-11T23:23:33.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.715+0000 7f14f82ac640 1 -- 172.21.15.43:0/1134230480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f01a7110 con 0x7f14f0103ff0 2024-09-11T23:23:33.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.723+0000 7f14e77fe640 1 -- 172.21.15.43:0/1134230480 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f14ec018020 con 0x7f14f0103ff0 2024-09-11T23:23:33.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.724+0000 7f14fa685740 1 Processor -- start 2024-09-11T23:23:33.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.724+0000 7f14fa685740 1 -- start start 2024-09-11T23:23:33.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.724+0000 7f14fa685740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56432c9ad370 0x56432c9ad750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.725+0000 7f14fa685740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56432c9adc90 0x56432c9b8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.725+0000 7f14fa685740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9b8550 0x56432c9ba940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.725+0000 7f14fa685740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56432c90a280 con 0x56432c9b8550 2024-09-11T23:23:33.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.725+0000 7f14fa685740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56432c8b1b10 con 0x56432c9ad370 2024-09-11T23:23:33.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.725+0000 7f14fa685740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56432c906240 con 0x56432c9adc90 2024-09-11T23:23:33.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.725+0000 7f14e77fe640 1 --2- 172.21.15.43:0/1134230480 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f14dc0a2ec0 0x7f14dc0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.725+0000 7f14f5847640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9b8550 0x56432c9ba940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.725+0000 7f14f6849640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56432c9ad370 0x56432c9ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.725+0000 7f14f6849640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56432c9ad370 0x56432c9ad750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:52964/0 (socket says 172.21.15.43:52964) 2024-09-11T23:23:33.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.725+0000 7f14f6849640 1 -- 172.21.15.43:0/1855478552 learned_addr learned my addr 172.21.15.43:0/1855478552 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:33.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.725+0000 7f14f5847640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9b8550 0x56432c9ba940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53898/0 (socket says 172.21.15.43:53898) 2024-09-11T23:23:33.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.726+0000 7f14f6048640 1 --2- 172.21.15.43:0/1855478552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56432c9adc90 0x56432c9b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.726+0000 7f14f5847640 1 -- 172.21.15.43:0/1855478552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56432c9adc90 msgr2=0x56432c9b8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:33.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.726+0000 7f14f6849640 1 --2- 172.21.15.43:0/1134230480 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f14dc0a2ec0 0x7f14dc0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.726+0000 7f14f5847640 1 --2- 172.21.15.43:0/1855478552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56432c9adc90 0x56432c9b8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.726+0000 7f14f5847640 1 -- 172.21.15.43:0/1855478552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56432c9ad370 msgr2=0x56432c9ad750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:33.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.726+0000 7f14f5847640 1 --2- 172.21.15.43:0/1855478552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56432c9ad370 0x56432c9ad750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.726+0000 7f14f5847640 1 -- 172.21.15.43:0/1855478552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56432c8be1f0 con 0x56432c9b8550 2024-09-11T23:23:33.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.726+0000 7f14e77fe640 1 -- 172.21.15.43:0/1134230480 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f14ec0cecf0 con 0x7f14f0103ff0 2024-09-11T23:23:33.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.726+0000 7f14f6849640 1 --2- 172.21.15.43:0/1134230480 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f14dc0a2ec0 0x7f14dc0a5380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009a60 tx=0x7f14e8005df0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:33.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.727+0000 7f14f5847640 1 --2- 172.21.15.43:0/1855478552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9b8550 0x56432c9ba940 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f14e00096f0 tx=0x7f14e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5290b248e9103f64 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:33.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.727+0000 7f14e4dea640 1 -- 172.21.15.43:0/1855478552 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0015020 con 0x56432c9b8550 2024-09-11T23:23:33.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.727+0000 7f14e4dea640 1 -- 172.21.15.43:0/1855478552 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e000bdb0 con 0x56432c9b8550 2024-09-11T23:23:33.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.727+0000 7f14e4dea640 1 -- 172.21.15.43:0/1855478552 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0010380 con 0x56432c9b8550 2024-09-11T23:23:33.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.728+0000 7f14fa685740 1 -- 172.21.15.43:0/1855478552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9b8550 msgr2=0x56432c9ba940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:33.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.728+0000 7f14fa685740 1 --2- 172.21.15.43:0/1855478552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9b8550 0x56432c9ba940 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f14e00096f0 tx=0x7f14e0009bb0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.729+0000 7f14fa685740 1 -- 172.21.15.43:0/1855478552 shutdown_connections 2024-09-11T23:23:33.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.729+0000 7f14fa685740 1 --2- 172.21.15.43:0/1855478552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9b8550 0x56432c9ba940 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.729+0000 7f14fa685740 1 --2- 172.21.15.43:0/1855478552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56432c9adc90 0x56432c9b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.729+0000 7f14fa685740 1 --2- 172.21.15.43:0/1855478552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56432c9ad370 0x56432c9ad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.729+0000 7f14fa685740 1 -- 172.21.15.43:0/1855478552 >> 172.21.15.43:0/1855478552 conn(0x56432c9a9900 msgr2=0x56432c9a9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:33.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.729+0000 7f14fa685740 1 -- 172.21.15.43:0/1855478552 shutdown_connections 2024-09-11T23:23:33.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.729+0000 7f14fa685740 1 -- 172.21.15.43:0/1855478552 wait complete. 2024-09-11T23:23:33.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.730+0000 7f14fa685740 1 -- start start 2024-09-11T23:23:33.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.730+0000 7f14fa685740 1 Processor -- start 2024-09-11T23:23:33.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.730+0000 7f14fa685740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9ad370 0x56432c9b5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.731+0000 7f14f6048640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9ad370 0x56432c9b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.731+0000 7f14fa685740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56432c9adc90 0x56432c9b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.731+0000 7f14fa685740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56432c9b8550 0x56432c9b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.731+0000 7f14fa685740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56432c9bb1c0 con 0x56432c9ad370 2024-09-11T23:23:33.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.731+0000 7f14f6048640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9ad370 0x56432c9b5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53902/0 (socket says 172.21.15.43:53902) 2024-09-11T23:23:33.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.732+0000 7f14f6048640 1 -- 172.21.15.43:0/1666133535 learned_addr learned my addr 172.21.15.43:0/1666133535 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:33.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.732+0000 7f14fa685740 1 -- 172.21.15.43:0/1666133535 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56432c8c4f10 con 0x56432c9b8550 2024-09-11T23:23:33.954 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.945+0000 7f650b59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56196b7ddaf0 0x56196b7dded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.732+0000 7f14fa685740 1 -- 172.21.15.43:0/1666133535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56432c9bb340 con 0x56432c9adc90 2024-09-11T23:23:33.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.732+0000 7f14fa685740 10 client.0 _ll_register_callbacks cb 0x56432c88da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:33.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.732+0000 7f14f6048640 1 -- 172.21.15.43:0/1666133535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56432c9adc90 msgr2=0x56432c9b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:33.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.732+0000 7f14f6048640 1 --2- 172.21.15.43:0/1666133535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56432c9adc90 0x56432c9b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.732+0000 7f14f6048640 1 -- 172.21.15.43:0/1666133535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56432c9b8550 msgr2=0x56432c9b5e00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:23:33.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.732+0000 7f14f6048640 1 --2- 172.21.15.43:0/1666133535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56432c9b8550 0x56432c9b5e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.732+0000 7f14f6048640 1 -- 172.21.15.43:0/1666133535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56432c67b6a0 con 0x56432c9ad370 2024-09-11T23:23:33.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.733+0000 7f14f6048640 1 --2- 172.21.15.43:0/1666133535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9ad370 0x56432c9b5380 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f14ec097720 tx=0x7f14ec09ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d39a08dafd1d526 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:33.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.733+0000 7f14d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:23:33.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.733+0000 7f14d9ffb640 1 -- 172.21.15.43:0/1666133535 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec0a3f30 con 0x56432c9ad370 2024-09-11T23:23:33.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.734+0000 7f14d9ffb640 1 -- 172.21.15.43:0/1666133535 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f14ec0cc660 con 0x56432c9ad370 2024-09-11T23:23:33.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.734+0000 7f14fa685740 1 -- 172.21.15.43:0/1666133535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14e000bdb0 con 0x56432c9ad370 2024-09-11T23:23:33.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.734+0000 7f14d9ffb640 1 -- 172.21.15.43:0/1666133535 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f14ec097930 con 0x56432c9ad370 2024-09-11T23:23:33.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.734+0000 7f14fa685740 10 client.15786 fetch_fsmap learned FSMap version 26 2024-09-11T23:23:33.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.734+0000 7f14fa685740 1 -- 172.21.15.43:0/1666133535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56432c9c6d70 con 0x56432c9ad370 2024-09-11T23:23:33.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.734+0000 7f14d9ffb640 1 -- 172.21.15.43:0/1666133535 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e000bdb0 con 0x56432c9ad370 2024-09-11T23:23:33.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.735+0000 7f14d9ffb640 1 -- 172.21.15.43:0/1666133535 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f14ec022060 con 0x56432c9ad370 2024-09-11T23:23:33.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.735+0000 7f14fa685740 10 client.15786 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:23:33.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.735+0000 7f14fa685740 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:33.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.735+0000 7f14fa685740 20 client.15786 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:33.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.735+0000 7f14fa685740 1 --2- 172.21.15.43:0/1666133535 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56432c9c8230 0x56432c9ca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.735+0000 7f14fa685740 4 client.15786 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:23:33.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.735+0000 7f14fa685740 1 -- 172.21.15.43:0/1666133535 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56432c9177f0 con 0x56432c9c8230 2024-09-11T23:23:33.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.737+0000 7f14f5847640 1 --2- 172.21.15.43:0/1666133535 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56432c9c8230 0x56432c9ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.742+0000 7f14f5847640 1 --2- 172.21.15.43:0/1666133535 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56432c9c8230 0x56432c9ca680 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67185a874362478 server_cookie=d531b92a9223b0f5 in_seq=0 out_seq=0 2024-09-11T23:23:33.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.742+0000 7f14d9ffb640 10 client.15786 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:33.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.744+0000 7f14d9ffb640 1 -- 172.21.15.43:0/1666133535 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56432c9177f0 con 0x56432c9c8230 2024-09-11T23:23:33.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.744+0000 7f14d9ffb640 10 client.15786 handle_command_reply: tid=0 2024-09-11T23:23:33.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.744+0000 7f14fa685740 1 client.15786 shutdown 2024-09-11T23:23:33.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.744+0000 7f14fa685740 1 -- 172.21.15.43:0/1666133535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9ad370 msgr2=0x56432c9b5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:33.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.744+0000 7f14fa685740 1 --2- 172.21.15.43:0/1666133535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9ad370 0x56432c9b5380 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f14ec097720 tx=0x7f14ec09ba80 comp rx=0 tx=0).stop 2024-09-11T23:23:33.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.744+0000 7f14fa685740 1 -- 172.21.15.43:0/1666133535 shutdown_connections 2024-09-11T23:23:33.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.744+0000 7f14fa685740 1 --2- 172.21.15.43:0/1666133535 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56432c9c8230 0x56432c9ca680 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.744+0000 7f14fa685740 1 --2- 172.21.15.43:0/1666133535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56432c9b8550 0x56432c9b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.744+0000 7f14fa685740 1 --2- 172.21.15.43:0/1666133535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56432c9adc90 0x56432c9b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.744+0000 7f14fa685740 1 --2- 172.21.15.43:0/1666133535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9ad370 0x56432c9b5380 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.744+0000 7f14d9ffb640 0 client.15786 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:33.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.744+0000 7f14fa685740 1 -- 172.21.15.43:0/1666133535 >> 172.21.15.43:0/1666133535 conn(0x56432c9a9900 msgr2=0x56432c9a0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:33.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.744+0000 7f14fa685740 1 -- 172.21.15.43:0/1666133535 shutdown_connections 2024-09-11T23:23:33.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.745+0000 7f14fa685740 1 -- 172.21.15.43:0/1666133535 wait complete. 2024-09-11T23:23:33.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.745+0000 7f14fa685740 20 client.15786 trim_cache size 0 max 16384 2024-09-11T23:23:33.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.796+0000 7f14fa685740 1 Processor -- start 2024-09-11T23:23:33.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.796+0000 7f14fa685740 1 -- start start 2024-09-11T23:23:33.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.797+0000 7f14fa685740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56432c9ad370 0x56432c9cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.797+0000 7f14fa685740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56432c9adc90 0x56432c9d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.797+0000 7f14fa685740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9b8550 0x56432c9b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.797+0000 7f14fa685740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14f01ae1d0 con 0x56432c9b8550 2024-09-11T23:23:33.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.798+0000 7f14fa685740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f14f01abaa0 con 0x56432c9adc90 2024-09-11T23:23:33.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.798+0000 7f14fa685740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f14f01a9930 con 0x56432c9ad370 2024-09-11T23:23:33.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.801+0000 7f14f6849640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9b8550 0x56432c9b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.801+0000 7f14f6849640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9b8550 0x56432c9b25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53910/0 (socket says 172.21.15.43:53910) 2024-09-11T23:23:33.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.801+0000 7f14f6849640 1 -- 172.21.15.43:0/3673819432 learned_addr learned my addr 172.21.15.43:0/3673819432 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:33.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.801+0000 7f14f6048640 1 --2- 172.21.15.43:0/3673819432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56432c9ad370 0x56432c9cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.801+0000 7f14f5847640 1 --2- 172.21.15.43:0/3673819432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56432c9adc90 0x56432c9d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.804+0000 7f14f6849640 1 -- 172.21.15.43:0/3673819432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56432c9ad370 msgr2=0x56432c9cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:33.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.804+0000 7f14f6849640 1 --2- 172.21.15.43:0/3673819432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56432c9ad370 0x56432c9cff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.804+0000 7f14f6849640 1 -- 172.21.15.43:0/3673819432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56432c9adc90 msgr2=0x56432c9d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:33.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.804+0000 7f14f6849640 1 --2- 172.21.15.43:0/3673819432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56432c9adc90 0x56432c9d04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.804+0000 7f14f6849640 1 -- 172.21.15.43:0/3673819432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56432c9aa4b0 con 0x56432c9b8550 2024-09-11T23:23:33.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.805+0000 7f14f6849640 1 --2- 172.21.15.43:0/3673819432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9b8550 0x56432c9b25e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009880 tx=0x7f14e8036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c057f8cc9dd18b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:33.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.806+0000 7f14dbfff640 1 -- 172.21.15.43:0/3673819432 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8031890 con 0x56432c9b8550 2024-09-11T23:23:33.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.806+0000 7f14dbfff640 1 -- 172.21.15.43:0/3673819432 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e8036070 con 0x56432c9b8550 2024-09-11T23:23:33.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.808+0000 7f14fa685740 1 -- 172.21.15.43:0/3673819432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9b8550 msgr2=0x56432c9b25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:33.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.808+0000 7f14fa685740 1 --2- 172.21.15.43:0/3673819432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9b8550 0x56432c9b25e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009880 tx=0x7f14e8036000 comp rx=0 tx=0).stop 2024-09-11T23:23:33.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.809+0000 7f14fa685740 1 -- 172.21.15.43:0/3673819432 shutdown_connections 2024-09-11T23:23:33.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.809+0000 7f14fa685740 1 --2- 172.21.15.43:0/3673819432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9b8550 0x56432c9b25e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.809+0000 7f14fa685740 1 --2- 172.21.15.43:0/3673819432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56432c9adc90 0x56432c9d04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.809+0000 7f14fa685740 1 --2- 172.21.15.43:0/3673819432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56432c9ad370 0x56432c9cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.809+0000 7f14fa685740 1 -- 172.21.15.43:0/3673819432 >> 172.21.15.43:0/3673819432 conn(0x56432c9a9900 msgr2=0x56432c9a0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:33.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.809+0000 7f14fa685740 1 -- 172.21.15.43:0/3673819432 shutdown_connections 2024-09-11T23:23:33.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.809+0000 7f14fa685740 1 -- 172.21.15.43:0/3673819432 wait complete. 2024-09-11T23:23:33.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.810+0000 7f14fa685740 1 -- start start 2024-09-11T23:23:33.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.810+0000 7f14fa685740 1 Processor -- start 2024-09-11T23:23:33.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.811+0000 7f14fa685740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9ad370 0x56432c9b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.811+0000 7f14fa685740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56432c9adc90 0x56432c9b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.811+0000 7f14fa685740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56432c9b8550 0x56432c9b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.811+0000 7f14fa685740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14f01a9f80 con 0x56432c9ad370 2024-09-11T23:23:33.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.811+0000 7f14fa685740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f14f01ab3d0 con 0x56432c9adc90 2024-09-11T23:23:33.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.811+0000 7f14fa685740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f14f01aa630 con 0x56432c9b8550 2024-09-11T23:23:33.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.811+0000 7f14fa685740 10 client.0 _ll_register_callbacks cb 0x7f14ec09ae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.945+0000 7f650a59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56196b7e8cb0 0x56196b7eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.945+0000 7f650a59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56196b7e8cb0 0x56196b7eb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:53020/0 (socket says 172.21.15.43:53020) 2024-09-11T23:23:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.945+0000 7f650a59c640 1 -- 172.21.15.43:0/1702309589 learned_addr learned my addr 172.21.15.43:0/1702309589 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.946+0000 7f6503fff640 1 --2- 172.21.15.43:0/371559841 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f64f80a2ec0 0x7f64f80a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.946+0000 7f650ad9d640 1 --2- 172.21.15.43:0/1702309589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7de410 0x56196b7e8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.946+0000 7f6503fff640 1 -- 172.21.15.43:0/371559841 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f64f40cf9e0 con 0x7f650c1091f0 2024-09-11T23:23:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.946+0000 7f650a59c640 1 -- 172.21.15.43:0/1702309589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56196b7ddaf0 msgr2=0x56196b7dded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.946+0000 7f650a59c640 1 --2- 172.21.15.43:0/1702309589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56196b7ddaf0 0x56196b7dded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.946+0000 7f650a59c640 1 -- 172.21.15.43:0/1702309589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7de410 msgr2=0x56196b7e8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.946+0000 7f650a59c640 1 --2- 172.21.15.43:0/1702309589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7de410 0x56196b7e8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.946+0000 7f650a59c640 1 -- 172.21.15.43:0/1702309589 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56196b6e9340 con 0x56196b7e8cb0 2024-09-11T23:23:33.962 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:23:33.962 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-11T23:23:33.962 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "49e8eaf5-3fe5-43ab-9b96-c1293a57cccb", 2024-09-11T23:23:33.962 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-11T23:23:33.962 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:23:33.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.813+0000 7f14f5847640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56432c9adc90 0x56432c9b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.813+0000 7f14f6849640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56432c9b8550 0x56432c9b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.818+0000 7f14f6048640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9ad370 0x56432c9b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.818+0000 7f14f6048640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9ad370 0x56432c9b2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53912/0 (socket says 172.21.15.43:53912) 2024-09-11T23:23:33.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.818+0000 7f14f6048640 1 -- 172.21.15.43:0/2772563357 learned_addr learned my addr 172.21.15.43:0/2772563357 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:33.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.823+0000 7f14f6849640 1 -- 172.21.15.43:0/2772563357 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56432c9adc90 msgr2=0x56432c9b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:33.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.823+0000 7f14f6849640 1 --2- 172.21.15.43:0/2772563357 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56432c9adc90 0x56432c9b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.823+0000 7f14f6849640 1 -- 172.21.15.43:0/2772563357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9ad370 msgr2=0x56432c9b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:33.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.823+0000 7f14f6849640 1 --2- 172.21.15.43:0/2772563357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9ad370 0x56432c9b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.823+0000 7f14f6849640 1 -- 172.21.15.43:0/2772563357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56432c9bec50 con 0x56432c9b8550 2024-09-11T23:23:33.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.829+0000 7f14f6849640 1 --2- 172.21.15.43:0/2772563357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56432c9b8550 0x56432c9b2f10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f14e8006010 tx=0x7f14e8031aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c04cb5bed732a93 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:33.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.829+0000 7f14d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-11T23:23:33.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.832+0000 7f14d9ffb640 1 -- 172.21.15.43:0/2772563357 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e802f3a0 con 0x56432c9b8550 2024-09-11T23:23:33.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.832+0000 7f14d9ffb640 1 -- 172.21.15.43:0/2772563357 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f14e8004d50 con 0x56432c9b8550 2024-09-11T23:23:33.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.832+0000 7f14fa685740 1 -- 172.21.15.43:0/2772563357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14e8036070 con 0x56432c9b8550 2024-09-11T23:23:33.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.835+0000 7f14d9ffb640 1 -- 172.21.15.43:0/2772563357 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f14e8063050 con 0x56432c9b8550 2024-09-11T23:23:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.835+0000 7f14fa685740 10 client.25354 fetch_fsmap learned FSMap version 26 2024-09-11T23:23:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.835+0000 7f14fa685740 1 -- 172.21.15.43:0/2772563357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56432ca2d0a0 con 0x56432c9b8550 2024-09-11T23:23:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.837+0000 7f14d9ffb640 1 -- 172.21.15.43:0/2772563357 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e8036070 con 0x56432c9b8550 2024-09-11T23:23:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.837+0000 7f14d9ffb640 1 -- 172.21.15.43:0/2772563357 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f14e80305f0 con 0x56432c9b8550 2024-09-11T23:23:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.840+0000 7f14fa685740 10 client.25354 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:23:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.840+0000 7f14fa685740 10 client.25354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.840+0000 7f14fa685740 20 client.25354 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.840+0000 7f14fa685740 1 --2- 172.21.15.43:0/2772563357 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56432ca2d720 0x56432ca2fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.840+0000 7f14fa685740 4 client.25354 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:23:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.840+0000 7f14fa685740 1 -- 172.21.15.43:0/2772563357 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56432c911d00 con 0x56432ca2d720 2024-09-11T23:23:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.841+0000 7f14f6048640 1 --2- 172.21.15.43:0/2772563357 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56432ca2d720 0x56432ca2fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.843+0000 7f14f6048640 1 --2- 172.21.15.43:0/2772563357 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56432ca2d720 0x56432ca2fb40 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c81182bccb1eccc server_cookie=719d9b1ffdf1f409 in_seq=0 out_seq=0 2024-09-11T23:23:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.843+0000 7f14d9ffb640 10 client.25354 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:33.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.844+0000 7f14d9ffb640 1 -- 172.21.15.43:0/2772563357 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56432c911d00 con 0x56432ca2d720 2024-09-11T23:23:33.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.844+0000 7f14d9ffb640 10 client.25354 handle_command_reply: tid=0 2024-09-11T23:23:33.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.844+0000 7f14fa685740 1 client.25354 shutdown 2024-09-11T23:23:33.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.845+0000 7f14fa685740 1 -- 172.21.15.43:0/2772563357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56432c9b8550 msgr2=0x56432c9b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:33.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.845+0000 7f14fa685740 1 --2- 172.21.15.43:0/2772563357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56432c9b8550 0x56432c9b2f10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f14e8006010 tx=0x7f14e8031aa0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.845+0000 7f14fa685740 1 -- 172.21.15.43:0/2772563357 shutdown_connections 2024-09-11T23:23:33.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.845+0000 7f14fa685740 1 --2- 172.21.15.43:0/2772563357 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56432ca2d720 0x56432ca2fb40 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.845+0000 7f14fa685740 1 --2- 172.21.15.43:0/2772563357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56432c9b8550 0x56432c9b2f10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.845+0000 7f14fa685740 1 --2- 172.21.15.43:0/2772563357 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56432c9adc90 0x56432c9b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.845+0000 7f14fa685740 1 --2- 172.21.15.43:0/2772563357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56432c9ad370 0x56432c9b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.845+0000 7f14fa685740 1 -- 172.21.15.43:0/2772563357 >> 172.21.15.43:0/2772563357 conn(0x56432c9a9900 msgr2=0x56432c9b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:33.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.846+0000 7f14d9ffb640 0 client.25354 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:33.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.846+0000 7f14fa685740 1 -- 172.21.15.43:0/2772563357 shutdown_connections 2024-09-11T23:23:33.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.846+0000 7f14fa685740 1 -- 172.21.15.43:0/2772563357 wait complete. 2024-09-11T23:23:33.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.846+0000 7f14fa685740 20 client.25354 trim_cache size 0 max 16384 2024-09-11T23:23:33.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.849+0000 7f14f82ac640 1 -- 172.21.15.43:0/1134230480 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f14dc0a2ec0 msgr2=0x7f14dc0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:33.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.849+0000 7f14f82ac640 1 --2- 172.21.15.43:0/1134230480 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f14dc0a2ec0 0x7f14dc0a5380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009a60 tx=0x7f14e8005df0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.849+0000 7f14f82ac640 1 -- 172.21.15.43:0/1134230480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14f0103ff0 msgr2=0x7f14f019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:33.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.849+0000 7f14f82ac640 1 --2- 172.21.15.43:0/1134230480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14f0103ff0 0x7f14f019a590 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00b450 tx=0x7f14ec00b910 comp rx=0 tx=0).stop 2024-09-11T23:23:33.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.849+0000 7f14f82ac640 1 -- 172.21.15.43:0/1134230480 shutdown_connections 2024-09-11T23:23:33.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.849+0000 7f14f82ac640 1 --2- 172.21.15.43:0/1134230480 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f14dc0a2ec0 0x7f14dc0a5380 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.849+0000 7f14f82ac640 1 --2- 172.21.15.43:0/1134230480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14f0109120 0x7f14f019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.849+0000 7f14f82ac640 1 --2- 172.21.15.43:0/1134230480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14f01049a0 0x7f14f019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.849+0000 7f14f82ac640 1 --2- 172.21.15.43:0/1134230480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14f0103ff0 0x7f14f019a590 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.850+0000 7f14f82ac640 1 -- 172.21.15.43:0/1134230480 >> 172.21.15.43:0/1134230480 conn(0x7f14f00ffab0 msgr2=0x7f14f0100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:33.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.850+0000 7f14f82ac640 1 -- 172.21.15.43:0/1134230480 shutdown_connections 2024-09-11T23:23:33.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:33.850+0000 7f14f82ac640 1 -- 172.21.15.43:0/1134230480 wait complete. 2024-09-11T23:23:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.948+0000 7f650a59c640 1 --2- 172.21.15.43:0/1702309589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56196b7e8cb0 0x56196b7eb0a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009b60 tx=0x7f64fc00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49bf437530cfb110 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.949+0000 7f65017fa640 1 -- 172.21.15.43:0/1702309589 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc009e50 con 0x56196b7e8cb0 2024-09-11T23:23:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.949+0000 7f65017fa640 1 -- 172.21.15.43:0/1702309589 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64fc005500 con 0x56196b7e8cb0 2024-09-11T23:23:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.949+0000 7f65017fa640 1 -- 172.21.15.43:0/1702309589 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc0116a0 con 0x56196b7e8cb0 2024-09-11T23:23:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.949+0000 7f650b59e640 1 --2- 172.21.15.43:0/371559841 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f64f80a2ec0 0x7f64f80a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.951+0000 7f6513fdf740 1 -- 172.21.15.43:0/1702309589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56196b7e8cb0 msgr2=0x56196b7eb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.951+0000 7f6513fdf740 1 --2- 172.21.15.43:0/1702309589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56196b7e8cb0 0x56196b7eb0a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009b60 tx=0x7f64fc00f730 comp rx=0 tx=0).stop 2024-09-11T23:23:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.952+0000 7f650b59e640 1 --2- 172.21.15.43:0/371559841 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f64f80a2ec0 0x7f64f80a5380 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6504009b30 tx=0x7f6504009500 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.953+0000 7f6513fdf740 1 -- 172.21.15.43:0/1702309589 shutdown_connections 2024-09-11T23:23:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.953+0000 7f6513fdf740 1 --2- 172.21.15.43:0/1702309589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56196b7e8cb0 0x56196b7eb0a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.953+0000 7f6513fdf740 1 --2- 172.21.15.43:0/1702309589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7de410 0x56196b7e8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.953+0000 7f6513fdf740 1 --2- 172.21.15.43:0/1702309589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56196b7ddaf0 0x56196b7dded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.953+0000 7f6513fdf740 1 -- 172.21.15.43:0/1702309589 >> 172.21.15.43:0/1702309589 conn(0x56196b7da060 msgr2=0x56196b7da440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.953+0000 7f6513fdf740 1 -- 172.21.15.43:0/1702309589 shutdown_connections 2024-09-11T23:23:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.953+0000 7f6513fdf740 1 -- 172.21.15.43:0/1702309589 wait complete. 2024-09-11T23:23:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.954+0000 7f6513fdf740 1 -- start start 2024-09-11T23:23:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.954+0000 7f6513fdf740 1 Processor -- start 2024-09-11T23:23:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.955+0000 7f6513fdf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56196b7ddaf0 0x56196b7e5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.955+0000 7f6513fdf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56196b7de410 0x56196b7e6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.955+0000 7f6513fdf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7e8cb0 0x56196b7e6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.956+0000 7f6513fdf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56196b7eb880 con 0x56196b7e8cb0 2024-09-11T23:23:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.956+0000 7f6513fdf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56196b6cb290 con 0x56196b7de410 2024-09-11T23:23:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.956+0000 7f6513fdf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56196b7eba00 con 0x56196b7ddaf0 2024-09-11T23:23:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.956+0000 7f6513fdf740 10 client.0 _ll_register_callbacks cb 0x56196b6fc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.956+0000 7f650b59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7e8cb0 0x56196b7e6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.956+0000 7f650b59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7e8cb0 0x56196b7e6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53940/0 (socket says 172.21.15.43:53940) 2024-09-11T23:23:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.956+0000 7f650b59e640 1 -- 172.21.15.43:0/2062156938 learned_addr learned my addr 172.21.15.43:0/2062156938 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.957+0000 7f650a59c640 1 --2- 172.21.15.43:0/2062156938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56196b7de410 0x56196b7e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.957+0000 7f650b59e640 1 -- 172.21.15.43:0/2062156938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56196b7ddaf0 msgr2=0x56196b7e5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.957+0000 7f650b59e640 1 --2- 172.21.15.43:0/2062156938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56196b7ddaf0 0x56196b7e5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.957+0000 7f650b59e640 1 -- 172.21.15.43:0/2062156938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56196b7de410 msgr2=0x56196b7e6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.957+0000 7f650b59e640 1 --2- 172.21.15.43:0/2062156938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56196b7de410 0x56196b7e6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.957+0000 7f650b59e640 1 -- 172.21.15.43:0/2062156938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56196b455210 con 0x56196b7e8cb0 2024-09-11T23:23:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.957+0000 7f650b59e640 1 --2- 172.21.15.43:0/2062156938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7e8cb0 0x56196b7e6640 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6504009c10 tx=0x7f6504005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9bb592c6f08ccb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.957+0000 7f64f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:23:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.957+0000 7f64f27fc640 1 -- 172.21.15.43:0/2062156938 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6504043070 con 0x56196b7e8cb0 2024-09-11T23:23:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.957+0000 7f64f27fc640 1 -- 172.21.15.43:0/2062156938 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f6504003b70 con 0x56196b7e8cb0 2024-09-11T23:23:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.958+0000 7f6513fdf740 1 -- 172.21.15.43:0/2062156938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64fc005500 con 0x56196b7e8cb0 2024-09-11T23:23:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.958+0000 7f64f27fc640 1 -- 172.21.15.43:0/2062156938 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6504036040 con 0x56196b7e8cb0 2024-09-11T23:23:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.958+0000 7f6513fdf740 10 client.15804 fetch_fsmap learned FSMap version 26 2024-09-11T23:23:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.958+0000 7f6513fdf740 1 -- 172.21.15.43:0/2062156938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56196b521620 con 0x56196b7e8cb0 2024-09-11T23:23:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.959+0000 7f64f27fc640 1 -- 172.21.15.43:0/2062156938 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64fc005500 con 0x56196b7e8cb0 2024-09-11T23:23:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.959+0000 7f64f27fc640 1 -- 172.21.15.43:0/2062156938 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6504052030 con 0x56196b7e8cb0 2024-09-11T23:23:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.959+0000 7f6513fdf740 10 client.15804 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:23:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.959+0000 7f6513fdf740 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.959+0000 7f6513fdf740 20 client.15804 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.959+0000 7f6513fdf740 1 --2- 172.21.15.43:0/2062156938 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56196b7f8c90 0x56196b7fb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.959+0000 7f6513fdf740 4 client.15804 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:23:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.959+0000 7f6513fdf740 1 -- 172.21.15.43:0/2062156938 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56196b74fa20 con 0x56196b7f8c90 2024-09-11T23:23:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.961+0000 7f650ad9d640 1 --2- 172.21.15.43:0/2062156938 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56196b7f8c90 0x56196b7fb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.969+0000 7f650ad9d640 1 --2- 172.21.15.43:0/2062156938 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56196b7f8c90 0x56196b7fb120 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64a03d4b15b2107 server_cookie=5e566b567de9b1de in_seq=0 out_seq=0 2024-09-11T23:23:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.969+0000 7f64f27fc640 10 client.15804 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.972+0000 7f64f27fc640 1 -- 172.21.15.43:0/2062156938 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56196b74fa20 con 0x56196b7f8c90 2024-09-11T23:23:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.972+0000 7f64f27fc640 10 client.15804 handle_command_reply: tid=0 2024-09-11T23:23:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.973+0000 7f6513fdf740 1 client.15804 shutdown 2024-09-11T23:23:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.975+0000 7f6513fdf740 1 -- 172.21.15.43:0/2062156938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7e8cb0 msgr2=0x56196b7e6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.975+0000 7f6513fdf740 1 --2- 172.21.15.43:0/2062156938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7e8cb0 0x56196b7e6640 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6504009c10 tx=0x7f6504005fd0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.975+0000 7f6513fdf740 1 -- 172.21.15.43:0/2062156938 shutdown_connections 2024-09-11T23:23:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.975+0000 7f6513fdf740 1 --2- 172.21.15.43:0/2062156938 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56196b7f8c90 0x56196b7fb120 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.975+0000 7f64f27fc640 0 client.15804 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.975+0000 7f6513fdf740 1 --2- 172.21.15.43:0/2062156938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7e8cb0 0x56196b7e6640 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.975+0000 7f6513fdf740 1 --2- 172.21.15.43:0/2062156938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56196b7de410 0x56196b7e6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.975+0000 7f6513fdf740 1 --2- 172.21.15.43:0/2062156938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56196b7ddaf0 0x56196b7e5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.975+0000 7f6513fdf740 1 -- 172.21.15.43:0/2062156938 >> 172.21.15.43:0/2062156938 conn(0x56196b7da060 msgr2=0x56196b7dfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.976+0000 7f6513fdf740 1 -- 172.21.15.43:0/2062156938 shutdown_connections 2024-09-11T23:23:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.979+0000 7f6513fdf740 1 -- 172.21.15.43:0/2062156938 wait complete. 2024-09-11T23:23:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:33.979+0000 7f6513fdf740 20 client.15804 trim_cache size 0 max 16384 2024-09-11T23:23:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.016+0000 7f6513fdf740 1 Processor -- start 2024-09-11T23:23:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.017+0000 7f6513fdf740 1 -- start start 2024-09-11T23:23:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.018+0000 7f6513fdf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7ddaf0 0x56196b7e6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.018+0000 7f6513fdf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56196b7de410 0x56196b800a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.018+0000 7f6513fdf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56196b7e8cb0 0x56196b800f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.018+0000 7f6513fdf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f650c1babd0 con 0x56196b7ddaf0 2024-09-11T23:23:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.018+0000 7f6513fdf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f650c1b84a0 con 0x56196b7de410 2024-09-11T23:23:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.018+0000 7f6513fdf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f650c1b62f0 con 0x56196b7e8cb0 2024-09-11T23:23:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.018+0000 7f650ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7ddaf0 0x56196b7e6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.018+0000 7f650ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7ddaf0 0x56196b7e6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53944/0 (socket says 172.21.15.43:53944) 2024-09-11T23:23:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.018+0000 7f650ad9d640 1 -- 172.21.15.43:0/4242866441 learned_addr learned my addr 172.21.15.43:0/4242866441 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.018+0000 7f650ad9d640 1 -- 172.21.15.43:0/4242866441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56196b7e8cb0 msgr2=0x56196b800f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:23:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.018+0000 7f650ad9d640 1 --2- 172.21.15.43:0/4242866441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56196b7e8cb0 0x56196b800f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.018+0000 7f650ad9d640 1 -- 172.21.15.43:0/4242866441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56196b7de410 msgr2=0x56196b800a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:23:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.018+0000 7f650ad9d640 1 --2- 172.21.15.43:0/4242866441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56196b7de410 0x56196b800a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.018+0000 7f650ad9d640 1 -- 172.21.15.43:0/4242866441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56196b7d9e90 con 0x56196b7ddaf0 2024-09-11T23:23:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.019+0000 7f650ad9d640 1 --2- 172.21.15.43:0/4242866441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7ddaf0 0x56196b7e6470 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f64f400c6a0 tx=0x7f64f4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a759a56d42f9ad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.019+0000 7f6500ff9640 1 -- 172.21.15.43:0/4242866441 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f40a0250 con 0x56196b7ddaf0 2024-09-11T23:23:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.019+0000 7f6500ff9640 1 -- 172.21.15.43:0/4242866441 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64f40a4b10 con 0x56196b7ddaf0 2024-09-11T23:23:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.019+0000 7f6500ff9640 1 -- 172.21.15.43:0/4242866441 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f409b2f0 con 0x56196b7ddaf0 2024-09-11T23:23:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.020+0000 7f6513fdf740 1 -- 172.21.15.43:0/4242866441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7ddaf0 msgr2=0x56196b7e6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.020+0000 7f6513fdf740 1 --2- 172.21.15.43:0/4242866441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7ddaf0 0x56196b7e6470 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f64f400c6a0 tx=0x7f64f4020000 comp rx=0 tx=0).stop 2024-09-11T23:23:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.020+0000 7f6513fdf740 1 -- 172.21.15.43:0/4242866441 shutdown_connections 2024-09-11T23:23:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.020+0000 7f6513fdf740 1 --2- 172.21.15.43:0/4242866441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56196b7e8cb0 0x56196b800f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.020+0000 7f6513fdf740 1 --2- 172.21.15.43:0/4242866441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56196b7de410 0x56196b800a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.020+0000 7f6513fdf740 1 --2- 172.21.15.43:0/4242866441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7ddaf0 0x56196b7e6470 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.020+0000 7f6513fdf740 1 -- 172.21.15.43:0/4242866441 >> 172.21.15.43:0/4242866441 conn(0x56196b7da060 msgr2=0x56196b7e7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.021+0000 7f6513fdf740 1 -- 172.21.15.43:0/4242866441 shutdown_connections 2024-09-11T23:23:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.021+0000 7f6513fdf740 1 -- 172.21.15.43:0/4242866441 wait complete. 2024-09-11T23:23:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.021+0000 7f6513fdf740 1 -- start start 2024-09-11T23:23:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.021+0000 7f6513fdf740 1 Processor -- start 2024-09-11T23:23:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.022+0000 7f6513fdf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56196b7ddaf0 0x56196b7f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.022+0000 7f6513fdf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56196b7de410 0x56196b7ec8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.022+0000 7f6513fdf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7e8cb0 0x56196b7ed060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.022+0000 7f6513fdf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f650c1b6960 con 0x56196b7e8cb0 2024-09-11T23:23:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.022+0000 7f6513fdf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f650c1b7dd0 con 0x56196b7ddaf0 2024-09-11T23:23:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.022+0000 7f6513fdf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f650c1b7030 con 0x56196b7de410 2024-09-11T23:23:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.022+0000 7f6513fdf740 10 client.0 _ll_register_callbacks cb 0x7f6504034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.022+0000 7f650b59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7e8cb0 0x56196b7ed060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.022+0000 7f650b59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7e8cb0 0x56196b7ed060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53948/0 (socket says 172.21.15.43:53948) 2024-09-11T23:23:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.022+0000 7f650b59e640 1 -- 172.21.15.43:0/2897607257 learned_addr learned my addr 172.21.15.43:0/2897607257 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.023+0000 7f650b59e640 1 -- 172.21.15.43:0/2897607257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56196b7de410 msgr2=0x56196b7ec8d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:23:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.023+0000 7f650b59e640 1 --2- 172.21.15.43:0/2897607257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56196b7de410 0x56196b7ec8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.023+0000 7f650b59e640 1 -- 172.21.15.43:0/2897607257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56196b7ddaf0 msgr2=0x56196b7f15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.023+0000 7f650b59e640 1 --2- 172.21.15.43:0/2897607257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56196b7ddaf0 0x56196b7f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.023+0000 7f650b59e640 1 -- 172.21.15.43:0/2897607257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56196b862a00 con 0x56196b7e8cb0 2024-09-11T23:23:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.023+0000 7f650b59e640 1 --2- 172.21.15.43:0/2897607257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7e8cb0 0x56196b7ed060 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f650402ac10 tx=0x7f6504003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99c6e5413af9b73 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.023+0000 7f64f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:23:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.023+0000 7f64f27fc640 1 -- 172.21.15.43:0/2897607257 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6504043040 con 0x56196b7e8cb0 2024-09-11T23:23:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.023+0000 7f6513fdf740 1 -- 172.21.15.43:0/2897607257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64f40a4b10 con 0x56196b7e8cb0 2024-09-11T23:23:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.023+0000 7f64f27fc640 1 -- 172.21.15.43:0/2897607257 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f6504004240 con 0x56196b7e8cb0 2024-09-11T23:23:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.024+0000 7f64f27fc640 1 -- 172.21.15.43:0/2897607257 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6504004440 con 0x56196b7e8cb0 2024-09-11T23:23:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.024+0000 7f6513fdf740 10 client.15810 fetch_fsmap learned FSMap version 26 2024-09-11T23:23:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.024+0000 7f6513fdf740 1 -- 172.21.15.43:0/2897607257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56196b863bf0 con 0x56196b7e8cb0 2024-09-11T23:23:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.024+0000 7f64f27fc640 1 -- 172.21.15.43:0/2897607257 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64f40a4b10 con 0x56196b7e8cb0 2024-09-11T23:23:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.024+0000 7f64f27fc640 1 -- 172.21.15.43:0/2897607257 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6504052070 con 0x56196b7e8cb0 2024-09-11T23:23:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.024+0000 7f6513fdf740 10 client.15810 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:23:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.024+0000 7f6513fdf740 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.024+0000 7f6513fdf740 20 client.15810 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.024+0000 7f6513fdf740 1 --2- 172.21.15.43:0/2897607257 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56196b7f8c90 0x56196b866260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.024+0000 7f6513fdf740 4 client.15810 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:23:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.024+0000 7f6513fdf740 1 -- 172.21.15.43:0/2897607257 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56196b749d70 con 0x56196b7f8c90 2024-09-11T23:23:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.028+0000 7f650ad9d640 1 --2- 172.21.15.43:0/2897607257 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56196b7f8c90 0x56196b866260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.037+0000 7f64f27fc640 10 client.15810 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.037+0000 7f650ad9d640 1 --2- 172.21.15.43:0/2897607257 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56196b7f8c90 0x56196b866260 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=672624dfd5e6b927 server_cookie=aed1d3c768090eae in_seq=0 out_seq=0 2024-09-11T23:23:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.109+0000 7f64f27fc640 1 -- 172.21.15.43:0/2897607257 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56196b749d70 con 0x56196b7f8c90 2024-09-11T23:23:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.109+0000 7f64f27fc640 10 client.15810 handle_command_reply: tid=0 2024-09-11T23:23:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.109+0000 7f6513fdf740 1 client.15810 shutdown 2024-09-11T23:23:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.110+0000 7f6513fdf740 1 -- 172.21.15.43:0/2897607257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7e8cb0 msgr2=0x56196b7ed060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.110+0000 7f6513fdf740 1 --2- 172.21.15.43:0/2897607257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7e8cb0 0x56196b7ed060 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f650402ac10 tx=0x7f6504003ba0 comp rx=0 tx=0).stop 2024-09-11T23:23:34.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:33 smithi043 ceph-mon[30950]: pgmap v54: 97 pgs: 97 active+clean; 2.7 GiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 227 MiB/s wr, 294 op/s 2024-09-11T23:23:34.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:33 smithi043 ceph-mon[30950]: scrub summary: active paths [/] 2024-09-11T23:23:34.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:33 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:23:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.111+0000 7f6513fdf740 1 -- 172.21.15.43:0/2897607257 shutdown_connections 2024-09-11T23:23:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.111+0000 7f6513fdf740 1 --2- 172.21.15.43:0/2897607257 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56196b7f8c90 0x56196b866260 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.111+0000 7f6513fdf740 1 --2- 172.21.15.43:0/2897607257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56196b7e8cb0 0x56196b7ed060 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.111+0000 7f6513fdf740 1 --2- 172.21.15.43:0/2897607257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56196b7de410 0x56196b7ec8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.111+0000 7f6513fdf740 1 --2- 172.21.15.43:0/2897607257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56196b7ddaf0 0x56196b7f15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.111+0000 7f6513fdf740 1 -- 172.21.15.43:0/2897607257 >> 172.21.15.43:0/2897607257 conn(0x56196b7da060 msgr2=0x56196b7d89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.111+0000 7f64f27fc640 0 client.15810 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.111+0000 7f6513fdf740 1 -- 172.21.15.43:0/2897607257 shutdown_connections 2024-09-11T23:23:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.111+0000 7f6513fdf740 1 -- 172.21.15.43:0/2897607257 wait complete. 2024-09-11T23:23:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.111+0000 7f6513fdf740 20 client.15810 trim_cache size 0 max 16384 2024-09-11T23:23:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:23:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:23:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:23:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:23:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.120+0000 7f6511247640 1 -- 172.21.15.43:0/371559841 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f64f80a2ec0 msgr2=0x7f64f80a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.120+0000 7f6511247640 1 --2- 172.21.15.43:0/371559841 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f64f80a2ec0 0x7f64f80a5380 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6504009b30 tx=0x7f6504009500 comp rx=0 tx=0).stop 2024-09-11T23:23:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.120+0000 7f6511247640 1 -- 172.21.15.43:0/371559841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f650c1091f0 msgr2=0x7f650c1a2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.120+0000 7f6511247640 1 --2- 172.21.15.43:0/371559841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f650c1091f0 0x7f650c1a2dd0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f64f400e9f0 tx=0x7f64f400eeb0 comp rx=0 tx=0).stop 2024-09-11T23:23:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.127+0000 7f6511247640 1 -- 172.21.15.43:0/371559841 shutdown_connections 2024-09-11T23:23:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.127+0000 7f6511247640 1 --2- 172.21.15.43:0/371559841 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f64f80a2ec0 0x7f64f80a5380 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.127+0000 7f6511247640 1 --2- 172.21.15.43:0/371559841 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f650c116ea0 0x7f650c1a7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.127+0000 7f6511247640 1 --2- 172.21.15.43:0/371559841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f650c109b10 0x7f650c1a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.127+0000 7f6511247640 1 --2- 172.21.15.43:0/371559841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f650c1091f0 0x7f650c1a2dd0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.127+0000 7f6511247640 1 -- 172.21.15.43:0/371559841 >> 172.21.15.43:0/371559841 conn(0x7f650c06d200 msgr2=0x7f650c117420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.127+0000 7f6511247640 1 -- 172.21.15.43:0/371559841 shutdown_connections 2024-09-11T23:23:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:34.127+0000 7f6511247640 1 -- 172.21.15.43:0/371559841 wait complete. 2024-09-11T23:23:34.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:33 smithi090 ceph-mon[35130]: pgmap v54: 97 pgs: 97 active+clean; 2.7 GiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 227 MiB/s wr, 294 op/s 2024-09-11T23:23:34.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:33 smithi090 ceph-mon[35130]: scrub summary: active paths [/] 2024-09-11T23:23:34.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:33 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:23:34.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:33 smithi097 ceph-mon[35562]: pgmap v54: 97 pgs: 97 active+clean; 2.7 GiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 227 MiB/s wr, 294 op/s 2024-09-11T23:23:34.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:33 smithi097 ceph-mon[35562]: scrub summary: active paths [/] 2024-09-11T23:23:34.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:33 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:23:34.763 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:23:35.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:34 smithi043 ceph-mon[30950]: scrub queued for path: / 2024-09-11T23:23:35.114 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:34 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:23:35.114 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:34 smithi043 ceph-mon[30950]: scrub summary: active paths [/] 2024-09-11T23:23:35.114 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:34 smithi043 ceph-mon[30950]: scrub summary: active paths [/] 2024-09-11T23:23:35.114 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:34 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:23:35.114 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:34 smithi043 ceph-mon[30950]: scrub summary: active paths [/] 2024-09-11T23:23:35.115 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:34 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:23:35.115 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:34 smithi043 ceph-mon[30950]: scrub summary: active paths [/] 2024-09-11T23:23:35.115 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:34 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:23:35.115 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:34 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:23:35.247 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:23:35.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:34 smithi090 ceph-mon[35130]: scrub queued for path: / 2024-09-11T23:23:35.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:34 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:23:35.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:34 smithi090 ceph-mon[35130]: scrub summary: active paths [/] 2024-09-11T23:23:35.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:34 smithi090 ceph-mon[35130]: scrub summary: active paths [/] 2024-09-11T23:23:35.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:34 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:23:35.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:34 smithi090 ceph-mon[35130]: scrub summary: active paths [/] 2024-09-11T23:23:35.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:34 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:23:35.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:34 smithi090 ceph-mon[35130]: scrub summary: active paths [/] 2024-09-11T23:23:35.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:34 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:23:35.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:34 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:23:35.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:34 smithi097 ceph-mon[35562]: scrub queued for path: / 2024-09-11T23:23:35.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:34 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:23:35.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:34 smithi097 ceph-mon[35562]: scrub summary: active paths [/] 2024-09-11T23:23:35.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:34 smithi097 ceph-mon[35562]: scrub summary: active paths [/] 2024-09-11T23:23:35.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:34 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:23:35.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:34 smithi097 ceph-mon[35562]: scrub summary: active paths [/] 2024-09-11T23:23:35.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:34 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:23:35.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:34 smithi097 ceph-mon[35562]: scrub summary: active paths [/] 2024-09-11T23:23:35.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:34 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:23:35.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:34 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:23:36.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.123+0000 7fc471501640 1 -- 172.21.15.43:0/1718864704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc46c102790 msgr2=0x7fc46c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:36.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.123+0000 7fc471501640 1 --2- 172.21.15.43:0/1718864704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc46c102790 0x7fc46c1067e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc460009860 tx=0x7fc46002a340 comp rx=0 tx=0).stop 2024-09-11T23:23:36.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.123+0000 7fc471501640 1 -- 172.21.15.43:0/1718864704 shutdown_connections 2024-09-11T23:23:36.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.123+0000 7fc471501640 1 --2- 172.21.15.43:0/1718864704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc46c106f10 0x7fc46c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.123+0000 7fc471501640 1 --2- 172.21.15.43:0/1718864704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc46c102790 0x7fc46c1067e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.123+0000 7fc471501640 1 --2- 172.21.15.43:0/1718864704 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc46c101de0 0x7fc46c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.123+0000 7fc471501640 1 -- 172.21.15.43:0/1718864704 >> 172.21.15.43:0/1718864704 conn(0x7fc46c0fd8e0 msgr2=0x7fc46c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:36.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.123+0000 7fc471501640 1 -- 172.21.15.43:0/1718864704 shutdown_connections 2024-09-11T23:23:36.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.123+0000 7fc471501640 1 -- 172.21.15.43:0/1718864704 wait complete. 2024-09-11T23:23:36.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.125+0000 7fc471501640 1 Processor -- start 2024-09-11T23:23:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.125+0000 7fc471501640 1 -- start start 2024-09-11T23:23:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.126+0000 7fc471501640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc46c101de0 0x7fc46c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.126+0000 7fc471501640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc46c102790 0x7fc46c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.126+0000 7fc471501640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc46c106f10 0x7fc46c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.126+0000 7fc471501640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c111f70 con 0x7fc46c102790 2024-09-11T23:23:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.126+0000 7fc471501640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c111df0 con 0x7fc46c106f10 2024-09-11T23:23:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.126+0000 7fc471501640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c1120f0 con 0x7fc46c101de0 2024-09-11T23:23:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.126+0000 7fc46a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc46c102790 0x7fc46c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.126+0000 7fc46a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc46c102790 0x7fc46c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53970/0 (socket says 172.21.15.43:53970) 2024-09-11T23:23:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.126+0000 7fc46a7fc640 1 -- 172.21.15.43:0/4122174754 learned_addr learned my addr 172.21.15.43:0/4122174754 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.127+0000 7fc46a7fc640 1 -- 172.21.15.43:0/4122174754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc46c101de0 msgr2=0x7fc46c0744f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:23:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.127+0000 7fc46a7fc640 1 --2- 172.21.15.43:0/4122174754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc46c101de0 0x7fc46c0744f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.127+0000 7fc46a7fc640 1 -- 172.21.15.43:0/4122174754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc46c106f10 msgr2=0x7fc46c074f70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:23:36.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.127+0000 7fc46a7fc640 1 --2- 172.21.15.43:0/4122174754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc46c106f10 0x7fc46c074f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.127+0000 7fc46a7fc640 1 -- 172.21.15.43:0/4122174754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46c1a68f0 con 0x7fc46c102790 2024-09-11T23:23:36.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.127+0000 7fc46a7fc640 1 --2- 172.21.15.43:0/4122174754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc46c102790 0x7fc46c074a30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc460035040 tx=0x7fc460033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:36.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.127+0000 7fc453fff640 1 -- 172.21.15.43:0/4122174754 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460037040 con 0x7fc46c102790 2024-09-11T23:23:36.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.127+0000 7fc471501640 1 -- 172.21.15.43:0/4122174754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46c1a6b80 con 0x7fc46c102790 2024-09-11T23:23:36.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.128+0000 7fc453fff640 1 -- 172.21.15.43:0/4122174754 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc460040070 con 0x7fc46c102790 2024-09-11T23:23:36.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.128+0000 7fc453fff640 1 -- 172.21.15.43:0/4122174754 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460003a50 con 0x7fc46c102790 2024-09-11T23:23:36.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.128+0000 7fc471501640 1 -- 172.21.15.43:0/4122174754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc46c1a70b0 con 0x7fc46c102790 2024-09-11T23:23:36.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.132+0000 7fc4738da740 1 Processor -- start 2024-09-11T23:23:36.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.132+0000 7fc4738da740 1 -- start start 2024-09-11T23:23:36.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.132+0000 7fc453fff640 1 -- 172.21.15.43:0/4122174754 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc460003bf0 con 0x7fc46c102790 2024-09-11T23:23:36.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.133+0000 7fc4738da740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8ea4a0 0x55761b8ea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:36.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.133+0000 7fc4738da740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55761b8eadc0 0x55761b8f5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:36.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.133+0000 7fc4738da740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55761b8f5680 0x55761b8f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:36.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.133+0000 7fc4738da740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55761b8502e0 con 0x55761b8ea4a0 2024-09-11T23:23:36.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.133+0000 7fc4738da740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55761b7f7ce0 con 0x55761b8eadc0 2024-09-11T23:23:36.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.133+0000 7fc4738da740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55761b84c2a0 con 0x55761b8f5680 2024-09-11T23:23:36.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.133+0000 7fc46b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8ea4a0 0x55761b8ea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:36.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.134+0000 7fc46b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8ea4a0 0x55761b8ea880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53982/0 (socket says 172.21.15.43:53982) 2024-09-11T23:23:36.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.134+0000 7fc46b7fe640 1 -- 172.21.15.43:0/2656028788 learned_addr learned my addr 172.21.15.43:0/2656028788 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:36.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.134+0000 7fc46affd640 1 --2- 172.21.15.43:0/2656028788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55761b8eadc0 0x55761b8f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:36.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.135+0000 7fc46b7fe640 1 -- 172.21.15.43:0/2656028788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55761b8f5680 msgr2=0x55761b8f7a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:23:36.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.135+0000 7fc46b7fe640 1 --2- 172.21.15.43:0/2656028788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55761b8f5680 0x55761b8f7a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.135+0000 7fc46b7fe640 1 -- 172.21.15.43:0/2656028788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55761b8eadc0 msgr2=0x55761b8f5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:36.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.135+0000 7fc46b7fe640 1 --2- 172.21.15.43:0/2656028788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55761b8eadc0 0x55761b8f5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.135+0000 7fc46b7fe640 1 -- 172.21.15.43:0/2656028788 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55761b7dc440 con 0x55761b8ea4a0 2024-09-11T23:23:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.136+0000 7fc453fff640 1 --2- 172.21.15.43:0/4122174754 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc4540a2cf0 0x7fc4540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.137+0000 7fc46affd640 1 --2- 172.21.15.43:0/4122174754 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc4540a2cf0 0x7fc4540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:36.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.137+0000 7fc453fff640 1 -- 172.21.15.43:0/4122174754 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fc4600ef490 con 0x7fc46c102790 2024-09-11T23:23:36.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.138+0000 7fc46b7fe640 1 --2- 172.21.15.43:0/2656028788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8ea4a0 0x55761b8ea880 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc464009a50 tx=0x7fc464009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76efa7d90460e3f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:36.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.138+0000 7fc4517fa640 1 -- 172.21.15.43:0/2656028788 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46400ec10 con 0x55761b8ea4a0 2024-09-11T23:23:36.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.139+0000 7fc46affd640 1 --2- 172.21.15.43:0/4122174754 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc4540a2cf0 0x7fc4540a51b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55761b8e7890 tx=0x7fc45c0094a0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:36.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.139+0000 7fc4517fa640 1 -- 172.21.15.43:0/2656028788 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc46400edb0 con 0x55761b8ea4a0 2024-09-11T23:23:36.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.139+0000 7fc4517fa640 1 -- 172.21.15.43:0/2656028788 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc464010430 con 0x55761b8ea4a0 2024-09-11T23:23:36.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.140+0000 7fc4738da740 1 -- 172.21.15.43:0/2656028788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8ea4a0 msgr2=0x55761b8ea880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:36.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.140+0000 7fc4738da740 1 --2- 172.21.15.43:0/2656028788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8ea4a0 0x55761b8ea880 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc464009a50 tx=0x7fc464009f10 comp rx=0 tx=0).stop 2024-09-11T23:23:36.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.140+0000 7fc4738da740 1 -- 172.21.15.43:0/2656028788 shutdown_connections 2024-09-11T23:23:36.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.140+0000 7fc4738da740 1 --2- 172.21.15.43:0/2656028788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55761b8f5680 0x55761b8f7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.140+0000 7fc4738da740 1 --2- 172.21.15.43:0/2656028788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55761b8eadc0 0x55761b8f5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.140+0000 7fc4738da740 1 --2- 172.21.15.43:0/2656028788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8ea4a0 0x55761b8ea880 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.140+0000 7fc4738da740 1 -- 172.21.15.43:0/2656028788 >> 172.21.15.43:0/2656028788 conn(0x55761b8e6a30 msgr2=0x55761b8e6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:36.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.141+0000 7fc4738da740 1 -- 172.21.15.43:0/2656028788 shutdown_connections 2024-09-11T23:23:36.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.141+0000 7fc4738da740 1 -- 172.21.15.43:0/2656028788 wait complete. 2024-09-11T23:23:36.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.142+0000 7fc4738da740 1 -- start start 2024-09-11T23:23:36.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.142+0000 7fc4738da740 1 Processor -- start 2024-09-11T23:23:36.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.143+0000 7fc4738da740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55761b8ea4a0 0x55761b8f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:36.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.143+0000 7fc4738da740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8eadc0 0x55761b8f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:36.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.143+0000 7fc4738da740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55761b8f5680 0x55761b8f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:36.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.143+0000 7fc4738da740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55761b8f8250 con 0x55761b8eadc0 2024-09-11T23:23:36.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.143+0000 7fc4738da740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55761b80b050 con 0x55761b8ea4a0 2024-09-11T23:23:36.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.143+0000 7fc4738da740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55761b8f83d0 con 0x55761b8f5680 2024-09-11T23:23:36.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.143+0000 7fc4738da740 10 client.0 _ll_register_callbacks cb 0x55761b84b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:36.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.143+0000 7fc46a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8eadc0 0x55761b8f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:36.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.143+0000 7fc46a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8eadc0 0x55761b8f21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53984/0 (socket says 172.21.15.43:53984) 2024-09-11T23:23:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.143+0000 7fc46a7fc640 1 -- 172.21.15.43:0/3301861545 learned_addr learned my addr 172.21.15.43:0/3301861545 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.143+0000 7fc46b7fe640 1 --2- 172.21.15.43:0/3301861545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55761b8f5680 0x55761b8f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.144+0000 7fc46a7fc640 1 -- 172.21.15.43:0/3301861545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55761b8f5680 msgr2=0x55761b8f26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.144+0000 7fc46a7fc640 1 --2- 172.21.15.43:0/3301861545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55761b8f5680 0x55761b8f26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.144+0000 7fc46a7fc640 1 -- 172.21.15.43:0/3301861545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55761b8ea4a0 msgr2=0x55761b8f4ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:23:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.144+0000 7fc46a7fc640 1 --2- 172.21.15.43:0/3301861545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55761b8ea4a0 0x55761b8f4ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.145+0000 7fc46a7fc640 1 -- 172.21.15.43:0/3301861545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55761b5c13c0 con 0x55761b8eadc0 2024-09-11T23:23:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.145+0000 7fc46a7fc640 1 --2- 172.21.15.43:0/3301861545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8eadc0 0x55761b8f21a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc4600ecda0 tx=0x7fc4600bb890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345746d4ca7a318f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.145+0000 7fc44eff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:23:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.145+0000 7fc44eff5640 1 -- 172.21.15.43:0/3301861545 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4600bc1c0 con 0x55761b8eadc0 2024-09-11T23:23:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.146+0000 7fc4738da740 1 -- 172.21.15.43:0/3301861545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc46400edb0 con 0x55761b8eadc0 2024-09-11T23:23:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.146+0000 7fc44eff5640 1 -- 172.21.15.43:0/3301861545 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fc4600c42a0 con 0x55761b8eadc0 2024-09-11T23:23:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.146+0000 7fc44eff5640 1 -- 172.21.15.43:0/3301861545 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc4600bbac0 con 0x55761b8eadc0 2024-09-11T23:23:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.146+0000 7fc4738da740 10 client.15822 fetch_fsmap learned FSMap version 26 2024-09-11T23:23:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.146+0000 7fc4738da740 1 -- 172.21.15.43:0/3301861545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55761b903ba0 con 0x55761b8eadc0 2024-09-11T23:23:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.146+0000 7fc44eff5640 1 -- 172.21.15.43:0/3301861545 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc46400edb0 con 0x55761b8eadc0 2024-09-11T23:23:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.146+0000 7fc44eff5640 1 -- 172.21.15.43:0/3301861545 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc46003a070 con 0x55761b8eadc0 2024-09-11T23:23:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.147+0000 7fc4738da740 10 client.15822 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:23:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.147+0000 7fc4738da740 10 client.15822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.147+0000 7fc4738da740 20 client.15822 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:36.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.147+0000 7fc4738da740 1 --2- 172.21.15.43:0/3301861545 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55761b905000 0x55761b907420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:36.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.147+0000 7fc4738da740 4 client.15822 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:23:36.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.147+0000 7fc4738da740 1 -- 172.21.15.43:0/3301861545 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55761b85d850 con 0x55761b905000 2024-09-11T23:23:36.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.148+0000 7fc46affd640 1 --2- 172.21.15.43:0/3301861545 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55761b905000 0x55761b907420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:36.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.154+0000 7fc46affd640 1 --2- 172.21.15.43:0/3301861545 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55761b905000 0x55761b907420 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b86a02d58e1adc91 server_cookie=fa0ae548da7e74b8 in_seq=0 out_seq=0 2024-09-11T23:23:36.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.154+0000 7fc44eff5640 10 client.15822 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:36.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.156+0000 7fc44eff5640 1 -- 172.21.15.43:0/3301861545 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55761b85d850 con 0x55761b905000 2024-09-11T23:23:36.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.156+0000 7fc44eff5640 10 client.15822 handle_command_reply: tid=0 2024-09-11T23:23:36.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.156+0000 7fc4738da740 1 client.15822 shutdown 2024-09-11T23:23:36.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.156+0000 7fc4738da740 1 -- 172.21.15.43:0/3301861545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8eadc0 msgr2=0x55761b8f21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:36.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.156+0000 7fc4738da740 1 --2- 172.21.15.43:0/3301861545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8eadc0 0x55761b8f21a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc4600ecda0 tx=0x7fc4600bb890 comp rx=0 tx=0).stop 2024-09-11T23:23:36.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.157+0000 7fc4738da740 1 -- 172.21.15.43:0/3301861545 shutdown_connections 2024-09-11T23:23:36.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.157+0000 7fc4738da740 1 --2- 172.21.15.43:0/3301861545 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55761b905000 0x55761b907420 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.157+0000 7fc4738da740 1 --2- 172.21.15.43:0/3301861545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55761b8f5680 0x55761b8f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.157+0000 7fc4738da740 1 --2- 172.21.15.43:0/3301861545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8eadc0 0x55761b8f21a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.157+0000 7fc4738da740 1 --2- 172.21.15.43:0/3301861545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55761b8ea4a0 0x55761b8f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.157+0000 7fc44eff5640 0 client.15822 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:36.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.157+0000 7fc4738da740 1 -- 172.21.15.43:0/3301861545 >> 172.21.15.43:0/3301861545 conn(0x55761b8e6a30 msgr2=0x55761b8ec760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:36.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.157+0000 7fc4738da740 1 -- 172.21.15.43:0/3301861545 shutdown_connections 2024-09-11T23:23:36.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.157+0000 7fc4738da740 1 -- 172.21.15.43:0/3301861545 wait complete. 2024-09-11T23:23:36.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.157+0000 7fc4738da740 20 client.15822 trim_cache size 0 max 16384 2024-09-11T23:23:36.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.196+0000 7fc4738da740 1 Processor -- start 2024-09-11T23:23:36.197 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:35 smithi043 ceph-mon[30950]: pgmap v55: 97 pgs: 97 active+clean; 2.7 GiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 227 MiB/s wr, 185 op/s 2024-09-11T23:23:36.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.196+0000 7fc4738da740 1 -- start start 2024-09-11T23:23:36.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.197+0000 7fc4738da740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55761b8e6a30 0x55761b90cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:36.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.197+0000 7fc4738da740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8ea4a0 0x55761b90d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:36.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.197+0000 7fc4738da740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55761b8eadc0 0x55761b8ef400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:36.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.197+0000 7fc4738da740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c1b2210 con 0x55761b8ea4a0 2024-09-11T23:23:36.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.197+0000 7fc4738da740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c1afae0 con 0x55761b8eadc0 2024-09-11T23:23:36.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.197+0000 7fc4738da740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c1ad930 con 0x55761b8e6a30 2024-09-11T23:23:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.198+0000 7fc46a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8ea4a0 0x55761b90d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.198+0000 7fc46a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8ea4a0 0x55761b90d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53994/0 (socket says 172.21.15.43:53994) 2024-09-11T23:23:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.198+0000 7fc46a7fc640 1 -- 172.21.15.43:0/470374881 learned_addr learned my addr 172.21.15.43:0/470374881 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.199+0000 7fc46a7fc640 1 -- 172.21.15.43:0/470374881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55761b8e6a30 msgr2=0x55761b90cd10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:23:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.199+0000 7fc46a7fc640 1 --2- 172.21.15.43:0/470374881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55761b8e6a30 0x55761b90cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.199+0000 7fc46a7fc640 1 -- 172.21.15.43:0/470374881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55761b8eadc0 msgr2=0x55761b8ef400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:23:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.199+0000 7fc46a7fc640 1 --2- 172.21.15.43:0/470374881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55761b8eadc0 0x55761b8ef400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.199+0000 7fc46a7fc640 1 -- 172.21.15.43:0/470374881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55761b8eabf0 con 0x55761b8ea4a0 2024-09-11T23:23:36.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.201+0000 7fc46a7fc640 1 --2- 172.21.15.43:0/470374881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8ea4a0 0x55761b90d250 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc4600c4200 tx=0x7fc460005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6c7acb5a325231 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:36.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.201+0000 7fc450ff9640 1 -- 172.21.15.43:0/470374881 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4600bb600 con 0x55761b8ea4a0 2024-09-11T23:23:36.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.201+0000 7fc450ff9640 1 -- 172.21.15.43:0/470374881 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4600bc1c0 con 0x55761b8ea4a0 2024-09-11T23:23:36.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.201+0000 7fc450ff9640 1 -- 172.21.15.43:0/470374881 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4600c42a0 con 0x55761b8ea4a0 2024-09-11T23:23:36.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.202+0000 7fc4738da740 1 -- 172.21.15.43:0/470374881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8ea4a0 msgr2=0x55761b90d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:36.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.202+0000 7fc4738da740 1 --2- 172.21.15.43:0/470374881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8ea4a0 0x55761b90d250 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc4600c4200 tx=0x7fc460005f90 comp rx=0 tx=0).stop 2024-09-11T23:23:36.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.203+0000 7fc4738da740 1 -- 172.21.15.43:0/470374881 shutdown_connections 2024-09-11T23:23:36.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.203+0000 7fc4738da740 1 --2- 172.21.15.43:0/470374881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55761b8eadc0 0x55761b8ef400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.203+0000 7fc4738da740 1 --2- 172.21.15.43:0/470374881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8ea4a0 0x55761b90d250 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.203+0000 7fc4738da740 1 --2- 172.21.15.43:0/470374881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55761b8e6a30 0x55761b90cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.203+0000 7fc4738da740 1 -- 172.21.15.43:0/470374881 >> 172.21.15.43:0/470374881 conn(0x55761b8f2840 msgr2=0x55761b8f0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:36.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.203+0000 7fc4738da740 1 -- 172.21.15.43:0/470374881 shutdown_connections 2024-09-11T23:23:36.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.203+0000 7fc4738da740 1 -- 172.21.15.43:0/470374881 wait complete. 2024-09-11T23:23:36.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.203+0000 7fc4738da740 1 -- start start 2024-09-11T23:23:36.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.204+0000 7fc4738da740 1 Processor -- start 2024-09-11T23:23:36.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.204+0000 7fc4738da740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8e6a30 0x55761b8f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:36.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.204+0000 7fc4738da740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55761b8eadc0 0x55761b8f99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:36.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.205+0000 7fc4738da740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55761b8f5680 0x55761b8f9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:36.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.205+0000 7fc4738da740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c1adfa0 con 0x55761b8e6a30 2024-09-11T23:23:36.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.205+0000 7fc4738da740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c1af410 con 0x55761b8f5680 2024-09-11T23:23:36.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.205+0000 7fc4738da740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c1ae670 con 0x55761b8eadc0 2024-09-11T23:23:36.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.205+0000 7fc4738da740 10 client.0 _ll_register_callbacks cb 0x7fc460009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:36.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.205+0000 7fc46affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8e6a30 0x55761b8f94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:36.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.205+0000 7fc46affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8e6a30 0x55761b8f94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54008/0 (socket says 172.21.15.43:54008) 2024-09-11T23:23:36.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.205+0000 7fc46affd640 1 -- 172.21.15.43:0/2097712452 learned_addr learned my addr 172.21.15.43:0/2097712452 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:36.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.206+0000 7fc46b7fe640 1 --2- 172.21.15.43:0/2097712452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55761b8f5680 0x55761b8f9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:36.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.206+0000 7fc46a7fc640 1 --2- 172.21.15.43:0/2097712452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55761b8eadc0 0x55761b8f99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:36.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.206+0000 7fc46affd640 1 -- 172.21.15.43:0/2097712452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55761b8eadc0 msgr2=0x55761b8f99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:36.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.206+0000 7fc46affd640 1 --2- 172.21.15.43:0/2097712452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55761b8eadc0 0x55761b8f99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.206+0000 7fc46affd640 1 -- 172.21.15.43:0/2097712452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55761b8f5680 msgr2=0x55761b8f9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:36.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.206+0000 7fc46affd640 1 --2- 172.21.15.43:0/2097712452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55761b8f5680 0x55761b8f9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.206+0000 7fc46affd640 1 -- 172.21.15.43:0/2097712452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55761b977660 con 0x55761b8e6a30 2024-09-11T23:23:36.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.206+0000 7fc46a7fc640 1 --2- 172.21.15.43:0/2097712452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55761b8eadc0 0x55761b8f99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:23:36.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.206+0000 7fc46affd640 1 --2- 172.21.15.43:0/2097712452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8e6a30 0x55761b8f94a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x55761b907c80 tx=0x7fc45c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb981b3bb010656 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:36.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.207+0000 7fc44eff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:23:36.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.207+0000 7fc44eff5640 1 -- 172.21.15.43:0/2097712452 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c003970 con 0x55761b8e6a30 2024-09-11T23:23:36.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.207+0000 7fc4738da740 1 -- 172.21.15.43:0/2097712452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4600bc1c0 con 0x55761b8e6a30 2024-09-11T23:23:36.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.207+0000 7fc44eff5640 1 -- 172.21.15.43:0/2097712452 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fc45c003b10 con 0x55761b8e6a30 2024-09-11T23:23:36.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.207+0000 7fc44eff5640 1 -- 172.21.15.43:0/2097712452 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc45c034050 con 0x55761b8e6a30 2024-09-11T23:23:36.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.208+0000 7fc4738da740 10 client.15828 fetch_fsmap learned FSMap version 26 2024-09-11T23:23:36.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.208+0000 7fc4738da740 1 -- 172.21.15.43:0/2097712452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55761b978990 con 0x55761b8e6a30 2024-09-11T23:23:36.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.208+0000 7fc44eff5640 1 -- 172.21.15.43:0/2097712452 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4600bc1c0 con 0x55761b8e6a30 2024-09-11T23:23:36.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.208+0000 7fc44eff5640 1 -- 172.21.15.43:0/2097712452 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc45c00f040 con 0x55761b8e6a30 2024-09-11T23:23:36.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.208+0000 7fc4738da740 10 client.15828 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:23:36.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.208+0000 7fc4738da740 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:36.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.208+0000 7fc4738da740 20 client.15828 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:36.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.209+0000 7fc4738da740 1 --2- 172.21.15.43:0/2097712452 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55761b979140 0x55761b97b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:36.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.209+0000 7fc4738da740 4 client.15828 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:23:36.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.209+0000 7fc4738da740 1 -- 172.21.15.43:0/2097712452 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55761b857d60 con 0x55761b979140 2024-09-11T23:23:36.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.210+0000 7fc46a7fc640 1 --2- 172.21.15.43:0/2097712452 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55761b979140 0x55761b97b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:36.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.212+0000 7fc44eff5640 10 client.15828 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:36.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.212+0000 7fc46a7fc640 1 --2- 172.21.15.43:0/2097712452 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55761b979140 0x55761b97b560 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4361881a6300f90 server_cookie=5d5996be540a74eb in_seq=0 out_seq=0 2024-09-11T23:23:36.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.213+0000 7fc44eff5640 1 -- 172.21.15.43:0/2097712452 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55761b857d60 con 0x55761b979140 2024-09-11T23:23:36.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.213+0000 7fc44eff5640 10 client.15828 handle_command_reply: tid=0 2024-09-11T23:23:36.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.213+0000 7fc4738da740 1 client.15828 shutdown 2024-09-11T23:23:36.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.213+0000 7fc4738da740 1 -- 172.21.15.43:0/2097712452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8e6a30 msgr2=0x55761b8f94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:36.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.213+0000 7fc4738da740 1 --2- 172.21.15.43:0/2097712452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8e6a30 0x55761b8f94a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x55761b907c80 tx=0x7fc45c004a40 comp rx=0 tx=0).stop 2024-09-11T23:23:36.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.213+0000 7fc4738da740 1 -- 172.21.15.43:0/2097712452 shutdown_connections 2024-09-11T23:23:36.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.213+0000 7fc4738da740 1 --2- 172.21.15.43:0/2097712452 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55761b979140 0x55761b97b560 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.213+0000 7fc4738da740 1 --2- 172.21.15.43:0/2097712452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55761b8f5680 0x55761b8f9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.213+0000 7fc4738da740 1 --2- 172.21.15.43:0/2097712452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55761b8eadc0 0x55761b8f99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.213+0000 7fc4738da740 1 --2- 172.21.15.43:0/2097712452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55761b8e6a30 0x55761b8f94a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.213+0000 7fc4738da740 1 -- 172.21.15.43:0/2097712452 >> 172.21.15.43:0/2097712452 conn(0x55761b8f2840 msgr2=0x55761b8f4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:36.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.213+0000 7fc44eff5640 0 client.15828 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:36.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.214+0000 7fc4738da740 1 -- 172.21.15.43:0/2097712452 shutdown_connections 2024-09-11T23:23:36.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.214+0000 7fc4738da740 1 -- 172.21.15.43:0/2097712452 wait complete. 2024-09-11T23:23:36.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.214+0000 7fc4738da740 20 client.15828 trim_cache size 0 max 16384 2024-09-11T23:23:36.215 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:23:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:23:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-09-11T23:23:36.215 INFO:teuthology.orchestra.run.smithi043.stdout: "49e8eaf5-3fe5-43ab-9b96-c1293a57cccb": { 2024-09-11T23:23:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-09-11T23:23:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "49e8eaf5-3fe5-43ab-9b96-c1293a57cccb", 2024-09-11T23:23:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-09-11T23:23:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:23:36.216 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:23:36.216 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:23:36.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.216+0000 7fc471501640 1 -- 172.21.15.43:0/4122174754 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc4540a2cf0 msgr2=0x7fc4540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.216+0000 7fc471501640 1 --2- 172.21.15.43:0/4122174754 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc4540a2cf0 0x7fc4540a51b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55761b8e7890 tx=0x7fc45c0094a0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.216+0000 7fc471501640 1 -- 172.21.15.43:0/4122174754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc46c102790 msgr2=0x7fc46c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.216+0000 7fc471501640 1 --2- 172.21.15.43:0/4122174754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc46c102790 0x7fc46c074a30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc460035040 tx=0x7fc460033d90 comp rx=0 tx=0).stop 2024-09-11T23:23:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.216+0000 7fc471501640 1 -- 172.21.15.43:0/4122174754 shutdown_connections 2024-09-11T23:23:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.216+0000 7fc471501640 1 --2- 172.21.15.43:0/4122174754 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc4540a2cf0 0x7fc4540a51b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.216+0000 7fc471501640 1 --2- 172.21.15.43:0/4122174754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc46c106f10 0x7fc46c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.216+0000 7fc471501640 1 --2- 172.21.15.43:0/4122174754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc46c102790 0x7fc46c074a30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.216+0000 7fc471501640 1 --2- 172.21.15.43:0/4122174754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc46c101de0 0x7fc46c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.216+0000 7fc471501640 1 -- 172.21.15.43:0/4122174754 >> 172.21.15.43:0/4122174754 conn(0x7fc46c0fd8e0 msgr2=0x7fc46c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:36.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.217+0000 7fc471501640 1 -- 172.21.15.43:0/4122174754 shutdown_connections 2024-09-11T23:23:36.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:36.217+0000 7fc471501640 1 -- 172.21.15.43:0/4122174754 wait complete. 2024-09-11T23:23:36.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:35 smithi090 ceph-mon[35130]: pgmap v55: 97 pgs: 97 active+clean; 2.7 GiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 227 MiB/s wr, 185 op/s 2024-09-11T23:23:36.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:35 smithi097 ceph-mon[35562]: pgmap v55: 97 pgs: 97 active+clean; 2.7 GiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 227 MiB/s wr, 185 op/s 2024-09-11T23:23:37.404 INFO:tasks.cephfs.filesystem:scrub status for tag:49e8eaf5-3fe5-43ab-9b96-c1293a57cccb - {'path': '/', 'tag': '49e8eaf5-3fe5-43ab-9b96-c1293a57cccb', 'options': 'recursive,force'} 2024-09-11T23:23:38.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:37 smithi090 ceph-mon[35130]: pgmap v56: 97 pgs: 97 active+clean; 7.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 597 MiB/s wr, 408 op/s 2024-09-11T23:23:38.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:37 smithi097 ceph-mon[35562]: pgmap v56: 97 pgs: 97 active+clean; 7.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 597 MiB/s wr, 408 op/s 2024-09-11T23:23:38.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:37 smithi043 ceph-mon[30950]: pgmap v56: 97 pgs: 97 active+clean; 7.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 597 MiB/s wr, 408 op/s 2024-09-11T23:23:39.147 INFO:tasks.workunit.client.0.smithi043.stdout:fs setup took 12 secs 2024-09-11T23:23:40.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:39 smithi090 ceph-mon[35130]: pgmap v57: 97 pgs: 97 active+clean; 7.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 597 MiB/s wr, 395 op/s 2024-09-11T23:23:40.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:39 smithi090 ceph-mon[35130]: scrub summary: idle 2024-09-11T23:23:40.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:39 smithi090 ceph-mon[35130]: scrub summary: idle 2024-09-11T23:23:40.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:39 smithi097 ceph-mon[35562]: pgmap v57: 97 pgs: 97 active+clean; 7.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 597 MiB/s wr, 395 op/s 2024-09-11T23:23:40.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:39 smithi097 ceph-mon[35562]: scrub summary: idle 2024-09-11T23:23:40.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:39 smithi097 ceph-mon[35562]: scrub summary: idle 2024-09-11T23:23:40.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:39 smithi043 ceph-mon[30950]: pgmap v57: 97 pgs: 97 active+clean; 7.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 597 MiB/s wr, 395 op/s 2024-09-11T23:23:40.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:39 smithi043 ceph-mon[30950]: scrub summary: idle 2024-09-11T23:23:40.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:39 smithi043 ceph-mon[30950]: scrub summary: idle 2024-09-11T23:23:41.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:40 smithi090 ceph-mon[35130]: scrub summary: idle 2024-09-11T23:23:41.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:40 smithi097 ceph-mon[35562]: scrub summary: idle 2024-09-11T23:23:41.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:40 smithi043 ceph-mon[30950]: scrub summary: idle 2024-09-11T23:23:42.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:41 smithi090 ceph-mon[35130]: pgmap v58: 97 pgs: 97 active+clean; 7.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 597 MiB/s wr, 395 op/s 2024-09-11T23:23:42.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:41 smithi097 ceph-mon[35562]: pgmap v58: 97 pgs: 97 active+clean; 7.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 597 MiB/s wr, 395 op/s 2024-09-11T23:23:42.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:41 smithi043 ceph-mon[30950]: pgmap v58: 97 pgs: 97 active+clean; 7.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 597 MiB/s wr, 395 op/s 2024-09-11T23:23:42.791 INFO:tasks.workunit.client.0.smithi043.stdout:Syncing()...3 sec 2024-09-11T23:23:42.791 INFO:tasks.workunit.client.0.smithi043.stdout:Starting Actual Benchmark At: Wed Sep 11 23:23:42 2024 2024-09-11T23:23:42.791 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-11T23:23:44.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:43 smithi090 ceph-mon[35130]: pgmap v59: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 947 MiB/s wr, 533 op/s 2024-09-11T23:23:44.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:43 smithi097 ceph-mon[35562]: pgmap v59: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 947 MiB/s wr, 533 op/s 2024-09-11T23:23:44.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:43 smithi043 ceph-mon[30950]: pgmap v59: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 947 MiB/s wr, 533 op/s 2024-09-11T23:23:46.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:45 smithi090 ceph-mon[35130]: pgmap v60: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 721 MiB/s wr, 360 op/s 2024-09-11T23:23:46.429 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:45 smithi043 ceph-mon[30950]: pgmap v60: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 721 MiB/s wr, 360 op/s 2024-09-11T23:23:46.430 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:45 smithi097 ceph-mon[35562]: pgmap v60: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 721 MiB/s wr, 360 op/s 2024-09-11T23:23:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.323+0000 7ff6bc023640 1 -- 172.21.15.43:0/761311629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6b4109e10 msgr2=0x7ff6b410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.323+0000 7ff6bc023640 1 --2- 172.21.15.43:0/761311629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6b4109e10 0x7ff6b410e240 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff6a40074e0 tx=0x7ff6a402b080 comp rx=0 tx=0).stop 2024-09-11T23:23:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.323+0000 7ff6bc023640 1 -- 172.21.15.43:0/761311629 shutdown_connections 2024-09-11T23:23:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.323+0000 7ff6bc023640 1 --2- 172.21.15.43:0/761311629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6b410e9a0 0x7ff6b4115410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.323+0000 7ff6bc023640 1 --2- 172.21.15.43:0/761311629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6b4109e10 0x7ff6b410e240 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.323+0000 7ff6bc023640 1 --2- 172.21.15.43:0/761311629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6b4109460 0x7ff6b4109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.323+0000 7ff6bc023640 1 -- 172.21.15.43:0/761311629 >> 172.21.15.43:0/761311629 conn(0x7ff6b406d200 msgr2=0x7ff6b406d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.325+0000 7ff6bc023640 1 -- 172.21.15.43:0/761311629 shutdown_connections 2024-09-11T23:23:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.326+0000 7ff6bc023640 1 -- 172.21.15.43:0/761311629 wait complete. 2024-09-11T23:23:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.327+0000 7ff6bc023640 1 Processor -- start 2024-09-11T23:23:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.327+0000 7ff6bc023640 1 -- start start 2024-09-11T23:23:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.328+0000 7ff6bc023640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6b4109460 0x7ff6b419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.328+0000 7ff6bc023640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6b410e9a0 0x7ff6b419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.328+0000 7ff6bc023640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6b419f450 0x7ff6b41a33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.328+0000 7ff6bc023640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b41185e0 con 0x7ff6b419f450 2024-09-11T23:23:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.328+0000 7ff6bc023640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b4118460 con 0x7ff6b410e9a0 2024-09-11T23:23:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.328+0000 7ff6bc023640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b4118760 con 0x7ff6b4109460 2024-09-11T23:23:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.328+0000 7ff6ba5c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6b419f450 0x7ff6b41a33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.329+0000 7ff6ba5c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6b419f450 0x7ff6b41a33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48920/0 (socket says 172.21.15.43:48920) 2024-09-11T23:23:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.329+0000 7ff6ba5c0640 1 -- 172.21.15.43:0/4221448807 learned_addr learned my addr 172.21.15.43:0/4221448807 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.329+0000 7ff6b95be640 1 --2- 172.21.15.43:0/4221448807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6b410e9a0 0x7ff6b419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.329+0000 7ff6ba5c0640 1 -- 172.21.15.43:0/4221448807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6b4109460 msgr2=0x7ff6b419e8c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:23:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.329+0000 7ff6ba5c0640 1 --2- 172.21.15.43:0/4221448807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6b4109460 0x7ff6b419e8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.329+0000 7ff6ba5c0640 1 -- 172.21.15.43:0/4221448807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6b410e9a0 msgr2=0x7ff6b419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.329+0000 7ff6ba5c0640 1 --2- 172.21.15.43:0/4221448807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6b410e9a0 0x7ff6b419ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.329+0000 7ff6ba5c0640 1 -- 172.21.15.43:0/4221448807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b41a3c60 con 0x7ff6b419f450 2024-09-11T23:23:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.330+0000 7ff6ba5c0640 1 --2- 172.21.15.43:0/4221448807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6b419f450 0x7ff6b41a33a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00b990 tx=0x7ff6ac00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.330+0000 7ff6aaffd640 1 -- 172.21.15.43:0/4221448807 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac019070 con 0x7ff6b419f450 2024-09-11T23:23:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.330+0000 7ff6aaffd640 1 -- 172.21.15.43:0/4221448807 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6ac00cba0 con 0x7ff6b419f450 2024-09-11T23:23:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.330+0000 7ff6aaffd640 1 -- 172.21.15.43:0/4221448807 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0076e0 con 0x7ff6b419f450 2024-09-11T23:23:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.332+0000 7ff6bc023640 1 -- 172.21.15.43:0/4221448807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6b41a3f50 con 0x7ff6b419f450 2024-09-11T23:23:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.332+0000 7ff6bc023640 1 -- 172.21.15.43:0/4221448807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6b41aba50 con 0x7ff6b419f450 2024-09-11T23:23:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.339+0000 7ff6bedbb740 1 Processor -- start 2024-09-11T23:23:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.341+0000 7ff6bedbb740 1 -- start start 2024-09-11T23:23:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.341+0000 7ff6bedbb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564dadd56af0 0x564dadd56ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.342+0000 7ff6bedbb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd57410 0x564dadd61770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.342+0000 7ff6bedbb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564dadd61cb0 0x564dadd640a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.342+0000 7ff6bedbb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564dadcbb2f0 con 0x564dadd57410 2024-09-11T23:23:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.342+0000 7ff6bedbb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564dadc63270 con 0x564dadd56af0 2024-09-11T23:23:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.342+0000 7ff6bedbb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564dadcb72b0 con 0x564dadd61cb0 2024-09-11T23:23:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.342+0000 7ff6aaffd640 1 -- 172.21.15.43:0/4221448807 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff6ac0149b0 con 0x7ff6b419f450 2024-09-11T23:23:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.343+0000 7ff6b9dbf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd57410 0x564dadd61770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.343+0000 7ff6b9dbf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd57410 0x564dadd61770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48934/0 (socket says 172.21.15.43:48934) 2024-09-11T23:23:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.343+0000 7ff6b9dbf640 1 -- 172.21.15.43:0/2994940586 learned_addr learned my addr 172.21.15.43:0/2994940586 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.343+0000 7ff6b9dbf640 1 -- 172.21.15.43:0/2994940586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564dadd61cb0 msgr2=0x564dadd640a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:23:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.343+0000 7ff6b9dbf640 1 --2- 172.21.15.43:0/2994940586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564dadd61cb0 0x564dadd640a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.343+0000 7ff6b9dbf640 1 -- 172.21.15.43:0/2994940586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564dadd56af0 msgr2=0x564dadd56ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:23:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.343+0000 7ff6b9dbf640 1 --2- 172.21.15.43:0/2994940586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564dadd56af0 0x564dadd56ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.343+0000 7ff6b9dbf640 1 -- 172.21.15.43:0/2994940586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dadc62340 con 0x564dadd57410 2024-09-11T23:23:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.343+0000 7ff6b9dbf640 1 --2- 172.21.15.43:0/2994940586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd57410 0x564dadd61770 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009780 tx=0x7ff6b0009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811c30da3f27fcfe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.343+0000 7ff69ffff640 1 -- 172.21.15.43:0/2994940586 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0015020 con 0x564dadd57410 2024-09-11T23:23:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.344+0000 7ff69ffff640 1 -- 172.21.15.43:0/2994940586 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6b0011420 con 0x564dadd57410 2024-09-11T23:23:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.344+0000 7ff6aaffd640 1 --2- 172.21.15.43:0/4221448807 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff6a00a2f00 0x7ff6a00a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.344+0000 7ff6aaffd640 1 -- 172.21.15.43:0/4221448807 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff6ac0ce880 con 0x7ff6b419f450 2024-09-11T23:23:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.344+0000 7ff69ffff640 1 -- 172.21.15.43:0/2994940586 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b00105c0 con 0x564dadd57410 2024-09-11T23:23:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.345+0000 7ff6bedbb740 1 -- 172.21.15.43:0/2994940586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd57410 msgr2=0x564dadd61770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.345+0000 7ff6bedbb740 1 --2- 172.21.15.43:0/2994940586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd57410 0x564dadd61770 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009780 tx=0x7ff6b0009c40 comp rx=0 tx=0).stop 2024-09-11T23:23:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.346+0000 7ff6bedbb740 1 -- 172.21.15.43:0/2994940586 shutdown_connections 2024-09-11T23:23:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.346+0000 7ff6bedbb740 1 --2- 172.21.15.43:0/2994940586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564dadd61cb0 0x564dadd640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.346+0000 7ff6bedbb740 1 --2- 172.21.15.43:0/2994940586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd57410 0x564dadd61770 secure :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009780 tx=0x7ff6b0009c40 comp rx=0 tx=0).stop 2024-09-11T23:23:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.346+0000 7ff6bedbb740 1 --2- 172.21.15.43:0/2994940586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564dadd56af0 0x564dadd56ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.346+0000 7ff6bedbb740 1 -- 172.21.15.43:0/2994940586 >> 172.21.15.43:0/2994940586 conn(0x564dadd53060 msgr2=0x564dadd53440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.349+0000 7ff6bedbb740 1 -- 172.21.15.43:0/2994940586 shutdown_connections 2024-09-11T23:23:47.351 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.350+0000 7ff6b95be640 1 --2- 172.21.15.43:0/4221448807 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff6a00a2f00 0x7ff6a00a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.352+0000 7ff6bedbb740 1 -- 172.21.15.43:0/2994940586 wait complete. 2024-09-11T23:23:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.352+0000 7ff6bedbb740 1 -- start start 2024-09-11T23:23:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.352+0000 7ff6bedbb740 1 Processor -- start 2024-09-11T23:23:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.353+0000 7ff6bedbb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd56af0 0x564dadd5edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.353+0000 7ff6bedbb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564dadd61cb0 0x564dadd5f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.353+0000 7ff6bedbb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564dadd5f830 0x564dadd6dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.353+0000 7ff6bedbb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564dadd64880 con 0x564dadd56af0 2024-09-11T23:23:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.353+0000 7ff6bedbb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564dadc44290 con 0x564dadd5f830 2024-09-11T23:23:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.353+0000 7ff6bedbb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564dadd64a00 con 0x564dadd61cb0 2024-09-11T23:23:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.353+0000 7ff6bedbb740 10 client.0 _ll_register_callbacks cb 0x564dadc75140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.355+0000 7ff6ba5c0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564dadd5f830 0x564dadd6dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.355+0000 7ff6ba5c0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564dadd5f830 0x564dadd6dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:34624/0 (socket says 172.21.15.43:34624) 2024-09-11T23:23:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.355+0000 7ff6ba5c0640 1 -- 172.21.15.43:0/847545328 learned_addr learned my addr 172.21.15.43:0/847545328 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.355+0000 7ff6b9dbf640 1 --2- 172.21.15.43:0/847545328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd56af0 0x564dadd5edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.355+0000 7ff6b95be640 1 --2- 172.21.15.43:0/847545328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564dadd61cb0 0x564dadd5f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.356+0000 7ff6ba5c0640 1 -- 172.21.15.43:0/847545328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564dadd61cb0 msgr2=0x564dadd5f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.356+0000 7ff6ba5c0640 1 --2- 172.21.15.43:0/847545328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564dadd61cb0 0x564dadd5f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.356+0000 7ff6ba5c0640 1 -- 172.21.15.43:0/847545328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd56af0 msgr2=0x564dadd5edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.356+0000 7ff6ba5c0640 1 --2- 172.21.15.43:0/847545328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd56af0 0x564dadd5edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.356+0000 7ff6ba5c0640 1 -- 172.21.15.43:0/847545328 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dad9ce210 con 0x564dadd5f830 2024-09-11T23:23:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.362+0000 7ff6ba5c0640 1 --2- 172.21.15.43:0/847545328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564dadd5f830 0x564dadd6dd60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00b960 tx=0x7ff6ac09ae50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc6c44655e4e844e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.363+0000 7ff69d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:23:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.364+0000 7ff69d7fa640 1 -- 172.21.15.43:0/847545328 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0cc1f0 con 0x564dadd5f830 2024-09-11T23:23:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.364+0000 7ff6b95be640 1 --2- 172.21.15.43:0/4221448807 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff6a00a2f00 0x7ff6a00a53c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x564dadd543e0 tx=0x7ff6a4004710 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.364+0000 7ff69d7fa640 1 -- 172.21.15.43:0/847545328 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff6ac0cc390 con 0x564dadd5f830 2024-09-11T23:23:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.364+0000 7ff6bedbb740 1 -- 172.21.15.43:0/847545328 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6b0011420 con 0x564dadd5f830 2024-09-11T23:23:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.365+0000 7ff69d7fa640 1 -- 172.21.15.43:0/847545328 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff6ac0a3630 con 0x564dadd5f830 2024-09-11T23:23:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.365+0000 7ff6bedbb740 10 client.25379 fetch_fsmap learned FSMap version 26 2024-09-11T23:23:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.365+0000 7ff6bedbb740 1 -- 172.21.15.43:0/847545328 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564dada9a620 con 0x564dadd5f830 2024-09-11T23:23:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.365+0000 7ff69d7fa640 1 -- 172.21.15.43:0/847545328 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6b0011420 con 0x564dadd5f830 2024-09-11T23:23:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.366+0000 7ff69d7fa640 1 -- 172.21.15.43:0/847545328 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff6ac099d10 con 0x564dadd5f830 2024-09-11T23:23:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.366+0000 7ff6bedbb740 10 client.25379 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:23:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.366+0000 7ff6bedbb740 10 client.25379 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.366+0000 7ff6bedbb740 20 client.25379 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.366+0000 7ff6bedbb740 1 --2- 172.21.15.43:0/847545328 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564dadd72290 0x564dadd74680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.366+0000 7ff6bedbb740 4 client.25379 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:23:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.366+0000 7ff6bedbb740 1 -- 172.21.15.43:0/847545328 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564dadcc8a20 con 0x564dadd72290 2024-09-11T23:23:47.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.371+0000 7ff6b9dbf640 1 --2- 172.21.15.43:0/847545328 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564dadd72290 0x564dadd74680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.373+0000 7ff6b9dbf640 1 --2- 172.21.15.43:0/847545328 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564dadd72290 0x564dadd74680 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db18e0b6103187d6 server_cookie=c5ec46111e2b561 in_seq=0 out_seq=0 2024-09-11T23:23:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.374+0000 7ff69d7fa640 10 client.25379 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.376+0000 7ff69d7fa640 1 -- 172.21.15.43:0/847545328 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564dadcc8a20 con 0x564dadd72290 2024-09-11T23:23:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.376+0000 7ff69d7fa640 10 client.25379 handle_command_reply: tid=0 2024-09-11T23:23:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.376+0000 7ff6bedbb740 1 client.25379 shutdown 2024-09-11T23:23:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.377+0000 7ff6bedbb740 1 -- 172.21.15.43:0/847545328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564dadd5f830 msgr2=0x564dadd6dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.377+0000 7ff6bedbb740 1 --2- 172.21.15.43:0/847545328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564dadd5f830 0x564dadd6dd60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00b960 tx=0x7ff6ac09ae50 comp rx=0 tx=0).stop 2024-09-11T23:23:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.377+0000 7ff6bedbb740 1 -- 172.21.15.43:0/847545328 shutdown_connections 2024-09-11T23:23:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.377+0000 7ff6bedbb740 1 --2- 172.21.15.43:0/847545328 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564dadd72290 0x564dadd74680 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.377+0000 7ff6bedbb740 1 --2- 172.21.15.43:0/847545328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564dadd5f830 0x564dadd6dd60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.377+0000 7ff6bedbb740 1 --2- 172.21.15.43:0/847545328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564dadd61cb0 0x564dadd5f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.377+0000 7ff6bedbb740 1 --2- 172.21.15.43:0/847545328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd56af0 0x564dadd5edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.377+0000 7ff6bedbb740 1 -- 172.21.15.43:0/847545328 >> 172.21.15.43:0/847545328 conn(0x564dadd53060 msgr2=0x564dadd58dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.377+0000 7ff69d7fa640 0 client.25379 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.379+0000 7ff6bedbb740 1 -- 172.21.15.43:0/847545328 shutdown_connections 2024-09-11T23:23:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.383+0000 7ff6bedbb740 1 -- 172.21.15.43:0/847545328 wait complete. 2024-09-11T23:23:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.384+0000 7ff6bedbb740 20 client.25379 trim_cache size 0 max 16384 2024-09-11T23:23:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.426+0000 7ff6bedbb740 1 Processor -- start 2024-09-11T23:23:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.429+0000 7ff6bedbb740 1 -- start start 2024-09-11T23:23:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.430+0000 7ff6bedbb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564dadd56af0 0x564dadd6db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.430+0000 7ff6bedbb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564dadd5f830 0x564dadd79f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.430+0000 7ff6bedbb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd61cb0 0x564dadd7a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.430+0000 7ff6bedbb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b41b6860 con 0x564dadd61cb0 2024-09-11T23:23:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.430+0000 7ff6bedbb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b41b4170 con 0x564dadd5f830 2024-09-11T23:23:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.430+0000 7ff6bedbb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b4109840 con 0x564dadd56af0 2024-09-11T23:23:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.431+0000 7ff6ba5c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd61cb0 0x564dadd7a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.431+0000 7ff6ba5c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd61cb0 0x564dadd7a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48950/0 (socket says 172.21.15.43:48950) 2024-09-11T23:23:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.431+0000 7ff6ba5c0640 1 -- 172.21.15.43:0/2276033321 learned_addr learned my addr 172.21.15.43:0/2276033321 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.431+0000 7ff6b9dbf640 1 --2- 172.21.15.43:0/2276033321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564dadd56af0 0x564dadd6db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.432+0000 7ff6b9dbf640 1 -- 172.21.15.43:0/2276033321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564dadd5f830 msgr2=0x564dadd79f50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:23:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.432+0000 7ff6b9dbf640 1 --2- 172.21.15.43:0/2276033321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564dadd5f830 0x564dadd79f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.432+0000 7ff6b9dbf640 1 -- 172.21.15.43:0/2276033321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd61cb0 msgr2=0x564dadd7a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.432+0000 7ff6b9dbf640 1 --2- 172.21.15.43:0/2276033321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd61cb0 0x564dadd7a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.433+0000 7ff6b9dbf640 1 -- 172.21.15.43:0/2276033321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dadd53cf0 con 0x564dadd56af0 2024-09-11T23:23:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.433+0000 7ff6b9dbf640 1 --2- 172.21.15.43:0/2276033321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564dadd56af0 0x564dadd6db50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x564dadd53fc0 tx=0x7ff6b0011f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c329fce2668e1801 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.435+0000 7ff69f7fe640 1 -- 172.21.15.43:0/2276033321 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0015040 con 0x564dadd56af0 2024-09-11T23:23:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.435+0000 7ff69f7fe640 1 -- 172.21.15.43:0/2276033321 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6b000b8d0 con 0x564dadd56af0 2024-09-11T23:23:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.435+0000 7ff69f7fe640 1 -- 172.21.15.43:0/2276033321 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b001a680 con 0x564dadd56af0 2024-09-11T23:23:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.437+0000 7ff6bedbb740 1 -- 172.21.15.43:0/2276033321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564dadd56af0 msgr2=0x564dadd6db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.437+0000 7ff6bedbb740 1 --2- 172.21.15.43:0/2276033321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564dadd56af0 0x564dadd6db50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x564dadd53fc0 tx=0x7ff6b0011f00 comp rx=0 tx=0).stop 2024-09-11T23:23:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.443+0000 7ff6bedbb740 1 -- 172.21.15.43:0/2276033321 shutdown_connections 2024-09-11T23:23:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.443+0000 7ff6bedbb740 1 --2- 172.21.15.43:0/2276033321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd61cb0 0x564dadd7a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.443+0000 7ff6bedbb740 1 --2- 172.21.15.43:0/2276033321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564dadd5f830 0x564dadd79f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.443+0000 7ff6bedbb740 1 --2- 172.21.15.43:0/2276033321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564dadd56af0 0x564dadd6db50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.443+0000 7ff6bedbb740 1 -- 172.21.15.43:0/2276033321 >> 172.21.15.43:0/2276033321 conn(0x564dadd53060 msgr2=0x564dadd60e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.450+0000 7ff6bedbb740 1 -- 172.21.15.43:0/2276033321 shutdown_connections 2024-09-11T23:23:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.450+0000 7ff6bedbb740 1 -- 172.21.15.43:0/2276033321 wait complete. 2024-09-11T23:23:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.450+0000 7ff6bedbb740 1 -- start start 2024-09-11T23:23:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.451+0000 7ff6bedbb740 1 Processor -- start 2024-09-11T23:23:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.451+0000 7ff6bedbb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564dadd56af0 0x564dadd69cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.452+0000 7ff6bedbb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564dadd5f830 0x564dadd6a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:47.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.452+0000 7ff6bedbb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd61cb0 0x564dadd66c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:47.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.452+0000 7ff6b9dbf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564dadd56af0 0x564dadd69cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:47.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.452+0000 7ff6b9dbf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564dadd56af0 0x564dadd69cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:34656/0 (socket says 172.21.15.43:34656) 2024-09-11T23:23:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.452+0000 7ff6b9dbf640 1 -- 172.21.15.43:0/2855349761 learned_addr learned my addr 172.21.15.43:0/2855349761 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.452+0000 7ff6bedbb740 1 -- 172.21.15.43:0/2855349761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b41b2660 con 0x564dadd61cb0 2024-09-11T23:23:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.452+0000 7ff6bedbb740 1 -- 172.21.15.43:0/2855349761 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b41b3aa0 con 0x564dadd56af0 2024-09-11T23:23:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.452+0000 7ff6bedbb740 1 -- 172.21.15.43:0/2855349761 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b41b2d00 con 0x564dadd5f830 2024-09-11T23:23:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.452+0000 7ff6bedbb740 10 client.0 _ll_register_callbacks cb 0x7ff6ac097170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:23:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.452+0000 7ff6ba5c0640 1 --2- 172.21.15.43:0/2855349761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd61cb0 0x564dadd66c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:47.454 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.453+0000 7ff6b95be640 1 --2- 172.21.15.43:0/2855349761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564dadd5f830 0x564dadd6a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:47.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.454+0000 7ff6b95be640 1 -- 172.21.15.43:0/2855349761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564dadd56af0 msgr2=0x564dadd69cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:47.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.454+0000 7ff6b95be640 1 --2- 172.21.15.43:0/2855349761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564dadd56af0 0x564dadd69cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.454+0000 7ff6b95be640 1 -- 172.21.15.43:0/2855349761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd61cb0 msgr2=0x564dadd66c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:23:47.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.454+0000 7ff6b95be640 1 --2- 172.21.15.43:0/2855349761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd61cb0 0x564dadd66c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:47.455 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.454+0000 7ff6b95be640 1 -- 172.21.15.43:0/2855349761 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dadd5f660 con 0x564dadd5f830 2024-09-11T23:23:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.455+0000 7ff6b95be640 1 --2- 172.21.15.43:0/2855349761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564dadd5f830 0x564dadd6a200 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff6b41a0080 tx=0x7ff6a402b080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eac35640d19db316 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.456+0000 7ff69d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-11T23:23:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.457+0000 7ff69d7fa640 1 -- 172.21.15.43:0/2855349761 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a402bd90 con 0x564dadd5f830 2024-09-11T23:23:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.457+0000 7ff69d7fa640 1 -- 172.21.15.43:0/2855349761 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff6a4004030 con 0x564dadd5f830 2024-09-11T23:23:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.458+0000 7ff6bedbb740 1 -- 172.21.15.43:0/2855349761 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6b000b8d0 con 0x564dadd5f830 2024-09-11T23:23:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.460+0000 7ff69d7fa640 1 -- 172.21.15.43:0/2855349761 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff6a4004230 con 0x564dadd5f830 2024-09-11T23:23:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.462+0000 7ff6bedbb740 10 client.25372 fetch_fsmap learned FSMap version 26 2024-09-11T23:23:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.462+0000 7ff6bedbb740 1 -- 172.21.15.43:0/2855349761 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564dadd61940 con 0x564dadd5f830 2024-09-11T23:23:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.464+0000 7ff69d7fa640 1 -- 172.21.15.43:0/2855349761 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6b000b8d0 con 0x564dadd5f830 2024-09-11T23:23:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.464+0000 7ff69d7fa640 1 -- 172.21.15.43:0/2855349761 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff6a40039a0 con 0x564dadd5f830 2024-09-11T23:23:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.464+0000 7ff6bedbb740 10 client.25372 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:23:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.464+0000 7ff6bedbb740 10 client.25372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:23:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.464+0000 7ff6bedbb740 20 client.25372 populate_metadata read hostname 'smithi043' 2024-09-11T23:23:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.464+0000 7ff6bedbb740 1 --2- 172.21.15.43:0/2855349761 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564dadddda80 0x564dadddfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.464+0000 7ff6bedbb740 4 client.25372 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:23:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.464+0000 7ff6bedbb740 1 -- 172.21.15.43:0/2855349761 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564dadcc2d70 con 0x564dadddda80 2024-09-11T23:23:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.467+0000 7ff6b9dbf640 1 --2- 172.21.15.43:0/2855349761 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564dadddda80 0x564dadddfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:47.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.478+0000 7ff6b9dbf640 1 --2- 172.21.15.43:0/2855349761 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564dadddda80 0x564dadddfea0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=164fed56598e370b server_cookie=a3514c1acf7443df in_seq=0 out_seq=0 2024-09-11T23:23:47.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:23:47.479+0000 7ff69d7fa640 10 client.25372 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:23:48.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:47 smithi090 ceph-mon[35130]: pgmap v61: 97 pgs: 97 active+clean; 12 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 807 MiB/s wr, 4.13k op/s 2024-09-11T23:23:48.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:47 smithi097 ceph-mon[35562]: pgmap v61: 97 pgs: 97 active+clean; 12 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 807 MiB/s wr, 4.13k op/s 2024-09-11T23:23:48.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:47 smithi043 ceph-mon[30950]: pgmap v61: 97 pgs: 97 active+clean; 12 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 807 MiB/s wr, 4.13k op/s 2024-09-11T23:23:48.929 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.4 2024-09-11T23:23:48.930 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:23:49.485 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:23:50.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:49 smithi090 ceph-mon[35130]: pgmap v62: 97 pgs: 97 active+clean; 13 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 529 MiB/s wr, 9.47k op/s 2024-09-11T23:23:50.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:49 smithi097 ceph-mon[35562]: pgmap v62: 97 pgs: 97 active+clean; 13 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 529 MiB/s wr, 9.47k op/s 2024-09-11T23:23:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:49 smithi043 ceph-mon[30950]: pgmap v62: 97 pgs: 97 active+clean; 13 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 529 MiB/s wr, 9.47k op/s 2024-09-11T23:23:52.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:51 smithi090 ceph-mon[35130]: pgmap v63: 97 pgs: 97 active+clean; 13 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 529 MiB/s wr, 9.47k op/s 2024-09-11T23:23:52.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:51 smithi097 ceph-mon[35562]: pgmap v63: 97 pgs: 97 active+clean; 13 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 529 MiB/s wr, 9.47k op/s 2024-09-11T23:23:52.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:51 smithi043 ceph-mon[30950]: pgmap v63: 97 pgs: 97 active+clean; 13 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 529 MiB/s wr, 9.47k op/s 2024-09-11T23:23:54.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:53 smithi043 ceph-mon[30950]: pgmap v64: 97 pgs: 97 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 646 MiB/s wr, 25.40k op/s 2024-09-11T23:23:54.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:53 smithi090 ceph-mon[35130]: pgmap v64: 97 pgs: 97 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 646 MiB/s wr, 25.40k op/s 2024-09-11T23:23:54.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:53 smithi097 ceph-mon[35562]: pgmap v64: 97 pgs: 97 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 646 MiB/s wr, 25.40k op/s 2024-09-11T23:23:54.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.811+0000 7fce69b7c640 1 -- 172.21.15.43:0/4104357564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce64110190 msgr2=0x7fce6406e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:54.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.811+0000 7fce69b7c640 1 --2- 172.21.15.43:0/4104357564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce64110190 0x7fce6406e4f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fce6001a910 tx=0x7fce6003b470 comp rx=0 tx=0).stop 2024-09-11T23:23:54.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.812+0000 7fce69b7c640 1 -- 172.21.15.43:0/4104357564 shutdown_connections 2024-09-11T23:23:54.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.812+0000 7fce69b7c640 1 --2- 172.21.15.43:0/4104357564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce6411b140 0x7fce6411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:54.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.812+0000 7fce69b7c640 1 --2- 172.21.15.43:0/4104357564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce6406ea30 0x7fce6406ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:54.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.812+0000 7fce69b7c640 1 --2- 172.21.15.43:0/4104357564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce64110190 0x7fce6406e4f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:54.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.812+0000 7fce69b7c640 1 -- 172.21.15.43:0/4104357564 >> 172.21.15.43:0/4104357564 conn(0x7fce64073180 msgr2=0x7fce64073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:54.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.812+0000 7fce69b7c640 1 -- 172.21.15.43:0/4104357564 shutdown_connections 2024-09-11T23:23:54.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.812+0000 7fce69b7c640 1 -- 172.21.15.43:0/4104357564 wait complete. 2024-09-11T23:23:54.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.813+0000 7fce69b7c640 1 Processor -- start 2024-09-11T23:23:54.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.814+0000 7fce69b7c640 1 -- start start 2024-09-11T23:23:54.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.814+0000 7fce69b7c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce6406ea30 0x7fce6411a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:54.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.814+0000 7fce69b7c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce64110190 0x7fce641168a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:54.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.814+0000 7fce69b7c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce6411b140 0x7fce64116de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:54.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.814+0000 7fce69b7c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fce64113360 con 0x7fce64110190 2024-09-11T23:23:54.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.814+0000 7fce69b7c640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fce641131e0 con 0x7fce6406ea30 2024-09-11T23:23:54.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.814+0000 7fce69b7c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fce641134e0 con 0x7fce6411b140 2024-09-11T23:23:54.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.815+0000 7fce6937b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce6411b140 0x7fce64116de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:54.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.815+0000 7fce68b7a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce6406ea30 0x7fce6411a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:54.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.815+0000 7fce68b7a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce6406ea30 0x7fce6411a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:42088/0 (socket says 172.21.15.43:42088) 2024-09-11T23:23:54.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.815+0000 7fce68b7a640 1 -- 172.21.15.43:0/1854525902 learned_addr learned my addr 172.21.15.43:0/1854525902 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:23:54.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.815+0000 7fce6937b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce6411b140 0x7fce64116de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:38070/0 (socket says 172.21.15.43:38070) 2024-09-11T23:23:54.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.817+0000 7fce5bfff640 1 --2- 172.21.15.43:0/1854525902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce64110190 0x7fce641168a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:54.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.818+0000 7fce5bfff640 1 -- 172.21.15.43:0/1854525902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce6411b140 msgr2=0x7fce64116de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:54.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.818+0000 7fce5bfff640 1 --2- 172.21.15.43:0/1854525902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce6411b140 0x7fce64116de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:54.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.818+0000 7fce5bfff640 1 -- 172.21.15.43:0/1854525902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce6406ea30 msgr2=0x7fce6411a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:54.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.818+0000 7fce5bfff640 1 --2- 172.21.15.43:0/1854525902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce6406ea30 0x7fce6411a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:54.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.818+0000 7fce5bfff640 1 -- 172.21.15.43:0/1854525902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce64117510 con 0x7fce64110190 2024-09-11T23:23:54.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.819+0000 7fce5bfff640 1 --2- 172.21.15.43:0/1854525902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce64110190 0x7fce641168a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fce5400c970 tx=0x7fce5400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:54.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.819+0000 7fce59ffb640 1 -- 172.21.15.43:0/1854525902 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54007bb0 con 0x7fce64110190 2024-09-11T23:23:54.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.819+0000 7fce59ffb640 1 -- 172.21.15.43:0/1854525902 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce54007d50 con 0x7fce64110190 2024-09-11T23:23:54.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.819+0000 7fce59ffb640 1 -- 172.21.15.43:0/1854525902 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54018890 con 0x7fce64110190 2024-09-11T23:23:54.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.820+0000 7fce69b7c640 1 -- 172.21.15.43:0/1854525902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce641aaf60 con 0x7fce64110190 2024-09-11T23:23:54.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.820+0000 7fce69b7c640 1 -- 172.21.15.43:0/1854525902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce641ab300 con 0x7fce64110190 2024-09-11T23:23:54.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.822+0000 7fce69b7c640 1 -- 172.21.15.43:0/1854525902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce641096f0 con 0x7fce64110190 2024-09-11T23:23:54.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.826+0000 7fce59ffb640 1 -- 172.21.15.43:0/1854525902 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fce54018a30 con 0x7fce64110190 2024-09-11T23:23:54.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.827+0000 7fce59ffb640 1 --2- 172.21.15.43:0/1854525902 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fce4c0a2ec0 0x7fce4c0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:23:54.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.828+0000 7fce59ffb640 1 -- 172.21.15.43:0/1854525902 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fce540cf7a0 con 0x7fce64110190 2024-09-11T23:23:54.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.830+0000 7fce68b7a640 1 --2- 172.21.15.43:0/1854525902 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fce4c0a2ec0 0x7fce4c0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:23:54.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.835+0000 7fce68b7a640 1 --2- 172.21.15.43:0/1854525902 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fce4c0a2ec0 0x7fce4c0a5380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fce60005ec0 tx=0x7fce60005df0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:23:54.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:53.841+0000 7fce59ffb640 1 -- 172.21.15.43:0/1854525902 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce5401d060 con 0x7fce64110190 2024-09-11T23:23:54.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:54.232+0000 7fce69b7c640 1 -- 172.21.15.43:0/1854525902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce64117dd0 con 0x7fce64110190 2024-09-11T23:23:54.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:54.234+0000 7fce59ffb640 1 -- 172.21.15.43:0/1854525902 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11899 (secure 0 0 0) 0x7fce64117dd0 con 0x7fce64110190 2024-09-11T23:23:54.755 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:23:54.755 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"btime":"2024-09-11T23:23:19:881806+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:23:19.722053+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:23:54.755 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-09-11T23:23:54.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:54.239+0000 7fce69b7c640 1 -- 172.21.15.43:0/1854525902 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fce4c0a2ec0 msgr2=0x7fce4c0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:54.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:54.239+0000 7fce69b7c640 1 --2- 172.21.15.43:0/1854525902 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fce4c0a2ec0 0x7fce4c0a5380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fce60005ec0 tx=0x7fce60005df0 comp rx=0 tx=0).stop 2024-09-11T23:23:54.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:54.239+0000 7fce69b7c640 1 -- 172.21.15.43:0/1854525902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce64110190 msgr2=0x7fce641168a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:23:54.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:54.239+0000 7fce69b7c640 1 --2- 172.21.15.43:0/1854525902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce64110190 0x7fce641168a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fce5400c970 tx=0x7fce5400ce30 comp rx=0 tx=0).stop 2024-09-11T23:23:54.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:54.241+0000 7fce69b7c640 1 -- 172.21.15.43:0/1854525902 shutdown_connections 2024-09-11T23:23:54.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:54.241+0000 7fce69b7c640 1 --2- 172.21.15.43:0/1854525902 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fce4c0a2ec0 0x7fce4c0a5380 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:54.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:54.241+0000 7fce69b7c640 1 --2- 172.21.15.43:0/1854525902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce6411b140 0x7fce64116de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:54.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:54.241+0000 7fce69b7c640 1 --2- 172.21.15.43:0/1854525902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce64110190 0x7fce641168a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:54.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:54.241+0000 7fce69b7c640 1 --2- 172.21.15.43:0/1854525902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce6406ea30 0x7fce6411a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:23:54.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:54.241+0000 7fce69b7c640 1 -- 172.21.15.43:0/1854525902 >> 172.21.15.43:0/1854525902 conn(0x7fce64073180 msgr2=0x7fce6411b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:23:54.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:54.246+0000 7fce69b7c640 1 -- 172.21.15.43:0/1854525902 shutdown_connections 2024-09-11T23:23:54.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:23:54.246+0000 7fce69b7c640 1 -- 172.21.15.43:0/1854525902 wait complete. 2024-09-11T23:23:55.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:55 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1854525902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:23:55.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:55 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1854525902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:23:55.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:55 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1854525902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:23:56.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:56 smithi090 ceph-mon[35130]: pgmap v65: 97 pgs: 97 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 296 MiB/s wr, 25.26k op/s 2024-09-11T23:23:56.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:56 smithi097 ceph-mon[35562]: pgmap v65: 97 pgs: 97 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 296 MiB/s wr, 25.26k op/s 2024-09-11T23:23:56.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:56 smithi043 ceph-mon[30950]: pgmap v65: 97 pgs: 97 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 296 MiB/s wr, 25.26k op/s 2024-09-11T23:23:58.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:23:58 smithi090 ceph-mon[35130]: pgmap v66: 97 pgs: 97 active+clean; 13 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 319 MiB/s wr, 27.93k op/s 2024-09-11T23:23:58.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:23:58 smithi097 ceph-mon[35562]: pgmap v66: 97 pgs: 97 active+clean; 13 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 319 MiB/s wr, 27.93k op/s 2024-09-11T23:23:58.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:23:58 smithi043 ceph-mon[30950]: pgmap v66: 97 pgs: 97 active+clean; 13 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 319 MiB/s wr, 27.93k op/s 2024-09-11T23:24:00.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:00 smithi090 ceph-mon[35130]: pgmap v67: 97 pgs: 97 active+clean; 13 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 318 MiB/s wr, 34.58k op/s 2024-09-11T23:24:00.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:00 smithi097 ceph-mon[35562]: pgmap v67: 97 pgs: 97 active+clean; 13 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 318 MiB/s wr, 34.58k op/s 2024-09-11T23:24:00.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:00 smithi043 ceph-mon[30950]: pgmap v67: 97 pgs: 97 active+clean; 13 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 318 MiB/s wr, 34.58k op/s 2024-09-11T23:24:02.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:02 smithi097 ceph-mon[35562]: pgmap v68: 97 pgs: 97 active+clean; 13 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 227 MiB/s wr, 29.03k op/s 2024-09-11T23:24:02.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:02 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:24:02.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:02 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:24:02.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:02 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:24:02.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:02 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:24:02.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:02 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:24:02.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:02 smithi043 ceph-mon[30950]: pgmap v68: 97 pgs: 97 active+clean; 13 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 227 MiB/s wr, 29.03k op/s 2024-09-11T23:24:02.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:02 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:24:02.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:02 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:24:02.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:02 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:24:02.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:02 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:24:02.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:02 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:24:02.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:02 smithi090 ceph-mon[35130]: pgmap v68: 97 pgs: 97 active+clean; 13 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 227 MiB/s wr, 29.03k op/s 2024-09-11T23:24:02.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:02 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:24:02.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:02 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:24:02.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:02 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:24:02.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:02 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:24:02.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:02 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:24:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.732+0000 7ff69d7fa640 1 -- 172.21.15.43:0/2855349761 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564dadcc2d70 con 0x564dadddda80 2024-09-11T23:24:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.732+0000 7ff69d7fa640 10 client.25372 handle_command_reply: tid=0 2024-09-11T23:24:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.732+0000 7ff6bedbb740 1 client.25372 shutdown 2024-09-11T23:24:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.732+0000 7ff6bedbb740 1 -- 172.21.15.43:0/2855349761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564dadd5f830 msgr2=0x564dadd6a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.732+0000 7ff6bedbb740 1 --2- 172.21.15.43:0/2855349761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564dadd5f830 0x564dadd6a200 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff6b41a0080 tx=0x7ff6a402b080 comp rx=0 tx=0).stop 2024-09-11T23:24:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.732+0000 7ff6bedbb740 1 -- 172.21.15.43:0/2855349761 shutdown_connections 2024-09-11T23:24:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.732+0000 7ff6bedbb740 1 --2- 172.21.15.43:0/2855349761 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564dadddda80 0x564dadddfea0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.732+0000 7ff6bedbb740 1 --2- 172.21.15.43:0/2855349761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564dadd61cb0 0x564dadd66c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.732+0000 7ff6bedbb740 1 --2- 172.21.15.43:0/2855349761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564dadd5f830 0x564dadd6a200 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.732+0000 7ff6bedbb740 1 --2- 172.21.15.43:0/2855349761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564dadd56af0 0x564dadd69cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.732+0000 7ff6bedbb740 1 -- 172.21.15.43:0/2855349761 >> 172.21.15.43:0/2855349761 conn(0x564dadd53060 msgr2=0x564dadd58550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.732+0000 7ff69d7fa640 0 client.25372 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.732+0000 7ff6bedbb740 1 -- 172.21.15.43:0/2855349761 shutdown_connections 2024-09-11T23:24:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.732+0000 7ff6bedbb740 1 -- 172.21.15.43:0/2855349761 wait complete. 2024-09-11T23:24:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.733+0000 7ff6bedbb740 20 client.25372 trim_cache size 0 max 16384 2024-09-11T23:24:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:24:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:24:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:24:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:24:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.733+0000 7ff6bc023640 1 -- 172.21.15.43:0/4221448807 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff6a00a2f00 msgr2=0x7ff6a00a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.733+0000 7ff6bc023640 1 --2- 172.21.15.43:0/4221448807 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff6a00a2f00 0x7ff6a00a53c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x564dadd543e0 tx=0x7ff6a4004710 comp rx=0 tx=0).stop 2024-09-11T23:24:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.733+0000 7ff6bc023640 1 -- 172.21.15.43:0/4221448807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6b419f450 msgr2=0x7ff6b41a33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.733+0000 7ff6bc023640 1 --2- 172.21.15.43:0/4221448807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6b419f450 0x7ff6b41a33a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00b990 tx=0x7ff6ac00be50 comp rx=0 tx=0).stop 2024-09-11T23:24:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.733+0000 7ff6bc023640 1 -- 172.21.15.43:0/4221448807 shutdown_connections 2024-09-11T23:24:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.733+0000 7ff6bc023640 1 --2- 172.21.15.43:0/4221448807 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff6a00a2f00 0x7ff6a00a53c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.733+0000 7ff6bc023640 1 --2- 172.21.15.43:0/4221448807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6b419f450 0x7ff6b41a33a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.733+0000 7ff6bc023640 1 --2- 172.21.15.43:0/4221448807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6b410e9a0 0x7ff6b419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.733+0000 7ff6bc023640 1 --2- 172.21.15.43:0/4221448807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6b4109460 0x7ff6b419e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.733+0000 7ff6bc023640 1 -- 172.21.15.43:0/4221448807 >> 172.21.15.43:0/4221448807 conn(0x7ff6b406d200 msgr2=0x7ff6b4072730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.733+0000 7ff6bc023640 1 -- 172.21.15.43:0/4221448807 shutdown_connections 2024-09-11T23:24:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:02.734+0000 7ff6bc023640 1 -- 172.21.15.43:0/4221448807 wait complete. 2024-09-11T23:24:03.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:03 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:24:03.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:03 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:24:03.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:03 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:24:03.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:03 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:24:03.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:03 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:24:03.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:03 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:24:03.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:03 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:24:03.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:03 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:24:03.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:03 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:24:04.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:04 smithi097 ceph-mon[35562]: pgmap v69: 97 pgs: 97 active+clean; 13 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 313 MiB/s wr, 37.62k op/s 2024-09-11T23:24:04.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:04 smithi043 ceph-mon[30950]: pgmap v69: 97 pgs: 97 active+clean; 13 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 313 MiB/s wr, 37.62k op/s 2024-09-11T23:24:04.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:04 smithi090 ceph-mon[35130]: pgmap v69: 97 pgs: 97 active+clean; 13 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 313 MiB/s wr, 37.62k op/s 2024-09-11T23:24:04.801 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0' 2024-09-11T23:24:04.802 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0 2024-09-11T23:24:05.345 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:24:06.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:06 smithi097 ceph-mon[35562]: pgmap v70: 97 pgs: 97 active+clean; 13 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 195 MiB/s wr, 21.68k op/s 2024-09-11T23:24:06.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:06 smithi043 ceph-mon[30950]: pgmap v70: 97 pgs: 97 active+clean; 13 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 195 MiB/s wr, 21.68k op/s 2024-09-11T23:24:06.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:06 smithi090 ceph-mon[35130]: pgmap v70: 97 pgs: 97 active+clean; 13 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 195 MiB/s wr, 21.68k op/s 2024-09-11T23:24:07.405 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:24:08.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:08 smithi097 ceph-mon[35562]: pgmap v71: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 204 MiB/s wr, 22.54k op/s 2024-09-11T23:24:08.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:08 smithi043 ceph-mon[30950]: pgmap v71: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 204 MiB/s wr, 22.54k op/s 2024-09-11T23:24:08.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:08 smithi090 ceph-mon[35130]: pgmap v71: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 204 MiB/s wr, 22.54k op/s 2024-09-11T23:24:10.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:10 smithi097 ceph-mon[35562]: pgmap v72: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 272 MiB/s wr, 29.41k op/s 2024-09-11T23:24:10.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:10 smithi043 ceph-mon[30950]: pgmap v72: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 272 MiB/s wr, 29.41k op/s 2024-09-11T23:24:10.521 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:10 smithi090 ceph-mon[35130]: pgmap v72: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 272 MiB/s wr, 29.41k op/s 2024-09-11T23:24:11.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.503+0000 7fcd4ffff640 1 -- 172.21.15.43:0/4145101707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd5011b140 msgr2=0x7fcd5011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:11.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.503+0000 7fcd4ffff640 1 --2- 172.21.15.43:0/4145101707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd5011b140 0x7fcd5011d530 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fcd4801acb0 tx=0x7fcd4803b900 comp rx=0 tx=0).stop 2024-09-11T23:24:11.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.505+0000 7fcd4ffff640 1 -- 172.21.15.43:0/4145101707 shutdown_connections 2024-09-11T23:24:11.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.505+0000 7fcd4ffff640 1 --2- 172.21.15.43:0/4145101707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd5011b140 0x7fcd5011d530 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.505+0000 7fcd4ffff640 1 --2- 172.21.15.43:0/4145101707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd5006ea30 0x7fcd5006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.505+0000 7fcd4ffff640 1 --2- 172.21.15.43:0/4145101707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd50110190 0x7fcd5006e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.505+0000 7fcd4ffff640 1 -- 172.21.15.43:0/4145101707 >> 172.21.15.43:0/4145101707 conn(0x7fcd50073180 msgr2=0x7fcd50073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:11.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.508+0000 7fcd4ffff640 1 -- 172.21.15.43:0/4145101707 shutdown_connections 2024-09-11T23:24:11.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.508+0000 7fcd4ffff640 1 -- 172.21.15.43:0/4145101707 wait complete. 2024-09-11T23:24:11.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.509+0000 7fcd4ffff640 1 Processor -- start 2024-09-11T23:24:11.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.515+0000 7fcd4ffff640 1 -- start start 2024-09-11T23:24:11.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.515+0000 7fcd4ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd5006ea30 0x7fcd5011a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:11.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.515+0000 7fcd4ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd50110190 0x7fcd501169c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:11.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.516+0000 7fcd4ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd5011b140 0x7fcd50116f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:11.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.516+0000 7fcd4ffff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcd50113310 con 0x7fcd50110190 2024-09-11T23:24:11.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.516+0000 7fcd4ffff640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcd50113190 con 0x7fcd5011b140 2024-09-11T23:24:11.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.516+0000 7fcd4ffff640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcd50113490 con 0x7fcd5006ea30 2024-09-11T23:24:11.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.516+0000 7fcd4effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd5006ea30 0x7fcd5011a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:11.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.516+0000 7fcd4f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd5011b140 0x7fcd50116f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:11.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.516+0000 7fcd4f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd5011b140 0x7fcd50116f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:40580/0 (socket says 172.21.15.43:40580) 2024-09-11T23:24:11.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.516+0000 7fcd4f7fe640 1 -- 172.21.15.43:0/1726737141 learned_addr learned my addr 172.21.15.43:0/1726737141 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:11.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.516+0000 7fcd4effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd5006ea30 0x7fcd5011a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:37628/0 (socket says 172.21.15.43:37628) 2024-09-11T23:24:11.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.517+0000 7fcd4e7fc640 1 --2- 172.21.15.43:0/1726737141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd50110190 0x7fcd501169c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:11.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.517+0000 7fcd4f7fe640 1 -- 172.21.15.43:0/1726737141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd5006ea30 msgr2=0x7fcd5011a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:11.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.517+0000 7fcd4f7fe640 1 --2- 172.21.15.43:0/1726737141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd5006ea30 0x7fcd5011a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.517+0000 7fcd4f7fe640 1 -- 172.21.15.43:0/1726737141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd50110190 msgr2=0x7fcd501169c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:11.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.517+0000 7fcd4f7fe640 1 --2- 172.21.15.43:0/1726737141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd50110190 0x7fcd501169c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.517+0000 7fcd4f7fe640 1 -- 172.21.15.43:0/1726737141 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd50117680 con 0x7fcd5011b140 2024-09-11T23:24:11.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.518+0000 7fcd4e7fc640 1 --2- 172.21.15.43:0/1726737141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd50110190 0x7fcd501169c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:24:11.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.526+0000 7fcd4f7fe640 1 --2- 172.21.15.43:0/1726737141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd5011b140 0x7fcd50116f00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcd4801ac00 tx=0x7fcd480405f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:11.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.527+0000 7fcd37fff640 1 -- 172.21.15.43:0/1726737141 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48045070 con 0x7fcd5011b140 2024-09-11T23:24:11.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.527+0000 7fcd37fff640 1 -- 172.21.15.43:0/1726737141 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd48003e50 con 0x7fcd5011b140 2024-09-11T23:24:11.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.528+0000 7fcd37fff640 1 -- 172.21.15.43:0/1726737141 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd480042a0 con 0x7fcd5011b140 2024-09-11T23:24:11.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.528+0000 7fcd4ffff640 1 -- 172.21.15.43:0/1726737141 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd501aaf60 con 0x7fcd5011b140 2024-09-11T23:24:11.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.528+0000 7fcd4ffff640 1 -- 172.21.15.43:0/1726737141 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd501ab300 con 0x7fcd5011b140 2024-09-11T23:24:11.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.535+0000 7fcd5790d740 1 Processor -- start 2024-09-11T23:24:11.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.540+0000 7fcd37fff640 1 -- 172.21.15.43:0/1726737141 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fcd48048020 con 0x7fcd5011b140 2024-09-11T23:24:11.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.541+0000 7fcd37fff640 1 --2- 172.21.15.43:0/1726737141 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fcd380a2ce0 0x7fcd380a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:11.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.542+0000 7fcd37fff640 1 -- 172.21.15.43:0/1726737141 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fcd48100c70 con 0x7fcd5011b140 2024-09-11T23:24:11.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.542+0000 7fcd4e7fc640 1 --2- 172.21.15.43:0/1726737141 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fcd380a2ce0 0x7fcd380a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:11.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.543+0000 7fcd5790d740 1 -- start start 2024-09-11T23:24:11.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.543+0000 7fcd5790d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f975900 0x562a4f975ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:11.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.543+0000 7fcd5790d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a4f976220 0x562a4f9805a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:11.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.543+0000 7fcd5790d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f980ae0 0x562a4f982ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:11.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.543+0000 7fcd5790d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562a4f8db2a0 con 0x562a4f980ae0 2024-09-11T23:24:11.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.543+0000 7fcd5790d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562a4f882b10 con 0x562a4f975900 2024-09-11T23:24:11.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.543+0000 7fcd5790d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562a4f8d7260 con 0x562a4f976220 2024-09-11T23:24:11.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.544+0000 7fcd4effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f975900 0x562a4f975ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:11.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.545+0000 7fcd4effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f975900 0x562a4f975ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:40582/0 (socket says 172.21.15.43:40582) 2024-09-11T23:24:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.545+0000 7fcd4effd640 1 -- 172.21.15.43:0/3606253538 learned_addr learned my addr 172.21.15.43:0/3606253538 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.545+0000 7fcd4f7fe640 1 --2- 172.21.15.43:0/3606253538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f980ae0 0x562a4f982ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.545+0000 7fcd4e7fc640 1 --2- 172.21.15.43:0/3606253538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a4f976220 0x562a4f9805a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.545+0000 7fcd4f7fe640 1 -- 172.21.15.43:0/3606253538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a4f976220 msgr2=0x562a4f9805a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.545+0000 7fcd4f7fe640 1 --2- 172.21.15.43:0/3606253538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a4f976220 0x562a4f9805a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.545+0000 7fcd4f7fe640 1 -- 172.21.15.43:0/3606253538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f975900 msgr2=0x562a4f975ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.545+0000 7fcd4f7fe640 1 --2- 172.21.15.43:0/3606253538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f975900 0x562a4f975ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.545+0000 7fcd4f7fe640 1 -- 172.21.15.43:0/3606253538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a4f64cb90 con 0x562a4f980ae0 2024-09-11T23:24:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.546+0000 7fcd4f7fe640 1 --2- 172.21.15.43:0/3606253538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f980ae0 0x562a4f982ed0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcd480fe470 tx=0x7fcd480cd130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144f5047f502634c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.546+0000 7fcd357fa640 1 -- 172.21.15.43:0/3606253538 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd480e6070 con 0x562a4f980ae0 2024-09-11T23:24:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.546+0000 7fcd357fa640 1 -- 172.21.15.43:0/3606253538 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd480e6210 con 0x562a4f980ae0 2024-09-11T23:24:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.547+0000 7fcd5790d740 1 -- 172.21.15.43:0/3606253538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f980ae0 msgr2=0x562a4f982ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.547+0000 7fcd5790d740 1 --2- 172.21.15.43:0/3606253538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f980ae0 0x562a4f982ed0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcd480fe470 tx=0x7fcd480cd130 comp rx=0 tx=0).stop 2024-09-11T23:24:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.547+0000 7fcd5790d740 1 -- 172.21.15.43:0/3606253538 shutdown_connections 2024-09-11T23:24:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.547+0000 7fcd5790d740 1 --2- 172.21.15.43:0/3606253538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f980ae0 0x562a4f982ed0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.547+0000 7fcd5790d740 1 --2- 172.21.15.43:0/3606253538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a4f976220 0x562a4f9805a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.547+0000 7fcd5790d740 1 --2- 172.21.15.43:0/3606253538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f975900 0x562a4f975ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.547+0000 7fcd5790d740 1 -- 172.21.15.43:0/3606253538 >> 172.21.15.43:0/3606253538 conn(0x562a4f971e90 msgr2=0x562a4f972270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.548+0000 7fcd4e7fc640 1 --2- 172.21.15.43:0/1726737141 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fcd380a2ce0 0x7fcd380a51a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcd50118180 tx=0x7fcd44008040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.550+0000 7fcd5790d740 1 -- 172.21.15.43:0/3606253538 shutdown_connections 2024-09-11T23:24:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.550+0000 7fcd5790d740 1 -- 172.21.15.43:0/3606253538 wait complete. 2024-09-11T23:24:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.550+0000 7fcd5790d740 1 -- start start 2024-09-11T23:24:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.550+0000 7fcd5790d740 1 Processor -- start 2024-09-11T23:24:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.551+0000 7fcd5790d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a4f975900 0x562a4f980290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.551+0000 7fcd5790d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f982fe0 0x562a4f9807d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.551+0000 7fcd5790d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f980e70 0x562a4f98c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.551+0000 7fcd5790d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562a4f8db2a0 con 0x562a4f982fe0 2024-09-11T23:24:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.551+0000 7fcd5790d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562a4f8636e0 con 0x562a4f980e70 2024-09-11T23:24:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.551+0000 7fcd5790d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562a4f983470 con 0x562a4f975900 2024-09-11T23:24:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.551+0000 7fcd5790d740 10 client.0 _ll_register_callbacks cb 0x562a4f887a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.552+0000 7fcd4f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f980e70 0x562a4f98c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.552+0000 7fcd4f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f980e70 0x562a4f98c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:40584/0 (socket says 172.21.15.43:40584) 2024-09-11T23:24:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.552+0000 7fcd4f7fe640 1 -- 172.21.15.43:0/976921178 learned_addr learned my addr 172.21.15.43:0/976921178 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.552+0000 7fcd4e7fc640 1 --2- 172.21.15.43:0/976921178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f982fe0 0x562a4f9807d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.553+0000 7fcd4e7fc640 1 -- 172.21.15.43:0/976921178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a4f975900 msgr2=0x562a4f980290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:24:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.553+0000 7fcd4e7fc640 1 --2- 172.21.15.43:0/976921178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a4f975900 0x562a4f980290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.553+0000 7fcd4e7fc640 1 -- 172.21.15.43:0/976921178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f980e70 msgr2=0x562a4f98c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.553+0000 7fcd4e7fc640 1 --2- 172.21.15.43:0/976921178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f980e70 0x562a4f98c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.553+0000 7fcd4e7fc640 1 -- 172.21.15.43:0/976921178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a4f98cf90 con 0x562a4f982fe0 2024-09-11T23:24:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.555+0000 7fcd4e7fc640 1 --2- 172.21.15.43:0/976921178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f982fe0 0x562a4f9807d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x562a4f972df0 tx=0x7fcd44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea4d1b072f5c5db server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:11.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.560+0000 7fcd32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:24:11.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.560+0000 7fcd32ff5640 1 -- 172.21.15.43:0/976921178 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4401f630 con 0x562a4f982fe0 2024-09-11T23:24:11.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.560+0000 7fcd32ff5640 1 -- 172.21.15.43:0/976921178 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fcd4401f7d0 con 0x562a4f982fe0 2024-09-11T23:24:11.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.560+0000 7fcd5790d740 1 -- 172.21.15.43:0/976921178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562a4f8e89d0 con 0x562a4f982fe0 2024-09-11T23:24:11.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.560+0000 7fcd32ff5640 1 -- 172.21.15.43:0/976921178 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcd4401f9d0 con 0x562a4f982fe0 2024-09-11T23:24:11.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.560+0000 7fcd5790d740 10 client.15855 fetch_fsmap learned FSMap version 26 2024-09-11T23:24:11.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.560+0000 7fcd5790d740 1 -- 172.21.15.43:0/976921178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562a4f98fa10 con 0x562a4f982fe0 2024-09-11T23:24:11.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.561+0000 7fcd32ff5640 1 -- 172.21.15.43:0/976921178 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562a4f8e89d0 con 0x562a4f982fe0 2024-09-11T23:24:11.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.561+0000 7fcd32ff5640 1 -- 172.21.15.43:0/976921178 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcd44027030 con 0x562a4f982fe0 2024-09-11T23:24:11.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.561+0000 7fcd5790d740 10 client.15855 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:24:11.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.561+0000 7fcd5790d740 10 client.15855 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:24:11.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.561+0000 7fcd5790d740 20 client.15855 populate_metadata read hostname 'smithi043' 2024-09-11T23:24:11.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.562+0000 7fcd5790d740 1 --2- 172.21.15.43:0/976921178 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562a4f991390 0x562a4f993850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:11.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.562+0000 7fcd5790d740 4 client.15855 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:24:11.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.562+0000 7fcd5790d740 1 -- 172.21.15.43:0/976921178 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a4f8e2d20 con 0x562a4f991390 2024-09-11T23:24:11.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.562+0000 7fcd4effd640 1 --2- 172.21.15.43:0/976921178 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562a4f991390 0x562a4f993850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:11.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.569+0000 7fcd4effd640 1 --2- 172.21.15.43:0/976921178 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562a4f991390 0x562a4f993850 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=494d2457a9e9ef1 server_cookie=2442d7b62fb5251a in_seq=0 out_seq=0 2024-09-11T23:24:11.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.569+0000 7fcd32ff5640 10 client.15855 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:11.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.572+0000 7fcd32ff5640 1 -- 172.21.15.43:0/976921178 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a4f8e2d20 con 0x562a4f991390 2024-09-11T23:24:11.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.572+0000 7fcd32ff5640 10 client.15855 handle_command_reply: tid=0 2024-09-11T23:24:11.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.573+0000 7fcd5790d740 1 client.15855 shutdown 2024-09-11T23:24:11.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.575+0000 7fcd5790d740 1 -- 172.21.15.43:0/976921178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f982fe0 msgr2=0x562a4f9807d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:11.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.575+0000 7fcd5790d740 1 --2- 172.21.15.43:0/976921178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f982fe0 0x562a4f9807d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x562a4f972df0 tx=0x7fcd44004750 comp rx=0 tx=0).stop 2024-09-11T23:24:11.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.577+0000 7fcd5790d740 1 -- 172.21.15.43:0/976921178 shutdown_connections 2024-09-11T23:24:11.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.577+0000 7fcd5790d740 1 --2- 172.21.15.43:0/976921178 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562a4f991390 0x562a4f993850 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.577+0000 7fcd5790d740 1 --2- 172.21.15.43:0/976921178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f980e70 0x562a4f98c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.577+0000 7fcd5790d740 1 --2- 172.21.15.43:0/976921178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f982fe0 0x562a4f9807d0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.577+0000 7fcd5790d740 1 --2- 172.21.15.43:0/976921178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a4f975900 0x562a4f980290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.577+0000 7fcd32ff5640 0 client.15855 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.577+0000 7fcd5790d740 1 -- 172.21.15.43:0/976921178 >> 172.21.15.43:0/976921178 conn(0x562a4f971e90 msgr2=0x562a4f977b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.582+0000 7fcd5790d740 1 -- 172.21.15.43:0/976921178 shutdown_connections 2024-09-11T23:24:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.582+0000 7fcd5790d740 1 -- 172.21.15.43:0/976921178 wait complete. 2024-09-11T23:24:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.582+0000 7fcd5790d740 20 client.15855 trim_cache size 0 max 16384 2024-09-11T23:24:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.611+0000 7fcd5790d740 1 Processor -- start 2024-09-11T23:24:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.611+0000 7fcd5790d740 1 -- start start 2024-09-11T23:24:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.611+0000 7fcd5790d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a4f975900 0x562a4f97b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.611+0000 7fcd5790d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f980e70 0x562a4f98c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.611+0000 7fcd5790d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f982fe0 0x562a4f982200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.611+0000 7fcd5790d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcd501b65a0 con 0x562a4f980e70 2024-09-11T23:24:11.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.611+0000 7fcd5790d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcd501b3e70 con 0x562a4f982fe0 2024-09-11T23:24:11.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.611+0000 7fcd5790d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcd501b1cc0 con 0x562a4f975900 2024-09-11T23:24:11.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.611+0000 7fcd4e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f980e70 0x562a4f98c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:11.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.611+0000 7fcd4e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f980e70 0x562a4f98c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59290/0 (socket says 172.21.15.43:59290) 2024-09-11T23:24:11.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.611+0000 7fcd4e7fc640 1 -- 172.21.15.43:0/1897015969 learned_addr learned my addr 172.21.15.43:0/1897015969 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:11.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.611+0000 7fcd4effd640 1 --2- 172.21.15.43:0/1897015969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a4f975900 0x562a4f97b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:11.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.611+0000 7fcd4f7fe640 1 --2- 172.21.15.43:0/1897015969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f982fe0 0x562a4f982200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:11.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.611+0000 7fcd4e7fc640 1 -- 172.21.15.43:0/1897015969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a4f975900 msgr2=0x562a4f97b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:11.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.611+0000 7fcd4e7fc640 1 --2- 172.21.15.43:0/1897015969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a4f975900 0x562a4f97b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.611+0000 7fcd4e7fc640 1 -- 172.21.15.43:0/1897015969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f982fe0 msgr2=0x562a4f982200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:11.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.611+0000 7fcd4e7fc640 1 --2- 172.21.15.43:0/1897015969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f982fe0 0x562a4f982200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.611+0000 7fcd4e7fc640 1 -- 172.21.15.43:0/1897015969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a4f97abe0 con 0x562a4f980e70 2024-09-11T23:24:11.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.611+0000 7fcd4f7fe640 1 --2- 172.21.15.43:0/1897015969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f982fe0 0x562a4f982200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:24:11.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.612+0000 7fcd4e7fc640 1 --2- 172.21.15.43:0/1897015969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f980e70 0x562a4f98c5a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcd44028f00 tx=0x7fcd44020d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955e83040a5f4fea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.612+0000 7fcd34ff9640 1 -- 172.21.15.43:0/1897015969 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44036070 con 0x562a4f980e70 2024-09-11T23:24:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.612+0000 7fcd34ff9640 1 -- 172.21.15.43:0/1897015969 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd4401b040 con 0x562a4f980e70 2024-09-11T23:24:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.612+0000 7fcd34ff9640 1 -- 172.21.15.43:0/1897015969 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44013800 con 0x562a4f980e70 2024-09-11T23:24:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.613+0000 7fcd5790d740 1 -- 172.21.15.43:0/1897015969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f980e70 msgr2=0x562a4f98c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.613+0000 7fcd5790d740 1 --2- 172.21.15.43:0/1897015969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f980e70 0x562a4f98c5a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcd44028f00 tx=0x7fcd44020d80 comp rx=0 tx=0).stop 2024-09-11T23:24:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.613+0000 7fcd5790d740 1 -- 172.21.15.43:0/1897015969 shutdown_connections 2024-09-11T23:24:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.613+0000 7fcd5790d740 1 --2- 172.21.15.43:0/1897015969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f982fe0 0x562a4f982200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.613+0000 7fcd5790d740 1 --2- 172.21.15.43:0/1897015969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f980e70 0x562a4f98c5a0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.613+0000 7fcd5790d740 1 --2- 172.21.15.43:0/1897015969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a4f975900 0x562a4f97b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.613+0000 7fcd5790d740 1 -- 172.21.15.43:0/1897015969 >> 172.21.15.43:0/1897015969 conn(0x562a4f971e90 msgr2=0x562a4f981af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.613+0000 7fcd5790d740 1 -- 172.21.15.43:0/1897015969 shutdown_connections 2024-09-11T23:24:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.613+0000 7fcd5790d740 1 -- 172.21.15.43:0/1897015969 wait complete. 2024-09-11T23:24:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.613+0000 7fcd5790d740 1 -- start start 2024-09-11T23:24:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.613+0000 7fcd5790d740 1 Processor -- start 2024-09-11T23:24:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.613+0000 7fcd5790d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f975900 0x562a4f988cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.613+0000 7fcd5790d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a4f980e70 0x562a4f97ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.613+0000 7fcd5790d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f982fe0 0x562a4f97d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.613+0000 7fcd5790d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcd501b2330 con 0x562a4f982fe0 2024-09-11T23:24:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.613+0000 7fcd5790d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcd501b37a0 con 0x562a4f975900 2024-09-11T23:24:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.613+0000 7fcd5790d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcd501b2a00 con 0x562a4f980e70 2024-09-11T23:24:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.614+0000 7fcd5790d740 10 client.0 _ll_register_callbacks cb 0x7fcd4400cb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.614+0000 7fcd4f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f982fe0 0x562a4f97d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.614+0000 7fcd4f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f982fe0 0x562a4f97d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59300/0 (socket says 172.21.15.43:59300) 2024-09-11T23:24:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.614+0000 7fcd4f7fe640 1 -- 172.21.15.43:0/3009759723 learned_addr learned my addr 172.21.15.43:0/3009759723 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.614+0000 7fcd4effd640 1 --2- 172.21.15.43:0/3009759723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f975900 0x562a4f988cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.614+0000 7fcd4e7fc640 1 --2- 172.21.15.43:0/3009759723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a4f980e70 0x562a4f97ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.614+0000 7fcd4f7fe640 1 -- 172.21.15.43:0/3009759723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a4f980e70 msgr2=0x562a4f97ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:11.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.614+0000 7fcd4f7fe640 1 --2- 172.21.15.43:0/3009759723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a4f980e70 0x562a4f97ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.614+0000 7fcd4f7fe640 1 -- 172.21.15.43:0/3009759723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f975900 msgr2=0x562a4f988cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:11.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.614+0000 7fcd4f7fe640 1 --2- 172.21.15.43:0/3009759723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f975900 0x562a4f988cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:11.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.614+0000 7fcd4f7fe640 1 -- 172.21.15.43:0/3009759723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a4f97da30 con 0x562a4f982fe0 2024-09-11T23:24:11.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.614+0000 7fcd4f7fe640 1 --2- 172.21.15.43:0/3009759723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f982fe0 0x562a4f97d3c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcd48005e30 tx=0x7fcd480c8ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2855d453980d9a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:11.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.614+0000 7fcd32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:24:11.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.614+0000 7fcd32ff5640 1 -- 172.21.15.43:0/3009759723 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd480eb060 con 0x562a4f982fe0 2024-09-11T23:24:11.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.614+0000 7fcd32ff5640 1 -- 172.21.15.43:0/3009759723 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fcd480d1170 con 0x562a4f982fe0 2024-09-11T23:24:11.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.614+0000 7fcd5790d740 1 -- 172.21.15.43:0/3009759723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd4401b040 con 0x562a4f982fe0 2024-09-11T23:24:11.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.614+0000 7fcd32ff5640 1 -- 172.21.15.43:0/3009759723 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcd480e6040 con 0x562a4f982fe0 2024-09-11T23:24:11.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.614+0000 7fcd5790d740 10 client.15861 fetch_fsmap learned FSMap version 26 2024-09-11T23:24:11.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.614+0000 7fcd5790d740 1 -- 172.21.15.43:0/3009759723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562a4f9fbf80 con 0x562a4f982fe0 2024-09-11T23:24:11.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.615+0000 7fcd32ff5640 1 -- 172.21.15.43:0/3009759723 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd4401b040 con 0x562a4f982fe0 2024-09-11T23:24:11.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.615+0000 7fcd32ff5640 1 -- 172.21.15.43:0/3009759723 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcd480d5a20 con 0x562a4f982fe0 2024-09-11T23:24:11.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.615+0000 7fcd5790d740 10 client.15861 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:24:11.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.615+0000 7fcd5790d740 10 client.15861 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:24:11.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.615+0000 7fcd5790d740 20 client.15861 populate_metadata read hostname 'smithi043' 2024-09-11T23:24:11.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.615+0000 7fcd5790d740 1 --2- 172.21.15.43:0/3009759723 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562a4f9fc670 0x562a4f9fea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:11.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.615+0000 7fcd5790d740 4 client.15861 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}] 2024-09-11T23:24:11.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.615+0000 7fcd5790d740 1 -- 172.21.15.43:0/3009759723 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}) -- 0x562a4f85eec0 con 0x562a4f9fc670 2024-09-11T23:24:11.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.615+0000 7fcd4effd640 1 --2- 172.21.15.43:0/3009759723 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562a4f9fc670 0x562a4f9fea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:11.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.616+0000 7fcd4effd640 1 --2- 172.21.15.43:0/3009759723 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562a4f9fc670 0x562a4f9fea90 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65f8af7abbb019c0 server_cookie=54cef86d6cf535be in_seq=0 out_seq=0 2024-09-11T23:24:11.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:10.616+0000 7fcd32ff5640 10 client.15861 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:11.988 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:24:12.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:12 smithi043 ceph-mon[30950]: pgmap v73: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 186 MiB/s wr, 18.98k op/s 2024-09-11T23:24:12.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:12 smithi090 ceph-mon[35130]: pgmap v73: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 186 MiB/s wr, 18.98k op/s 2024-09-11T23:24:12.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:12 smithi097 ceph-mon[35562]: pgmap v73: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 186 MiB/s wr, 18.98k op/s 2024-09-11T23:24:14.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:14 smithi043 ceph-mon[30950]: pgmap v74: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 258 MiB/s wr, 28.32k op/s 2024-09-11T23:24:14.321 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:14 smithi090 ceph-mon[35130]: pgmap v74: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 258 MiB/s wr, 28.32k op/s 2024-09-11T23:24:14.353 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:14 smithi097 ceph-mon[35562]: pgmap v74: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 258 MiB/s wr, 28.32k op/s 2024-09-11T23:24:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.898+0000 7f9ef5166640 1 -- 172.21.15.43:0/1174055079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ef01091f0 msgr2=0x7f9ef0110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.898+0000 7f9ef5166640 1 --2- 172.21.15.43:0/1174055079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ef01091f0 0x7f9ef0110250 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009900 tx=0x7f9ee002a420 comp rx=0 tx=0).stop 2024-09-11T23:24:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.899+0000 7f9ef5166640 1 -- 172.21.15.43:0/1174055079 shutdown_connections 2024-09-11T23:24:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.899+0000 7f9ef5166640 1 --2- 172.21.15.43:0/1174055079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ef011b140 0x7f9ef011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.899+0000 7f9ef5166640 1 --2- 172.21.15.43:0/1174055079 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ef01095d0 0x7f9ef0109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.899+0000 7f9ef5166640 1 --2- 172.21.15.43:0/1174055079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ef01091f0 0x7f9ef0110250 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.899+0000 7f9ef5166640 1 -- 172.21.15.43:0/1174055079 >> 172.21.15.43:0/1174055079 conn(0x7f9ef006cdf0 msgr2=0x7f9ef006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.906+0000 7f9ef5166640 1 -- 172.21.15.43:0/1174055079 shutdown_connections 2024-09-11T23:24:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.907+0000 7f9ef5166640 1 -- 172.21.15.43:0/1174055079 wait complete. 2024-09-11T23:24:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.908+0000 7f9ef5166640 1 Processor -- start 2024-09-11T23:24:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.908+0000 7f9ef5166640 1 -- start start 2024-09-11T23:24:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.908+0000 7f9ef5166640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ef01091f0 0x7f9ef01a7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.908+0000 7f9ef5166640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ef01095d0 0x7f9ef01a7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.908+0000 7f9ef5166640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ef011b140 0x7f9ef01abad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.908+0000 7f9ef5166640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef0120d50 con 0x7f9ef011b140 2024-09-11T23:24:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.908+0000 7f9ef5166640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef0120bd0 con 0x7f9ef01095d0 2024-09-11T23:24:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.908+0000 7f9ef5166640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef0120ed0 con 0x7f9ef01091f0 2024-09-11T23:24:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.909+0000 7f9eeed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ef01091f0 0x7f9ef01a7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.909+0000 7f9eeed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ef01091f0 0x7f9ef01a7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:37676/0 (socket says 172.21.15.43:37676) 2024-09-11T23:24:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.909+0000 7f9eeed9d640 1 -- 172.21.15.43:0/3144954541 learned_addr learned my addr 172.21.15.43:0/3144954541 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.910+0000 7f9eef59e640 1 --2- 172.21.15.43:0/3144954541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ef011b140 0x7f9ef01abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.910+0000 7f9eeed9d640 1 -- 172.21.15.43:0/3144954541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ef01095d0 msgr2=0x7f9ef01a7750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T23:24:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.910+0000 7f9eeed9d640 1 --2- 172.21.15.43:0/3144954541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ef01095d0 0x7f9ef01a7750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.910+0000 7f9eeed9d640 1 -- 172.21.15.43:0/3144954541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ef011b140 msgr2=0x7f9ef01abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.910+0000 7f9eeed9d640 1 --2- 172.21.15.43:0/3144954541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ef011b140 0x7f9ef01abad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.910+0000 7f9eeed9d640 1 -- 172.21.15.43:0/3144954541 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef01ac200 con 0x7f9ef01091f0 2024-09-11T23:24:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.911+0000 7f9eeed9d640 1 --2- 172.21.15.43:0/3144954541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ef01091f0 0x7f9ef01a7210 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ee002a990 tx=0x7f9ee0003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.912+0000 7f9ed7fff640 1 -- 172.21.15.43:0/3144954541 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee003f070 con 0x7f9ef01091f0 2024-09-11T23:24:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.913+0000 7f9ed7fff640 1 -- 172.21.15.43:0/3144954541 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee0031c20 con 0x7f9ef01091f0 2024-09-11T23:24:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.913+0000 7f9ed7fff640 1 -- 172.21.15.43:0/3144954541 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee003f070 con 0x7f9ef01091f0 2024-09-11T23:24:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.914+0000 7f9ef5166640 1 -- 172.21.15.43:0/3144954541 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef01ac490 con 0x7f9ef01091f0 2024-09-11T23:24:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.914+0000 7f9ef5166640 1 -- 172.21.15.43:0/3144954541 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef01b3d80 con 0x7f9ef01091f0 2024-09-11T23:24:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.919+0000 7f9ed7fff640 1 -- 172.21.15.43:0/3144954541 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9ee0042070 con 0x7f9ef01091f0 2024-09-11T23:24:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.920+0000 7f9ed7fff640 1 --2- 172.21.15.43:0/3144954541 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9ed80a3030 0x7f9ed80a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.921+0000 7f9ed7fff640 1 -- 172.21.15.43:0/3144954541 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f9ee00ef8d0 con 0x7f9ef01091f0 2024-09-11T23:24:15.925 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.922+0000 7f9eee59c640 1 --2- 172.21.15.43:0/3144954541 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9ed80a3030 0x7f9ed80a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.927+0000 7f9ef7fe0740 1 Processor -- start 2024-09-11T23:24:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.927+0000 7f9eee59c640 1 --2- 172.21.15.43:0/3144954541 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9ed80a3030 0x7f9ed80a54f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9ef011d630 tx=0x7f9ee4006bd0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.930+0000 7f9ef7fe0740 1 -- start start 2024-09-11T23:24:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.930+0000 7f9ef7fe0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de344af0 0x55c2de344ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.930+0000 7f9ef7fe0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de345410 0x55c2de34f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.930+0000 7f9ef7fe0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de34fcb0 0x55c2de3520a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.930+0000 7f9ef7fe0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c2de2a92f0 con 0x55c2de34fcb0 2024-09-11T23:24:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.930+0000 7f9ef7fe0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c2de251270 con 0x55c2de345410 2024-09-11T23:24:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.930+0000 7f9ef7fe0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c2de2a52b0 con 0x55c2de344af0 2024-09-11T23:24:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.930+0000 7f9eeed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de344af0 0x55c2de344ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.930+0000 7f9eeed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de344af0 0x55c2de344ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:37690/0 (socket says 172.21.15.43:37690) 2024-09-11T23:24:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.930+0000 7f9eeed9d640 1 -- 172.21.15.43:0/3531805232 learned_addr learned my addr 172.21.15.43:0/3531805232 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.931+0000 7f9eee59c640 1 --2- 172.21.15.43:0/3531805232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de345410 0x55c2de34f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.931+0000 7f9eeed9d640 1 -- 172.21.15.43:0/3531805232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de345410 msgr2=0x55c2de34f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.931+0000 7f9eeed9d640 1 --2- 172.21.15.43:0/3531805232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de345410 0x55c2de34f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.931+0000 7f9eeed9d640 1 -- 172.21.15.43:0/3531805232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de34fcb0 msgr2=0x55c2de3520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.931+0000 7f9eeed9d640 1 --2- 172.21.15.43:0/3531805232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de34fcb0 0x55c2de3520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:15.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.931+0000 7f9eeed9d640 1 -- 172.21.15.43:0/3531805232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2de250340 con 0x55c2de344af0 2024-09-11T23:24:15.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.931+0000 7f9eeed9d640 1 --2- 172.21.15.43:0/3531805232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de344af0 0x55c2de344ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00c44d0 tx=0x7f9ee00bc550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc7dcd7d5cc891 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:15.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.932+0000 7f9ed57fa640 1 -- 172.21.15.43:0/3531805232 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0049070 con 0x55c2de344af0 2024-09-11T23:24:15.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.933+0000 7f9ed57fa640 1 -- 172.21.15.43:0/3531805232 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee00c4a00 con 0x55c2de344af0 2024-09-11T23:24:15.933 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.933+0000 7f9ed57fa640 1 -- 172.21.15.43:0/3531805232 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee00ed240 con 0x55c2de344af0 2024-09-11T23:24:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.935+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/3531805232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de344af0 msgr2=0x55c2de344ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.935+0000 7f9ef7fe0740 1 --2- 172.21.15.43:0/3531805232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de344af0 0x55c2de344ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00c44d0 tx=0x7f9ee00bc550 comp rx=0 tx=0).stop 2024-09-11T23:24:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.938+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/3531805232 shutdown_connections 2024-09-11T23:24:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.938+0000 7f9ef7fe0740 1 --2- 172.21.15.43:0/3531805232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de34fcb0 0x55c2de3520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.938+0000 7f9ef7fe0740 1 --2- 172.21.15.43:0/3531805232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de345410 0x55c2de34f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.938+0000 7f9ef7fe0740 1 --2- 172.21.15.43:0/3531805232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de344af0 0x55c2de344ed0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.938+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/3531805232 >> 172.21.15.43:0/3531805232 conn(0x55c2de341060 msgr2=0x55c2de341440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.943+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/3531805232 shutdown_connections 2024-09-11T23:24:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.943+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/3531805232 wait complete. 2024-09-11T23:24:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.943+0000 7f9ef7fe0740 1 -- start start 2024-09-11T23:24:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.944+0000 7f9ef7fe0740 1 Processor -- start 2024-09-11T23:24:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.944+0000 7f9ef7fe0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de344af0 0x55c2de34cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.945+0000 7f9ef7fe0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de345410 0x55c2de34d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.945+0000 7f9eeed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de344af0 0x55c2de34cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.945+0000 7f9eeed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de344af0 0x55c2de34cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:37696/0 (socket says 172.21.15.43:37696) 2024-09-11T23:24:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.945+0000 7f9eeed9d640 1 -- 172.21.15.43:0/4126141258 learned_addr learned my addr 172.21.15.43:0/4126141258 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.945+0000 7f9eee59c640 1 --2- 172.21.15.43:0/4126141258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de345410 0x55c2de34d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.945+0000 7f9ef7fe0740 1 --2- 172.21.15.43:0/4126141258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de34fcb0 0x55c2de34d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.945+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/4126141258 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c2de352640 con 0x55c2de345410 2024-09-11T23:24:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.945+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/4126141258 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c2de232290 con 0x55c2de34fcb0 2024-09-11T23:24:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.946+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/4126141258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c2de3527c0 con 0x55c2de344af0 2024-09-11T23:24:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.946+0000 7f9eef59e640 1 --2- 172.21.15.43:0/4126141258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de34fcb0 0x55c2de34d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.946+0000 7f9ef7fe0740 10 client.0 _ll_register_callbacks cb 0x55c2de263140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.947+0000 7f9eef59e640 1 -- 172.21.15.43:0/4126141258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de344af0 msgr2=0x55c2de34cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.947+0000 7f9eef59e640 1 --2- 172.21.15.43:0/4126141258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de344af0 0x55c2de34cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.947+0000 7f9eef59e640 1 -- 172.21.15.43:0/4126141258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de345410 msgr2=0x55c2de34d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.947+0000 7f9eef59e640 1 --2- 172.21.15.43:0/4126141258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de345410 0x55c2de34d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.947+0000 7f9eef59e640 1 -- 172.21.15.43:0/4126141258 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2ddfbc210 con 0x55c2de34fcb0 2024-09-11T23:24:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.949+0000 7f9eef59e640 1 --2- 172.21.15.43:0/4126141258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de34fcb0 0x55c2de34d680 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55c2de3423e0 tx=0x7f9ee800fe40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a51422d305ac64d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.949+0000 7f9ed2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:24:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.949+0000 7f9ed2ff5640 1 -- 172.21.15.43:0/4126141258 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8004d20 con 0x55c2de34fcb0 2024-09-11T23:24:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.950+0000 7f9ed2ff5640 1 -- 172.21.15.43:0/4126141258 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f9ee800da80 con 0x55c2de34fcb0 2024-09-11T23:24:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.951+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/4126141258 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ee00c4a00 con 0x55c2de34fcb0 2024-09-11T23:24:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.951+0000 7f9ed2ff5640 1 -- 172.21.15.43:0/4126141258 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9ee800dc80 con 0x55c2de34fcb0 2024-09-11T23:24:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.952+0000 7f9ef7fe0740 10 client.25409 fetch_fsmap learned FSMap version 26 2024-09-11T23:24:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.952+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/4126141258 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c2de088620 con 0x55c2de34fcb0 2024-09-11T23:24:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.956+0000 7f9ed2ff5640 1 -- 172.21.15.43:0/4126141258 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee8003e50 con 0x55c2de34fcb0 2024-09-11T23:24:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.956+0000 7f9ed2ff5640 1 -- 172.21.15.43:0/4126141258 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9ee8004690 con 0x55c2de34fcb0 2024-09-11T23:24:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.957+0000 7f9ef7fe0740 10 client.25409 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:24:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.957+0000 7f9ef7fe0740 10 client.25409 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:24:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.957+0000 7f9ef7fe0740 20 client.25409 populate_metadata read hostname 'smithi043' 2024-09-11T23:24:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.957+0000 7f9ef7fe0740 1 --2- 172.21.15.43:0/4126141258 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c2de35faa0 0x55c2de361f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.957+0000 7f9ef7fe0740 4 client.25409 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:24:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.957+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/4126141258 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9ee00c4a00 con 0x55c2de35faa0 2024-09-11T23:24:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.959+0000 7f9eeed9d640 1 --2- 172.21.15.43:0/4126141258 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c2de35faa0 0x55c2de361f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.961+0000 7f9eeed9d640 1 --2- 172.21.15.43:0/4126141258 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c2de35faa0 0x55c2de361f00 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e98dd44216a0e2c8 server_cookie=2f9b0f451ba36d13 in_seq=0 out_seq=0 2024-09-11T23:24:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.961+0000 7f9ed2ff5640 10 client.25409 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.962+0000 7f9ed2ff5640 1 -- 172.21.15.43:0/4126141258 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9ee00c4a00 con 0x55c2de35faa0 2024-09-11T23:24:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.963+0000 7f9ed2ff5640 10 client.25409 handle_command_reply: tid=0 2024-09-11T23:24:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.963+0000 7f9ef7fe0740 1 client.25409 shutdown 2024-09-11T23:24:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.963+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/4126141258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de34fcb0 msgr2=0x55c2de34d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.963+0000 7f9ef7fe0740 1 --2- 172.21.15.43:0/4126141258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de34fcb0 0x55c2de34d680 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55c2de3423e0 tx=0x7f9ee800fe40 comp rx=0 tx=0).stop 2024-09-11T23:24:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.964+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/4126141258 shutdown_connections 2024-09-11T23:24:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.964+0000 7f9ef7fe0740 1 --2- 172.21.15.43:0/4126141258 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c2de35faa0 0x55c2de361f00 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.964+0000 7f9ef7fe0740 1 --2- 172.21.15.43:0/4126141258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de34fcb0 0x55c2de34d680 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.964+0000 7f9ef7fe0740 1 --2- 172.21.15.43:0/4126141258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de345410 0x55c2de34d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.964+0000 7f9ef7fe0740 1 --2- 172.21.15.43:0/4126141258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de344af0 0x55c2de34cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.964+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/4126141258 >> 172.21.15.43:0/4126141258 conn(0x55c2de341060 msgr2=0x55c2de346d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.965+0000 7f9ed2ff5640 0 client.25409 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.970+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/4126141258 shutdown_connections 2024-09-11T23:24:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.970+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/4126141258 wait complete. 2024-09-11T23:24:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:15.971+0000 7f9ef7fe0740 20 client.25409 trim_cache size 0 max 16384 2024-09-11T23:24:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.004+0000 7f9ef7fe0740 1 Processor -- start 2024-09-11T23:24:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.004+0000 7f9ef7fe0740 1 -- start start 2024-09-11T23:24:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.004+0000 7f9ef7fe0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de344af0 0x55c2de347560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.004+0000 7f9ef7fe0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de345410 0x55c2de34d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.004+0000 7f9ef7fe0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de34fcb0 0x55c2de367810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.004+0000 7f9eeed9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de344af0 0x55c2de347560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.004+0000 7f9eeed9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de344af0 0x55c2de347560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:40650/0 (socket says 172.21.15.43:40650) 2024-09-11T23:24:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.004+0000 7f9eeed9d640 1 -- 172.21.15.43:0/1253324227 learned_addr learned my addr 172.21.15.43:0/1253324227 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.004+0000 7f9eee59c640 1 --2- 172.21.15.43:0/1253324227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de345410 0x55c2de34d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.004+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/1253324227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef01bf0b0 con 0x55c2de34fcb0 2024-09-11T23:24:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.005+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/1253324227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef01bc980 con 0x55c2de344af0 2024-09-11T23:24:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.005+0000 7f9eef59e640 1 --2- 172.21.15.43:0/1253324227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de34fcb0 0x55c2de367810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.005+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/1253324227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef01ba7d0 con 0x55c2de345410 2024-09-11T23:24:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.005+0000 7f9eeed9d640 1 -- 172.21.15.43:0/1253324227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de345410 msgr2=0x55c2de34d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.005+0000 7f9eeed9d640 1 --2- 172.21.15.43:0/1253324227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de345410 0x55c2de34d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.005+0000 7f9eeed9d640 1 -- 172.21.15.43:0/1253324227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de34fcb0 msgr2=0x55c2de367810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.005+0000 7f9eeed9d640 1 --2- 172.21.15.43:0/1253324227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de34fcb0 0x55c2de367810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.005+0000 7f9eeed9d640 1 -- 172.21.15.43:0/1253324227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2de344870 con 0x55c2de344af0 2024-09-11T23:24:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.005+0000 7f9eef59e640 1 --2- 172.21.15.43:0/1253324227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de34fcb0 0x55c2de367810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:24:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.005+0000 7f9eeed9d640 1 --2- 172.21.15.43:0/1253324227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de344af0 0x55c2de347560 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00b9f70 tx=0x7f9ee0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c634320c00d1260 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.006+0000 7f9ed4ff9640 1 -- 172.21.15.43:0/1253324227 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee00d9070 con 0x55c2de344af0 2024-09-11T23:24:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.006+0000 7f9ed4ff9640 1 -- 172.21.15.43:0/1253324227 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee00a4070 con 0x55c2de344af0 2024-09-11T23:24:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.006+0000 7f9ed4ff9640 1 -- 172.21.15.43:0/1253324227 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0049040 con 0x55c2de344af0 2024-09-11T23:24:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.007+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/1253324227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de344af0 msgr2=0x55c2de347560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.007+0000 7f9ef7fe0740 1 --2- 172.21.15.43:0/1253324227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de344af0 0x55c2de347560 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00b9f70 tx=0x7f9ee0009260 comp rx=0 tx=0).stop 2024-09-11T23:24:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.007+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/1253324227 shutdown_connections 2024-09-11T23:24:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.007+0000 7f9ef7fe0740 1 --2- 172.21.15.43:0/1253324227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de34fcb0 0x55c2de367810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.007+0000 7f9ef7fe0740 1 --2- 172.21.15.43:0/1253324227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de345410 0x55c2de34d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.007+0000 7f9ef7fe0740 1 --2- 172.21.15.43:0/1253324227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de344af0 0x55c2de347560 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.007+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/1253324227 >> 172.21.15.43:0/1253324227 conn(0x55c2de341060 msgr2=0x55c2de34ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.007+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/1253324227 shutdown_connections 2024-09-11T23:24:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.007+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/1253324227 wait complete. 2024-09-11T23:24:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.007+0000 7f9ef7fe0740 1 -- start start 2024-09-11T23:24:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.007+0000 7f9ef7fe0740 1 Processor -- start 2024-09-11T23:24:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.008+0000 7f9ef7fe0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de344af0 0x55c2de349d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.008+0000 7f9ef7fe0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de345410 0x55c2de34a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.008+0000 7f9ef7fe0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de34fcb0 0x55c2de34a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.008+0000 7f9ef7fe0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef01bae40 con 0x55c2de345410 2024-09-11T23:24:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.008+0000 7f9ef7fe0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef01bc2b0 con 0x55c2de34fcb0 2024-09-11T23:24:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.008+0000 7f9ef7fe0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef01bb510 con 0x55c2de344af0 2024-09-11T23:24:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.008+0000 7f9ef7fe0740 10 client.0 _ll_register_callbacks cb 0x7f9ee800f5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.008+0000 7f9eee59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de345410 0x55c2de34a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.008+0000 7f9eee59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de345410 0x55c2de34a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59344/0 (socket says 172.21.15.43:59344) 2024-09-11T23:24:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.008+0000 7f9eee59c640 1 -- 172.21.15.43:0/49598566 learned_addr learned my addr 172.21.15.43:0/49598566 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.008+0000 7f9eeed9d640 1 --2- 172.21.15.43:0/49598566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de344af0 0x55c2de349d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.008+0000 7f9eee59c640 1 -- 172.21.15.43:0/49598566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de344af0 msgr2=0x55c2de349d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.008+0000 7f9eee59c640 1 --2- 172.21.15.43:0/49598566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de344af0 0x55c2de349d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.008+0000 7f9eee59c640 1 -- 172.21.15.43:0/49598566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de34fcb0 msgr2=0x55c2de34a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.008+0000 7f9eee59c640 1 --2- 172.21.15.43:0/49598566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de34fcb0 0x55c2de34a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.008+0000 7f9eee59c640 1 -- 172.21.15.43:0/49598566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2de3da8a0 con 0x55c2de345410 2024-09-11T23:24:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.008+0000 7f9eee59c640 1 --2- 172.21.15.43:0/49598566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de345410 0x55c2de34a240 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009ed0 tx=0x7f9ee4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367ade261f32642a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.008+0000 7f9ed2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:24:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.009+0000 7f9ed2ff5640 1 -- 172.21.15.43:0/49598566 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4024030 con 0x55c2de345410 2024-09-11T23:24:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.009+0000 7f9ed2ff5640 1 -- 172.21.15.43:0/49598566 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f9ee4020640 con 0x55c2de345410 2024-09-11T23:24:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.009+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/49598566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ee00a4070 con 0x55c2de345410 2024-09-11T23:24:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.009+0000 7f9ed2ff5640 1 -- 172.21.15.43:0/49598566 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9ee4020840 con 0x55c2de345410 2024-09-11T23:24:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.009+0000 7f9ef7fe0740 10 client.15876 fetch_fsmap learned FSMap version 26 2024-09-11T23:24:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.009+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/49598566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c2de3dbc20 con 0x55c2de345410 2024-09-11T23:24:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.009+0000 7f9ed2ff5640 1 -- 172.21.15.43:0/49598566 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee00a4070 con 0x55c2de345410 2024-09-11T23:24:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.009+0000 7f9ed2ff5640 1 -- 172.21.15.43:0/49598566 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9ee4012070 con 0x55c2de345410 2024-09-11T23:24:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.009+0000 7f9ef7fe0740 10 client.15876 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:24:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.009+0000 7f9ef7fe0740 10 client.15876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:24:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.009+0000 7f9ef7fe0740 20 client.15876 populate_metadata read hostname 'smithi043' 2024-09-11T23:24:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.009+0000 7f9ef7fe0740 1 --2- 172.21.15.43:0/49598566 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c2de3dc460 0x55c2de3de880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.009+0000 7f9ef7fe0740 4 client.15876 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:24:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.009+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/49598566 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c2de2b6a20 con 0x55c2de3dc460 2024-09-11T23:24:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.011+0000 7f9eeed9d640 1 --2- 172.21.15.43:0/49598566 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c2de3dc460 0x55c2de3de880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.012+0000 7f9eeed9d640 1 --2- 172.21.15.43:0/49598566 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c2de3dc460 0x55c2de3de880 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f80ba119a6d0b016 server_cookie=97b595bec6206d5f in_seq=0 out_seq=0 2024-09-11T23:24:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:16.012+0000 7f9ed2ff5640 10 client.15876 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:16 smithi043 ceph-mon[30950]: pgmap v75: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 192 MiB/s wr, 22.20k op/s 2024-09-11T23:24:16.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:16 smithi090 ceph-mon[35130]: pgmap v75: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 192 MiB/s wr, 22.20k op/s 2024-09-11T23:24:16.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:16 smithi097 ceph-mon[35562]: pgmap v75: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 192 MiB/s wr, 22.20k op/s 2024-09-11T23:24:17.899 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:24:17.899 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-11T23:24:17.899 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-11T23:24:17.900 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:24:17.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.854+0000 7f77278a4640 1 -- 172.21.15.43:0/1547459333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7720102570 msgr2=0x7f77201029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:17.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.854+0000 7f77278a4640 1 --2- 172.21.15.43:0/1547459333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7720102570 0x7f77201029b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7710009900 tx=0x7f771002a480 comp rx=0 tx=0).stop 2024-09-11T23:24:17.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.854+0000 7f77278a4640 1 -- 172.21.15.43:0/1547459333 shutdown_connections 2024-09-11T23:24:17.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.854+0000 7f77278a4640 1 --2- 172.21.15.43:0/1547459333 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7720102ef0 0x7f772010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.854+0000 7f77278a4640 1 --2- 172.21.15.43:0/1547459333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7720102570 0x7f77201029b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.854+0000 7f77278a4640 1 --2- 172.21.15.43:0/1547459333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7720100e20 0x7f7720101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.854+0000 7f77278a4640 1 -- 172.21.15.43:0/1547459333 >> 172.21.15.43:0/1547459333 conn(0x7f77200fc7d0 msgr2=0x7f77200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:17.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.854+0000 7f77278a4640 1 -- 172.21.15.43:0/1547459333 shutdown_connections 2024-09-11T23:24:17.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.854+0000 7f77278a4640 1 -- 172.21.15.43:0/1547459333 wait complete. 2024-09-11T23:24:17.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.855+0000 7f77278a4640 1 Processor -- start 2024-09-11T23:24:17.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.855+0000 7f77278a4640 1 -- start start 2024-09-11T23:24:17.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.855+0000 7f77278a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7720100e20 0x7f77200fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.855+0000 7f77278a4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7720102570 0x7f77200fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.856+0000 7f77278a4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7720102ef0 0x7f77200fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.856+0000 7f77278a4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7720110f50 con 0x7f7720100e20 2024-09-11T23:24:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.856+0000 7f77278a4640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7720110dd0 con 0x7f7720102ef0 2024-09-11T23:24:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.856+0000 7f77278a4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f77201110d0 con 0x7f7720102570 2024-09-11T23:24:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.856+0000 7f7725e41640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7720102ef0 0x7f77200fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.856+0000 7f7725e41640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7720102ef0 0x7f77200fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:40668/0 (socket says 172.21.15.43:40668) 2024-09-11T23:24:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.856+0000 7f7725e41640 1 -- 172.21.15.43:0/1349393365 learned_addr learned my addr 172.21.15.43:0/1349393365 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.856+0000 7f7724e3f640 1 --2- 172.21.15.43:0/1349393365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7720102570 0x7f77200fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.858+0000 7f7724e3f640 1 -- 172.21.15.43:0/1349393365 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7720102ef0 msgr2=0x7f77200fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.858+0000 7f7725640640 1 --2- 172.21.15.43:0/1349393365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7720100e20 0x7f77200fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.858+0000 7f7724e3f640 1 --2- 172.21.15.43:0/1349393365 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7720102ef0 0x7f77200fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.858+0000 7f7724e3f640 1 -- 172.21.15.43:0/1349393365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7720100e20 msgr2=0x7f77200fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.858+0000 7f7724e3f640 1 --2- 172.21.15.43:0/1349393365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7720100e20 0x7f77200fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.858+0000 7f7724e3f640 1 -- 172.21.15.43:0/1349393365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77201aaf50 con 0x7f7720102570 2024-09-11T23:24:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.858+0000 7f7725e41640 1 --2- 172.21.15.43:0/1349393365 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7720102ef0 0x7f77200fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:24:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.858+0000 7f7725640640 1 --2- 172.21.15.43:0/1349393365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7720100e20 0x7f77200fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:24:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.859+0000 7f7724e3f640 1 --2- 172.21.15.43:0/1349393365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7720102570 0x7f77200fd1c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7710005940 tx=0x7f7710003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.860+0000 7f77167fc640 1 -- 172.21.15.43:0/1349393365 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771003f070 con 0x7f7720102570 2024-09-11T23:24:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.860+0000 7f77278a4640 1 -- 172.21.15.43:0/1349393365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77201ab120 con 0x7f7720102570 2024-09-11T23:24:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.860+0000 7f77278a4640 1 -- 172.21.15.43:0/1349393365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77201ab6b0 con 0x7f7720102570 2024-09-11T23:24:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.860+0000 7f77167fc640 1 -- 172.21.15.43:0/1349393365 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7710031c30 con 0x7f7720102570 2024-09-11T23:24:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.861+0000 7f77167fc640 1 -- 172.21.15.43:0/1349393365 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7710049060 con 0x7f7720102570 2024-09-11T23:24:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.865+0000 7f7729c7d740 1 Processor -- start 2024-09-11T23:24:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.866+0000 7f7729c7d740 1 -- start start 2024-09-11T23:24:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.867+0000 7f7729c7d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614e50f2360 0x5614e50f2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.867+0000 7f77167fc640 1 -- 172.21.15.43:0/1349393365 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7710042070 con 0x7f7720102570 2024-09-11T23:24:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.868+0000 7f7725e41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614e50f2360 0x5614e50f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.868+0000 7f7725e41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614e50f2360 0x5614e50f2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59364/0 (socket says 172.21.15.43:59364) 2024-09-11T23:24:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.869+0000 7f7729c7d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e50f2c80 0x5614e510e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.869+0000 7f7725e41640 1 -- 172.21.15.43:0/3874275308 learned_addr learned my addr 172.21.15.43:0/3874275308 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.869+0000 7f7729c7d740 1 --2- 172.21.15.43:0/3874275308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e510e690 0x5614e5110a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.869+0000 7f7729c7d740 1 -- 172.21.15.43:0/3874275308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5614e50582e0 con 0x5614e50f2360 2024-09-11T23:24:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.869+0000 7f7729c7d740 1 -- 172.21.15.43:0/3874275308 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5614e4fffce0 con 0x5614e50f2c80 2024-09-11T23:24:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.869+0000 7f7729c7d740 1 -- 172.21.15.43:0/3874275308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5614e50542a0 con 0x5614e510e690 2024-09-11T23:24:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.870+0000 7f77167fc640 1 --2- 172.21.15.43:0/1349393365 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f770c0a2bd0 0x7f770c0a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.870+0000 7f77167fc640 1 -- 172.21.15.43:0/1349393365 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f77100efe80 con 0x7f7720102570 2024-09-11T23:24:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.871+0000 7f7725640640 1 --2- 172.21.15.43:0/3874275308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e50f2c80 0x5614e510e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.871+0000 7f7725e41640 1 --2- 172.21.15.43:0/1349393365 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f770c0a2bd0 0x7f770c0a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.873+0000 7f7725640640 1 -- 172.21.15.43:0/3874275308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e510e690 msgr2=0x5614e5110a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:24:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.873+0000 7f7725640640 1 --2- 172.21.15.43:0/3874275308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e510e690 0x5614e5110a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.873+0000 7f7725640640 1 -- 172.21.15.43:0/3874275308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614e50f2360 msgr2=0x5614e50f2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.873+0000 7f7725640640 1 --2- 172.21.15.43:0/3874275308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614e50f2360 0x5614e50f2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.873+0000 7f7725640640 1 -- 172.21.15.43:0/3874275308 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614e4fe4440 con 0x5614e50f2c80 2024-09-11T23:24:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.873+0000 7f7725640640 1 --2- 172.21.15.43:0/3874275308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e50f2c80 0x5614e510e150 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f771c005fd0 tx=0x7f771c00e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a855ed7aa17b85c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.873+0000 7f770b7fe640 1 -- 172.21.15.43:0/3874275308 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c016070 con 0x5614e50f2c80 2024-09-11T23:24:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.874+0000 7f770b7fe640 1 -- 172.21.15.43:0/3874275308 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f771c005500 con 0x5614e50f2c80 2024-09-11T23:24:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.874+0000 7f770b7fe640 1 -- 172.21.15.43:0/3874275308 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c0116d0 con 0x5614e50f2c80 2024-09-11T23:24:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.875+0000 7f7729c7d740 1 -- 172.21.15.43:0/3874275308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e50f2c80 msgr2=0x5614e510e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.875+0000 7f7729c7d740 1 --2- 172.21.15.43:0/3874275308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e50f2c80 0x5614e510e150 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f771c005fd0 tx=0x7f771c00e6c0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.875+0000 7f7729c7d740 1 -- 172.21.15.43:0/3874275308 shutdown_connections 2024-09-11T23:24:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.875+0000 7f7729c7d740 1 --2- 172.21.15.43:0/3874275308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e510e690 0x5614e5110a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.876+0000 7f7729c7d740 1 --2- 172.21.15.43:0/3874275308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e50f2c80 0x5614e510e150 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.876+0000 7f7729c7d740 1 --2- 172.21.15.43:0/3874275308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614e50f2360 0x5614e50f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.876+0000 7f7729c7d740 1 -- 172.21.15.43:0/3874275308 >> 172.21.15.43:0/3874275308 conn(0x5614e50eea30 msgr2=0x5614e50eee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.883+0000 7f7725e41640 1 --2- 172.21.15.43:0/1349393365 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f770c0a2bd0 0x7f770c0a5090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f77180094f0 tx=0x7f7718004ab0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.885+0000 7f7729c7d740 1 -- 172.21.15.43:0/3874275308 shutdown_connections 2024-09-11T23:24:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.885+0000 7f7729c7d740 1 -- 172.21.15.43:0/3874275308 wait complete. 2024-09-11T23:24:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.886+0000 7f7729c7d740 1 -- start start 2024-09-11T23:24:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.886+0000 7f7729c7d740 1 Processor -- start 2024-09-11T23:24:17.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.887+0000 7f7729c7d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614e50f2360 0x5614e510e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:17.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.888+0000 7f7729c7d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e50f2c80 0x5614e510b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:17.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.888+0000 7f7729c7d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e510e690 0x5614e510b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:17.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.888+0000 7f7729c7d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5614e5111260 con 0x5614e50f2360 2024-09-11T23:24:17.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.888+0000 7f7729c7d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5614e5013050 con 0x5614e50f2c80 2024-09-11T23:24:17.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.888+0000 7f7729c7d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5614e51113e0 con 0x5614e510e690 2024-09-11T23:24:17.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.888+0000 7f7729c7d740 10 client.0 _ll_register_callbacks cb 0x5614e5053160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:17.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.888+0000 7f7724e3f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e50f2c80 0x5614e510b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:17.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.888+0000 7f7724e3f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e50f2c80 0x5614e510b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:40678/0 (socket says 172.21.15.43:40678) 2024-09-11T23:24:17.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.888+0000 7f7724e3f640 1 -- 172.21.15.43:0/2853488261 learned_addr learned my addr 172.21.15.43:0/2853488261 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:17.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.889+0000 7f7724e3f640 1 -- 172.21.15.43:0/2853488261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e510e690 msgr2=0x5614e510b840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:24:17.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.889+0000 7f7724e3f640 1 --2- 172.21.15.43:0/2853488261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e510e690 0x5614e510b840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.889+0000 7f7724e3f640 1 -- 172.21.15.43:0/2853488261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614e50f2360 msgr2=0x5614e510e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:17.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.889+0000 7f7724e3f640 1 --2- 172.21.15.43:0/2853488261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614e50f2360 0x5614e510e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.889+0000 7f7724e3f640 1 -- 172.21.15.43:0/2853488261 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614e4dc93c0 con 0x5614e50f2c80 2024-09-11T23:24:17.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.890+0000 7f7724e3f640 1 --2- 172.21.15.43:0/2853488261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e50f2c80 0x5614e510b2c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7710009900 tx=0x7f77100bbdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ed6fa5dec44e7a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:17.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.890+0000 7f7708ff9640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:24:17.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.892+0000 7f7708ff9640 1 -- 172.21.15.43:0/2853488261 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77100bc3a0 con 0x5614e50f2c80 2024-09-11T23:24:17.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.893+0000 7f7729c7d740 1 -- 172.21.15.43:0/2853488261 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f771c005500 con 0x5614e50f2c80 2024-09-11T23:24:17.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.893+0000 7f7708ff9640 1 -- 172.21.15.43:0/2853488261 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f77100d52b0 con 0x5614e50f2c80 2024-09-11T23:24:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.894+0000 7f7708ff9640 1 -- 172.21.15.43:0/2853488261 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f77100c0750 con 0x5614e50f2c80 2024-09-11T23:24:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.897+0000 7f7729c7d740 10 client.25424 fetch_fsmap learned FSMap version 26 2024-09-11T23:24:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.897+0000 7f7729c7d740 1 -- 172.21.15.43:0/2853488261 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5614e511ccc0 con 0x5614e50f2c80 2024-09-11T23:24:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.899+0000 7f7708ff9640 1 -- 172.21.15.43:0/2853488261 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f771c005500 con 0x5614e50f2c80 2024-09-11T23:24:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.899+0000 7f7708ff9640 1 -- 172.21.15.43:0/2853488261 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f77100bc600 con 0x5614e50f2c80 2024-09-11T23:24:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.901+0000 7f7729c7d740 10 client.25424 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:24:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.901+0000 7f7729c7d740 10 client.25424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:24:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.901+0000 7f7729c7d740 20 client.25424 populate_metadata read hostname 'smithi043' 2024-09-11T23:24:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.901+0000 7f7729c7d740 1 --2- 172.21.15.43:0/2853488261 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5614e511e150 0x5614e51205f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.901+0000 7f7729c7d740 4 client.25424 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:24:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.901+0000 7f7729c7d740 1 -- 172.21.15.43:0/2853488261 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614e5065850 con 0x5614e511e150 2024-09-11T23:24:17.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.901+0000 7f7725640640 1 --2- 172.21.15.43:0/2853488261 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5614e511e150 0x5614e51205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:17.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.902+0000 7f7708ff9640 10 client.25424 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:17.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.902+0000 7f7725640640 1 --2- 172.21.15.43:0/2853488261 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5614e511e150 0x5614e51205f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=518d8c7901cde03a server_cookie=da24ed3df416be2c in_seq=0 out_seq=0 2024-09-11T23:24:17.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.905+0000 7f7708ff9640 1 -- 172.21.15.43:0/2853488261 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5614e5065850 con 0x5614e511e150 2024-09-11T23:24:17.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.905+0000 7f7708ff9640 10 client.25424 handle_command_reply: tid=0 2024-09-11T23:24:17.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.905+0000 7f7729c7d740 1 client.25424 shutdown 2024-09-11T23:24:17.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.907+0000 7f7729c7d740 1 -- 172.21.15.43:0/2853488261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e50f2c80 msgr2=0x5614e510b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:17.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.907+0000 7f7729c7d740 1 --2- 172.21.15.43:0/2853488261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e50f2c80 0x5614e510b2c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7710009900 tx=0x7f77100bbdc0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.907+0000 7f7729c7d740 1 -- 172.21.15.43:0/2853488261 shutdown_connections 2024-09-11T23:24:17.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.907+0000 7f7729c7d740 1 --2- 172.21.15.43:0/2853488261 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5614e511e150 0x5614e51205f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.907+0000 7f7729c7d740 1 --2- 172.21.15.43:0/2853488261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e510e690 0x5614e510b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.907+0000 7f7729c7d740 1 --2- 172.21.15.43:0/2853488261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e50f2c80 0x5614e510b2c0 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7710009900 tx=0x7f77100bbdc0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.907+0000 7f7729c7d740 1 --2- 172.21.15.43:0/2853488261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614e50f2360 0x5614e510e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.907+0000 7f7729c7d740 1 -- 172.21.15.43:0/2853488261 >> 172.21.15.43:0/2853488261 conn(0x5614e50eea30 msgr2=0x5614e50f4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:17.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.907+0000 7f7708ff9640 0 client.25424 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:17.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.907+0000 7f7729c7d740 1 -- 172.21.15.43:0/2853488261 shutdown_connections 2024-09-11T23:24:17.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.907+0000 7f7729c7d740 1 -- 172.21.15.43:0/2853488261 wait complete. 2024-09-11T23:24:17.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.908+0000 7f7729c7d740 20 client.25424 trim_cache size 0 max 16384 2024-09-11T23:24:17.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.947+0000 7f7729c7d740 1 Processor -- start 2024-09-11T23:24:17.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.947+0000 7f7729c7d740 1 -- start start 2024-09-11T23:24:17.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.948+0000 7f7729c7d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e50f2360 0x5614e511a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:17.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.948+0000 7f7729c7d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614e510e690 0x5614e50ef040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:17.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.948+0000 7f7725640640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e50f2360 0x5614e511a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:17.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.948+0000 7f7725640640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e50f2360 0x5614e511a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:37760/0 (socket says 172.21.15.43:37760) 2024-09-11T23:24:17.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.948+0000 7f7725640640 1 -- 172.21.15.43:0/2789620209 learned_addr learned my addr 172.21.15.43:0/2789620209 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:17.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.949+0000 7f7724e3f640 1 --2- 172.21.15.43:0/2789620209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614e510e690 0x5614e50ef040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:17.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.949+0000 7f7729c7d740 1 --2- 172.21.15.43:0/2789620209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e5106800 0x5614e5125ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:17.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.949+0000 7f7729c7d740 1 -- 172.21.15.43:0/2789620209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f77201adf00 con 0x5614e510e690 2024-09-11T23:24:17.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.949+0000 7f7729c7d740 1 -- 172.21.15.43:0/2789620209 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f77201a4cc0 con 0x5614e5106800 2024-09-11T23:24:17.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.949+0000 7f7729c7d740 1 -- 172.21.15.43:0/2789620209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f77201a2b10 con 0x5614e50f2360 2024-09-11T23:24:17.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.949+0000 7f7725e41640 1 --2- 172.21.15.43:0/2789620209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e5106800 0x5614e5125ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:17.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.950+0000 7f7725640640 1 -- 172.21.15.43:0/2789620209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e5106800 msgr2=0x5614e5125ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:17.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.950+0000 7f7725640640 1 --2- 172.21.15.43:0/2789620209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e5106800 0x5614e5125ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.950+0000 7f7725640640 1 -- 172.21.15.43:0/2789620209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614e510e690 msgr2=0x5614e50ef040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:17.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.950+0000 7f7725640640 1 --2- 172.21.15.43:0/2789620209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614e510e690 0x5614e50ef040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.950+0000 7f7725640640 1 -- 172.21.15.43:0/2789620209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614e510a270 con 0x5614e50f2360 2024-09-11T23:24:17.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.950+0000 7f7724e3f640 1 --2- 172.21.15.43:0/2789620209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614e510e690 0x5614e50ef040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:24:17.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.950+0000 7f7725e41640 1 --2- 172.21.15.43:0/2789620209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e5106800 0x5614e5125ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:24:17.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.950+0000 7f7725640640 1 --2- 172.21.15.43:0/2789620209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e50f2360 0x5614e511a530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f771c004bd0 tx=0x7f771c00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50f3d2d7ab60926d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:17.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.952+0000 7f770affd640 1 -- 172.21.15.43:0/2789620209 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c016040 con 0x5614e50f2360 2024-09-11T23:24:17.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.952+0000 7f770affd640 1 -- 172.21.15.43:0/2789620209 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f771c0097c0 con 0x5614e50f2360 2024-09-11T23:24:17.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.952+0000 7f770affd640 1 -- 172.21.15.43:0/2789620209 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c003850 con 0x5614e50f2360 2024-09-11T23:24:17.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.953+0000 7f7729c7d740 1 -- 172.21.15.43:0/2789620209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e50f2360 msgr2=0x5614e511a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:17.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.953+0000 7f7729c7d740 1 --2- 172.21.15.43:0/2789620209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e50f2360 0x5614e511a530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f771c004bd0 tx=0x7f771c00ef60 comp rx=0 tx=0).stop 2024-09-11T23:24:17.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.954+0000 7f7729c7d740 1 -- 172.21.15.43:0/2789620209 shutdown_connections 2024-09-11T23:24:17.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.954+0000 7f7729c7d740 1 --2- 172.21.15.43:0/2789620209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e5106800 0x5614e5125ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.954+0000 7f7729c7d740 1 --2- 172.21.15.43:0/2789620209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614e510e690 0x5614e50ef040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.954+0000 7f7729c7d740 1 --2- 172.21.15.43:0/2789620209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e50f2360 0x5614e511a530 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.954+0000 7f7729c7d740 1 -- 172.21.15.43:0/2789620209 >> 172.21.15.43:0/2789620209 conn(0x5614e50eea30 msgr2=0x5614e51060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:17.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.954+0000 7f7729c7d740 1 -- 172.21.15.43:0/2789620209 shutdown_connections 2024-09-11T23:24:17.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.955+0000 7f7729c7d740 1 -- 172.21.15.43:0/2789620209 wait complete. 2024-09-11T23:24:17.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.956+0000 7f7729c7d740 1 -- start start 2024-09-11T23:24:17.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.956+0000 7f7729c7d740 1 Processor -- start 2024-09-11T23:24:17.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.957+0000 7f7729c7d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e50f2360 0x5614e51123d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:17.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.957+0000 7f7729c7d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614e5106800 0x5614e5112910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:17.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.957+0000 7f7729c7d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e510e690 0x5614e5112e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:17.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.957+0000 7f7729c7d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f77201a3180 con 0x5614e5106800 2024-09-11T23:24:17.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.957+0000 7f7729c7d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f77201a45f0 con 0x5614e50f2360 2024-09-11T23:24:17.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.957+0000 7f7725640640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e50f2360 0x5614e51123d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:17.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.957+0000 7f7729c7d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f77201a3850 con 0x5614e510e690 2024-09-11T23:24:17.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.958+0000 7f7729c7d740 10 client.0 _ll_register_callbacks cb 0x7f7710009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:17.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.958+0000 7f7725e41640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e510e690 0x5614e5112e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:17.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.958+0000 7f7725e41640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e510e690 0x5614e5112e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:37774/0 (socket says 172.21.15.43:37774) 2024-09-11T23:24:17.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.958+0000 7f7725e41640 1 -- 172.21.15.43:0/682144984 learned_addr learned my addr 172.21.15.43:0/682144984 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:17.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.958+0000 7f7725e41640 1 -- 172.21.15.43:0/682144984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e50f2360 msgr2=0x5614e51123d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:17.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.958+0000 7f7725e41640 1 --2- 172.21.15.43:0/682144984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e50f2360 0x5614e51123d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.958+0000 7f7725e41640 1 -- 172.21.15.43:0/682144984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614e5106800 msgr2=0x5614e5112910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:17.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.958+0000 7f7725e41640 1 --2- 172.21.15.43:0/682144984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614e5106800 0x5614e5112910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.958+0000 7f7725e41640 1 -- 172.21.15.43:0/682144984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614e50f2ab0 con 0x5614e510e690 2024-09-11T23:24:17.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.960+0000 7f7725e41640 1 --2- 172.21.15.43:0/682144984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e510e690 0x5614e5112e50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7718013390 tx=0x7f7718004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39ea7462b7550c53 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:17.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.960+0000 7f7708ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-11T23:24:17.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.960+0000 7f7708ff9640 1 -- 172.21.15.43:0/682144984 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7718013dd0 con 0x5614e510e690 2024-09-11T23:24:17.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.960+0000 7f7708ff9640 1 -- 172.21.15.43:0/682144984 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f7718004b20 con 0x5614e510e690 2024-09-11T23:24:17.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.960+0000 7f7729c7d740 1 -- 172.21.15.43:0/682144984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f771c0097c0 con 0x5614e510e690 2024-09-11T23:24:17.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.963+0000 7f7708ff9640 1 -- 172.21.15.43:0/682144984 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7718004d20 con 0x5614e510e690 2024-09-11T23:24:17.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.963+0000 7f7729c7d740 10 client.25411 fetch_fsmap learned FSMap version 26 2024-09-11T23:24:17.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.963+0000 7f7729c7d740 1 -- 172.21.15.43:0/682144984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5614e5188f50 con 0x5614e510e690 2024-09-11T23:24:17.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.965+0000 7f7708ff9640 1 -- 172.21.15.43:0/682144984 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7718034960 con 0x5614e510e690 2024-09-11T23:24:17.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.965+0000 7f7708ff9640 1 -- 172.21.15.43:0/682144984 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f771800c050 con 0x5614e510e690 2024-09-11T23:24:17.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.965+0000 7f7729c7d740 10 client.25411 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:24:17.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.965+0000 7f7729c7d740 10 client.25411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:24:17.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.965+0000 7f7729c7d740 20 client.25411 populate_metadata read hostname 'smithi043' 2024-09-11T23:24:17.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.965+0000 7f7729c7d740 1 --2- 172.21.15.43:0/682144984 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5614e5189730 0x5614e518bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:17.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.965+0000 7f7729c7d740 4 client.25411 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:24:17.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.965+0000 7f7729c7d740 1 -- 172.21.15.43:0/682144984 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f771c0097c0 con 0x5614e5189730 2024-09-11T23:24:17.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.966+0000 7f7725640640 1 --2- 172.21.15.43:0/682144984 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5614e5189730 0x5614e518bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:17.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.966+0000 7f7725640640 1 --2- 172.21.15.43:0/682144984 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5614e5189730 0x5614e518bb50 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d905801c8c61ba server_cookie=986fdb184d75467d in_seq=0 out_seq=0 2024-09-11T23:24:17.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.967+0000 7f7708ff9640 10 client.25411 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:17.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.967+0000 7f7708ff9640 1 -- 172.21.15.43:0/682144984 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f771c0097c0 con 0x5614e5189730 2024-09-11T23:24:17.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.967+0000 7f7708ff9640 10 client.25411 handle_command_reply: tid=0 2024-09-11T23:24:17.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.967+0000 7f7729c7d740 1 client.25411 shutdown 2024-09-11T23:24:17.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.967+0000 7f7729c7d740 1 -- 172.21.15.43:0/682144984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e510e690 msgr2=0x5614e5112e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:17.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.967+0000 7f7729c7d740 1 --2- 172.21.15.43:0/682144984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e510e690 0x5614e5112e50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7718013390 tx=0x7f7718004a40 comp rx=0 tx=0).stop 2024-09-11T23:24:17.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.968+0000 7f7729c7d740 1 -- 172.21.15.43:0/682144984 shutdown_connections 2024-09-11T23:24:17.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.968+0000 7f7729c7d740 1 --2- 172.21.15.43:0/682144984 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5614e5189730 0x5614e518bb50 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.968+0000 7f7729c7d740 1 --2- 172.21.15.43:0/682144984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614e510e690 0x5614e5112e50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.968+0000 7f7708ff9640 0 client.25411 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:17.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.968+0000 7f7729c7d740 1 --2- 172.21.15.43:0/682144984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5614e5106800 0x5614e5112910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.968+0000 7f7729c7d740 1 --2- 172.21.15.43:0/682144984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5614e50f2360 0x5614e51123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.968+0000 7f7729c7d740 1 -- 172.21.15.43:0/682144984 >> 172.21.15.43:0/682144984 conn(0x5614e50eea30 msgr2=0x5614e510cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:17.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.978+0000 7f7729c7d740 1 -- 172.21.15.43:0/682144984 shutdown_connections 2024-09-11T23:24:17.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.978+0000 7f7729c7d740 1 -- 172.21.15.43:0/682144984 wait complete. 2024-09-11T23:24:17.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.978+0000 7f7729c7d740 20 client.25411 trim_cache size 0 max 16384 2024-09-11T23:24:17.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.980+0000 7f77278a4640 1 -- 172.21.15.43:0/1349393365 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f770c0a2bd0 msgr2=0x7f770c0a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:17.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.980+0000 7f77278a4640 1 --2- 172.21.15.43:0/1349393365 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f770c0a2bd0 0x7f770c0a5090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f77180094f0 tx=0x7f7718004ab0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.980+0000 7f77278a4640 1 -- 172.21.15.43:0/1349393365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7720102570 msgr2=0x7f77200fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:17.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.980+0000 7f77278a4640 1 --2- 172.21.15.43:0/1349393365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7720102570 0x7f77200fd1c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7710005940 tx=0x7f7710003c10 comp rx=0 tx=0).stop 2024-09-11T23:24:17.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.981+0000 7f77278a4640 1 -- 172.21.15.43:0/1349393365 shutdown_connections 2024-09-11T23:24:17.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.981+0000 7f77278a4640 1 --2- 172.21.15.43:0/1349393365 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f770c0a2bd0 0x7f770c0a5090 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.981+0000 7f77278a4640 1 --2- 172.21.15.43:0/1349393365 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7720102ef0 0x7f77200fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.981+0000 7f77278a4640 1 --2- 172.21.15.43:0/1349393365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7720102570 0x7f77200fd1c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.981+0000 7f77278a4640 1 --2- 172.21.15.43:0/1349393365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7720100e20 0x7f77200fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:17.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.981+0000 7f77278a4640 1 -- 172.21.15.43:0/1349393365 >> 172.21.15.43:0/1349393365 conn(0x7f77200fc7d0 msgr2=0x7f772006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:17.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.982+0000 7f77278a4640 1 -- 172.21.15.43:0/1349393365 shutdown_connections 2024-09-11T23:24:17.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:16.983+0000 7f77278a4640 1 -- 172.21.15.43:0/1349393365 wait complete. 2024-09-11T23:24:18.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:18 smithi043 ceph-mon[30950]: pgmap v76: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 192 MiB/s wr, 22.20k op/s 2024-09-11T23:24:18.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:18 smithi090 ceph-mon[35130]: pgmap v76: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 192 MiB/s wr, 22.20k op/s 2024-09-11T23:24:18.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:18 smithi097 ceph-mon[35562]: pgmap v76: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 192 MiB/s wr, 22.20k op/s 2024-09-11T23:24:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.691+0000 7f9ed2ff5640 1 -- 172.21.15.43:0/49598566 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c2de2b6a20 con 0x55c2de3dc460 2024-09-11T23:24:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.691+0000 7f9ed2ff5640 10 client.15876 handle_command_reply: tid=0 2024-09-11T23:24:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.691+0000 7f9ef7fe0740 1 client.15876 shutdown 2024-09-11T23:24:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.691+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/49598566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de345410 msgr2=0x55c2de34a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.691+0000 7f9ef7fe0740 1 --2- 172.21.15.43:0/49598566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de345410 0x55c2de34a240 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009ed0 tx=0x7f9ee4004750 comp rx=0 tx=0).stop 2024-09-11T23:24:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.691+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/49598566 shutdown_connections 2024-09-11T23:24:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.691+0000 7f9ef7fe0740 1 --2- 172.21.15.43:0/49598566 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c2de3dc460 0x55c2de3de880 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.691+0000 7f9ed2ff5640 0 client.15876 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.691+0000 7f9ef7fe0740 1 --2- 172.21.15.43:0/49598566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c2de34fcb0 0x55c2de34a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.691+0000 7f9ef7fe0740 1 --2- 172.21.15.43:0/49598566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2de345410 0x55c2de34a240 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.691+0000 7f9ef7fe0740 1 --2- 172.21.15.43:0/49598566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c2de344af0 0x55c2de349d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.691+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/49598566 >> 172.21.15.43:0/49598566 conn(0x55c2de341060 msgr2=0x55c2de34e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.692+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/49598566 shutdown_connections 2024-09-11T23:24:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.692+0000 7f9ef7fe0740 1 -- 172.21.15.43:0/49598566 wait complete. 2024-09-11T23:24:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.692+0000 7f9ef7fe0740 20 client.15876 trim_cache size 0 max 16384 2024-09-11T23:24:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:24:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:24:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:24:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:24:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.693+0000 7f9ef5166640 1 -- 172.21.15.43:0/3144954541 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9ed80a3030 msgr2=0x7f9ed80a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.693+0000 7f9ef5166640 1 --2- 172.21.15.43:0/3144954541 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9ed80a3030 0x7f9ed80a54f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9ef011d630 tx=0x7f9ee4006bd0 comp rx=0 tx=0).stop 2024-09-11T23:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.693+0000 7f9ef5166640 1 -- 172.21.15.43:0/3144954541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ef01091f0 msgr2=0x7f9ef01a7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.693+0000 7f9ef5166640 1 --2- 172.21.15.43:0/3144954541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ef01091f0 0x7f9ef01a7210 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ee002a990 tx=0x7f9ee0003c00 comp rx=0 tx=0).stop 2024-09-11T23:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.693+0000 7f9ef5166640 1 -- 172.21.15.43:0/3144954541 shutdown_connections 2024-09-11T23:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.693+0000 7f9ef5166640 1 --2- 172.21.15.43:0/3144954541 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9ed80a3030 0x7f9ed80a54f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.693+0000 7f9ef5166640 1 --2- 172.21.15.43:0/3144954541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ef011b140 0x7f9ef01abad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.693+0000 7f9ef5166640 1 --2- 172.21.15.43:0/3144954541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ef01095d0 0x7f9ef01a7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.693+0000 7f9ef5166640 1 --2- 172.21.15.43:0/3144954541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ef01091f0 0x7f9ef01a7210 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.693+0000 7f9ef5166640 1 -- 172.21.15.43:0/3144954541 >> 172.21.15.43:0/3144954541 conn(0x7f9ef006cdf0 msgr2=0x7f9ef011b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.693+0000 7f9ef5166640 1 -- 172.21.15.43:0/3144954541 shutdown_connections 2024-09-11T23:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:18.693+0000 7f9ef5166640 1 -- 172.21.15.43:0/3144954541 wait complete. 2024-09-11T23:24:20.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:20 smithi043 ceph-mon[30950]: pgmap v77: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 252 MiB/s wr, 33.19k op/s 2024-09-11T23:24:20.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:20 smithi090 ceph-mon[35130]: pgmap v77: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 252 MiB/s wr, 33.19k op/s 2024-09-11T23:24:20.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:20 smithi097 ceph-mon[35562]: pgmap v77: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 252 MiB/s wr, 33.19k op/s 2024-09-11T23:24:22.340 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:22 smithi043 ceph-mon[30950]: pgmap v78: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 167 MiB/s wr, 24.66k op/s 2024-09-11T23:24:22.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:22 smithi090 ceph-mon[35130]: pgmap v78: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 167 MiB/s wr, 24.66k op/s 2024-09-11T23:24:22.557 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:22 smithi097 ceph-mon[35562]: pgmap v78: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 167 MiB/s wr, 24.66k op/s 2024-09-11T23:24:24.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.284+0000 7fcd32ff5640 1 -- 172.21.15.43:0/3009759723 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x562a4f85eec0 con 0x562a4f9fc670 2024-09-11T23:24:24.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.284+0000 7fcd32ff5640 10 client.15861 handle_command_reply: tid=0 2024-09-11T23:24:24.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.284+0000 7fcd5790d740 1 client.15861 shutdown 2024-09-11T23:24:24.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.284+0000 7fcd5790d740 1 -- 172.21.15.43:0/3009759723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f982fe0 msgr2=0x562a4f97d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:24.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.284+0000 7fcd5790d740 1 --2- 172.21.15.43:0/3009759723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f982fe0 0x562a4f97d3c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcd48005e30 tx=0x7fcd480c8ec0 comp rx=0 tx=0).stop 2024-09-11T23:24:24.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.284+0000 7fcd5790d740 1 -- 172.21.15.43:0/3009759723 shutdown_connections 2024-09-11T23:24:24.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.284+0000 7fcd5790d740 1 --2- 172.21.15.43:0/3009759723 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562a4f9fc670 0x562a4f9fea90 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:24.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.284+0000 7fcd5790d740 1 --2- 172.21.15.43:0/3009759723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562a4f982fe0 0x562a4f97d3c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:24.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.284+0000 7fcd5790d740 1 --2- 172.21.15.43:0/3009759723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a4f980e70 0x562a4f97ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:24.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.284+0000 7fcd32ff5640 0 client.15861 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:24.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.284+0000 7fcd5790d740 1 --2- 172.21.15.43:0/3009759723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562a4f975900 0x562a4f988cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:24.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.284+0000 7fcd5790d740 1 -- 172.21.15.43:0/3009759723 >> 172.21.15.43:0/3009759723 conn(0x562a4f971e90 msgr2=0x562a4f982310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:24.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.284+0000 7fcd5790d740 1 -- 172.21.15.43:0/3009759723 shutdown_connections 2024-09-11T23:24:24.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.284+0000 7fcd5790d740 1 -- 172.21.15.43:0/3009759723 wait complete. 2024-09-11T23:24:24.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.285+0000 7fcd5790d740 20 client.15861 trim_cache size 0 max 16384 2024-09-11T23:24:24.286 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:24:24.286 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 26, 2024-09-11T23:24:24.286 INFO:teuthology.orchestra.run.smithi043.stdout: "leader": 24497, 2024-09-11T23:24:24.286 INFO:teuthology.orchestra.run.smithi043.stdout: "set_version": 6, 2024-09-11T23:24:24.287 INFO:teuthology.orchestra.run.smithi043.stdout: "sets": { 2024-09-11T23:24:24.287 INFO:teuthology.orchestra.run.smithi043.stdout: "993ae2f2": { 2024-09-11T23:24:24.287 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 6, 2024-09-11T23:24:24.287 INFO:teuthology.orchestra.run.smithi043.stdout: "age_ref": 0.0, 2024-09-11T23:24:24.287 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:24:24.287 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "QUIESCED", 2024-09-11T23:24:24.287 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-09-11T23:24:24.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:24:24.287 INFO:teuthology.orchestra.run.smithi043.stdout: "timeout": 90.0, 2024-09-11T23:24:24.287 INFO:teuthology.orchestra.run.smithi043.stdout: "expiration": 133.4, 2024-09-11T23:24:24.287 INFO:teuthology.orchestra.run.smithi043.stdout: "members": { 2024-09-11T23:24:24.287 INFO:teuthology.orchestra.run.smithi043.stdout: "file:/": { 2024-09-11T23:24:24.287 INFO:teuthology.orchestra.run.smithi043.stdout: "excluded": false, 2024-09-11T23:24:24.287 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:24:24.288 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "QUIESCED", 2024-09-11T23:24:24.288 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-09-11T23:24:24.288 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:24:24.288 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:24:24.288 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:24:24.288 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:24:24.288 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:24:24.288 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:24:24.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.285+0000 7fcd4ffff640 1 -- 172.21.15.43:0/1726737141 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fcd380a2ce0 msgr2=0x7fcd380a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:24.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.285+0000 7fcd4ffff640 1 --2- 172.21.15.43:0/1726737141 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fcd380a2ce0 0x7fcd380a51a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcd50118180 tx=0x7fcd44008040 comp rx=0 tx=0).stop 2024-09-11T23:24:24.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.285+0000 7fcd4ffff640 1 -- 172.21.15.43:0/1726737141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd5011b140 msgr2=0x7fcd50116f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:24.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.285+0000 7fcd4ffff640 1 --2- 172.21.15.43:0/1726737141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd5011b140 0x7fcd50116f00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcd4801ac00 tx=0x7fcd480405f0 comp rx=0 tx=0).stop 2024-09-11T23:24:24.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.285+0000 7fcd4ffff640 1 -- 172.21.15.43:0/1726737141 shutdown_connections 2024-09-11T23:24:24.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.285+0000 7fcd4ffff640 1 --2- 172.21.15.43:0/1726737141 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fcd380a2ce0 0x7fcd380a51a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:24.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.285+0000 7fcd4ffff640 1 --2- 172.21.15.43:0/1726737141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd5011b140 0x7fcd50116f00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:24.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.285+0000 7fcd4ffff640 1 --2- 172.21.15.43:0/1726737141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd50110190 0x7fcd501169c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:24.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.285+0000 7fcd4ffff640 1 --2- 172.21.15.43:0/1726737141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd5006ea30 0x7fcd5011a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:24.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.285+0000 7fcd4ffff640 1 -- 172.21.15.43:0/1726737141 >> 172.21.15.43:0/1726737141 conn(0x7fcd50073180 msgr2=0x7fcd5011b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:24.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.286+0000 7fcd4ffff640 1 -- 172.21.15.43:0/1726737141 shutdown_connections 2024-09-11T23:24:24.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:24.286+0000 7fcd4ffff640 1 -- 172.21.15.43:0/1726737141 wait complete. 2024-09-11T23:24:24.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:24 smithi043 ceph-mon[30950]: pgmap v79: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 193 MiB/s wr, 29.16k op/s 2024-09-11T23:24:24.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:24 smithi090 ceph-mon[35130]: pgmap v79: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 193 MiB/s wr, 29.16k op/s 2024-09-11T23:24:24.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:24 smithi097 ceph-mon[35562]: pgmap v79: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 193 MiB/s wr, 29.16k op/s 2024-09-11T23:24:26.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:26 smithi043 ceph-mon[30950]: pgmap v80: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 147 MiB/s wr, 24.22k op/s 2024-09-11T23:24:26.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:26 smithi090 ceph-mon[35130]: pgmap v80: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 147 MiB/s wr, 24.22k op/s 2024-09-11T23:24:26.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:26 smithi097 ceph-mon[35562]: pgmap v80: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 147 MiB/s wr, 24.22k op/s 2024-09-11T23:24:28.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:28 smithi043 ceph-mon[30950]: pgmap v81: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 126 MiB/s wr, 21.75k op/s 2024-09-11T23:24:28.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:28 smithi090 ceph-mon[35130]: pgmap v81: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 126 MiB/s wr, 21.75k op/s 2024-09-11T23:24:28.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:28 smithi097 ceph-mon[35562]: pgmap v81: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 126 MiB/s wr, 21.75k op/s 2024-09-11T23:24:30.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:30 smithi043 ceph-mon[30950]: pgmap v82: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 149 MiB/s wr, 26.11k op/s 2024-09-11T23:24:30.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:30 smithi090 ceph-mon[35130]: pgmap v82: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 149 MiB/s wr, 26.11k op/s 2024-09-11T23:24:30.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:30 smithi097 ceph-mon[35562]: pgmap v82: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 149 MiB/s wr, 26.11k op/s 2024-09-11T23:24:31.803 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.802+0000 7f6649952640 1 -- 172.21.15.43:0/1097269991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6644101370 msgr2=0x7f664410d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:31.803 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.802+0000 7f6649952640 1 --2- 172.21.15.43:0/1097269991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6644101370 0x7f664410d8a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f663c009cd0 tx=0x7f663c02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.803 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.802+0000 7f6649952640 1 -- 172.21.15.43:0/1097269991 shutdown_connections 2024-09-11T23:24:31.803 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.802+0000 7f6649952640 1 --2- 172.21.15.43:0/1097269991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6644101370 0x7f664410d8a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.802+0000 7f6649952640 1 --2- 172.21.15.43:0/1097269991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66441009d0 0x7f6644100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.802+0000 7f6649952640 1 --2- 172.21.15.43:0/1097269991 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66441069e0 0x7f6644106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.802+0000 7f6649952640 1 -- 172.21.15.43:0/1097269991 >> 172.21.15.43:0/1097269991 conn(0x7f66440fc6b0 msgr2=0x7f66440fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.802+0000 7f6649952640 1 -- 172.21.15.43:0/1097269991 shutdown_connections 2024-09-11T23:24:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.802+0000 7f6649952640 1 -- 172.21.15.43:0/1097269991 wait complete. 2024-09-11T23:24:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.802+0000 7f6649952640 1 Processor -- start 2024-09-11T23:24:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.802+0000 7f6649952640 1 -- start start 2024-09-11T23:24:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.803+0000 7f6649952640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66441009d0 0x7f6644198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.803+0000 7f6649952640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6644101370 0x7f66441988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.803+0000 7f6649952640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66441069e0 0x7f664419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.803+0000 7f6649952640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6644111f10 con 0x7f66441069e0 2024-09-11T23:24:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.803+0000 7f6649952640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6644111d90 con 0x7f66441009d0 2024-09-11T23:24:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.803+0000 7f6649952640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6644112090 con 0x7f6644101370 2024-09-11T23:24:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.803+0000 7f66437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66441069e0 0x7f664419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.803+0000 7f66437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66441069e0 0x7f664419cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44774/0 (socket says 172.21.15.43:44774) 2024-09-11T23:24:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.803+0000 7f66437fe640 1 -- 172.21.15.43:0/3444010507 learned_addr learned my addr 172.21.15.43:0/3444010507 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.803+0000 7f6642ffd640 1 --2- 172.21.15.43:0/3444010507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66441009d0 0x7f6644198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.803+0000 7f663bfff640 1 --2- 172.21.15.43:0/3444010507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6644101370 0x7f66441988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.803+0000 7f66437fe640 1 -- 172.21.15.43:0/3444010507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6644101370 msgr2=0x7f66441988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.803+0000 7f66437fe640 1 --2- 172.21.15.43:0/3444010507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6644101370 0x7f66441988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.803+0000 7f66437fe640 1 -- 172.21.15.43:0/3444010507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66441009d0 msgr2=0x7f6644198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.803+0000 7f66437fe640 1 --2- 172.21.15.43:0/3444010507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66441009d0 0x7f6644198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.803+0000 7f66437fe640 1 -- 172.21.15.43:0/3444010507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664419d350 con 0x7f66441069e0 2024-09-11T23:24:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.803+0000 7f66437fe640 1 --2- 172.21.15.43:0/3444010507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66441069e0 0x7f664419cc20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f663c005cc0 tx=0x7f663c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.804+0000 7f6648950640 1 -- 172.21.15.43:0/3444010507 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c004040 con 0x7f66441069e0 2024-09-11T23:24:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.804+0000 7f6648950640 1 -- 172.21.15.43:0/3444010507 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663c0046c0 con 0x7f66441069e0 2024-09-11T23:24:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.804+0000 7f6649952640 1 -- 172.21.15.43:0/3444010507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f664419d5e0 con 0x7f66441069e0 2024-09-11T23:24:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.804+0000 7f6648950640 1 -- 172.21.15.43:0/3444010507 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c046480 con 0x7f66441069e0 2024-09-11T23:24:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.804+0000 7f6649952640 1 -- 172.21.15.43:0/3444010507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66441a4ed0 con 0x7f66441069e0 2024-09-11T23:24:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.806+0000 7f6648950640 1 -- 172.21.15.43:0/3444010507 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f663c0305a0 con 0x7f66441069e0 2024-09-11T23:24:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.807+0000 7f664c7cb740 1 Processor -- start 2024-09-11T23:24:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.807+0000 7f664c7cb740 1 -- start start 2024-09-11T23:24:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.807+0000 7f664c7cb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5604cdc899d0 0x5604cdc89db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.807+0000 7f664c7cb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604cdc8a2f0 0x5604cdc98b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.807+0000 7f664c7cb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc990c0 0x5604cdc9b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.807+0000 7f664c7cb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5604cdbee2f0 con 0x5604cdc990c0 2024-09-11T23:24:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.807+0000 7f664c7cb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5604cdb96270 con 0x5604cdc899d0 2024-09-11T23:24:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.807+0000 7f664c7cb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5604cdbea2b0 con 0x5604cdc8a2f0 2024-09-11T23:24:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.807+0000 7f6648950640 1 --2- 172.21.15.43:0/3444010507 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f66300a2cf0 0x7f66300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.807+0000 7f663bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc990c0 0x5604cdc9b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.807+0000 7f663bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc990c0 0x5604cdc9b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44788/0 (socket says 172.21.15.43:44788) 2024-09-11T23:24:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.807+0000 7f663bfff640 1 -- 172.21.15.43:0/1903055212 learned_addr learned my addr 172.21.15.43:0/1903055212 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.807+0000 7f6642ffd640 1 --2- 172.21.15.43:0/1903055212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604cdc8a2f0 0x5604cdc98b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.808+0000 7f663bfff640 1 --2- 172.21.15.43:0/1903055212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5604cdc899d0 0x5604cdc89db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.808+0000 7f66437fe640 1 --2- 172.21.15.43:0/3444010507 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f66300a2cf0 0x7f66300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.808+0000 7f6648950640 1 -- 172.21.15.43:0/3444010507 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f663c02e030 con 0x7f66441069e0 2024-09-11T23:24:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.808+0000 7f663bfff640 1 -- 172.21.15.43:0/1903055212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604cdc8a2f0 msgr2=0x5604cdc98b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.808+0000 7f663bfff640 1 --2- 172.21.15.43:0/1903055212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604cdc8a2f0 0x5604cdc98b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.808+0000 7f663bfff640 1 -- 172.21.15.43:0/1903055212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5604cdc899d0 msgr2=0x5604cdc89db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.808+0000 7f663bfff640 1 --2- 172.21.15.43:0/1903055212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5604cdc899d0 0x5604cdc89db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.808+0000 7f663bfff640 1 -- 172.21.15.43:0/1903055212 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604cdb95340 con 0x5604cdc990c0 2024-09-11T23:24:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.808+0000 7f6642ffd640 1 --2- 172.21.15.43:0/1903055212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604cdc8a2f0 0x5604cdc98b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:24:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.808+0000 7f663bfff640 1 --2- 172.21.15.43:0/1903055212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc990c0 0x5604cdc9b4b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f662c0127f0 tx=0x7f662c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f487ffc6474ff4b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.808+0000 7f66437fe640 1 --2- 172.21.15.43:0/3444010507 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f66300a2cf0 0x7f66300a51b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f663c0bcde0 tx=0x7f663c090150 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.808+0000 7f6639ffb640 1 -- 172.21.15.43:0/1903055212 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c012e30 con 0x5604cdc990c0 2024-09-11T23:24:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.808+0000 7f6639ffb640 1 -- 172.21.15.43:0/1903055212 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f662c010c50 con 0x5604cdc990c0 2024-09-11T23:24:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.808+0000 7f6639ffb640 1 -- 172.21.15.43:0/1903055212 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c01eb30 con 0x5604cdc990c0 2024-09-11T23:24:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.809+0000 7f664c7cb740 1 -- 172.21.15.43:0/1903055212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc990c0 msgr2=0x5604cdc9b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.809+0000 7f664c7cb740 1 --2- 172.21.15.43:0/1903055212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc990c0 0x5604cdc9b4b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f662c0127f0 tx=0x7f662c00b7c0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.809+0000 7f664c7cb740 1 -- 172.21.15.43:0/1903055212 shutdown_connections 2024-09-11T23:24:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.809+0000 7f664c7cb740 1 --2- 172.21.15.43:0/1903055212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc990c0 0x5604cdc9b4b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.809+0000 7f664c7cb740 1 --2- 172.21.15.43:0/1903055212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604cdc8a2f0 0x5604cdc98b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.809+0000 7f664c7cb740 1 --2- 172.21.15.43:0/1903055212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5604cdc899d0 0x5604cdc89db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.809+0000 7f664c7cb740 1 -- 172.21.15.43:0/1903055212 >> 172.21.15.43:0/1903055212 conn(0x5604cdc86060 msgr2=0x5604cdc86440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.809+0000 7f664c7cb740 1 -- 172.21.15.43:0/1903055212 shutdown_connections 2024-09-11T23:24:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.809+0000 7f664c7cb740 1 -- 172.21.15.43:0/1903055212 wait complete. 2024-09-11T23:24:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.809+0000 7f664c7cb740 1 -- start start 2024-09-11T23:24:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.810+0000 7f664c7cb740 1 Processor -- start 2024-09-11T23:24:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.810+0000 7f664c7cb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc899d0 0x5604cdc96040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.810+0000 7f664c7cb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604cdc8a2f0 0x5604cdc96580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.810+0000 7f664c7cb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5604cdc990c0 0x5604cdc96ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.810+0000 7f664c7cb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5604cdc9bcf0 con 0x5604cdc899d0 2024-09-11T23:24:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.810+0000 7f664c7cb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5604cdb77290 con 0x5604cdc990c0 2024-09-11T23:24:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.810+0000 7f664c7cb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5604cdc9be70 con 0x5604cdc8a2f0 2024-09-11T23:24:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.810+0000 7f664c7cb740 10 client.0 _ll_register_callbacks cb 0x5604cdba8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.810+0000 7f663bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc899d0 0x5604cdc96040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.810+0000 7f663bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc899d0 0x5604cdc96040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44790/0 (socket says 172.21.15.43:44790) 2024-09-11T23:24:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.810+0000 7f663bfff640 1 -- 172.21.15.43:0/2280936962 learned_addr learned my addr 172.21.15.43:0/2280936962 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.810+0000 7f6642ffd640 1 --2- 172.21.15.43:0/2280936962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604cdc8a2f0 0x5604cdc96580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.810+0000 7f663bfff640 1 --2- 172.21.15.43:0/2280936962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5604cdc990c0 0x5604cdc96ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.811+0000 7f663bfff640 1 -- 172.21.15.43:0/2280936962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604cdc8a2f0 msgr2=0x5604cdc96580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.811+0000 7f663bfff640 1 --2- 172.21.15.43:0/2280936962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604cdc8a2f0 0x5604cdc96580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.811+0000 7f663bfff640 1 -- 172.21.15.43:0/2280936962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5604cdc990c0 msgr2=0x5604cdc96ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.811+0000 7f663bfff640 1 --2- 172.21.15.43:0/2280936962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5604cdc990c0 0x5604cdc96ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.811+0000 7f663bfff640 1 -- 172.21.15.43:0/2280936962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604cd901210 con 0x5604cdc899d0 2024-09-11T23:24:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.811+0000 7f663bfff640 1 --2- 172.21.15.43:0/2280936962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc899d0 0x5604cdc96040 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f662c005d00 tx=0x7f662c01e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f26da1e62ffc2e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.811+0000 7f6622ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:24:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.811+0000 7f6622ffd640 1 -- 172.21.15.43:0/2280936962 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c024040 con 0x5604cdc899d0 2024-09-11T23:24:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.811+0000 7f6622ffd640 1 -- 172.21.15.43:0/2280936962 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f662c010980 con 0x5604cdc899d0 2024-09-11T23:24:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.811+0000 7f664c7cb740 1 -- 172.21.15.43:0/2280936962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f662c010c50 con 0x5604cdc899d0 2024-09-11T23:24:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.811+0000 7f6622ffd640 1 -- 172.21.15.43:0/2280936962 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f662c010e00 con 0x5604cdc899d0 2024-09-11T23:24:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.811+0000 7f664c7cb740 10 client.15894 fetch_fsmap learned FSMap version 26 2024-09-11T23:24:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.811+0000 7f664c7cb740 1 -- 172.21.15.43:0/2280936962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5604cd9cd620 con 0x5604cdc899d0 2024-09-11T23:24:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.811+0000 7f6622ffd640 1 -- 172.21.15.43:0/2280936962 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f662c010c50 con 0x5604cdc899d0 2024-09-11T23:24:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.811+0000 7f6622ffd640 1 -- 172.21.15.43:0/2280936962 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f662c0054d0 con 0x5604cdc899d0 2024-09-11T23:24:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.812+0000 7f664c7cb740 10 client.15894 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:24:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.812+0000 7f664c7cb740 10 client.15894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:24:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.812+0000 7f664c7cb740 20 client.15894 populate_metadata read hostname 'smithi043' 2024-09-11T23:24:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.812+0000 7f664c7cb740 1 --2- 172.21.15.43:0/2280936962 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5604cdca8ff0 0x5604cdcab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.812+0000 7f664c7cb740 4 client.15894 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:24:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.812+0000 7f664c7cb740 1 -- 172.21.15.43:0/2280936962 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604cdbfba20 con 0x5604cdca8ff0 2024-09-11T23:24:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.812+0000 7f6642ffd640 1 --2- 172.21.15.43:0/2280936962 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5604cdca8ff0 0x5604cdcab3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.812+0000 7f6622ffd640 10 client.15894 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.812+0000 7f6642ffd640 1 --2- 172.21.15.43:0/2280936962 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5604cdca8ff0 0x5604cdcab3e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dc4d47e13a2eaba server_cookie=723a2cd249c269c3 in_seq=0 out_seq=0 2024-09-11T23:24:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.813+0000 7f6622ffd640 1 -- 172.21.15.43:0/2280936962 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604cdbfba20 con 0x5604cdca8ff0 2024-09-11T23:24:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.813+0000 7f6622ffd640 10 client.15894 handle_command_reply: tid=0 2024-09-11T23:24:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.813+0000 7f664c7cb740 1 client.15894 shutdown 2024-09-11T23:24:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.813+0000 7f664c7cb740 1 -- 172.21.15.43:0/2280936962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc899d0 msgr2=0x5604cdc96040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.813+0000 7f664c7cb740 1 --2- 172.21.15.43:0/2280936962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc899d0 0x5604cdc96040 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f662c005d00 tx=0x7f662c01e710 comp rx=0 tx=0).stop 2024-09-11T23:24:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.813+0000 7f664c7cb740 1 -- 172.21.15.43:0/2280936962 shutdown_connections 2024-09-11T23:24:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.813+0000 7f664c7cb740 1 --2- 172.21.15.43:0/2280936962 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5604cdca8ff0 0x5604cdcab3e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.813+0000 7f664c7cb740 1 --2- 172.21.15.43:0/2280936962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5604cdc990c0 0x5604cdc96ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.813+0000 7f664c7cb740 1 --2- 172.21.15.43:0/2280936962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604cdc8a2f0 0x5604cdc96580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.813+0000 7f664c7cb740 1 --2- 172.21.15.43:0/2280936962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc899d0 0x5604cdc96040 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.813+0000 7f6622ffd640 0 client.15894 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.813+0000 7f664c7cb740 1 -- 172.21.15.43:0/2280936962 >> 172.21.15.43:0/2280936962 conn(0x5604cdc86060 msgr2=0x5604cdc8bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.813+0000 7f664c7cb740 1 -- 172.21.15.43:0/2280936962 shutdown_connections 2024-09-11T23:24:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.813+0000 7f664c7cb740 1 -- 172.21.15.43:0/2280936962 wait complete. 2024-09-11T23:24:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.813+0000 7f664c7cb740 20 client.15894 trim_cache size 0 max 16384 2024-09-11T23:24:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.828+0000 7f664c7cb740 1 Processor -- start 2024-09-11T23:24:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.828+0000 7f664c7cb740 1 -- start start 2024-09-11T23:24:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.828+0000 7f664c7cb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5604cdc899d0 0x5604cdc850b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.828+0000 7f664c7cb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc8a2f0 0x5604cdcb0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.828+0000 7f664c7cb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604cdc990c0 0x5604cdcb1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.828+0000 7f664c7cb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f66441adf00 con 0x5604cdc8a2f0 2024-09-11T23:24:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.828+0000 7f664c7cb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f66441ab7d0 con 0x5604cdc899d0 2024-09-11T23:24:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.828+0000 7f664c7cb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f66441a9620 con 0x5604cdc990c0 2024-09-11T23:24:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.828+0000 7f6642ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc8a2f0 0x5604cdcb0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.828+0000 7f6642ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc8a2f0 0x5604cdcb0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44806/0 (socket says 172.21.15.43:44806) 2024-09-11T23:24:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.828+0000 7f6642ffd640 1 -- 172.21.15.43:0/1218395500 learned_addr learned my addr 172.21.15.43:0/1218395500 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.828+0000 7f663bfff640 1 --2- 172.21.15.43:0/1218395500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5604cdc899d0 0x5604cdc850b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.828+0000 7f663bfff640 1 --2- 172.21.15.43:0/1218395500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604cdc990c0 0x5604cdcb1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.829+0000 7f6642ffd640 1 -- 172.21.15.43:0/1218395500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604cdc990c0 msgr2=0x5604cdcb1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.829+0000 7f6642ffd640 1 --2- 172.21.15.43:0/1218395500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604cdc990c0 0x5604cdcb1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.829+0000 7f6642ffd640 1 -- 172.21.15.43:0/1218395500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5604cdc899d0 msgr2=0x5604cdc850b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.829+0000 7f6642ffd640 1 --2- 172.21.15.43:0/1218395500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5604cdc899d0 0x5604cdc850b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.829+0000 7f6642ffd640 1 -- 172.21.15.43:0/1218395500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604cdc802f0 con 0x5604cdc8a2f0 2024-09-11T23:24:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.829+0000 7f663bfff640 1 --2- 172.21.15.43:0/1218395500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5604cdc899d0 0x5604cdc850b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:24:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.829+0000 7f6642ffd640 1 --2- 172.21.15.43:0/1218395500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc8a2f0 0x5604cdcb0b50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x5604cdcabaf0 tx=0x7f663400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bf97b6dc07df55 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.829+0000 7f66397fa640 1 -- 172.21.15.43:0/1218395500 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6634003ca0 con 0x5604cdc8a2f0 2024-09-11T23:24:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.829+0000 7f66397fa640 1 -- 172.21.15.43:0/1218395500 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6634003e40 con 0x5604cdc8a2f0 2024-09-11T23:24:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.829+0000 7f66397fa640 1 -- 172.21.15.43:0/1218395500 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66340119a0 con 0x5604cdc8a2f0 2024-09-11T23:24:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.829+0000 7f664c7cb740 1 -- 172.21.15.43:0/1218395500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc8a2f0 msgr2=0x5604cdcb0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.829+0000 7f664c7cb740 1 --2- 172.21.15.43:0/1218395500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc8a2f0 0x5604cdcb0b50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x5604cdcabaf0 tx=0x7f663400b390 comp rx=0 tx=0).stop 2024-09-11T23:24:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.830+0000 7f664c7cb740 1 -- 172.21.15.43:0/1218395500 shutdown_connections 2024-09-11T23:24:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.830+0000 7f664c7cb740 1 --2- 172.21.15.43:0/1218395500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604cdc990c0 0x5604cdcb1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.830+0000 7f664c7cb740 1 --2- 172.21.15.43:0/1218395500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc8a2f0 0x5604cdcb0b50 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.830+0000 7f664c7cb740 1 --2- 172.21.15.43:0/1218395500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5604cdc899d0 0x5604cdc850b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.830+0000 7f664c7cb740 1 -- 172.21.15.43:0/1218395500 >> 172.21.15.43:0/1218395500 conn(0x5604cdc86060 msgr2=0x5604cdc849a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.830+0000 7f664c7cb740 1 -- 172.21.15.43:0/1218395500 shutdown_connections 2024-09-11T23:24:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.830+0000 7f664c7cb740 1 -- 172.21.15.43:0/1218395500 wait complete. 2024-09-11T23:24:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.830+0000 7f664c7cb740 1 -- start start 2024-09-11T23:24:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.830+0000 7f664c7cb740 1 Processor -- start 2024-09-11T23:24:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.830+0000 7f664c7cb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc899d0 0x5604cdc9ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.830+0000 7f664c7cb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604cdc8a2f0 0x5604cdc9d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.830+0000 7f664c7cb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5604cdc990c0 0x5604cdc9d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.830+0000 7f664c7cb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f66441a9c90 con 0x5604cdc899d0 2024-09-11T23:24:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.830+0000 7f664c7cb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f66441ab100 con 0x5604cdc990c0 2024-09-11T23:24:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.830+0000 7f664c7cb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f66441aa360 con 0x5604cdc8a2f0 2024-09-11T23:24:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.831+0000 7f664c7cb740 10 client.0 _ll_register_callbacks cb 0x7f662c00be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.831+0000 7f663bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc899d0 0x5604cdc9ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.831+0000 7f663bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc899d0 0x5604cdc9ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44810/0 (socket says 172.21.15.43:44810) 2024-09-11T23:24:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.831+0000 7f663bfff640 1 -- 172.21.15.43:0/1776544431 learned_addr learned my addr 172.21.15.43:0/1776544431 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.831+0000 7f663bfff640 1 --2- 172.21.15.43:0/1776544431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5604cdc990c0 0x5604cdc9d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.831+0000 7f6642ffd640 1 --2- 172.21.15.43:0/1776544431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604cdc8a2f0 0x5604cdc9d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.831+0000 7f663bfff640 1 -- 172.21.15.43:0/1776544431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604cdc8a2f0 msgr2=0x5604cdc9d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.831+0000 7f663bfff640 1 --2- 172.21.15.43:0/1776544431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604cdc8a2f0 0x5604cdc9d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.831+0000 7f663bfff640 1 -- 172.21.15.43:0/1776544431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5604cdc990c0 msgr2=0x5604cdc9d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.831+0000 7f663bfff640 1 --2- 172.21.15.43:0/1776544431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5604cdc990c0 0x5604cdc9d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.831+0000 7f663bfff640 1 -- 172.21.15.43:0/1776544431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604cdd12b20 con 0x5604cdc899d0 2024-09-11T23:24:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.831+0000 7f663bfff640 1 --2- 172.21.15.43:0/1776544431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc899d0 0x5604cdc9ce60 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f662c010740 tx=0x7f662c02d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203a4ccca6277cf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.831+0000 7f6622ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:24:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.831+0000 7f6622ffd640 1 -- 172.21.15.43:0/1776544431 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c039020 con 0x5604cdc899d0 2024-09-11T23:24:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.831+0000 7f6622ffd640 1 -- 172.21.15.43:0/1776544431 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f662c00bab0 con 0x5604cdc899d0 2024-09-11T23:24:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.831+0000 7f664c7cb740 1 -- 172.21.15.43:0/1776544431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6634003e40 con 0x5604cdc899d0 2024-09-11T23:24:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.831+0000 7f6622ffd640 1 -- 172.21.15.43:0/1776544431 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f662c010c50 con 0x5604cdc899d0 2024-09-11T23:24:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.831+0000 7f664c7cb740 10 client.15900 fetch_fsmap learned FSMap version 26 2024-09-11T23:24:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.831+0000 7f664c7cb740 1 -- 172.21.15.43:0/1776544431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5604cdd13d90 con 0x5604cdc899d0 2024-09-11T23:24:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.832+0000 7f6622ffd640 1 -- 172.21.15.43:0/1776544431 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6634003e40 con 0x5604cdc899d0 2024-09-11T23:24:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.832+0000 7f6622ffd640 1 -- 172.21.15.43:0/1776544431 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f662c033430 con 0x5604cdc899d0 2024-09-11T23:24:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.832+0000 7f664c7cb740 10 client.15900 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:24:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.832+0000 7f664c7cb740 10 client.15900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:24:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.832+0000 7f664c7cb740 20 client.15900 populate_metadata read hostname 'smithi043' 2024-09-11T23:24:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.832+0000 7f664c7cb740 1 --2- 172.21.15.43:0/1776544431 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5604cdd144d0 0x5604cdd168f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.832+0000 7f664c7cb740 4 client.15900 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:24:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.832+0000 7f664c7cb740 1 -- 172.21.15.43:0/1776544431 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5604cdbf5d70 con 0x5604cdd144d0 2024-09-11T23:24:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.832+0000 7f6642ffd640 1 --2- 172.21.15.43:0/1776544431 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5604cdd144d0 0x5604cdd168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.833+0000 7f6642ffd640 1 --2- 172.21.15.43:0/1776544431 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5604cdd144d0 0x5604cdd168f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2f2083ed434155 server_cookie=6c01a796113dea64 in_seq=0 out_seq=0 2024-09-11T23:24:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:31.833+0000 7f6622ffd640 10 client.15900 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.028+0000 7f6622ffd640 1 -- 172.21.15.43:0/1776544431 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5604cdbf5d70 con 0x5604cdd144d0 2024-09-11T23:24:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.028+0000 7f6622ffd640 10 client.15900 handle_command_reply: tid=0 2024-09-11T23:24:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.028+0000 7f664c7cb740 1 client.15900 shutdown 2024-09-11T23:24:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.028+0000 7f664c7cb740 1 -- 172.21.15.43:0/1776544431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc899d0 msgr2=0x5604cdc9ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.028+0000 7f664c7cb740 1 --2- 172.21.15.43:0/1776544431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc899d0 0x5604cdc9ce60 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f662c010740 tx=0x7f662c02d040 comp rx=0 tx=0).stop 2024-09-11T23:24:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.028+0000 7f664c7cb740 1 -- 172.21.15.43:0/1776544431 shutdown_connections 2024-09-11T23:24:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.028+0000 7f664c7cb740 1 --2- 172.21.15.43:0/1776544431 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5604cdd144d0 0x5604cdd168f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.028+0000 7f664c7cb740 1 --2- 172.21.15.43:0/1776544431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5604cdc990c0 0x5604cdc9d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.028+0000 7f664c7cb740 1 --2- 172.21.15.43:0/1776544431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604cdc8a2f0 0x5604cdc9d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.028+0000 7f664c7cb740 1 --2- 172.21.15.43:0/1776544431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5604cdc899d0 0x5604cdc9ce60 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.028+0000 7f6622ffd640 0 client.15900 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.028+0000 7f664c7cb740 1 -- 172.21.15.43:0/1776544431 >> 172.21.15.43:0/1776544431 conn(0x5604cdc86060 msgr2=0x5604cdc851c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.029+0000 7f664c7cb740 1 -- 172.21.15.43:0/1776544431 shutdown_connections 2024-09-11T23:24:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.029+0000 7f664c7cb740 1 -- 172.21.15.43:0/1776544431 wait complete. 2024-09-11T23:24:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.029+0000 7f664c7cb740 20 client.15900 trim_cache size 0 max 16384 2024-09-11T23:24:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:24:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:24:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:24:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:24:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.029+0000 7f6649952640 1 -- 172.21.15.43:0/3444010507 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f66300a2cf0 msgr2=0x7f66300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.029+0000 7f6649952640 1 --2- 172.21.15.43:0/3444010507 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f66300a2cf0 0x7f66300a51b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f663c0bcde0 tx=0x7f663c090150 comp rx=0 tx=0).stop 2024-09-11T23:24:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.029+0000 7f6649952640 1 -- 172.21.15.43:0/3444010507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66441069e0 msgr2=0x7f664419cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.029+0000 7f6649952640 1 --2- 172.21.15.43:0/3444010507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66441069e0 0x7f664419cc20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f663c005cc0 tx=0x7f663c003970 comp rx=0 tx=0).stop 2024-09-11T23:24:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.029+0000 7f6649952640 1 -- 172.21.15.43:0/3444010507 shutdown_connections 2024-09-11T23:24:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.029+0000 7f6649952640 1 --2- 172.21.15.43:0/3444010507 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f66300a2cf0 0x7f66300a51b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.029+0000 7f6649952640 1 --2- 172.21.15.43:0/3444010507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66441069e0 0x7f664419cc20 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.029+0000 7f6649952640 1 --2- 172.21.15.43:0/3444010507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6644101370 0x7f66441988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.029+0000 7f6649952640 1 --2- 172.21.15.43:0/3444010507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66441009d0 0x7f6644198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.029+0000 7f6649952640 1 -- 172.21.15.43:0/3444010507 >> 172.21.15.43:0/3444010507 conn(0x7f66440fc6b0 msgr2=0x7f66440fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.030+0000 7f6649952640 1 -- 172.21.15.43:0/3444010507 shutdown_connections 2024-09-11T23:24:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:32.030+0000 7f6649952640 1 -- 172.21.15.43:0/3444010507 wait complete. 2024-09-11T23:24:32.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:32 smithi043 ceph-mon[30950]: pgmap v83: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 84 MiB/s wr, 14.85k op/s 2024-09-11T23:24:32.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:32 smithi090 ceph-mon[35130]: pgmap v83: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 84 MiB/s wr, 14.85k op/s 2024-09-11T23:24:32.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:32 smithi097 ceph-mon[35562]: pgmap v83: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 84 MiB/s wr, 14.85k op/s 2024-09-11T23:24:34.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:34 smithi043 ceph-mon[30950]: pgmap v84: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 78 MiB/s wr, 14.01k op/s 2024-09-11T23:24:34.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:34 smithi090 ceph-mon[35130]: pgmap v84: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 78 MiB/s wr, 14.01k op/s 2024-09-11T23:24:34.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:34 smithi097 ceph-mon[35562]: pgmap v84: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 78 MiB/s wr, 14.01k op/s 2024-09-11T23:24:35.573 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:24:35.574 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:24:35.979 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '993ae2f2', quiesce took 47.0 seconds. Will release after: -33.6 2024-09-11T23:24:35.979 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:24:36.022 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:24:36.299 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:36 smithi043 ceph-mon[30950]: pgmap v85: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 53 MiB/s wr, 9.54k op/s 2024-09-11T23:24:36.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:36 smithi090 ceph-mon[35130]: pgmap v85: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 53 MiB/s wr, 9.54k op/s 2024-09-11T23:24:36.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.535+0000 7f5620378640 1 -- 172.21.15.43:0/2372033533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5618102d30 msgr2=0x7f5618103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:36.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.535+0000 7f5620378640 1 --2- 172.21.15.43:0/2372033533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5618102d30 0x7f5618103110 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5614009900 tx=0x7f561402a480 comp rx=0 tx=0).stop 2024-09-11T23:24:36.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.535+0000 7f5620378640 1 -- 172.21.15.43:0/2372033533 shutdown_connections 2024-09-11T23:24:36.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.535+0000 7f5620378640 1 --2- 172.21.15.43:0/2372033533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f561810c160 0x7f561810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:36.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.535+0000 7f5620378640 1 --2- 172.21.15.43:0/2372033533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5618103650 0x7f561810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:36.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.535+0000 7f5620378640 1 --2- 172.21.15.43:0/2372033533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5618102d30 0x7f5618103110 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:36.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.535+0000 7f5620378640 1 -- 172.21.15.43:0/2372033533 >> 172.21.15.43:0/2372033533 conn(0x7f56180fc790 msgr2=0x7f56180febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:36.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.535+0000 7f5620378640 1 -- 172.21.15.43:0/2372033533 shutdown_connections 2024-09-11T23:24:36.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.535+0000 7f5620378640 1 -- 172.21.15.43:0/2372033533 wait complete. 2024-09-11T23:24:36.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.536+0000 7f5620378640 1 Processor -- start 2024-09-11T23:24:36.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.536+0000 7f5620378640 1 -- start start 2024-09-11T23:24:36.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.536+0000 7f5620378640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5618102d30 0x7f5618100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:36.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.536+0000 7f5620378640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5618103650 0x7f5618100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:36.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.536+0000 7f5620378640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f561810c160 0x7f5618101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:36.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.536+0000 7f5620378640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5618111f60 con 0x7f561810c160 2024-09-11T23:24:36.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.536+0000 7f5620378640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5618111de0 con 0x7f5618102d30 2024-09-11T23:24:36.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.536+0000 7f5620378640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f56181120e0 con 0x7f5618103650 2024-09-11T23:24:36.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.536+0000 7f561e915640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f561810c160 0x7f5618101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:36.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.536+0000 7f561e114640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5618102d30 0x7f5618100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:36.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.536+0000 7f561e915640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f561810c160 0x7f5618101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44826/0 (socket says 172.21.15.43:44826) 2024-09-11T23:24:36.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.536+0000 7f561e915640 1 -- 172.21.15.43:0/3630075948 learned_addr learned my addr 172.21.15.43:0/3630075948 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:36.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.536+0000 7f561d913640 1 --2- 172.21.15.43:0/3630075948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5618103650 0x7f5618100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:36.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.537+0000 7f561d913640 1 -- 172.21.15.43:0/3630075948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5618102d30 msgr2=0x7f5618100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:36.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.537+0000 7f561d913640 1 --2- 172.21.15.43:0/3630075948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5618102d30 0x7f5618100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:36.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.537+0000 7f561d913640 1 -- 172.21.15.43:0/3630075948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f561810c160 msgr2=0x7f5618101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:36.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.537+0000 7f561d913640 1 --2- 172.21.15.43:0/3630075948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f561810c160 0x7f5618101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:36.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.537+0000 7f561d913640 1 -- 172.21.15.43:0/3630075948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56181a6900 con 0x7f5618103650 2024-09-11T23:24:36.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.537+0000 7f561e915640 1 --2- 172.21.15.43:0/3630075948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f561810c160 0x7f5618101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:24:36.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.537+0000 7f561d913640 1 --2- 172.21.15.43:0/3630075948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5618103650 0x7f5618100ed0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f560800e7a0 tx=0x7f560800ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:36.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.537+0000 7f560f7fe640 1 -- 172.21.15.43:0/3630075948 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560800ccc0 con 0x7f5618103650 2024-09-11T23:24:36.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.537+0000 7f5620378640 1 -- 172.21.15.43:0/3630075948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56181a6bf0 con 0x7f5618103650 2024-09-11T23:24:36.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.537+0000 7f560f7fe640 1 -- 172.21.15.43:0/3630075948 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5608009530 con 0x7f5618103650 2024-09-11T23:24:36.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.537+0000 7f560f7fe640 1 -- 172.21.15.43:0/3630075948 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5608018880 con 0x7f5618103650 2024-09-11T23:24:36.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.537+0000 7f5620378640 1 -- 172.21.15.43:0/3630075948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56181a7180 con 0x7f5618103650 2024-09-11T23:24:36.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.538+0000 7f5620378640 1 -- 172.21.15.43:0/3630075948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5618103110 con 0x7f5618103650 2024-09-11T23:24:36.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.540+0000 7f560f7fe640 1 -- 172.21.15.43:0/3630075948 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5608018a20 con 0x7f5618103650 2024-09-11T23:24:36.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.540+0000 7f560f7fe640 1 --2- 172.21.15.43:0/3630075948 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56040a2d70 0x7f56040a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:36.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.540+0000 7f561e114640 1 --2- 172.21.15.43:0/3630075948 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56040a2d70 0x7f56040a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:36.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.540+0000 7f560f7fe640 1 -- 172.21.15.43:0/3630075948 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f56080cfaa0 con 0x7f5618103650 2024-09-11T23:24:36.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.541+0000 7f561e114640 1 --2- 172.21.15.43:0/3630075948 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56040a2d70 0x7f56040a5230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5614009230 tx=0x7f56140058d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:36.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.544+0000 7f560f7fe640 1 -- 172.21.15.43:0/3630075948 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f560809c010 con 0x7f5618103650 2024-09-11T23:24:36.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:36 smithi097 ceph-mon[35562]: pgmap v85: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 53 MiB/s wr, 9.54k op/s 2024-09-11T23:24:36.569 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:24:36.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.776+0000 7f5620378640 1 -- 172.21.15.43:0/3630075948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5618101ec0 con 0x7f5618103650 2024-09-11T23:24:36.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.777+0000 7f560f7fe640 1 -- 172.21.15.43:0/3630075948 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11899 (secure 0 0 0) 0x7f5618101ec0 con 0x7f5618103650 2024-09-11T23:24:36.778 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:24:36.778 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"btime":"2024-09-11T23:23:19:881806+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:23:19.722053+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:24:36.778 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-09-11T23:24:36.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.779+0000 7f5620378640 1 -- 172.21.15.43:0/3630075948 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56040a2d70 msgr2=0x7f56040a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:36.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.779+0000 7f5620378640 1 --2- 172.21.15.43:0/3630075948 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56040a2d70 0x7f56040a5230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5614009230 tx=0x7f56140058d0 comp rx=0 tx=0).stop 2024-09-11T23:24:36.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.779+0000 7f5620378640 1 -- 172.21.15.43:0/3630075948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5618103650 msgr2=0x7f5618100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:36.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.779+0000 7f5620378640 1 --2- 172.21.15.43:0/3630075948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5618103650 0x7f5618100ed0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f560800e7a0 tx=0x7f560800ec60 comp rx=0 tx=0).stop 2024-09-11T23:24:36.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.779+0000 7f5620378640 1 -- 172.21.15.43:0/3630075948 shutdown_connections 2024-09-11T23:24:36.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.779+0000 7f5620378640 1 --2- 172.21.15.43:0/3630075948 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56040a2d70 0x7f56040a5230 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:36.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.779+0000 7f5620378640 1 --2- 172.21.15.43:0/3630075948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f561810c160 0x7f5618101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:36.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.779+0000 7f5620378640 1 --2- 172.21.15.43:0/3630075948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5618103650 0x7f5618100ed0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:36.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.779+0000 7f5620378640 1 --2- 172.21.15.43:0/3630075948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5618102d30 0x7f5618100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:36.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.779+0000 7f5620378640 1 -- 172.21.15.43:0/3630075948 >> 172.21.15.43:0/3630075948 conn(0x7f56180fc790 msgr2=0x7f56180fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:36.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.779+0000 7f5620378640 1 -- 172.21.15.43:0/3630075948 shutdown_connections 2024-09-11T23:24:36.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:36.779+0000 7f5620378640 1 -- 172.21.15.43:0/3630075948 wait complete. 2024-09-11T23:24:37.216 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:37 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3630075948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:24:37.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:37 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3630075948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:24:37.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:37 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3630075948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:24:37.677 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:24:37.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.927+0000 7fa2c9cb8640 1 -- 172.21.15.43:0/338817019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2c4102790 msgr2=0x7fa2c41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:37.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.927+0000 7fa2c9cb8640 1 --2- 172.21.15.43:0/338817019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2c4102790 0x7fa2c41067e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009900 tx=0x7fa2b802a410 comp rx=0 tx=0).stop 2024-09-11T23:24:37.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.927+0000 7fa2c9cb8640 1 -- 172.21.15.43:0/338817019 shutdown_connections 2024-09-11T23:24:37.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.927+0000 7fa2c9cb8640 1 --2- 172.21.15.43:0/338817019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2c4106f10 0x7fa2c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:37.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.927+0000 7fa2c9cb8640 1 --2- 172.21.15.43:0/338817019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2c4102790 0x7fa2c41067e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:37.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.927+0000 7fa2c9cb8640 1 --2- 172.21.15.43:0/338817019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2c4101de0 0x7fa2c41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:37.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.927+0000 7fa2c9cb8640 1 -- 172.21.15.43:0/338817019 >> 172.21.15.43:0/338817019 conn(0x7fa2c40fd8c0 msgr2=0x7fa2c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:37.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.927+0000 7fa2c9cb8640 1 -- 172.21.15.43:0/338817019 shutdown_connections 2024-09-11T23:24:37.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.927+0000 7fa2c9cb8640 1 -- 172.21.15.43:0/338817019 wait complete. 2024-09-11T23:24:37.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.928+0000 7fa2c9cb8640 1 Processor -- start 2024-09-11T23:24:37.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.929+0000 7fa2c9cb8640 1 -- start start 2024-09-11T23:24:37.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.929+0000 7fa2c9cb8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2c4101de0 0x7fa2c40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:37.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.929+0000 7fa2c9cb8640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2c4102790 0x7fa2c4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:37.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.929+0000 7fa2c9cb8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2c4106f10 0x7fa2c4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:37.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.929+0000 7fa2c9cb8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c4111f70 con 0x7fa2c4106f10 2024-09-11T23:24:37.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.929+0000 7fa2c9cb8640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c4111df0 con 0x7fa2c4102790 2024-09-11T23:24:37.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.929+0000 7fa2c9cb8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c41120f0 con 0x7fa2c4101de0 2024-09-11T23:24:37.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.929+0000 7fa2c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2c4106f10 0x7fa2c4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:37.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.929+0000 7fa2c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2c4106f10 0x7fa2c4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44840/0 (socket says 172.21.15.43:44840) 2024-09-11T23:24:37.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.929+0000 7fa2c3fff640 1 -- 172.21.15.43:0/4098294376 learned_addr learned my addr 172.21.15.43:0/4098294376 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:37.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.929+0000 7fa2c37fe640 1 --2- 172.21.15.43:0/4098294376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2c4101de0 0x7fa2c40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:37.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.929+0000 7fa2c2ffd640 1 --2- 172.21.15.43:0/4098294376 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2c4102790 0x7fa2c4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:37.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.929+0000 7fa2c3fff640 1 -- 172.21.15.43:0/4098294376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2c4101de0 msgr2=0x7fa2c40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:37.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.929+0000 7fa2c3fff640 1 --2- 172.21.15.43:0/4098294376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2c4101de0 0x7fa2c40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:37.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.929+0000 7fa2c3fff640 1 -- 172.21.15.43:0/4098294376 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2c4102790 msgr2=0x7fa2c4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:37.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.929+0000 7fa2c3fff640 1 --2- 172.21.15.43:0/4098294376 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2c4102790 0x7fa2c4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:37.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.929+0000 7fa2c3fff640 1 -- 172.21.15.43:0/4098294376 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c41a69d0 con 0x7fa2c4106f10 2024-09-11T23:24:37.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.930+0000 7fa2c3fff640 1 --2- 172.21.15.43:0/4098294376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2c4106f10 0x7fa2c4074f70 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009fd0 tx=0x7fa2bc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:37.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.930+0000 7fa2c0ff9640 1 -- 172.21.15.43:0/4098294376 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc017070 con 0x7fa2c4106f10 2024-09-11T23:24:37.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.930+0000 7fa2c0ff9640 1 -- 172.21.15.43:0/4098294376 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2bc003e50 con 0x7fa2c4106f10 2024-09-11T23:24:37.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.930+0000 7fa2c0ff9640 1 -- 172.21.15.43:0/4098294376 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc0106c0 con 0x7fa2c4106f10 2024-09-11T23:24:37.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.930+0000 7fa2c9cb8640 1 -- 172.21.15.43:0/4098294376 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c41a6cc0 con 0x7fa2c4106f10 2024-09-11T23:24:37.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.930+0000 7fa2c9cb8640 1 -- 172.21.15.43:0/4098294376 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c41a7180 con 0x7fa2c4106f10 2024-09-11T23:24:37.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.931+0000 7fa2c9cb8640 1 -- 172.21.15.43:0/4098294376 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c41021c0 con 0x7fa2c4106f10 2024-09-11T23:24:37.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.932+0000 7fa2c0ff9640 1 -- 172.21.15.43:0/4098294376 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa2bc003970 con 0x7fa2c4106f10 2024-09-11T23:24:37.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.933+0000 7fa2c0ff9640 1 --2- 172.21.15.43:0/4098294376 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa2ac0a2cf0 0x7fa2ac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:37.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.933+0000 7fa2c0ff9640 1 -- 172.21.15.43:0/4098294376 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fa2bc0cf0c0 con 0x7fa2c4106f10 2024-09-11T23:24:37.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.933+0000 7fa2c37fe640 1 --2- 172.21.15.43:0/4098294376 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa2ac0a2cf0 0x7fa2ac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:37.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.934+0000 7fa2c37fe640 1 --2- 172.21.15.43:0/4098294376 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa2ac0a2cf0 0x7fa2ac0a51b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009bd0 tx=0x7fa2b4005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:37.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:37.936+0000 7fa2c0ff9640 1 -- 172.21.15.43:0/4098294376 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa2bc09b630 con 0x7fa2c4106f10 2024-09-11T23:24:38.042 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:24:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:38.159+0000 7fa2c9cb8640 1 -- 172.21.15.43:0/4098294376 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2c410a150 con 0x7fa2c4106f10 2024-09-11T23:24:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:38.160+0000 7fa2c0ff9640 1 -- 172.21.15.43:0/4098294376 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11899 (secure 0 0 0) 0x7fa2bc09f930 con 0x7fa2c4106f10 2024-09-11T23:24:38.162 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:24:38.162 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"btime":"2024-09-11T23:23:19:881806+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:23:19.722053+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/4233140081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4233140081},{"type":"v1","addr":"172.21.15.90:6835","nonce":4233140081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:24:38.162 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2024-09-11T23:24:38.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:38.162+0000 7fa2c9cb8640 1 -- 172.21.15.43:0/4098294376 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa2ac0a2cf0 msgr2=0x7fa2ac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:38.162+0000 7fa2c9cb8640 1 --2- 172.21.15.43:0/4098294376 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa2ac0a2cf0 0x7fa2ac0a51b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009bd0 tx=0x7fa2b4005f90 comp rx=0 tx=0).stop 2024-09-11T23:24:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:38.162+0000 7fa2c9cb8640 1 -- 172.21.15.43:0/4098294376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2c4106f10 msgr2=0x7fa2c4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:38.162+0000 7fa2c9cb8640 1 --2- 172.21.15.43:0/4098294376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2c4106f10 0x7fa2c4074f70 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009fd0 tx=0x7fa2bc00b260 comp rx=0 tx=0).stop 2024-09-11T23:24:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:38.162+0000 7fa2c9cb8640 1 -- 172.21.15.43:0/4098294376 shutdown_connections 2024-09-11T23:24:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:38.162+0000 7fa2c9cb8640 1 --2- 172.21.15.43:0/4098294376 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa2ac0a2cf0 0x7fa2ac0a51b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:38.162+0000 7fa2c9cb8640 1 --2- 172.21.15.43:0/4098294376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2c4106f10 0x7fa2c4074f70 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:38.162+0000 7fa2c9cb8640 1 --2- 172.21.15.43:0/4098294376 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2c4102790 0x7fa2c4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:38.162+0000 7fa2c9cb8640 1 --2- 172.21.15.43:0/4098294376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2c4101de0 0x7fa2c40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:38.162+0000 7fa2c9cb8640 1 -- 172.21.15.43:0/4098294376 >> 172.21.15.43:0/4098294376 conn(0x7fa2c40fd8c0 msgr2=0x7fa2c40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:38.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:38.162+0000 7fa2c9cb8640 1 -- 172.21.15.43:0/4098294376 shutdown_connections 2024-09-11T23:24:38.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:38.162+0000 7fa2c9cb8640 1 -- 172.21.15.43:0/4098294376 wait complete. 2024-09-11T23:24:38.205 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:38 smithi043 ceph-mon[30950]: pgmap v86: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 MiB/s wr, 5.15k op/s 2024-09-11T23:24:38.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:38 smithi090 ceph-mon[35130]: pgmap v86: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 MiB/s wr, 5.15k op/s 2024-09-11T23:24:38.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:38 smithi097 ceph-mon[35562]: pgmap v86: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 MiB/s wr, 5.15k op/s 2024-09-11T23:24:38.896 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 993ae2f2 --release --await' 2024-09-11T23:24:38.896 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 993ae2f2 --release --await 2024-09-11T23:24:39.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:39 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/4098294376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:24:39.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:39 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/4098294376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:24:39.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:39 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/4098294376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:24:40.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:40 smithi090 ceph-mon[35130]: pgmap v87: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 27 MiB/s wr, 5.15k op/s 2024-09-11T23:24:40.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:40 smithi097 ceph-mon[35562]: pgmap v87: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 27 MiB/s wr, 5.15k op/s 2024-09-11T23:24:40.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:40 smithi043 ceph-mon[30950]: pgmap v87: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 27 MiB/s wr, 5.15k op/s 2024-09-11T23:24:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.802+0000 7f9c1687b640 1 -- 172.21.15.43:0/1101437873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c101008d0 msgr2=0x7f9c10100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.802+0000 7f9c1687b640 1 --2- 172.21.15.43:0/1101437873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c101008d0 0x7f9c10100cb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009900 tx=0x7f9c0002a410 comp rx=0 tx=0).stop 2024-09-11T23:24:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.802+0000 7f9c1687b640 1 -- 172.21.15.43:0/1101437873 shutdown_connections 2024-09-11T23:24:40.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.802+0000 7f9c1687b640 1 --2- 172.21.15.43:0/1101437873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c1010e480 0x7f9c10110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.802+0000 7f9c1687b640 1 --2- 172.21.15.43:0/1101437873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c10101280 0x7f9c1010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.802+0000 7f9c1687b640 1 --2- 172.21.15.43:0/1101437873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c101008d0 0x7f9c10100cb0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.802+0000 7f9c1687b640 1 -- 172.21.15.43:0/1101437873 >> 172.21.15.43:0/1101437873 conn(0x7f9c100fc7a0 msgr2=0x7f9c100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:40.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.802+0000 7f9c1687b640 1 -- 172.21.15.43:0/1101437873 shutdown_connections 2024-09-11T23:24:40.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.802+0000 7f9c1687b640 1 -- 172.21.15.43:0/1101437873 wait complete. 2024-09-11T23:24:40.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.802+0000 7f9c1687b640 1 Processor -- start 2024-09-11T23:24:40.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.803+0000 7f9c1687b640 1 -- start start 2024-09-11T23:24:40.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.803+0000 7f9c1687b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c101008d0 0x7f9c1019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:40.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.803+0000 7f9c1687b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c10101280 0x7f9c1019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:40.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.803+0000 7f9c1687b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c1010e480 0x7f9c1019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:40.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.803+0000 7f9c1687b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101141b0 con 0x7f9c10101280 2024-09-11T23:24:40.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.803+0000 7f9c1687b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9c10114030 con 0x7f9c101008d0 2024-09-11T23:24:40.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.803+0000 7f9c1687b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9c10114330 con 0x7f9c1010e480 2024-09-11T23:24:40.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.803+0000 7f9c0f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c10101280 0x7f9c1019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:40.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.803+0000 7f9c0f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c10101280 0x7f9c1019abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44860/0 (socket says 172.21.15.43:44860) 2024-09-11T23:24:40.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.803+0000 7f9c0f7fe640 1 -- 172.21.15.43:0/2470404573 learned_addr learned my addr 172.21.15.43:0/2470404573 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:40.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.803+0000 7f9c0ffff640 1 --2- 172.21.15.43:0/2470404573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c101008d0 0x7f9c1019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.803+0000 7f9c14e18640 1 --2- 172.21.15.43:0/2470404573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c1010e480 0x7f9c1019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.803+0000 7f9c0f7fe640 1 -- 172.21.15.43:0/2470404573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c1010e480 msgr2=0x7f9c1019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.804+0000 7f9c0f7fe640 1 --2- 172.21.15.43:0/2470404573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c1010e480 0x7f9c1019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.804+0000 7f9c0f7fe640 1 -- 172.21.15.43:0/2470404573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c101008d0 msgr2=0x7f9c1019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.804+0000 7f9c0f7fe640 1 --2- 172.21.15.43:0/2470404573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c101008d0 0x7f9c1019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.804+0000 7f9c0f7fe640 1 -- 172.21.15.43:0/2470404573 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1019f640 con 0x7f9c10101280 2024-09-11T23:24:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.804+0000 7f9c0ffff640 1 --2- 172.21.15.43:0/2470404573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c101008d0 0x7f9c1019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:24:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.804+0000 7f9c0f7fe640 1 --2- 172.21.15.43:0/2470404573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c10101280 0x7f9c1019abd0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400c890 tx=0x7f9c0400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.804+0000 7f9c0d7fa640 1 -- 172.21.15.43:0/2470404573 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04004220 con 0x7f9c10101280 2024-09-11T23:24:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.804+0000 7f9c1687b640 1 -- 172.21.15.43:0/2470404573 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1019f930 con 0x7f9c10101280 2024-09-11T23:24:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.804+0000 7f9c0d7fa640 1 -- 172.21.15.43:0/2470404573 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c040043c0 con 0x7f9c10101280 2024-09-11T23:24:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.804+0000 7f9c0d7fa640 1 -- 172.21.15.43:0/2470404573 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04004a00 con 0x7f9c10101280 2024-09-11T23:24:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.804+0000 7f9c1687b640 1 -- 172.21.15.43:0/2470404573 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c101a7210 con 0x7f9c10101280 2024-09-11T23:24:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.806+0000 7f9c0d7fa640 1 -- 172.21.15.43:0/2470404573 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9c040079b0 con 0x7f9c10101280 2024-09-11T23:24:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.807+0000 7f9c18c54740 1 Processor -- start 2024-09-11T23:24:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.807+0000 7f9c18c54740 1 -- start start 2024-09-11T23:24:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.807+0000 7f9c18c54740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb08b40 0x55f9edb07090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.807+0000 7f9c18c54740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9edb075d0 0x55f9edb07a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.807+0000 7f9c18c54740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb12370 0x55f9edb14760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.807+0000 7f9c18c54740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f9eda6d2f0 con 0x55f9edb08b40 2024-09-11T23:24:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.807+0000 7f9c18c54740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f9eda14cf0 con 0x55f9edb12370 2024-09-11T23:24:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.807+0000 7f9c18c54740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f9eda692b0 con 0x55f9edb075d0 2024-09-11T23:24:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.807+0000 7f9c0d7fa640 1 --2- 172.21.15.43:0/2470404573 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9bf80a2cf0 0x7f9bf80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.807+0000 7f9c14e18640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb08b40 0x55f9edb07090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.807+0000 7f9c14e18640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb08b40 0x55f9edb07090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44862/0 (socket says 172.21.15.43:44862) 2024-09-11T23:24:40.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.807+0000 7f9c14e18640 1 -- 172.21.15.43:0/3481903099 learned_addr learned my addr 172.21.15.43:0/3481903099 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:40.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.807+0000 7f9c0ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9edb075d0 0x55f9edb07a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:40.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.807+0000 7f9c14e18640 1 --2- 172.21.15.43:0/2470404573 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9bf80a2cf0 0x7f9bf80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:40.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.807+0000 7f9c0f7fe640 1 --2- 172.21.15.43:0/3481903099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb12370 0x55f9edb14760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:40.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.808+0000 7f9c0d7fa640 1 -- 172.21.15.43:0/2470404573 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f9c040cf1b0 con 0x7f9c10101280 2024-09-11T23:24:40.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.808+0000 7f9c14e18640 1 -- 172.21.15.43:0/3481903099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9edb075d0 msgr2=0x55f9edb07a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:40.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.808+0000 7f9c14e18640 1 --2- 172.21.15.43:0/3481903099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9edb075d0 0x55f9edb07a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.808+0000 7f9c14e18640 1 -- 172.21.15.43:0/3481903099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb12370 msgr2=0x55f9edb14760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:40.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.808+0000 7f9c14e18640 1 --2- 172.21.15.43:0/3481903099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb12370 0x55f9edb14760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.808+0000 7f9c14e18640 1 -- 172.21.15.43:0/3481903099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9ed9f93a0 con 0x55f9edb08b40 2024-09-11T23:24:40.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.808+0000 7f9c0ffff640 1 --2- 172.21.15.43:0/3481903099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9edb075d0 0x55f9edb07a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:24:40.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.808+0000 7f9c0f7fe640 1 --2- 172.21.15.43:0/3481903099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb12370 0x55f9edb14760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:24:40.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.808+0000 7f9c14e18640 1 --2- 172.21.15.43:0/3481903099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb08b40 0x55f9edb07090 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9c08012cb0 tx=0x7f9c08010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98dd9111114a0a42 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.808+0000 7f9c14e18640 1 --2- 172.21.15.43:0/2470404573 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9bf80a2cf0 0x7f9bf80a51b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009fd0 tx=0x7f9c0800c000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.808+0000 7f9bf67fc640 1 -- 172.21.15.43:0/3481903099 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08023070 con 0x55f9edb08b40 2024-09-11T23:24:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.808+0000 7f9bf67fc640 1 -- 172.21.15.43:0/3481903099 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c080124f0 con 0x55f9edb08b40 2024-09-11T23:24:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.808+0000 7f9bf67fc640 1 -- 172.21.15.43:0/3481903099 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0800ce20 con 0x55f9edb08b40 2024-09-11T23:24:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.809+0000 7f9c18c54740 1 -- 172.21.15.43:0/3481903099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb08b40 msgr2=0x55f9edb07090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.809+0000 7f9c18c54740 1 --2- 172.21.15.43:0/3481903099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb08b40 0x55f9edb07090 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9c08012cb0 tx=0x7f9c08010740 comp rx=0 tx=0).stop 2024-09-11T23:24:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.809+0000 7f9c18c54740 1 -- 172.21.15.43:0/3481903099 shutdown_connections 2024-09-11T23:24:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.809+0000 7f9c18c54740 1 --2- 172.21.15.43:0/3481903099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb12370 0x55f9edb14760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.809+0000 7f9c18c54740 1 --2- 172.21.15.43:0/3481903099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9edb075d0 0x55f9edb07a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.809+0000 7f9c18c54740 1 --2- 172.21.15.43:0/3481903099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb08b40 0x55f9edb07090 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.809+0000 7f9c18c54740 1 -- 172.21.15.43:0/3481903099 >> 172.21.15.43:0/3481903099 conn(0x55f9edad0310 msgr2=0x55f9edb03c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.809+0000 7f9c18c54740 1 -- 172.21.15.43:0/3481903099 shutdown_connections 2024-09-11T23:24:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.809+0000 7f9c18c54740 1 -- 172.21.15.43:0/3481903099 wait complete. 2024-09-11T23:24:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.809+0000 7f9c18c54740 1 -- start start 2024-09-11T23:24:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.810+0000 7f9c18c54740 1 Processor -- start 2024-09-11T23:24:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.810+0000 7f9c18c54740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb075d0 0x55f9edb11da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.810+0000 7f9c18c54740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb08b40 0x55f9edb0f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.810+0000 7f9c18c54740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9edb12370 0x55f9edb0f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.810+0000 7f9c18c54740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f9edb15010 con 0x55f9edb075d0 2024-09-11T23:24:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.810+0000 7f9c18c54740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f9eda28060 con 0x55f9edb08b40 2024-09-11T23:24:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.810+0000 7f9c18c54740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f9edb15190 con 0x55f9edb12370 2024-09-11T23:24:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.810+0000 7f9c18c54740 10 client.0 _ll_register_callbacks cb 0x55f9ed77b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.810+0000 7f9c0ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb075d0 0x55f9edb11da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.810+0000 7f9c0ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb075d0 0x55f9edb11da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44878/0 (socket says 172.21.15.43:44878) 2024-09-11T23:24:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.810+0000 7f9c0ffff640 1 -- 172.21.15.43:0/1894283569 learned_addr learned my addr 172.21.15.43:0/1894283569 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:40.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.810+0000 7f9c14e18640 1 --2- 172.21.15.43:0/1894283569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9edb12370 0x55f9edb0f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:40.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.810+0000 7f9c0f7fe640 1 --2- 172.21.15.43:0/1894283569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb08b40 0x55f9edb0f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:40.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.810+0000 7f9c0ffff640 1 -- 172.21.15.43:0/1894283569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9edb12370 msgr2=0x55f9edb0f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:40.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.810+0000 7f9c0ffff640 1 --2- 172.21.15.43:0/1894283569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9edb12370 0x55f9edb0f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.811+0000 7f9c0ffff640 1 -- 172.21.15.43:0/1894283569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb08b40 msgr2=0x55f9edb0f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:40.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.811+0000 7f9c0ffff640 1 --2- 172.21.15.43:0/1894283569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb08b40 0x55f9edb0f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.811+0000 7f9c0ffff640 1 -- 172.21.15.43:0/1894283569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9ed7de3c0 con 0x55f9edb075d0 2024-09-11T23:24:40.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.811+0000 7f9c0ffff640 1 --2- 172.21.15.43:0/1894283569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb075d0 0x55f9edb11da0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009900 tx=0x7f9c00031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bddd1469037fd4d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:40.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.811+0000 7f9bef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:24:40.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.811+0000 7f9bef7fe640 1 -- 172.21.15.43:0/1894283569 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00035040 con 0x55f9edb075d0 2024-09-11T23:24:40.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.811+0000 7f9bef7fe640 1 -- 172.21.15.43:0/1894283569 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f9c00005230 con 0x55f9edb075d0 2024-09-11T23:24:40.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.811+0000 7f9c18c54740 1 -- 172.21.15.43:0/1894283569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c080124f0 con 0x55f9edb075d0 2024-09-11T23:24:40.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.811+0000 7f9bef7fe640 1 -- 172.21.15.43:0/1894283569 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9c00004490 con 0x55f9edb075d0 2024-09-11T23:24:40.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.811+0000 7f9c18c54740 10 client.15924 fetch_fsmap learned FSMap version 26 2024-09-11T23:24:40.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.811+0000 7f9c18c54740 1 -- 172.21.15.43:0/1894283569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f9edb20af0 con 0x55f9edb075d0 2024-09-11T23:24:40.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.811+0000 7f9bef7fe640 1 -- 172.21.15.43:0/1894283569 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c080124f0 con 0x55f9edb075d0 2024-09-11T23:24:40.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.812+0000 7f9bef7fe640 1 -- 172.21.15.43:0/1894283569 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9c000036f0 con 0x55f9edb075d0 2024-09-11T23:24:40.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.812+0000 7f9c18c54740 10 client.15924 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:24:40.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.812+0000 7f9c18c54740 10 client.15924 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T23:24:40.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.812+0000 7f9c18c54740 20 client.15924 populate_metadata read hostname 'smithi043' 2024-09-11T23:24:40.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.812+0000 7f9c18c54740 1 --2- 172.21.15.43:0/1894283569 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55f9edb21f00 0x55f9edb24310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:40.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.812+0000 7f9c18c54740 4 client.15924 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:24:40.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.812+0000 7f9c18c54740 1 -- 172.21.15.43:0/1894283569 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9eda7a860 con 0x55f9edb21f00 2024-09-11T23:24:40.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.812+0000 7f9c0f7fe640 1 --2- 172.21.15.43:0/1894283569 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55f9edb21f00 0x55f9edb24310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:40.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.812+0000 7f9c0f7fe640 1 --2- 172.21.15.43:0/1894283569 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55f9edb21f00 0x55f9edb24310 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db80caf5db19fec1 server_cookie=8908c8d89b533030 in_seq=0 out_seq=0 2024-09-11T23:24:40.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.812+0000 7f9bef7fe640 10 client.15924 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:24:40.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.813+0000 7f9bef7fe640 1 -- 172.21.15.43:0/1894283569 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9eda7a860 con 0x55f9edb21f00 2024-09-11T23:24:40.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.813+0000 7f9bef7fe640 10 client.15924 handle_command_reply: tid=0 2024-09-11T23:24:40.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.813+0000 7f9c18c54740 1 client.15924 shutdown 2024-09-11T23:24:40.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.813+0000 7f9c18c54740 1 -- 172.21.15.43:0/1894283569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb075d0 msgr2=0x55f9edb11da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:40.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.813+0000 7f9c18c54740 1 --2- 172.21.15.43:0/1894283569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb075d0 0x55f9edb11da0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009900 tx=0x7f9c00031af0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.813+0000 7f9c18c54740 1 -- 172.21.15.43:0/1894283569 shutdown_connections 2024-09-11T23:24:40.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.813+0000 7f9c18c54740 1 --2- 172.21.15.43:0/1894283569 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55f9edb21f00 0x55f9edb24310 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.813+0000 7f9c18c54740 1 --2- 172.21.15.43:0/1894283569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9edb12370 0x55f9edb0f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.813+0000 7f9bef7fe640 0 client.15924 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:24:40.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.813+0000 7f9c18c54740 1 --2- 172.21.15.43:0/1894283569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb08b40 0x55f9edb0f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.813+0000 7f9c18c54740 1 --2- 172.21.15.43:0/1894283569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb075d0 0x55f9edb11da0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.813+0000 7f9c18c54740 1 -- 172.21.15.43:0/1894283569 >> 172.21.15.43:0/1894283569 conn(0x55f9edad0310 msgr2=0x55f9edb09960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:40.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.814+0000 7f9c18c54740 1 -- 172.21.15.43:0/1894283569 shutdown_connections 2024-09-11T23:24:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.814+0000 7f9c18c54740 1 -- 172.21.15.43:0/1894283569 wait complete. 2024-09-11T23:24:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.814+0000 7f9c18c54740 20 client.15924 trim_cache size 0 max 16384 2024-09-11T23:24:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.827+0000 7f9c18c54740 1 Processor -- start 2024-09-11T23:24:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.827+0000 7f9c18c54740 1 -- start start 2024-09-11T23:24:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.827+0000 7f9c18c54740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9edb075d0 0x55f9edb0e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.827+0000 7f9c18c54740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb08b40 0x55f9edb29be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.827+0000 7f9c18c54740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb12370 0x55f9edb2a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.827+0000 7f9c18c54740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101b2410 con 0x55f9edb12370 2024-09-11T23:24:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.827+0000 7f9c18c54740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101afce0 con 0x55f9edb08b40 2024-09-11T23:24:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.827+0000 7f9c18c54740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101adb30 con 0x55f9edb075d0 2024-09-11T23:24:40.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.827+0000 7f9c14e18640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb12370 0x55f9edb2a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:40.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.827+0000 7f9c14e18640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb12370 0x55f9edb2a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44884/0 (socket says 172.21.15.43:44884) 2024-09-11T23:24:40.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.827+0000 7f9c14e18640 1 -- 172.21.15.43:0/1990608336 learned_addr learned my addr 172.21.15.43:0/1990608336 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:40.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.827+0000 7f9c0ffff640 1 --2- 172.21.15.43:0/1990608336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9edb075d0 0x55f9edb0e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:40.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.827+0000 7f9c0f7fe640 1 --2- 172.21.15.43:0/1990608336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb08b40 0x55f9edb29be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:40.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.827+0000 7f9c14e18640 1 -- 172.21.15.43:0/1990608336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9edb075d0 msgr2=0x55f9edb0e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:40.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.827+0000 7f9c14e18640 1 --2- 172.21.15.43:0/1990608336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9edb075d0 0x55f9edb0e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.827+0000 7f9c14e18640 1 -- 172.21.15.43:0/1990608336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb08b40 msgr2=0x55f9edb29be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:40.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.827+0000 7f9c14e18640 1 --2- 172.21.15.43:0/1990608336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb08b40 0x55f9edb29be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.827+0000 7f9c14e18640 1 -- 172.21.15.43:0/1990608336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9edb07be0 con 0x55f9edb12370 2024-09-11T23:24:40.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.828+0000 7f9c0f7fe640 1 --2- 172.21.15.43:0/1990608336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb08b40 0x55f9edb29be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:24:40.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.828+0000 7f9c14e18640 1 --2- 172.21.15.43:0/1990608336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb12370 0x55f9edb2a120 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9c080122d0 tx=0x7f9c08012260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd7eb7a9deb696a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.828+0000 7f9bf5ffb640 1 -- 172.21.15.43:0/1990608336 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08023040 con 0x55f9edb12370 2024-09-11T23:24:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.828+0000 7f9bf5ffb640 1 -- 172.21.15.43:0/1990608336 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c080127c0 con 0x55f9edb12370 2024-09-11T23:24:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.828+0000 7f9bf5ffb640 1 -- 172.21.15.43:0/1990608336 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c080052a0 con 0x55f9edb12370 2024-09-11T23:24:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.829+0000 7f9c18c54740 1 -- 172.21.15.43:0/1990608336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb12370 msgr2=0x55f9edb2a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.829+0000 7f9c18c54740 1 --2- 172.21.15.43:0/1990608336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb12370 0x55f9edb2a120 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9c080122d0 tx=0x7f9c08012260 comp rx=0 tx=0).stop 2024-09-11T23:24:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.829+0000 7f9c18c54740 1 -- 172.21.15.43:0/1990608336 shutdown_connections 2024-09-11T23:24:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.829+0000 7f9c18c54740 1 --2- 172.21.15.43:0/1990608336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb12370 0x55f9edb2a120 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.829+0000 7f9c18c54740 1 --2- 172.21.15.43:0/1990608336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb08b40 0x55f9edb29be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.829+0000 7f9c18c54740 1 --2- 172.21.15.43:0/1990608336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9edb075d0 0x55f9edb0e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.829+0000 7f9c18c54740 1 -- 172.21.15.43:0/1990608336 >> 172.21.15.43:0/1990608336 conn(0x55f9edad0310 msgr2=0x55f9edb0d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.829+0000 7f9c18c54740 1 -- 172.21.15.43:0/1990608336 shutdown_connections 2024-09-11T23:24:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.829+0000 7f9c18c54740 1 -- 172.21.15.43:0/1990608336 wait complete. 2024-09-11T23:24:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.829+0000 7f9c18c54740 1 -- start start 2024-09-11T23:24:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.829+0000 7f9c18c54740 1 Processor -- start 2024-09-11T23:24:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.830+0000 7f9c18c54740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb075d0 0x55f9edb1abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.830+0000 7f9c18c54740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb08b40 0x55f9edb15ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.830+0000 7f9c18c54740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9edb12370 0x55f9edb16430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.830+0000 7f9c18c54740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101ae1a0 con 0x55f9edb075d0 2024-09-11T23:24:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.830+0000 7f9c18c54740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101af610 con 0x55f9edb08b40 2024-09-11T23:24:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.830+0000 7f9c18c54740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101ae870 con 0x55f9edb12370 2024-09-11T23:24:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.830+0000 7f9c0ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb075d0 0x55f9edb1abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.830+0000 7f9c0ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb075d0 0x55f9edb1abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44888/0 (socket says 172.21.15.43:44888) 2024-09-11T23:24:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.830+0000 7f9c18c54740 10 client.0 _ll_register_callbacks cb 0x7f9c00031780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.830+0000 7f9c0ffff640 1 -- 172.21.15.43:0/3765224160 learned_addr learned my addr 172.21.15.43:0/3765224160 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.830+0000 7f9c14e18640 1 --2- 172.21.15.43:0/3765224160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9edb12370 0x55f9edb16430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.830+0000 7f9c0f7fe640 1 --2- 172.21.15.43:0/3765224160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb08b40 0x55f9edb15ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.830+0000 7f9c0ffff640 1 -- 172.21.15.43:0/3765224160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9edb12370 msgr2=0x55f9edb16430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.830+0000 7f9c0ffff640 1 --2- 172.21.15.43:0/3765224160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9edb12370 0x55f9edb16430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.830+0000 7f9c0ffff640 1 -- 172.21.15.43:0/3765224160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb08b40 msgr2=0x55f9edb15ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.830+0000 7f9c0ffff640 1 --2- 172.21.15.43:0/3765224160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb08b40 0x55f9edb15ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.830+0000 7f9c0ffff640 1 -- 172.21.15.43:0/3765224160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9edb16c60 con 0x55f9edb075d0 2024-09-11T23:24:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.830+0000 7f9c0ffff640 1 --2- 172.21.15.43:0/3765224160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb075d0 0x55f9edb1abc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9c000059d0 tx=0x7f9c00032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f399859d79c40409 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.830+0000 7f9bef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:24:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.831+0000 7f9bef7fe640 1 -- 172.21.15.43:0/3765224160 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00035030 con 0x55f9edb075d0 2024-09-11T23:24:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.831+0000 7f9bef7fe640 1 -- 172.21.15.43:0/3765224160 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f9c00005450 con 0x55f9edb075d0 2024-09-11T23:24:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.831+0000 7f9c18c54740 1 -- 172.21.15.43:0/3765224160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c080127c0 con 0x55f9edb075d0 2024-09-11T23:24:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.831+0000 7f9bef7fe640 1 -- 172.21.15.43:0/3765224160 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9c00005650 con 0x55f9edb075d0 2024-09-11T23:24:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.831+0000 7f9c18c54740 10 client.15930 fetch_fsmap learned FSMap version 26 2024-09-11T23:24:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.831+0000 7f9c18c54740 1 -- 172.21.15.43:0/3765224160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f9edb8cb80 con 0x55f9edb075d0 2024-09-11T23:24:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.831+0000 7f9bef7fe640 1 -- 172.21.15.43:0/3765224160 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c080127c0 con 0x55f9edb075d0 2024-09-11T23:24:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.831+0000 7f9bef7fe640 1 -- 172.21.15.43:0/3765224160 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9c0004d5d0 con 0x55f9edb075d0 2024-09-11T23:24:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.831+0000 7f9c18c54740 10 client.15930 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:24:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.831+0000 7f9c18c54740 10 client.15930 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T23:24:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.831+0000 7f9c18c54740 20 client.15930 populate_metadata read hostname 'smithi043' 2024-09-11T23:24:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.831+0000 7f9c18c54740 1 --2- 172.21.15.43:0/3765224160 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55f9edb21f00 0x55f9edb8f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.831+0000 7f9c18c54740 4 client.15930 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:24:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.831+0000 7f9c18c54740 1 -- 172.21.15.43:0/3765224160 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9eda74d70 con 0x55f9edb21f00 2024-09-11T23:24:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.831+0000 7f9c0f7fe640 1 --2- 172.21.15.43:0/3765224160 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55f9edb21f00 0x55f9edb8f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.832+0000 7f9c0f7fe640 1 --2- 172.21.15.43:0/3765224160 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55f9edb21f00 0x55f9edb8f280 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6a3fb653511fb8 server_cookie=d991de4888ecca1e in_seq=0 out_seq=0 2024-09-11T23:24:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.832+0000 7f9bef7fe640 10 client.15930 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:24:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.832+0000 7f9bef7fe640 1 -- 172.21.15.43:0/3765224160 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9eda74d70 con 0x55f9edb21f00 2024-09-11T23:24:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.832+0000 7f9bef7fe640 10 client.15930 handle_command_reply: tid=0 2024-09-11T23:24:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.832+0000 7f9c18c54740 1 client.15930 shutdown 2024-09-11T23:24:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.832+0000 7f9c18c54740 1 -- 172.21.15.43:0/3765224160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb075d0 msgr2=0x55f9edb1abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.832+0000 7f9c18c54740 1 --2- 172.21.15.43:0/3765224160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb075d0 0x55f9edb1abc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9c000059d0 tx=0x7f9c00032fc0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.874 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:24:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.832+0000 7f9c18c54740 1 -- 172.21.15.43:0/3765224160 shutdown_connections 2024-09-11T23:24:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.832+0000 7f9c18c54740 1 --2- 172.21.15.43:0/3765224160 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55f9edb21f00 0x55f9edb8f280 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.832+0000 7f9c18c54740 1 --2- 172.21.15.43:0/3765224160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9edb12370 0x55f9edb16430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.832+0000 7f9c18c54740 1 --2- 172.21.15.43:0/3765224160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9edb08b40 0x55f9edb15ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.832+0000 7f9bef7fe640 0 client.15930 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:24:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.832+0000 7f9c18c54740 1 --2- 172.21.15.43:0/3765224160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9edb075d0 0x55f9edb1abc0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.832+0000 7f9c18c54740 1 -- 172.21.15.43:0/3765224160 >> 172.21.15.43:0/3765224160 conn(0x55f9edad0310 msgr2=0x55f9edb0e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.832+0000 7f9c18c54740 1 -- 172.21.15.43:0/3765224160 shutdown_connections 2024-09-11T23:24:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.832+0000 7f9c18c54740 1 -- 172.21.15.43:0/3765224160 wait complete. 2024-09-11T23:24:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.833+0000 7f9c18c54740 20 client.15930 trim_cache size 0 max 16384 2024-09-11T23:24:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.833+0000 7f9c1687b640 1 -- 172.21.15.43:0/2470404573 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9bf80a2cf0 msgr2=0x7f9bf80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.833+0000 7f9c1687b640 1 --2- 172.21.15.43:0/2470404573 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9bf80a2cf0 0x7f9bf80a51b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009fd0 tx=0x7f9c0800c000 comp rx=0 tx=0).stop 2024-09-11T23:24:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.833+0000 7f9c1687b640 1 -- 172.21.15.43:0/2470404573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c10101280 msgr2=0x7f9c1019abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.833+0000 7f9c1687b640 1 --2- 172.21.15.43:0/2470404573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c10101280 0x7f9c1019abd0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400c890 tx=0x7f9c0400cd50 comp rx=0 tx=0).stop 2024-09-11T23:24:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.833+0000 7f9c1687b640 1 -- 172.21.15.43:0/2470404573 shutdown_connections 2024-09-11T23:24:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.833+0000 7f9c1687b640 1 --2- 172.21.15.43:0/2470404573 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9bf80a2cf0 0x7f9bf80a51b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.833+0000 7f9c1687b640 1 --2- 172.21.15.43:0/2470404573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c1010e480 0x7f9c1019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.833+0000 7f9c1687b640 1 --2- 172.21.15.43:0/2470404573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c10101280 0x7f9c1019abd0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.833+0000 7f9c1687b640 1 --2- 172.21.15.43:0/2470404573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c101008d0 0x7f9c1019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.833+0000 7f9c1687b640 1 -- 172.21.15.43:0/2470404573 >> 172.21.15.43:0/2470404573 conn(0x7f9c100fc7a0 msgr2=0x7f9c100feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.833+0000 7f9c1687b640 1 -- 172.21.15.43:0/2470404573 shutdown_connections 2024-09-11T23:24:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:39.833+0000 7f9c1687b640 1 -- 172.21.15.43:0/2470404573 wait complete. 2024-09-11T23:24:41.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:41 smithi090 ceph-mon[35130]: pgmap v88: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 794 op/s 2024-09-11T23:24:41.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:41 smithi097 ceph-mon[35562]: pgmap v88: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 794 op/s 2024-09-11T23:24:41.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:41 smithi043 ceph-mon[30950]: pgmap v88: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 794 op/s 2024-09-11T23:24:43.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:43 smithi043 ceph-mon[30950]: pgmap v89: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 874 KiB/s wr, 209 op/s 2024-09-11T23:24:43.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:43 smithi090 ceph-mon[35130]: pgmap v89: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 874 KiB/s wr, 209 op/s 2024-09-11T23:24:43.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:43 smithi097 ceph-mon[35562]: pgmap v89: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 874 KiB/s wr, 209 op/s 2024-09-11T23:24:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.132+0000 7f6c9762e640 1 -- 172.21.15.43:0/562419336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c9010e310 msgr2=0x7f6c90110760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.132+0000 7f6c9762e640 1 --2- 172.21.15.43:0/562419336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c9010e310 0x7f6c90110760 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009ca0 tx=0x7f6c8802a950 comp rx=0 tx=0).stop 2024-09-11T23:24:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.132+0000 7f6c9762e640 1 -- 172.21.15.43:0/562419336 shutdown_connections 2024-09-11T23:24:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.132+0000 7f6c9762e640 1 --2- 172.21.15.43:0/562419336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c9010e310 0x7f6c90110760 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.132+0000 7f6c9762e640 1 --2- 172.21.15.43:0/562419336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c90074f40 0x7f6c9010ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.132+0000 7f6c9762e640 1 --2- 172.21.15.43:0/562419336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c90074620 0x7f6c90074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.132+0000 7f6c9762e640 1 -- 172.21.15.43:0/562419336 >> 172.21.15.43:0/562419336 conn(0x7f6c900fd7d0 msgr2=0x7f6c900ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.132+0000 7f6c9762e640 1 -- 172.21.15.43:0/562419336 shutdown_connections 2024-09-11T23:24:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.132+0000 7f6c9762e640 1 -- 172.21.15.43:0/562419336 wait complete. 2024-09-11T23:24:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.132+0000 7f6c9762e640 1 Processor -- start 2024-09-11T23:24:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.133+0000 7f6c9762e640 1 -- start start 2024-09-11T23:24:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.133+0000 7f6c9762e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c90074620 0x7f6c9019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.133+0000 7f6c9762e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c90074f40 0x7f6c9019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.133+0000 7f6c9762e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c9010e310 0x7f6c9019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.133+0000 7f6c9762e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6c901140e0 con 0x7f6c90074620 2024-09-11T23:24:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.133+0000 7f6c9762e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6c90113f60 con 0x7f6c90074f40 2024-09-11T23:24:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.133+0000 7f6c9762e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6c90114260 con 0x7f6c9010e310 2024-09-11T23:24:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.133+0000 7f6c953ca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c90074620 0x7f6c9019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.133+0000 7f6c953ca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c90074620 0x7f6c9019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48042/0 (socket says 172.21.15.43:48042) 2024-09-11T23:24:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.133+0000 7f6c953ca640 1 -- 172.21.15.43:0/805605587 learned_addr learned my addr 172.21.15.43:0/805605587 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.133+0000 7f6c95bcb640 1 --2- 172.21.15.43:0/805605587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c9010e310 0x7f6c9019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.133+0000 7f6c94bc9640 1 --2- 172.21.15.43:0/805605587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c90074f40 0x7f6c9019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.133+0000 7f6c953ca640 1 -- 172.21.15.43:0/805605587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c9010e310 msgr2=0x7f6c9019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.133+0000 7f6c953ca640 1 --2- 172.21.15.43:0/805605587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c9010e310 0x7f6c9019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.133+0000 7f6c953ca640 1 -- 172.21.15.43:0/805605587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c90074f40 msgr2=0x7f6c9019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.133+0000 7f6c953ca640 1 --2- 172.21.15.43:0/805605587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c90074f40 0x7f6c9019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.133+0000 7f6c953ca640 1 -- 172.21.15.43:0/805605587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c9019f560 con 0x7f6c90074620 2024-09-11T23:24:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.133+0000 7f6c94bc9640 1 --2- 172.21.15.43:0/805605587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c90074f40 0x7f6c9019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:24:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.134+0000 7f6c953ca640 1 --2- 172.21.15.43:0/805605587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c90074620 0x7f6c9019a570 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00d950 tx=0x7f6c8c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.134+0000 7f6c867fc640 1 -- 172.21.15.43:0/805605587 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c017070 con 0x7f6c90074620 2024-09-11T23:24:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.134+0000 7f6c9762e640 1 -- 172.21.15.43:0/805605587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c9019f850 con 0x7f6c90074620 2024-09-11T23:24:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.134+0000 7f6c867fc640 1 -- 172.21.15.43:0/805605587 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c8c009c30 con 0x7f6c90074620 2024-09-11T23:24:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.134+0000 7f6c867fc640 1 -- 172.21.15.43:0/805605587 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c00f6b0 con 0x7f6c90074620 2024-09-11T23:24:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.134+0000 7f6c9762e640 1 -- 172.21.15.43:0/805605587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c901a70e0 con 0x7f6c90074620 2024-09-11T23:24:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.136+0000 7f6c867fc640 1 -- 172.21.15.43:0/805605587 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6c8c0109d0 con 0x7f6c90074620 2024-09-11T23:24:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.137+0000 7f6c9a3cb740 1 Processor -- start 2024-09-11T23:24:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.137+0000 7f6c9a3cb740 1 -- start start 2024-09-11T23:24:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.137+0000 7f6c9a3cb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa1f6f7af0 0x55aa1f6f7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.137+0000 7f6c9a3cb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f6f8410 0x55aa1f702770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.137+0000 7f6c867fc640 1 --2- 172.21.15.43:0/805605587 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6c7c0a2cf0 0x7f6c7c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.137+0000 7f6c9a3cb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f702cb0 0x55aa1f7050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.137+0000 7f6c9a3cb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aa1f65c2f0 con 0x55aa1f6f8410 2024-09-11T23:24:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.137+0000 7f6c9a3cb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55aa1f604270 con 0x55aa1f6f7af0 2024-09-11T23:24:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.137+0000 7f6c9a3cb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55aa1f6582b0 con 0x55aa1f702cb0 2024-09-11T23:24:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.137+0000 7f6c94bc9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f6f8410 0x55aa1f702770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.137+0000 7f6c94bc9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f6f8410 0x55aa1f702770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48048/0 (socket says 172.21.15.43:48048) 2024-09-11T23:24:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.137+0000 7f6c94bc9640 1 -- 172.21.15.43:0/2748537677 learned_addr learned my addr 172.21.15.43:0/2748537677 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.137+0000 7f6c867fc640 1 -- 172.21.15.43:0/805605587 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f6c8c0cf0c0 con 0x7f6c90074620 2024-09-11T23:24:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.137+0000 7f6c953ca640 1 --2- 172.21.15.43:0/805605587 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6c7c0a2cf0 0x7f6c7c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.137+0000 7f6c95bcb640 1 --2- 172.21.15.43:0/2748537677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa1f6f7af0 0x55aa1f6f7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.137+0000 7f6c95bcb640 1 --2- 172.21.15.43:0/2748537677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f702cb0 0x55aa1f7050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.138+0000 7f6c94bc9640 1 -- 172.21.15.43:0/2748537677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f702cb0 msgr2=0x55aa1f7050a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.138+0000 7f6c94bc9640 1 --2- 172.21.15.43:0/2748537677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f702cb0 0x55aa1f7050a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.138+0000 7f6c94bc9640 1 -- 172.21.15.43:0/2748537677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa1f6f7af0 msgr2=0x55aa1f6f7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.138+0000 7f6c94bc9640 1 --2- 172.21.15.43:0/2748537677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa1f6f7af0 0x55aa1f6f7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.138+0000 7f6c94bc9640 1 -- 172.21.15.43:0/2748537677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa1f603340 con 0x55aa1f6f8410 2024-09-11T23:24:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.138+0000 7f6c95bcb640 1 --2- 172.21.15.43:0/2748537677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f702cb0 0x55aa1f7050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:24:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.138+0000 7f6c953ca640 1 --2- 172.21.15.43:0/805605587 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6c7c0a2cf0 0x7f6c7c0a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c000c00 tx=0x7f6c8c010d70 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.138+0000 7f6c94bc9640 1 --2- 172.21.15.43:0/2748537677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f6f8410 0x55aa1f702770 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000bb60 tx=0x7f6c8000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ffe2d5bf0b0ebb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.138+0000 7f6c7b7fe640 1 -- 172.21.15.43:0/2748537677 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8000be60 con 0x55aa1f6f8410 2024-09-11T23:24:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.138+0000 7f6c7b7fe640 1 -- 172.21.15.43:0/2748537677 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c80017b00 con 0x55aa1f6f8410 2024-09-11T23:24:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.138+0000 7f6c7b7fe640 1 -- 172.21.15.43:0/2748537677 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8000fc50 con 0x55aa1f6f8410 2024-09-11T23:24:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.139+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/2748537677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f6f8410 msgr2=0x55aa1f702770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.139+0000 7f6c9a3cb740 1 --2- 172.21.15.43:0/2748537677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f6f8410 0x55aa1f702770 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000bb60 tx=0x7f6c8000d710 comp rx=0 tx=0).stop 2024-09-11T23:24:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.139+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/2748537677 shutdown_connections 2024-09-11T23:24:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.139+0000 7f6c9a3cb740 1 --2- 172.21.15.43:0/2748537677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f702cb0 0x55aa1f7050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.139+0000 7f6c9a3cb740 1 --2- 172.21.15.43:0/2748537677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f6f8410 0x55aa1f702770 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.139+0000 7f6c9a3cb740 1 --2- 172.21.15.43:0/2748537677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa1f6f7af0 0x55aa1f6f7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.139+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/2748537677 >> 172.21.15.43:0/2748537677 conn(0x55aa1f6f4060 msgr2=0x55aa1f6f4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.139+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/2748537677 shutdown_connections 2024-09-11T23:24:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.139+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/2748537677 wait complete. 2024-09-11T23:24:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.139+0000 7f6c9a3cb740 1 -- start start 2024-09-11T23:24:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.139+0000 7f6c9a3cb740 1 Processor -- start 2024-09-11T23:24:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.140+0000 7f6c9a3cb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f6f7af0 0x55aa1f6ffb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.140+0000 7f6c9a3cb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa1f6f8410 0x55aa1f7000c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.140+0000 7f6c9a3cb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f702cb0 0x55aa1f700600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.140+0000 7f6c9a3cb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aa1f705880 con 0x55aa1f702cb0 2024-09-11T23:24:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.140+0000 7f6c9a3cb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55aa1f5e5290 con 0x55aa1f6f8410 2024-09-11T23:24:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.140+0000 7f6c9a3cb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55aa1f705a00 con 0x55aa1f6f7af0 2024-09-11T23:24:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.140+0000 7f6c9a3cb740 10 client.0 _ll_register_callbacks cb 0x55aa1f616140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.140+0000 7f6c95bcb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f702cb0 0x55aa1f700600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.140+0000 7f6c95bcb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f702cb0 0x55aa1f700600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48054/0 (socket says 172.21.15.43:48054) 2024-09-11T23:24:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.140+0000 7f6c95bcb640 1 -- 172.21.15.43:0/1373981004 learned_addr learned my addr 172.21.15.43:0/1373981004 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.140+0000 7f6c94bc9640 1 --2- 172.21.15.43:0/1373981004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa1f6f8410 0x55aa1f7000c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.140+0000 7f6c95bcb640 1 --2- 172.21.15.43:0/1373981004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f6f7af0 0x55aa1f6ffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.140+0000 7f6c95bcb640 1 -- 172.21.15.43:0/1373981004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f6f7af0 msgr2=0x55aa1f6ffb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.140+0000 7f6c95bcb640 1 --2- 172.21.15.43:0/1373981004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f6f7af0 0x55aa1f6ffb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.140+0000 7f6c95bcb640 1 -- 172.21.15.43:0/1373981004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa1f6f8410 msgr2=0x55aa1f7000c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.140+0000 7f6c95bcb640 1 --2- 172.21.15.43:0/1373981004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa1f6f8410 0x55aa1f7000c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.140+0000 7f6c95bcb640 1 -- 172.21.15.43:0/1373981004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa1f36f210 con 0x55aa1f702cb0 2024-09-11T23:24:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.141+0000 7f6c95bcb640 1 --2- 172.21.15.43:0/1373981004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f702cb0 0x55aa1f700600 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009ca0 tx=0x7f6c8803d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8557247522dabd23 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.141+0000 7f6c78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:24:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.141+0000 7f6c78ff9640 1 -- 172.21.15.43:0/1373981004 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c88041030 con 0x55aa1f702cb0 2024-09-11T23:24:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.141+0000 7f6c78ff9640 1 -- 172.21.15.43:0/1373981004 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f6c8802fb50 con 0x55aa1f702cb0 2024-09-11T23:24:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.141+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/1373981004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c80017b00 con 0x55aa1f702cb0 2024-09-11T23:24:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.141+0000 7f6c78ff9640 1 -- 172.21.15.43:0/1373981004 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6c88034040 con 0x55aa1f702cb0 2024-09-11T23:24:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.141+0000 7f6c9a3cb740 10 client.15942 fetch_fsmap learned FSMap version 26 2024-09-11T23:24:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.141+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/1373981004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aa1f43b620 con 0x55aa1f702cb0 2024-09-11T23:24:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.141+0000 7f6c78ff9640 1 -- 172.21.15.43:0/1373981004 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c80017b00 con 0x55aa1f702cb0 2024-09-11T23:24:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.141+0000 7f6c78ff9640 1 -- 172.21.15.43:0/1373981004 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6c88047030 con 0x55aa1f702cb0 2024-09-11T23:24:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.141+0000 7f6c9a3cb740 10 client.15942 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:24:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.141+0000 7f6c9a3cb740 10 client.15942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:24:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.141+0000 7f6c9a3cb740 20 client.15942 populate_metadata read hostname 'smithi043' 2024-09-11T23:24:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.142+0000 7f6c9a3cb740 1 --2- 172.21.15.43:0/1373981004 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55aa1f712bf0 0x55aa1f714fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.142+0000 7f6c9a3cb740 4 client.15942 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:24:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.142+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/1373981004 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa1f669a20 con 0x55aa1f712bf0 2024-09-11T23:24:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.142+0000 7f6c94bc9640 1 --2- 172.21.15.43:0/1373981004 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55aa1f712bf0 0x55aa1f714fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.142+0000 7f6c78ff9640 10 client.15942 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.142+0000 7f6c94bc9640 1 --2- 172.21.15.43:0/1373981004 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55aa1f712bf0 0x55aa1f714fe0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ff2841375b1e336 server_cookie=176073741cf61529 in_seq=0 out_seq=0 2024-09-11T23:24:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.143+0000 7f6c78ff9640 1 -- 172.21.15.43:0/1373981004 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aa1f669a20 con 0x55aa1f712bf0 2024-09-11T23:24:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.143+0000 7f6c78ff9640 10 client.15942 handle_command_reply: tid=0 2024-09-11T23:24:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.143+0000 7f6c9a3cb740 1 client.15942 shutdown 2024-09-11T23:24:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.143+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/1373981004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f702cb0 msgr2=0x55aa1f700600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.143+0000 7f6c9a3cb740 1 --2- 172.21.15.43:0/1373981004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f702cb0 0x55aa1f700600 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009ca0 tx=0x7f6c8803d040 comp rx=0 tx=0).stop 2024-09-11T23:24:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.143+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/1373981004 shutdown_connections 2024-09-11T23:24:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.143+0000 7f6c9a3cb740 1 --2- 172.21.15.43:0/1373981004 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55aa1f712bf0 0x55aa1f714fe0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.143+0000 7f6c9a3cb740 1 --2- 172.21.15.43:0/1373981004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f702cb0 0x55aa1f700600 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.143+0000 7f6c78ff9640 0 client.15942 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.143+0000 7f6c9a3cb740 1 --2- 172.21.15.43:0/1373981004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa1f6f8410 0x55aa1f7000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.143+0000 7f6c9a3cb740 1 --2- 172.21.15.43:0/1373981004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f6f7af0 0x55aa1f6ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.143+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/1373981004 >> 172.21.15.43:0/1373981004 conn(0x55aa1f6f4060 msgr2=0x55aa1f6f9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.143+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/1373981004 shutdown_connections 2024-09-11T23:24:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.143+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/1373981004 wait complete. 2024-09-11T23:24:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.144+0000 7f6c9a3cb740 20 client.15942 trim_cache size 0 max 16384 2024-09-11T23:24:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.158+0000 7f6c9a3cb740 1 Processor -- start 2024-09-11T23:24:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.158+0000 7f6c9a3cb740 1 -- start start 2024-09-11T23:24:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.158+0000 7f6c9a3cb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa1f6f7af0 0x55aa1f6fa560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.158+0000 7f6c9a3cb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f6f8410 0x55aa1f700430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.158+0000 7f6c9a3cb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f702cb0 0x55aa1f71a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.158+0000 7f6c9a3cb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6c901b2330 con 0x55aa1f6f8410 2024-09-11T23:24:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.158+0000 7f6c9a3cb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6c901afc00 con 0x55aa1f6f7af0 2024-09-11T23:24:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.158+0000 7f6c9a3cb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6c901ada50 con 0x55aa1f702cb0 2024-09-11T23:24:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.159+0000 7f6c94bc9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f6f8410 0x55aa1f700430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.159+0000 7f6c94bc9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f6f8410 0x55aa1f700430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48066/0 (socket says 172.21.15.43:48066) 2024-09-11T23:24:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.159+0000 7f6c94bc9640 1 -- 172.21.15.43:0/846980550 learned_addr learned my addr 172.21.15.43:0/846980550 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.159+0000 7f6c95bcb640 1 --2- 172.21.15.43:0/846980550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa1f6f7af0 0x55aa1f6fa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.159+0000 7f6c95bcb640 1 --2- 172.21.15.43:0/846980550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f702cb0 0x55aa1f71a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.159+0000 7f6c94bc9640 1 -- 172.21.15.43:0/846980550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f702cb0 msgr2=0x55aa1f71a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.159+0000 7f6c94bc9640 1 --2- 172.21.15.43:0/846980550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f702cb0 0x55aa1f71a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.159+0000 7f6c94bc9640 1 -- 172.21.15.43:0/846980550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa1f6f7af0 msgr2=0x55aa1f6fa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.159+0000 7f6c94bc9640 1 --2- 172.21.15.43:0/846980550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa1f6f7af0 0x55aa1f6fa560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.159+0000 7f6c94bc9640 1 -- 172.21.15.43:0/846980550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa1f6ef2d0 con 0x55aa1f6f8410 2024-09-11T23:24:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.159+0000 7f6c95bcb640 1 --2- 172.21.15.43:0/846980550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa1f6f7af0 0x55aa1f6fa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:24:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.159+0000 7f6c94bc9640 1 --2- 172.21.15.43:0/846980550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f6f8410 0x55aa1f700430 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x55aa1f6f4ec0 tx=0x7f6c80005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f8f8c86347df2a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.159+0000 7f6c7affd640 1 -- 172.21.15.43:0/846980550 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8000f4a0 con 0x55aa1f6f8410 2024-09-11T23:24:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.159+0000 7f6c7affd640 1 -- 172.21.15.43:0/846980550 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c80005520 con 0x55aa1f6f8410 2024-09-11T23:24:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.159+0000 7f6c7affd640 1 -- 172.21.15.43:0/846980550 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c80004660 con 0x55aa1f6f8410 2024-09-11T23:24:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.160+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/846980550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f6f8410 msgr2=0x55aa1f700430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.160+0000 7f6c9a3cb740 1 --2- 172.21.15.43:0/846980550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f6f8410 0x55aa1f700430 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x55aa1f6f4ec0 tx=0x7f6c80005c60 comp rx=0 tx=0).stop 2024-09-11T23:24:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.160+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/846980550 shutdown_connections 2024-09-11T23:24:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.160+0000 7f6c9a3cb740 1 --2- 172.21.15.43:0/846980550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f702cb0 0x55aa1f71a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.160+0000 7f6c9a3cb740 1 --2- 172.21.15.43:0/846980550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f6f8410 0x55aa1f700430 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.160+0000 7f6c9a3cb740 1 --2- 172.21.15.43:0/846980550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa1f6f7af0 0x55aa1f6fa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.160+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/846980550 >> 172.21.15.43:0/846980550 conn(0x55aa1f6f4060 msgr2=0x55aa1f701c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.160+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/846980550 shutdown_connections 2024-09-11T23:24:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.160+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/846980550 wait complete. 2024-09-11T23:24:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.160+0000 7f6c9a3cb740 1 -- start start 2024-09-11T23:24:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.160+0000 7f6c9a3cb740 1 Processor -- start 2024-09-11T23:24:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.161+0000 7f6c9a3cb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f6f7af0 0x55aa1f70b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.161+0000 7f6c9a3cb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f6f8410 0x55aa1f6fc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.161+0000 7f6c9a3cb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa1f702cb0 0x55aa1f6fce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.161+0000 7f6c9a3cb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6c901ae0c0 con 0x55aa1f6f8410 2024-09-11T23:24:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.161+0000 7f6c9a3cb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6c901af530 con 0x55aa1f702cb0 2024-09-11T23:24:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.161+0000 7f6c9a3cb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6c901ae790 con 0x55aa1f6f7af0 2024-09-11T23:24:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.161+0000 7f6c9a3cb740 10 client.0 _ll_register_callbacks cb 0x7f6c88003eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.161+0000 7f6c94bc9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f6f8410 0x55aa1f6fc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.161+0000 7f6c94bc9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f6f8410 0x55aa1f6fc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48070/0 (socket says 172.21.15.43:48070) 2024-09-11T23:24:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.161+0000 7f6c94bc9640 1 -- 172.21.15.43:0/1760127480 learned_addr learned my addr 172.21.15.43:0/1760127480 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.161+0000 7f6c95bcb640 1 --2- 172.21.15.43:0/1760127480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f6f7af0 0x55aa1f70b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.161+0000 7f6c95bcb640 1 --2- 172.21.15.43:0/1760127480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa1f702cb0 0x55aa1f6fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.161+0000 7f6c94bc9640 1 -- 172.21.15.43:0/1760127480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f6f7af0 msgr2=0x55aa1f70b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.161+0000 7f6c94bc9640 1 --2- 172.21.15.43:0/1760127480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f6f7af0 0x55aa1f70b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.161+0000 7f6c94bc9640 1 -- 172.21.15.43:0/1760127480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa1f702cb0 msgr2=0x55aa1f6fce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.161+0000 7f6c94bc9640 1 --2- 172.21.15.43:0/1760127480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa1f702cb0 0x55aa1f6fce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.161+0000 7f6c94bc9640 1 -- 172.21.15.43:0/1760127480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa1f6fd640 con 0x55aa1f6f8410 2024-09-11T23:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.161+0000 7f6c95bcb640 1 --2- 172.21.15.43:0/1760127480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f6f7af0 0x55aa1f70b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.161+0000 7f6c94bc9640 1 --2- 172.21.15.43:0/1760127480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f6f8410 0x55aa1f6fc8d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000b310 tx=0x7f6c8000b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e348b716e8b92b46 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.161+0000 7f6c78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.162+0000 7f6c78ff9640 1 -- 172.21.15.43:0/1760127480 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8003d060 con 0x55aa1f6f8410 2024-09-11T23:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.162+0000 7f6c78ff9640 1 -- 172.21.15.43:0/1760127480 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f6c80038430 con 0x55aa1f6f8410 2024-09-11T23:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.162+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/1760127480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c80005520 con 0x55aa1f6f8410 2024-09-11T23:24:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.162+0000 7f6c78ff9640 1 -- 172.21.15.43:0/1760127480 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6c80034050 con 0x55aa1f6f8410 2024-09-11T23:24:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.162+0000 7f6c9a3cb740 10 client.15948 fetch_fsmap learned FSMap version 26 2024-09-11T23:24:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.162+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/1760127480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aa1f77d890 con 0x55aa1f6f8410 2024-09-11T23:24:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.162+0000 7f6c78ff9640 1 -- 172.21.15.43:0/1760127480 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c80005520 con 0x55aa1f6f8410 2024-09-11T23:24:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.162+0000 7f6c78ff9640 1 -- 172.21.15.43:0/1760127480 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6c800415f0 con 0x55aa1f6f8410 2024-09-11T23:24:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.162+0000 7f6c9a3cb740 10 client.15948 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:24:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.162+0000 7f6c9a3cb740 10 client.15948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:24:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.162+0000 7f6c9a3cb740 20 client.15948 populate_metadata read hostname 'smithi043' 2024-09-11T23:24:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.162+0000 7f6c9a3cb740 1 --2- 172.21.15.43:0/1760127480 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55aa1f77ddb0 0x55aa1f7801a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.162+0000 7f6c9a3cb740 4 client.15948 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:24:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.162+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/1760127480 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aa1f663d70 con 0x55aa1f77ddb0 2024-09-11T23:24:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.163+0000 7f6c95bcb640 1 --2- 172.21.15.43:0/1760127480 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55aa1f77ddb0 0x55aa1f7801a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.163+0000 7f6c95bcb640 1 --2- 172.21.15.43:0/1760127480 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55aa1f77ddb0 0x55aa1f7801a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba43121f29ac27db server_cookie=e84585c7379140e9 in_seq=0 out_seq=0 2024-09-11T23:24:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.163+0000 7f6c78ff9640 10 client.15948 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.177+0000 7f6c78ff9640 1 -- 172.21.15.43:0/1760127480 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aa1f663d70 con 0x55aa1f77ddb0 2024-09-11T23:24:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.177+0000 7f6c78ff9640 10 client.15948 handle_command_reply: tid=0 2024-09-11T23:24:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.178+0000 7f6c9a3cb740 1 client.15948 shutdown 2024-09-11T23:24:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.178+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/1760127480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f6f8410 msgr2=0x55aa1f6fc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.178+0000 7f6c9a3cb740 1 --2- 172.21.15.43:0/1760127480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f6f8410 0x55aa1f6fc8d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000b310 tx=0x7f6c8000b6e0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.178+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/1760127480 shutdown_connections 2024-09-11T23:24:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.178+0000 7f6c9a3cb740 1 --2- 172.21.15.43:0/1760127480 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55aa1f77ddb0 0x55aa1f7801a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.178+0000 7f6c9a3cb740 1 --2- 172.21.15.43:0/1760127480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa1f702cb0 0x55aa1f6fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.178+0000 7f6c78ff9640 0 client.15948 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.178+0000 7f6c9a3cb740 1 --2- 172.21.15.43:0/1760127480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa1f6f8410 0x55aa1f6fc8d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.178+0000 7f6c9a3cb740 1 --2- 172.21.15.43:0/1760127480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa1f6f7af0 0x55aa1f70b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.178+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/1760127480 >> 172.21.15.43:0/1760127480 conn(0x55aa1f6f4060 msgr2=0x55aa1f6f29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.178+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/1760127480 shutdown_connections 2024-09-11T23:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.178+0000 7f6c9a3cb740 1 -- 172.21.15.43:0/1760127480 wait complete. 2024-09-11T23:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.178+0000 7f6c9a3cb740 20 client.15948 trim_cache size 0 max 16384 2024-09-11T23:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.179+0000 7f6c9762e640 1 -- 172.21.15.43:0/805605587 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6c7c0a2cf0 msgr2=0x7f6c7c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.179+0000 7f6c9762e640 1 --2- 172.21.15.43:0/805605587 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6c7c0a2cf0 0x7f6c7c0a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c000c00 tx=0x7f6c8c010d70 comp rx=0 tx=0).stop 2024-09-11T23:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.179+0000 7f6c9762e640 1 -- 172.21.15.43:0/805605587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c90074620 msgr2=0x7f6c9019a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.179+0000 7f6c9762e640 1 --2- 172.21.15.43:0/805605587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c90074620 0x7f6c9019a570 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00d950 tx=0x7f6c8c00de10 comp rx=0 tx=0).stop 2024-09-11T23:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.179+0000 7f6c9762e640 1 -- 172.21.15.43:0/805605587 shutdown_connections 2024-09-11T23:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.179+0000 7f6c9762e640 1 --2- 172.21.15.43:0/805605587 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6c7c0a2cf0 0x7f6c7c0a51b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.179+0000 7f6c9762e640 1 --2- 172.21.15.43:0/805605587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c9010e310 0x7f6c9019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.179+0000 7f6c9762e640 1 --2- 172.21.15.43:0/805605587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c90074f40 0x7f6c9019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.179+0000 7f6c9762e640 1 --2- 172.21.15.43:0/805605587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c90074620 0x7f6c9019a570 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.179+0000 7f6c9762e640 1 -- 172.21.15.43:0/805605587 >> 172.21.15.43:0/805605587 conn(0x7f6c900fd7d0 msgr2=0x7f6c900fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.179+0000 7f6c9762e640 1 -- 172.21.15.43:0/805605587 shutdown_connections 2024-09-11T23:24:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:45.179+0000 7f6c9762e640 1 -- 172.21.15.43:0/805605587 wait complete. 2024-09-11T23:24:45.603 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:45 smithi097 ceph-mon[35562]: pgmap v90: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 142 KiB/s wr, 40 op/s 2024-09-11T23:24:45.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:45 smithi043 ceph-mon[30950]: pgmap v90: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 142 KiB/s wr, 40 op/s 2024-09-11T23:24:45.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:45 smithi090 ceph-mon[35130]: pgmap v90: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 142 KiB/s wr, 40 op/s 2024-09-11T23:24:47.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:47 smithi043 ceph-mon[30950]: pgmap v91: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-11T23:24:47.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:47 smithi090 ceph-mon[35130]: pgmap v91: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-11T23:24:47.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:47 smithi097 ceph-mon[35562]: pgmap v91: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-11T23:24:49.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:49 smithi043 ceph-mon[30950]: pgmap v92: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-11T23:24:49.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:49 smithi090 ceph-mon[35130]: pgmap v92: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-11T23:24:49.787 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:24:49.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:49 smithi097 ceph-mon[35562]: pgmap v92: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-11T23:24:49.864 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:24:51.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:51 smithi043 ceph-mon[30950]: pgmap v93: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-11T23:24:51.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:51 smithi090 ceph-mon[35130]: pgmap v93: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-11T23:24:51.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:51 smithi097 ceph-mon[35562]: pgmap v93: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-11T23:24:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.065+0000 7f08aefe1640 1 -- 172.21.15.43:0/804511725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a8103660 msgr2=0x7f08a810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.065+0000 7f08aefe1640 1 --2- 172.21.15.43:0/804511725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a8103660 0x7f08a810bb50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f089c009900 tx=0x7f089c02a480 comp rx=0 tx=0).stop 2024-09-11T23:24:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.065+0000 7f08aefe1640 1 -- 172.21.15.43:0/804511725 shutdown_connections 2024-09-11T23:24:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.065+0000 7f08aefe1640 1 --2- 172.21.15.43:0/804511725 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08a810c170 0x7f08a810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.065+0000 7f08aefe1640 1 --2- 172.21.15.43:0/804511725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a8103660 0x7f08a810bb50 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.065+0000 7f08aefe1640 1 --2- 172.21.15.43:0/804511725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a8102d40 0x7f08a8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.065+0000 7f08aefe1640 1 -- 172.21.15.43:0/804511725 >> 172.21.15.43:0/804511725 conn(0x7f08a80fc780 msgr2=0x7f08a80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.065+0000 7f08aefe1640 1 -- 172.21.15.43:0/804511725 shutdown_connections 2024-09-11T23:24:53.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.065+0000 7f08aefe1640 1 -- 172.21.15.43:0/804511725 wait complete. 2024-09-11T23:24:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.066+0000 7f08aefe1640 1 Processor -- start 2024-09-11T23:24:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.066+0000 7f08aefe1640 1 -- start start 2024-09-11T23:24:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.066+0000 7f08aefe1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a8102d40 0x7f08a819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.066+0000 7f08aefe1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08a8103660 0x7f08a819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.066+0000 7f08aefe1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a810c170 0x7f08a819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.066+0000 7f08aefe1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f08a8111f70 con 0x7f08a8102d40 2024-09-11T23:24:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.066+0000 7f08aefe1640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f08a8111df0 con 0x7f08a8103660 2024-09-11T23:24:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.066+0000 7f08aefe1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81120f0 con 0x7f08a810c170 2024-09-11T23:24:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.066+0000 7f08acd7d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a8102d40 0x7f08a819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.066+0000 7f08acd7d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a8102d40 0x7f08a819a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41424/0 (socket says 172.21.15.43:41424) 2024-09-11T23:24:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.066+0000 7f08acd7d640 1 -- 172.21.15.43:0/811446176 learned_addr learned my addr 172.21.15.43:0/811446176 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.067+0000 7f08a7fff640 1 --2- 172.21.15.43:0/811446176 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08a8103660 0x7f08a819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:53.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.067+0000 7f08ad57e640 1 --2- 172.21.15.43:0/811446176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a810c170 0x7f08a819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:53.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.067+0000 7f08acd7d640 1 -- 172.21.15.43:0/811446176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a810c170 msgr2=0x7f08a819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:53.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.067+0000 7f08acd7d640 1 --2- 172.21.15.43:0/811446176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a810c170 0x7f08a819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.067+0000 7f08acd7d640 1 -- 172.21.15.43:0/811446176 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08a8103660 msgr2=0x7f08a819ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:53.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.067+0000 7f08acd7d640 1 --2- 172.21.15.43:0/811446176 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08a8103660 0x7f08a819ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.067+0000 7f08acd7d640 1 -- 172.21.15.43:0/811446176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a819f6d0 con 0x7f08a8102d40 2024-09-11T23:24:53.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.067+0000 7f08acd7d640 1 --2- 172.21.15.43:0/811446176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a8102d40 0x7f08a819a720 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f089800d770 tx=0x7f089800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:53.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.067+0000 7f08a5ffb640 1 -- 172.21.15.43:0/811446176 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898017070 con 0x7f08a8102d40 2024-09-11T23:24:53.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.067+0000 7f08a5ffb640 1 -- 172.21.15.43:0/811446176 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0898005500 con 0x7f08a8102d40 2024-09-11T23:24:53.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.067+0000 7f08aefe1640 1 -- 172.21.15.43:0/811446176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a819f9c0 con 0x7f08a8102d40 2024-09-11T23:24:53.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.067+0000 7f08a5ffb640 1 -- 172.21.15.43:0/811446176 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898004e00 con 0x7f08a8102d40 2024-09-11T23:24:53.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.068+0000 7f08aefe1640 1 -- 172.21.15.43:0/811446176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a81a72a0 con 0x7f08a8102d40 2024-09-11T23:24:53.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.070+0000 7f08a5ffb640 1 -- 172.21.15.43:0/811446176 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0898004fa0 con 0x7f08a8102d40 2024-09-11T23:24:53.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.070+0000 7f08b13ba740 1 Processor -- start 2024-09-11T23:24:53.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.070+0000 7f08b13ba740 1 -- start start 2024-09-11T23:24:53.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.070+0000 7f08b13ba740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e82060 0x561154e82440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:53.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.070+0000 7f08b13ba740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e82980 0x561154e8cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:53.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.071+0000 7f08a5ffb640 1 --2- 172.21.15.43:0/811446176 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f08900a2d70 0x7f08900a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:53.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.071+0000 7f08b13ba740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e8d240 0x561154e8f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:53.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.071+0000 7f08b13ba740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561154de7620 con 0x561154e82980 2024-09-11T23:24:53.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.071+0000 7f08b13ba740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561154d8ee50 con 0x561154e82060 2024-09-11T23:24:53.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.071+0000 7f08b13ba740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561154de35e0 con 0x561154e8d240 2024-09-11T23:24:53.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.071+0000 7f08acd7d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e82980 0x561154e8cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:53.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.071+0000 7f08acd7d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e82980 0x561154e8cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41426/0 (socket says 172.21.15.43:41426) 2024-09-11T23:24:53.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.071+0000 7f08acd7d640 1 -- 172.21.15.43:0/3868214528 learned_addr learned my addr 172.21.15.43:0/3868214528 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:53.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.071+0000 7f08a7fff640 1 --2- 172.21.15.43:0/811446176 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f08900a2d70 0x7f08900a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:53.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.071+0000 7f08a5ffb640 1 -- 172.21.15.43:0/811446176 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f08980ced70 con 0x7f08a8102d40 2024-09-11T23:24:53.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.071+0000 7f08ad57e640 1 --2- 172.21.15.43:0/3868214528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e8d240 0x561154e8f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:53.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.071+0000 7f08ad57e640 1 --2- 172.21.15.43:0/3868214528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e82060 0x561154e82440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:53.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.071+0000 7f08acd7d640 1 -- 172.21.15.43:0/3868214528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e8d240 msgr2=0x561154e8f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:53.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.071+0000 7f08acd7d640 1 --2- 172.21.15.43:0/3868214528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e8d240 0x561154e8f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.071+0000 7f08acd7d640 1 -- 172.21.15.43:0/3868214528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e82060 msgr2=0x561154e82440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.071+0000 7f08acd7d640 1 --2- 172.21.15.43:0/3868214528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e82060 0x561154e82440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.071+0000 7f08acd7d640 1 -- 172.21.15.43:0/3868214528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561154d7fd40 con 0x561154e82980 2024-09-11T23:24:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.071+0000 7f08acd7d640 1 --2- 172.21.15.43:0/3868214528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e82980 0x561154e8cd00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f089800d740 tx=0x7f089806b1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb411b9171f1406b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.072+0000 7f08a7fff640 1 --2- 172.21.15.43:0/811446176 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f08900a2d70 0x7f08900a5230 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f089c009230 tx=0x7f089c0058d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.072+0000 7f088effd640 1 -- 172.21.15.43:0/3868214528 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898020070 con 0x561154e82980 2024-09-11T23:24:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.072+0000 7f088effd640 1 -- 172.21.15.43:0/3868214528 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08980046c0 con 0x561154e82980 2024-09-11T23:24:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.072+0000 7f088effd640 1 -- 172.21.15.43:0/3868214528 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898009cc0 con 0x561154e82980 2024-09-11T23:24:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.072+0000 7f08b13ba740 1 -- 172.21.15.43:0/3868214528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e82980 msgr2=0x561154e8cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.072+0000 7f08b13ba740 1 --2- 172.21.15.43:0/3868214528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e82980 0x561154e8cd00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f089800d740 tx=0x7f089806b1a0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.072+0000 7f08b13ba740 1 -- 172.21.15.43:0/3868214528 shutdown_connections 2024-09-11T23:24:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.072+0000 7f08b13ba740 1 --2- 172.21.15.43:0/3868214528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e8d240 0x561154e8f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.072+0000 7f08b13ba740 1 --2- 172.21.15.43:0/3868214528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e82980 0x561154e8cd00 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.072+0000 7f08b13ba740 1 --2- 172.21.15.43:0/3868214528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e82060 0x561154e82440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.072+0000 7f08b13ba740 1 -- 172.21.15.43:0/3868214528 >> 172.21.15.43:0/3868214528 conn(0x561154e7e5f0 msgr2=0x561154e7e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.073+0000 7f08b13ba740 1 -- 172.21.15.43:0/3868214528 shutdown_connections 2024-09-11T23:24:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.073+0000 7f08b13ba740 1 -- 172.21.15.43:0/3868214528 wait complete. 2024-09-11T23:24:53.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.073+0000 7f08b13ba740 1 -- start start 2024-09-11T23:24:53.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.073+0000 7f08b13ba740 1 Processor -- start 2024-09-11T23:24:53.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.074+0000 7f08b13ba740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e82060 0x561154e8a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:53.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.074+0000 7f08b13ba740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e82980 0x561154e8a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:53.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.074+0000 7f08b13ba740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e8d240 0x561154e8abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:53.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.074+0000 7f08b13ba740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561154e8fe10 con 0x561154e8d240 2024-09-11T23:24:53.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.074+0000 7f08b13ba740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561154d6ff10 con 0x561154e82060 2024-09-11T23:24:53.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.074+0000 7f08b13ba740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561154e8ff90 con 0x561154e82980 2024-09-11T23:24:53.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.074+0000 7f08b13ba740 10 client.0 _ll_register_callbacks cb 0x561154d93db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:53.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.074+0000 7f08ad57e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e8d240 0x561154e8abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:53.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.074+0000 7f08acd7d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e82060 0x561154e8a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:53.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.074+0000 7f08a7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e82980 0x561154e8a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:53.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.074+0000 7f08a7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e82980 0x561154e8a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:46266/0 (socket says 172.21.15.43:46266) 2024-09-11T23:24:53.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.074+0000 7f08a7fff640 1 -- 172.21.15.43:0/366089858 learned_addr learned my addr 172.21.15.43:0/366089858 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:53.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.074+0000 7f08acd7d640 1 -- 172.21.15.43:0/366089858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e82980 msgr2=0x561154e8a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:53.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.074+0000 7f08acd7d640 1 --2- 172.21.15.43:0/366089858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e82980 0x561154e8a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.074+0000 7f08acd7d640 1 -- 172.21.15.43:0/366089858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e8d240 msgr2=0x561154e8abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:53.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.074+0000 7f08acd7d640 1 --2- 172.21.15.43:0/366089858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e8d240 0x561154e8abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.074+0000 7f08acd7d640 1 -- 172.21.15.43:0/366089858 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561154af6580 con 0x561154e82060 2024-09-11T23:24:53.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.074+0000 7f08a7fff640 1 --2- 172.21.15.43:0/366089858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e82980 0x561154e8a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:24:53.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.074+0000 7f08ad57e640 1 --2- 172.21.15.43:0/366089858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e8d240 0x561154e8abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:24:53.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.075+0000 7f08acd7d640 1 --2- 172.21.15.43:0/366089858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e82060 0x561154e8a170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x561154e82f20 tx=0x7f08980cc4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ce9d681845ca53e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:53.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.075+0000 7f0887fff640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:24:53.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.075+0000 7f0887fff640 1 -- 172.21.15.43:0/366089858 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898020040 con 0x561154e82060 2024-09-11T23:24:53.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.075+0000 7f0887fff640 1 -- 172.21.15.43:0/366089858 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f0898009cc0 con 0x561154e82060 2024-09-11T23:24:53.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.075+0000 7f08b13ba740 1 -- 172.21.15.43:0/366089858 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561154df4b90 con 0x561154e82060 2024-09-11T23:24:53.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.075+0000 7f0887fff640 1 -- 172.21.15.43:0/366089858 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f089809aca0 con 0x561154e82060 2024-09-11T23:24:53.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.075+0000 7f08b13ba740 10 client.25496 fetch_fsmap learned FSMap version 26 2024-09-11T23:24:53.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.075+0000 7f08b13ba740 1 -- 172.21.15.43:0/366089858 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561154bc0960 con 0x561154e82060 2024-09-11T23:24:53.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.075+0000 7f0887fff640 1 -- 172.21.15.43:0/366089858 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561154df4b90 con 0x561154e82060 2024-09-11T23:24:53.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.075+0000 7f0887fff640 1 -- 172.21.15.43:0/366089858 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f08980cc740 con 0x561154e82060 2024-09-11T23:24:53.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.075+0000 7f08b13ba740 10 client.25496 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:24:53.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.075+0000 7f08b13ba740 10 client.25496 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:24:53.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.076+0000 7f08b13ba740 20 client.25496 populate_metadata read hostname 'smithi043' 2024-09-11T23:24:53.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.076+0000 7f08b13ba740 1 --2- 172.21.15.43:0/366089858 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561154e9cc90 0x561154e9f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:53.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.076+0000 7f08b13ba740 4 client.25496 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:24:53.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.076+0000 7f08b13ba740 1 -- 172.21.15.43:0/366089858 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561154def0a0 con 0x561154e9cc90 2024-09-11T23:24:53.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.076+0000 7f08a7fff640 1 --2- 172.21.15.43:0/366089858 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561154e9cc90 0x561154e9f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:53.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.076+0000 7f08a7fff640 1 --2- 172.21.15.43:0/366089858 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561154e9cc90 0x561154e9f080 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d947484c48b8681 server_cookie=dd84c1b8f7851e30 in_seq=0 out_seq=0 2024-09-11T23:24:53.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.076+0000 7f0887fff640 10 client.25496 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:53.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.077+0000 7f0887fff640 1 -- 172.21.15.43:0/366089858 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561154def0a0 con 0x561154e9cc90 2024-09-11T23:24:53.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.077+0000 7f0887fff640 10 client.25496 handle_command_reply: tid=0 2024-09-11T23:24:53.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.077+0000 7f08b13ba740 1 client.25496 shutdown 2024-09-11T23:24:53.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.077+0000 7f08b13ba740 1 -- 172.21.15.43:0/366089858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e82060 msgr2=0x561154e8a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:53.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.077+0000 7f08b13ba740 1 --2- 172.21.15.43:0/366089858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e82060 0x561154e8a170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x561154e82f20 tx=0x7f08980cc4b0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.077+0000 7f08b13ba740 1 -- 172.21.15.43:0/366089858 shutdown_connections 2024-09-11T23:24:53.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.077+0000 7f08b13ba740 1 --2- 172.21.15.43:0/366089858 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561154e9cc90 0x561154e9f080 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.077+0000 7f08b13ba740 1 --2- 172.21.15.43:0/366089858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e8d240 0x561154e8abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.077+0000 7f08b13ba740 1 --2- 172.21.15.43:0/366089858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e82980 0x561154e8a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.077+0000 7f08b13ba740 1 --2- 172.21.15.43:0/366089858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e82060 0x561154e8a170 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.077+0000 7f0887fff640 0 client.25496 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:53.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.077+0000 7f08b13ba740 1 -- 172.21.15.43:0/366089858 >> 172.21.15.43:0/366089858 conn(0x561154e7e5f0 msgr2=0x561154e7d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:53.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.077+0000 7f08b13ba740 1 -- 172.21.15.43:0/366089858 shutdown_connections 2024-09-11T23:24:53.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.077+0000 7f08b13ba740 1 -- 172.21.15.43:0/366089858 wait complete. 2024-09-11T23:24:53.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.078+0000 7f08b13ba740 20 client.25496 trim_cache size 0 max 16384 2024-09-11T23:24:53.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.095+0000 7f08b13ba740 1 Processor -- start 2024-09-11T23:24:53.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.095+0000 7f08b13ba740 1 -- start start 2024-09-11T23:24:53.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.095+0000 7f08b13ba740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e82060 0x561154e841c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:53.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.095+0000 7f08b13ba740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e82980 0x561154e84700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:53.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.095+0000 7f08b13ba740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e8d240 0x561154ea46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:53.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.095+0000 7f08b13ba740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81b2410 con 0x561154e82980 2024-09-11T23:24:53.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.095+0000 7f08b13ba740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81afce0 con 0x561154e82060 2024-09-11T23:24:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.095+0000 7f08b13ba740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81adb30 con 0x561154e8d240 2024-09-11T23:24:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.095+0000 7f08a7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e82980 0x561154e84700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.095+0000 7f08a7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e82980 0x561154e84700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41442/0 (socket says 172.21.15.43:41442) 2024-09-11T23:24:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.095+0000 7f08a7fff640 1 -- 172.21.15.43:0/1133762829 learned_addr learned my addr 172.21.15.43:0/1133762829 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.096+0000 7f08acd7d640 1 --2- 172.21.15.43:0/1133762829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e82060 0x561154e841c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.096+0000 7f08ad57e640 1 --2- 172.21.15.43:0/1133762829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e8d240 0x561154ea46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.096+0000 7f08a7fff640 1 -- 172.21.15.43:0/1133762829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e8d240 msgr2=0x561154ea46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.096+0000 7f08a7fff640 1 --2- 172.21.15.43:0/1133762829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e8d240 0x561154ea46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.096+0000 7f08a7fff640 1 -- 172.21.15.43:0/1133762829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e82060 msgr2=0x561154e841c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.096+0000 7f08a7fff640 1 --2- 172.21.15.43:0/1133762829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e82060 0x561154e841c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.096+0000 7f08a7fff640 1 -- 172.21.15.43:0/1133762829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561154e7f780 con 0x561154e82980 2024-09-11T23:24:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.096+0000 7f08a7fff640 1 --2- 172.21.15.43:0/1133762829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e82980 0x561154e84700 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f089c004770 tx=0x7f089c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28da55a188ef1263 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.096+0000 7f088e7fc640 1 -- 172.21.15.43:0/1133762829 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c035040 con 0x561154e82980 2024-09-11T23:24:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.096+0000 7f088e7fc640 1 -- 172.21.15.43:0/1133762829 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f089c057070 con 0x561154e82980 2024-09-11T23:24:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.096+0000 7f088e7fc640 1 -- 172.21.15.43:0/1133762829 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c0514e0 con 0x561154e82980 2024-09-11T23:24:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.097+0000 7f08b13ba740 1 -- 172.21.15.43:0/1133762829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e82980 msgr2=0x561154e84700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.097+0000 7f08b13ba740 1 --2- 172.21.15.43:0/1133762829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e82980 0x561154e84700 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f089c004770 tx=0x7f089c040000 comp rx=0 tx=0).stop 2024-09-11T23:24:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.097+0000 7f08b13ba740 1 -- 172.21.15.43:0/1133762829 shutdown_connections 2024-09-11T23:24:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.097+0000 7f08b13ba740 1 --2- 172.21.15.43:0/1133762829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e8d240 0x561154ea46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.097+0000 7f08b13ba740 1 --2- 172.21.15.43:0/1133762829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e82980 0x561154e84700 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.097+0000 7f08b13ba740 1 --2- 172.21.15.43:0/1133762829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e82060 0x561154e841c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.097+0000 7f08b13ba740 1 -- 172.21.15.43:0/1133762829 >> 172.21.15.43:0/1133762829 conn(0x561154e7e5f0 msgr2=0x561154e83ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.097+0000 7f08b13ba740 1 -- 172.21.15.43:0/1133762829 shutdown_connections 2024-09-11T23:24:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.097+0000 7f08b13ba740 1 -- 172.21.15.43:0/1133762829 wait complete. 2024-09-11T23:24:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.097+0000 7f08b13ba740 1 -- start start 2024-09-11T23:24:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.097+0000 7f08b13ba740 1 Processor -- start 2024-09-11T23:24:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.098+0000 7f08b13ba740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e82060 0x561154e90f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.098+0000 7f08b13ba740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e82980 0x561154e91440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.098+0000 7f08b13ba740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e8d240 0x561154e91980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.098+0000 7f08b13ba740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81ae1a0 con 0x561154e82980 2024-09-11T23:24:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.098+0000 7f08b13ba740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81af610 con 0x561154e8d240 2024-09-11T23:24:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.098+0000 7f08b13ba740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81ae870 con 0x561154e82060 2024-09-11T23:24:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.098+0000 7f08b13ba740 10 client.0 _ll_register_callbacks cb 0x7f089800de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.098+0000 7f08a7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e82980 0x561154e91440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.098+0000 7f08a7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e82980 0x561154e91440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41446/0 (socket says 172.21.15.43:41446) 2024-09-11T23:24:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.098+0000 7f08a7fff640 1 -- 172.21.15.43:0/2931782331 learned_addr learned my addr 172.21.15.43:0/2931782331 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.098+0000 7f08acd7d640 1 --2- 172.21.15.43:0/2931782331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e82060 0x561154e90f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.098+0000 7f08ad57e640 1 --2- 172.21.15.43:0/2931782331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e8d240 0x561154e91980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.098+0000 7f08a7fff640 1 -- 172.21.15.43:0/2931782331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e82060 msgr2=0x561154e90f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.098+0000 7f08a7fff640 1 --2- 172.21.15.43:0/2931782331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e82060 0x561154e90f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.098+0000 7f08a7fff640 1 -- 172.21.15.43:0/2931782331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e8d240 msgr2=0x561154e91980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.098+0000 7f08a7fff640 1 --2- 172.21.15.43:0/2931782331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e8d240 0x561154e91980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.098+0000 7f08a7fff640 1 -- 172.21.15.43:0/2931782331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561154f06680 con 0x561154e82980 2024-09-11T23:24:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.098+0000 7f08ad57e640 1 --2- 172.21.15.43:0/2931782331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e8d240 0x561154e91980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:24:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.099+0000 7f08a7fff640 1 --2- 172.21.15.43:0/2931782331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e82980 0x561154e91440 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f089c031980 tx=0x7f089c0316c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42bbf90f7aca3808 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.099+0000 7f0887fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:24:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.099+0000 7f0887fff640 1 -- 172.21.15.43:0/2931782331 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c060060 con 0x561154e82980 2024-09-11T23:24:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.099+0000 7f0887fff640 1 -- 172.21.15.43:0/2931782331 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f089c003830 con 0x561154e82980 2024-09-11T23:24:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.099+0000 7f08b13ba740 1 -- 172.21.15.43:0/2931782331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f089c057070 con 0x561154e82980 2024-09-11T23:24:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.099+0000 7f0887fff640 1 -- 172.21.15.43:0/2931782331 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f089c035070 con 0x561154e82980 2024-09-11T23:24:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.099+0000 7f08b13ba740 10 client.15966 fetch_fsmap learned FSMap version 26 2024-09-11T23:24:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.099+0000 7f08b13ba740 1 -- 172.21.15.43:0/2931782331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561154f07970 con 0x561154e82980 2024-09-11T23:24:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.099+0000 7f0887fff640 1 -- 172.21.15.43:0/2931782331 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f089c057070 con 0x561154e82980 2024-09-11T23:24:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.099+0000 7f0887fff640 1 -- 172.21.15.43:0/2931782331 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f089c03cce0 con 0x561154e82980 2024-09-11T23:24:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.099+0000 7f08b13ba740 10 client.15966 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:24:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.099+0000 7f08b13ba740 10 client.15966 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:24:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.099+0000 7f08b13ba740 20 client.15966 populate_metadata read hostname 'smithi043' 2024-09-11T23:24:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.099+0000 7f08b13ba740 1 --2- 172.21.15.43:0/2931782331 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561154f080c0 0x561154f0a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.099+0000 7f08b13ba740 4 client.15966 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "993ae2f2", "release": true, "await": true}] 2024-09-11T23:24:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.099+0000 7f08b13ba740 1 -- 172.21.15.43:0/2931782331 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "quiesce db", "set_id": "993ae2f2", "release": true, "await": true}) -- 0x561154d6b5a0 con 0x561154f080c0 2024-09-11T23:24:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.100+0000 7f08acd7d640 1 --2- 172.21.15.43:0/2931782331 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561154f080c0 0x561154f0a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.100+0000 7f08acd7d640 1 --2- 172.21.15.43:0/2931782331 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561154f080c0 0x561154f0a4e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e772bf3c9f1f807 server_cookie=5de7524a03fb0e4f in_seq=0 out_seq=0 2024-09-11T23:24:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.100+0000 7f0887fff640 10 client.15966 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.129+0000 7f0887fff640 1 -- 172.21.15.43:0/2931782331 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x561154d6b5a0 con 0x561154f080c0 2024-09-11T23:24:53.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.129+0000 7f0887fff640 10 client.15966 handle_command_reply: tid=0 2024-09-11T23:24:53.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.129+0000 7f08b13ba740 1 client.15966 shutdown 2024-09-11T23:24:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.137+0000 7f08b13ba740 1 -- 172.21.15.43:0/2931782331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e82980 msgr2=0x561154e91440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.137+0000 7f08b13ba740 1 --2- 172.21.15.43:0/2931782331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e82980 0x561154e91440 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f089c031980 tx=0x7f089c0316c0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.138+0000 7f08b13ba740 1 -- 172.21.15.43:0/2931782331 shutdown_connections 2024-09-11T23:24:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.138+0000 7f08b13ba740 1 --2- 172.21.15.43:0/2931782331 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561154f080c0 0x561154f0a4e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.138+0000 7f08b13ba740 1 --2- 172.21.15.43:0/2931782331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561154e8d240 0x561154e91980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.138+0000 7f08b13ba740 1 --2- 172.21.15.43:0/2931782331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561154e82980 0x561154e91440 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.138+0000 7f08b13ba740 1 --2- 172.21.15.43:0/2931782331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561154e82060 0x561154e90f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.138+0000 7f08b13ba740 1 -- 172.21.15.43:0/2931782331 >> 172.21.15.43:0/2931782331 conn(0x561154e7e5f0 msgr2=0x561154e84240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.138+0000 7f0887fff640 0 client.15966 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:53.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.143+0000 7f08b13ba740 1 -- 172.21.15.43:0/2931782331 shutdown_connections 2024-09-11T23:24:53.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.143+0000 7f08b13ba740 1 -- 172.21.15.43:0/2931782331 wait complete. 2024-09-11T23:24:53.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.143+0000 7f08b13ba740 20 client.15966 trim_cache size 0 max 16384 2024-09-11T23:24:53.143 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:24:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 26, 2024-09-11T23:24:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "leader": 24497, 2024-09-11T23:24:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "set_version": 8, 2024-09-11T23:24:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "sets": { 2024-09-11T23:24:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "993ae2f2": { 2024-09-11T23:24:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 8, 2024-09-11T23:24:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "age_ref": 0.0, 2024-09-11T23:24:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:24:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RELEASED", 2024-09-11T23:24:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-09-11T23:24:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:24:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "timeout": 90.0, 2024-09-11T23:24:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "expiration": 133.4, 2024-09-11T23:24:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "members": { 2024-09-11T23:24:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "file:/": { 2024-09-11T23:24:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "excluded": false, 2024-09-11T23:24:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:24:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RELEASED", 2024-09-11T23:24:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-09-11T23:24:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:24:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:24:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:24:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:24:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:24:53.145 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:24:53.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.145+0000 7f08aefe1640 1 -- 172.21.15.43:0/811446176 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f08900a2d70 msgr2=0x7f08900a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:53.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.145+0000 7f08aefe1640 1 --2- 172.21.15.43:0/811446176 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f08900a2d70 0x7f08900a5230 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f089c009230 tx=0x7f089c0058d0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.145+0000 7f08aefe1640 1 -- 172.21.15.43:0/811446176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a8102d40 msgr2=0x7f08a819a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:53.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.145+0000 7f08aefe1640 1 --2- 172.21.15.43:0/811446176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a8102d40 0x7f08a819a720 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f089800d770 tx=0x7f089800dc30 comp rx=0 tx=0).stop 2024-09-11T23:24:53.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.145+0000 7f08aefe1640 1 -- 172.21.15.43:0/811446176 shutdown_connections 2024-09-11T23:24:53.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.145+0000 7f08aefe1640 1 --2- 172.21.15.43:0/811446176 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f08900a2d70 0x7f08900a5230 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.145+0000 7f08aefe1640 1 --2- 172.21.15.43:0/811446176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08a810c170 0x7f08a819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.145+0000 7f08aefe1640 1 --2- 172.21.15.43:0/811446176 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08a8103660 0x7f08a819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.145+0000 7f08aefe1640 1 --2- 172.21.15.43:0/811446176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08a8102d40 0x7f08a819a720 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:53.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.145+0000 7f08aefe1640 1 -- 172.21.15.43:0/811446176 >> 172.21.15.43:0/811446176 conn(0x7f08a80fc780 msgr2=0x7f08a81008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:53.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.149+0000 7f08aefe1640 1 -- 172.21.15.43:0/811446176 shutdown_connections 2024-09-11T23:24:53.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:24:53.149+0000 7f08aefe1640 1 -- 172.21.15.43:0/811446176 wait complete. 2024-09-11T23:24:53.445 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:24:53.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:53 smithi043 ceph-mon[30950]: pgmap v94: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.2 KiB/s wr, 10 op/s 2024-09-11T23:24:53.772 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:53 smithi090 ceph-mon[35130]: pgmap v94: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.2 KiB/s wr, 10 op/s 2024-09-11T23:24:53.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:53 smithi097 ceph-mon[35562]: pgmap v94: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.2 KiB/s wr, 10 op/s 2024-09-11T23:24:55.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:55 smithi043 ceph-mon[30950]: pgmap v95: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.4 KiB/s wr, 7 op/s 2024-09-11T23:24:55.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:55 smithi090 ceph-mon[35130]: pgmap v95: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.4 KiB/s wr, 7 op/s 2024-09-11T23:24:55.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:55 smithi097 ceph-mon[35562]: pgmap v95: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.4 KiB/s wr, 7 op/s 2024-09-11T23:24:56.746 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '993ae2f2', total seconds elapsed: 67.8 2024-09-11T23:24:57.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:57 smithi090 ceph-mon[35130]: pgmap v96: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.4 KiB/s wr, 7 op/s 2024-09-11T23:24:57.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:57 smithi097 ceph-mon[35562]: pgmap v96: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.4 KiB/s wr, 7 op/s 2024-09-11T23:24:57.862 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:57 smithi043 ceph-mon[30950]: pgmap v96: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.4 KiB/s wr, 7 op/s 2024-09-11T23:24:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.301+0000 7fd9f128d640 1 -- 172.21.15.43:0/2306567542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9ec10c0a0 msgr2=0x7fd9ec10e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.301+0000 7fd9f128d640 1 --2- 172.21.15.43:0/2306567542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9ec10c0a0 0x7fd9ec10e540 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009ca0 tx=0x7fd9e402a950 comp rx=0 tx=0).stop 2024-09-11T23:24:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.302+0000 7fd9f128d640 1 -- 172.21.15.43:0/2306567542 shutdown_connections 2024-09-11T23:24:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.302+0000 7fd9f128d640 1 --2- 172.21.15.43:0/2306567542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9ec10c0a0 0x7fd9ec10e540 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.302+0000 7fd9f128d640 1 --2- 172.21.15.43:0/2306567542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ec1035a0 0x7fd9ec10ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.302+0000 7fd9f128d640 1 --2- 172.21.15.43:0/2306567542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9ec102c80 0x7fd9ec103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.302+0000 7fd9f128d640 1 -- 172.21.15.43:0/2306567542 >> 172.21.15.43:0/2306567542 conn(0x7fd9ec0779a0 msgr2=0x7fd9ec0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.302+0000 7fd9f128d640 1 -- 172.21.15.43:0/2306567542 shutdown_connections 2024-09-11T23:24:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.302+0000 7fd9f128d640 1 -- 172.21.15.43:0/2306567542 wait complete. 2024-09-11T23:24:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.302+0000 7fd9f128d640 1 Processor -- start 2024-09-11T23:24:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.302+0000 7fd9f128d640 1 -- start start 2024-09-11T23:24:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.303+0000 7fd9f128d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ec102c80 0x7fd9ec19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.303+0000 7fd9f128d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9ec1035a0 0x7fd9ec19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.303+0000 7fd9f128d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9ec10c0a0 0x7fd9ec19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.303+0000 7fd9f128d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec101d70 con 0x7fd9ec10c0a0 2024-09-11T23:24:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.303+0000 7fd9f128d640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec101bf0 con 0x7fd9ec102c80 2024-09-11T23:24:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.303+0000 7fd9f128d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec101ef0 con 0x7fd9ec1035a0 2024-09-11T23:24:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.303+0000 7fd9eb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9ec10c0a0 0x7fd9ec19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.303+0000 7fd9eb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9ec10c0a0 0x7fd9ec19ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41452/0 (socket says 172.21.15.43:41452) 2024-09-11T23:24:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.303+0000 7fd9eb7fe640 1 -- 172.21.15.43:0/389688002 learned_addr learned my addr 172.21.15.43:0/389688002 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.303+0000 7fd9eaffd640 1 --2- 172.21.15.43:0/389688002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ec102c80 0x7fd9ec19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.303+0000 7fd9ea7fc640 1 --2- 172.21.15.43:0/389688002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9ec1035a0 0x7fd9ec19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.303+0000 7fd9eb7fe640 1 -- 172.21.15.43:0/389688002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9ec1035a0 msgr2=0x7fd9ec19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.303+0000 7fd9eb7fe640 1 --2- 172.21.15.43:0/389688002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9ec1035a0 0x7fd9ec19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.303+0000 7fd9eb7fe640 1 -- 172.21.15.43:0/389688002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ec102c80 msgr2=0x7fd9ec19a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.303+0000 7fd9eb7fe640 1 --2- 172.21.15.43:0/389688002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ec102c80 0x7fd9ec19a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.303+0000 7fd9eb7fe640 1 -- 172.21.15.43:0/389688002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ec19f560 con 0x7fd9ec10c0a0 2024-09-11T23:24:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.303+0000 7fd9ea7fc640 1 --2- 172.21.15.43:0/389688002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9ec1035a0 0x7fd9ec19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:24:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.303+0000 7fd9eb7fe640 1 --2- 172.21.15.43:0/389688002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9ec10c0a0 0x7fd9ec19ee30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd9e402ae70 tx=0x7fd9e402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.304+0000 7fd9d3fff640 1 -- 172.21.15.43:0/389688002 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4034040 con 0x7fd9ec10c0a0 2024-09-11T23:24:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.304+0000 7fd9d3fff640 1 -- 172.21.15.43:0/389688002 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9e4030920 con 0x7fd9ec10c0a0 2024-09-11T23:24:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.304+0000 7fd9f128d640 1 -- 172.21.15.43:0/389688002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9ec19f7f0 con 0x7fd9ec10c0a0 2024-09-11T23:24:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.304+0000 7fd9d3fff640 1 -- 172.21.15.43:0/389688002 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4045480 con 0x7fd9ec10c0a0 2024-09-11T23:24:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.304+0000 7fd9f128d640 1 -- 172.21.15.43:0/389688002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9ec1a70e0 con 0x7fd9ec10c0a0 2024-09-11T23:24:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.306+0000 7fd9d3fff640 1 -- 172.21.15.43:0/389688002 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd9e4037020 con 0x7fd9ec10c0a0 2024-09-11T23:24:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.307+0000 7fd9f402c740 1 Processor -- start 2024-09-11T23:24:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.307+0000 7fd9f402c740 1 -- start start 2024-09-11T23:24:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.307+0000 7fd9f402c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989aaf0 0x5634c989aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.307+0000 7fd9f402c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c989b410 0x5634c98a5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.307+0000 7fd9f402c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c98a5cb0 0x5634c98a80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.307+0000 7fd9f402c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5634c97ff2f0 con 0x5634c98a5cb0 2024-09-11T23:24:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.307+0000 7fd9f402c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5634c97a7270 con 0x5634c989b410 2024-09-11T23:24:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.307+0000 7fd9f402c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5634c97fb2b0 con 0x5634c989aaf0 2024-09-11T23:24:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.307+0000 7fd9ea7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c98a5cb0 0x5634c98a80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.307+0000 7fd9ea7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c98a5cb0 0x5634c98a80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41456/0 (socket says 172.21.15.43:41456) 2024-09-11T23:24:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.307+0000 7fd9ea7fc640 1 -- 172.21.15.43:0/3926570012 learned_addr learned my addr 172.21.15.43:0/3926570012 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.307+0000 7fd9eaffd640 1 --2- 172.21.15.43:0/3926570012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c989b410 0x5634c98a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.307+0000 7fd9d3fff640 1 --2- 172.21.15.43:0/389688002 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fd9d40a2d70 0x7fd9d40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.307+0000 7fd9ea7fc640 1 --2- 172.21.15.43:0/3926570012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989aaf0 0x5634c989aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.307+0000 7fd9eaffd640 1 -- 172.21.15.43:0/3926570012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989aaf0 msgr2=0x5634c989aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.307+0000 7fd9eaffd640 1 --2- 172.21.15.43:0/3926570012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989aaf0 0x5634c989aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.307+0000 7fd9eaffd640 1 -- 172.21.15.43:0/3926570012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c98a5cb0 msgr2=0x5634c98a80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.307+0000 7fd9eaffd640 1 --2- 172.21.15.43:0/3926570012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c98a5cb0 0x5634c98a80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.307+0000 7fd9eaffd640 1 -- 172.21.15.43:0/3926570012 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c97a6340 con 0x5634c989b410 2024-09-11T23:24:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.307+0000 7fd9d3fff640 1 -- 172.21.15.43:0/389688002 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fd9e40f0290 con 0x7fd9ec10c0a0 2024-09-11T23:24:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.307+0000 7fd9ea7fc640 1 --2- 172.21.15.43:0/3926570012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989aaf0 0x5634c989aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:24:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.308+0000 7fd9eb7fe640 1 --2- 172.21.15.43:0/389688002 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fd9d40a2d70 0x7fd9d40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.308+0000 7fd9eaffd640 1 --2- 172.21.15.43:0/3926570012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c989b410 0x5634c98a5770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc004630 tx=0x7fd9dc00bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7525823c0eb01b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.308+0000 7fd9d17fa640 1 -- 172.21.15.43:0/3926570012 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc003bd0 con 0x5634c989b410 2024-09-11T23:24:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.308+0000 7fd9eb7fe640 1 --2- 172.21.15.43:0/389688002 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fd9d40a2d70 0x7fd9d40a5230 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd9e402fda0 tx=0x7fd9e408b790 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.308+0000 7fd9d17fa640 1 -- 172.21.15.43:0/3926570012 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9dc015070 con 0x5634c989b410 2024-09-11T23:24:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.308+0000 7fd9d17fa640 1 -- 172.21.15.43:0/3926570012 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc004d90 con 0x5634c989b410 2024-09-11T23:24:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.309+0000 7fd9f402c740 1 -- 172.21.15.43:0/3926570012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c989b410 msgr2=0x5634c98a5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.309+0000 7fd9f402c740 1 --2- 172.21.15.43:0/3926570012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c989b410 0x5634c98a5770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc004630 tx=0x7fd9dc00bb20 comp rx=0 tx=0).stop 2024-09-11T23:24:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.309+0000 7fd9f402c740 1 -- 172.21.15.43:0/3926570012 shutdown_connections 2024-09-11T23:24:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.309+0000 7fd9f402c740 1 --2- 172.21.15.43:0/3926570012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c98a5cb0 0x5634c98a80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.309+0000 7fd9f402c740 1 --2- 172.21.15.43:0/3926570012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c989b410 0x5634c98a5770 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.309+0000 7fd9f402c740 1 --2- 172.21.15.43:0/3926570012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989aaf0 0x5634c989aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.309+0000 7fd9f402c740 1 -- 172.21.15.43:0/3926570012 >> 172.21.15.43:0/3926570012 conn(0x5634c9897060 msgr2=0x5634c9897440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.309+0000 7fd9f402c740 1 -- 172.21.15.43:0/3926570012 shutdown_connections 2024-09-11T23:24:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.309+0000 7fd9f402c740 1 -- 172.21.15.43:0/3926570012 wait complete. 2024-09-11T23:24:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.310+0000 7fd9f402c740 1 -- start start 2024-09-11T23:24:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.310+0000 7fd9f402c740 1 Processor -- start 2024-09-11T23:24:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.310+0000 7fd9f402c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c989aaf0 0x5634c98a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.310+0000 7fd9f402c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989b410 0x5634c98a30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.310+0000 7fd9f402c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c98a5cb0 0x5634c98a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.310+0000 7fd9f402c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5634c98a8880 con 0x5634c98a5cb0 2024-09-11T23:24:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.310+0000 7fd9f402c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5634c9788290 con 0x5634c989aaf0 2024-09-11T23:24:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.310+0000 7fd9f402c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5634c98a8a00 con 0x5634c989b410 2024-09-11T23:24:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.310+0000 7fd9f402c740 10 client.0 _ll_register_callbacks cb 0x5634c97b9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.310+0000 7fd9ea7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c98a5cb0 0x5634c98a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.310+0000 7fd9ea7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c98a5cb0 0x5634c98a35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41466/0 (socket says 172.21.15.43:41466) 2024-09-11T23:24:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.310+0000 7fd9ea7fc640 1 -- 172.21.15.43:0/1733447914 learned_addr learned my addr 172.21.15.43:0/1733447914 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.310+0000 7fd9ea7fc640 1 --2- 172.21.15.43:0/1733447914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c989aaf0 0x5634c98a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.311+0000 7fd9eaffd640 1 --2- 172.21.15.43:0/1733447914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989b410 0x5634c98a30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.311+0000 7fd9ea7fc640 1 -- 172.21.15.43:0/1733447914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989b410 msgr2=0x5634c98a30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.311+0000 7fd9ea7fc640 1 --2- 172.21.15.43:0/1733447914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989b410 0x5634c98a30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.311+0000 7fd9ea7fc640 1 -- 172.21.15.43:0/1733447914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c989aaf0 msgr2=0x5634c98a2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.311+0000 7fd9ea7fc640 1 --2- 172.21.15.43:0/1733447914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c989aaf0 0x5634c98a2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.311+0000 7fd9ea7fc640 1 -- 172.21.15.43:0/1733447914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634c9512210 con 0x5634c98a5cb0 2024-09-11T23:24:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.311+0000 7fd9eaffd640 1 --2- 172.21.15.43:0/1733447914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989b410 0x5634c98a30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:24:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.311+0000 7fd9ea7fc640 1 --2- 172.21.15.43:0/1733447914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c98a5cb0 0x5634c98a35e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0012ae0 tx=0x7fd9e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8a236b0e1217d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.311+0000 7fd9ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:24:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.311+0000 7fd9ceff5640 1 -- 172.21.15.43:0/1733447914 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0010b00 con 0x5634c98a5cb0 2024-09-11T23:24:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.311+0000 7fd9f402c740 1 -- 172.21.15.43:0/1733447914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9dc015070 con 0x5634c98a5cb0 2024-09-11T23:24:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.311+0000 7fd9ceff5640 1 -- 172.21.15.43:0/1733447914 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fd9e0010ca0 con 0x5634c98a5cb0 2024-09-11T23:24:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.312+0000 7fd9ceff5640 1 -- 172.21.15.43:0/1733447914 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd9e0004020 con 0x5634c98a5cb0 2024-09-11T23:24:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.312+0000 7fd9f402c740 10 client.15978 fetch_fsmap learned FSMap version 26 2024-09-11T23:24:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.312+0000 7fd9f402c740 1 -- 172.21.15.43:0/1733447914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5634c95de620 con 0x5634c98a5cb0 2024-09-11T23:24:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.312+0000 7fd9ceff5640 1 -- 172.21.15.43:0/1733447914 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9dc015070 con 0x5634c98a5cb0 2024-09-11T23:24:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.312+0000 7fd9ceff5640 1 -- 172.21.15.43:0/1733447914 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd9e001f070 con 0x5634c98a5cb0 2024-09-11T23:24:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.312+0000 7fd9f402c740 10 client.15978 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:24:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.312+0000 7fd9f402c740 10 client.15978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:24:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.312+0000 7fd9f402c740 20 client.15978 populate_metadata read hostname 'smithi043' 2024-09-11T23:24:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.312+0000 7fd9f402c740 1 --2- 172.21.15.43:0/1733447914 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5634c98b5c50 0x5634c98b8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.312+0000 7fd9f402c740 4 client.15978 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:24:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.312+0000 7fd9f402c740 1 -- 172.21.15.43:0/1733447914 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634c980ca20 con 0x5634c98b5c50 2024-09-11T23:24:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.312+0000 7fd9eaffd640 1 --2- 172.21.15.43:0/1733447914 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5634c98b5c50 0x5634c98b8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.316+0000 7fd9eaffd640 1 --2- 172.21.15.43:0/1733447914 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5634c98b5c50 0x5634c98b8040 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=939737520b0756a3 server_cookie=dfa80fa4269928f5 in_seq=0 out_seq=0 2024-09-11T23:24:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.316+0000 7fd9ceff5640 10 client.15978 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.322+0000 7fd9ceff5640 1 -- 172.21.15.43:0/1733447914 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634c980ca20 con 0x5634c98b5c50 2024-09-11T23:24:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.322+0000 7fd9ceff5640 10 client.15978 handle_command_reply: tid=0 2024-09-11T23:24:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.327+0000 7fd9f402c740 1 client.15978 shutdown 2024-09-11T23:24:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.331+0000 7fd9f402c740 1 -- 172.21.15.43:0/1733447914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c98a5cb0 msgr2=0x5634c98a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.331+0000 7fd9f402c740 1 --2- 172.21.15.43:0/1733447914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c98a5cb0 0x5634c98a35e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0012ae0 tx=0x7fd9e0004730 comp rx=0 tx=0).stop 2024-09-11T23:24:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.333+0000 7fd9f402c740 1 -- 172.21.15.43:0/1733447914 shutdown_connections 2024-09-11T23:24:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.333+0000 7fd9f402c740 1 --2- 172.21.15.43:0/1733447914 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5634c98b5c50 0x5634c98b8040 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.333+0000 7fd9f402c740 1 --2- 172.21.15.43:0/1733447914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c98a5cb0 0x5634c98a35e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.333+0000 7fd9f402c740 1 --2- 172.21.15.43:0/1733447914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989b410 0x5634c98a30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.333+0000 7fd9f402c740 1 --2- 172.21.15.43:0/1733447914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c989aaf0 0x5634c98a2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.333+0000 7fd9f402c740 1 -- 172.21.15.43:0/1733447914 >> 172.21.15.43:0/1733447914 conn(0x5634c9897060 msgr2=0x5634c989cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.334+0000 7fd9ceff5640 0 client.15978 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.338+0000 7fd9f402c740 1 -- 172.21.15.43:0/1733447914 shutdown_connections 2024-09-11T23:24:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.341+0000 7fd9f402c740 1 -- 172.21.15.43:0/1733447914 wait complete. 2024-09-11T23:24:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.343+0000 7fd9f402c740 20 client.15978 trim_cache size 0 max 16384 2024-09-11T23:24:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.375+0000 7fd9f402c740 1 Processor -- start 2024-09-11T23:24:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.375+0000 7fd9f402c740 1 -- start start 2024-09-11T23:24:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.375+0000 7fd9f402c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c989aaf0 0x5634c98a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.375+0000 7fd9f402c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989b410 0x5634c98a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.375+0000 7fd9f402c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c98a5cb0 0x5634c98bd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.375+0000 7fd9f402c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec1b22e0 con 0x5634c989aaf0 2024-09-11T23:24:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.375+0000 7fd9f402c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec1afbb0 con 0x5634c98a5cb0 2024-09-11T23:24:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.375+0000 7fd9f402c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec1ada00 con 0x5634c989b410 2024-09-11T23:24:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.377+0000 7fd9ea7fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c98a5cb0 0x5634c98bd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.377+0000 7fd9ea7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c989aaf0 0x5634c98a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.377+0000 7fd9ea7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c989aaf0 0x5634c98a33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41476/0 (socket says 172.21.15.43:41476) 2024-09-11T23:24:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.377+0000 7fd9ea7fc640 1 -- 172.21.15.43:0/2742424664 learned_addr learned my addr 172.21.15.43:0/2742424664 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.379+0000 7fd9eaffd640 1 --2- 172.21.15.43:0/2742424664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989b410 0x5634c98a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.381+0000 7fd9ea7fc640 1 -- 172.21.15.43:0/2742424664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989b410 msgr2=0x5634c98a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.381+0000 7fd9ea7fc640 1 --2- 172.21.15.43:0/2742424664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989b410 0x5634c98a4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.381+0000 7fd9ea7fc640 1 -- 172.21.15.43:0/2742424664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c989aaf0 msgr2=0x5634c98a33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.381+0000 7fd9ea7fc640 1 --2- 172.21.15.43:0/2742424664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c989aaf0 0x5634c98a33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.381+0000 7fd9ea7fc640 1 -- 172.21.15.43:0/2742424664 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c989b240 con 0x5634c98a5cb0 2024-09-11T23:24:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.381+0000 7fd9eaffd640 1 --2- 172.21.15.43:0/2742424664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989b410 0x5634c98a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:24:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.384+0000 7fd9ea7fc640 1 --2- 172.21.15.43:0/2742424664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c98a5cb0 0x5634c98bd930 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0005ef0 tx=0x7fd9e0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdc62104de2e7e02 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.388+0000 7fd9d0ff9640 1 -- 172.21.15.43:0/2742424664 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e000e790 con 0x5634c98a5cb0 2024-09-11T23:24:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.389+0000 7fd9d0ff9640 1 -- 172.21.15.43:0/2742424664 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9e000e350 con 0x5634c98a5cb0 2024-09-11T23:24:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.389+0000 7fd9d0ff9640 1 -- 172.21.15.43:0/2742424664 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0010b00 con 0x5634c98a5cb0 2024-09-11T23:24:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.390+0000 7fd9f402c740 1 -- 172.21.15.43:0/2742424664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c98a5cb0 msgr2=0x5634c98bd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:24:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.390+0000 7fd9f402c740 1 --2- 172.21.15.43:0/2742424664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c98a5cb0 0x5634c98bd930 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0005ef0 tx=0x7fd9e0004730 comp rx=0 tx=0).stop 2024-09-11T23:24:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.390+0000 7fd9f402c740 1 -- 172.21.15.43:0/2742424664 shutdown_connections 2024-09-11T23:24:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.390+0000 7fd9f402c740 1 --2- 172.21.15.43:0/2742424664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c98a5cb0 0x5634c98bd930 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.390+0000 7fd9f402c740 1 --2- 172.21.15.43:0/2742424664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989b410 0x5634c98a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.390+0000 7fd9f402c740 1 --2- 172.21.15.43:0/2742424664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c989aaf0 0x5634c98a33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.390+0000 7fd9f402c740 1 -- 172.21.15.43:0/2742424664 >> 172.21.15.43:0/2742424664 conn(0x5634c9897060 msgr2=0x5634c98a43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:24:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.390+0000 7fd9f402c740 1 -- 172.21.15.43:0/2742424664 shutdown_connections 2024-09-11T23:24:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.390+0000 7fd9f402c740 1 -- 172.21.15.43:0/2742424664 wait complete. 2024-09-11T23:24:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.391+0000 7fd9f402c740 1 -- start start 2024-09-11T23:24:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.391+0000 7fd9f402c740 1 Processor -- start 2024-09-11T23:24:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.391+0000 7fd9f402c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c989aaf0 0x5634c98ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.391+0000 7fd9f402c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989b410 0x5634c98a9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.391+0000 7fd9f402c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c98a9b50 0x5634c98a9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.391+0000 7fd9f402c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec1ae070 con 0x5634c989aaf0 2024-09-11T23:24:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.391+0000 7fd9f402c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec1af4e0 con 0x5634c98a9b50 2024-09-11T23:24:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.391+0000 7fd9f402c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec1ae740 con 0x5634c989b410 2024-09-11T23:24:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.391+0000 7fd9f402c740 10 client.0 _ll_register_callbacks cb 0x7fd9e000ee10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:24:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.392+0000 7fd9ea7fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c98a9b50 0x5634c98a9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.392+0000 7fd9ea7fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c98a9b50 0x5634c98a9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:50136/0 (socket says 172.21.15.43:50136) 2024-09-11T23:24:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.392+0000 7fd9ea7fc640 1 -- 172.21.15.43:0/2703362434 learned_addr learned my addr 172.21.15.43:0/2703362434 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:24:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.393+0000 7fd9ea7fc640 1 --2- 172.21.15.43:0/2703362434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c989aaf0 0x5634c98ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.393+0000 7fd9ea7fc640 1 -- 172.21.15.43:0/2703362434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989b410 msgr2=0x5634c98a9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.393+0000 7fd9ea7fc640 1 --2- 172.21.15.43:0/2703362434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989b410 0x5634c98a9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.393+0000 7fd9ea7fc640 1 -- 172.21.15.43:0/2703362434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c989aaf0 msgr2=0x5634c98ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:24:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.393+0000 7fd9ea7fc640 1 --2- 172.21.15.43:0/2703362434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c989aaf0 0x5634c98ae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:24:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.393+0000 7fd9ea7fc640 1 -- 172.21.15.43:0/2703362434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634c9928160 con 0x5634c98a9b50 2024-09-11T23:24:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.397+0000 7fd9ea7fc640 1 --2- 172.21.15.43:0/2703362434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c98a9b50 0x5634c98a9fd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00126d0 tx=0x7fd9e00041c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d76174a0a0dd454f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:24:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.400+0000 7fd9ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:24:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.400+0000 7fd9ceff5640 1 -- 172.21.15.43:0/2703362434 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e000e790 con 0x5634c98a9b50 2024-09-11T23:24:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.400+0000 7fd9f402c740 1 -- 172.21.15.43:0/2703362434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9e000e350 con 0x5634c98a9b50 2024-09-11T23:24:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.400+0000 7fd9ceff5640 1 -- 172.21.15.43:0/2703362434 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fd9e0010b00 con 0x5634c98a9b50 2024-09-11T23:24:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.401+0000 7fd9ceff5640 1 -- 172.21.15.43:0/2703362434 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd9e0010d00 con 0x5634c98a9b50 2024-09-11T23:24:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.403+0000 7fd9f402c740 10 client.25520 fetch_fsmap learned FSMap version 26 2024-09-11T23:24:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.403+0000 7fd9f402c740 1 -- 172.21.15.43:0/2703362434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5634c99293a0 con 0x5634c98a9b50 2024-09-11T23:24:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.404+0000 7fd9ceff5640 1 -- 172.21.15.43:0/2703362434 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9e000e350 con 0x5634c98a9b50 2024-09-11T23:24:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.405+0000 7fd9ceff5640 1 -- 172.21.15.43:0/2703362434 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd9e001f070 con 0x5634c98a9b50 2024-09-11T23:24:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.405+0000 7fd9f402c740 10 client.25520 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:24:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.405+0000 7fd9f402c740 10 client.25520 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:24:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.405+0000 7fd9f402c740 20 client.25520 populate_metadata read hostname 'smithi043' 2024-09-11T23:24:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.405+0000 7fd9f402c740 1 --2- 172.21.15.43:0/2703362434 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5634c9929a40 0x5634c992be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:24:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.405+0000 7fd9f402c740 4 client.25520 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:24:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.405+0000 7fd9f402c740 1 -- 172.21.15.43:0/2703362434 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5634c9806d70 con 0x5634c9929a40 2024-09-11T23:24:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.406+0000 7fd9eaffd640 1 --2- 172.21.15.43:0/2703362434 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5634c9929a40 0x5634c992be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:24:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.410+0000 7fd9eaffd640 1 --2- 172.21.15.43:0/2703362434 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5634c9929a40 0x5634c992be60 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ff0c74971b8810 server_cookie=eb24a4b238e274c9 in_seq=0 out_seq=0 2024-09-11T23:24:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:24:58.410+0000 7fd9ceff5640 10 client.25520 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:24:59.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:24:59 smithi043 ceph-mon[30950]: pgmap v97: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 34 MiB/s wr, 5.23k op/s 2024-09-11T23:24:59.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:24:59 smithi090 ceph-mon[35130]: pgmap v97: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 34 MiB/s wr, 5.23k op/s 2024-09-11T23:24:59.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:24:59 smithi097 ceph-mon[35562]: pgmap v97: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 34 MiB/s wr, 5.23k op/s 2024-09-11T23:25:01.466 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:01 smithi043 ceph-mon[30950]: pgmap v98: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 39 MiB/s wr, 5.98k op/s 2024-09-11T23:25:01.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:01 smithi090 ceph-mon[35130]: pgmap v98: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 39 MiB/s wr, 5.98k op/s 2024-09-11T23:25:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:01 smithi097 ceph-mon[35562]: pgmap v98: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 39 MiB/s wr, 5.98k op/s 2024-09-11T23:25:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.879+0000 7fd9ceff5640 1 -- 172.21.15.43:0/2703362434 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5634c9806d70 con 0x5634c9929a40 2024-09-11T23:25:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.879+0000 7fd9ceff5640 10 client.25520 handle_command_reply: tid=0 2024-09-11T23:25:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.880+0000 7fd9f402c740 1 client.25520 shutdown 2024-09-11T23:25:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.880+0000 7fd9f402c740 1 -- 172.21.15.43:0/2703362434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c98a9b50 msgr2=0x5634c98a9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.880+0000 7fd9f402c740 1 --2- 172.21.15.43:0/2703362434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c98a9b50 0x5634c98a9fd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00126d0 tx=0x7fd9e00041c0 comp rx=0 tx=0).stop 2024-09-11T23:25:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.880+0000 7fd9f402c740 1 -- 172.21.15.43:0/2703362434 shutdown_connections 2024-09-11T23:25:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.880+0000 7fd9f402c740 1 --2- 172.21.15.43:0/2703362434 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5634c9929a40 0x5634c992be60 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.880+0000 7fd9f402c740 1 --2- 172.21.15.43:0/2703362434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634c98a9b50 0x5634c98a9fd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.880+0000 7fd9f402c740 1 --2- 172.21.15.43:0/2703362434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5634c989b410 0x5634c98a9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.880+0000 7fd9f402c740 1 --2- 172.21.15.43:0/2703362434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634c989aaf0 0x5634c98ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.880+0000 7fd9f402c740 1 -- 172.21.15.43:0/2703362434 >> 172.21.15.43:0/2703362434 conn(0x5634c9897060 msgr2=0x5634c98a43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:25:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:25:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:25:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:25:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.880+0000 7fd9ceff5640 0 client.25520 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:25:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.880+0000 7fd9f402c740 1 -- 172.21.15.43:0/2703362434 shutdown_connections 2024-09-11T23:25:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.880+0000 7fd9f402c740 1 -- 172.21.15.43:0/2703362434 wait complete. 2024-09-11T23:25:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.880+0000 7fd9f402c740 20 client.25520 trim_cache size 0 max 16384 2024-09-11T23:25:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.881+0000 7fd9f128d640 1 -- 172.21.15.43:0/389688002 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fd9d40a2d70 msgr2=0x7fd9d40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.881+0000 7fd9f128d640 1 --2- 172.21.15.43:0/389688002 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fd9d40a2d70 0x7fd9d40a5230 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd9e402fda0 tx=0x7fd9e408b790 comp rx=0 tx=0).stop 2024-09-11T23:25:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.881+0000 7fd9f128d640 1 -- 172.21.15.43:0/389688002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9ec10c0a0 msgr2=0x7fd9ec19ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.881+0000 7fd9f128d640 1 --2- 172.21.15.43:0/389688002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9ec10c0a0 0x7fd9ec19ee30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd9e402ae70 tx=0x7fd9e402f5d0 comp rx=0 tx=0).stop 2024-09-11T23:25:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.881+0000 7fd9f128d640 1 -- 172.21.15.43:0/389688002 shutdown_connections 2024-09-11T23:25:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.881+0000 7fd9f128d640 1 --2- 172.21.15.43:0/389688002 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fd9d40a2d70 0x7fd9d40a5230 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.881+0000 7fd9f128d640 1 --2- 172.21.15.43:0/389688002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9ec10c0a0 0x7fd9ec19ee30 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.881+0000 7fd9f128d640 1 --2- 172.21.15.43:0/389688002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9ec1035a0 0x7fd9ec19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.881+0000 7fd9f128d640 1 --2- 172.21.15.43:0/389688002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ec102c80 0x7fd9ec19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.881+0000 7fd9f128d640 1 -- 172.21.15.43:0/389688002 >> 172.21.15.43:0/389688002 conn(0x7fd9ec0779a0 msgr2=0x7fd9ec0fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.882+0000 7fd9f128d640 1 -- 172.21.15.43:0/389688002 shutdown_connections 2024-09-11T23:25:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:01.882+0000 7fd9f128d640 1 -- 172.21.15.43:0/389688002 wait complete. 2024-09-11T23:25:02.502 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:25:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.381+0000 7fd7e20dd640 1 -- 172.21.15.43:0/1784789569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7dc109830 msgr2=0x7fd7dc109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.381+0000 7fd7e20dd640 1 --2- 172.21.15.43:0/1784789569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7dc109830 0x7fd7dc109c90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009860 tx=0x7fd7d002a360 comp rx=0 tx=0).stop 2024-09-11T23:25:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.381+0000 7fd7e20dd640 1 -- 172.21.15.43:0/1784789569 shutdown_connections 2024-09-11T23:25:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.381+0000 7fd7e20dd640 1 --2- 172.21.15.43:0/1784789569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7dc11b140 0x7fd7dc11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.381+0000 7fd7e20dd640 1 --2- 172.21.15.43:0/1784789569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7dc109830 0x7fd7dc109c90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.381+0000 7fd7e20dd640 1 --2- 172.21.15.43:0/1784789569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7dc109450 0x7fd7dc110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.381+0000 7fd7e20dd640 1 -- 172.21.15.43:0/1784789569 >> 172.21.15.43:0/1784789569 conn(0x7fd7dc06e140 msgr2=0x7fd7dc06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.382+0000 7fd7e20dd640 1 -- 172.21.15.43:0/1784789569 shutdown_connections 2024-09-11T23:25:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.383+0000 7fd7e20dd640 1 -- 172.21.15.43:0/1784789569 wait complete. 2024-09-11T23:25:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.384+0000 7fd7e20dd640 1 Processor -- start 2024-09-11T23:25:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.384+0000 7fd7e20dd640 1 -- start start 2024-09-11T23:25:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.384+0000 7fd7e20dd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7dc109450 0x7fd7dc1a73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.384+0000 7fd7e20dd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7dc109830 0x7fd7dc1a7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.384+0000 7fd7e20dd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7dc11b140 0x7fd7dc1abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.384+0000 7fd7e20dd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc120e30 con 0x7fd7dc109450 2024-09-11T23:25:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.384+0000 7fd7e20dd640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc120cb0 con 0x7fd7dc11b140 2024-09-11T23:25:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.385+0000 7fd7e20dd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc120fb0 con 0x7fd7dc109830 2024-09-11T23:25:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.386+0000 7fd7db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7dc109450 0x7fd7dc1a73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.386+0000 7fd7db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7dc109450 0x7fd7dc1a73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55688/0 (socket says 172.21.15.43:55688) 2024-09-11T23:25:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.386+0000 7fd7db7fe640 1 -- 172.21.15.43:0/944644711 learned_addr learned my addr 172.21.15.43:0/944644711 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.386+0000 7fd7dbfff640 1 --2- 172.21.15.43:0/944644711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7dc11b140 0x7fd7dc1abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.386+0000 7fd7db7fe640 1 -- 172.21.15.43:0/944644711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7dc109830 msgr2=0x7fd7dc1a7930 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:25:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.386+0000 7fd7db7fe640 1 --2- 172.21.15.43:0/944644711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7dc109830 0x7fd7dc1a7930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.386+0000 7fd7db7fe640 1 -- 172.21.15.43:0/944644711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7dc11b140 msgr2=0x7fd7dc1abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.386+0000 7fd7db7fe640 1 --2- 172.21.15.43:0/944644711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7dc11b140 0x7fd7dc1abcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.386+0000 7fd7db7fe640 1 -- 172.21.15.43:0/944644711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7dc1ac3a0 con 0x7fd7dc109450 2024-09-11T23:25:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.387+0000 7fd7db7fe640 1 --2- 172.21.15.43:0/944644711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7dc109450 0x7fd7dc1a73f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00e9c0 tx=0x7fd7cc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.387+0000 7fd7d8ff9640 1 -- 172.21.15.43:0/944644711 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc00ccc0 con 0x7fd7dc109450 2024-09-11T23:25:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.387+0000 7fd7e20dd640 1 -- 172.21.15.43:0/944644711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7dc1ac690 con 0x7fd7dc109450 2024-09-11T23:25:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.388+0000 7fd7e20dd640 1 -- 172.21.15.43:0/944644711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7dc110920 con 0x7fd7dc109450 2024-09-11T23:25:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.388+0000 7fd7d8ff9640 1 -- 172.21.15.43:0/944644711 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7cc005500 con 0x7fd7dc109450 2024-09-11T23:25:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.388+0000 7fd7d8ff9640 1 -- 172.21.15.43:0/944644711 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc010430 con 0x7fd7dc109450 2024-09-11T23:25:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.393+0000 7fd7d8ff9640 1 -- 172.21.15.43:0/944644711 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd7cc0046c0 con 0x7fd7dc109450 2024-09-11T23:25:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.394+0000 7fd7d8ff9640 1 --2- 172.21.15.43:0/944644711 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fd7c40b3e90 0x7fd7c40b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.394+0000 7fd7d8ff9640 1 -- 172.21.15.43:0/944644711 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fd7cc099c30 con 0x7fd7dc109450 2024-09-11T23:25:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.394+0000 7fd7daffd640 1 --2- 172.21.15.43:0/944644711 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fd7c40b3e90 0x7fd7c40b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.395+0000 7fd7daffd640 1 --2- 172.21.15.43:0/944644711 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fd7c40b3e90 0x7fd7c40b6350 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00095f0 tx=0x7fd7d0005760 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:02.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.398+0000 7fd7e44b6740 1 Processor -- start 2024-09-11T23:25:02.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.398+0000 7fd7e44b6740 1 -- start start 2024-09-11T23:25:02.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.398+0000 7fd7e44b6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72a2b40 0x5565c72a1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.398+0000 7fd7e44b6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565c72a15d0 0x5565c72a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.398+0000 7fd7e44b6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72ac370 0x5565c72ae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.399+0000 7fd7db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72a2b40 0x5565c72a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.399+0000 7fd7db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72a2b40 0x5565c72a1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55700/0 (socket says 172.21.15.43:55700) 2024-09-11T23:25:02.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.400+0000 7fd7daffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565c72a15d0 0x5565c72a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.400+0000 7fd7db7fe640 1 -- 172.21.15.43:0/1050056680 learned_addr learned my addr 172.21.15.43:0/1050056680 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:02.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.400+0000 7fd7e44b6740 1 -- 172.21.15.43:0/1050056680 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5565c72072f0 con 0x5565c72a2b40 2024-09-11T23:25:02.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.400+0000 7fd7e44b6740 1 -- 172.21.15.43:0/1050056680 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5565c71aecf0 con 0x5565c72ac370 2024-09-11T23:25:02.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.400+0000 7fd7e44b6740 1 -- 172.21.15.43:0/1050056680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5565c72032b0 con 0x5565c72a15d0 2024-09-11T23:25:02.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.402+0000 7fd7daffd640 1 -- 172.21.15.43:0/1050056680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72ac370 msgr2=0x5565c72ae760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:25:02.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.402+0000 7fd7daffd640 1 --2- 172.21.15.43:0/1050056680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72ac370 0x5565c72ae760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.402+0000 7fd7daffd640 1 -- 172.21.15.43:0/1050056680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72a2b40 msgr2=0x5565c72a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:02.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.402+0000 7fd7daffd640 1 --2- 172.21.15.43:0/1050056680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72a2b40 0x5565c72a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.402+0000 7fd7daffd640 1 -- 172.21.15.43:0/1050056680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565c71933a0 con 0x5565c72a15d0 2024-09-11T23:25:02.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.403+0000 7fd7db7fe640 1 --2- 172.21.15.43:0/1050056680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72a2b40 0x5565c72a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:25:02.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.405+0000 7fd7daffd640 1 --2- 172.21.15.43:0/1050056680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565c72a15d0 0x5565c72a1a10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009780 tx=0x7fd7d0039000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48c1e2f2bf2f02d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:02.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.405+0000 7fd7c1ffb640 1 -- 172.21.15.43:0/1050056680 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0037040 con 0x5565c72a15d0 2024-09-11T23:25:02.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.405+0000 7fd7c1ffb640 1 -- 172.21.15.43:0/1050056680 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7d0030580 con 0x5565c72a15d0 2024-09-11T23:25:02.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.406+0000 7fd7c1ffb640 1 -- 172.21.15.43:0/1050056680 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0003a90 con 0x5565c72a15d0 2024-09-11T23:25:02.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.409+0000 7fd7e44b6740 1 -- 172.21.15.43:0/1050056680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565c72a15d0 msgr2=0x5565c72a1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:02.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.409+0000 7fd7e44b6740 1 --2- 172.21.15.43:0/1050056680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565c72a15d0 0x5565c72a1a10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009780 tx=0x7fd7d0039000 comp rx=0 tx=0).stop 2024-09-11T23:25:02.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.412+0000 7fd7e44b6740 1 -- 172.21.15.43:0/1050056680 shutdown_connections 2024-09-11T23:25:02.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.412+0000 7fd7e44b6740 1 --2- 172.21.15.43:0/1050056680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72ac370 0x5565c72ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.412+0000 7fd7e44b6740 1 --2- 172.21.15.43:0/1050056680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565c72a15d0 0x5565c72a1a10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.412+0000 7fd7e44b6740 1 --2- 172.21.15.43:0/1050056680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72a2b40 0x5565c72a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.412+0000 7fd7e44b6740 1 -- 172.21.15.43:0/1050056680 >> 172.21.15.43:0/1050056680 conn(0x5565c726a310 msgr2=0x5565c729dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:02.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.414+0000 7fd7e44b6740 1 -- 172.21.15.43:0/1050056680 shutdown_connections 2024-09-11T23:25:02.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.414+0000 7fd7e44b6740 1 -- 172.21.15.43:0/1050056680 wait complete. 2024-09-11T23:25:02.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.415+0000 7fd7e44b6740 1 -- start start 2024-09-11T23:25:02.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.415+0000 7fd7e44b6740 1 Processor -- start 2024-09-11T23:25:02.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.416+0000 7fd7e44b6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72a15d0 0x5565c72abd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.416+0000 7fd7e44b6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72a2b40 0x5565c72a8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.416+0000 7fd7e44b6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565c72ac370 0x5565c72a9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.416+0000 7fd7e44b6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5565c72aefb0 con 0x5565c72a2b40 2024-09-11T23:25:02.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.416+0000 7fd7e44b6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5565c71c2060 con 0x5565c72a15d0 2024-09-11T23:25:02.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.416+0000 7fd7db7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72a15d0 0x5565c72abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.416+0000 7fd7e44b6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5565c72af130 con 0x5565c72ac370 2024-09-11T23:25:02.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.417+0000 7fd7e44b6740 10 client.0 _ll_register_callbacks cb 0x5565c6f158c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:02.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.418+0000 7fd7db7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72a15d0 0x5565c72abd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:41558/0 (socket says 172.21.15.43:41558) 2024-09-11T23:25:02.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.418+0000 7fd7db7fe640 1 -- 172.21.15.43:0/3083727240 learned_addr learned my addr 172.21.15.43:0/3083727240 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:02.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.418+0000 7fd7daffd640 1 --2- 172.21.15.43:0/3083727240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72a2b40 0x5565c72a8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.417+0000 7fd7dbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565c72ac370 0x5565c72a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.419+0000 7fd7db7fe640 1 -- 172.21.15.43:0/3083727240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565c72ac370 msgr2=0x5565c72a9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:02.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.419+0000 7fd7db7fe640 1 --2- 172.21.15.43:0/3083727240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565c72ac370 0x5565c72a9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.419+0000 7fd7db7fe640 1 -- 172.21.15.43:0/3083727240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72a2b40 msgr2=0x5565c72a8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:02.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.419+0000 7fd7db7fe640 1 --2- 172.21.15.43:0/3083727240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72a2b40 0x5565c72a8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.419+0000 7fd7db7fe640 1 -- 172.21.15.43:0/3083727240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565c6f783c0 con 0x5565c72a15d0 2024-09-11T23:25:02.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.419+0000 7fd7db7fe640 1 --2- 172.21.15.43:0/3083727240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72a15d0 0x5565c72abd20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc09bb20 tx=0x7fd7cc09c300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8a7c8b7dc3949da server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:02.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.419+0000 7fd7baffd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:25:02.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.420+0000 7fd7baffd640 1 -- 172.21.15.43:0/3083727240 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0b8020 con 0x5565c72a15d0 2024-09-11T23:25:02.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.420+0000 7fd7baffd640 1 -- 172.21.15.43:0/3083727240 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fd7cc0ba2c0 con 0x5565c72a15d0 2024-09-11T23:25:02.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.420+0000 7fd7e44b6740 1 -- 172.21.15.43:0/3083727240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7d0030580 con 0x5565c72a15d0 2024-09-11T23:25:02.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.422+0000 7fd7baffd640 1 -- 172.21.15.43:0/3083727240 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd7cc0a4800 con 0x5565c72a15d0 2024-09-11T23:25:02.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.422+0000 7fd7e44b6740 10 client.25529 fetch_fsmap learned FSMap version 26 2024-09-11T23:25:02.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.422+0000 7fd7e44b6740 1 -- 172.21.15.43:0/3083727240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5565c72baa60 con 0x5565c72a15d0 2024-09-11T23:25:02.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.422+0000 7fd7baffd640 1 -- 172.21.15.43:0/3083727240 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7cc0049b0 con 0x5565c72a15d0 2024-09-11T23:25:02.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.422+0000 7fd7baffd640 1 -- 172.21.15.43:0/3083727240 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd7cc097970 con 0x5565c72a15d0 2024-09-11T23:25:02.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.422+0000 7fd7e44b6740 10 client.25529 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:25:02.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.423+0000 7fd7e44b6740 10 client.25529 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T23:25:02.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.423+0000 7fd7e44b6740 20 client.25529 populate_metadata read hostname 'smithi043' 2024-09-11T23:25:02.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.423+0000 7fd7e44b6740 1 --2- 172.21.15.43:0/3083727240 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5565c72bbfd0 0x5565c72be3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.423+0000 7fd7e44b6740 4 client.25529 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:25:02.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.423+0000 7fd7e44b6740 1 -- 172.21.15.43:0/3083727240 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd7d0030580 con 0x5565c72bbfd0 2024-09-11T23:25:02.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.427+0000 7fd7daffd640 1 --2- 172.21.15.43:0/3083727240 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5565c72bbfd0 0x5565c72be3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.429+0000 7fd7daffd640 1 --2- 172.21.15.43:0/3083727240 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5565c72bbfd0 0x5565c72be3f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a74bc1ce203a15c server_cookie=f689464982a9bcc4 in_seq=0 out_seq=0 2024-09-11T23:25:02.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.429+0000 7fd7baffd640 10 client.25529 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:25:02.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.431+0000 7fd7baffd640 1 -- 172.21.15.43:0/3083727240 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fd7d0030580 con 0x5565c72bbfd0 2024-09-11T23:25:02.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.431+0000 7fd7baffd640 10 client.25529 handle_command_reply: tid=0 2024-09-11T23:25:02.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.431+0000 7fd7e44b6740 1 client.25529 shutdown 2024-09-11T23:25:02.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.432+0000 7fd7e44b6740 1 -- 172.21.15.43:0/3083727240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72a15d0 msgr2=0x5565c72abd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:02.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.432+0000 7fd7e44b6740 1 --2- 172.21.15.43:0/3083727240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72a15d0 0x5565c72abd20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc09bb20 tx=0x7fd7cc09c300 comp rx=0 tx=0).stop 2024-09-11T23:25:02.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.432+0000 7fd7e44b6740 1 -- 172.21.15.43:0/3083727240 shutdown_connections 2024-09-11T23:25:02.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.432+0000 7fd7e44b6740 1 --2- 172.21.15.43:0/3083727240 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5565c72bbfd0 0x5565c72be3f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.432+0000 7fd7e44b6740 1 --2- 172.21.15.43:0/3083727240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565c72ac370 0x5565c72a9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.432+0000 7fd7baffd640 0 client.25529 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:25:02.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.432+0000 7fd7e44b6740 1 --2- 172.21.15.43:0/3083727240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72a2b40 0x5565c72a8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.432+0000 7fd7e44b6740 1 --2- 172.21.15.43:0/3083727240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72a15d0 0x5565c72abd20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.432+0000 7fd7e44b6740 1 -- 172.21.15.43:0/3083727240 >> 172.21.15.43:0/3083727240 conn(0x5565c726a310 msgr2=0x5565c72a39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:02.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.433+0000 7fd7e44b6740 1 -- 172.21.15.43:0/3083727240 shutdown_connections 2024-09-11T23:25:02.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.433+0000 7fd7e44b6740 1 -- 172.21.15.43:0/3083727240 wait complete. 2024-09-11T23:25:02.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.434+0000 7fd7e44b6740 20 client.25529 trim_cache size 0 max 16384 2024-09-11T23:25:02.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.465+0000 7fd7e44b6740 1 Processor -- start 2024-09-11T23:25:02.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.466+0000 7fd7e44b6740 1 -- start start 2024-09-11T23:25:02.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.466+0000 7fd7e44b6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72a15d0 0x5565c72a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.466+0000 7fd7e44b6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565c72a2b40 0x5565c72b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.466+0000 7fd7e44b6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72ac370 0x5565c72c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.466+0000 7fd7e44b6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c8011150 con 0x5565c72ac370 2024-09-11T23:25:02.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.466+0000 7fd7e44b6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c800ea20 con 0x5565c72a15d0 2024-09-11T23:25:02.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.466+0000 7fd7e44b6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c800c870 con 0x5565c72a2b40 2024-09-11T23:25:02.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.467+0000 7fd7db7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72a15d0 0x5565c72a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.467+0000 7fd7db7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72a15d0 0x5565c72a8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:41570/0 (socket says 172.21.15.43:41570) 2024-09-11T23:25:02.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.467+0000 7fd7db7fe640 1 -- 172.21.15.43:0/557875366 learned_addr learned my addr 172.21.15.43:0/557875366 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:02.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.467+0000 7fd7daffd640 1 --2- 172.21.15.43:0/557875366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565c72a2b40 0x5565c72b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.468+0000 7fd7db7fe640 1 -- 172.21.15.43:0/557875366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565c72a2b40 msgr2=0x5565c72b82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:02.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.468+0000 7fd7db7fe640 1 --2- 172.21.15.43:0/557875366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565c72a2b40 0x5565c72b82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.468+0000 7fd7db7fe640 1 -- 172.21.15.43:0/557875366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72ac370 msgr2=0x5565c72c3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:02.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.468+0000 7fd7db7fe640 1 --2- 172.21.15.43:0/557875366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72ac370 0x5565c72c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.468+0000 7fd7db7fe640 1 -- 172.21.15.43:0/557875366 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565c72aead0 con 0x5565c72a15d0 2024-09-11T23:25:02.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.469+0000 7fd7db7fe640 1 --2- 172.21.15.43:0/557875366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72a15d0 0x5565c72a8fa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc098560 tx=0x7fd7cc0a00c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af49193193085dcc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:02.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.470+0000 7fd7c17fa640 1 -- 172.21.15.43:0/557875366 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0b8070 con 0x5565c72a15d0 2024-09-11T23:25:02.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.470+0000 7fd7c17fa640 1 -- 172.21.15.43:0/557875366 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7cc09add0 con 0x5565c72a15d0 2024-09-11T23:25:02.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.470+0000 7fd7c17fa640 1 -- 172.21.15.43:0/557875366 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc097b40 con 0x5565c72a15d0 2024-09-11T23:25:02.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.474+0000 7fd7e44b6740 1 -- 172.21.15.43:0/557875366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72a15d0 msgr2=0x5565c72a8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:02.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.474+0000 7fd7e44b6740 1 --2- 172.21.15.43:0/557875366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72a15d0 0x5565c72a8fa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc098560 tx=0x7fd7cc0a00c0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.477+0000 7fd7e44b6740 1 -- 172.21.15.43:0/557875366 shutdown_connections 2024-09-11T23:25:02.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.477+0000 7fd7e44b6740 1 --2- 172.21.15.43:0/557875366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72ac370 0x5565c72c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.477+0000 7fd7e44b6740 1 --2- 172.21.15.43:0/557875366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565c72a2b40 0x5565c72b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.477+0000 7fd7e44b6740 1 --2- 172.21.15.43:0/557875366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72a15d0 0x5565c72a8fa0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.477+0000 7fd7e44b6740 1 -- 172.21.15.43:0/557875366 >> 172.21.15.43:0/557875366 conn(0x5565c726a310 msgr2=0x5565c72a8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:02.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.480+0000 7fd7e44b6740 1 -- 172.21.15.43:0/557875366 shutdown_connections 2024-09-11T23:25:02.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.480+0000 7fd7e44b6740 1 -- 172.21.15.43:0/557875366 wait complete. 2024-09-11T23:25:02.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.481+0000 7fd7e44b6740 1 -- start start 2024-09-11T23:25:02.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.481+0000 7fd7e44b6740 1 Processor -- start 2024-09-11T23:25:02.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.482+0000 7fd7e44b6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72a15d0 0x5565c72b0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.482+0000 7fd7e44b6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72a2b40 0x5565c72b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.482+0000 7fd7db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72a15d0 0x5565c72b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.482+0000 7fd7db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72a15d0 0x5565c72b0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55728/0 (socket says 172.21.15.43:55728) 2024-09-11T23:25:02.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.482+0000 7fd7e44b6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565c72ac370 0x5565c7325cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.482+0000 7fd7daffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72a2b40 0x5565c72b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.482+0000 7fd7daffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72a2b40 0x5565c72b0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:41584/0 (socket says 172.21.15.43:41584) 2024-09-11T23:25:02.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.482+0000 7fd7daffd640 1 -- 172.21.15.43:0/1917864978 learned_addr learned my addr 172.21.15.43:0/1917864978 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:02.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.483+0000 7fd7dbfff640 1 --2- 172.21.15.43:0/1917864978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565c72ac370 0x5565c7325cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.483+0000 7fd7e44b6740 1 -- 172.21.15.43:0/1917864978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c800cee0 con 0x5565c72a15d0 2024-09-11T23:25:02.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.483+0000 7fd7e44b6740 1 -- 172.21.15.43:0/1917864978 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c800e350 con 0x5565c72a2b40 2024-09-11T23:25:02.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.483+0000 7fd7e44b6740 1 -- 172.21.15.43:0/1917864978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c800d5b0 con 0x5565c72ac370 2024-09-11T23:25:02.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.483+0000 7fd7e44b6740 10 client.0 _ll_register_callbacks cb 0x7fd7cc097e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:02.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.483+0000 7fd7daffd640 1 -- 172.21.15.43:0/1917864978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565c72ac370 msgr2=0x5565c7325cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:02.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.483+0000 7fd7daffd640 1 --2- 172.21.15.43:0/1917864978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565c72ac370 0x5565c7325cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.483+0000 7fd7daffd640 1 -- 172.21.15.43:0/1917864978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72a15d0 msgr2=0x5565c72b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:02.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.483+0000 7fd7daffd640 1 --2- 172.21.15.43:0/1917864978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72a15d0 0x5565c72b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.483+0000 7fd7daffd640 1 -- 172.21.15.43:0/1917864978 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565c72b1320 con 0x5565c72a2b40 2024-09-11T23:25:02.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.484+0000 7fd7e44b6740 1 -- 172.21.15.43:0/1917864978 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7cc09add0 con 0x5565c72a2b40 2024-09-11T23:25:02.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.484+0000 7fd7daffd640 1 --2- 172.21.15.43:0/1917864978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72a2b40 0x5565c72b0a40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0000c00 tx=0x7fd7d0030fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38bd896de41dc14c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:02.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.485+0000 7fd7baffd640 10 client.25535 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:25:02.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.485+0000 7fd7baffd640 1 -- 172.21.15.43:0/1917864978 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0037040 con 0x5565c72a2b40 2024-09-11T23:25:02.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.485+0000 7fd7baffd640 1 -- 172.21.15.43:0/1917864978 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fd7d0030640 con 0x5565c72a2b40 2024-09-11T23:25:02.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.486+0000 7fd7baffd640 1 -- 172.21.15.43:0/1917864978 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd7d003d440 con 0x5565c72a2b40 2024-09-11T23:25:02.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.486+0000 7fd7e44b6740 10 client.25535 fetch_fsmap learned FSMap version 26 2024-09-11T23:25:02.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.486+0000 7fd7e44b6740 1 -- 172.21.15.43:0/1917864978 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5565c7327240 con 0x5565c72a2b40 2024-09-11T23:25:02.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.486+0000 7fd7baffd640 1 -- 172.21.15.43:0/1917864978 <== mon.2 v2:172.21.15.90:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd7d003a030 con 0x5565c72a2b40 2024-09-11T23:25:02.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.487+0000 7fd7e44b6740 10 client.25535 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:25:02.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.487+0000 7fd7e44b6740 10 client.25535 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T23:25:02.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.487+0000 7fd7e44b6740 20 client.25535 populate_metadata read hostname 'smithi043' 2024-09-11T23:25:02.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.487+0000 7fd7e44b6740 1 --2- 172.21.15.43:0/1917864978 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5565c72bbfd0 0x5565c7329420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:02.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.487+0000 7fd7e44b6740 4 client.25535 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:25:02.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.487+0000 7fd7e44b6740 1 -- 172.21.15.43:0/1917864978 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565c7214860 con 0x5565c72bbfd0 2024-09-11T23:25:02.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.487+0000 7fd7db7fe640 1 --2- 172.21.15.43:0/1917864978 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5565c72bbfd0 0x5565c7329420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:02.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.494+0000 7fd7db7fe640 1 --2- 172.21.15.43:0/1917864978 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5565c72bbfd0 0x5565c7329420 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c835a15151107244 server_cookie=125b66e01410b13b in_seq=0 out_seq=0 2024-09-11T23:25:02.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.494+0000 7fd7baffd640 10 client.25535 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:25:02.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.497+0000 7fd7baffd640 1 -- 172.21.15.43:0/1917864978 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565c7214860 con 0x5565c72bbfd0 2024-09-11T23:25:02.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.497+0000 7fd7baffd640 10 client.25535 handle_command_reply: tid=0 2024-09-11T23:25:02.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.497+0000 7fd7e44b6740 1 client.25535 shutdown 2024-09-11T23:25:02.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.498+0000 7fd7e44b6740 1 -- 172.21.15.43:0/1917864978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72a2b40 msgr2=0x5565c72b0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:02.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.498+0000 7fd7e44b6740 1 --2- 172.21.15.43:0/1917864978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72a2b40 0x5565c72b0a40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0000c00 tx=0x7fd7d0030fc0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.501+0000 7fd7e44b6740 1 -- 172.21.15.43:0/1917864978 shutdown_connections 2024-09-11T23:25:02.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.501+0000 7fd7e44b6740 1 --2- 172.21.15.43:0/1917864978 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5565c72bbfd0 0x5565c7329420 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.501+0000 7fd7e44b6740 1 --2- 172.21.15.43:0/1917864978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565c72ac370 0x5565c7325cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.501+0000 7fd7e44b6740 1 --2- 172.21.15.43:0/1917864978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565c72a2b40 0x5565c72b0a40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.501+0000 7fd7e44b6740 1 --2- 172.21.15.43:0/1917864978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5565c72a15d0 0x5565c72b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.501+0000 7fd7e44b6740 1 -- 172.21.15.43:0/1917864978 >> 172.21.15.43:0/1917864978 conn(0x5565c726a310 msgr2=0x5565c72a3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.501+0000 7fd7baffd640 0 client.25535 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:25:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.501+0000 7fd7e44b6740 1 -- 172.21.15.43:0/1917864978 shutdown_connections 2024-09-11T23:25:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.501+0000 7fd7e44b6740 1 -- 172.21.15.43:0/1917864978 wait complete. 2024-09-11T23:25:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.501+0000 7fd7e44b6740 20 client.25535 trim_cache size 0 max 16384 2024-09-11T23:25:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.502+0000 7fd7e20dd640 1 -- 172.21.15.43:0/944644711 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fd7c40b3e90 msgr2=0x7fd7c40b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.502+0000 7fd7e20dd640 1 --2- 172.21.15.43:0/944644711 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fd7c40b3e90 0x7fd7c40b6350 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00095f0 tx=0x7fd7d0005760 comp rx=0 tx=0).stop 2024-09-11T23:25:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.502+0000 7fd7e20dd640 1 -- 172.21.15.43:0/944644711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7dc109450 msgr2=0x7fd7dc1a73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.502+0000 7fd7e20dd640 1 --2- 172.21.15.43:0/944644711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7dc109450 0x7fd7dc1a73f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00e9c0 tx=0x7fd7cc00ee80 comp rx=0 tx=0).stop 2024-09-11T23:25:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.502+0000 7fd7e20dd640 1 -- 172.21.15.43:0/944644711 shutdown_connections 2024-09-11T23:25:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.502+0000 7fd7e20dd640 1 --2- 172.21.15.43:0/944644711 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fd7c40b3e90 0x7fd7c40b6350 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.502+0000 7fd7e20dd640 1 --2- 172.21.15.43:0/944644711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7dc11b140 0x7fd7dc1abcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.502+0000 7fd7e20dd640 1 --2- 172.21.15.43:0/944644711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7dc109830 0x7fd7dc1a7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.502+0000 7fd7e20dd640 1 --2- 172.21.15.43:0/944644711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7dc109450 0x7fd7dc1a73f0 secure :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00e9c0 tx=0x7fd7cc00ee80 comp rx=0 tx=0).stop 2024-09-11T23:25:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.502+0000 7fd7e20dd640 1 -- 172.21.15.43:0/944644711 >> 172.21.15.43:0/944644711 conn(0x7fd7dc06e140 msgr2=0x7fd7dc11bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.504+0000 7fd7e20dd640 1 -- 172.21.15.43:0/944644711 shutdown_connections 2024-09-11T23:25:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:01.505+0000 7fd7e20dd640 1 -- 172.21.15.43:0/944644711 wait complete. 2024-09-11T23:25:03.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:03 smithi090 ceph-mon[35130]: pgmap v99: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 61 MiB/s wr, 8.93k op/s 2024-09-11T23:25:03.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:03 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:25:03.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:03 smithi097 ceph-mon[35562]: pgmap v99: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 61 MiB/s wr, 8.93k op/s 2024-09-11T23:25:03.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:03 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:25:03.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:03 smithi043 ceph-mon[30950]: pgmap v99: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 61 MiB/s wr, 8.93k op/s 2024-09-11T23:25:03.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:03 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:25:05.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:05 smithi090 ceph-mon[35130]: pgmap v100: 97 pgs: 97 active+clean; 14 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 88 MiB/s wr, 12.72k op/s 2024-09-11T23:25:05.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:05 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:25:05.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:05 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:25:05.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:05 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:25:05.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:05 smithi097 ceph-mon[35562]: pgmap v100: 97 pgs: 97 active+clean; 14 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 88 MiB/s wr, 12.72k op/s 2024-09-11T23:25:05.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:05 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:25:05.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:05 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:25:05.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:05 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:25:05.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:05 smithi043 ceph-mon[30950]: pgmap v100: 97 pgs: 97 active+clean; 14 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 88 MiB/s wr, 12.72k op/s 2024-09-11T23:25:05.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:05 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:25:05.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:05 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:25:05.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:05 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:25:07.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:07 smithi090 ceph-mon[35130]: pgmap v101: 97 pgs: 97 active+clean; 14 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 96 MiB/s wr, 13.64k op/s 2024-09-11T23:25:07.800 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:07 smithi097 ceph-mon[35562]: pgmap v101: 97 pgs: 97 active+clean; 14 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 96 MiB/s wr, 13.64k op/s 2024-09-11T23:25:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:07 smithi043 ceph-mon[30950]: pgmap v101: 97 pgs: 97 active+clean; 14 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 96 MiB/s wr, 13.64k op/s 2024-09-11T23:25:09.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:09 smithi043 ceph-mon[30950]: pgmap v102: 97 pgs: 97 active+clean; 14 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 134 MiB/s wr, 18.93k op/s 2024-09-11T23:25:09.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:09 smithi090 ceph-mon[35130]: pgmap v102: 97 pgs: 97 active+clean; 14 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 134 MiB/s wr, 18.93k op/s 2024-09-11T23:25:09.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:09 smithi097 ceph-mon[35562]: pgmap v102: 97 pgs: 97 active+clean; 14 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 134 MiB/s wr, 18.93k op/s 2024-09-11T23:25:11.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:11 smithi090 ceph-mon[35130]: pgmap v103: 97 pgs: 97 active+clean; 14 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 117 MiB/s wr, 15.98k op/s 2024-09-11T23:25:11.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:11 smithi097 ceph-mon[35562]: pgmap v103: 97 pgs: 97 active+clean; 14 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 117 MiB/s wr, 15.98k op/s 2024-09-11T23:25:11.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:11 smithi043 ceph-mon[30950]: pgmap v103: 97 pgs: 97 active+clean; 14 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 117 MiB/s wr, 15.98k op/s 2024-09-11T23:25:13.145 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:25:13.494 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:25:13.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:13 smithi090 ceph-mon[35130]: pgmap v104: 97 pgs: 97 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 127 MiB/s wr, 17.20k op/s 2024-09-11T23:25:13.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:13 smithi097 ceph-mon[35562]: pgmap v104: 97 pgs: 97 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 127 MiB/s wr, 17.20k op/s 2024-09-11T23:25:13.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:13 smithi043 ceph-mon[30950]: pgmap v104: 97 pgs: 97 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 127 MiB/s wr, 17.20k op/s 2024-09-11T23:25:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.044+0000 7f7fd15e8640 1 -- 172.21.15.43:0/2041350711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fcc100810 msgr2=0x7f7fcc100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.044+0000 7f7fd15e8640 1 --2- 172.21.15.43:0/2041350711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fcc100810 0x7f7fcc100bf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009900 tx=0x7f7fbc02e820 comp rx=0 tx=0).stop 2024-09-11T23:25:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.047+0000 7f7fd15e8640 1 -- 172.21.15.43:0/2041350711 shutdown_connections 2024-09-11T23:25:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.047+0000 7f7fd15e8640 1 --2- 172.21.15.43:0/2041350711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fcc1080d0 0x7f7fcc10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.047+0000 7f7fd15e8640 1 --2- 172.21.15.43:0/2041350711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fcc1011c0 0x7f7fcc107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.047+0000 7f7fd15e8640 1 --2- 172.21.15.43:0/2041350711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fcc100810 0x7f7fcc100bf0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.047+0000 7f7fd15e8640 1 -- 172.21.15.43:0/2041350711 >> 172.21.15.43:0/2041350711 conn(0x7f7fcc075eb0 msgr2=0x7f7fcc0762c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.051+0000 7f7fd15e8640 1 -- 172.21.15.43:0/2041350711 shutdown_connections 2024-09-11T23:25:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.052+0000 7f7fd15e8640 1 -- 172.21.15.43:0/2041350711 wait complete. 2024-09-11T23:25:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.052+0000 7f7fd15e8640 1 Processor -- start 2024-09-11T23:25:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.053+0000 7f7fd15e8640 1 -- start start 2024-09-11T23:25:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.053+0000 7f7fd15e8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fcc100810 0x7f7fcc19d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.053+0000 7f7fd15e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fcc1080d0 0x7f7fcc1961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.053+0000 7f7fd15e8640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fcc1966f0 0x7f7fcc196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.053+0000 7f7fd15e8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc10fdd0 con 0x7f7fcc1080d0 2024-09-11T23:25:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.053+0000 7f7fd15e8640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc10fc50 con 0x7f7fcc1966f0 2024-09-11T23:25:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.053+0000 7f7fd15e8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc10ff50 con 0x7f7fcc100810 2024-09-11T23:25:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.054+0000 7f7fcaffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fcc100810 0x7f7fcc19d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.054+0000 7f7fcb7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fcc1966f0 0x7f7fcc196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:15.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.055+0000 7f7fcaffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fcc100810 0x7f7fcc19d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:37338/0 (socket says 172.21.15.43:37338) 2024-09-11T23:25:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.055+0000 7f7fcaffd640 1 -- 172.21.15.43:0/60983878 learned_addr learned my addr 172.21.15.43:0/60983878 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.056+0000 7f7fca7fc640 1 --2- 172.21.15.43:0/60983878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fcc1080d0 0x7f7fcc1961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.056+0000 7f7fcaffd640 1 -- 172.21.15.43:0/60983878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fcc1966f0 msgr2=0x7f7fcc196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.056+0000 7f7fcaffd640 1 --2- 172.21.15.43:0/60983878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fcc1966f0 0x7f7fcc196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.056+0000 7f7fcaffd640 1 -- 172.21.15.43:0/60983878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fcc1080d0 msgr2=0x7f7fcc1961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.056+0000 7f7fcaffd640 1 --2- 172.21.15.43:0/60983878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fcc1080d0 0x7f7fcc1961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.056+0000 7f7fcaffd640 1 -- 172.21.15.43:0/60983878 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fcc19ac50 con 0x7f7fcc100810 2024-09-11T23:25:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.061+0000 7f7fcaffd640 1 --2- 172.21.15.43:0/60983878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fcc100810 0x7f7fcc19d130 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc036450 tx=0x7f7fbc036820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.065+0000 7f7fb3fff640 1 -- 172.21.15.43:0/60983878 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc045070 con 0x7f7fcc100810 2024-09-11T23:25:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.065+0000 7f7fb3fff640 1 -- 172.21.15.43:0/60983878 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fbc003ad0 con 0x7f7fcc100810 2024-09-11T23:25:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.065+0000 7f7fb3fff640 1 -- 172.21.15.43:0/60983878 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc0407c0 con 0x7f7fcc100810 2024-09-11T23:25:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.066+0000 7f7fd15e8640 1 -- 172.21.15.43:0/60983878 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fcc19aee0 con 0x7f7fcc100810 2024-09-11T23:25:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.066+0000 7f7fd15e8640 1 -- 172.21.15.43:0/60983878 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fcc19b3b0 con 0x7f7fcc100810 2024-09-11T23:25:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.073+0000 7f7fb3fff640 1 -- 172.21.15.43:0/60983878 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7fbc040960 con 0x7f7fcc100810 2024-09-11T23:25:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.076+0000 7f7fb3fff640 1 --2- 172.21.15.43:0/60983878 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7fb40a2f00 0x7f7fb40a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.076+0000 7f7fb3fff640 1 -- 172.21.15.43:0/60983878 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f7fbc0f0500 con 0x7f7fcc100810 2024-09-11T23:25:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.077+0000 7f7fca7fc640 1 --2- 172.21.15.43:0/60983878 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7fb40a2f00 0x7f7fb40a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.080+0000 7f7fd4462740 1 Processor -- start 2024-09-11T23:25:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.080+0000 7f7fca7fc640 1 --2- 172.21.15.43:0/60983878 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7fb40a2f00 0x7f7fb40a53c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0006fd0 tx=0x7f7fc0008040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.081+0000 7f7fd4462740 1 -- start start 2024-09-11T23:25:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.082+0000 7f7fd4462740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0857daf0 0x55ad0857ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.082+0000 7f7fd4462740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad0857e410 0x55ad08588770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.082+0000 7f7fd4462740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad08588cb0 0x55ad0858b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.082+0000 7f7fd4462740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ad084e22f0 con 0x55ad0857e410 2024-09-11T23:25:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.082+0000 7f7fd4462740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ad0848a270 con 0x55ad0857daf0 2024-09-11T23:25:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.082+0000 7f7fd4462740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ad084de2b0 con 0x55ad08588cb0 2024-09-11T23:25:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.083+0000 7f7fcb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad08588cb0 0x55ad0858b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.083+0000 7f7fca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad0857e410 0x55ad08588770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.083+0000 7f7fca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad0857e410 0x55ad08588770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58928/0 (socket says 172.21.15.43:58928) 2024-09-11T23:25:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.083+0000 7f7fca7fc640 1 -- 172.21.15.43:0/4294161134 learned_addr learned my addr 172.21.15.43:0/4294161134 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.084+0000 7f7fcaffd640 1 --2- 172.21.15.43:0/4294161134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0857daf0 0x55ad0857ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.084+0000 7f7fca7fc640 1 -- 172.21.15.43:0/4294161134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad08588cb0 msgr2=0x55ad0858b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.084+0000 7f7fca7fc640 1 --2- 172.21.15.43:0/4294161134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad08588cb0 0x55ad0858b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.084+0000 7f7fca7fc640 1 -- 172.21.15.43:0/4294161134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0857daf0 msgr2=0x55ad0857ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.084+0000 7f7fca7fc640 1 --2- 172.21.15.43:0/4294161134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0857daf0 0x55ad0857ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.084+0000 7f7fca7fc640 1 -- 172.21.15.43:0/4294161134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad08489340 con 0x55ad0857e410 2024-09-11T23:25:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.086+0000 7f7fca7fc640 1 --2- 172.21.15.43:0/4294161134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad0857e410 0x55ad08588770 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000d7e0 tx=0x7f7fc0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91eafe8e6e32e1e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:15.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.088+0000 7f7fb17fa640 1 -- 172.21.15.43:0/4294161134 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0024070 con 0x55ad0857e410 2024-09-11T23:25:15.088 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.088+0000 7f7fb17fa640 1 -- 172.21.15.43:0/4294161134 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc001f400 con 0x55ad0857e410 2024-09-11T23:25:15.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.090+0000 7f7fd4462740 1 -- 172.21.15.43:0/4294161134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad0857e410 msgr2=0x55ad08588770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:15.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.090+0000 7f7fd4462740 1 --2- 172.21.15.43:0/4294161134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad0857e410 0x55ad08588770 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000d7e0 tx=0x7f7fc0010000 comp rx=0 tx=0).stop 2024-09-11T23:25:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.091+0000 7f7fd4462740 1 -- 172.21.15.43:0/4294161134 shutdown_connections 2024-09-11T23:25:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.091+0000 7f7fd4462740 1 --2- 172.21.15.43:0/4294161134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad08588cb0 0x55ad0858b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.091+0000 7f7fd4462740 1 --2- 172.21.15.43:0/4294161134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad0857e410 0x55ad08588770 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.091+0000 7f7fd4462740 1 --2- 172.21.15.43:0/4294161134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0857daf0 0x55ad0857ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.091+0000 7f7fd4462740 1 -- 172.21.15.43:0/4294161134 >> 172.21.15.43:0/4294161134 conn(0x55ad0857a060 msgr2=0x55ad0857a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.096+0000 7f7fd4462740 1 -- 172.21.15.43:0/4294161134 shutdown_connections 2024-09-11T23:25:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.097+0000 7f7fd4462740 1 -- 172.21.15.43:0/4294161134 wait complete. 2024-09-11T23:25:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.097+0000 7f7fd4462740 1 -- start start 2024-09-11T23:25:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.098+0000 7f7fd4462740 1 Processor -- start 2024-09-11T23:25:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.098+0000 7f7fd4462740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad0857daf0 0x55ad08585b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.099+0000 7f7fd4462740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0857e410 0x55ad08586070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.099+0000 7f7fd4462740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad08588cb0 0x55ad085865b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.099+0000 7f7fd4462740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ad0858b8d0 con 0x55ad08588cb0 2024-09-11T23:25:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.099+0000 7f7fd4462740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ad0846b290 con 0x55ad0857e410 2024-09-11T23:25:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.099+0000 7f7fd4462740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ad0858ba50 con 0x55ad0857daf0 2024-09-11T23:25:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.099+0000 7f7fd4462740 10 client.0 _ll_register_callbacks cb 0x55ad0849c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.100+0000 7f7fcaffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad0857daf0 0x55ad08585b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.100+0000 7f7fcb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad08588cb0 0x55ad085865b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.100+0000 7f7fcb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad08588cb0 0x55ad085865b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58934/0 (socket says 172.21.15.43:58934) 2024-09-11T23:25:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.100+0000 7f7fcb7fe640 1 -- 172.21.15.43:0/2084521488 learned_addr learned my addr 172.21.15.43:0/2084521488 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.100+0000 7f7fca7fc640 1 --2- 172.21.15.43:0/2084521488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0857e410 0x55ad08586070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.100+0000 7f7fcb7fe640 1 -- 172.21.15.43:0/2084521488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad0857daf0 msgr2=0x55ad08585b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.100+0000 7f7fcb7fe640 1 --2- 172.21.15.43:0/2084521488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad0857daf0 0x55ad08585b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.100+0000 7f7fcb7fe640 1 -- 172.21.15.43:0/2084521488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0857e410 msgr2=0x55ad08586070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.100+0000 7f7fcb7fe640 1 --2- 172.21.15.43:0/2084521488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0857e410 0x55ad08586070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.100+0000 7f7fcb7fe640 1 -- 172.21.15.43:0/2084521488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad081f5210 con 0x55ad08588cb0 2024-09-11T23:25:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.101+0000 7f7fcb7fe640 1 --2- 172.21.15.43:0/2084521488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad08588cb0 0x55ad085865b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400be80 tx=0x7f7fc4007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d13099fc5a4340 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.101+0000 7f7faeff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:25:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.102+0000 7f7faeff5640 1 -- 172.21.15.43:0/2084521488 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4009e20 con 0x55ad08588cb0 2024-09-11T23:25:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.102+0000 7f7fd4462740 1 -- 172.21.15.43:0/2084521488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc001f400 con 0x55ad08588cb0 2024-09-11T23:25:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.102+0000 7f7faeff5640 1 -- 172.21.15.43:0/2084521488 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f7fc4018400 con 0x55ad08588cb0 2024-09-11T23:25:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.102+0000 7f7faeff5640 1 -- 172.21.15.43:0/2084521488 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7fc4025050 con 0x55ad08588cb0 2024-09-11T23:25:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.110+0000 7f7fd4462740 10 client.16005 fetch_fsmap learned FSMap version 26 2024-09-11T23:25:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.110+0000 7f7fd4462740 1 -- 172.21.15.43:0/2084521488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ad082c1620 con 0x55ad08588cb0 2024-09-11T23:25:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.111+0000 7f7faeff5640 1 -- 172.21.15.43:0/2084521488 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc001f400 con 0x55ad08588cb0 2024-09-11T23:25:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.111+0000 7f7faeff5640 1 -- 172.21.15.43:0/2084521488 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7fc4020030 con 0x55ad08588cb0 2024-09-11T23:25:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.112+0000 7f7fd4462740 10 client.16005 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:25:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.112+0000 7f7fd4462740 10 client.16005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:25:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.112+0000 7f7fd4462740 20 client.16005 populate_metadata read hostname 'smithi043' 2024-09-11T23:25:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.112+0000 7f7fd4462740 1 --2- 172.21.15.43:0/2084521488 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55ad08598d80 0x55ad0859b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.112+0000 7f7fd4462740 4 client.16005 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:25:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.112+0000 7f7fd4462740 1 -- 172.21.15.43:0/2084521488 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad084efa20 con 0x55ad08598d80 2024-09-11T23:25:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.116+0000 7f7fcaffd640 1 --2- 172.21.15.43:0/2084521488 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55ad08598d80 0x55ad0859b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.118+0000 7f7fcaffd640 1 --2- 172.21.15.43:0/2084521488 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55ad08598d80 0x55ad0859b200 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93c85d133f8e4545 server_cookie=f19450fa3a5361d9 in_seq=0 out_seq=0 2024-09-11T23:25:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.118+0000 7f7faeff5640 10 client.16005 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:25:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.119+0000 7f7faeff5640 1 -- 172.21.15.43:0/2084521488 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad084efa20 con 0x55ad08598d80 2024-09-11T23:25:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.119+0000 7f7faeff5640 10 client.16005 handle_command_reply: tid=0 2024-09-11T23:25:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.119+0000 7f7fd4462740 1 client.16005 shutdown 2024-09-11T23:25:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.131+0000 7f7fd4462740 1 -- 172.21.15.43:0/2084521488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad08588cb0 msgr2=0x55ad085865b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.131+0000 7f7fd4462740 1 --2- 172.21.15.43:0/2084521488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad08588cb0 0x55ad085865b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400be80 tx=0x7f7fc4007680 comp rx=0 tx=0).stop 2024-09-11T23:25:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.132+0000 7f7fd4462740 1 -- 172.21.15.43:0/2084521488 shutdown_connections 2024-09-11T23:25:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.132+0000 7f7fd4462740 1 --2- 172.21.15.43:0/2084521488 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55ad08598d80 0x55ad0859b200 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.132+0000 7f7fd4462740 1 --2- 172.21.15.43:0/2084521488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad08588cb0 0x55ad085865b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.132+0000 7f7fd4462740 1 --2- 172.21.15.43:0/2084521488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0857e410 0x55ad08586070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.132+0000 7f7fd4462740 1 --2- 172.21.15.43:0/2084521488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad0857daf0 0x55ad08585b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.132+0000 7f7fd4462740 1 -- 172.21.15.43:0/2084521488 >> 172.21.15.43:0/2084521488 conn(0x55ad0857a060 msgr2=0x55ad0857fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.132+0000 7f7faeff5640 0 client.16005 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:25:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.135+0000 7f7fd4462740 1 -- 172.21.15.43:0/2084521488 shutdown_connections 2024-09-11T23:25:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.138+0000 7f7fd4462740 1 -- 172.21.15.43:0/2084521488 wait complete. 2024-09-11T23:25:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.138+0000 7f7fd4462740 20 client.16005 trim_cache size 0 max 16384 2024-09-11T23:25:15.171 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.171+0000 7f7fd4462740 1 Processor -- start 2024-09-11T23:25:15.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.172+0000 7f7fd4462740 1 -- start start 2024-09-11T23:25:15.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.172+0000 7f7fd4462740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0857daf0 0x55ad08580df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:15.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.172+0000 7f7fd4462740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad0857e410 0x55ad085863e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:15.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.172+0000 7f7fd4462740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad08588cb0 0x55ad085a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:15.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.172+0000 7f7fd4462740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb8011200 con 0x55ad0857e410 2024-09-11T23:25:15.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.172+0000 7f7fd4462740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb800ead0 con 0x55ad0857daf0 2024-09-11T23:25:15.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.172+0000 7f7fd4462740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb800c920 con 0x55ad08588cb0 2024-09-11T23:25:15.173 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.173+0000 7f7fcb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad08588cb0 0x55ad085a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:15.173 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.174+0000 7f7fca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad0857e410 0x55ad085863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:15.174 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.174+0000 7f7fcb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad08588cb0 0x55ad085a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:37360/0 (socket says 172.21.15.43:37360) 2024-09-11T23:25:15.174 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.174+0000 7f7fcb7fe640 1 -- 172.21.15.43:0/2734762097 learned_addr learned my addr 172.21.15.43:0/2734762097 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:15.174 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.174+0000 7f7fcaffd640 1 --2- 172.21.15.43:0/2734762097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0857daf0 0x55ad08580df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:15.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.176+0000 7f7fcaffd640 1 -- 172.21.15.43:0/2734762097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad08588cb0 msgr2=0x55ad085a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:15.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.176+0000 7f7fcaffd640 1 --2- 172.21.15.43:0/2734762097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad08588cb0 0x55ad085a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.176+0000 7f7fcaffd640 1 -- 172.21.15.43:0/2734762097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad0857e410 msgr2=0x55ad085863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:15.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.176+0000 7f7fcaffd640 1 --2- 172.21.15.43:0/2734762097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad0857e410 0x55ad085863e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.176+0000 7f7fcaffd640 1 -- 172.21.15.43:0/2734762097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad085752d0 con 0x55ad0857daf0 2024-09-11T23:25:15.177 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.177+0000 7f7fcaffd640 1 --2- 172.21.15.43:0/2734762097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0857daf0 0x55ad08580df0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0edbf0 tx=0x7f7fbc036990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26b40e50d35fffc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.178+0000 7f7fb0ff9640 1 -- 172.21.15.43:0/2734762097 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc0ca070 con 0x55ad0857daf0 2024-09-11T23:25:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.178+0000 7f7fb0ff9640 1 -- 172.21.15.43:0/2734762097 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fbc0c56b0 con 0x55ad0857daf0 2024-09-11T23:25:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.178+0000 7f7fb0ff9640 1 -- 172.21.15.43:0/2734762097 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc0ede70 con 0x55ad0857daf0 2024-09-11T23:25:15.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.182+0000 7f7fd4462740 1 -- 172.21.15.43:0/2734762097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0857daf0 msgr2=0x55ad08580df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:15.182 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.182+0000 7f7fd4462740 1 --2- 172.21.15.43:0/2734762097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0857daf0 0x55ad08580df0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0edbf0 tx=0x7f7fbc036990 comp rx=0 tx=0).stop 2024-09-11T23:25:15.184 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.183+0000 7f7fd4462740 1 -- 172.21.15.43:0/2734762097 shutdown_connections 2024-09-11T23:25:15.184 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.183+0000 7f7fd4462740 1 --2- 172.21.15.43:0/2734762097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad08588cb0 0x55ad085a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.184 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.183+0000 7f7fd4462740 1 --2- 172.21.15.43:0/2734762097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad0857e410 0x55ad085863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.184 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.183+0000 7f7fd4462740 1 --2- 172.21.15.43:0/2734762097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0857daf0 0x55ad08580df0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.184 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.183+0000 7f7fd4462740 1 -- 172.21.15.43:0/2734762097 >> 172.21.15.43:0/2734762097 conn(0x55ad0857a060 msgr2=0x55ad08587bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.187+0000 7f7fd4462740 1 -- 172.21.15.43:0/2734762097 shutdown_connections 2024-09-11T23:25:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.188+0000 7f7fd4462740 1 -- 172.21.15.43:0/2734762097 wait complete. 2024-09-11T23:25:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.189+0000 7f7fd4462740 1 -- start start 2024-09-11T23:25:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.189+0000 7f7fd4462740 1 Processor -- start 2024-09-11T23:25:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.189+0000 7f7fd4462740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad0857daf0 0x55ad086010a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.189+0000 7f7fd4462740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0857e410 0x55ad086015e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.189+0000 7f7fd4462740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad08588cb0 0x55ad08591000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.189+0000 7f7fd4462740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb800cf90 con 0x55ad0857daf0 2024-09-11T23:25:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.189+0000 7f7fd4462740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb800e400 con 0x55ad0857e410 2024-09-11T23:25:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.189+0000 7f7fd4462740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb800d660 con 0x55ad08588cb0 2024-09-11T23:25:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.190+0000 7f7fd4462740 10 client.0 _ll_register_callbacks cb 0x7f7fc4009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.190+0000 7f7fcaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad0857daf0 0x55ad086010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.190+0000 7f7fcaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad0857daf0 0x55ad086010a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58948/0 (socket says 172.21.15.43:58948) 2024-09-11T23:25:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.190+0000 7f7fcaffd640 1 -- 172.21.15.43:0/779935677 learned_addr learned my addr 172.21.15.43:0/779935677 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.190+0000 7f7fcb7fe640 1 --2- 172.21.15.43:0/779935677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad08588cb0 0x55ad08591000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.190+0000 7f7fcb7fe640 1 -- 172.21.15.43:0/779935677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0857e410 msgr2=0x55ad086015e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:25:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.190+0000 7f7fcb7fe640 1 --2- 172.21.15.43:0/779935677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0857e410 0x55ad086015e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.190+0000 7f7fcb7fe640 1 -- 172.21.15.43:0/779935677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad0857daf0 msgr2=0x55ad086010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.190+0000 7f7fcb7fe640 1 --2- 172.21.15.43:0/779935677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad0857daf0 0x55ad086010a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.191+0000 7f7fcb7fe640 1 -- 172.21.15.43:0/779935677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad08591800 con 0x55ad08588cb0 2024-09-11T23:25:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.196+0000 7f7fcb7fe640 1 --2- 172.21.15.43:0/779935677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad08588cb0 0x55ad08591000 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4007b80 tx=0x7f7fc4017eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e96924c44637e95 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.196+0000 7f7faeff5640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-11T23:25:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.202+0000 7f7faeff5640 1 -- 172.21.15.43:0/779935677 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40175a0 con 0x55ad08588cb0 2024-09-11T23:25:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.202+0000 7f7fd4462740 1 -- 172.21.15.43:0/779935677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fbc0c56b0 con 0x55ad08588cb0 2024-09-11T23:25:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.202+0000 7f7faeff5640 1 -- 172.21.15.43:0/779935677 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f7fc4017740 con 0x55ad08588cb0 2024-09-11T23:25:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.204+0000 7f7faeff5640 1 -- 172.21.15.43:0/779935677 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7fc4009e50 con 0x55ad08588cb0 2024-09-11T23:25:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.208+0000 7f7fd4462740 10 client.25531 fetch_fsmap learned FSMap version 26 2024-09-11T23:25:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.208+0000 7f7fd4462740 1 -- 172.21.15.43:0/779935677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ad08614b40 con 0x55ad08588cb0 2024-09-11T23:25:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.213+0000 7f7faeff5640 1 -- 172.21.15.43:0/779935677 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fbc0c56b0 con 0x55ad08588cb0 2024-09-11T23:25:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.215+0000 7f7faeff5640 1 -- 172.21.15.43:0/779935677 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7fc4020040 con 0x55ad08588cb0 2024-09-11T23:25:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.215+0000 7f7fd4462740 10 client.25531 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:25:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.215+0000 7f7fd4462740 10 client.25531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:25:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.215+0000 7f7fd4462740 20 client.25531 populate_metadata read hostname 'smithi043' 2024-09-11T23:25:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.215+0000 7f7fd4462740 1 --2- 172.21.15.43:0/779935677 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55ad086150e0 0x55ad08617500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.215+0000 7f7fd4462740 4 client.25531 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:25:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.215+0000 7f7fd4462740 1 -- 172.21.15.43:0/779935677 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ad084e9d70 con 0x55ad086150e0 2024-09-11T23:25:15.220 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.221+0000 7f7fcaffd640 1 --2- 172.21.15.43:0/779935677 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55ad086150e0 0x55ad08617500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.234+0000 7f7fcaffd640 1 --2- 172.21.15.43:0/779935677 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55ad086150e0 0x55ad08617500 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=593f0c630f919fc2 server_cookie=8aae8e0169222abb in_seq=0 out_seq=0 2024-09-11T23:25:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:15.234+0000 7f7faeff5640 10 client.25531 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:25:15.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:15 smithi090 ceph-mon[35130]: pgmap v105: 97 pgs: 97 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 118 MiB/s wr, 15.91k op/s 2024-09-11T23:25:15.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:15 smithi097 ceph-mon[35562]: pgmap v105: 97 pgs: 97 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 118 MiB/s wr, 15.91k op/s 2024-09-11T23:25:15.831 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:15 smithi043 ceph-mon[30950]: pgmap v105: 97 pgs: 97 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 118 MiB/s wr, 15.91k op/s 2024-09-11T23:25:17.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:17 smithi090 ceph-mon[35130]: pgmap v106: 97 pgs: 97 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 95 MiB/s wr, 12.82k op/s 2024-09-11T23:25:17.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:17 smithi097 ceph-mon[35562]: pgmap v106: 97 pgs: 97 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 95 MiB/s wr, 12.82k op/s 2024-09-11T23:25:17.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:17 smithi043 ceph-mon[30950]: pgmap v106: 97 pgs: 97 active+clean; 15 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 95 MiB/s wr, 12.82k op/s 2024-09-11T23:25:19.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:19 smithi043 ceph-mon[30950]: pgmap v107: 97 pgs: 97 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 114 MiB/s wr, 15.68k op/s 2024-09-11T23:25:19.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:19 smithi090 ceph-mon[35130]: pgmap v107: 97 pgs: 97 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 114 MiB/s wr, 15.68k op/s 2024-09-11T23:25:19.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:19 smithi097 ceph-mon[35562]: pgmap v107: 97 pgs: 97 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 114 MiB/s wr, 15.68k op/s 2024-09-11T23:25:20.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.244+0000 7f132e89a640 1 -- 172.21.15.43:0/2917738195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1328100e20 msgr2=0x7f1328101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.244+0000 7f132e89a640 1 --2- 172.21.15.43:0/2917738195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1328100e20 0x7f1328101200 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1318009900 tx=0x7f131802a480 comp rx=0 tx=0).stop 2024-09-11T23:25:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.244+0000 7f132e89a640 1 -- 172.21.15.43:0/2917738195 shutdown_connections 2024-09-11T23:25:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.244+0000 7f132e89a640 1 --2- 172.21.15.43:0/2917738195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1328102ef0 0x7f132810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.244+0000 7f132e89a640 1 --2- 172.21.15.43:0/2917738195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1328102570 0x7f13281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.244+0000 7f132e89a640 1 --2- 172.21.15.43:0/2917738195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1328100e20 0x7f1328101200 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.244+0000 7f132e89a640 1 -- 172.21.15.43:0/2917738195 >> 172.21.15.43:0/2917738195 conn(0x7f13280fc7d0 msgr2=0x7f13280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.244+0000 7f132e89a640 1 -- 172.21.15.43:0/2917738195 shutdown_connections 2024-09-11T23:25:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.244+0000 7f132e89a640 1 -- 172.21.15.43:0/2917738195 wait complete. 2024-09-11T23:25:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.245+0000 7f132e89a640 1 Processor -- start 2024-09-11T23:25:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.245+0000 7f132e89a640 1 -- start start 2024-09-11T23:25:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.245+0000 7f132e89a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1328100e20 0x7f13281993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.245+0000 7f132e89a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1328102570 0x7f13281998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.245+0000 7f132e89a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1328102ef0 0x7f13281a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.245+0000 7f132e89a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1328110fa0 con 0x7f1328102ef0 2024-09-11T23:25:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.245+0000 7f132e89a640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1328110e20 con 0x7f1328102570 2024-09-11T23:25:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.245+0000 7f132e89a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1328111120 con 0x7f1328100e20 2024-09-11T23:25:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.245+0000 7f132ce37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1328102ef0 0x7f13281a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.245+0000 7f132ce37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1328102ef0 0x7f13281a4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58962/0 (socket says 172.21.15.43:58962) 2024-09-11T23:25:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.246+0000 7f132ce37640 1 -- 172.21.15.43:0/1341087296 learned_addr learned my addr 172.21.15.43:0/1341087296 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.246+0000 7f13277fe640 1 --2- 172.21.15.43:0/1341087296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1328102570 0x7f13281998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.246+0000 7f1327fff640 1 --2- 172.21.15.43:0/1341087296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1328100e20 0x7f13281993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.246+0000 7f132ce37640 1 -- 172.21.15.43:0/1341087296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1328100e20 msgr2=0x7f13281993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.246+0000 7f132ce37640 1 --2- 172.21.15.43:0/1341087296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1328100e20 0x7f13281993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.246+0000 7f132ce37640 1 -- 172.21.15.43:0/1341087296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1328102570 msgr2=0x7f13281998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.246+0000 7f132ce37640 1 --2- 172.21.15.43:0/1341087296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1328102570 0x7f13281998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.246+0000 7f132ce37640 1 -- 172.21.15.43:0/1341087296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13281a4970 con 0x7f1328102ef0 2024-09-11T23:25:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.246+0000 7f1327fff640 1 --2- 172.21.15.43:0/1341087296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1328100e20 0x7f13281993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:25:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.246+0000 7f132ce37640 1 --2- 172.21.15.43:0/1341087296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1328102ef0 0x7f13281a4430 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f132000b970 tx=0x7f132000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.246+0000 7f13257fa640 1 -- 172.21.15.43:0/1341087296 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320019070 con 0x7f1328102ef0 2024-09-11T23:25:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.246+0000 7f13257fa640 1 -- 172.21.15.43:0/1341087296 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f132000cb40 con 0x7f1328102ef0 2024-09-11T23:25:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.246+0000 7f13257fa640 1 -- 172.21.15.43:0/1341087296 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320007650 con 0x7f1328102ef0 2024-09-11T23:25:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.246+0000 7f132e89a640 1 -- 172.21.15.43:0/1341087296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13281a4b40 con 0x7f1328102ef0 2024-09-11T23:25:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.246+0000 7f132e89a640 1 -- 172.21.15.43:0/1341087296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13281a5000 con 0x7f1328102ef0 2024-09-11T23:25:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.249+0000 7f13257fa640 1 -- 172.21.15.43:0/1341087296 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f132001c070 con 0x7f1328102ef0 2024-09-11T23:25:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.249+0000 7f1330c73740 1 Processor -- start 2024-09-11T23:25:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.249+0000 7f1330c73740 1 -- start start 2024-09-11T23:25:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.250+0000 7f13257fa640 1 --2- 172.21.15.43:0/1341087296 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f13100a2cf0 0x7f13100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.250+0000 7f1330c73740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556672756b40 0x556672755090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.250+0000 7f1330c73740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566727555d0 0x556672755a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.250+0000 7f1330c73740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556672764780 0x556672766b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.250+0000 7f1330c73740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5566726bb2f0 con 0x5566727555d0 2024-09-11T23:25:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.250+0000 7f1330c73740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556672662cf0 con 0x556672764780 2024-09-11T23:25:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.250+0000 7f1330c73740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5566726b72b0 con 0x556672756b40 2024-09-11T23:25:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.250+0000 7f13257fa640 1 -- 172.21.15.43:0/1341087296 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f13200ce920 con 0x7f1328102ef0 2024-09-11T23:25:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.250+0000 7f13277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566727555d0 0x556672755a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.250+0000 7f1327fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556672756b40 0x556672755090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:20.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.250+0000 7f1327fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556672756b40 0x556672755090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:37392/0 (socket says 172.21.15.43:37392) 2024-09-11T23:25:20.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.250+0000 7f1327fff640 1 -- 172.21.15.43:0/995397410 learned_addr learned my addr 172.21.15.43:0/995397410 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:20.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.250+0000 7f13277fe640 1 --2- 172.21.15.43:0/1341087296 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f13100a2cf0 0x7f13100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:20.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.250+0000 7f132ce37640 1 --2- 172.21.15.43:0/995397410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556672764780 0x556672766b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:20.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.250+0000 7f1327fff640 1 -- 172.21.15.43:0/995397410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556672764780 msgr2=0x556672766b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:20.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.250+0000 7f1327fff640 1 --2- 172.21.15.43:0/995397410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556672764780 0x556672766b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.250+0000 7f1327fff640 1 -- 172.21.15.43:0/995397410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566727555d0 msgr2=0x556672755a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:20.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.250+0000 7f1327fff640 1 --2- 172.21.15.43:0/995397410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566727555d0 0x556672755a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.250+0000 7f1327fff640 1 -- 172.21.15.43:0/995397410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566726473a0 con 0x556672756b40 2024-09-11T23:25:20.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.250+0000 7f13277fe640 1 --2- 172.21.15.43:0/995397410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566727555d0 0x556672755a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:20.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.251+0000 7f13277fe640 1 --2- 172.21.15.43:0/1341087296 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f13100a2cf0 0x7f13100a51b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f131c005d00 tx=0x7f131c00b7f0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:20.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.251+0000 7f1327fff640 1 --2- 172.21.15.43:0/995397410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556672756b40 0x556672755090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f13180314e0 tx=0x7f1318004200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bb6a8bafb24d522 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:20.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.251+0000 7f130e7fc640 1 -- 172.21.15.43:0/995397410 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1318035030 con 0x556672756b40 2024-09-11T23:25:20.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.251+0000 7f130e7fc640 1 -- 172.21.15.43:0/995397410 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1318005520 con 0x556672756b40 2024-09-11T23:25:20.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.251+0000 7f130e7fc640 1 -- 172.21.15.43:0/995397410 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1318003850 con 0x556672756b40 2024-09-11T23:25:20.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.252+0000 7f1330c73740 1 -- 172.21.15.43:0/995397410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556672756b40 msgr2=0x556672755090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:20.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.252+0000 7f1330c73740 1 --2- 172.21.15.43:0/995397410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556672756b40 0x556672755090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f13180314e0 tx=0x7f1318004200 comp rx=0 tx=0).stop 2024-09-11T23:25:20.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.252+0000 7f1330c73740 1 -- 172.21.15.43:0/995397410 shutdown_connections 2024-09-11T23:25:20.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.252+0000 7f1330c73740 1 --2- 172.21.15.43:0/995397410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556672764780 0x556672766b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.252+0000 7f1330c73740 1 --2- 172.21.15.43:0/995397410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5566727555d0 0x556672755a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.252+0000 7f1330c73740 1 --2- 172.21.15.43:0/995397410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556672756b40 0x556672755090 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.252+0000 7f1330c73740 1 -- 172.21.15.43:0/995397410 >> 172.21.15.43:0/995397410 conn(0x55667271e310 msgr2=0x556672751c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:20.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.252+0000 7f1330c73740 1 -- 172.21.15.43:0/995397410 shutdown_connections 2024-09-11T23:25:20.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.252+0000 7f1330c73740 1 -- 172.21.15.43:0/995397410 wait complete. 2024-09-11T23:25:20.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.252+0000 7f1330c73740 1 -- start start 2024-09-11T23:25:20.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.252+0000 7f1330c73740 1 Processor -- start 2024-09-11T23:25:20.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.253+0000 7f1330c73740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5566727555d0 0x556672764100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:20.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.253+0000 7f1330c73740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556672756b40 0x5566727613b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:20.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.253+0000 7f1330c73740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556672764780 0x556672761930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:20.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.253+0000 7f1330c73740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556672767420 con 0x556672756b40 2024-09-11T23:25:20.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.253+0000 7f1330c73740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556672676060 con 0x556672764780 2024-09-11T23:25:20.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.253+0000 7f1330c73740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5566727675a0 con 0x5566727555d0 2024-09-11T23:25:20.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.253+0000 7f1330c73740 10 client.0 _ll_register_callbacks cb 0x5566723c98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:20.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.253+0000 7f13277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556672756b40 0x5566727613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:20.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.253+0000 7f13277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556672756b40 0x5566727613b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58982/0 (socket says 172.21.15.43:58982) 2024-09-11T23:25:20.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.253+0000 7f13277fe640 1 -- 172.21.15.43:0/402425276 learned_addr learned my addr 172.21.15.43:0/402425276 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:20.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.253+0000 7f132ce37640 1 --2- 172.21.15.43:0/402425276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556672764780 0x556672761930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:20.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.253+0000 7f1327fff640 1 --2- 172.21.15.43:0/402425276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5566727555d0 0x556672764100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:20.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.253+0000 7f13277fe640 1 -- 172.21.15.43:0/402425276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5566727555d0 msgr2=0x556672764100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:20.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.253+0000 7f13277fe640 1 --2- 172.21.15.43:0/402425276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5566727555d0 0x556672764100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.253+0000 7f13277fe640 1 -- 172.21.15.43:0/402425276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556672764780 msgr2=0x556672761930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.253+0000 7f13277fe640 1 --2- 172.21.15.43:0/402425276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556672764780 0x556672761930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.253+0000 7f13277fe640 1 -- 172.21.15.43:0/402425276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55667242c3c0 con 0x556672756b40 2024-09-11T23:25:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.253+0000 7f132ce37640 1 --2- 172.21.15.43:0/402425276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556672764780 0x556672761930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:25:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.253+0000 7f13277fe640 1 --2- 172.21.15.43:0/402425276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556672756b40 0x5566727613b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f131c004270 tx=0x7f131c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e50a762a92d0ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.253+0000 7f13077fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:25:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.253+0000 7f13077fe640 1 -- 172.21.15.43:0/402425276 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c010e50 con 0x556672756b40 2024-09-11T23:25:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.254+0000 7f13077fe640 1 -- 172.21.15.43:0/402425276 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f131c0042a0 con 0x556672756b40 2024-09-11T23:25:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.254+0000 7f1330c73740 1 -- 172.21.15.43:0/402425276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1318005520 con 0x556672756b40 2024-09-11T23:25:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.254+0000 7f13077fe640 1 -- 172.21.15.43:0/402425276 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f131c0044a0 con 0x556672756b40 2024-09-11T23:25:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.254+0000 7f1330c73740 10 client.16023 fetch_fsmap learned FSMap version 26 2024-09-11T23:25:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.254+0000 7f1330c73740 1 -- 172.21.15.43:0/402425276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556672772e50 con 0x556672756b40 2024-09-11T23:25:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.254+0000 7f13077fe640 1 -- 172.21.15.43:0/402425276 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1318005520 con 0x556672756b40 2024-09-11T23:25:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.254+0000 7f13077fe640 1 -- 172.21.15.43:0/402425276 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f131c01f070 con 0x556672756b40 2024-09-11T23:25:20.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.254+0000 7f1330c73740 10 client.16023 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:25:20.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.254+0000 7f1330c73740 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:25:20.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.254+0000 7f1330c73740 20 client.16023 populate_metadata read hostname 'smithi043' 2024-09-11T23:25:20.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.254+0000 7f1330c73740 1 --2- 172.21.15.43:0/402425276 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5566727743d0 0x556672776830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:20.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.254+0000 7f1330c73740 4 client.16023 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:25:20.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.254+0000 7f1330c73740 1 -- 172.21.15.43:0/402425276 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566726c8860 con 0x5566727743d0 2024-09-11T23:25:20.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.255+0000 7f1327fff640 1 --2- 172.21.15.43:0/402425276 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5566727743d0 0x556672776830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:20.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.255+0000 7f1327fff640 1 --2- 172.21.15.43:0/402425276 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5566727743d0 0x556672776830 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ed9ee500ff38d13 server_cookie=8a3602a616fa0f4c in_seq=0 out_seq=0 2024-09-11T23:25:20.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.255+0000 7f13077fe640 10 client.16023 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:25:20.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.255+0000 7f13077fe640 1 -- 172.21.15.43:0/402425276 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566726c8860 con 0x5566727743d0 2024-09-11T23:25:20.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.255+0000 7f13077fe640 10 client.16023 handle_command_reply: tid=0 2024-09-11T23:25:20.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.256+0000 7f1330c73740 1 client.16023 shutdown 2024-09-11T23:25:20.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.256+0000 7f1330c73740 1 -- 172.21.15.43:0/402425276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556672756b40 msgr2=0x5566727613b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:20.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.256+0000 7f1330c73740 1 --2- 172.21.15.43:0/402425276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556672756b40 0x5566727613b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f131c004270 tx=0x7f131c005c90 comp rx=0 tx=0).stop 2024-09-11T23:25:20.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.256+0000 7f1330c73740 1 -- 172.21.15.43:0/402425276 shutdown_connections 2024-09-11T23:25:20.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.256+0000 7f1330c73740 1 --2- 172.21.15.43:0/402425276 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5566727743d0 0x556672776830 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.256+0000 7f1330c73740 1 --2- 172.21.15.43:0/402425276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556672764780 0x556672761930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.256+0000 7f1330c73740 1 --2- 172.21.15.43:0/402425276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556672756b40 0x5566727613b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.256+0000 7f1330c73740 1 --2- 172.21.15.43:0/402425276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5566727555d0 0x556672764100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.256+0000 7f13077fe640 0 client.16023 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:25:20.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.256+0000 7f1330c73740 1 -- 172.21.15.43:0/402425276 >> 172.21.15.43:0/402425276 conn(0x55667271e310 msgr2=0x556672757960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:20.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.256+0000 7f1330c73740 1 -- 172.21.15.43:0/402425276 shutdown_connections 2024-09-11T23:25:20.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.256+0000 7f1330c73740 1 -- 172.21.15.43:0/402425276 wait complete. 2024-09-11T23:25:20.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.256+0000 7f1330c73740 20 client.16023 trim_cache size 0 max 16384 2024-09-11T23:25:20.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.269+0000 7f1330c73740 1 Processor -- start 2024-09-11T23:25:20.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.269+0000 7f1330c73740 1 -- start start 2024-09-11T23:25:20.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.270+0000 7f1330c73740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5566727555d0 0x5566727706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:20.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.270+0000 7f1330c73740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556672756b40 0x556672763320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:20.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.270+0000 7f1330c73740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556672764780 0x55667277c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:20.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.270+0000 7f1330c73740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f13281adf40 con 0x556672756b40 2024-09-11T23:25:20.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.270+0000 7f1330c73740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f13281ab810 con 0x5566727555d0 2024-09-11T23:25:20.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.270+0000 7f1330c73740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f13281a9660 con 0x556672764780 2024-09-11T23:25:20.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.270+0000 7f13277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556672756b40 0x556672763320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:20.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.270+0000 7f13277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556672756b40 0x556672763320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58986/0 (socket says 172.21.15.43:58986) 2024-09-11T23:25:20.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.270+0000 7f1327fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5566727555d0 0x5566727706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:20.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.270+0000 7f13277fe640 1 -- 172.21.15.43:0/1923100680 learned_addr learned my addr 172.21.15.43:0/1923100680 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:20.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.270+0000 7f132ce37640 1 --2- 172.21.15.43:0/1923100680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556672764780 0x55667277c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:20.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.270+0000 7f13277fe640 1 -- 172.21.15.43:0/1923100680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556672764780 msgr2=0x55667277c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:20.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.270+0000 7f13277fe640 1 --2- 172.21.15.43:0/1923100680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556672764780 0x55667277c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.270+0000 7f13277fe640 1 -- 172.21.15.43:0/1923100680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5566727555d0 msgr2=0x5566727706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:20.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.270+0000 7f13277fe640 1 --2- 172.21.15.43:0/1923100680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5566727555d0 0x5566727706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.270+0000 7f13277fe640 1 -- 172.21.15.43:0/1923100680 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556672751ed0 con 0x556672756b40 2024-09-11T23:25:20.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.270+0000 7f13277fe640 1 --2- 172.21.15.43:0/1923100680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556672756b40 0x556672763320 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f131c029d40 tx=0x7f131c00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf93f47416f5685 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:20.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.270+0000 7f130dffb640 1 -- 172.21.15.43:0/1923100680 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c012070 con 0x556672756b40 2024-09-11T23:25:20.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.271+0000 7f130dffb640 1 -- 172.21.15.43:0/1923100680 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f131c010e50 con 0x556672756b40 2024-09-11T23:25:20.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.271+0000 7f130dffb640 1 -- 172.21.15.43:0/1923100680 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c029980 con 0x556672756b40 2024-09-11T23:25:20.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.271+0000 7f1330c73740 1 -- 172.21.15.43:0/1923100680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556672756b40 msgr2=0x556672763320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:20.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.271+0000 7f1330c73740 1 --2- 172.21.15.43:0/1923100680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556672756b40 0x556672763320 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f131c029d40 tx=0x7f131c00b8c0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.271+0000 7f1330c73740 1 -- 172.21.15.43:0/1923100680 shutdown_connections 2024-09-11T23:25:20.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.271+0000 7f1330c73740 1 --2- 172.21.15.43:0/1923100680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556672764780 0x55667277c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.271+0000 7f1330c73740 1 --2- 172.21.15.43:0/1923100680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556672756b40 0x556672763320 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.271+0000 7f1330c73740 1 --2- 172.21.15.43:0/1923100680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5566727555d0 0x5566727706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.271+0000 7f1330c73740 1 -- 172.21.15.43:0/1923100680 >> 172.21.15.43:0/1923100680 conn(0x55667271e310 msgr2=0x556672762c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.271+0000 7f1330c73740 1 -- 172.21.15.43:0/1923100680 shutdown_connections 2024-09-11T23:25:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.271+0000 7f1330c73740 1 -- 172.21.15.43:0/1923100680 wait complete. 2024-09-11T23:25:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.272+0000 7f1330c73740 1 -- start start 2024-09-11T23:25:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.272+0000 7f1330c73740 1 Processor -- start 2024-09-11T23:25:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.272+0000 7f1330c73740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5566727555d0 0x55667275e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.272+0000 7f1330c73740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556672756b40 0x55667275e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.272+0000 7f1330c73740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556672764780 0x55667275ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.272+0000 7f1330c73740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f13281a9cd0 con 0x556672764780 2024-09-11T23:25:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.272+0000 7f1330c73740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f13281ab140 con 0x5566727555d0 2024-09-11T23:25:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.272+0000 7f1330c73740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f13281aa3a0 con 0x556672756b40 2024-09-11T23:25:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.272+0000 7f1330c73740 10 client.0 _ll_register_callbacks cb 0x7f131c00bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.272+0000 7f132ce37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556672764780 0x55667275ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.273+0000 7f1327fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5566727555d0 0x55667275e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.273+0000 7f132ce37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556672764780 0x55667275ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58988/0 (socket says 172.21.15.43:58988) 2024-09-11T23:25:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.273+0000 7f132ce37640 1 -- 172.21.15.43:0/3560681494 learned_addr learned my addr 172.21.15.43:0/3560681494 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.273+0000 7f13277fe640 1 --2- 172.21.15.43:0/3560681494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556672756b40 0x55667275e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.273+0000 7f1327fff640 1 -- 172.21.15.43:0/3560681494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556672756b40 msgr2=0x55667275e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.273+0000 7f1327fff640 1 --2- 172.21.15.43:0/3560681494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556672756b40 0x55667275e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.273+0000 7f1327fff640 1 -- 172.21.15.43:0/3560681494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556672764780 msgr2=0x55667275ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.273+0000 7f1327fff640 1 --2- 172.21.15.43:0/3560681494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556672764780 0x55667275ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.273+0000 7f1327fff640 1 -- 172.21.15.43:0/3560681494 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566727de0d0 con 0x5566727555d0 2024-09-11T23:25:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.273+0000 7f13277fe640 1 --2- 172.21.15.43:0/3560681494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556672756b40 0x55667275e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:25:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.273+0000 7f132ce37640 1 --2- 172.21.15.43:0/3560681494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556672764780 0x55667275ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.273+0000 7f1327fff640 1 --2- 172.21.15.43:0/3560681494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5566727555d0 0x55667275e390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1318005c40 tx=0x7f13180058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40670bddd8dd0f2f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.273+0000 7f13077fe640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:25:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.273+0000 7f13077fe640 1 -- 172.21.15.43:0/3560681494 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1318035040 con 0x5566727555d0 2024-09-11T23:25:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.273+0000 7f1330c73740 1 -- 172.21.15.43:0/3560681494 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f131c010e50 con 0x5566727555d0 2024-09-11T23:25:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.273+0000 7f13077fe640 1 -- 172.21.15.43:0/3560681494 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f1318004fa0 con 0x5566727555d0 2024-09-11T23:25:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.273+0000 7f13077fe640 1 -- 172.21.15.43:0/3560681494 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f131805d050 con 0x5566727555d0 2024-09-11T23:25:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.273+0000 7f1330c73740 10 client.25565 fetch_fsmap learned FSMap version 26 2024-09-11T23:25:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.273+0000 7f1330c73740 1 -- 172.21.15.43:0/3560681494 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5566727df370 con 0x5566727555d0 2024-09-11T23:25:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.274+0000 7f13077fe640 1 -- 172.21.15.43:0/3560681494 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f131c010e50 con 0x5566727555d0 2024-09-11T23:25:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.274+0000 7f13077fe640 1 -- 172.21.15.43:0/3560681494 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1318003a00 con 0x5566727555d0 2024-09-11T23:25:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.274+0000 7f1330c73740 10 client.25565 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T23:25:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.274+0000 7f1330c73740 10 client.25565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:25:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.274+0000 7f1330c73740 20 client.25565 populate_metadata read hostname 'smithi043' 2024-09-11T23:25:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.274+0000 7f1330c73740 1 --2- 172.21.15.43:0/3560681494 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5566727df7c0 0x5566727e1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.274+0000 7f1330c73740 4 client.25565 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:25:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.274+0000 7f1330c73740 1 -- 172.21.15.43:0/3560681494 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566726c2d70 con 0x5566727df7c0 2024-09-11T23:25:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.274+0000 7f13277fe640 1 --2- 172.21.15.43:0/3560681494 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5566727df7c0 0x5566727e1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.275+0000 7f13077fe640 10 client.25565 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:25:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.275+0000 7f13277fe640 1 --2- 172.21.15.43:0/3560681494 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5566727df7c0 0x5566727e1bb0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbec66e50b005f45 server_cookie=dfdc4c7670d233de in_seq=0 out_seq=0 2024-09-11T23:25:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.275+0000 7f13077fe640 1 -- 172.21.15.43:0/3560681494 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566726c2d70 con 0x5566727df7c0 2024-09-11T23:25:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.275+0000 7f13077fe640 10 client.25565 handle_command_reply: tid=0 2024-09-11T23:25:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.275+0000 7f1330c73740 1 client.25565 shutdown 2024-09-11T23:25:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.275+0000 7f1330c73740 1 -- 172.21.15.43:0/3560681494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5566727555d0 msgr2=0x55667275e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.275+0000 7f1330c73740 1 --2- 172.21.15.43:0/3560681494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5566727555d0 0x55667275e390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1318005c40 tx=0x7f13180058d0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.275+0000 7f1330c73740 1 -- 172.21.15.43:0/3560681494 shutdown_connections 2024-09-11T23:25:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.275+0000 7f1330c73740 1 --2- 172.21.15.43:0/3560681494 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5566727df7c0 0x5566727e1bb0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.275+0000 7f1330c73740 1 --2- 172.21.15.43:0/3560681494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556672764780 0x55667275ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.275+0000 7f1330c73740 1 --2- 172.21.15.43:0/3560681494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556672756b40 0x55667275e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.275+0000 7f1330c73740 1 --2- 172.21.15.43:0/3560681494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5566727555d0 0x55667275e390 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.275+0000 7f13077fe640 0 client.25565 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:25:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.275+0000 7f1330c73740 1 -- 172.21.15.43:0/3560681494 >> 172.21.15.43:0/3560681494 conn(0x55667271e310 msgr2=0x55667275f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.275+0000 7f1330c73740 1 -- 172.21.15.43:0/3560681494 shutdown_connections 2024-09-11T23:25:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.275+0000 7f1330c73740 1 -- 172.21.15.43:0/3560681494 wait complete. 2024-09-11T23:25:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.275+0000 7f1330c73740 20 client.25565 trim_cache size 0 max 16384 2024-09-11T23:25:20.561 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:25:20.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.276+0000 7f132e89a640 1 -- 172.21.15.43:0/1341087296 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f13100a2cf0 msgr2=0x7f13100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:20.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.276+0000 7f132e89a640 1 --2- 172.21.15.43:0/1341087296 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f13100a2cf0 0x7f13100a51b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f131c005d00 tx=0x7f131c00b7f0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.276+0000 7f132e89a640 1 -- 172.21.15.43:0/1341087296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1328102ef0 msgr2=0x7f13281a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:20.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.276+0000 7f132e89a640 1 --2- 172.21.15.43:0/1341087296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1328102ef0 0x7f13281a4430 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f132000b970 tx=0x7f132000be30 comp rx=0 tx=0).stop 2024-09-11T23:25:20.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.276+0000 7f132e89a640 1 -- 172.21.15.43:0/1341087296 shutdown_connections 2024-09-11T23:25:20.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.276+0000 7f132e89a640 1 --2- 172.21.15.43:0/1341087296 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f13100a2cf0 0x7f13100a51b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.276+0000 7f132e89a640 1 --2- 172.21.15.43:0/1341087296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1328102ef0 0x7f13281a4430 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.276+0000 7f132e89a640 1 --2- 172.21.15.43:0/1341087296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1328102570 0x7f13281998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.276+0000 7f132e89a640 1 --2- 172.21.15.43:0/1341087296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1328100e20 0x7f13281993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:20.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.276+0000 7f132e89a640 1 -- 172.21.15.43:0/1341087296 >> 172.21.15.43:0/1341087296 conn(0x7f13280fc7d0 msgr2=0x7f13280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:20.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.276+0000 7f132e89a640 1 -- 172.21.15.43:0/1341087296 shutdown_connections 2024-09-11T23:25:20.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:19.276+0000 7f132e89a640 1 -- 172.21.15.43:0/1341087296 wait complete. 2024-09-11T23:25:21.633 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.7 2024-09-11T23:25:21.633 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:25:21.690 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:21 smithi043 ceph-mon[30950]: pgmap v108: 97 pgs: 97 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 79 MiB/s wr, 11.05k op/s 2024-09-11T23:25:21.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:21 smithi090 ceph-mon[35130]: pgmap v108: 97 pgs: 97 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 79 MiB/s wr, 11.05k op/s 2024-09-11T23:25:21.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:21 smithi097 ceph-mon[35562]: pgmap v108: 97 pgs: 97 active+clean; 15 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 79 MiB/s wr, 11.05k op/s 2024-09-11T23:25:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.346+0000 7f7faeff5640 1 -- 172.21.15.43:0/779935677 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ad084e9d70 con 0x55ad086150e0 2024-09-11T23:25:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.346+0000 7f7faeff5640 10 client.25531 handle_command_reply: tid=0 2024-09-11T23:25:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.346+0000 7f7fd4462740 1 client.25531 shutdown 2024-09-11T23:25:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.346+0000 7f7fd4462740 1 -- 172.21.15.43:0/779935677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad08588cb0 msgr2=0x55ad08591000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.346+0000 7f7fd4462740 1 --2- 172.21.15.43:0/779935677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad08588cb0 0x55ad08591000 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4007b80 tx=0x7f7fc4017eb0 comp rx=0 tx=0).stop 2024-09-11T23:25:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.346+0000 7f7fd4462740 1 -- 172.21.15.43:0/779935677 shutdown_connections 2024-09-11T23:25:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.346+0000 7f7fd4462740 1 --2- 172.21.15.43:0/779935677 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55ad086150e0 0x55ad08617500 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.346+0000 7f7fd4462740 1 --2- 172.21.15.43:0/779935677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad08588cb0 0x55ad08591000 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.346+0000 7f7fd4462740 1 --2- 172.21.15.43:0/779935677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0857e410 0x55ad086015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.346+0000 7f7fd4462740 1 --2- 172.21.15.43:0/779935677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ad0857daf0 0x55ad086010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.346+0000 7f7faeff5640 0 client.25531 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:25:22.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.346+0000 7f7fd4462740 1 -- 172.21.15.43:0/779935677 >> 172.21.15.43:0/779935677 conn(0x55ad0857a060 msgr2=0x55ad08587390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:22.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.346+0000 7f7fd4462740 1 -- 172.21.15.43:0/779935677 shutdown_connections 2024-09-11T23:25:22.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.347+0000 7f7fd4462740 1 -- 172.21.15.43:0/779935677 wait complete. 2024-09-11T23:25:22.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.347+0000 7f7fd4462740 20 client.25531 trim_cache size 0 max 16384 2024-09-11T23:25:22.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:25:22.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:25:22.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:25:22.348 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:25:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.347+0000 7f7fd15e8640 1 -- 172.21.15.43:0/60983878 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7fb40a2f00 msgr2=0x7f7fb40a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.347+0000 7f7fd15e8640 1 --2- 172.21.15.43:0/60983878 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7fb40a2f00 0x7f7fb40a53c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0006fd0 tx=0x7f7fc0008040 comp rx=0 tx=0).stop 2024-09-11T23:25:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.347+0000 7f7fd15e8640 1 -- 172.21.15.43:0/60983878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fcc100810 msgr2=0x7f7fcc19d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.347+0000 7f7fd15e8640 1 --2- 172.21.15.43:0/60983878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fcc100810 0x7f7fcc19d130 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc036450 tx=0x7f7fbc036820 comp rx=0 tx=0).stop 2024-09-11T23:25:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.347+0000 7f7fd15e8640 1 -- 172.21.15.43:0/60983878 shutdown_connections 2024-09-11T23:25:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.347+0000 7f7fd15e8640 1 --2- 172.21.15.43:0/60983878 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7fb40a2f00 0x7f7fb40a53c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.347+0000 7f7fd15e8640 1 --2- 172.21.15.43:0/60983878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fcc1966f0 0x7f7fcc196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.347+0000 7f7fd15e8640 1 --2- 172.21.15.43:0/60983878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fcc1080d0 0x7f7fcc1961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.347+0000 7f7fd15e8640 1 --2- 172.21.15.43:0/60983878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fcc100810 0x7f7fcc19d130 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.347+0000 7f7fd15e8640 1 -- 172.21.15.43:0/60983878 >> 172.21.15.43:0/60983878 conn(0x7f7fcc075eb0 msgr2=0x7f7fcc105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.347+0000 7f7fd15e8640 1 -- 172.21.15.43:0/60983878 shutdown_connections 2024-09-11T23:25:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:22.347+0000 7f7fd15e8640 1 -- 172.21.15.43:0/60983878 wait complete. 2024-09-11T23:25:23.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:23 smithi090 ceph-mon[35130]: pgmap v109: 97 pgs: 97 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 75 MiB/s wr, 10.75k op/s 2024-09-11T23:25:23.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:23 smithi097 ceph-mon[35562]: pgmap v109: 97 pgs: 97 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 75 MiB/s wr, 10.75k op/s 2024-09-11T23:25:23.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:23 smithi043 ceph-mon[30950]: pgmap v109: 97 pgs: 97 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 75 MiB/s wr, 10.75k op/s 2024-09-11T23:25:25.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:25 smithi090 ceph-mon[35130]: pgmap v110: 97 pgs: 97 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 72 MiB/s wr, 10.69k op/s 2024-09-11T23:25:25.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:25 smithi097 ceph-mon[35562]: pgmap v110: 97 pgs: 97 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 72 MiB/s wr, 10.69k op/s 2024-09-11T23:25:25.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:25 smithi043 ceph-mon[30950]: pgmap v110: 97 pgs: 97 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 72 MiB/s wr, 10.69k op/s 2024-09-11T23:25:27.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:27 smithi090 ceph-mon[35130]: pgmap v111: 97 pgs: 97 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 59 MiB/s wr, 9.03k op/s 2024-09-11T23:25:27.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:27 smithi097 ceph-mon[35562]: pgmap v111: 97 pgs: 97 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 59 MiB/s wr, 9.03k op/s 2024-09-11T23:25:27.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:27 smithi043 ceph-mon[30950]: pgmap v111: 97 pgs: 97 active+clean; 15 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 59 MiB/s wr, 9.03k op/s 2024-09-11T23:25:29.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:29 smithi090 ceph-mon[35130]: pgmap v112: 97 pgs: 97 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 58 MiB/s wr, 9.01k op/s 2024-09-11T23:25:29.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:29 smithi097 ceph-mon[35562]: pgmap v112: 97 pgs: 97 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 58 MiB/s wr, 9.01k op/s 2024-09-11T23:25:29.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:29 smithi043 ceph-mon[30950]: pgmap v112: 97 pgs: 97 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 58 MiB/s wr, 9.01k op/s 2024-09-11T23:25:30.079 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:25:30.957 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:25:30.957 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:25:31.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:31 smithi090 ceph-mon[35130]: pgmap v113: 97 pgs: 97 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 36 MiB/s wr, 5.82k op/s 2024-09-11T23:25:31.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:31 smithi097 ceph-mon[35562]: pgmap v113: 97 pgs: 97 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 36 MiB/s wr, 5.82k op/s 2024-09-11T23:25:31.834 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:31 smithi043 ceph-mon[30950]: pgmap v113: 97 pgs: 97 active+clean; 15 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 36 MiB/s wr, 5.82k op/s 2024-09-11T23:25:32.270 INFO:journalctl@ceph.mds.e.smithi090.stdout:Sep 11 23:25:31 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-e[64566]: ignoring --setuser ceph since I am not root 2024-09-11T23:25:32.270 INFO:journalctl@ceph.mds.e.smithi090.stdout:Sep 11 23:25:31 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-e[64566]: ignoring --setgroup ceph since I am not root 2024-09-11T23:25:32.270 INFO:journalctl@ceph.mds.e.smithi090.stdout:Sep 11 23:25:31 smithi090 ceph-mds[64570]: -- 172.21.15.90:0/3034290980 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557637323860 con 0x557637301000 2024-09-11T23:25:32.270 INFO:journalctl@ceph.mds.e.smithi090.stdout:Sep 11 23:25:31 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-e[64566]: starting mds.e at 2024-09-11T23:25:32.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:32 smithi090 ceph-mon[35130]: daemon mds.e restarted 2024-09-11T23:25:32.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:32 smithi097 ceph-mon[35562]: daemon mds.e restarted 2024-09-11T23:25:32.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:32 smithi043 ceph-mon[30950]: daemon mds.e restarted 2024-09-11T23:25:33.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:33 smithi043 ceph-mon[30950]: pgmap v114: 97 pgs: 97 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 37 MiB/s wr, 6.02k op/s 2024-09-11T23:25:33.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:33 smithi043 ceph-mon[30950]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T23:25:33.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:33 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.90:6834/568491708,v1:172.21.15.90:6835/568491708] up:boot 2024-09-11T23:25:33.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:33 smithi043 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-09-11T23:25:33.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:33 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:25:33.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:33 smithi043 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:25:33.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:33 smithi090 ceph-mon[35130]: pgmap v114: 97 pgs: 97 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 37 MiB/s wr, 6.02k op/s 2024-09-11T23:25:33.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:33 smithi090 ceph-mon[35130]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T23:25:33.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:33 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.90:6834/568491708,v1:172.21.15.90:6835/568491708] up:boot 2024-09-11T23:25:33.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:33 smithi090 ceph-mon[35130]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-09-11T23:25:33.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:33 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:25:33.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:33 smithi090 ceph-mon[35130]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:25:33.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:33 smithi097 ceph-mon[35562]: pgmap v114: 97 pgs: 97 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 37 MiB/s wr, 6.02k op/s 2024-09-11T23:25:33.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:33 smithi097 ceph-mon[35562]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T23:25:33.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:33 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.90:6834/568491708,v1:172.21.15.90:6835/568491708] up:boot 2024-09-11T23:25:33.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:33 smithi097 ceph-mon[35562]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-09-11T23:25:33.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:33 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:25:33.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:33 smithi097 ceph-mon[35562]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:25:35.441 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.439+0000 7f3580752640 1 -- 172.21.15.43:0/1738549468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35781069b0 msgr2=0x7f3578106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.439+0000 7f3580752640 1 --2- 172.21.15.43:0/1738549468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35781069b0 0x7f3578106d90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3574009860 tx=0x7f357402a340 comp rx=0 tx=0).stop 2024-09-11T23:25:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.440+0000 7f3580752640 1 -- 172.21.15.43:0/1738549468 shutdown_connections 2024-09-11T23:25:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.440+0000 7f3580752640 1 --2- 172.21.15.43:0/1738549468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3578101340 0x7f357810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.440+0000 7f3580752640 1 --2- 172.21.15.43:0/1738549468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35781009a0 0x7f3578100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.440+0000 7f3580752640 1 --2- 172.21.15.43:0/1738549468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35781069b0 0x7f3578106d90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.440+0000 7f3580752640 1 -- 172.21.15.43:0/1738549468 >> 172.21.15.43:0/1738549468 conn(0x7f35780779a0 msgr2=0x7f35780fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.440+0000 7f3580752640 1 -- 172.21.15.43:0/1738549468 shutdown_connections 2024-09-11T23:25:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.440+0000 7f3580752640 1 -- 172.21.15.43:0/1738549468 wait complete. 2024-09-11T23:25:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.440+0000 7f3580752640 1 Processor -- start 2024-09-11T23:25:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.441+0000 7f3580752640 1 -- start start 2024-09-11T23:25:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.441+0000 7f3580752640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35781009a0 0x7f3578198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.441+0000 7f3580752640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3578101340 0x7f35781988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.441+0000 7f3580752640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35781069b0 0x7f357819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.441+0000 7f3580752640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3578110df0 con 0x7f3578101340 2024-09-11T23:25:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.441+0000 7f3580752640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3578110c70 con 0x7f35781009a0 2024-09-11T23:25:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.441+0000 7f3580752640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3578110f70 con 0x7f35781069b0 2024-09-11T23:25:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.441+0000 7f357dced640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3578101340 0x7f35781988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.441+0000 7f357e4ee640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35781009a0 0x7f3578198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.441+0000 7f357ecef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35781069b0 0x7f357819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.441+0000 7f357e4ee640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35781009a0 0x7f3578198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:35656/0 (socket says 172.21.15.43:35656) 2024-09-11T23:25:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.441+0000 7f357e4ee640 1 -- 172.21.15.43:0/2977511162 learned_addr learned my addr 172.21.15.43:0/2977511162 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.441+0000 7f357dced640 1 -- 172.21.15.43:0/2977511162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35781069b0 msgr2=0x7f357819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.441+0000 7f357dced640 1 --2- 172.21.15.43:0/2977511162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35781069b0 0x7f357819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.441+0000 7f357dced640 1 -- 172.21.15.43:0/2977511162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35781009a0 msgr2=0x7f3578198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.441+0000 7f357dced640 1 --2- 172.21.15.43:0/2977511162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35781009a0 0x7f3578198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.441+0000 7f357dced640 1 -- 172.21.15.43:0/2977511162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357819d370 con 0x7f3578101340 2024-09-11T23:25:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.441+0000 7f357ecef640 1 --2- 172.21.15.43:0/2977511162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35781069b0 0x7f357819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.442+0000 7f357dced640 1 --2- 172.21.15.43:0/2977511162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3578101340 0x7f35781988c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3568005fd0 tx=0x7f356800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.442+0000 7f356f7fe640 1 -- 172.21.15.43:0/2977511162 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356800cd00 con 0x7f3578101340 2024-09-11T23:25:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.442+0000 7f3580752640 1 -- 172.21.15.43:0/2977511162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f357819d660 con 0x7f3578101340 2024-09-11T23:25:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.442+0000 7f356f7fe640 1 -- 172.21.15.43:0/2977511162 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35680035f0 con 0x7f3578101340 2024-09-11T23:25:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.442+0000 7f356f7fe640 1 -- 172.21.15.43:0/2977511162 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3568010690 con 0x7f3578101340 2024-09-11T23:25:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.442+0000 7f3580752640 1 -- 172.21.15.43:0/2977511162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35781a4ef0 con 0x7f3578101340 2024-09-11T23:25:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.444+0000 7f356f7fe640 1 -- 172.21.15.43:0/2977511162 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f35680037a0 con 0x7f3578101340 2024-09-11T23:25:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.445+0000 7f356f7fe640 1 --2- 172.21.15.43:0/2977511162 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f35640a2cf0 0x7f35640a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.445+0000 7f35835c9740 1 Processor -- start 2024-09-11T23:25:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.445+0000 7f356f7fe640 1 -- 172.21.15.43:0/2977511162 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f35680cf690 con 0x7f3578101340 2024-09-11T23:25:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.445+0000 7f357e4ee640 1 --2- 172.21.15.43:0/2977511162 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f35640a2cf0 0x7f35640a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.445+0000 7f35835c9740 1 -- start start 2024-09-11T23:25:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.446+0000 7f35835c9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8ba1af0 0x563ae8ba1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.446+0000 7f35835c9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba2410 0x563ae8bac770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.446+0000 7f35835c9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8baccb0 0x563ae8baf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.446+0000 7f35835c9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563ae8b062f0 con 0x563ae8ba2410 2024-09-11T23:25:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.446+0000 7f35835c9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563ae8aae270 con 0x563ae8baccb0 2024-09-11T23:25:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.446+0000 7f35835c9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563ae8b022b0 con 0x563ae8ba1af0 2024-09-11T23:25:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.446+0000 7f357dced640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba2410 0x563ae8bac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.446+0000 7f357e4ee640 1 --2- 172.21.15.43:0/2977511162 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f35640a2cf0 0x7f35640a51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f35740095f0 tx=0x7f35740056d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.446+0000 7f357dced640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba2410 0x563ae8bac770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44042/0 (socket says 172.21.15.43:44042) 2024-09-11T23:25:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.446+0000 7f357dced640 1 -- 172.21.15.43:0/1989066329 learned_addr learned my addr 172.21.15.43:0/1989066329 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.446+0000 7f357ecef640 1 --2- 172.21.15.43:0/1989066329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8baccb0 0x563ae8baf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.446+0000 7f357e4ee640 1 --2- 172.21.15.43:0/1989066329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8ba1af0 0x563ae8ba1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.446+0000 7f357dced640 1 -- 172.21.15.43:0/1989066329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8ba1af0 msgr2=0x563ae8ba1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.446+0000 7f357dced640 1 --2- 172.21.15.43:0/1989066329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8ba1af0 0x563ae8ba1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.446+0000 7f357dced640 1 -- 172.21.15.43:0/1989066329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8baccb0 msgr2=0x563ae8baf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.446+0000 7f357dced640 1 --2- 172.21.15.43:0/1989066329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8baccb0 0x563ae8baf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.446+0000 7f357dced640 1 -- 172.21.15.43:0/1989066329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae8aad340 con 0x563ae8ba2410 2024-09-11T23:25:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.446+0000 7f357e4ee640 1 --2- 172.21.15.43:0/1989066329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8ba1af0 0x563ae8ba1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:25:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.446+0000 7f357dced640 1 --2- 172.21.15.43:0/1989066329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba2410 0x563ae8bac770 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3568098010 tx=0x7f3568005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821d8c5e3b60fe2e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.447+0000 7f356cff9640 1 -- 172.21.15.43:0/1989066329 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35680b2030 con 0x563ae8ba2410 2024-09-11T23:25:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.447+0000 7f356cff9640 1 -- 172.21.15.43:0/1989066329 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f356809bbb0 con 0x563ae8ba2410 2024-09-11T23:25:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.447+0000 7f356cff9640 1 -- 172.21.15.43:0/1989066329 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3568098220 con 0x563ae8ba2410 2024-09-11T23:25:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.448+0000 7f35835c9740 1 -- 172.21.15.43:0/1989066329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba2410 msgr2=0x563ae8bac770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.448+0000 7f35835c9740 1 --2- 172.21.15.43:0/1989066329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba2410 0x563ae8bac770 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3568098010 tx=0x7f3568005790 comp rx=0 tx=0).stop 2024-09-11T23:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.448+0000 7f35835c9740 1 -- 172.21.15.43:0/1989066329 shutdown_connections 2024-09-11T23:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.448+0000 7f35835c9740 1 --2- 172.21.15.43:0/1989066329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8baccb0 0x563ae8baf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.448+0000 7f35835c9740 1 --2- 172.21.15.43:0/1989066329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba2410 0x563ae8bac770 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.448+0000 7f35835c9740 1 --2- 172.21.15.43:0/1989066329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8ba1af0 0x563ae8ba1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.448+0000 7f35835c9740 1 -- 172.21.15.43:0/1989066329 >> 172.21.15.43:0/1989066329 conn(0x563ae8b9e060 msgr2=0x563ae8b9e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.448+0000 7f35835c9740 1 -- 172.21.15.43:0/1989066329 shutdown_connections 2024-09-11T23:25:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.448+0000 7f35835c9740 1 -- 172.21.15.43:0/1989066329 wait complete. 2024-09-11T23:25:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.448+0000 7f35835c9740 1 -- start start 2024-09-11T23:25:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.448+0000 7f35835c9740 1 Processor -- start 2024-09-11T23:25:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.449+0000 7f35835c9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba1af0 0x563ae8bac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.449+0000 7f35835c9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8ba2410 0x563ae8ba9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.449+0000 7f35835c9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8baccb0 0x563ae8baa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.449+0000 7f35835c9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563ae8baf880 con 0x563ae8ba1af0 2024-09-11T23:25:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.449+0000 7f35835c9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563ae8a8f290 con 0x563ae8ba2410 2024-09-11T23:25:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.449+0000 7f35835c9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563ae8bafa00 con 0x563ae8baccb0 2024-09-11T23:25:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.449+0000 7f35835c9740 10 client.0 _ll_register_callbacks cb 0x563ae8ac0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.449+0000 7f357e4ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba1af0 0x563ae8bac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.449+0000 7f357e4ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba1af0 0x563ae8bac6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44058/0 (socket says 172.21.15.43:44058) 2024-09-11T23:25:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.449+0000 7f357dced640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8ba2410 0x563ae8ba9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.449+0000 7f357e4ee640 1 -- 172.21.15.43:0/2006375653 learned_addr learned my addr 172.21.15.43:0/2006375653 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.449+0000 7f357ecef640 1 --2- 172.21.15.43:0/2006375653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8baccb0 0x563ae8baa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.449+0000 7f357dced640 1 -- 172.21.15.43:0/2006375653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8baccb0 msgr2=0x563ae8baa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.449+0000 7f357dced640 1 --2- 172.21.15.43:0/2006375653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8baccb0 0x563ae8baa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.449+0000 7f357dced640 1 -- 172.21.15.43:0/2006375653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba1af0 msgr2=0x563ae8bac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.449+0000 7f357dced640 1 --2- 172.21.15.43:0/2006375653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba1af0 0x563ae8bac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.449+0000 7f357dced640 1 -- 172.21.15.43:0/2006375653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae8819210 con 0x563ae8ba2410 2024-09-11T23:25:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.449+0000 7f357e4ee640 1 --2- 172.21.15.43:0/2006375653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba1af0 0x563ae8bac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.449+0000 7f357dced640 1 --2- 172.21.15.43:0/2006375653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8ba2410 0x563ae8ba9bc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f35680a49e0 tx=0x7f3568003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53a8f20fa2b54364 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.449+0000 7f3561ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:25:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.450+0000 7f3561ffb640 1 -- 172.21.15.43:0/2006375653 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35680b2070 con 0x563ae8ba2410 2024-09-11T23:25:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.450+0000 7f3561ffb640 1 -- 172.21.15.43:0/2006375653 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f35680ba4e0 con 0x563ae8ba2410 2024-09-11T23:25:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.450+0000 7f35835c9740 1 -- 172.21.15.43:0/2006375653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f356809bbb0 con 0x563ae8ba2410 2024-09-11T23:25:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.450+0000 7f3561ffb640 1 -- 172.21.15.43:0/2006375653 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f356809b000 con 0x563ae8ba2410 2024-09-11T23:25:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.450+0000 7f35835c9740 10 client.25589 fetch_fsmap learned FSMap version 29 2024-09-11T23:25:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.450+0000 7f35835c9740 1 -- 172.21.15.43:0/2006375653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563ae88e5620 con 0x563ae8ba2410 2024-09-11T23:25:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.450+0000 7f3561ffb640 1 -- 172.21.15.43:0/2006375653 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f356809bbb0 con 0x563ae8ba2410 2024-09-11T23:25:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.450+0000 7f3561ffb640 1 -- 172.21.15.43:0/2006375653 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f3568082030 con 0x563ae8ba2410 2024-09-11T23:25:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.450+0000 7f35835c9740 10 client.25589 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:25:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.450+0000 7f35835c9740 10 client.25589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:25:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.450+0000 7f35835c9740 20 client.25589 populate_metadata read hostname 'smithi043' 2024-09-11T23:25:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.450+0000 7f35835c9740 1 --2- 172.21.15.43:0/2006375653 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x563ae8bc52c0 0x563ae8bc76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.450+0000 7f35835c9740 4 client.25589 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:25:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.450+0000 7f35835c9740 1 -- 172.21.15.43:0/2006375653 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ae8b13a20 con 0x563ae8bc52c0 2024-09-11T23:25:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.450+0000 7f357e4ee640 1 --2- 172.21.15.43:0/2006375653 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x563ae8bc52c0 0x563ae8bc76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.451+0000 7f3561ffb640 10 client.25589 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:25:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.451+0000 7f357e4ee640 1 --2- 172.21.15.43:0/2006375653 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x563ae8bc52c0 0x563ae8bc76b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bf294b196382ae7 server_cookie=72b5d84db6422e75 in_seq=0 out_seq=0 2024-09-11T23:25:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.451+0000 7f3561ffb640 1 -- 172.21.15.43:0/2006375653 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563ae8b13a20 con 0x563ae8bc52c0 2024-09-11T23:25:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.451+0000 7f3561ffb640 10 client.25589 handle_command_reply: tid=0 2024-09-11T23:25:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.452+0000 7f35835c9740 1 client.25589 shutdown 2024-09-11T23:25:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.452+0000 7f35835c9740 1 -- 172.21.15.43:0/2006375653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8ba2410 msgr2=0x563ae8ba9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.452+0000 7f35835c9740 1 --2- 172.21.15.43:0/2006375653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8ba2410 0x563ae8ba9bc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f35680a49e0 tx=0x7f3568003c10 comp rx=0 tx=0).stop 2024-09-11T23:25:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.452+0000 7f35835c9740 1 -- 172.21.15.43:0/2006375653 shutdown_connections 2024-09-11T23:25:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.452+0000 7f35835c9740 1 --2- 172.21.15.43:0/2006375653 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x563ae8bc52c0 0x563ae8bc76b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.452+0000 7f35835c9740 1 --2- 172.21.15.43:0/2006375653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8baccb0 0x563ae8baa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.452+0000 7f35835c9740 1 --2- 172.21.15.43:0/2006375653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8ba2410 0x563ae8ba9bc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.452+0000 7f35835c9740 1 --2- 172.21.15.43:0/2006375653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba1af0 0x563ae8bac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.452+0000 7f3561ffb640 0 client.25589 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:25:35.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.452+0000 7f35835c9740 1 -- 172.21.15.43:0/2006375653 >> 172.21.15.43:0/2006375653 conn(0x563ae8b9e060 msgr2=0x563ae8ba3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:35.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.452+0000 7f35835c9740 1 -- 172.21.15.43:0/2006375653 shutdown_connections 2024-09-11T23:25:35.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.452+0000 7f35835c9740 1 -- 172.21.15.43:0/2006375653 wait complete. 2024-09-11T23:25:35.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.452+0000 7f35835c9740 20 client.25589 trim_cache size 0 max 16384 2024-09-11T23:25:35.476 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.466+0000 7f35835c9740 1 Processor -- start 2024-09-11T23:25:35.476 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.466+0000 7f35835c9740 1 -- start start 2024-09-11T23:25:35.476 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f35835c9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8ba1af0 0x563ae8bccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.476 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f35835c9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba2410 0x563ae8bcd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.476 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f35835c9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8baccb0 0x563ae8ba6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.476 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f35835c9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f35781ae030 con 0x563ae8ba2410 2024-09-11T23:25:35.476 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f35835c9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f35781ab8d0 con 0x563ae8baccb0 2024-09-11T23:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f35835c9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f35781a9720 con 0x563ae8ba1af0 2024-09-11T23:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f357dced640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba2410 0x563ae8bcd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f357ecef640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8baccb0 0x563ae8ba6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f357e4ee640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8ba1af0 0x563ae8bccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f357dced640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba2410 0x563ae8bcd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44072/0 (socket says 172.21.15.43:44072) 2024-09-11T23:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f357e4ee640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8ba1af0 0x563ae8bccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:35416/0 (socket says 172.21.15.43:35416) 2024-09-11T23:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f357ecef640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8baccb0 0x563ae8ba6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:35684/0 (socket says 172.21.15.43:35684) 2024-09-11T23:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f357dced640 1 -- 172.21.15.43:0/420269545 learned_addr learned my addr 172.21.15.43:0/420269545 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f357dced640 1 -- 172.21.15.43:0/420269545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8ba1af0 msgr2=0x563ae8bccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f357dced640 1 --2- 172.21.15.43:0/420269545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8ba1af0 0x563ae8bccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f357dced640 1 -- 172.21.15.43:0/420269545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8baccb0 msgr2=0x563ae8ba6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f357dced640 1 --2- 172.21.15.43:0/420269545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8baccb0 0x563ae8ba6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f357dced640 1 -- 172.21.15.43:0/420269545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae8ba1920 con 0x563ae8ba2410 2024-09-11T23:25:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f357dced640 1 --2- 172.21.15.43:0/420269545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba2410 0x563ae8bcd1c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3568098580 tx=0x7f3568097f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2da7482dcc8300d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f3563fff640 1 -- 172.21.15.43:0/420269545 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35680b2070 con 0x563ae8ba2410 2024-09-11T23:25:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f3563fff640 1 -- 172.21.15.43:0/420269545 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f356809bbb0 con 0x563ae8ba2410 2024-09-11T23:25:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.467+0000 7f3563fff640 1 -- 172.21.15.43:0/420269545 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3568097900 con 0x563ae8ba2410 2024-09-11T23:25:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.468+0000 7f35835c9740 1 -- 172.21.15.43:0/420269545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba2410 msgr2=0x563ae8bcd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.468+0000 7f35835c9740 1 --2- 172.21.15.43:0/420269545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba2410 0x563ae8bcd1c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3568098580 tx=0x7f3568097f80 comp rx=0 tx=0).stop 2024-09-11T23:25:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.468+0000 7f35835c9740 1 -- 172.21.15.43:0/420269545 shutdown_connections 2024-09-11T23:25:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.468+0000 7f35835c9740 1 --2- 172.21.15.43:0/420269545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8baccb0 0x563ae8ba6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.468+0000 7f35835c9740 1 --2- 172.21.15.43:0/420269545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba2410 0x563ae8bcd1c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.468+0000 7f35835c9740 1 --2- 172.21.15.43:0/420269545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8ba1af0 0x563ae8bccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.468+0000 7f35835c9740 1 -- 172.21.15.43:0/420269545 >> 172.21.15.43:0/420269545 conn(0x563ae8b9e060 msgr2=0x563ae8babc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.468+0000 7f35835c9740 1 -- 172.21.15.43:0/420269545 shutdown_connections 2024-09-11T23:25:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.468+0000 7f35835c9740 1 -- 172.21.15.43:0/420269545 wait complete. 2024-09-11T23:25:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.468+0000 7f35835c9740 1 -- start start 2024-09-11T23:25:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.469+0000 7f35835c9740 1 Processor -- start 2024-09-11T23:25:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.469+0000 7f35835c9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba1af0 0x563ae8bbd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.469+0000 7f35835c9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8ba2410 0x563ae8bbdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.469+0000 7f35835c9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8baccb0 0x563ae8bbe3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.469+0000 7f35835c9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f35781a9d90 con 0x563ae8ba1af0 2024-09-11T23:25:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.469+0000 7f35835c9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f35781ab200 con 0x563ae8ba2410 2024-09-11T23:25:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.469+0000 7f35835c9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f35781aa460 con 0x563ae8baccb0 2024-09-11T23:25:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.469+0000 7f35835c9740 10 client.0 _ll_register_callbacks cb 0x7f3570009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.469+0000 7f357e4ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba1af0 0x563ae8bbd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.469+0000 7f357e4ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba1af0 0x563ae8bbd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44088/0 (socket says 172.21.15.43:44088) 2024-09-11T23:25:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.469+0000 7f357e4ee640 1 -- 172.21.15.43:0/2438716429 learned_addr learned my addr 172.21.15.43:0/2438716429 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.469+0000 7f357dced640 1 --2- 172.21.15.43:0/2438716429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8ba2410 0x563ae8bbdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.469+0000 7f357ecef640 1 --2- 172.21.15.43:0/2438716429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8baccb0 0x563ae8bbe3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.469+0000 7f357e4ee640 1 -- 172.21.15.43:0/2438716429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8baccb0 msgr2=0x563ae8bbe3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.469+0000 7f357e4ee640 1 --2- 172.21.15.43:0/2438716429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8baccb0 0x563ae8bbe3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.469+0000 7f357e4ee640 1 -- 172.21.15.43:0/2438716429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8ba2410 msgr2=0x563ae8bbdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.469+0000 7f357e4ee640 1 --2- 172.21.15.43:0/2438716429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8ba2410 0x563ae8bbdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.469+0000 7f357e4ee640 1 -- 172.21.15.43:0/2438716429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae8c2edf0 con 0x563ae8ba1af0 2024-09-11T23:25:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.469+0000 7f357e4ee640 1 --2- 172.21.15.43:0/2438716429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba1af0 0x563ae8bbd970 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f357402a930 tx=0x7f3574005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eaa04781d70be81 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.469+0000 7f3561ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:25:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.470+0000 7f3561ffb640 1 -- 172.21.15.43:0/2438716429 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3574037040 con 0x563ae8ba1af0 2024-09-11T23:25:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.470+0000 7f3561ffb640 1 -- 172.21.15.43:0/2438716429 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f35740036b0 con 0x563ae8ba1af0 2024-09-11T23:25:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.470+0000 7f35835c9740 1 -- 172.21.15.43:0/2438716429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f356809bbb0 con 0x563ae8ba1af0 2024-09-11T23:25:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.470+0000 7f3561ffb640 1 -- 172.21.15.43:0/2438716429 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f357405e050 con 0x563ae8ba1af0 2024-09-11T23:25:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.470+0000 7f35835c9740 10 client.16059 fetch_fsmap learned FSMap version 29 2024-09-11T23:25:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.470+0000 7f35835c9740 1 -- 172.21.15.43:0/2438716429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563ae8c30100 con 0x563ae8ba1af0 2024-09-11T23:25:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.470+0000 7f3561ffb640 1 -- 172.21.15.43:0/2438716429 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f356809bbb0 con 0x563ae8ba1af0 2024-09-11T23:25:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.470+0000 7f3561ffb640 1 -- 172.21.15.43:0/2438716429 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f3574032b90 con 0x563ae8ba1af0 2024-09-11T23:25:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.470+0000 7f35835c9740 10 client.16059 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T23:25:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.470+0000 7f35835c9740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:25:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.470+0000 7f35835c9740 20 client.16059 populate_metadata read hostname 'smithi043' 2024-09-11T23:25:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.470+0000 7f35835c9740 1 --2- 172.21.15.43:0/2438716429 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x563ae8c30850 0x563ae8c32c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.470+0000 7f35835c9740 4 client.16059 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:25:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.470+0000 7f35835c9740 1 -- 172.21.15.43:0/2438716429 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ae8b0dd70 con 0x563ae8c30850 2024-09-11T23:25:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.470+0000 7f357dced640 1 --2- 172.21.15.43:0/2438716429 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x563ae8c30850 0x563ae8c32c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.471+0000 7f3561ffb640 10 client.16059 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:25:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.471+0000 7f357dced640 1 --2- 172.21.15.43:0/2438716429 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x563ae8c30850 0x563ae8c32c70 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec2c9b971888cea server_cookie=91b2ef572e351524 in_seq=0 out_seq=0 2024-09-11T23:25:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f3561ffb640 1 -- 172.21.15.43:0/2438716429 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563ae8b0dd70 con 0x563ae8c30850 2024-09-11T23:25:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f3561ffb640 10 client.16059 handle_command_reply: tid=0 2024-09-11T23:25:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f35835c9740 1 client.16059 shutdown 2024-09-11T23:25:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f35835c9740 1 -- 172.21.15.43:0/2438716429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba1af0 msgr2=0x563ae8bbd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f35835c9740 1 --2- 172.21.15.43:0/2438716429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba1af0 0x563ae8bbd970 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f357402a930 tx=0x7f3574005f90 comp rx=0 tx=0).stop 2024-09-11T23:25:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f35835c9740 1 -- 172.21.15.43:0/2438716429 shutdown_connections 2024-09-11T23:25:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f35835c9740 1 --2- 172.21.15.43:0/2438716429 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x563ae8c30850 0x563ae8c32c70 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f35835c9740 1 --2- 172.21.15.43:0/2438716429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ae8baccb0 0x563ae8bbe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f35835c9740 1 --2- 172.21.15.43:0/2438716429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ae8ba2410 0x563ae8bbdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f35835c9740 1 --2- 172.21.15.43:0/2438716429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563ae8ba1af0 0x563ae8bbd970 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f3561ffb640 0 client.16059 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:25:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f35835c9740 1 -- 172.21.15.43:0/2438716429 >> 172.21.15.43:0/2438716429 conn(0x563ae8b9e060 msgr2=0x563ae8bb1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f35835c9740 1 -- 172.21.15.43:0/2438716429 shutdown_connections 2024-09-11T23:25:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f35835c9740 1 -- 172.21.15.43:0/2438716429 wait complete. 2024-09-11T23:25:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f35835c9740 20 client.16059 trim_cache size 0 max 16384 2024-09-11T23:25:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:25:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:25:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:25:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:25:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f3580752640 1 -- 172.21.15.43:0/2977511162 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f35640a2cf0 msgr2=0x7f35640a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f3580752640 1 --2- 172.21.15.43:0/2977511162 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f35640a2cf0 0x7f35640a51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f35740095f0 tx=0x7f35740056d0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f3580752640 1 -- 172.21.15.43:0/2977511162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3578101340 msgr2=0x7f35781988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f3580752640 1 --2- 172.21.15.43:0/2977511162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3578101340 0x7f35781988c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3568005fd0 tx=0x7f356800ee50 comp rx=0 tx=0).stop 2024-09-11T23:25:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f3580752640 1 -- 172.21.15.43:0/2977511162 shutdown_connections 2024-09-11T23:25:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f3580752640 1 --2- 172.21.15.43:0/2977511162 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f35640a2cf0 0x7f35640a51b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f3580752640 1 --2- 172.21.15.43:0/2977511162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35781069b0 0x7f357819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f3580752640 1 --2- 172.21.15.43:0/2977511162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3578101340 0x7f35781988c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f3580752640 1 --2- 172.21.15.43:0/2977511162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35781009a0 0x7f3578198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.496+0000 7f3580752640 1 -- 172.21.15.43:0/2977511162 >> 172.21.15.43:0/2977511162 conn(0x7f35780779a0 msgr2=0x7f3578069ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.497+0000 7f3580752640 1 -- 172.21.15.43:0/2977511162 shutdown_connections 2024-09-11T23:25:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:35.497+0000 7f3580752640 1 -- 172.21.15.43:0/2977511162 wait complete. 2024-09-11T23:25:35.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:35 smithi043 ceph-mon[30950]: pgmap v116: 97 pgs: 97 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 23 MiB/s wr, 3.57k op/s 2024-09-11T23:25:35.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:35 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1872309435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:25:35.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:35 smithi090 ceph-mon[35130]: pgmap v116: 97 pgs: 97 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 23 MiB/s wr, 3.57k op/s 2024-09-11T23:25:35.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:35 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1872309435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:25:35.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:35 smithi097 ceph-mon[35562]: pgmap v116: 97 pgs: 97 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 23 MiB/s wr, 3.57k op/s 2024-09-11T23:25:35.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:35 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1872309435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:25:35.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.676+0000 7fa0891a4640 1 -- 172.21.15.43:0/860200624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa084101420 msgr2=0x7fa08410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:35.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.676+0000 7fa0891a4640 1 --2- 172.21.15.43:0/860200624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa084101420 0x7fa08410d950 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009cd0 tx=0x7fa07c02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.676+0000 7fa0891a4640 1 -- 172.21.15.43:0/860200624 shutdown_connections 2024-09-11T23:25:35.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.676+0000 7fa0891a4640 1 --2- 172.21.15.43:0/860200624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa084101420 0x7fa08410d950 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.676+0000 7fa0891a4640 1 --2- 172.21.15.43:0/860200624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa084100a80 0x7fa084100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.676+0000 7fa0891a4640 1 --2- 172.21.15.43:0/860200624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa084106a80 0x7fa084106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.676+0000 7fa0891a4640 1 -- 172.21.15.43:0/860200624 >> 172.21.15.43:0/860200624 conn(0x7fa0840fc7a0 msgr2=0x7fa0840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:35.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.676+0000 7fa0891a4640 1 -- 172.21.15.43:0/860200624 shutdown_connections 2024-09-11T23:25:35.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.676+0000 7fa0891a4640 1 -- 172.21.15.43:0/860200624 wait complete. 2024-09-11T23:25:35.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.677+0000 7fa0891a4640 1 Processor -- start 2024-09-11T23:25:35.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.677+0000 7fa0891a4640 1 -- start start 2024-09-11T23:25:35.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.677+0000 7fa0891a4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa084100a80 0x7fa0840fce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.677+0000 7fa0891a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa084101420 0x7fa0840fd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.677+0000 7fa0891a4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa084106a80 0x7fa0840fd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.677+0000 7fa0891a4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa084110e80 con 0x7fa084101420 2024-09-11T23:25:35.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.677+0000 7fa0891a4640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa084110d00 con 0x7fa084100a80 2024-09-11T23:25:35.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.677+0000 7fa0891a4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa084111000 con 0x7fa084106a80 2024-09-11T23:25:35.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.677+0000 7fa07a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa084101420 0x7fa0840fd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.947 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:25:35.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.677+0000 7fa082d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa084100a80 0x7fa0840fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.677+0000 7fa07a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa084101420 0x7fa0840fd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44014/0 (socket says 172.21.15.43:44014) 2024-09-11T23:25:35.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.677+0000 7fa07a59c640 1 -- 172.21.15.43:0/1872309435 learned_addr learned my addr 172.21.15.43:0/1872309435 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:35.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.678+0000 7fa08359e640 1 --2- 172.21.15.43:0/1872309435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa084106a80 0x7fa0840fd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.678+0000 7fa07a59c640 1 -- 172.21.15.43:0/1872309435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa084106a80 msgr2=0x7fa0840fd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:35.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.678+0000 7fa07a59c640 1 --2- 172.21.15.43:0/1872309435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa084106a80 0x7fa0840fd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.678+0000 7fa07a59c640 1 -- 172.21.15.43:0/1872309435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa084100a80 msgr2=0x7fa0840fce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:35.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.678+0000 7fa07a59c640 1 --2- 172.21.15.43:0/1872309435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa084100a80 0x7fa0840fce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.678+0000 7fa07a59c640 1 -- 172.21.15.43:0/1872309435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0841a68b0 con 0x7fa084101420 2024-09-11T23:25:35.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.678+0000 7fa07a59c640 1 --2- 172.21.15.43:0/1872309435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa084101420 0x7fa0840fd340 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa06c00e9f0 tx=0x7fa06c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:35.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.678+0000 7fa080d99640 1 -- 172.21.15.43:0/1872309435 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06c00cd80 con 0x7fa084101420 2024-09-11T23:25:35.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.678+0000 7fa080d99640 1 -- 172.21.15.43:0/1872309435 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa06c010910 con 0x7fa084101420 2024-09-11T23:25:35.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.678+0000 7fa0891a4640 1 -- 172.21.15.43:0/1872309435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0841a6ba0 con 0x7fa084101420 2024-09-11T23:25:35.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.678+0000 7fa080d99640 1 -- 172.21.15.43:0/1872309435 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06c018990 con 0x7fa084101420 2024-09-11T23:25:35.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.678+0000 7fa0891a4640 1 -- 172.21.15.43:0/1872309435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0841a70e0 con 0x7fa084101420 2024-09-11T23:25:35.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.679+0000 7fa0891a4640 1 -- 172.21.15.43:0/1872309435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa084100ee0 con 0x7fa084101420 2024-09-11T23:25:35.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.681+0000 7fa080d99640 1 -- 172.21.15.43:0/1872309435 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa06c010ac0 con 0x7fa084101420 2024-09-11T23:25:35.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.681+0000 7fa080d99640 1 --2- 172.21.15.43:0/1872309435 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa0700a2cf0 0x7fa0700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:35.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.681+0000 7fa080d99640 1 -- 172.21.15.43:0/1872309435 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7fa06c0cfb30 con 0x7fa084101420 2024-09-11T23:25:35.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.681+0000 7fa082d9d640 1 --2- 172.21.15.43:0/1872309435 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa0700a2cf0 0x7fa0700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:35.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.682+0000 7fa082d9d640 1 --2- 172.21.15.43:0/1872309435 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa0700a2cf0 0x7fa0700a51b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa0740058c0 tx=0x7fa07400a2d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:35.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.685+0000 7fa080d99640 1 -- 172.21.15.43:0/1872309435 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa06c09c100 con 0x7fa084101420 2024-09-11T23:25:35.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.901+0000 7fa0891a4640 1 -- 172.21.15.43:0/1872309435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0840fe330 con 0x7fa084101420 2024-09-11T23:25:35.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.901+0000 7fa080d99640 1 -- 172.21.15.43:0/1872309435 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11896 (secure 0 0 0) 0x7fa0840fe330 con 0x7fa084101420 2024-09-11T23:25:35.949 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2024-09-11T23:25:35.949 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"btime":"2024-09-11T23:25:32:437970+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:25:32.437954+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25571":{"gid":25571,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/568491708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":568491708},{"type":"v1","addr":"172.21.15.90:6835","nonce":568491708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:25:35.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.903+0000 7fa0891a4640 1 -- 172.21.15.43:0/1872309435 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa0700a2cf0 msgr2=0x7fa0700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:35.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.903+0000 7fa0891a4640 1 --2- 172.21.15.43:0/1872309435 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa0700a2cf0 0x7fa0700a51b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa0740058c0 tx=0x7fa07400a2d0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.903+0000 7fa0891a4640 1 -- 172.21.15.43:0/1872309435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa084101420 msgr2=0x7fa0840fd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:35.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.903+0000 7fa0891a4640 1 --2- 172.21.15.43:0/1872309435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa084101420 0x7fa0840fd340 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa06c00e9f0 tx=0x7fa06c00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.903+0000 7fa0891a4640 1 -- 172.21.15.43:0/1872309435 shutdown_connections 2024-09-11T23:25:35.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.903+0000 7fa0891a4640 1 --2- 172.21.15.43:0/1872309435 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa0700a2cf0 0x7fa0700a51b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.903+0000 7fa0891a4640 1 --2- 172.21.15.43:0/1872309435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa084106a80 0x7fa0840fd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.903+0000 7fa0891a4640 1 --2- 172.21.15.43:0/1872309435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa084101420 0x7fa0840fd340 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.903+0000 7fa0891a4640 1 --2- 172.21.15.43:0/1872309435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa084100a80 0x7fa0840fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:35.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.903+0000 7fa0891a4640 1 -- 172.21.15.43:0/1872309435 >> 172.21.15.43:0/1872309435 conn(0x7fa0840fc7a0 msgr2=0x7fa08406aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:35.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.903+0000 7fa0891a4640 1 -- 172.21.15.43:0/1872309435 shutdown_connections 2024-09-11T23:25:35.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:34.903+0000 7fa0891a4640 1 -- 172.21.15.43:0/1872309435 wait complete. 2024-09-11T23:25:36.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:36 smithi090 ceph-mon[35130]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:25:36.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:36 smithi097 ceph-mon[35562]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:25:36.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:36 smithi043 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:25:37.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:37 smithi090 ceph-mon[35130]: pgmap v117: 97 pgs: 97 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 25 MiB/s wr, 3.84k op/s 2024-09-11T23:25:37.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:37 smithi097 ceph-mon[35562]: pgmap v117: 97 pgs: 97 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 25 MiB/s wr, 3.84k op/s 2024-09-11T23:25:37.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:37 smithi043 ceph-mon[30950]: pgmap v117: 97 pgs: 97 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 25 MiB/s wr, 3.84k op/s 2024-09-11T23:25:39.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:39 smithi090 ceph-mon[35130]: pgmap v118: 97 pgs: 97 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 20 MiB/s wr, 3.06k op/s 2024-09-11T23:25:39.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:39 smithi097 ceph-mon[35562]: pgmap v118: 97 pgs: 97 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 20 MiB/s wr, 3.06k op/s 2024-09-11T23:25:39.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:39 smithi043 ceph-mon[30950]: pgmap v118: 97 pgs: 97 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 20 MiB/s wr, 3.06k op/s 2024-09-11T23:25:41.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:41 smithi090 ceph-mon[35130]: pgmap v119: 97 pgs: 97 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 18 MiB/s wr, 2.67k op/s 2024-09-11T23:25:41.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:41 smithi090 ceph-mon[35130]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:25:41.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:41 smithi097 ceph-mon[35562]: pgmap v119: 97 pgs: 97 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 18 MiB/s wr, 2.67k op/s 2024-09-11T23:25:41.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:41 smithi097 ceph-mon[35562]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:25:41.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:41 smithi043 ceph-mon[30950]: pgmap v119: 97 pgs: 97 active+clean; 16 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 18 MiB/s wr, 2.67k op/s 2024-09-11T23:25:41.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:41 smithi043 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:25:43.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:43 smithi090 ceph-mon[35130]: pgmap v120: 97 pgs: 97 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 25 MiB/s wr, 3.64k op/s 2024-09-11T23:25:43.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:43 smithi097 ceph-mon[35562]: pgmap v120: 97 pgs: 97 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 25 MiB/s wr, 3.64k op/s 2024-09-11T23:25:43.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:43 smithi043 ceph-mon[30950]: pgmap v120: 97 pgs: 97 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 25 MiB/s wr, 3.64k op/s 2024-09-11T23:25:45.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:45 smithi090 ceph-mon[35130]: pgmap v121: 97 pgs: 97 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 27 MiB/s wr, 3.92k op/s 2024-09-11T23:25:45.802 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:45 smithi097 ceph-mon[35562]: pgmap v121: 97 pgs: 97 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 27 MiB/s wr, 3.92k op/s 2024-09-11T23:25:45.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:45 smithi043 ceph-mon[30950]: pgmap v121: 97 pgs: 97 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 27 MiB/s wr, 3.92k op/s 2024-09-11T23:25:46.139 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0' 2024-09-11T23:25:46.140 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0 2024-09-11T23:25:46.302 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:25:47.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:47 smithi090 ceph-mon[35130]: pgmap v122: 97 pgs: 97 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 MiB/s wr, 3.46k op/s 2024-09-11T23:25:47.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:47 smithi097 ceph-mon[35562]: pgmap v122: 97 pgs: 97 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 MiB/s wr, 3.46k op/s 2024-09-11T23:25:47.816 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:47 smithi043 ceph-mon[30950]: pgmap v122: 97 pgs: 97 active+clean; 16 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 MiB/s wr, 3.46k op/s 2024-09-11T23:25:49.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:49 smithi090 ceph-mon[35130]: pgmap v123: 97 pgs: 97 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 34 MiB/s wr, 4.97k op/s 2024-09-11T23:25:49.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:49 smithi097 ceph-mon[35562]: pgmap v123: 97 pgs: 97 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 34 MiB/s wr, 4.97k op/s 2024-09-11T23:25:49.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:49 smithi043 ceph-mon[30950]: pgmap v123: 97 pgs: 97 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 34 MiB/s wr, 4.97k op/s 2024-09-11T23:25:50.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.044+0000 7f6e6fc73640 1 -- 172.21.15.43:0/2502746135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e681009d0 msgr2=0x7f6e68100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:50.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.044+0000 7f6e6fc73640 1 --2- 172.21.15.43:0/2502746135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e681009d0 0x7f6e68100db0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009900 tx=0x7f6e6402a410 comp rx=0 tx=0).stop 2024-09-11T23:25:50.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.044+0000 7f6e6fc73640 1 -- 172.21.15.43:0/2502746135 shutdown_connections 2024-09-11T23:25:50.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.044+0000 7f6e6fc73640 1 --2- 172.21.15.43:0/2502746135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e6810e4f0 0x7f6e68110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.044+0000 7f6e6fc73640 1 --2- 172.21.15.43:0/2502746135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e68101380 0x7f6e6810dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.044+0000 7f6e6fc73640 1 --2- 172.21.15.43:0/2502746135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e681009d0 0x7f6e68100db0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.044+0000 7f6e6fc73640 1 -- 172.21.15.43:0/2502746135 >> 172.21.15.43:0/2502746135 conn(0x7f6e680fc7d0 msgr2=0x7f6e680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:50.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.045+0000 7f6e6fc73640 1 -- 172.21.15.43:0/2502746135 shutdown_connections 2024-09-11T23:25:50.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.045+0000 7f6e6fc73640 1 -- 172.21.15.43:0/2502746135 wait complete. 2024-09-11T23:25:50.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.045+0000 7f6e6fc73640 1 Processor -- start 2024-09-11T23:25:50.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.045+0000 7f6e6fc73640 1 -- start start 2024-09-11T23:25:50.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.046+0000 7f6e6fc73640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e681009d0 0x7f6e6819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:50.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.046+0000 7f6e6fc73640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e68101380 0x7f6e6819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:50.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.046+0000 7f6e6fc73640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e6810e4f0 0x7f6e6819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:50.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.046+0000 7f6e6da0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e681009d0 0x7f6e6819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:50.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.046+0000 7f6e6fc73640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6e68114270 con 0x7f6e681009d0 2024-09-11T23:25:50.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.046+0000 7f6e6da0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e681009d0 0x7f6e6819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54788/0 (socket says 172.21.15.43:54788) 2024-09-11T23:25:50.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.046+0000 7f6e6da0f640 1 -- 172.21.15.43:0/1330074008 learned_addr learned my addr 172.21.15.43:0/1330074008 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:50.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.046+0000 7f6e6fc73640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6e681140f0 con 0x7f6e6810e4f0 2024-09-11T23:25:50.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.046+0000 7f6e6d20e640 1 --2- 172.21.15.43:0/1330074008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e68101380 0x7f6e6819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:50.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.046+0000 7f6e6fc73640 1 -- 172.21.15.43:0/1330074008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6e681143f0 con 0x7f6e68101380 2024-09-11T23:25:50.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.046+0000 7f6e6e210640 1 --2- 172.21.15.43:0/1330074008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e6810e4f0 0x7f6e6819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:50.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.046+0000 7f6e6d20e640 1 -- 172.21.15.43:0/1330074008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e6810e4f0 msgr2=0x7f6e6819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:50.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.046+0000 7f6e6d20e640 1 --2- 172.21.15.43:0/1330074008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e6810e4f0 0x7f6e6819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.046+0000 7f6e6d20e640 1 -- 172.21.15.43:0/1330074008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e681009d0 msgr2=0x7f6e6819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:50.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.046+0000 7f6e6d20e640 1 --2- 172.21.15.43:0/1330074008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e681009d0 0x7f6e6819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.046+0000 7f6e6d20e640 1 -- 172.21.15.43:0/1330074008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6819f700 con 0x7f6e68101380 2024-09-11T23:25:50.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.046+0000 7f6e6e210640 1 --2- 172.21.15.43:0/1330074008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e6810e4f0 0x7f6e6819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:50.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.046+0000 7f6e6da0f640 1 --2- 172.21.15.43:0/1330074008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e681009d0 0x7f6e6819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:50.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.047+0000 7f6e6d20e640 1 --2- 172.21.15.43:0/1330074008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e68101380 0x7f6e6819ac90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800e9f0 tx=0x7f6e5800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:50.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.047+0000 7f6e5effd640 1 -- 172.21.15.43:0/1330074008 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5800ccc0 con 0x7f6e68101380 2024-09-11T23:25:50.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.047+0000 7f6e5effd640 1 -- 172.21.15.43:0/1330074008 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e58010910 con 0x7f6e68101380 2024-09-11T23:25:50.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.047+0000 7f6e6fc73640 1 -- 172.21.15.43:0/1330074008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6819f9f0 con 0x7f6e68101380 2024-09-11T23:25:50.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.047+0000 7f6e5effd640 1 -- 172.21.15.43:0/1330074008 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58018960 con 0x7f6e68101380 2024-09-11T23:25:50.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.047+0000 7f6e6fc73640 1 -- 172.21.15.43:0/1330074008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e681a72d0 con 0x7f6e68101380 2024-09-11T23:25:50.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.050+0000 7f6e7204c740 1 Processor -- start 2024-09-11T23:25:50.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.050+0000 7f6e7204c740 1 -- start start 2024-09-11T23:25:50.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.051+0000 7f6e7204c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265db60 0x55b74265df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:50.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.051+0000 7f6e7204c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b74265e480 0x55b742668800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:50.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.051+0000 7f6e7204c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b742668d40 0x55b74266b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:50.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.051+0000 7f6e7204c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b7425c3280 con 0x55b74265db60 2024-09-11T23:25:50.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.051+0000 7f6e7204c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b74256ab10 con 0x55b74265e480 2024-09-11T23:25:50.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.051+0000 7f6e7204c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b7425bf240 con 0x55b742668d40 2024-09-11T23:25:50.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.051+0000 7f6e5effd640 1 -- 172.21.15.43:0/1330074008 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6e58010ac0 con 0x7f6e68101380 2024-09-11T23:25:50.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.051+0000 7f6e6da0f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b74265e480 0x55b742668800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:50.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.051+0000 7f6e6da0f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b74265e480 0x55b742668800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:53760/0 (socket says 172.21.15.43:53760) 2024-09-11T23:25:50.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.051+0000 7f6e6da0f640 1 -- 172.21.15.43:0/814916748 learned_addr learned my addr 172.21.15.43:0/814916748 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:50.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.051+0000 7f6e6e210640 1 --2- 172.21.15.43:0/814916748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265db60 0x55b74265df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:50.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.052+0000 7f6e6d20e640 1 --2- 172.21.15.43:0/814916748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b742668d40 0x55b74266b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:50.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.052+0000 7f6e6da0f640 1 -- 172.21.15.43:0/814916748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b742668d40 msgr2=0x55b74266b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:50.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.052+0000 7f6e6da0f640 1 --2- 172.21.15.43:0/814916748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b742668d40 0x55b74266b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.052+0000 7f6e6da0f640 1 -- 172.21.15.43:0/814916748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265db60 msgr2=0x55b74265df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:50.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.052+0000 7f6e6da0f640 1 --2- 172.21.15.43:0/814916748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265db60 0x55b74265df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.052+0000 7f6e6da0f640 1 -- 172.21.15.43:0/814916748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7425771f0 con 0x55b74265e480 2024-09-11T23:25:50.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.052+0000 7f6e6d20e640 1 --2- 172.21.15.43:0/814916748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b742668d40 0x55b74266b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:25:50.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.052+0000 7f6e6e210640 1 --2- 172.21.15.43:0/814916748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265db60 0x55b74265df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:50.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.052+0000 7f6e5effd640 1 --2- 172.21.15.43:0/1330074008 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6e540a2d70 0x7f6e540a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:50.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.053+0000 7f6e6da0f640 1 --2- 172.21.15.43:0/814916748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b74265e480 0x55b742668800 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009a20 tx=0x7f6e64005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=249a361bfe44a0f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:50.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.053+0000 7f6e6e210640 1 --2- 172.21.15.43:0/1330074008 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6e540a2d70 0x7f6e540a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:50.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.053+0000 7f6e5effd640 1 -- 172.21.15.43:0/1330074008 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f6e580cf840 con 0x7f6e68101380 2024-09-11T23:25:50.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.053+0000 7f6e53fff640 1 -- 172.21.15.43:0/814916748 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e64040070 con 0x55b74265e480 2024-09-11T23:25:50.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.053+0000 7f6e53fff640 1 -- 172.21.15.43:0/814916748 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e64031970 con 0x55b74265e480 2024-09-11T23:25:50.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.054+0000 7f6e53fff640 1 -- 172.21.15.43:0/814916748 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e64040070 con 0x55b74265e480 2024-09-11T23:25:50.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.054+0000 7f6e7204c740 1 -- 172.21.15.43:0/814916748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b74265e480 msgr2=0x55b742668800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:50.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.054+0000 7f6e6e210640 1 --2- 172.21.15.43:0/1330074008 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6e540a2d70 0x7f6e540a5230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009fd0 tx=0x7f6e60009500 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:50.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.054+0000 7f6e7204c740 1 --2- 172.21.15.43:0/814916748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b74265e480 0x55b742668800 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009a20 tx=0x7f6e64005dd0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.054+0000 7f6e7204c740 1 -- 172.21.15.43:0/814916748 shutdown_connections 2024-09-11T23:25:50.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.054+0000 7f6e7204c740 1 --2- 172.21.15.43:0/814916748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b742668d40 0x55b74266b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.054+0000 7f6e7204c740 1 --2- 172.21.15.43:0/814916748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b74265e480 0x55b742668800 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.054+0000 7f6e7204c740 1 --2- 172.21.15.43:0/814916748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265db60 0x55b74265df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.054+0000 7f6e7204c740 1 -- 172.21.15.43:0/814916748 >> 172.21.15.43:0/814916748 conn(0x55b74265a0f0 msgr2=0x55b74265a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:50.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.054+0000 7f6e7204c740 1 -- 172.21.15.43:0/814916748 shutdown_connections 2024-09-11T23:25:50.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.054+0000 7f6e7204c740 1 -- 172.21.15.43:0/814916748 wait complete. 2024-09-11T23:25:50.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.055+0000 7f6e7204c740 1 -- start start 2024-09-11T23:25:50.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.055+0000 7f6e7204c740 1 Processor -- start 2024-09-11T23:25:50.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.055+0000 7f6e7204c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b74265db60 0x55b7426687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:50.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.055+0000 7f6e7204c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265e480 0x55b742665af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:50.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.056+0000 7f6e7204c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b742668d40 0x55b742666280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:50.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.056+0000 7f6e6da0f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b74265db60 0x55b7426687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:50.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.056+0000 7f6e6da0f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b74265db60 0x55b7426687a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:59982/0 (socket says 172.21.15.43:59982) 2024-09-11T23:25:50.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.056+0000 7f6e6d20e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265e480 0x55b742665af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:50.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.056+0000 7f6e6d20e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265e480 0x55b742665af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54810/0 (socket says 172.21.15.43:54810) 2024-09-11T23:25:50.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.056+0000 7f6e6da0f640 1 -- 172.21.15.43:0/3581686479 learned_addr learned my addr 172.21.15.43:0/3581686479 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:50.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.056+0000 7f6e7204c740 1 -- 172.21.15.43:0/3581686479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b74266b960 con 0x55b74265e480 2024-09-11T23:25:50.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.056+0000 7f6e7204c740 1 -- 172.21.15.43:0/3581686479 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b74257df10 con 0x55b742668d40 2024-09-11T23:25:50.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.056+0000 7f6e7204c740 1 -- 172.21.15.43:0/3581686479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b74266bae0 con 0x55b74265db60 2024-09-11T23:25:50.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.056+0000 7f6e6e210640 1 --2- 172.21.15.43:0/3581686479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b742668d40 0x55b742666280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:50.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.056+0000 7f6e7204c740 10 client.0 _ll_register_callbacks cb 0x55b742546a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:50.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.056+0000 7f6e6d20e640 1 -- 172.21.15.43:0/3581686479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b74265db60 msgr2=0x55b7426687a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:50.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.056+0000 7f6e6d20e640 1 --2- 172.21.15.43:0/3581686479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b74265db60 0x55b7426687a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.056+0000 7f6e6d20e640 1 -- 172.21.15.43:0/3581686479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b742668d40 msgr2=0x55b742666280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:50.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.056+0000 7f6e6d20e640 1 --2- 172.21.15.43:0/3581686479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b742668d40 0x55b742666280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.056+0000 7f6e6d20e640 1 -- 172.21.15.43:0/3581686479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7423346a0 con 0x55b74265e480 2024-09-11T23:25:50.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.056+0000 7f6e6da0f640 1 --2- 172.21.15.43:0/3581686479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b74265db60 0x55b7426687a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:50.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.057+0000 7f6e6d20e640 1 --2- 172.21.15.43:0/3581686479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265e480 0x55b742665af0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6e580ccf30 tx=0x7f6e5809b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205a0ad1ed90f98a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:50.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.057+0000 7f6e517fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:25:50.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.057+0000 7f6e517fa640 1 -- 172.21.15.43:0/3581686479 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5809b400 con 0x55b74265e480 2024-09-11T23:25:50.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.057+0000 7f6e517fa640 1 -- 172.21.15.43:0/3581686479 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f6e58010430 con 0x55b74265e480 2024-09-11T23:25:50.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.057+0000 7f6e7204c740 1 -- 172.21.15.43:0/3581686479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e64031970 con 0x55b74265e480 2024-09-11T23:25:50.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.057+0000 7f6e517fa640 1 -- 172.21.15.43:0/3581686479 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6e580a4970 con 0x55b74265e480 2024-09-11T23:25:50.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.057+0000 7f6e7204c740 10 client.16071 fetch_fsmap learned FSMap version 30 2024-09-11T23:25:50.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.057+0000 7f6e7204c740 1 -- 172.21.15.43:0/3581686479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b7426773e0 con 0x55b74265e480 2024-09-11T23:25:50.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.058+0000 7f6e517fa640 1 -- 172.21.15.43:0/3581686479 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e64031970 con 0x55b74265e480 2024-09-11T23:25:50.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.058+0000 7f6e517fa640 1 -- 172.21.15.43:0/3581686479 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f6e58086040 con 0x55b74265e480 2024-09-11T23:25:50.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.058+0000 7f6e7204c740 10 client.16071 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:25:50.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.058+0000 7f6e7204c740 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:25:50.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.058+0000 7f6e7204c740 20 client.16071 populate_metadata read hostname 'smithi043' 2024-09-11T23:25:50.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.058+0000 7f6e7204c740 1 --2- 172.21.15.43:0/3581686479 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b742678d90 0x55b74267b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:50.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.058+0000 7f6e7204c740 4 client.16071 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:25:50.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.058+0000 7f6e7204c740 1 -- 172.21.15.43:0/3581686479 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7425d07f0 con 0x55b742678d90 2024-09-11T23:25:50.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.058+0000 7f6e6da0f640 1 --2- 172.21.15.43:0/3581686479 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b742678d90 0x55b74267b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:50.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.059+0000 7f6e517fa640 10 client.16071 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:25:50.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.059+0000 7f6e6da0f640 1 --2- 172.21.15.43:0/3581686479 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b742678d90 0x55b74267b200 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbb2d0da429402fa server_cookie=314677cb4dac792b in_seq=0 out_seq=0 2024-09-11T23:25:50.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.060+0000 7f6e517fa640 1 -- 172.21.15.43:0/3581686479 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b7425d07f0 con 0x55b742678d90 2024-09-11T23:25:50.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.060+0000 7f6e517fa640 10 client.16071 handle_command_reply: tid=0 2024-09-11T23:25:50.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.060+0000 7f6e7204c740 1 client.16071 shutdown 2024-09-11T23:25:50.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.062+0000 7f6e7204c740 1 -- 172.21.15.43:0/3581686479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265e480 msgr2=0x55b742665af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:50.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.062+0000 7f6e7204c740 1 --2- 172.21.15.43:0/3581686479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265e480 0x55b742665af0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6e580ccf30 tx=0x7f6e5809b710 comp rx=0 tx=0).stop 2024-09-11T23:25:50.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.062+0000 7f6e7204c740 1 -- 172.21.15.43:0/3581686479 shutdown_connections 2024-09-11T23:25:50.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.062+0000 7f6e7204c740 1 --2- 172.21.15.43:0/3581686479 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b742678d90 0x55b74267b200 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.062+0000 7f6e7204c740 1 --2- 172.21.15.43:0/3581686479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b742668d40 0x55b742666280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.062+0000 7f6e517fa640 0 client.16071 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:25:50.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.062+0000 7f6e7204c740 1 --2- 172.21.15.43:0/3581686479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265e480 0x55b742665af0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.062+0000 7f6e7204c740 1 --2- 172.21.15.43:0/3581686479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b74265db60 0x55b7426687a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.062+0000 7f6e7204c740 1 -- 172.21.15.43:0/3581686479 >> 172.21.15.43:0/3581686479 conn(0x55b74265a0f0 msgr2=0x55b74265fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:50.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.063+0000 7f6e7204c740 1 -- 172.21.15.43:0/3581686479 shutdown_connections 2024-09-11T23:25:50.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.063+0000 7f6e7204c740 1 -- 172.21.15.43:0/3581686479 wait complete. 2024-09-11T23:25:50.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.064+0000 7f6e7204c740 20 client.16071 trim_cache size 0 max 16384 2024-09-11T23:25:50.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.092+0000 7f6e7204c740 1 Processor -- start 2024-09-11T23:25:50.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.093+0000 7f6e7204c740 1 -- start start 2024-09-11T23:25:50.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.093+0000 7f6e7204c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b74265db60 0x55b7426643c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:50.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.093+0000 7f6e7204c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265e480 0x55b742664900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:50.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.093+0000 7f6e7204c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b742668d40 0x55b742680ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:50.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.093+0000 7f6e7204c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6e681b24c0 con 0x55b74265e480 2024-09-11T23:25:50.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.093+0000 7f6e7204c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6e681afd90 con 0x55b742668d40 2024-09-11T23:25:50.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.093+0000 7f6e7204c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6e681adbe0 con 0x55b74265db60 2024-09-11T23:25:50.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.093+0000 7f6e6da0f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b74265db60 0x55b7426643c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:50.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.093+0000 7f6e6da0f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b74265db60 0x55b7426643c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:59994/0 (socket says 172.21.15.43:59994) 2024-09-11T23:25:50.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.093+0000 7f6e6da0f640 1 -- 172.21.15.43:0/881189622 learned_addr learned my addr 172.21.15.43:0/881189622 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:50.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.093+0000 7f6e6e210640 1 --2- 172.21.15.43:0/881189622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b742668d40 0x55b742680ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:50.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.093+0000 7f6e6da0f640 1 -- 172.21.15.43:0/881189622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b742668d40 msgr2=0x55b742680ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:50.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.093+0000 7f6e6da0f640 1 --2- 172.21.15.43:0/881189622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b742668d40 0x55b742680ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.093+0000 7f6e6da0f640 1 -- 172.21.15.43:0/881189622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265e480 msgr2=0x55b742664900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:50.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.093+0000 7f6e6d20e640 1 --2- 172.21.15.43:0/881189622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265e480 0x55b742664900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:50.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.094+0000 7f6e6da0f640 1 --2- 172.21.15.43:0/881189622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265e480 0x55b742664900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.094+0000 7f6e6da0f640 1 -- 172.21.15.43:0/881189622 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7426665f0 con 0x55b74265db60 2024-09-11T23:25:50.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.094+0000 7f6e6d20e640 1 --2- 172.21.15.43:0/881189622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265e480 0x55b742664900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:25:50.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.094+0000 7f6e6da0f640 1 --2- 172.21.15.43:0/881189622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b74265db60 0x55b7426643c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6e64005560 tx=0x7f6e64047040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85ee8744c62d7c39 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:50.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.094+0000 7f6e537fe640 1 -- 172.21.15.43:0/881189622 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e64040040 con 0x55b74265db60 2024-09-11T23:25:50.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.095+0000 7f6e537fe640 1 -- 172.21.15.43:0/881189622 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e64066070 con 0x55b74265db60 2024-09-11T23:25:50.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.095+0000 7f6e537fe640 1 -- 172.21.15.43:0/881189622 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e64035040 con 0x55b74265db60 2024-09-11T23:25:50.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.095+0000 7f6e7204c740 1 -- 172.21.15.43:0/881189622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b74265db60 msgr2=0x55b7426643c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:50.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.095+0000 7f6e7204c740 1 --2- 172.21.15.43:0/881189622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b74265db60 0x55b7426643c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6e64005560 tx=0x7f6e64047040 comp rx=0 tx=0).stop 2024-09-11T23:25:50.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.096+0000 7f6e7204c740 1 -- 172.21.15.43:0/881189622 shutdown_connections 2024-09-11T23:25:50.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.096+0000 7f6e7204c740 1 --2- 172.21.15.43:0/881189622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b742668d40 0x55b742680ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.096+0000 7f6e7204c740 1 --2- 172.21.15.43:0/881189622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265e480 0x55b742664900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.096+0000 7f6e7204c740 1 --2- 172.21.15.43:0/881189622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b74265db60 0x55b7426643c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.096+0000 7f6e7204c740 1 -- 172.21.15.43:0/881189622 >> 172.21.15.43:0/881189622 conn(0x55b74265a0f0 msgr2=0x55b742660e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:50.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.096+0000 7f6e7204c740 1 -- 172.21.15.43:0/881189622 shutdown_connections 2024-09-11T23:25:50.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.096+0000 7f6e7204c740 1 -- 172.21.15.43:0/881189622 wait complete. 2024-09-11T23:25:50.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.096+0000 7f6e7204c740 1 -- start start 2024-09-11T23:25:50.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.097+0000 7f6e7204c740 1 Processor -- start 2024-09-11T23:25:50.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.098+0000 7f6e7204c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b74265db60 0x55b742671680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:50.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.098+0000 7f6e7204c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265e480 0x55b742662980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:50.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.098+0000 7f6e7204c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b742668d40 0x55b7426630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:50.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.098+0000 7f6e6da0f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b74265db60 0x55b742671680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:50.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.098+0000 7f6e7204c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6e681ae250 con 0x55b74265e480 2024-09-11T23:25:50.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.098+0000 7f6e7204c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6e681af6c0 con 0x55b74265db60 2024-09-11T23:25:50.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.098+0000 7f6e7204c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6e681ae920 con 0x55b742668d40 2024-09-11T23:25:50.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.098+0000 7f6e6da0f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b74265db60 0x55b742671680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:53790/0 (socket says 172.21.15.43:53790) 2024-09-11T23:25:50.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.098+0000 7f6e6da0f640 1 -- 172.21.15.43:0/3626614645 learned_addr learned my addr 172.21.15.43:0/3626614645 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:50.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.098+0000 7f6e7204c740 10 client.0 _ll_register_callbacks cb 0x7f6e5809bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:50.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.098+0000 7f6e6d20e640 1 --2- 172.21.15.43:0/3626614645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265e480 0x55b742662980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:50.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.098+0000 7f6e6e210640 1 --2- 172.21.15.43:0/3626614645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b742668d40 0x55b7426630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:50.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.098+0000 7f6e6da0f640 1 -- 172.21.15.43:0/3626614645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b742668d40 msgr2=0x55b7426630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:50.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.098+0000 7f6e6da0f640 1 --2- 172.21.15.43:0/3626614645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b742668d40 0x55b7426630c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.098+0000 7f6e6da0f640 1 -- 172.21.15.43:0/3626614645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265e480 msgr2=0x55b742662980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:50.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.098+0000 7f6e6da0f640 1 --2- 172.21.15.43:0/3626614645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265e480 0x55b742662980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.098+0000 7f6e6da0f640 1 -- 172.21.15.43:0/3626614645 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b74265e2b0 con 0x55b74265db60 2024-09-11T23:25:50.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.098+0000 7f6e6d20e640 1 --2- 172.21.15.43:0/3626614645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265e480 0x55b742662980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:50.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.099+0000 7f6e6da0f640 1 --2- 172.21.15.43:0/3626614645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b74265db60 0x55b742671680 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6e64051040 tx=0x7f6e64003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a9059eeabc0bffe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:50.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.099+0000 7f6e517fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:25:50.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.099+0000 7f6e517fa640 1 -- 172.21.15.43:0/3626614645 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6406f060 con 0x55b74265db60 2024-09-11T23:25:50.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.099+0000 7f6e517fa640 1 -- 172.21.15.43:0/3626614645 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f6e64004040 con 0x55b74265db60 2024-09-11T23:25:50.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.099+0000 7f6e7204c740 1 -- 172.21.15.43:0/3626614645 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e64066070 con 0x55b74265db60 2024-09-11T23:25:50.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.100+0000 7f6e517fa640 1 -- 172.21.15.43:0/3626614645 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6e64040030 con 0x55b74265db60 2024-09-11T23:25:50.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.100+0000 7f6e7204c740 10 client.25613 fetch_fsmap learned FSMap version 30 2024-09-11T23:25:50.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.100+0000 7f6e7204c740 1 -- 172.21.15.43:0/3626614645 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b7426ec2c0 con 0x55b74265db60 2024-09-11T23:25:50.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.100+0000 7f6e517fa640 1 -- 172.21.15.43:0/3626614645 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e64066070 con 0x55b74265db60 2024-09-11T23:25:50.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.100+0000 7f6e517fa640 1 -- 172.21.15.43:0/3626614645 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f6e6404e070 con 0x55b74265db60 2024-09-11T23:25:50.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.100+0000 7f6e7204c740 10 client.25613 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:25:50.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.100+0000 7f6e7204c740 10 client.25613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:25:50.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.100+0000 7f6e7204c740 20 client.25613 populate_metadata read hostname 'smithi043' 2024-09-11T23:25:50.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.100+0000 7f6e7204c740 1 --2- 172.21.15.43:0/3626614645 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b7426ec4f0 0x55b7426ee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:50.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.100+0000 7f6e7204c740 4 client.25613 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:25:50.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.100+0000 7f6e7204c740 1 -- 172.21.15.43:0/3626614645 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b7425cad00 con 0x55b7426ec4f0 2024-09-11T23:25:50.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.100+0000 7f6e6d20e640 1 --2- 172.21.15.43:0/3626614645 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b7426ec4f0 0x55b7426ee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:50.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.101+0000 7f6e517fa640 10 client.25613 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:25:50.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.101+0000 7f6e6d20e640 1 --2- 172.21.15.43:0/3626614645 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b7426ec4f0 0x55b7426ee8e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4194235442e77c25 server_cookie=55a40178ae2ab66c in_seq=0 out_seq=0 2024-09-11T23:25:50.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.102+0000 7f6e517fa640 1 -- 172.21.15.43:0/3626614645 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b7425cad00 con 0x55b7426ec4f0 2024-09-11T23:25:50.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.102+0000 7f6e517fa640 10 client.25613 handle_command_reply: tid=0 2024-09-11T23:25:50.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.102+0000 7f6e7204c740 1 client.25613 shutdown 2024-09-11T23:25:50.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.103+0000 7f6e7204c740 1 -- 172.21.15.43:0/3626614645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b74265db60 msgr2=0x55b742671680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:50.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.103+0000 7f6e7204c740 1 --2- 172.21.15.43:0/3626614645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b74265db60 0x55b742671680 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6e64051040 tx=0x7f6e64003820 comp rx=0 tx=0).stop 2024-09-11T23:25:50.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.103+0000 7f6e7204c740 1 -- 172.21.15.43:0/3626614645 shutdown_connections 2024-09-11T23:25:50.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.103+0000 7f6e7204c740 1 --2- 172.21.15.43:0/3626614645 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b7426ec4f0 0x55b7426ee8e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.103+0000 7f6e517fa640 0 client.25613 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:25:50.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.103+0000 7f6e7204c740 1 --2- 172.21.15.43:0/3626614645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b742668d40 0x55b7426630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.103+0000 7f6e7204c740 1 --2- 172.21.15.43:0/3626614645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b74265e480 0x55b742662980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.103+0000 7f6e7204c740 1 --2- 172.21.15.43:0/3626614645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b74265db60 0x55b742671680 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.103+0000 7f6e7204c740 1 -- 172.21.15.43:0/3626614645 >> 172.21.15.43:0/3626614645 conn(0x55b74265a0f0 msgr2=0x55b742664380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:50.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.103+0000 7f6e7204c740 1 -- 172.21.15.43:0/3626614645 shutdown_connections 2024-09-11T23:25:50.857 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:25:50.857 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-11T23:25:50.857 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "ddf9b449-9095-42dd-a28c-7f5af91a6fc0", 2024-09-11T23:25:50.857 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-11T23:25:50.857 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:25:50.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.103+0000 7f6e7204c740 1 -- 172.21.15.43:0/3626614645 wait complete. 2024-09-11T23:25:50.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.103+0000 7f6e7204c740 20 client.25613 trim_cache size 0 max 16384 2024-09-11T23:25:50.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.104+0000 7f6e6fc73640 1 -- 172.21.15.43:0/1330074008 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6e540a2d70 msgr2=0x7f6e540a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:50.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.104+0000 7f6e6fc73640 1 --2- 172.21.15.43:0/1330074008 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6e540a2d70 0x7f6e540a5230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009fd0 tx=0x7f6e60009500 comp rx=0 tx=0).stop 2024-09-11T23:25:50.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.104+0000 7f6e6fc73640 1 -- 172.21.15.43:0/1330074008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e68101380 msgr2=0x7f6e6819ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:50.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.104+0000 7f6e6fc73640 1 --2- 172.21.15.43:0/1330074008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e68101380 0x7f6e6819ac90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800e9f0 tx=0x7f6e5800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.104+0000 7f6e6fc73640 1 -- 172.21.15.43:0/1330074008 shutdown_connections 2024-09-11T23:25:50.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.104+0000 7f6e6fc73640 1 --2- 172.21.15.43:0/1330074008 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6e540a2d70 0x7f6e540a5230 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.104+0000 7f6e6fc73640 1 --2- 172.21.15.43:0/1330074008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e6810e4f0 0x7f6e6819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.104+0000 7f6e6fc73640 1 --2- 172.21.15.43:0/1330074008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e68101380 0x7f6e6819ac90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.104+0000 7f6e6fc73640 1 --2- 172.21.15.43:0/1330074008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e681009d0 0x7f6e6819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:50.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.104+0000 7f6e6fc73640 1 -- 172.21.15.43:0/1330074008 >> 172.21.15.43:0/1330074008 conn(0x7f6e680fc7d0 msgr2=0x7f6e680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:50.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.104+0000 7f6e6fc73640 1 -- 172.21.15.43:0/1330074008 shutdown_connections 2024-09-11T23:25:50.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:25:50.104+0000 7f6e6fc73640 1 -- 172.21.15.43:0/1330074008 wait complete. 2024-09-11T23:25:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.122+0000 7f3944a5e640 1 -- 172.21.15.43:0/204146613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39401009a0 msgr2=0x7f3940100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.122+0000 7f3944a5e640 1 --2- 172.21.15.43:0/204146613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39401009a0 0x7f3940100e00 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3934009930 tx=0x7f393402a420 comp rx=0 tx=0).stop 2024-09-11T23:25:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.123+0000 7f3944a5e640 1 -- 172.21.15.43:0/204146613 shutdown_connections 2024-09-11T23:25:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.123+0000 7f3944a5e640 1 --2- 172.21.15.43:0/204146613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3940101340 0x7f394010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.123+0000 7f3944a5e640 1 --2- 172.21.15.43:0/204146613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39401009a0 0x7f3940100e00 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.123+0000 7f3944a5e640 1 --2- 172.21.15.43:0/204146613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39401069b0 0x7f3940106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.123+0000 7f3944a5e640 1 -- 172.21.15.43:0/204146613 >> 172.21.15.43:0/204146613 conn(0x7f39400779a0 msgr2=0x7f39400fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.123+0000 7f3944a5e640 1 -- 172.21.15.43:0/204146613 shutdown_connections 2024-09-11T23:25:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.123+0000 7f3944a5e640 1 -- 172.21.15.43:0/204146613 wait complete. 2024-09-11T23:25:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.123+0000 7f3944a5e640 1 Processor -- start 2024-09-11T23:25:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.123+0000 7f3944a5e640 1 -- start start 2024-09-11T23:25:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.124+0000 7f3944a5e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39401009a0 0x7f39401972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.124+0000 7f3944a5e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3940101340 0x7f3940197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.124+0000 7f3944a5e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39401069b0 0x7f394019bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.124+0000 7f3944a5e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3940110dc0 con 0x7f3940101340 2024-09-11T23:25:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.124+0000 7f3944a5e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3940110c40 con 0x7f39401009a0 2024-09-11T23:25:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.124+0000 7f3944a5e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3940110f40 con 0x7f39401069b0 2024-09-11T23:25:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.124+0000 7f393dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3940101340 0x7f3940197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.124+0000 7f393dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3940101340 0x7f3940197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51556/0 (socket says 172.21.15.43:51556) 2024-09-11T23:25:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.124+0000 7f393dd9b640 1 -- 172.21.15.43:0/1463840281 learned_addr learned my addr 172.21.15.43:0/1463840281 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.124+0000 7f393e59c640 1 --2- 172.21.15.43:0/1463840281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39401009a0 0x7f39401972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.124+0000 7f393ed9d640 1 --2- 172.21.15.43:0/1463840281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39401069b0 0x7f394019bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.124+0000 7f393dd9b640 1 -- 172.21.15.43:0/1463840281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39401069b0 msgr2=0x7f394019bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.124+0000 7f393dd9b640 1 --2- 172.21.15.43:0/1463840281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39401069b0 0x7f394019bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.124+0000 7f393dd9b640 1 -- 172.21.15.43:0/1463840281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39401009a0 msgr2=0x7f39401972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.124+0000 7f393dd9b640 1 --2- 172.21.15.43:0/1463840281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39401009a0 0x7f39401972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.124+0000 7f393dd9b640 1 -- 172.21.15.43:0/1463840281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394019c2e0 con 0x7f3940101340 2024-09-11T23:25:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.124+0000 7f393dd9b640 1 --2- 172.21.15.43:0/1463840281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3940101340 0x7f3940197830 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3934009230 tx=0x7f3934004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.124+0000 7f392f7fe640 1 -- 172.21.15.43:0/1463840281 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393403f030 con 0x7f3940101340 2024-09-11T23:25:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.125+0000 7f392f7fe640 1 -- 172.21.15.43:0/1463840281 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3934003ad0 con 0x7f3940101340 2024-09-11T23:25:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.125+0000 7f3944a5e640 1 -- 172.21.15.43:0/1463840281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394019c570 con 0x7f3940101340 2024-09-11T23:25:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.125+0000 7f392f7fe640 1 -- 172.21.15.43:0/1463840281 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3934035070 con 0x7f3940101340 2024-09-11T23:25:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.125+0000 7f3944a5e640 1 -- 172.21.15.43:0/1463840281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39401a3e60 con 0x7f3940101340 2024-09-11T23:25:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.127+0000 7f392f7fe640 1 -- 172.21.15.43:0/1463840281 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3934042030 con 0x7f3940101340 2024-09-11T23:25:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.127+0000 7f392f7fe640 1 --2- 172.21.15.43:0/1463840281 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f39280a2d70 0x7f39280a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:51.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.127+0000 7f393e59c640 1 --2- 172.21.15.43:0/1463840281 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f39280a2d70 0x7f39280a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:51.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.128+0000 7f392f7fe640 1 -- 172.21.15.43:0/1463840281 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f39340ef4a0 con 0x7f3940101340 2024-09-11T23:25:51.266 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.128+0000 7f393e59c640 1 --2- 172.21.15.43:0/1463840281 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f39280a2d70 0x7f39280a5230 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f393000bb60 tx=0x7f393000b390 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.147+0000 7f39477fa740 1 Processor -- start 2024-09-11T23:25:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.148+0000 7f39477fa740 1 -- start start 2024-09-11T23:25:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.148+0000 7f39477fa740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f797a10970 0x55f797a10d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.148+0000 7f39477fa740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f797a11290 0x55f797a1d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.148+0000 7f39477fa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a1dec0 0x55f797a202b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.148+0000 7f39477fa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f7979752f0 con 0x55f797a1dec0 2024-09-11T23:25:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.148+0000 7f39477fa740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f79791d270 con 0x55f797a10970 2024-09-11T23:25:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.148+0000 7f39477fa740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f7979712b0 con 0x55f797a11290 2024-09-11T23:25:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.148+0000 7f393ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a1dec0 0x55f797a202b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.148+0000 7f393ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a1dec0 0x55f797a202b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51572/0 (socket says 172.21.15.43:51572) 2024-09-11T23:25:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.148+0000 7f393ed9d640 1 -- 172.21.15.43:0/1693668432 learned_addr learned my addr 172.21.15.43:0/1693668432 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.148+0000 7f393dd9b640 1 --2- 172.21.15.43:0/1693668432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f797a11290 0x55f797a1d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.148+0000 7f393e59c640 1 --2- 172.21.15.43:0/1693668432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f797a10970 0x55f797a10d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.148+0000 7f393ed9d640 1 -- 172.21.15.43:0/1693668432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f797a11290 msgr2=0x55f797a1d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.148+0000 7f393ed9d640 1 --2- 172.21.15.43:0/1693668432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f797a11290 0x55f797a1d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.148+0000 7f393ed9d640 1 -- 172.21.15.43:0/1693668432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f797a10970 msgr2=0x55f797a10d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.148+0000 7f393ed9d640 1 --2- 172.21.15.43:0/1693668432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f797a10970 0x55f797a10d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.148+0000 7f393ed9d640 1 -- 172.21.15.43:0/1693668432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f79791c340 con 0x55f797a1dec0 2024-09-11T23:25:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.148+0000 7f393dd9b640 1 --2- 172.21.15.43:0/1693668432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f797a11290 0x55f797a1d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:25:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.149+0000 7f393ed9d640 1 --2- 172.21.15.43:0/1693668432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a1dec0 0x55f797a202b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3938010370 tx=0x7f3938010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113f6ba170691487 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.149+0000 7f392cff9640 1 -- 172.21.15.43:0/1693668432 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938016070 con 0x55f797a1dec0 2024-09-11T23:25:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.149+0000 7f392cff9640 1 -- 172.21.15.43:0/1693668432 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3938004d20 con 0x55f797a1dec0 2024-09-11T23:25:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.149+0000 7f392cff9640 1 -- 172.21.15.43:0/1693668432 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938005420 con 0x55f797a1dec0 2024-09-11T23:25:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.149+0000 7f39477fa740 1 -- 172.21.15.43:0/1693668432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a1dec0 msgr2=0x55f797a202b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.149+0000 7f39477fa740 1 --2- 172.21.15.43:0/1693668432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a1dec0 0x55f797a202b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3938010370 tx=0x7f3938010830 comp rx=0 tx=0).stop 2024-09-11T23:25:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.149+0000 7f39477fa740 1 -- 172.21.15.43:0/1693668432 shutdown_connections 2024-09-11T23:25:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.149+0000 7f39477fa740 1 --2- 172.21.15.43:0/1693668432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a1dec0 0x55f797a202b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.149+0000 7f39477fa740 1 --2- 172.21.15.43:0/1693668432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f797a11290 0x55f797a1d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.149+0000 7f39477fa740 1 --2- 172.21.15.43:0/1693668432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f797a10970 0x55f797a10d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.149+0000 7f39477fa740 1 -- 172.21.15.43:0/1693668432 >> 172.21.15.43:0/1693668432 conn(0x55f797a0d060 msgr2=0x55f797a0d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.150+0000 7f39477fa740 1 -- 172.21.15.43:0/1693668432 shutdown_connections 2024-09-11T23:25:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.150+0000 7f39477fa740 1 -- 172.21.15.43:0/1693668432 wait complete. 2024-09-11T23:25:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.150+0000 7f39477fa740 1 -- start start 2024-09-11T23:25:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.150+0000 7f39477fa740 1 Processor -- start 2024-09-11T23:25:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.150+0000 7f39477fa740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f797a10970 0x55f797a1adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.150+0000 7f39477fa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a11290 0x55f797a1b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.150+0000 7f39477fa740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f797a1dec0 0x55f797a1b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.150+0000 7f39477fa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f797a20af0 con 0x55f797a11290 2024-09-11T23:25:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.150+0000 7f39477fa740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f7978fe290 con 0x55f797a10970 2024-09-11T23:25:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.150+0000 7f39477fa740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f797a20c70 con 0x55f797a1dec0 2024-09-11T23:25:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.150+0000 7f39477fa740 10 client.0 _ll_register_callbacks cb 0x55f79792f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.150+0000 7f393dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a11290 0x55f797a1b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.150+0000 7f393dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a11290 0x55f797a1b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51576/0 (socket says 172.21.15.43:51576) 2024-09-11T23:25:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.150+0000 7f393dd9b640 1 -- 172.21.15.43:0/1448353634 learned_addr learned my addr 172.21.15.43:0/1448353634 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.150+0000 7f393ed9d640 1 --2- 172.21.15.43:0/1448353634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f797a1dec0 0x55f797a1b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.150+0000 7f393e59c640 1 --2- 172.21.15.43:0/1448353634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f797a10970 0x55f797a1adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.151+0000 7f393dd9b640 1 -- 172.21.15.43:0/1448353634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f797a1dec0 msgr2=0x55f797a1b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.151+0000 7f393dd9b640 1 --2- 172.21.15.43:0/1448353634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f797a1dec0 0x55f797a1b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.151+0000 7f393dd9b640 1 -- 172.21.15.43:0/1448353634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f797a10970 msgr2=0x55f797a1adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.151+0000 7f393dd9b640 1 --2- 172.21.15.43:0/1448353634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f797a10970 0x55f797a1adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.151+0000 7f393dd9b640 1 -- 172.21.15.43:0/1448353634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f797688210 con 0x55f797a11290 2024-09-11T23:25:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.151+0000 7f393dd9b640 1 --2- 172.21.15.43:0/1448353634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a11290 0x55f797a1b330 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f39340ecc50 tx=0x7f39340bfe30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700065b19428f4f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.151+0000 7f3925ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:25:51.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.151+0000 7f3925ffb640 1 -- 172.21.15.43:0/1448353634 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39340d4060 con 0x55f797a11290 2024-09-11T23:25:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.151+0000 7f3925ffb640 1 -- 172.21.15.43:0/1448353634 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f39340b80e0 con 0x55f797a11290 2024-09-11T23:25:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.151+0000 7f39477fa740 1 -- 172.21.15.43:0/1448353634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3938004d20 con 0x55f797a11290 2024-09-11T23:25:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.151+0000 7f3925ffb640 1 -- 172.21.15.43:0/1448353634 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3934049040 con 0x55f797a11290 2024-09-11T23:25:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.151+0000 7f39477fa740 10 client.16086 fetch_fsmap learned FSMap version 30 2024-09-11T23:25:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.151+0000 7f39477fa740 1 -- 172.21.15.43:0/1448353634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f797754620 con 0x55f797a11290 2024-09-11T23:25:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.151+0000 7f3925ffb640 1 -- 172.21.15.43:0/1448353634 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3938004d20 con 0x55f797a11290 2024-09-11T23:25:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.151+0000 7f3925ffb640 1 -- 172.21.15.43:0/1448353634 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f39340c41a0 con 0x55f797a11290 2024-09-11T23:25:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.152+0000 7f39477fa740 10 client.16086 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:25:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.152+0000 7f39477fa740 10 client.16086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:25:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.152+0000 7f39477fa740 20 client.16086 populate_metadata read hostname 'smithi043' 2024-09-11T23:25:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.152+0000 7f39477fa740 1 --2- 172.21.15.43:0/1448353634 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f797a2de40 0x55f797a30260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.152+0000 7f39477fa740 4 client.16086 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:25:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.152+0000 7f39477fa740 1 -- 172.21.15.43:0/1448353634 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f797982a20 con 0x55f797a2de40 2024-09-11T23:25:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.152+0000 7f393e59c640 1 --2- 172.21.15.43:0/1448353634 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f797a2de40 0x55f797a30260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.152+0000 7f3925ffb640 10 client.16086 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:25:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.152+0000 7f393e59c640 1 --2- 172.21.15.43:0/1448353634 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f797a2de40 0x55f797a30260 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96cacfdd5f00d4bf server_cookie=5c8311ccd00071c0 in_seq=0 out_seq=0 2024-09-11T23:25:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.153+0000 7f3925ffb640 1 -- 172.21.15.43:0/1448353634 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f797982a20 con 0x55f797a2de40 2024-09-11T23:25:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.153+0000 7f3925ffb640 10 client.16086 handle_command_reply: tid=0 2024-09-11T23:25:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.153+0000 7f39477fa740 1 client.16086 shutdown 2024-09-11T23:25:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.153+0000 7f39477fa740 1 -- 172.21.15.43:0/1448353634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a11290 msgr2=0x55f797a1b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.153+0000 7f39477fa740 1 --2- 172.21.15.43:0/1448353634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a11290 0x55f797a1b330 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f39340ecc50 tx=0x7f39340bfe30 comp rx=0 tx=0).stop 2024-09-11T23:25:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.153+0000 7f39477fa740 1 -- 172.21.15.43:0/1448353634 shutdown_connections 2024-09-11T23:25:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.153+0000 7f39477fa740 1 --2- 172.21.15.43:0/1448353634 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f797a2de40 0x55f797a30260 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.153+0000 7f39477fa740 1 --2- 172.21.15.43:0/1448353634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f797a1dec0 0x55f797a1b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.153+0000 7f3925ffb640 0 client.16086 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:25:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.153+0000 7f39477fa740 1 --2- 172.21.15.43:0/1448353634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a11290 0x55f797a1b330 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.153+0000 7f39477fa740 1 --2- 172.21.15.43:0/1448353634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f797a10970 0x55f797a1adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.153+0000 7f39477fa740 1 -- 172.21.15.43:0/1448353634 >> 172.21.15.43:0/1448353634 conn(0x55f797a0d060 msgr2=0x55f797a12dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.153+0000 7f39477fa740 1 -- 172.21.15.43:0/1448353634 shutdown_connections 2024-09-11T23:25:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.153+0000 7f39477fa740 1 -- 172.21.15.43:0/1448353634 wait complete. 2024-09-11T23:25:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.153+0000 7f39477fa740 20 client.16086 trim_cache size 0 max 16384 2024-09-11T23:25:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.168+0000 7f39477fa740 1 Processor -- start 2024-09-11T23:25:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.173+0000 7f39477fa740 1 -- start start 2024-09-11T23:25:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.173+0000 7f39477fa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a10970 0x55f797a35b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.173+0000 7f39477fa740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f797a11290 0x55f797a360b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.173+0000 7f39477fa740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f797a1dec0 0x55f797a18070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.173+0000 7f39477fa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f39401adee0 con 0x55f797a10970 2024-09-11T23:25:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.173+0000 7f39477fa740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f39401ab7b0 con 0x55f797a1dec0 2024-09-11T23:25:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.173+0000 7f39477fa740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f39401a9600 con 0x55f797a11290 2024-09-11T23:25:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.173+0000 7f393e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a10970 0x55f797a35b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.173+0000 7f393e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a10970 0x55f797a35b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51578/0 (socket says 172.21.15.43:51578) 2024-09-11T23:25:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.173+0000 7f393e59c640 1 -- 172.21.15.43:0/1535158772 learned_addr learned my addr 172.21.15.43:0/1535158772 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.174+0000 7f393e59c640 1 -- 172.21.15.43:0/1535158772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f797a11290 msgr2=0x55f797a360b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:25:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.174+0000 7f393e59c640 1 --2- 172.21.15.43:0/1535158772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f797a11290 0x55f797a360b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.174+0000 7f393e59c640 1 -- 172.21.15.43:0/1535158772 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f797a1dec0 msgr2=0x55f797a18070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:25:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.174+0000 7f393e59c640 1 --2- 172.21.15.43:0/1535158772 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f797a1dec0 0x55f797a18070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.174+0000 7f393e59c640 1 -- 172.21.15.43:0/1535158772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f797a308e0 con 0x55f797a10970 2024-09-11T23:25:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.174+0000 7f393e59c640 1 --2- 172.21.15.43:0/1535158772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a10970 0x55f797a35b70 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x55f797a30ae0 tx=0x7f3930013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc272792eccb9a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.175+0000 7f3927fff640 1 -- 172.21.15.43:0/1535158772 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393001eaf0 con 0x55f797a10970 2024-09-11T23:25:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.175+0000 7f3927fff640 1 -- 172.21.15.43:0/1535158772 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3930011e10 con 0x55f797a10970 2024-09-11T23:25:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.175+0000 7f3927fff640 1 -- 172.21.15.43:0/1535158772 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930029600 con 0x55f797a10970 2024-09-11T23:25:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.177+0000 7f39477fa740 1 -- 172.21.15.43:0/1535158772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a10970 msgr2=0x55f797a35b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.177+0000 7f39477fa740 1 --2- 172.21.15.43:0/1535158772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a10970 0x55f797a35b70 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x55f797a30ae0 tx=0x7f3930013000 comp rx=0 tx=0).stop 2024-09-11T23:25:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.179+0000 7f39477fa740 1 -- 172.21.15.43:0/1535158772 shutdown_connections 2024-09-11T23:25:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.179+0000 7f39477fa740 1 --2- 172.21.15.43:0/1535158772 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f797a1dec0 0x55f797a18070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.179+0000 7f39477fa740 1 --2- 172.21.15.43:0/1535158772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f797a11290 0x55f797a360b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.179+0000 7f39477fa740 1 --2- 172.21.15.43:0/1535158772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a10970 0x55f797a35b70 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.179+0000 7f39477fa740 1 -- 172.21.15.43:0/1535158772 >> 172.21.15.43:0/1535158772 conn(0x55f797a0d060 msgr2=0x55f797a1ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.181+0000 7f39477fa740 1 -- 172.21.15.43:0/1535158772 shutdown_connections 2024-09-11T23:25:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.186+0000 7f39477fa740 1 -- 172.21.15.43:0/1535158772 wait complete. 2024-09-11T23:25:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.186+0000 7f39477fa740 1 -- start start 2024-09-11T23:25:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.186+0000 7f39477fa740 1 Processor -- start 2024-09-11T23:25:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.187+0000 7f39477fa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a10970 0x55f797a94620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.187+0000 7f39477fa740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f797a11290 0x55f797a25ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.187+0000 7f39477fa740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f797a1dec0 0x55f797a26420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.187+0000 7f39477fa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f39401a9c70 con 0x55f797a10970 2024-09-11T23:25:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.187+0000 7f39477fa740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f39401ab0e0 con 0x55f797a1dec0 2024-09-11T23:25:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.187+0000 7f39477fa740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f39401aa340 con 0x55f797a11290 2024-09-11T23:25:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.187+0000 7f39477fa740 10 client.0 _ll_register_callbacks cb 0x7f39340bb8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:25:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.187+0000 7f393ed9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f797a1dec0 0x55f797a26420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.189+0000 7f393ed9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f797a1dec0 0x55f797a26420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:48640/0 (socket says 172.21.15.43:48640) 2024-09-11T23:25:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.189+0000 7f393ed9d640 1 -- 172.21.15.43:0/4041486246 learned_addr learned my addr 172.21.15.43:0/4041486246 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:25:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.189+0000 7f393dd9b640 1 --2- 172.21.15.43:0/4041486246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f797a11290 0x55f797a25ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.191+0000 7f393ed9d640 1 -- 172.21.15.43:0/4041486246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f797a11290 msgr2=0x55f797a25ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.191+0000 7f393ed9d640 1 --2- 172.21.15.43:0/4041486246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f797a11290 0x55f797a25ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.191+0000 7f393ed9d640 1 -- 172.21.15.43:0/4041486246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a10970 msgr2=0x55f797a94620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:25:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.191+0000 7f393ed9d640 1 --2- 172.21.15.43:0/4041486246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a10970 0x55f797a94620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.191+0000 7f393ed9d640 1 -- 172.21.15.43:0/4041486246 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f797a26ac0 con 0x55f797a1dec0 2024-09-11T23:25:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.192+0000 7f393ed9d640 1 --2- 172.21.15.43:0/4041486246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f797a1dec0 0x55f797a26420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55f797a0e150 tx=0x7f3938003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=206f504d564541d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:25:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.192+0000 7f3925ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:25:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.193+0000 7f3925ffb640 1 -- 172.21.15.43:0/4041486246 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938016070 con 0x55f797a1dec0 2024-09-11T23:25:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.193+0000 7f39477fa740 1 -- 172.21.15.43:0/4041486246 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3930011e10 con 0x55f797a1dec0 2024-09-11T23:25:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.193+0000 7f3925ffb640 1 -- 172.21.15.43:0/4041486246 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f39380052c0 con 0x55f797a1dec0 2024-09-11T23:25:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.193+0000 7f3925ffb640 1 -- 172.21.15.43:0/4041486246 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f39380054c0 con 0x55f797a1dec0 2024-09-11T23:25:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.194+0000 7f39477fa740 10 client.25628 fetch_fsmap learned FSMap version 30 2024-09-11T23:25:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.194+0000 7f39477fa740 1 -- 172.21.15.43:0/4041486246 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f797ab0740 con 0x55f797a1dec0 2024-09-11T23:25:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.198+0000 7f3925ffb640 1 -- 172.21.15.43:0/4041486246 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3930011e10 con 0x55f797a1dec0 2024-09-11T23:25:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.198+0000 7f3925ffb640 1 -- 172.21.15.43:0/4041486246 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f393800f040 con 0x55f797a1dec0 2024-09-11T23:25:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.201+0000 7f39477fa740 10 client.25628 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:25:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.201+0000 7f39477fa740 10 client.25628 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:25:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.201+0000 7f39477fa740 20 client.25628 populate_metadata read hostname 'smithi043' 2024-09-11T23:25:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.201+0000 7f39477fa740 1 --2- 172.21.15.43:0/4041486246 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f797ab0cd0 0x55f797ab30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:25:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.201+0000 7f39477fa740 4 client.25628 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:25:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.201+0000 7f39477fa740 1 -- 172.21.15.43:0/4041486246 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f79797cd70 con 0x55f797ab0cd0 2024-09-11T23:25:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.206+0000 7f393e59c640 1 --2- 172.21.15.43:0/4041486246 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f797ab0cd0 0x55f797ab30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:25:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.214+0000 7f393e59c640 1 --2- 172.21.15.43:0/4041486246 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f797ab0cd0 0x55f797ab30c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c240773ca3a9c4bb server_cookie=dbf13a1c847ad3d5 in_seq=0 out_seq=0 2024-09-11T23:25:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.215+0000 7f3925ffb640 10 client.25628 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:25:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.756+0000 7f3925ffb640 1 -- 172.21.15.43:0/4041486246 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f79797cd70 con 0x55f797ab0cd0 2024-09-11T23:25:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.756+0000 7f3925ffb640 10 client.25628 handle_command_reply: tid=0 2024-09-11T23:25:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.756+0000 7f39477fa740 1 client.25628 shutdown 2024-09-11T23:25:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.756+0000 7f39477fa740 1 -- 172.21.15.43:0/4041486246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f797a1dec0 msgr2=0x55f797a26420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.756+0000 7f39477fa740 1 --2- 172.21.15.43:0/4041486246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f797a1dec0 0x55f797a26420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55f797a0e150 tx=0x7f3938003ba0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.756+0000 7f39477fa740 1 -- 172.21.15.43:0/4041486246 shutdown_connections 2024-09-11T23:25:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.756+0000 7f39477fa740 1 --2- 172.21.15.43:0/4041486246 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f797ab0cd0 0x55f797ab30c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.756+0000 7f39477fa740 1 --2- 172.21.15.43:0/4041486246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f797a1dec0 0x55f797a26420 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.756+0000 7f39477fa740 1 --2- 172.21.15.43:0/4041486246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f797a11290 0x55f797a25ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.756+0000 7f39477fa740 1 --2- 172.21.15.43:0/4041486246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f797a10970 0x55f797a94620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.756+0000 7f3925ffb640 0 client.25628 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:25:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.756+0000 7f39477fa740 1 -- 172.21.15.43:0/4041486246 >> 172.21.15.43:0/4041486246 conn(0x55f797a0d060 msgr2=0x55f797a1adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.756+0000 7f39477fa740 1 -- 172.21.15.43:0/4041486246 shutdown_connections 2024-09-11T23:25:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.756+0000 7f39477fa740 1 -- 172.21.15.43:0/4041486246 wait complete. 2024-09-11T23:25:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.756+0000 7f39477fa740 20 client.25628 trim_cache size 0 max 16384 2024-09-11T23:25:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:25:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:25:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:25:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:25:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.757+0000 7f3944a5e640 1 -- 172.21.15.43:0/1463840281 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f39280a2d70 msgr2=0x7f39280a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.757+0000 7f3944a5e640 1 --2- 172.21.15.43:0/1463840281 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f39280a2d70 0x7f39280a5230 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f393000bb60 tx=0x7f393000b390 comp rx=0 tx=0).stop 2024-09-11T23:25:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.757+0000 7f3944a5e640 1 -- 172.21.15.43:0/1463840281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3940101340 msgr2=0x7f3940197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:25:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.757+0000 7f3944a5e640 1 --2- 172.21.15.43:0/1463840281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3940101340 0x7f3940197830 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3934009230 tx=0x7f3934004270 comp rx=0 tx=0).stop 2024-09-11T23:25:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.757+0000 7f3944a5e640 1 -- 172.21.15.43:0/1463840281 shutdown_connections 2024-09-11T23:25:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.757+0000 7f3944a5e640 1 --2- 172.21.15.43:0/1463840281 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f39280a2d70 0x7f39280a5230 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.757+0000 7f3944a5e640 1 --2- 172.21.15.43:0/1463840281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39401069b0 0x7f394019bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.757+0000 7f3944a5e640 1 --2- 172.21.15.43:0/1463840281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3940101340 0x7f3940197830 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.757+0000 7f3944a5e640 1 --2- 172.21.15.43:0/1463840281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39401009a0 0x7f39401972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:25:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.757+0000 7f3944a5e640 1 -- 172.21.15.43:0/1463840281 >> 172.21.15.43:0/1463840281 conn(0x7f39400779a0 msgr2=0x7f3940103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:25:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.757+0000 7f3944a5e640 1 -- 172.21.15.43:0/1463840281 shutdown_connections 2024-09-11T23:25:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:25:51.757+0000 7f3944a5e640 1 -- 172.21.15.43:0/1463840281 wait complete. 2024-09-11T23:25:51.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:51 smithi090 ceph-mon[35130]: scrub queued for path: / 2024-09-11T23:25:51.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:51 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:25:51.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:51 smithi090 ceph-mon[35130]: scrub summary: active paths [/] 2024-09-11T23:25:51.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:51 smithi090 ceph-mon[35130]: scrub summary: active paths [/] 2024-09-11T23:25:51.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:51 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:25:51.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:51 smithi090 ceph-mon[35130]: pgmap v124: 97 pgs: 97 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 38 MiB/s wr, 5.60k op/s 2024-09-11T23:25:51.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:51 smithi097 ceph-mon[35562]: scrub queued for path: / 2024-09-11T23:25:51.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:51 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:25:51.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:51 smithi097 ceph-mon[35562]: scrub summary: active paths [/] 2024-09-11T23:25:51.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:51 smithi097 ceph-mon[35562]: scrub summary: active paths [/] 2024-09-11T23:25:51.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:51 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:25:51.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:51 smithi097 ceph-mon[35562]: pgmap v124: 97 pgs: 97 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 38 MiB/s wr, 5.60k op/s 2024-09-11T23:25:51.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:51 smithi043 ceph-mon[30950]: scrub queued for path: / 2024-09-11T23:25:51.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:51 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:25:51.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:51 smithi043 ceph-mon[30950]: scrub summary: active paths [/] 2024-09-11T23:25:51.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:51 smithi043 ceph-mon[30950]: scrub summary: active paths [/] 2024-09-11T23:25:51.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:51 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:25:51.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:51 smithi043 ceph-mon[30950]: pgmap v124: 97 pgs: 97 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 38 MiB/s wr, 5.60k op/s 2024-09-11T23:25:53.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:53 smithi090 ceph-mon[35130]: pgmap v125: 97 pgs: 97 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 44 MiB/s wr, 6.55k op/s 2024-09-11T23:25:53.787 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:53 smithi043 ceph-mon[30950]: pgmap v125: 97 pgs: 97 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 44 MiB/s wr, 6.55k op/s 2024-09-11T23:25:53.800 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:53 smithi097 ceph-mon[35562]: pgmap v125: 97 pgs: 97 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 44 MiB/s wr, 6.55k op/s 2024-09-11T23:25:55.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:55 smithi090 ceph-mon[35130]: pgmap v126: 97 pgs: 97 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 35 MiB/s wr, 5.14k op/s 2024-09-11T23:25:55.800 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:55 smithi097 ceph-mon[35562]: pgmap v126: 97 pgs: 97 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 35 MiB/s wr, 5.14k op/s 2024-09-11T23:25:55.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:55 smithi043 ceph-mon[30950]: pgmap v126: 97 pgs: 97 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 35 MiB/s wr, 5.14k op/s 2024-09-11T23:25:57.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:57 smithi090 ceph-mon[35130]: pgmap v127: 97 pgs: 97 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 31 MiB/s wr, 4.52k op/s 2024-09-11T23:25:57.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:57 smithi097 ceph-mon[35562]: pgmap v127: 97 pgs: 97 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 31 MiB/s wr, 4.52k op/s 2024-09-11T23:25:57.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:57 smithi043 ceph-mon[30950]: pgmap v127: 97 pgs: 97 active+clean; 16 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 31 MiB/s wr, 4.52k op/s 2024-09-11T23:25:59.465 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:25:59.510 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:25:59.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:25:59 smithi090 ceph-mon[35130]: pgmap v128: 97 pgs: 97 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 40 MiB/s wr, 5.63k op/s 2024-09-11T23:25:59.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:25:59 smithi097 ceph-mon[35562]: pgmap v128: 97 pgs: 97 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 40 MiB/s wr, 5.63k op/s 2024-09-11T23:25:59.863 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:25:59 smithi043 ceph-mon[30950]: pgmap v128: 97 pgs: 97 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 40 MiB/s wr, 5.63k op/s 2024-09-11T23:26:01.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:01 smithi090 ceph-mon[35130]: pgmap v129: 97 pgs: 97 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 31 MiB/s wr, 4.25k op/s 2024-09-11T23:26:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:01 smithi097 ceph-mon[35562]: pgmap v129: 97 pgs: 97 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 31 MiB/s wr, 4.25k op/s 2024-09-11T23:26:01.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:01 smithi043 ceph-mon[30950]: pgmap v129: 97 pgs: 97 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 31 MiB/s wr, 4.25k op/s 2024-09-11T23:26:03.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:03 smithi090 ceph-mon[35130]: pgmap v130: 97 pgs: 97 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 35 MiB/s wr, 4.73k op/s 2024-09-11T23:26:03.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:03 smithi097 ceph-mon[35562]: pgmap v130: 97 pgs: 97 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 35 MiB/s wr, 4.73k op/s 2024-09-11T23:26:03.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:03 smithi043 ceph-mon[30950]: pgmap v130: 97 pgs: 97 active+clean; 16 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 35 MiB/s wr, 4.73k op/s 2024-09-11T23:26:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.871+0000 7f04538c9640 1 -- 172.21.15.43:0/2986179625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f044c106a00 msgr2=0x7f044c106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.871+0000 7f04538c9640 1 --2- 172.21.15.43:0/2986179625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f044c106a00 0x7f044c106de0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0448009930 tx=0x7f044802a420 comp rx=0 tx=0).stop 2024-09-11T23:26:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.871+0000 7f04538c9640 1 -- 172.21.15.43:0/2986179625 shutdown_connections 2024-09-11T23:26:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.871+0000 7f04538c9640 1 --2- 172.21.15.43:0/2986179625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f044c101390 0x7f044c10d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.871+0000 7f04538c9640 1 --2- 172.21.15.43:0/2986179625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f044c1009f0 0x7f044c100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.871+0000 7f04538c9640 1 --2- 172.21.15.43:0/2986179625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f044c106a00 0x7f044c106de0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.871+0000 7f04538c9640 1 -- 172.21.15.43:0/2986179625 >> 172.21.15.43:0/2986179625 conn(0x7f044c0fc6b0 msgr2=0x7f044c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.871+0000 7f04538c9640 1 -- 172.21.15.43:0/2986179625 shutdown_connections 2024-09-11T23:26:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.871+0000 7f04538c9640 1 -- 172.21.15.43:0/2986179625 wait complete. 2024-09-11T23:26:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.872+0000 7f04538c9640 1 Processor -- start 2024-09-11T23:26:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.879+0000 7f04538c9640 1 -- start start 2024-09-11T23:26:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.879+0000 7f04538c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f044c1009f0 0x7f044c073ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.879+0000 7f04538c9640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f044c101390 0x7f044c06cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.879+0000 7f04538c9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f044c106a00 0x7f044c06d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.879+0000 7f04538c9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f044c110e40 con 0x7f044c1009f0 2024-09-11T23:26:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.879+0000 7f04538c9640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f044c110cc0 con 0x7f044c101390 2024-09-11T23:26:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.879+0000 7f04538c9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f044c110fc0 con 0x7f044c106a00 2024-09-11T23:26:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.880+0000 7f0451e66640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f044c106a00 0x7f044c06d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.880+0000 7f0451e66640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f044c106a00 0x7f044c06d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:33250/0 (socket says 172.21.15.43:33250) 2024-09-11T23:26:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.880+0000 7f0451e66640 1 -- 172.21.15.43:0/315114390 learned_addr learned my addr 172.21.15.43:0/315114390 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.880+0000 7f0451e66640 1 -- 172.21.15.43:0/315114390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f044c101390 msgr2=0x7f044c06cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.880+0000 7f0450e64640 1 --2- 172.21.15.43:0/315114390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f044c101390 0x7f044c06cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.880+0000 7f0451e66640 1 --2- 172.21.15.43:0/315114390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f044c101390 0x7f044c06cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.880+0000 7f0451e66640 1 -- 172.21.15.43:0/315114390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f044c1009f0 msgr2=0x7f044c073ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.880+0000 7f0451e66640 1 --2- 172.21.15.43:0/315114390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f044c1009f0 0x7f044c073ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.880+0000 7f0451e66640 1 -- 172.21.15.43:0/315114390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044c06d720 con 0x7f044c106a00 2024-09-11T23:26:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.881+0000 7f0450e64640 1 --2- 172.21.15.43:0/315114390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f044c101390 0x7f044c06cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:26:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.881+0000 7f0451e66640 1 --2- 172.21.15.43:0/315114390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f044c106a00 0x7f044c06d080 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0444009fd0 tx=0x7f044400b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.882+0000 7f04427fc640 1 -- 172.21.15.43:0/315114390 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0444017070 con 0x7f044c106a00 2024-09-11T23:26:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.882+0000 7f04538c9640 1 -- 172.21.15.43:0/315114390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f044c0715e0 con 0x7f044c106a00 2024-09-11T23:26:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.882+0000 7f04427fc640 1 -- 172.21.15.43:0/315114390 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0444003e50 con 0x7f044c106a00 2024-09-11T23:26:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.882+0000 7f04427fc640 1 -- 172.21.15.43:0/315114390 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04440106c0 con 0x7f044c106a00 2024-09-11T23:26:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.883+0000 7f04538c9640 1 -- 172.21.15.43:0/315114390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f044c071b30 con 0x7f044c106a00 2024-09-11T23:26:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.888+0000 7f04427fc640 1 -- 172.21.15.43:0/315114390 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0444003970 con 0x7f044c106a00 2024-09-11T23:26:04.888 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.889+0000 7f04427fc640 1 --2- 172.21.15.43:0/315114390 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f04380a50d0 0x7f04380a7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.889+0000 7f04427fc640 1 -- 172.21.15.43:0/315114390 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f04440cf160 con 0x7f044c106a00 2024-09-11T23:26:04.890 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.891+0000 7f0451665640 1 --2- 172.21.15.43:0/315114390 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f04380a50d0 0x7f04380a7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:04.904+0000 7f0451665640 1 --2- 172.21.15.43:0/315114390 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f04380a50d0 0x7f04380a7590 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f044802a940 tx=0x7f0448000bc0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.025+0000 7f045665e740 1 Processor -- start 2024-09-11T23:26:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.025+0000 7f045665e740 1 -- start start 2024-09-11T23:26:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.026+0000 7f045665e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5624e6ff7af0 0x5624e6ff7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.026+0000 7f045665e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e6ff8410 0x5624e7002770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.026+0000 7f045665e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624e7002cb0 0x5624e70050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.026+0000 7f045665e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5624e6f5c2f0 con 0x5624e6ff8410 2024-09-11T23:26:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.026+0000 7f045665e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5624e6f04270 con 0x5624e7002cb0 2024-09-11T23:26:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.026+0000 7f045665e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5624e6f582b0 con 0x5624e6ff7af0 2024-09-11T23:26:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.028+0000 7f0450e64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e6ff8410 0x5624e7002770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.028+0000 7f0450e64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e6ff8410 0x5624e7002770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54472/0 (socket says 172.21.15.43:54472) 2024-09-11T23:26:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.028+0000 7f0450e64640 1 -- 172.21.15.43:0/289400733 learned_addr learned my addr 172.21.15.43:0/289400733 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.028+0000 7f0451e66640 1 --2- 172.21.15.43:0/289400733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624e7002cb0 0x5624e70050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.030+0000 7f0451665640 1 --2- 172.21.15.43:0/289400733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5624e6ff7af0 0x5624e6ff7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.030+0000 7f0451665640 1 -- 172.21.15.43:0/289400733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624e7002cb0 msgr2=0x5624e70050a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.030+0000 7f0451665640 1 --2- 172.21.15.43:0/289400733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624e7002cb0 0x5624e70050a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.030+0000 7f0451665640 1 -- 172.21.15.43:0/289400733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e6ff8410 msgr2=0x5624e7002770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.030+0000 7f0451665640 1 --2- 172.21.15.43:0/289400733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e6ff8410 0x5624e7002770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.030+0000 7f0451665640 1 -- 172.21.15.43:0/289400733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624e6f03340 con 0x5624e6ff7af0 2024-09-11T23:26:05.032 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.032+0000 7f0451665640 1 --2- 172.21.15.43:0/289400733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5624e6ff7af0 0x5624e6ff7ed0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0448009230 tx=0x7f044803a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae8c9765e61fb5b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.035+0000 7f04377fe640 1 -- 172.21.15.43:0/289400733 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448035040 con 0x5624e6ff7af0 2024-09-11T23:26:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.035+0000 7f04377fe640 1 -- 172.21.15.43:0/289400733 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04480318e0 con 0x5624e6ff7af0 2024-09-11T23:26:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.035+0000 7f04377fe640 1 -- 172.21.15.43:0/289400733 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448042910 con 0x5624e6ff7af0 2024-09-11T23:26:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.038+0000 7f045665e740 1 -- 172.21.15.43:0/289400733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5624e6ff7af0 msgr2=0x5624e6ff7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.038+0000 7f045665e740 1 --2- 172.21.15.43:0/289400733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5624e6ff7af0 0x5624e6ff7ed0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0448009230 tx=0x7f044803a000 comp rx=0 tx=0).stop 2024-09-11T23:26:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.038+0000 7f045665e740 1 -- 172.21.15.43:0/289400733 shutdown_connections 2024-09-11T23:26:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.038+0000 7f045665e740 1 --2- 172.21.15.43:0/289400733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624e7002cb0 0x5624e70050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.038+0000 7f045665e740 1 --2- 172.21.15.43:0/289400733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e6ff8410 0x5624e7002770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.038+0000 7f045665e740 1 --2- 172.21.15.43:0/289400733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5624e6ff7af0 0x5624e6ff7ed0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.038+0000 7f045665e740 1 -- 172.21.15.43:0/289400733 >> 172.21.15.43:0/289400733 conn(0x5624e6ff4060 msgr2=0x5624e6ff4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.043+0000 7f045665e740 1 -- 172.21.15.43:0/289400733 shutdown_connections 2024-09-11T23:26:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.043+0000 7f045665e740 1 -- 172.21.15.43:0/289400733 wait complete. 2024-09-11T23:26:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.043+0000 7f045665e740 1 -- start start 2024-09-11T23:26:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.044+0000 7f045665e740 1 Processor -- start 2024-09-11T23:26:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.044+0000 7f045665e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5624e6ff7af0 0x5624e6fffc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.044+0000 7f045665e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e6ff8410 0x5624e7000140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.044+0000 7f045665e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624e7002cb0 0x5624e7000680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.044+0000 7f045665e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5624e7005820 con 0x5624e6ff8410 2024-09-11T23:26:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.044+0000 7f045665e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5624e6ee5290 con 0x5624e7002cb0 2024-09-11T23:26:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.044+0000 7f045665e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5624e70059a0 con 0x5624e6ff7af0 2024-09-11T23:26:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.044+0000 7f045665e740 10 client.0 _ll_register_callbacks cb 0x5624e6f16140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.044+0000 7f0450e64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e6ff8410 0x5624e7000140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.044+0000 7f0450e64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e6ff8410 0x5624e7000140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54482/0 (socket says 172.21.15.43:54482) 2024-09-11T23:26:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.044+0000 7f0450e64640 1 -- 172.21.15.43:0/1172291066 learned_addr learned my addr 172.21.15.43:0/1172291066 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.044+0000 7f0451665640 1 --2- 172.21.15.43:0/1172291066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5624e6ff7af0 0x5624e6fffc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.044+0000 7f0451e66640 1 --2- 172.21.15.43:0/1172291066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624e7002cb0 0x5624e7000680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.045+0000 7f0450e64640 1 -- 172.21.15.43:0/1172291066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5624e6ff7af0 msgr2=0x5624e6fffc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.045+0000 7f0450e64640 1 --2- 172.21.15.43:0/1172291066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5624e6ff7af0 0x5624e6fffc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.045+0000 7f0450e64640 1 -- 172.21.15.43:0/1172291066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624e7002cb0 msgr2=0x5624e7000680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.045+0000 7f0450e64640 1 --2- 172.21.15.43:0/1172291066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624e7002cb0 0x5624e7000680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.045+0000 7f0450e64640 1 -- 172.21.15.43:0/1172291066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624e6c6f210 con 0x5624e6ff8410 2024-09-11T23:26:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.045+0000 7f0450e64640 1 --2- 172.21.15.43:0/1172291066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e6ff8410 0x5624e7000140 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f043c0126a0 tx=0x7f043c012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688140e3d992dfb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.045+0000 7f0434ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:26:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.045+0000 7f0434ff9640 1 -- 172.21.15.43:0/1172291066 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c004460 con 0x5624e6ff8410 2024-09-11T23:26:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.045+0000 7f0434ff9640 1 -- 172.21.15.43:0/1172291066 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f043c010950 con 0x5624e6ff8410 2024-09-11T23:26:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.045+0000 7f045665e740 1 -- 172.21.15.43:0/1172291066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04480318e0 con 0x5624e6ff8410 2024-09-11T23:26:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.045+0000 7f0434ff9640 1 -- 172.21.15.43:0/1172291066 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f043c010b50 con 0x5624e6ff8410 2024-09-11T23:26:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.045+0000 7f045665e740 10 client.16110 fetch_fsmap learned FSMap version 30 2024-09-11T23:26:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.045+0000 7f045665e740 1 -- 172.21.15.43:0/1172291066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5624e6d3b620 con 0x5624e6ff8410 2024-09-11T23:26:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.045+0000 7f0434ff9640 1 -- 172.21.15.43:0/1172291066 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04480318e0 con 0x5624e6ff8410 2024-09-11T23:26:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.046+0000 7f0434ff9640 1 -- 172.21.15.43:0/1172291066 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f043c019070 con 0x5624e6ff8410 2024-09-11T23:26:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.046+0000 7f045665e740 10 client.16110 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:26:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.046+0000 7f045665e740 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:26:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.046+0000 7f045665e740 20 client.16110 populate_metadata read hostname 'smithi043' 2024-09-11T23:26:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.046+0000 7f045665e740 1 --2- 172.21.15.43:0/1172291066 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5624e701b690 0x5624e701dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.046+0000 7f045665e740 4 client.16110 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:26:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.046+0000 7f045665e740 1 -- 172.21.15.43:0/1172291066 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624e6f69a20 con 0x5624e701b690 2024-09-11T23:26:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.046+0000 7f0451665640 1 --2- 172.21.15.43:0/1172291066 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5624e701b690 0x5624e701dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.047+0000 7f0451665640 1 --2- 172.21.15.43:0/1172291066 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5624e701b690 0x5624e701dad0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bb591a66c04abb4 server_cookie=ea8ec346db358a42 in_seq=0 out_seq=0 2024-09-11T23:26:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.047+0000 7f0434ff9640 10 client.16110 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.048+0000 7f0434ff9640 1 -- 172.21.15.43:0/1172291066 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624e6f69a20 con 0x5624e701b690 2024-09-11T23:26:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.048+0000 7f0434ff9640 10 client.16110 handle_command_reply: tid=0 2024-09-11T23:26:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.048+0000 7f045665e740 1 client.16110 shutdown 2024-09-11T23:26:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.048+0000 7f045665e740 1 -- 172.21.15.43:0/1172291066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e6ff8410 msgr2=0x5624e7000140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.048+0000 7f045665e740 1 --2- 172.21.15.43:0/1172291066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e6ff8410 0x5624e7000140 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f043c0126a0 tx=0x7f043c012b60 comp rx=0 tx=0).stop 2024-09-11T23:26:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.048+0000 7f045665e740 1 -- 172.21.15.43:0/1172291066 shutdown_connections 2024-09-11T23:26:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.048+0000 7f045665e740 1 --2- 172.21.15.43:0/1172291066 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5624e701b690 0x5624e701dad0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.048+0000 7f045665e740 1 --2- 172.21.15.43:0/1172291066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624e7002cb0 0x5624e7000680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.048+0000 7f045665e740 1 --2- 172.21.15.43:0/1172291066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e6ff8410 0x5624e7000140 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.048+0000 7f0434ff9640 0 client.16110 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.048+0000 7f045665e740 1 --2- 172.21.15.43:0/1172291066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5624e6ff7af0 0x5624e6fffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.048+0000 7f045665e740 1 -- 172.21.15.43:0/1172291066 >> 172.21.15.43:0/1172291066 conn(0x5624e6ff4060 msgr2=0x5624e6ff9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.048+0000 7f045665e740 1 -- 172.21.15.43:0/1172291066 shutdown_connections 2024-09-11T23:26:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.048+0000 7f045665e740 1 -- 172.21.15.43:0/1172291066 wait complete. 2024-09-11T23:26:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.048+0000 7f045665e740 20 client.16110 trim_cache size 0 max 16384 2024-09-11T23:26:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.063+0000 7f045665e740 1 Processor -- start 2024-09-11T23:26:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.063+0000 7f045665e740 1 -- start start 2024-09-11T23:26:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.064+0000 7f045665e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624e6ff7af0 0x5624e70004b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.064+0000 7f045665e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e6ff8410 0x5624e70233a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.064+0000 7f045665e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5624e7002cb0 0x5624e70238e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.064+0000 7f045665e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1b6850 con 0x5624e6ff8410 2024-09-11T23:26:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.064+0000 7f045665e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1b4120 con 0x5624e6ff7af0 2024-09-11T23:26:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.064+0000 7f045665e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1b1f70 con 0x5624e7002cb0 2024-09-11T23:26:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.064+0000 7f0450e64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e6ff8410 0x5624e70233a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.064+0000 7f0450e64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e6ff8410 0x5624e70233a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54494/0 (socket says 172.21.15.43:54494) 2024-09-11T23:26:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.064+0000 7f0450e64640 1 -- 172.21.15.43:0/1784217600 learned_addr learned my addr 172.21.15.43:0/1784217600 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.064+0000 7f0451665640 1 --2- 172.21.15.43:0/1784217600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624e6ff7af0 0x5624e70004b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.064+0000 7f0451e66640 1 --2- 172.21.15.43:0/1784217600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5624e7002cb0 0x5624e70238e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.064+0000 7f0450e64640 1 -- 172.21.15.43:0/1784217600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5624e7002cb0 msgr2=0x5624e70238e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.064+0000 7f0450e64640 1 --2- 172.21.15.43:0/1784217600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5624e7002cb0 0x5624e70238e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.064+0000 7f0450e64640 1 -- 172.21.15.43:0/1784217600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624e6ff7af0 msgr2=0x5624e70004b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.064+0000 7f0450e64640 1 --2- 172.21.15.43:0/1784217600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624e6ff7af0 0x5624e70004b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.064+0000 7f0450e64640 1 -- 172.21.15.43:0/1784217600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624e6ff7870 con 0x5624e6ff8410 2024-09-11T23:26:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.064+0000 7f0451665640 1 --2- 172.21.15.43:0/1784217600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624e6ff7af0 0x5624e70004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:26:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.064+0000 7f0450e64640 1 --2- 172.21.15.43:0/1784217600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e6ff8410 0x5624e70233a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f043c01b810 tx=0x7f043c005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa846e892edc235a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.064+0000 7f0436ffd640 1 -- 172.21.15.43:0/1784217600 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c01b890 con 0x5624e6ff8410 2024-09-11T23:26:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.064+0000 7f0436ffd640 1 -- 172.21.15.43:0/1784217600 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f043c01ba30 con 0x5624e6ff8410 2024-09-11T23:26:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.065+0000 7f0436ffd640 1 -- 172.21.15.43:0/1784217600 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c003bf0 con 0x5624e6ff8410 2024-09-11T23:26:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.065+0000 7f045665e740 1 -- 172.21.15.43:0/1784217600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e6ff8410 msgr2=0x5624e70233a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.065+0000 7f045665e740 1 --2- 172.21.15.43:0/1784217600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e6ff8410 0x5624e70233a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f043c01b810 tx=0x7f043c005e70 comp rx=0 tx=0).stop 2024-09-11T23:26:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.065+0000 7f045665e740 1 -- 172.21.15.43:0/1784217600 shutdown_connections 2024-09-11T23:26:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.065+0000 7f045665e740 1 --2- 172.21.15.43:0/1784217600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5624e7002cb0 0x5624e70238e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.065+0000 7f045665e740 1 --2- 172.21.15.43:0/1784217600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e6ff8410 0x5624e70233a0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.065+0000 7f045665e740 1 --2- 172.21.15.43:0/1784217600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624e6ff7af0 0x5624e70004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.065+0000 7f045665e740 1 -- 172.21.15.43:0/1784217600 >> 172.21.15.43:0/1784217600 conn(0x5624e6ff4060 msgr2=0x5624e7001c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.065+0000 7f045665e740 1 -- 172.21.15.43:0/1784217600 shutdown_connections 2024-09-11T23:26:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.065+0000 7f045665e740 1 -- 172.21.15.43:0/1784217600 wait complete. 2024-09-11T23:26:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.066+0000 7f045665e740 1 -- start start 2024-09-11T23:26:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.066+0000 7f045665e740 1 Processor -- start 2024-09-11T23:26:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.066+0000 7f045665e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624e6ff7af0 0x5624e7013d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.066+0000 7f045665e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5624e6ff8410 0x5624e700f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.066+0000 7f045665e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e7002cb0 0x5624e700f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.066+0000 7f045665e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1b25e0 con 0x5624e7002cb0 2024-09-11T23:26:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.066+0000 7f045665e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1b3a50 con 0x5624e6ff7af0 2024-09-11T23:26:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.066+0000 7f045665e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1b2cb0 con 0x5624e6ff8410 2024-09-11T23:26:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.066+0000 7f045665e740 10 client.0 _ll_register_callbacks cb 0x7f043c0095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.066+0000 7f0451e66640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e7002cb0 0x5624e700f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.066+0000 7f0451e66640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e7002cb0 0x5624e700f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54500/0 (socket says 172.21.15.43:54500) 2024-09-11T23:26:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.066+0000 7f0451e66640 1 -- 172.21.15.43:0/4252913101 learned_addr learned my addr 172.21.15.43:0/4252913101 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.066+0000 7f0450e64640 1 --2- 172.21.15.43:0/4252913101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5624e6ff8410 0x5624e700f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.066+0000 7f0451665640 1 --2- 172.21.15.43:0/4252913101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624e6ff7af0 0x5624e7013d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.066+0000 7f0451e66640 1 -- 172.21.15.43:0/4252913101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5624e6ff8410 msgr2=0x5624e700f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.066+0000 7f0451e66640 1 --2- 172.21.15.43:0/4252913101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5624e6ff8410 0x5624e700f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.066+0000 7f0451e66640 1 -- 172.21.15.43:0/4252913101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624e6ff7af0 msgr2=0x5624e7013d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.066+0000 7f0451e66640 1 --2- 172.21.15.43:0/4252913101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624e6ff7af0 0x5624e7013d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.067+0000 7f0451e66640 1 -- 172.21.15.43:0/4252913101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624e700fe60 con 0x5624e7002cb0 2024-09-11T23:26:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.067+0000 7f0451e66640 1 --2- 172.21.15.43:0/4252913101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e7002cb0 0x5624e700f660 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0444097ff0 tx=0x7f044409b0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411cc5620b537db7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.067+0000 7f0434ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:26:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.067+0000 7f0434ff9640 1 -- 172.21.15.43:0/4252913101 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04440b3070 con 0x5624e7002cb0 2024-09-11T23:26:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.067+0000 7f0434ff9640 1 -- 172.21.15.43:0/4252913101 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f04440ccad0 con 0x5624e7002cb0 2024-09-11T23:26:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.067+0000 7f045665e740 1 -- 172.21.15.43:0/4252913101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f043c01ba30 con 0x5624e7002cb0 2024-09-11T23:26:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.067+0000 7f0434ff9640 1 -- 172.21.15.43:0/4252913101 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0444020040 con 0x5624e7002cb0 2024-09-11T23:26:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.067+0000 7f045665e740 10 client.16116 fetch_fsmap learned FSMap version 30 2024-09-11T23:26:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.067+0000 7f045665e740 1 -- 172.21.15.43:0/4252913101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5624e70864c0 con 0x5624e7002cb0 2024-09-11T23:26:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.067+0000 7f0434ff9640 1 -- 172.21.15.43:0/4252913101 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f043c01ba30 con 0x5624e7002cb0 2024-09-11T23:26:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.067+0000 7f0434ff9640 1 -- 172.21.15.43:0/4252913101 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f04440afc60 con 0x5624e7002cb0 2024-09-11T23:26:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.067+0000 7f045665e740 10 client.16116 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:26:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.067+0000 7f045665e740 10 client.16116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:26:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.067+0000 7f045665e740 20 client.16116 populate_metadata read hostname 'smithi043' 2024-09-11T23:26:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.067+0000 7f045665e740 1 --2- 172.21.15.43:0/4252913101 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5624e7086c30 0x5624e7089020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.067+0000 7f045665e740 4 client.16116 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:26:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.067+0000 7f045665e740 1 -- 172.21.15.43:0/4252913101 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5624e6f63d70 con 0x5624e7086c30 2024-09-11T23:26:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.068+0000 7f0451665640 1 --2- 172.21.15.43:0/4252913101 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5624e7086c30 0x5624e7089020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.068+0000 7f0451665640 1 --2- 172.21.15.43:0/4252913101 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5624e7086c30 0x5624e7089020 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce159ac72075f667 server_cookie=7b26192817e4c5c6 in_seq=0 out_seq=0 2024-09-11T23:26:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:05.068+0000 7f0434ff9640 10 client.16116 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:05.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.250+0000 7f224e4b6640 1 -- 172.21.15.43:0/2200445502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2248102790 msgr2=0x7f22481067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:05.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.250+0000 7f224e4b6640 1 --2- 172.21.15.43:0/2200445502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2248102790 0x7f22481067e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2230009900 tx=0x7f223002a410 comp rx=0 tx=0).stop 2024-09-11T23:26:05.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.250+0000 7f224e4b6640 1 -- 172.21.15.43:0/2200445502 shutdown_connections 2024-09-11T23:26:05.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.250+0000 7f224e4b6640 1 --2- 172.21.15.43:0/2200445502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2248106f10 0x7f224810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.250+0000 7f224e4b6640 1 --2- 172.21.15.43:0/2200445502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2248102790 0x7f22481067e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.250+0000 7f224e4b6640 1 --2- 172.21.15.43:0/2200445502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2248101de0 0x7f22481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.250+0000 7f224e4b6640 1 -- 172.21.15.43:0/2200445502 >> 172.21.15.43:0/2200445502 conn(0x7f22480fd8c0 msgr2=0x7f22480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:05.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.250+0000 7f224e4b6640 1 -- 172.21.15.43:0/2200445502 shutdown_connections 2024-09-11T23:26:05.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.250+0000 7f224e4b6640 1 -- 172.21.15.43:0/2200445502 wait complete. 2024-09-11T23:26:05.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.251+0000 7f224e4b6640 1 Processor -- start 2024-09-11T23:26:05.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.251+0000 7f224e4b6640 1 -- start start 2024-09-11T23:26:05.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.251+0000 7f224e4b6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2248101de0 0x7f2248198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.251+0000 7f224e4b6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2248102790 0x7f22481989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.251+0000 7f224e4b6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2248106f10 0x7f224819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.251+0000 7f224e4b6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2248111f70 con 0x7f2248101de0 2024-09-11T23:26:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.251+0000 7f224e4b6640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2248111df0 con 0x7f2248102790 2024-09-11T23:26:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.251+0000 7f224e4b6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f22481120f0 con 0x7f2248106f10 2024-09-11T23:26:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.251+0000 7f2247fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2248101de0 0x7f2248198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.251+0000 7f2247fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2248101de0 0x7f2248198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54412/0 (socket says 172.21.15.43:54412) 2024-09-11T23:26:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.251+0000 7f2247fff640 1 -- 172.21.15.43:0/905457447 learned_addr learned my addr 172.21.15.43:0/905457447 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.251+0000 7f223ffff640 1 --2- 172.21.15.43:0/905457447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2248102790 0x7f22481989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.251+0000 7f224ca53640 1 --2- 172.21.15.43:0/905457447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2248106f10 0x7f224819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.252+0000 7f2247fff640 1 -- 172.21.15.43:0/905457447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2248106f10 msgr2=0x7f224819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.252+0000 7f2247fff640 1 --2- 172.21.15.43:0/905457447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2248106f10 0x7f224819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.252+0000 7f2247fff640 1 -- 172.21.15.43:0/905457447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2248102790 msgr2=0x7f22481989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.252+0000 7f2247fff640 1 --2- 172.21.15.43:0/905457447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2248102790 0x7f22481989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.252+0000 7f2247fff640 1 -- 172.21.15.43:0/905457447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224819d440 con 0x7f2248101de0 2024-09-11T23:26:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.252+0000 7f2247fff640 1 --2- 172.21.15.43:0/905457447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2248101de0 0x7f2248198490 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f223800ea60 tx=0x7f223800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:05.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.252+0000 7f2245ffb640 1 -- 172.21.15.43:0/905457447 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223800cde0 con 0x7f2248101de0 2024-09-11T23:26:05.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.252+0000 7f224e4b6640 1 -- 172.21.15.43:0/905457447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224819d730 con 0x7f2248101de0 2024-09-11T23:26:05.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.252+0000 7f2245ffb640 1 -- 172.21.15.43:0/905457447 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22380035f0 con 0x7f2248101de0 2024-09-11T23:26:05.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.252+0000 7f2245ffb640 1 -- 172.21.15.43:0/905457447 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2238010670 con 0x7f2248101de0 2024-09-11T23:26:05.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.252+0000 7f224e4b6640 1 -- 172.21.15.43:0/905457447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22481a5010 con 0x7f2248101de0 2024-09-11T23:26:05.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.255+0000 7f225088f740 1 Processor -- start 2024-09-11T23:26:05.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.255+0000 7f225088f740 1 -- start start 2024-09-11T23:26:05.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.256+0000 7f225088f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cc3760 0x564802cc3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.256+0000 7f225088f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc4080 0x564802cce400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.256+0000 7f225088f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564802cce940 0x564802cd0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.256+0000 7f225088f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564802c292a0 con 0x564802cc4080 2024-09-11T23:26:05.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.256+0000 7f225088f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564802bd0b10 con 0x564802cc3760 2024-09-11T23:26:05.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.256+0000 7f225088f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564802c25260 con 0x564802cce940 2024-09-11T23:26:05.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.256+0000 7f2245ffb640 1 -- 172.21.15.43:0/905457447 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f22380037a0 con 0x7f2248101de0 2024-09-11T23:26:05.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.256+0000 7f224ca53640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cc3760 0x564802cc3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.256+0000 7f224ca53640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cc3760 0x564802cc3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:55240/0 (socket says 172.21.15.43:55240) 2024-09-11T23:26:05.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.256+0000 7f224ca53640 1 -- 172.21.15.43:0/1605417213 learned_addr learned my addr 172.21.15.43:0/1605417213 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:05.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.256+0000 7f223ffff640 1 --2- 172.21.15.43:0/1605417213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564802cce940 0x564802cd0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.256+0000 7f224ca53640 1 -- 172.21.15.43:0/1605417213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564802cce940 msgr2=0x564802cd0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:05.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.256+0000 7f224ca53640 1 --2- 172.21.15.43:0/1605417213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564802cce940 0x564802cd0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.256+0000 7f224ca53640 1 -- 172.21.15.43:0/1605417213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc4080 msgr2=0x564802cce400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:26:05.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.256+0000 7f224ca53640 1 --2- 172.21.15.43:0/1605417213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc4080 0x564802cce400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.256+0000 7f224ca53640 1 -- 172.21.15.43:0/1605417213 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56480299ab90 con 0x564802cc3760 2024-09-11T23:26:05.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.256+0000 7f224ca53640 1 --2- 172.21.15.43:0/1605417213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cc3760 0x564802cc3b40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2240009f40 tx=0x7f2240010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=989c353c0c67c12 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:05.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.257+0000 7f223e7fc640 1 -- 172.21.15.43:0/1605417213 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240003b70 con 0x564802cc3760 2024-09-11T23:26:05.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.257+0000 7f223e7fc640 1 -- 172.21.15.43:0/1605417213 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2240003d10 con 0x564802cc3760 2024-09-11T23:26:05.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.257+0000 7f223e7fc640 1 -- 172.21.15.43:0/1605417213 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240011af0 con 0x564802cc3760 2024-09-11T23:26:05.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.257+0000 7f2245ffb640 1 --2- 172.21.15.43:0/905457447 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f22340a2cf0 0x7f22340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.257+0000 7f2247fff640 1 --2- 172.21.15.43:0/905457447 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f22340a2cf0 0x7f22340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.257+0000 7f2245ffb640 1 -- 172.21.15.43:0/905457447 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f22380cf7d0 con 0x7f2248101de0 2024-09-11T23:26:05.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.258+0000 7f225088f740 1 -- 172.21.15.43:0/1605417213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cc3760 msgr2=0x564802cc3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:05.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.258+0000 7f225088f740 1 --2- 172.21.15.43:0/1605417213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cc3760 0x564802cc3b40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2240009f40 tx=0x7f2240010860 comp rx=0 tx=0).stop 2024-09-11T23:26:05.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.258+0000 7f2247fff640 1 --2- 172.21.15.43:0/905457447 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f22340a2cf0 0x7f22340a51b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x564802cc0b50 tx=0x7f223801b6d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:05.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.258+0000 7f225088f740 1 -- 172.21.15.43:0/1605417213 shutdown_connections 2024-09-11T23:26:05.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.258+0000 7f225088f740 1 --2- 172.21.15.43:0/1605417213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564802cce940 0x564802cd0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.258+0000 7f225088f740 1 --2- 172.21.15.43:0/1605417213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc4080 0x564802cce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.258+0000 7f225088f740 1 --2- 172.21.15.43:0/1605417213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cc3760 0x564802cc3b40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.258+0000 7f225088f740 1 -- 172.21.15.43:0/1605417213 >> 172.21.15.43:0/1605417213 conn(0x564802cbfcf0 msgr2=0x564802cc00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:05.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.258+0000 7f225088f740 1 -- 172.21.15.43:0/1605417213 shutdown_connections 2024-09-11T23:26:05.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.258+0000 7f225088f740 1 -- 172.21.15.43:0/1605417213 wait complete. 2024-09-11T23:26:05.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.258+0000 7f225088f740 1 -- start start 2024-09-11T23:26:05.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.259+0000 7f225088f740 1 Processor -- start 2024-09-11T23:26:05.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.259+0000 7f225088f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564802cc3760 0x564802ccb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.259+0000 7f225088f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc4080 0x564802ccbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.260+0000 7f225088f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cce940 0x564802ccc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.260+0000 7f225088f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564802cd1510 con 0x564802cc4080 2024-09-11T23:26:05.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.260+0000 7f225088f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564802bb16e0 con 0x564802cce940 2024-09-11T23:26:05.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.260+0000 7f225088f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564802cd1690 con 0x564802cc3760 2024-09-11T23:26:05.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.260+0000 7f223ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc4080 0x564802ccbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.260+0000 7f224ca53640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564802cc3760 0x564802ccb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.260+0000 7f224ca53640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cce940 0x564802ccc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.260+0000 7f223ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc4080 0x564802ccbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54424/0 (socket says 172.21.15.43:54424) 2024-09-11T23:26:05.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.260+0000 7f224ca53640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cce940 0x564802ccc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:55250/0 (socket says 172.21.15.43:55250) 2024-09-11T23:26:05.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.260+0000 7f223ffff640 1 -- 172.21.15.43:0/3003083159 learned_addr learned my addr 172.21.15.43:0/3003083159 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:05.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.260+0000 7f225088f740 10 client.0 _ll_register_callbacks cb 0x564802bd5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:05.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.260+0000 7f223ffff640 1 -- 172.21.15.43:0/3003083159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564802cc3760 msgr2=0x564802ccb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:05.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.260+0000 7f223ffff640 1 --2- 172.21.15.43:0/3003083159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564802cc3760 0x564802ccb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.260+0000 7f223ffff640 1 -- 172.21.15.43:0/3003083159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cce940 msgr2=0x564802ccc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:05.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.260+0000 7f223ffff640 1 --2- 172.21.15.43:0/3003083159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cce940 0x564802ccc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.260+0000 7f223ffff640 1 -- 172.21.15.43:0/3003083159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564802cdab20 con 0x564802cc4080 2024-09-11T23:26:05.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.260+0000 7f224ca53640 1 --2- 172.21.15.43:0/3003083159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cce940 0x564802ccc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:26:05.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.260+0000 7f223ffff640 1 --2- 172.21.15.43:0/3003083159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc4080 0x564802ccbe60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f223002a930 tx=0x7f22300317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7a6cd6e116e09b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:05.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.260+0000 7f22277fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:26:05.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.260+0000 7f225088f740 1 -- 172.21.15.43:0/3003083159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2240003d10 con 0x564802cc4080 2024-09-11T23:26:05.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.261+0000 7f22277fe640 1 -- 172.21.15.43:0/3003083159 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2230035040 con 0x564802cc4080 2024-09-11T23:26:05.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.261+0000 7f22277fe640 1 -- 172.21.15.43:0/3003083159 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f2230004040 con 0x564802cc4080 2024-09-11T23:26:05.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.261+0000 7f22277fe640 1 -- 172.21.15.43:0/3003083159 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2230004240 con 0x564802cc4080 2024-09-11T23:26:05.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.261+0000 7f225088f740 10 client.16098 fetch_fsmap learned FSMap version 30 2024-09-11T23:26:05.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.261+0000 7f225088f740 1 -- 172.21.15.43:0/3003083159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({fsmap=30}) -- 0x564802cdd3b0 con 0x564802cc4080 2024-09-11T23:26:05.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.261+0000 7f22277fe640 1 -- 172.21.15.43:0/3003083159 <== mon.0 v2:172.21.15.43:3300/0 4 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f22300035f0 con 0x564802cc4080 2024-09-11T23:26:05.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.261+0000 7f225088f740 10 client.16098 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:26:05.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.261+0000 7f225088f740 10 client.16098 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:26:05.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.261+0000 7f225088f740 20 client.16098 populate_metadata read hostname 'smithi043' 2024-09-11T23:26:05.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.261+0000 7f225088f740 1 --2- 172.21.15.43:0/3003083159 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564802cde760 0x564802ce0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.261+0000 7f225088f740 4 client.16098 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:26:05.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.261+0000 7f225088f740 1 -- 172.21.15.43:0/3003083159 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564802c369d0 con 0x564802cde760 2024-09-11T23:26:05.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.261+0000 7f224ca53640 1 --2- 172.21.15.43:0/3003083159 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564802cde760 0x564802ce0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.262+0000 7f224ca53640 1 --2- 172.21.15.43:0/3003083159 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564802cde760 0x564802ce0b50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6650a2d6d6e0887 server_cookie=6abcfe1d90ac42d in_seq=0 out_seq=0 2024-09-11T23:26:05.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.262+0000 7f22277fe640 10 client.16098 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:05.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.263+0000 7f22277fe640 1 -- 172.21.15.43:0/3003083159 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564802c369d0 con 0x564802cde760 2024-09-11T23:26:05.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.263+0000 7f22277fe640 10 client.16098 handle_command_reply: tid=0 2024-09-11T23:26:05.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.263+0000 7f225088f740 1 client.16098 shutdown 2024-09-11T23:26:05.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.263+0000 7f225088f740 1 -- 172.21.15.43:0/3003083159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc4080 msgr2=0x564802ccbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:05.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.263+0000 7f225088f740 1 --2- 172.21.15.43:0/3003083159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc4080 0x564802ccbe60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f223002a930 tx=0x7f22300317e0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.263+0000 7f225088f740 1 -- 172.21.15.43:0/3003083159 shutdown_connections 2024-09-11T23:26:05.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.263+0000 7f225088f740 1 --2- 172.21.15.43:0/3003083159 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564802cde760 0x564802ce0b50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.263+0000 7f225088f740 1 --2- 172.21.15.43:0/3003083159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cce940 0x564802ccc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.263+0000 7f225088f740 1 --2- 172.21.15.43:0/3003083159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc4080 0x564802ccbe60 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.263+0000 7f22277fe640 0 client.16098 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:05.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.263+0000 7f225088f740 1 --2- 172.21.15.43:0/3003083159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564802cc3760 0x564802ccb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.263+0000 7f225088f740 1 -- 172.21.15.43:0/3003083159 >> 172.21.15.43:0/3003083159 conn(0x564802cbfcf0 msgr2=0x564802cc5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:05.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.263+0000 7f225088f740 1 -- 172.21.15.43:0/3003083159 shutdown_connections 2024-09-11T23:26:05.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.263+0000 7f225088f740 1 -- 172.21.15.43:0/3003083159 wait complete. 2024-09-11T23:26:05.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.263+0000 7f225088f740 20 client.16098 trim_cache size 0 max 16384 2024-09-11T23:26:05.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.286+0000 7f225088f740 1 Processor -- start 2024-09-11T23:26:05.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.286+0000 7f225088f740 1 -- start start 2024-09-11T23:26:05.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.287+0000 7f225088f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc3760 0x564802ccd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.287+0000 7f225088f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564802cc4080 0x564802ccc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.287+0000 7f225088f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cce940 0x564802ce63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.287+0000 7f225088f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f22481adfd0 con 0x564802cc3760 2024-09-11T23:26:05.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.287+0000 7f225088f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f22481ab8a0 con 0x564802cce940 2024-09-11T23:26:05.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.287+0000 7f225088f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f22481a96f0 con 0x564802cc4080 2024-09-11T23:26:05.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.287+0000 7f224ca53640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cce940 0x564802ce63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.287+0000 7f224ca53640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cce940 0x564802ce63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:55266/0 (socket says 172.21.15.43:55266) 2024-09-11T23:26:05.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.287+0000 7f224ca53640 1 -- 172.21.15.43:0/861229560 learned_addr learned my addr 172.21.15.43:0/861229560 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:05.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.287+0000 7f224ca53640 1 -- 172.21.15.43:0/861229560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564802cc4080 msgr2=0x564802ccc1f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:26:05.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.287+0000 7f224ca53640 1 --2- 172.21.15.43:0/861229560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564802cc4080 0x564802ccc1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.287+0000 7f224ca53640 1 -- 172.21.15.43:0/861229560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc3760 msgr2=0x564802ccd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:05.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.287+0000 7f224ca53640 1 --2- 172.21.15.43:0/861229560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc3760 0x564802ccd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.287+0000 7f224ca53640 1 -- 172.21.15.43:0/861229560 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564802cbb260 con 0x564802cce940 2024-09-11T23:26:05.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.288+0000 7f224ca53640 1 --2- 172.21.15.43:0/861229560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cce940 0x564802ce63e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2240009ee0 tx=0x7f2240009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90c4b1f07febe10f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:05.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.289+0000 7f223dffb640 1 -- 172.21.15.43:0/861229560 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240011d80 con 0x564802cce940 2024-09-11T23:26:05.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.289+0000 7f223dffb640 1 -- 172.21.15.43:0/861229560 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2240005050 con 0x564802cce940 2024-09-11T23:26:05.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.289+0000 7f223dffb640 1 -- 172.21.15.43:0/861229560 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240032620 con 0x564802cce940 2024-09-11T23:26:05.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.290+0000 7f225088f740 1 -- 172.21.15.43:0/861229560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cce940 msgr2=0x564802ce63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:05.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.290+0000 7f225088f740 1 --2- 172.21.15.43:0/861229560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cce940 0x564802ce63e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2240009ee0 tx=0x7f2240009fd0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.292+0000 7f225088f740 1 -- 172.21.15.43:0/861229560 shutdown_connections 2024-09-11T23:26:05.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.292+0000 7f225088f740 1 --2- 172.21.15.43:0/861229560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cce940 0x564802ce63e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.292+0000 7f225088f740 1 --2- 172.21.15.43:0/861229560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564802cc4080 0x564802ccc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.292+0000 7f225088f740 1 --2- 172.21.15.43:0/861229560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc3760 0x564802ccd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.292+0000 7f225088f740 1 -- 172.21.15.43:0/861229560 >> 172.21.15.43:0/861229560 conn(0x564802cbfcf0 msgr2=0x564802cbee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:05.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.295+0000 7f225088f740 1 -- 172.21.15.43:0/861229560 shutdown_connections 2024-09-11T23:26:05.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.295+0000 7f225088f740 1 -- 172.21.15.43:0/861229560 wait complete. 2024-09-11T23:26:05.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.295+0000 7f225088f740 1 -- start start 2024-09-11T23:26:05.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.296+0000 7f225088f740 1 Processor -- start 2024-09-11T23:26:05.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.296+0000 7f225088f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cc3760 0x564802cd6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.296+0000 7f225088f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc4080 0x564802cd38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.296+0000 7f225088f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564802cce940 0x564802cd3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.296+0000 7f225088f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f22481a9d60 con 0x564802cc4080 2024-09-11T23:26:05.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.296+0000 7f225088f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f22481ab1d0 con 0x564802cc3760 2024-09-11T23:26:05.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.296+0000 7f225088f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f22481aa430 con 0x564802cce940 2024-09-11T23:26:05.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.296+0000 7f225088f740 10 client.0 _ll_register_callbacks cb 0x7f2230005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:05.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.299+0000 7f223ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc4080 0x564802cd38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.299+0000 7f223ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc4080 0x564802cd38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54448/0 (socket says 172.21.15.43:54448) 2024-09-11T23:26:05.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.299+0000 7f223ffff640 1 -- 172.21.15.43:0/3557255245 learned_addr learned my addr 172.21.15.43:0/3557255245 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:05.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.301+0000 7f224ca53640 1 --2- 172.21.15.43:0/3557255245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cc3760 0x564802cd6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.301+0000 7f224ca53640 1 --2- 172.21.15.43:0/3557255245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564802cce940 0x564802cd3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.301+0000 7f223ffff640 1 -- 172.21.15.43:0/3557255245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564802cce940 msgr2=0x564802cd3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:05.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.301+0000 7f223ffff640 1 --2- 172.21.15.43:0/3557255245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564802cce940 0x564802cd3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.301+0000 7f223ffff640 1 -- 172.21.15.43:0/3557255245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cc3760 msgr2=0x564802cd6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:05.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.301+0000 7f223ffff640 1 --2- 172.21.15.43:0/3557255245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cc3760 0x564802cd6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:05.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.301+0000 7f223ffff640 1 -- 172.21.15.43:0/3557255245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564802cbf580 con 0x564802cc4080 2024-09-11T23:26:05.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.301+0000 7f223ffff640 1 --2- 172.21.15.43:0/3557255245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc4080 0x564802cd38b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2230009230 tx=0x7f2230009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dab8f20ddd7afd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:05.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.301+0000 7f22277fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:26:05.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.305+0000 7f22277fe640 1 -- 172.21.15.43:0/3557255245 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223005a060 con 0x564802cc4080 2024-09-11T23:26:05.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.305+0000 7f22277fe640 1 -- 172.21.15.43:0/3557255245 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f22300318d0 con 0x564802cc4080 2024-09-11T23:26:05.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.306+0000 7f225088f740 1 -- 172.21.15.43:0/3557255245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2240005050 con 0x564802cc4080 2024-09-11T23:26:05.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.314+0000 7f22277fe640 1 -- 172.21.15.43:0/3557255245 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2230035070 con 0x564802cc4080 2024-09-11T23:26:05.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.317+0000 7f225088f740 10 client.16101 fetch_fsmap learned FSMap version 30 2024-09-11T23:26:05.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.317+0000 7f225088f740 1 -- 172.21.15.43:0/3557255245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564802d6c060 con 0x564802cc4080 2024-09-11T23:26:05.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.319+0000 7f22277fe640 1 -- 172.21.15.43:0/3557255245 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2240005050 con 0x564802cc4080 2024-09-11T23:26:05.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.319+0000 7f22277fe640 1 -- 172.21.15.43:0/3557255245 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f2230005030 con 0x564802cc4080 2024-09-11T23:26:05.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.319+0000 7f225088f740 10 client.16101 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:26:05.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.319+0000 7f225088f740 10 client.16101 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:26:05.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.319+0000 7f225088f740 20 client.16101 populate_metadata read hostname 'smithi043' 2024-09-11T23:26:05.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.319+0000 7f225088f740 1 --2- 172.21.15.43:0/3557255245 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564802d6c7f0 0x564802d6ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:05.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.319+0000 7f225088f740 4 client.16101 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}] 2024-09-11T23:26:05.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.319+0000 7f225088f740 1 -- 172.21.15.43:0/3557255245 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}) -- 0x564802c30d20 con 0x564802d6c7f0 2024-09-11T23:26:05.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.320+0000 7f224ca53640 1 --2- 172.21.15.43:0/3557255245 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564802d6c7f0 0x564802d6ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:05.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.323+0000 7f224ca53640 1 --2- 172.21.15.43:0/3557255245 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564802d6c7f0 0x564802d6ec10 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c11a09f9a4a4e52f server_cookie=b6e77d1b53f20541 in_seq=0 out_seq=0 2024-09-11T23:26:05.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:04.324+0000 7f22277fe640 10 client.16101 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:05.551 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:26:05.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:05 smithi043 ceph-mon[30950]: pgmap v131: 97 pgs: 97 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 49 MiB/s wr, 6.61k op/s 2024-09-11T23:26:05.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:05 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:26:05.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:05 smithi090 ceph-mon[35130]: pgmap v131: 97 pgs: 97 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 49 MiB/s wr, 6.61k op/s 2024-09-11T23:26:05.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:05 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:26:05.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:05 smithi097 ceph-mon[35562]: pgmap v131: 97 pgs: 97 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 49 MiB/s wr, 6.61k op/s 2024-09-11T23:26:05.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:05 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:26:07.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:07 smithi090 ceph-mon[35130]: pgmap v132: 97 pgs: 97 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 50 MiB/s wr, 6.87k op/s 2024-09-11T23:26:07.774 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:07 smithi043 ceph-mon[30950]: pgmap v132: 97 pgs: 97 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 50 MiB/s wr, 6.87k op/s 2024-09-11T23:26:07.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:07 smithi097 ceph-mon[35562]: pgmap v132: 97 pgs: 97 active+clean; 17 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 50 MiB/s wr, 6.87k op/s 2024-09-11T23:26:09.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:09 smithi090 ceph-mon[35130]: pgmap v133: 97 pgs: 97 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 74 MiB/s wr, 10.64k op/s 2024-09-11T23:26:09.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:09 smithi097 ceph-mon[35562]: pgmap v133: 97 pgs: 97 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 74 MiB/s wr, 10.64k op/s 2024-09-11T23:26:09.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:09 smithi043 ceph-mon[30950]: pgmap v133: 97 pgs: 97 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 74 MiB/s wr, 10.64k op/s 2024-09-11T23:26:10.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:10 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:26:10.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:10 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:26:10.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:10 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:26:10.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:10 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:26:10.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:10 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:26:10.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:10 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:26:10.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:10 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:26:10.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:10 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:26:10.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:10 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:26:11.806 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:26:11.806 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "scrub active (22 inodes in the stack)", 2024-09-11T23:26:11.807 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-09-11T23:26:11.807 INFO:teuthology.orchestra.run.smithi043.stdout: "ddf9b449-9095-42dd-a28c-7f5af91a6fc0": { 2024-09-11T23:26:11.807 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-09-11T23:26:11.807 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "ddf9b449-9095-42dd-a28c-7f5af91a6fc0", 2024-09-11T23:26:11.807 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-09-11T23:26:11.807 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:26:11.807 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:26:11.807 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:26:11.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.618+0000 7fb841f9c640 1 -- 172.21.15.43:0/1197717814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb83c106a80 msgr2=0x7fb83c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:11.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.618+0000 7fb841f9c640 1 --2- 172.21.15.43:0/1197717814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb83c106a80 0x7fb83c106e60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009930 tx=0x7fb82c02a420 comp rx=0 tx=0).stop 2024-09-11T23:26:11.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.619+0000 7fb841f9c640 1 -- 172.21.15.43:0/1197717814 shutdown_connections 2024-09-11T23:26:11.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.619+0000 7fb841f9c640 1 --2- 172.21.15.43:0/1197717814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb83c101420 0x7fb83c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.619+0000 7fb841f9c640 1 --2- 172.21.15.43:0/1197717814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c100a80 0x7fb83c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.619+0000 7fb841f9c640 1 --2- 172.21.15.43:0/1197717814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb83c106a80 0x7fb83c106e60 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.619+0000 7fb841f9c640 1 -- 172.21.15.43:0/1197717814 >> 172.21.15.43:0/1197717814 conn(0x7fb83c0fc7a0 msgr2=0x7fb83c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:11.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.619+0000 7fb841f9c640 1 -- 172.21.15.43:0/1197717814 shutdown_connections 2024-09-11T23:26:11.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.619+0000 7fb841f9c640 1 -- 172.21.15.43:0/1197717814 wait complete. 2024-09-11T23:26:11.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.619+0000 7fb841f9c640 1 Processor -- start 2024-09-11T23:26:11.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.619+0000 7fb841f9c640 1 -- start start 2024-09-11T23:26:11.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.620+0000 7fb841f9c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb83c100a80 0x7fb83c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:11.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.620+0000 7fb841f9c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c101420 0x7fb83c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:11.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.620+0000 7fb841f9c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb83c106a80 0x7fb83c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:11.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.620+0000 7fb841f9c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c111fe0 con 0x7fb83c106a80 2024-09-11T23:26:11.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.620+0000 7fb841f9c640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c111e60 con 0x7fb83c101420 2024-09-11T23:26:11.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.620+0000 7fb841f9c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c112160 con 0x7fb83c100a80 2024-09-11T23:26:11.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.620+0000 7fb83affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c101420 0x7fb83c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:11.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.620+0000 7fb83b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb83c100a80 0x7fb83c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:11.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.620+0000 7fb83bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb83c106a80 0x7fb83c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:11.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.620+0000 7fb83affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c101420 0x7fb83c19ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:46052/0 (socket says 172.21.15.43:46052) 2024-09-11T23:26:11.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.620+0000 7fb83affd640 1 -- 172.21.15.43:0/885023220 learned_addr learned my addr 172.21.15.43:0/885023220 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:11.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.620+0000 7fb83b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb83c100a80 0x7fb83c19a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:38004/0 (socket says 172.21.15.43:38004) 2024-09-11T23:26:11.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.620+0000 7fb83affd640 1 -- 172.21.15.43:0/885023220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb83c100a80 msgr2=0x7fb83c19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:11.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.620+0000 7fb83affd640 1 --2- 172.21.15.43:0/885023220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb83c100a80 0x7fb83c19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.620+0000 7fb83affd640 1 -- 172.21.15.43:0/885023220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb83c106a80 msgr2=0x7fb83c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:11.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.620+0000 7fb83affd640 1 --2- 172.21.15.43:0/885023220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb83c106a80 0x7fb83c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.620+0000 7fb83affd640 1 -- 172.21.15.43:0/885023220 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83c19f600 con 0x7fb83c101420 2024-09-11T23:26:11.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.620+0000 7fb83bfff640 1 --2- 172.21.15.43:0/885023220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb83c106a80 0x7fb83c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:26:11.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.620+0000 7fb83b7fe640 1 --2- 172.21.15.43:0/885023220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb83c100a80 0x7fb83c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:26:11.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.621+0000 7fb83affd640 1 --2- 172.21.15.43:0/885023220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c101420 0x7fb83c19ab90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb83000e9f0 tx=0x7fb83000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:11.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.621+0000 7fb838ff9640 1 -- 172.21.15.43:0/885023220 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83000cd80 con 0x7fb83c101420 2024-09-11T23:26:11.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.621+0000 7fb838ff9640 1 -- 172.21.15.43:0/885023220 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb830010910 con 0x7fb83c101420 2024-09-11T23:26:11.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.621+0000 7fb841f9c640 1 -- 172.21.15.43:0/885023220 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb83c19f8f0 con 0x7fb83c101420 2024-09-11T23:26:11.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.621+0000 7fb838ff9640 1 -- 172.21.15.43:0/885023220 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830018990 con 0x7fb83c101420 2024-09-11T23:26:11.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.621+0000 7fb841f9c640 1 -- 172.21.15.43:0/885023220 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb83c1a71d0 con 0x7fb83c101420 2024-09-11T23:26:11.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.623+0000 7fb838ff9640 1 -- 172.21.15.43:0/885023220 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb830018b30 con 0x7fb83c101420 2024-09-11T23:26:11.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.624+0000 7fb844375740 1 Processor -- start 2024-09-11T23:26:11.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.624+0000 7fb844375740 1 -- start start 2024-09-11T23:26:11.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.624+0000 7fb838ff9640 1 --2- 172.21.15.43:0/885023220 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb8240a2d70 0x7fb8240a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:11.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.624+0000 7fb844375740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b204a0 0x562955b20880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:11.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.624+0000 7fb844375740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b20dc0 0x562955b2b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:11.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.624+0000 7fb844375740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b2b680 0x562955b2da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:11.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.624+0000 7fb844375740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562955a862e0 con 0x562955b20dc0 2024-09-11T23:26:11.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.624+0000 7fb844375740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562955a2dce0 con 0x562955b2b680 2024-09-11T23:26:11.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.624+0000 7fb844375740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562955a822a0 con 0x562955b204a0 2024-09-11T23:26:11.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.624+0000 7fb83bfff640 1 --2- 172.21.15.43:0/885023220 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb8240a2d70 0x7fb8240a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:11.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.624+0000 7fb838ff9640 1 -- 172.21.15.43:0/885023220 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7fb8300cf780 con 0x7fb83c101420 2024-09-11T23:26:11.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.624+0000 7fb83b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b204a0 0x562955b20880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.624+0000 7fb83b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b204a0 0x562955b20880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:38016/0 (socket says 172.21.15.43:38016) 2024-09-11T23:26:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.624+0000 7fb83b7fe640 1 -- 172.21.15.43:0/2557807511 learned_addr learned my addr 172.21.15.43:0/2557807511 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.624+0000 7fb83affd640 1 --2- 172.21.15.43:0/2557807511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b20dc0 0x562955b2b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.624+0000 7fb83bfff640 1 --2- 172.21.15.43:0/2557807511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b2b680 0x562955b2da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.624+0000 7fb83b7fe640 1 -- 172.21.15.43:0/2557807511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b2b680 msgr2=0x562955b2da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.624+0000 7fb83b7fe640 1 --2- 172.21.15.43:0/2557807511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b2b680 0x562955b2da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.624+0000 7fb83b7fe640 1 -- 172.21.15.43:0/2557807511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b20dc0 msgr2=0x562955b2b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.624+0000 7fb83b7fe640 1 --2- 172.21.15.43:0/2557807511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b20dc0 0x562955b2b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.624+0000 7fb83b7fe640 1 -- 172.21.15.43:0/2557807511 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562955a12440 con 0x562955b204a0 2024-09-11T23:26:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.625+0000 7fb83bfff640 1 --2- 172.21.15.43:0/885023220 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb8240a2d70 0x7fb8240a5230 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb834009fd0 tx=0x7fb834011f60 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.625+0000 7fb83b7fe640 1 --2- 172.21.15.43:0/2557807511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b204a0 0x562955b20880 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009930 tx=0x7fb82c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5f2c31e92086e43 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.625+0000 7fb821ffb640 1 -- 172.21.15.43:0/2557807511 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c035040 con 0x562955b204a0 2024-09-11T23:26:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.625+0000 7fb821ffb640 1 -- 172.21.15.43:0/2557807511 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb82c005520 con 0x562955b204a0 2024-09-11T23:26:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.625+0000 7fb821ffb640 1 -- 172.21.15.43:0/2557807511 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c0038c0 con 0x562955b204a0 2024-09-11T23:26:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.626+0000 7fb844375740 1 -- 172.21.15.43:0/2557807511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b204a0 msgr2=0x562955b20880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.626+0000 7fb844375740 1 --2- 172.21.15.43:0/2557807511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b204a0 0x562955b20880 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009930 tx=0x7fb82c000bc0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.626+0000 7fb844375740 1 -- 172.21.15.43:0/2557807511 shutdown_connections 2024-09-11T23:26:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.626+0000 7fb844375740 1 --2- 172.21.15.43:0/2557807511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b2b680 0x562955b2da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.626+0000 7fb844375740 1 --2- 172.21.15.43:0/2557807511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b20dc0 0x562955b2b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.626+0000 7fb844375740 1 --2- 172.21.15.43:0/2557807511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b204a0 0x562955b20880 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.626+0000 7fb844375740 1 -- 172.21.15.43:0/2557807511 >> 172.21.15.43:0/2557807511 conn(0x562955b1ca30 msgr2=0x562955b1ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.626+0000 7fb844375740 1 -- 172.21.15.43:0/2557807511 shutdown_connections 2024-09-11T23:26:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.626+0000 7fb844375740 1 -- 172.21.15.43:0/2557807511 wait complete. 2024-09-11T23:26:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.626+0000 7fb844375740 1 -- start start 2024-09-11T23:26:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.626+0000 7fb844375740 1 Processor -- start 2024-09-11T23:26:11.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb844375740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b204a0 0x562955b2afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:11.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb844375740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b20dc0 0x562955b28270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:11.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb844375740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b2b680 0x562955b287b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:11.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb844375740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562955b2e250 con 0x562955b204a0 2024-09-11T23:26:11.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb844375740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562955a41050 con 0x562955b20dc0 2024-09-11T23:26:11.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb844375740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562955b2e3d0 con 0x562955b2b680 2024-09-11T23:26:11.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb844375740 10 client.0 _ll_register_callbacks cb 0x562955a81160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:11.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb83affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b20dc0 0x562955b28270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:11.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb83b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b204a0 0x562955b2afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:11.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb83bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b2b680 0x562955b287b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:11.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb83affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b20dc0 0x562955b28270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:46074/0 (socket says 172.21.15.43:46074) 2024-09-11T23:26:11.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb83b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b204a0 0x562955b2afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48590/0 (socket says 172.21.15.43:48590) 2024-09-11T23:26:11.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb83affd640 1 -- 172.21.15.43:0/1993721815 learned_addr learned my addr 172.21.15.43:0/1993721815 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:11.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb83affd640 1 -- 172.21.15.43:0/1993721815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b2b680 msgr2=0x562955b287b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:11.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb83affd640 1 --2- 172.21.15.43:0/1993721815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b2b680 0x562955b287b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb83affd640 1 -- 172.21.15.43:0/1993721815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b204a0 msgr2=0x562955b2afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:11.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb83affd640 1 --2- 172.21.15.43:0/1993721815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b204a0 0x562955b2afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb83affd640 1 -- 172.21.15.43:0/1993721815 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629557f73c0 con 0x562955b20dc0 2024-09-11T23:26:11.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb83b7fe640 1 --2- 172.21.15.43:0/1993721815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b204a0 0x562955b2afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:26:11.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb83affd640 1 --2- 172.21.15.43:0/1993721815 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b20dc0 0x562955b28270 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb830098150 tx=0x7fb83009bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8175a3d11ed79dc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:11.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb81affd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:26:11.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb81affd640 1 -- 172.21.15.43:0/1993721815 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830098360 con 0x562955b20dc0 2024-09-11T23:26:11.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb81affd640 1 -- 172.21.15.43:0/1993721815 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7fb83009b140 con 0x562955b20dc0 2024-09-11T23:26:11.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.627+0000 7fb844375740 1 -- 172.21.15.43:0/1993721815 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb82c005520 con 0x562955b20dc0 2024-09-11T23:26:11.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.628+0000 7fb81affd640 1 -- 172.21.15.43:0/1993721815 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb830098500 con 0x562955b20dc0 2024-09-11T23:26:11.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.628+0000 7fb844375740 10 client.25670 fetch_fsmap learned FSMap version 30 2024-09-11T23:26:11.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.628+0000 7fb844375740 1 -- 172.21.15.43:0/1993721815 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562955b39c60 con 0x562955b20dc0 2024-09-11T23:26:11.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.628+0000 7fb81affd640 1 -- 172.21.15.43:0/1993721815 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb82c005520 con 0x562955b20dc0 2024-09-11T23:26:11.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.628+0000 7fb81affd640 1 -- 172.21.15.43:0/1993721815 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb8300ba2c0 con 0x562955b20dc0 2024-09-11T23:26:11.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.628+0000 7fb844375740 10 client.25670 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:26:11.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.628+0000 7fb844375740 10 client.25670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:26:11.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.628+0000 7fb844375740 20 client.25670 populate_metadata read hostname 'smithi043' 2024-09-11T23:26:11.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.628+0000 7fb844375740 1 --2- 172.21.15.43:0/1993721815 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562955b3b120 0x562955b3d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:11.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.628+0000 7fb844375740 4 client.25670 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:26:11.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.628+0000 7fb844375740 1 -- 172.21.15.43:0/1993721815 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562955a93850 con 0x562955b3b120 2024-09-11T23:26:11.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.628+0000 7fb83b7fe640 1 --2- 172.21.15.43:0/1993721815 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562955b3b120 0x562955b3d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:11.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.629+0000 7fb81affd640 10 client.25670 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:11.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.629+0000 7fb83b7fe640 1 --2- 172.21.15.43:0/1993721815 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562955b3b120 0x562955b3d530 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3823e76a9a00ac9 server_cookie=1e02cd7d1bf6ff37 in_seq=0 out_seq=0 2024-09-11T23:26:11.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.629+0000 7fb81affd640 1 -- 172.21.15.43:0/1993721815 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562955a93850 con 0x562955b3b120 2024-09-11T23:26:11.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.629+0000 7fb81affd640 10 client.25670 handle_command_reply: tid=0 2024-09-11T23:26:11.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.629+0000 7fb844375740 1 client.25670 shutdown 2024-09-11T23:26:11.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.629+0000 7fb844375740 1 -- 172.21.15.43:0/1993721815 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b20dc0 msgr2=0x562955b28270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:11.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.629+0000 7fb844375740 1 --2- 172.21.15.43:0/1993721815 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b20dc0 0x562955b28270 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb830098150 tx=0x7fb83009bd70 comp rx=0 tx=0).stop 2024-09-11T23:26:11.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.629+0000 7fb844375740 1 -- 172.21.15.43:0/1993721815 shutdown_connections 2024-09-11T23:26:11.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.629+0000 7fb844375740 1 --2- 172.21.15.43:0/1993721815 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562955b3b120 0x562955b3d530 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.629+0000 7fb844375740 1 --2- 172.21.15.43:0/1993721815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b2b680 0x562955b287b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.629+0000 7fb844375740 1 --2- 172.21.15.43:0/1993721815 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b20dc0 0x562955b28270 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.629+0000 7fb844375740 1 --2- 172.21.15.43:0/1993721815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b204a0 0x562955b2afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.629+0000 7fb81affd640 0 client.25670 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:11.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.629+0000 7fb844375740 1 -- 172.21.15.43:0/1993721815 >> 172.21.15.43:0/1993721815 conn(0x562955b1ca30 msgr2=0x562955b1bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:11.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.630+0000 7fb844375740 1 -- 172.21.15.43:0/1993721815 shutdown_connections 2024-09-11T23:26:11.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.630+0000 7fb844375740 1 -- 172.21.15.43:0/1993721815 wait complete. 2024-09-11T23:26:11.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.630+0000 7fb844375740 20 client.25670 trim_cache size 0 max 16384 2024-09-11T23:26:11.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.662+0000 7fb844375740 1 Processor -- start 2024-09-11T23:26:11.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.662+0000 7fb844375740 1 -- start start 2024-09-11T23:26:11.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.663+0000 7fb844375740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b1ca30 0x562955b2a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:11.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.663+0000 7fb844375740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b204a0 0x562955b42e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:11.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.663+0000 7fb844375740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b20dc0 0x562955b43340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:11.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.663+0000 7fb844375740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c1b2410 con 0x562955b20dc0 2024-09-11T23:26:11.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.663+0000 7fb844375740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c1afce0 con 0x562955b1ca30 2024-09-11T23:26:11.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.663+0000 7fb844375740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c1adb30 con 0x562955b204a0 2024-09-11T23:26:11.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.663+0000 7fb83bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b20dc0 0x562955b43340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:11.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.663+0000 7fb83bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b20dc0 0x562955b43340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48600/0 (socket says 172.21.15.43:48600) 2024-09-11T23:26:11.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.663+0000 7fb83bfff640 1 -- 172.21.15.43:0/168970216 learned_addr learned my addr 172.21.15.43:0/168970216 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:11.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.665+0000 7fb83bfff640 1 -- 172.21.15.43:0/168970216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b204a0 msgr2=0x562955b42e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:26:11.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.665+0000 7fb83b7fe640 1 --2- 172.21.15.43:0/168970216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b1ca30 0x562955b2a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:11.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.665+0000 7fb83bfff640 1 --2- 172.21.15.43:0/168970216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b204a0 0x562955b42e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.665+0000 7fb83bfff640 1 -- 172.21.15.43:0/168970216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b1ca30 msgr2=0x562955b2a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:11.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.665+0000 7fb83bfff640 1 --2- 172.21.15.43:0/168970216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b1ca30 0x562955b2a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.665+0000 7fb83bfff640 1 -- 172.21.15.43:0/168970216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562955b20bf0 con 0x562955b20dc0 2024-09-11T23:26:11.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.665+0000 7fb83bfff640 1 --2- 172.21.15.43:0/168970216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b20dc0 0x562955b43340 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb83400fec0 tx=0x7fb834009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4e8851b475cbc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:11.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.667+0000 7fb8217fa640 1 -- 172.21.15.43:0/168970216 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83401a040 con 0x562955b20dc0 2024-09-11T23:26:11.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.667+0000 7fb8217fa640 1 -- 172.21.15.43:0/168970216 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb834004730 con 0x562955b20dc0 2024-09-11T23:26:11.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.667+0000 7fb8217fa640 1 -- 172.21.15.43:0/168970216 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8340426a0 con 0x562955b20dc0 2024-09-11T23:26:11.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.668+0000 7fb844375740 1 -- 172.21.15.43:0/168970216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b20dc0 msgr2=0x562955b43340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:11.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.668+0000 7fb844375740 1 --2- 172.21.15.43:0/168970216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b20dc0 0x562955b43340 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb83400fec0 tx=0x7fb834009500 comp rx=0 tx=0).stop 2024-09-11T23:26:11.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.668+0000 7fb844375740 1 -- 172.21.15.43:0/168970216 shutdown_connections 2024-09-11T23:26:11.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.668+0000 7fb844375740 1 --2- 172.21.15.43:0/168970216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b20dc0 0x562955b43340 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.668+0000 7fb844375740 1 --2- 172.21.15.43:0/168970216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b204a0 0x562955b42e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.668+0000 7fb844375740 1 --2- 172.21.15.43:0/168970216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b1ca30 0x562955b2a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.668+0000 7fb844375740 1 -- 172.21.15.43:0/168970216 >> 172.21.15.43:0/168970216 conn(0x562955b28910 msgr2=0x562955b29ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.671+0000 7fb844375740 1 -- 172.21.15.43:0/168970216 shutdown_connections 2024-09-11T23:26:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.671+0000 7fb844375740 1 -- 172.21.15.43:0/168970216 wait complete. 2024-09-11T23:26:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.672+0000 7fb844375740 1 -- start start 2024-09-11T23:26:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.672+0000 7fb844375740 1 Processor -- start 2024-09-11T23:26:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.672+0000 7fb844375740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b1ca30 0x562955b312b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.673+0000 7fb844375740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b204a0 0x562955b317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.673+0000 7fb844375740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b20dc0 0x562955b31d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.673+0000 7fb844375740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c1ae1a0 con 0x562955b204a0 2024-09-11T23:26:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.673+0000 7fb844375740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c1af610 con 0x562955b20dc0 2024-09-11T23:26:11.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.673+0000 7fb844375740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c1ae870 con 0x562955b1ca30 2024-09-11T23:26:11.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.673+0000 7fb844375740 10 client.0 _ll_register_callbacks cb 0x7fb83009bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:11.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.673+0000 7fb83b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b1ca30 0x562955b312b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:11.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.673+0000 7fb83b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b1ca30 0x562955b312b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:38032/0 (socket says 172.21.15.43:38032) 2024-09-11T23:26:11.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.673+0000 7fb83b7fe640 1 -- 172.21.15.43:0/2079649853 learned_addr learned my addr 172.21.15.43:0/2079649853 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:11.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.673+0000 7fb83bfff640 1 --2- 172.21.15.43:0/2079649853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b20dc0 0x562955b31d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:11.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.673+0000 7fb83b7fe640 1 -- 172.21.15.43:0/2079649853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b20dc0 msgr2=0x562955b31d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:11.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.673+0000 7fb83b7fe640 1 --2- 172.21.15.43:0/2079649853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b20dc0 0x562955b31d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.673+0000 7fb83b7fe640 1 -- 172.21.15.43:0/2079649853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b204a0 msgr2=0x562955b317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:11.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.673+0000 7fb83b7fe640 1 --2- 172.21.15.43:0/2079649853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b204a0 0x562955b317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.674+0000 7fb83b7fe640 1 -- 172.21.15.43:0/2079649853 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562955bb5df0 con 0x562955b1ca30 2024-09-11T23:26:11.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.674+0000 7fb83b7fe640 1 --2- 172.21.15.43:0/2079649853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b1ca30 0x562955b312b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb82c02aa10 tx=0x7fb82c004eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aa9f16b0baadd1d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:11.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.675+0000 7fb81affd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-11T23:26:11.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.675+0000 7fb81affd640 1 -- 172.21.15.43:0/2079649853 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c035030 con 0x562955b1ca30 2024-09-11T23:26:11.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.675+0000 7fb81affd640 1 -- 172.21.15.43:0/2079649853 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7fb82c004bf0 con 0x562955b1ca30 2024-09-11T23:26:11.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.675+0000 7fb844375740 1 -- 172.21.15.43:0/2079649853 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb834004730 con 0x562955b1ca30 2024-09-11T23:26:11.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.675+0000 7fb81affd640 1 -- 172.21.15.43:0/2079649853 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb82c060050 con 0x562955b1ca30 2024-09-11T23:26:11.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.678+0000 7fb844375740 10 client.25645 fetch_fsmap learned FSMap version 30 2024-09-11T23:26:11.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.678+0000 7fb844375740 1 -- 172.21.15.43:0/2079649853 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562955bb6ff0 con 0x562955b1ca30 2024-09-11T23:26:11.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.679+0000 7fb81affd640 1 -- 172.21.15.43:0/2079649853 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb834004730 con 0x562955b1ca30 2024-09-11T23:26:11.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.679+0000 7fb81affd640 1 -- 172.21.15.43:0/2079649853 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb82c003b90 con 0x562955b1ca30 2024-09-11T23:26:11.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.680+0000 7fb844375740 10 client.25645 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:26:11.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.680+0000 7fb844375740 10 client.25645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:26:11.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.680+0000 7fb844375740 20 client.25645 populate_metadata read hostname 'smithi043' 2024-09-11T23:26:11.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.680+0000 7fb844375740 1 --2- 172.21.15.43:0/2079649853 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562955b2b680 0x562955bb9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:11.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.680+0000 7fb844375740 4 client.25645 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:26:11.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.680+0000 7fb844375740 1 -- 172.21.15.43:0/2079649853 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562955a8dd60 con 0x562955b2b680 2024-09-11T23:26:11.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.680+0000 7fb83affd640 1 --2- 172.21.15.43:0/2079649853 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562955b2b680 0x562955bb9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:11.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.684+0000 7fb83affd640 1 --2- 172.21.15.43:0/2079649853 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562955b2b680 0x562955bb9750 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=758424ff1722ef7f server_cookie=4e300272ab22a365 in_seq=0 out_seq=0 2024-09-11T23:26:11.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.685+0000 7fb81affd640 10 client.25645 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:11.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.685+0000 7fb81affd640 1 -- 172.21.15.43:0/2079649853 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x562955a8dd60 con 0x562955b2b680 2024-09-11T23:26:11.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.685+0000 7fb81affd640 10 client.25645 handle_command_reply: tid=0 2024-09-11T23:26:11.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.685+0000 7fb844375740 1 client.25645 shutdown 2024-09-11T23:26:11.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.687+0000 7fb844375740 1 -- 172.21.15.43:0/2079649853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b1ca30 msgr2=0x562955b312b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:11.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.687+0000 7fb844375740 1 --2- 172.21.15.43:0/2079649853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b1ca30 0x562955b312b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb82c02aa10 tx=0x7fb82c004eb0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.691+0000 7fb844375740 1 -- 172.21.15.43:0/2079649853 shutdown_connections 2024-09-11T23:26:11.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.691+0000 7fb844375740 1 --2- 172.21.15.43:0/2079649853 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562955b2b680 0x562955bb9750 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.691+0000 7fb844375740 1 --2- 172.21.15.43:0/2079649853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562955b20dc0 0x562955b31d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.691+0000 7fb844375740 1 --2- 172.21.15.43:0/2079649853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562955b204a0 0x562955b317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.691+0000 7fb844375740 1 --2- 172.21.15.43:0/2079649853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562955b1ca30 0x562955b312b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.691+0000 7fb844375740 1 -- 172.21.15.43:0/2079649853 >> 172.21.15.43:0/2079649853 conn(0x562955b28910 msgr2=0x562955b2a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:11.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.691+0000 7fb81affd640 0 client.25645 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:11.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.691+0000 7fb844375740 1 -- 172.21.15.43:0/2079649853 shutdown_connections 2024-09-11T23:26:11.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.692+0000 7fb844375740 1 -- 172.21.15.43:0/2079649853 wait complete. 2024-09-11T23:26:11.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.692+0000 7fb844375740 20 client.25645 trim_cache size 0 max 16384 2024-09-11T23:26:11.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.695+0000 7fb841f9c640 1 -- 172.21.15.43:0/885023220 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb8240a2d70 msgr2=0x7fb8240a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:11.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.695+0000 7fb841f9c640 1 --2- 172.21.15.43:0/885023220 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb8240a2d70 0x7fb8240a5230 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb834009fd0 tx=0x7fb834011f60 comp rx=0 tx=0).stop 2024-09-11T23:26:11.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.695+0000 7fb841f9c640 1 -- 172.21.15.43:0/885023220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c101420 msgr2=0x7fb83c19ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:11.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.695+0000 7fb841f9c640 1 --2- 172.21.15.43:0/885023220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c101420 0x7fb83c19ab90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb83000e9f0 tx=0x7fb83000eeb0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.695+0000 7fb841f9c640 1 -- 172.21.15.43:0/885023220 shutdown_connections 2024-09-11T23:26:11.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.695+0000 7fb841f9c640 1 --2- 172.21.15.43:0/885023220 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fb8240a2d70 0x7fb8240a5230 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.695+0000 7fb841f9c640 1 --2- 172.21.15.43:0/885023220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb83c106a80 0x7fb83c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.695+0000 7fb841f9c640 1 --2- 172.21.15.43:0/885023220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c101420 0x7fb83c19ab90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.695+0000 7fb841f9c640 1 --2- 172.21.15.43:0/885023220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb83c100a80 0x7fb83c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:11.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.695+0000 7fb841f9c640 1 -- 172.21.15.43:0/885023220 >> 172.21.15.43:0/885023220 conn(0x7fb83c0fc7a0 msgr2=0x7fb83c10c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:11.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.696+0000 7fb841f9c640 1 -- 172.21.15.43:0/885023220 shutdown_connections 2024-09-11T23:26:11.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:10.698+0000 7fb841f9c640 1 -- 172.21.15.43:0/885023220 wait complete. 2024-09-11T23:26:12.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:11 smithi090 ceph-mon[35130]: pgmap v134: 97 pgs: 97 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 75 MiB/s wr, 11.07k op/s 2024-09-11T23:26:12.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:11 smithi097 ceph-mon[35562]: pgmap v134: 97 pgs: 97 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 75 MiB/s wr, 11.07k op/s 2024-09-11T23:26:12.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:11 smithi043 ceph-mon[30950]: pgmap v134: 97 pgs: 97 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 75 MiB/s wr, 11.07k op/s 2024-09-11T23:26:14.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:13 smithi090 ceph-mon[35130]: pgmap v135: 97 pgs: 97 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 78 MiB/s wr, 11.51k op/s 2024-09-11T23:26:14.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:13 smithi097 ceph-mon[35562]: pgmap v135: 97 pgs: 97 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 78 MiB/s wr, 11.51k op/s 2024-09-11T23:26:14.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:13 smithi043 ceph-mon[30950]: pgmap v135: 97 pgs: 97 active+clean; 17 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 78 MiB/s wr, 11.51k op/s 2024-09-11T23:26:16.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:15 smithi090 ceph-mon[35130]: pgmap v136: 97 pgs: 97 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 78 MiB/s wr, 11.71k op/s 2024-09-11T23:26:16.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:15 smithi097 ceph-mon[35562]: pgmap v136: 97 pgs: 97 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 78 MiB/s wr, 11.71k op/s 2024-09-11T23:26:16.104 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:15 smithi043 ceph-mon[30950]: pgmap v136: 97 pgs: 97 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 78 MiB/s wr, 11.71k op/s 2024-09-11T23:26:18.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:17 smithi090 ceph-mon[35130]: pgmap v137: 97 pgs: 97 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 65 MiB/s wr, 10.03k op/s 2024-09-11T23:26:18.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:17 smithi097 ceph-mon[35562]: pgmap v137: 97 pgs: 97 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 65 MiB/s wr, 10.03k op/s 2024-09-11T23:26:18.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:17 smithi043 ceph-mon[30950]: pgmap v137: 97 pgs: 97 active+clean; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 65 MiB/s wr, 10.03k op/s 2024-09-11T23:26:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.016+0000 7f0434ff9640 1 -- 172.21.15.43:0/4252913101 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5624e6f63d70 con 0x5624e7086c30 2024-09-11T23:26:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.016+0000 7f0434ff9640 10 client.16116 handle_command_reply: tid=0 2024-09-11T23:26:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.016+0000 7f045665e740 1 client.16116 shutdown 2024-09-11T23:26:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.016+0000 7f045665e740 1 -- 172.21.15.43:0/4252913101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e7002cb0 msgr2=0x5624e700f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.016+0000 7f045665e740 1 --2- 172.21.15.43:0/4252913101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e7002cb0 0x5624e700f660 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0444097ff0 tx=0x7f044409b0b0 comp rx=0 tx=0).stop 2024-09-11T23:26:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.016+0000 7f045665e740 1 -- 172.21.15.43:0/4252913101 shutdown_connections 2024-09-11T23:26:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.016+0000 7f045665e740 1 --2- 172.21.15.43:0/4252913101 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5624e7086c30 0x5624e7089020 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.016+0000 7f045665e740 1 --2- 172.21.15.43:0/4252913101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5624e7002cb0 0x5624e700f660 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.016+0000 7f045665e740 1 --2- 172.21.15.43:0/4252913101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5624e6ff8410 0x5624e700f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.016+0000 7f045665e740 1 --2- 172.21.15.43:0/4252913101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624e6ff7af0 0x5624e7013d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.016+0000 7f045665e740 1 -- 172.21.15.43:0/4252913101 >> 172.21.15.43:0/4252913101 conn(0x5624e6ff4060 msgr2=0x5624e6ff95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.016+0000 7f0434ff9640 0 client.16116 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.016+0000 7f045665e740 1 -- 172.21.15.43:0/4252913101 shutdown_connections 2024-09-11T23:26:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.016+0000 7f045665e740 1 -- 172.21.15.43:0/4252913101 wait complete. 2024-09-11T23:26:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.016+0000 7f045665e740 20 client.16116 trim_cache size 0 max 16384 2024-09-11T23:26:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:26:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:26:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:26:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:26:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.017+0000 7f04538c9640 1 -- 172.21.15.43:0/315114390 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f04380a50d0 msgr2=0x7f04380a7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.017+0000 7f04538c9640 1 --2- 172.21.15.43:0/315114390 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f04380a50d0 0x7f04380a7590 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f044802a940 tx=0x7f0448000bc0 comp rx=0 tx=0).stop 2024-09-11T23:26:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.017+0000 7f04538c9640 1 -- 172.21.15.43:0/315114390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f044c106a00 msgr2=0x7f044c06d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.017+0000 7f04538c9640 1 --2- 172.21.15.43:0/315114390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f044c106a00 0x7f044c06d080 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0444009fd0 tx=0x7f044400b260 comp rx=0 tx=0).stop 2024-09-11T23:26:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.017+0000 7f04538c9640 1 -- 172.21.15.43:0/315114390 shutdown_connections 2024-09-11T23:26:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.017+0000 7f04538c9640 1 --2- 172.21.15.43:0/315114390 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f04380a50d0 0x7f04380a7590 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.017+0000 7f04538c9640 1 --2- 172.21.15.43:0/315114390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f044c106a00 0x7f044c06d080 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.017+0000 7f04538c9640 1 --2- 172.21.15.43:0/315114390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f044c101390 0x7f044c06cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.017+0000 7f04538c9640 1 --2- 172.21.15.43:0/315114390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f044c1009f0 0x7f044c073ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.017+0000 7f04538c9640 1 -- 172.21.15.43:0/315114390 >> 172.21.15.43:0/315114390 conn(0x7f044c0fc6b0 msgr2=0x7f044c0fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.017+0000 7f04538c9640 1 -- 172.21.15.43:0/315114390 shutdown_connections 2024-09-11T23:26:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:20.017+0000 7f04538c9640 1 -- 172.21.15.43:0/315114390 wait complete. 2024-09-11T23:26:20.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:19 smithi090 ceph-mon[35130]: pgmap v138: 97 pgs: 97 active+clean; 18 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 70 MiB/s wr, 10.55k op/s 2024-09-11T23:26:20.050 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:19 smithi097 ceph-mon[35562]: pgmap v138: 97 pgs: 97 active+clean; 18 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 70 MiB/s wr, 10.55k op/s 2024-09-11T23:26:20.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:19 smithi043 ceph-mon[30950]: pgmap v138: 97 pgs: 97 active+clean; 18 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 70 MiB/s wr, 10.55k op/s 2024-09-11T23:26:21.752 INFO:tasks.cephfs.filesystem:scrub status for tag:ddf9b449-9095-42dd-a28c-7f5af91a6fc0 - {'path': '/', 'tag': 'ddf9b449-9095-42dd-a28c-7f5af91a6fc0', 'options': 'recursive,force'} 2024-09-11T23:26:22.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:21 smithi090 ceph-mon[35130]: pgmap v139: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 MiB/s wr, 7.76k op/s 2024-09-11T23:26:22.115 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:21 smithi043 ceph-mon[30950]: pgmap v139: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 MiB/s wr, 7.76k op/s 2024-09-11T23:26:22.116 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:21 smithi097 ceph-mon[35562]: pgmap v139: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 MiB/s wr, 7.76k op/s 2024-09-11T23:26:24.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:23 smithi090 ceph-mon[35130]: pgmap v140: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 44 MiB/s wr, 6.70k op/s 2024-09-11T23:26:24.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:23 smithi097 ceph-mon[35562]: pgmap v140: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 44 MiB/s wr, 6.70k op/s 2024-09-11T23:26:24.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:23 smithi043 ceph-mon[30950]: pgmap v140: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 44 MiB/s wr, 6.70k op/s 2024-09-11T23:26:26.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:25 smithi090 ceph-mon[35130]: pgmap v141: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 MiB/s wr, 6.39k op/s 2024-09-11T23:26:26.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:25 smithi097 ceph-mon[35562]: pgmap v141: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 MiB/s wr, 6.39k op/s 2024-09-11T23:26:26.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:25 smithi043 ceph-mon[30950]: pgmap v141: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 MiB/s wr, 6.39k op/s 2024-09-11T23:26:27.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:27 smithi043 ceph-mon[30950]: pgmap v142: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 38 MiB/s wr, 5.83k op/s 2024-09-11T23:26:28.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:27 smithi090 ceph-mon[35130]: pgmap v142: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 38 MiB/s wr, 5.83k op/s 2024-09-11T23:26:28.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:27 smithi097 ceph-mon[35562]: pgmap v142: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 38 MiB/s wr, 5.83k op/s 2024-09-11T23:26:30.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:29 smithi090 ceph-mon[35130]: pgmap v143: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 4.50k op/s 2024-09-11T23:26:30.031 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:29 smithi043 ceph-mon[30950]: pgmap v143: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 4.50k op/s 2024-09-11T23:26:30.050 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:29 smithi097 ceph-mon[35562]: pgmap v143: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 4.50k op/s 2024-09-11T23:26:30.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:30 smithi090 ceph-mon[35130]: scrub summary: active paths [/] 2024-09-11T23:26:30.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:30 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:26:30.770 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:26:30 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-b[65178]: 2024-09-11T23:26:30.449+0000 7fa495419640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000215 [...4,head] /volumes/qa/sv_1/2b469744-514b-43ef-9c93-a53c45581893/client.0/tmp/ffsb/.git/refs/ auth{1=1,2=1} v405 ap=3 RANDEPHEMERALPIN f(v1 m2024-09-11T23:23:15.144864+0000 3=0+3) n(v3 rc2024-09-11T23:23:15.250863+0000 b73 7=2+5) old_inodes=1 (inest sync->mix g=1,2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:768 nref=3)) caps={24820=pAsLsXs/-@18} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x55b6eefef600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000215:[<0x10000000008/refs v360>,<0x10000000007/.git v1663>,<0x10000000006/ffsb v1323>,<0x10000000005/tmp v15>,<0x100000001fb/client.0 v16>,<0x100000001fa/2b469744-514b-43ef-9c93-a53c45581893 v32>,<0x10000000001/sv_1 v54>,<0x10000000000/qa v47>,<0x1/volumes v44>]//[]","memoryvalue":"(2)0x10000000215:[<0x10000000008/refs v405>,<0x10000000007/.git v3113>,<0x10000000006/ffsb v2831>,<0x10000000005/tmp v310>,<0x100000001fb/client.0 v304>,<0x100000001fa/2b469744-514b-43ef-9c93-a53c45581893 v314>,<0x10000000001/sv_1 v320>,<0x10000000000/qa v313>,<0x1/volumes v298>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-11T23:23:14.524871+0000 2=0+2)","ondisk_value.rstat":"n(v0 rc2024-09-11T23:23:15.250863+0000 b73 7=2+5)","memory_value.dirstat":"f(v1 m2024-09-11T23:23:15.144864+0000 3=0+3)","memory_value.rstat":"n(v3 rc2024-09-11T23:23:15.250863+0000 b73 7=2+5)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-11T23:26:31.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:30 smithi097 ceph-mon[35562]: scrub summary: active paths [/] 2024-09-11T23:26:31.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:30 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:26:31.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:30 smithi043 ceph-mon[30950]: scrub summary: active paths [/] 2024-09-11T23:26:31.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:30 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:26:32.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:31 smithi090 ceph-mon[35130]: pgmap v144: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 MiB/s wr, 4.45k op/s 2024-09-11T23:26:32.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:31 smithi090 ceph-mon[35130]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/2b469744-514b-43ef-9c93-a53c45581893/client.0/tmp/ffsb/.git/refs) see mds.b log and `damage ls` output for details 2024-09-11T23:26:32.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:31 smithi090 ceph-mon[35130]: scrub summary: active paths [/] 2024-09-11T23:26:32.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:31 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:26:32.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:31 smithi090 ceph-mon[35130]: scrub summary: active paths [/] 2024-09-11T23:26:32.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:31 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:26:32.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:31 smithi097 ceph-mon[35562]: pgmap v144: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 MiB/s wr, 4.45k op/s 2024-09-11T23:26:32.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:31 smithi097 ceph-mon[35562]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/2b469744-514b-43ef-9c93-a53c45581893/client.0/tmp/ffsb/.git/refs) see mds.b log and `damage ls` output for details 2024-09-11T23:26:32.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:31 smithi097 ceph-mon[35562]: scrub summary: active paths [/] 2024-09-11T23:26:32.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:31 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:26:32.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:31 smithi097 ceph-mon[35562]: scrub summary: active paths [/] 2024-09-11T23:26:32.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:31 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:26:32.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:31 smithi043 ceph-mon[30950]: pgmap v144: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 MiB/s wr, 4.45k op/s 2024-09-11T23:26:32.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:31 smithi043 ceph-mon[30950]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/2b469744-514b-43ef-9c93-a53c45581893/client.0/tmp/ffsb/.git/refs) see mds.b log and `damage ls` output for details 2024-09-11T23:26:32.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:31 smithi043 ceph-mon[30950]: scrub summary: active paths [/] 2024-09-11T23:26:32.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:31 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:26:32.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:31 smithi043 ceph-mon[30950]: scrub summary: active paths [/] 2024-09-11T23:26:32.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:31 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:26:33.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:32 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:26:33.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:32 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:26:33.109 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:32 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:26:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.123+0000 7f1e65e4c640 1 -- 172.21.15.43:0/4085450386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e601069e0 msgr2=0x7f1e60106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.123+0000 7f1e65e4c640 1 --2- 172.21.15.43:0/4085450386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e601069e0 0x7f1e60106dc0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000fb40 tx=0x7f1e5002e5b0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.123+0000 7f1e65e4c640 1 -- 172.21.15.43:0/4085450386 shutdown_connections 2024-09-11T23:26:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.123+0000 7f1e65e4c640 1 --2- 172.21.15.43:0/4085450386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e60101370 0x7f1e6010d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.123+0000 7f1e65e4c640 1 --2- 172.21.15.43:0/4085450386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e601009d0 0x7f1e60100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.123+0000 7f1e65e4c640 1 --2- 172.21.15.43:0/4085450386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e601069e0 0x7f1e60106dc0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.123+0000 7f1e65e4c640 1 -- 172.21.15.43:0/4085450386 >> 172.21.15.43:0/4085450386 conn(0x7f1e600fc6b0 msgr2=0x7f1e600fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.123+0000 7f1e65e4c640 1 -- 172.21.15.43:0/4085450386 shutdown_connections 2024-09-11T23:26:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.123+0000 7f1e65e4c640 1 -- 172.21.15.43:0/4085450386 wait complete. 2024-09-11T23:26:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.123+0000 7f1e65e4c640 1 Processor -- start 2024-09-11T23:26:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.123+0000 7f1e65e4c640 1 -- start start 2024-09-11T23:26:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.124+0000 7f1e65e4c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e601009d0 0x7f1e60196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.124+0000 7f1e65e4c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e60101370 0x7f1e60196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.124+0000 7f1e65e4c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e601069e0 0x7f1e6019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.124+0000 7f1e65e4c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6010fd20 con 0x7f1e60101370 2024-09-11T23:26:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.124+0000 7f1e65e4c640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6010fba0 con 0x7f1e601009d0 2024-09-11T23:26:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.124+0000 7f1e65e4c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6010fea0 con 0x7f1e601069e0 2024-09-11T23:26:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.124+0000 7f1e5effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e60101370 0x7f1e60196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.124+0000 7f1e5effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e60101370 0x7f1e60196770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54158/0 (socket says 172.21.15.43:54158) 2024-09-11T23:26:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.124+0000 7f1e5f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e601009d0 0x7f1e60196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.124+0000 7f1e5effd640 1 -- 172.21.15.43:0/4061477917 learned_addr learned my addr 172.21.15.43:0/4061477917 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.124+0000 7f1e5ffff640 1 --2- 172.21.15.43:0/4061477917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e601069e0 0x7f1e6019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.124+0000 7f1e5effd640 1 -- 172.21.15.43:0/4061477917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e601069e0 msgr2=0x7f1e6019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.124+0000 7f1e5effd640 1 --2- 172.21.15.43:0/4061477917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e601069e0 0x7f1e6019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.124+0000 7f1e5effd640 1 -- 172.21.15.43:0/4061477917 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e601009d0 msgr2=0x7f1e60196230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.124+0000 7f1e5effd640 1 --2- 172.21.15.43:0/4061477917 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e601009d0 0x7f1e60196230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.124+0000 7f1e5effd640 1 -- 172.21.15.43:0/4061477917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6019b220 con 0x7f1e60101370 2024-09-11T23:26:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.124+0000 7f1e5effd640 1 --2- 172.21.15.43:0/4061477917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e60101370 0x7f1e60196770 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1e54007880 tx=0x7f1e54007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.124+0000 7f1e5cff9640 1 -- 172.21.15.43:0/4061477917 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54019030 con 0x7f1e60101370 2024-09-11T23:26:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.124+0000 7f1e65e4c640 1 -- 172.21.15.43:0/4061477917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6019b510 con 0x7f1e60101370 2024-09-11T23:26:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.124+0000 7f1e5cff9640 1 -- 172.21.15.43:0/4061477917 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e5400fe00 con 0x7f1e60101370 2024-09-11T23:26:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.124+0000 7f1e5cff9640 1 -- 172.21.15.43:0/4061477917 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54004ba0 con 0x7f1e60101370 2024-09-11T23:26:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.125+0000 7f1e65e4c640 1 -- 172.21.15.43:0/4061477917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e601a2da0 con 0x7f1e60101370 2024-09-11T23:26:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.127+0000 7f1e5cff9640 1 -- 172.21.15.43:0/4061477917 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1e5401c070 con 0x7f1e60101370 2024-09-11T23:26:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.127+0000 7f1e68be5740 1 Processor -- start 2024-09-11T23:26:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.127+0000 7f1e68be5740 1 -- start start 2024-09-11T23:26:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.128+0000 7f1e68be5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1b8af0 0x556bab1b8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.128+0000 7f1e68be5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1b9410 0x556bab1c3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.128+0000 7f1e68be5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1c3cb0 0x556bab1c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.128+0000 7f1e68be5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556bab11d2f0 con 0x556bab1b9410 2024-09-11T23:26:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.128+0000 7f1e68be5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556bab0c5270 con 0x556bab1b8af0 2024-09-11T23:26:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.128+0000 7f1e68be5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556bab1192b0 con 0x556bab1c3cb0 2024-09-11T23:26:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.128+0000 7f1e5f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1b9410 0x556bab1c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.128+0000 7f1e5f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1b9410 0x556bab1c3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54172/0 (socket says 172.21.15.43:54172) 2024-09-11T23:26:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.128+0000 7f1e5f7fe640 1 -- 172.21.15.43:0/2999513396 learned_addr learned my addr 172.21.15.43:0/2999513396 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.128+0000 7f1e5effd640 1 --2- 172.21.15.43:0/2999513396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1c3cb0 0x556bab1c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.128+0000 7f1e5ffff640 1 --2- 172.21.15.43:0/2999513396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1b8af0 0x556bab1b8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.128+0000 7f1e5cff9640 1 --2- 172.21.15.43:0/4061477917 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f1e480a2d70 0x7f1e480a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.128+0000 7f1e5ffff640 1 --2- 172.21.15.43:0/4061477917 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f1e480a2d70 0x7f1e480a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.128+0000 7f1e5f7fe640 1 -- 172.21.15.43:0/2999513396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1c3cb0 msgr2=0x556bab1c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.128+0000 7f1e5f7fe640 1 --2- 172.21.15.43:0/2999513396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1c3cb0 0x556bab1c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.128+0000 7f1e5f7fe640 1 -- 172.21.15.43:0/2999513396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1b8af0 msgr2=0x556bab1b8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.128+0000 7f1e5f7fe640 1 --2- 172.21.15.43:0/2999513396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1b8af0 0x556bab1b8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.128+0000 7f1e5f7fe640 1 -- 172.21.15.43:0/2999513396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bab0c4340 con 0x556bab1b9410 2024-09-11T23:26:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.128+0000 7f1e5cff9640 1 -- 172.21.15.43:0/4061477917 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f1e540cf420 con 0x7f1e60101370 2024-09-11T23:26:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.128+0000 7f1e5ffff640 1 --2- 172.21.15.43:0/2999513396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1b8af0 0x556bab1b8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:26:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.128+0000 7f1e5f7fe640 1 --2- 172.21.15.43:0/2999513396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1b9410 0x556bab1c3770 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000fb10 tx=0x7f1e50005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862b36a665771326 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.129+0000 7f1e5ffff640 1 --2- 172.21.15.43:0/4061477917 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f1e480a2d70 0x7f1e480a5230 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1e58009fd0 tx=0x7f1e58004ab0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.129+0000 7f1e45ffb640 1 -- 172.21.15.43:0/2999513396 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5000ec70 con 0x556bab1b9410 2024-09-11T23:26:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.129+0000 7f1e45ffb640 1 -- 172.21.15.43:0/2999513396 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e50045070 con 0x556bab1b9410 2024-09-11T23:26:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.129+0000 7f1e45ffb640 1 -- 172.21.15.43:0/2999513396 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e500042a0 con 0x556bab1b9410 2024-09-11T23:26:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.129+0000 7f1e68be5740 1 -- 172.21.15.43:0/2999513396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1b9410 msgr2=0x556bab1c3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.129+0000 7f1e68be5740 1 --2- 172.21.15.43:0/2999513396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1b9410 0x556bab1c3770 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000fb10 tx=0x7f1e50005e80 comp rx=0 tx=0).stop 2024-09-11T23:26:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.129+0000 7f1e68be5740 1 -- 172.21.15.43:0/2999513396 shutdown_connections 2024-09-11T23:26:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.129+0000 7f1e68be5740 1 --2- 172.21.15.43:0/2999513396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1c3cb0 0x556bab1c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.129+0000 7f1e68be5740 1 --2- 172.21.15.43:0/2999513396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1b9410 0x556bab1c3770 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.129+0000 7f1e68be5740 1 --2- 172.21.15.43:0/2999513396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1b8af0 0x556bab1b8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.129+0000 7f1e68be5740 1 -- 172.21.15.43:0/2999513396 >> 172.21.15.43:0/2999513396 conn(0x556bab1b5060 msgr2=0x556bab1b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.129+0000 7f1e68be5740 1 -- 172.21.15.43:0/2999513396 shutdown_connections 2024-09-11T23:26:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.129+0000 7f1e68be5740 1 -- 172.21.15.43:0/2999513396 wait complete. 2024-09-11T23:26:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.130+0000 7f1e68be5740 1 -- start start 2024-09-11T23:26:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.130+0000 7f1e68be5740 1 Processor -- start 2024-09-11T23:26:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.130+0000 7f1e68be5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1b8af0 0x556bab1c0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.130+0000 7f1e68be5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1b9410 0x556bab1c10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.130+0000 7f1e68be5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1c3cb0 0x556bab1c15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.130+0000 7f1e68be5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556bab1c6880 con 0x556bab1b8af0 2024-09-11T23:26:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.130+0000 7f1e68be5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556bab0a6290 con 0x556bab1c3cb0 2024-09-11T23:26:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.130+0000 7f1e68be5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556bab1c6a00 con 0x556bab1b9410 2024-09-11T23:26:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.130+0000 7f1e68be5740 10 client.0 _ll_register_callbacks cb 0x556bab0d7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.130+0000 7f1e5f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1b8af0 0x556bab1c0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.130+0000 7f1e5f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1b8af0 0x556bab1c0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54174/0 (socket says 172.21.15.43:54174) 2024-09-11T23:26:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.130+0000 7f1e5f7fe640 1 -- 172.21.15.43:0/2127033172 learned_addr learned my addr 172.21.15.43:0/2127033172 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.130+0000 7f1e5effd640 1 --2- 172.21.15.43:0/2127033172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1b9410 0x556bab1c10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.130+0000 7f1e5ffff640 1 --2- 172.21.15.43:0/2127033172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1c3cb0 0x556bab1c15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.131+0000 7f1e5effd640 1 -- 172.21.15.43:0/2127033172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1c3cb0 msgr2=0x556bab1c15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.131+0000 7f1e5effd640 1 --2- 172.21.15.43:0/2127033172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1c3cb0 0x556bab1c15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.131+0000 7f1e5effd640 1 -- 172.21.15.43:0/2127033172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1b8af0 msgr2=0x556bab1c0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.131+0000 7f1e5effd640 1 --2- 172.21.15.43:0/2127033172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1b8af0 0x556bab1c0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.131+0000 7f1e5effd640 1 -- 172.21.15.43:0/2127033172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556baae30210 con 0x556bab1b9410 2024-09-11T23:26:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.131+0000 7f1e5f7fe640 1 --2- 172.21.15.43:0/2127033172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1b8af0 0x556bab1c0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:26:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.131+0000 7f1e5effd640 1 --2- 172.21.15.43:0/2127033172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1b9410 0x556bab1c10a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1e540ccb10 tx=0x7f1e54007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19bd86ddecccf23a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.131+0000 7f1e3effd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-11T23:26:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.131+0000 7f1e3effd640 1 -- 172.21.15.43:0/2127033172 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5409b8e0 con 0x556bab1b9410 2024-09-11T23:26:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.131+0000 7f1e3effd640 1 -- 172.21.15.43:0/2127033172 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f1e540ccd90 con 0x556bab1b9410 2024-09-11T23:26:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.131+0000 7f1e68be5740 1 -- 172.21.15.43:0/2127033172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e50045070 con 0x556bab1b9410 2024-09-11T23:26:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.131+0000 7f1e3effd640 1 -- 172.21.15.43:0/2127033172 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1e540ccf90 con 0x556bab1b9410 2024-09-11T23:26:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.131+0000 7f1e68be5740 10 client.25657 fetch_fsmap learned FSMap version 30 2024-09-11T23:26:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.131+0000 7f1e68be5740 1 -- 172.21.15.43:0/2127033172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556baaefc620 con 0x556bab1b9410 2024-09-11T23:26:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.131+0000 7f1e3effd640 1 -- 172.21.15.43:0/2127033172 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e50045070 con 0x556bab1b9410 2024-09-11T23:26:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.131+0000 7f1e3effd640 1 -- 172.21.15.43:0/2127033172 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f1e54026060 con 0x556bab1b9410 2024-09-11T23:26:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.132+0000 7f1e68be5740 10 client.25657 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:26:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.132+0000 7f1e68be5740 10 client.25657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:26:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.132+0000 7f1e68be5740 20 client.25657 populate_metadata read hostname 'smithi043' 2024-09-11T23:26:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.132+0000 7f1e68be5740 1 --2- 172.21.15.43:0/2127033172 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x556bab1d3c30 0x556bab1d6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.132+0000 7f1e68be5740 4 client.25657 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:26:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.132+0000 7f1e68be5740 1 -- 172.21.15.43:0/2127033172 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bab12aa20 con 0x556bab1d3c30 2024-09-11T23:26:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.132+0000 7f1e5f7fe640 1 --2- 172.21.15.43:0/2127033172 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x556bab1d3c30 0x556bab1d6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.132+0000 7f1e5f7fe640 1 --2- 172.21.15.43:0/2127033172 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x556bab1d3c30 0x556bab1d6020 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dff3618ac747c963 server_cookie=b6fe8312a2a723be in_seq=0 out_seq=0 2024-09-11T23:26:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.132+0000 7f1e3effd640 10 client.25657 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.133+0000 7f1e3effd640 1 -- 172.21.15.43:0/2127033172 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556bab12aa20 con 0x556bab1d3c30 2024-09-11T23:26:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.133+0000 7f1e3effd640 10 client.25657 handle_command_reply: tid=0 2024-09-11T23:26:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.133+0000 7f1e68be5740 1 client.25657 shutdown 2024-09-11T23:26:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.133+0000 7f1e68be5740 1 -- 172.21.15.43:0/2127033172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1b9410 msgr2=0x556bab1c10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.133+0000 7f1e68be5740 1 --2- 172.21.15.43:0/2127033172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1b9410 0x556bab1c10a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1e540ccb10 tx=0x7f1e54007f90 comp rx=0 tx=0).stop 2024-09-11T23:26:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.133+0000 7f1e68be5740 1 -- 172.21.15.43:0/2127033172 shutdown_connections 2024-09-11T23:26:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.133+0000 7f1e68be5740 1 --2- 172.21.15.43:0/2127033172 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x556bab1d3c30 0x556bab1d6020 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.133+0000 7f1e68be5740 1 --2- 172.21.15.43:0/2127033172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1c3cb0 0x556bab1c15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.133+0000 7f1e68be5740 1 --2- 172.21.15.43:0/2127033172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1b9410 0x556bab1c10a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.133+0000 7f1e68be5740 1 --2- 172.21.15.43:0/2127033172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1b8af0 0x556bab1c0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.133+0000 7f1e3effd640 0 client.25657 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.133+0000 7f1e68be5740 1 -- 172.21.15.43:0/2127033172 >> 172.21.15.43:0/2127033172 conn(0x556bab1b5060 msgr2=0x556bab1badc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.133+0000 7f1e68be5740 1 -- 172.21.15.43:0/2127033172 shutdown_connections 2024-09-11T23:26:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.133+0000 7f1e68be5740 1 -- 172.21.15.43:0/2127033172 wait complete. 2024-09-11T23:26:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.133+0000 7f1e68be5740 20 client.25657 trim_cache size 0 max 16384 2024-09-11T23:26:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.152+0000 7f1e68be5740 1 Processor -- start 2024-09-11T23:26:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.152+0000 7f1e68be5740 1 -- start start 2024-09-11T23:26:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.152+0000 7f1e68be5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1b8af0 0x556bab1c1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.152+0000 7f1e68be5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1b9410 0x556bab1db890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.152+0000 7f1e68be5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1c3cb0 0x556bab1dbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.152+0000 7f1e68be5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601adf00 con 0x556bab1c3cb0 2024-09-11T23:26:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.152+0000 7f1e68be5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601ab7d0 con 0x556bab1b8af0 2024-09-11T23:26:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.152+0000 7f1e68be5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601a9620 con 0x556bab1b9410 2024-09-11T23:26:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.152+0000 7f1e5ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1c3cb0 0x556bab1dbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.152+0000 7f1e5ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1c3cb0 0x556bab1dbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54186/0 (socket says 172.21.15.43:54186) 2024-09-11T23:26:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.152+0000 7f1e5ffff640 1 -- 172.21.15.43:0/1487197517 learned_addr learned my addr 172.21.15.43:0/1487197517 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.152+0000 7f1e5f7fe640 1 --2- 172.21.15.43:0/1487197517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1b8af0 0x556bab1c1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.152+0000 7f1e5effd640 1 --2- 172.21.15.43:0/1487197517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1b9410 0x556bab1db890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.152+0000 7f1e5ffff640 1 -- 172.21.15.43:0/1487197517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1b9410 msgr2=0x556bab1db890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.152+0000 7f1e5ffff640 1 --2- 172.21.15.43:0/1487197517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1b9410 0x556bab1db890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.152+0000 7f1e5ffff640 1 -- 172.21.15.43:0/1487197517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1b8af0 msgr2=0x556bab1c1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.152+0000 7f1e5ffff640 1 --2- 172.21.15.43:0/1487197517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1b8af0 0x556bab1c1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.152+0000 7f1e5ffff640 1 -- 172.21.15.43:0/1487197517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bab1b02d0 con 0x556bab1c3cb0 2024-09-11T23:26:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.153+0000 7f1e5ffff640 1 --2- 172.21.15.43:0/1487197517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1c3cb0 0x556bab1dbdd0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1e58013430 tx=0x7f1e58009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f37726565cf727 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.153+0000 7f1e457fa640 1 -- 172.21.15.43:0/1487197517 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e58022bd0 con 0x556bab1c3cb0 2024-09-11T23:26:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.153+0000 7f1e457fa640 1 -- 172.21.15.43:0/1487197517 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e5800fe40 con 0x556bab1c3cb0 2024-09-11T23:26:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.153+0000 7f1e457fa640 1 -- 172.21.15.43:0/1487197517 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5802a640 con 0x556bab1c3cb0 2024-09-11T23:26:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.154+0000 7f1e68be5740 1 -- 172.21.15.43:0/1487197517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1c3cb0 msgr2=0x556bab1dbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.154+0000 7f1e68be5740 1 --2- 172.21.15.43:0/1487197517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1c3cb0 0x556bab1dbdd0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1e58013430 tx=0x7f1e58009c80 comp rx=0 tx=0).stop 2024-09-11T23:26:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.154+0000 7f1e68be5740 1 -- 172.21.15.43:0/1487197517 shutdown_connections 2024-09-11T23:26:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.154+0000 7f1e68be5740 1 --2- 172.21.15.43:0/1487197517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1c3cb0 0x556bab1dbdd0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.154+0000 7f1e68be5740 1 --2- 172.21.15.43:0/1487197517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1b9410 0x556bab1db890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.154+0000 7f1e68be5740 1 --2- 172.21.15.43:0/1487197517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1b8af0 0x556bab1c1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.154+0000 7f1e68be5740 1 -- 172.21.15.43:0/1487197517 >> 172.21.15.43:0/1487197517 conn(0x556bab1b5060 msgr2=0x556bab1c2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.154+0000 7f1e68be5740 1 -- 172.21.15.43:0/1487197517 shutdown_connections 2024-09-11T23:26:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.154+0000 7f1e68be5740 1 -- 172.21.15.43:0/1487197517 wait complete. 2024-09-11T23:26:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.154+0000 7f1e68be5740 1 -- start start 2024-09-11T23:26:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.154+0000 7f1e68be5740 1 Processor -- start 2024-09-11T23:26:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.154+0000 7f1e68be5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1b8af0 0x556bab1cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.154+0000 7f1e68be5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1b9410 0x556bab1cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.154+0000 7f1e68be5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1c3cb0 0x556bab1c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.154+0000 7f1e68be5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601a9c90 con 0x556bab1c3cb0 2024-09-11T23:26:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.154+0000 7f1e68be5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601ab100 con 0x556bab1b9410 2024-09-11T23:26:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.154+0000 7f1e68be5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601aa360 con 0x556bab1b8af0 2024-09-11T23:26:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.154+0000 7f1e68be5740 10 client.0 _ll_register_callbacks cb 0x7f1e540046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.155+0000 7f1e5ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1c3cb0 0x556bab1c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.155+0000 7f1e5effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1b9410 0x556bab1cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.155+0000 7f1e5ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1c3cb0 0x556bab1c73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54190/0 (socket says 172.21.15.43:54190) 2024-09-11T23:26:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.155+0000 7f1e5effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1b9410 0x556bab1cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:43706/0 (socket says 172.21.15.43:43706) 2024-09-11T23:26:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.155+0000 7f1e5ffff640 1 -- 172.21.15.43:0/2771660935 learned_addr learned my addr 172.21.15.43:0/2771660935 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.155+0000 7f1e5f7fe640 1 --2- 172.21.15.43:0/2771660935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1b8af0 0x556bab1cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.155+0000 7f1e5effd640 1 -- 172.21.15.43:0/2771660935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1b8af0 msgr2=0x556bab1cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.155+0000 7f1e5effd640 1 --2- 172.21.15.43:0/2771660935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1b8af0 0x556bab1cbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.155+0000 7f1e5effd640 1 -- 172.21.15.43:0/2771660935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1c3cb0 msgr2=0x556bab1c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.155+0000 7f1e5effd640 1 --2- 172.21.15.43:0/2771660935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1c3cb0 0x556bab1c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.155+0000 7f1e5effd640 1 -- 172.21.15.43:0/2771660935 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bab1c7bd0 con 0x556bab1b9410 2024-09-11T23:26:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.155+0000 7f1e5f7fe640 1 --2- 172.21.15.43:0/2771660935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1b8af0 0x556bab1cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:26:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.155+0000 7f1e5effd640 1 --2- 172.21.15.43:0/2771660935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1b9410 0x556bab1cc4c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400b580 tx=0x7f1e540ba410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de0ce3a14838e571 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.155+0000 7f1e3effd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:26:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.155+0000 7f1e3effd640 1 -- 172.21.15.43:0/2771660935 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5409b240 con 0x556bab1b9410 2024-09-11T23:26:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.155+0000 7f1e68be5740 1 -- 172.21.15.43:0/2771660935 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556bab124d70 con 0x556bab1b9410 2024-09-11T23:26:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.155+0000 7f1e3effd640 1 -- 172.21.15.43:0/2771660935 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f1e5409ad30 con 0x556bab1b9410 2024-09-11T23:26:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.155+0000 7f1e3effd640 1 -- 172.21.15.43:0/2771660935 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1e54005040 con 0x556bab1b9410 2024-09-11T23:26:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.155+0000 7f1e68be5740 10 client.25688 fetch_fsmap learned FSMap version 30 2024-09-11T23:26:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.155+0000 7f1e68be5740 1 -- 172.21.15.43:0/2771660935 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556bab1c7bd0 con 0x556bab1b9410 2024-09-11T23:26:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.156+0000 7f1e3effd640 1 -- 172.21.15.43:0/2771660935 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556bab124d70 con 0x556bab1b9410 2024-09-11T23:26:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.156+0000 7f1e3effd640 1 -- 172.21.15.43:0/2771660935 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f1e54097630 con 0x556bab1b9410 2024-09-11T23:26:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.156+0000 7f1e68be5740 10 client.25688 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:26:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.156+0000 7f1e68be5740 10 client.25688 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:26:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.156+0000 7f1e68be5740 20 client.25688 populate_metadata read hostname 'smithi043' 2024-09-11T23:26:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.156+0000 7f1e68be5740 1 --2- 172.21.15.43:0/2771660935 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x556bab23e880 0x556bab240ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.156+0000 7f1e68be5740 4 client.25688 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:26:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.156+0000 7f1e68be5740 1 -- 172.21.15.43:0/2771660935 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556bab0a1900 con 0x556bab23e880 2024-09-11T23:26:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.156+0000 7f1e5f7fe640 1 --2- 172.21.15.43:0/2771660935 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x556bab23e880 0x556bab240ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.156+0000 7f1e5f7fe640 1 --2- 172.21.15.43:0/2771660935 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x556bab23e880 0x556bab240ca0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71a21405491a56e5 server_cookie=d2b355f6acd9b44 in_seq=0 out_seq=0 2024-09-11T23:26:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.156+0000 7f1e3effd640 10 client.25688 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.285+0000 7f1e3effd640 1 -- 172.21.15.43:0/2771660935 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556bab0a1900 con 0x556bab23e880 2024-09-11T23:26:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.285+0000 7f1e3effd640 10 client.25688 handle_command_reply: tid=0 2024-09-11T23:26:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.285+0000 7f1e68be5740 1 client.25688 shutdown 2024-09-11T23:26:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.285+0000 7f1e68be5740 1 -- 172.21.15.43:0/2771660935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1b9410 msgr2=0x556bab1cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.285+0000 7f1e68be5740 1 --2- 172.21.15.43:0/2771660935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1b9410 0x556bab1cc4c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400b580 tx=0x7f1e540ba410 comp rx=0 tx=0).stop 2024-09-11T23:26:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.285+0000 7f1e68be5740 1 -- 172.21.15.43:0/2771660935 shutdown_connections 2024-09-11T23:26:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.285+0000 7f1e68be5740 1 --2- 172.21.15.43:0/2771660935 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x556bab23e880 0x556bab240ca0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.285+0000 7f1e68be5740 1 --2- 172.21.15.43:0/2771660935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bab1c3cb0 0x556bab1c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.285+0000 7f1e68be5740 1 --2- 172.21.15.43:0/2771660935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bab1b9410 0x556bab1cc4c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.285+0000 7f1e68be5740 1 --2- 172.21.15.43:0/2771660935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bab1b8af0 0x556bab1cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.285+0000 7f1e3effd640 0 client.25688 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.286+0000 7f1e68be5740 1 -- 172.21.15.43:0/2771660935 >> 172.21.15.43:0/2771660935 conn(0x556bab1b5060 msgr2=0x556bab1c23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.286+0000 7f1e68be5740 1 -- 172.21.15.43:0/2771660935 shutdown_connections 2024-09-11T23:26:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.286+0000 7f1e68be5740 1 -- 172.21.15.43:0/2771660935 wait complete. 2024-09-11T23:26:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.286+0000 7f1e68be5740 20 client.25688 trim_cache size 0 max 16384 2024-09-11T23:26:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:26:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:26:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:26:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:26:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.286+0000 7f1e65e4c640 1 -- 172.21.15.43:0/4061477917 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f1e480a2d70 msgr2=0x7f1e480a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.286+0000 7f1e65e4c640 1 --2- 172.21.15.43:0/4061477917 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f1e480a2d70 0x7f1e480a5230 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1e58009fd0 tx=0x7f1e58004ab0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.286+0000 7f1e65e4c640 1 -- 172.21.15.43:0/4061477917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e60101370 msgr2=0x7f1e60196770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.286+0000 7f1e65e4c640 1 --2- 172.21.15.43:0/4061477917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e60101370 0x7f1e60196770 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1e54007880 tx=0x7f1e54007d40 comp rx=0 tx=0).stop 2024-09-11T23:26:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.286+0000 7f1e65e4c640 1 -- 172.21.15.43:0/4061477917 shutdown_connections 2024-09-11T23:26:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.286+0000 7f1e65e4c640 1 --2- 172.21.15.43:0/4061477917 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f1e480a2d70 0x7f1e480a5230 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.286+0000 7f1e65e4c640 1 --2- 172.21.15.43:0/4061477917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e601069e0 0x7f1e6019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.286+0000 7f1e65e4c640 1 --2- 172.21.15.43:0/4061477917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e60101370 0x7f1e60196770 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.286+0000 7f1e65e4c640 1 --2- 172.21.15.43:0/4061477917 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e601009d0 0x7f1e60196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.286+0000 7f1e65e4c640 1 -- 172.21.15.43:0/4061477917 >> 172.21.15.43:0/4061477917 conn(0x7f1e600fc6b0 msgr2=0x7f1e600fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.287+0000 7f1e65e4c640 1 -- 172.21.15.43:0/4061477917 shutdown_connections 2024-09-11T23:26:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:33.287+0000 7f1e65e4c640 1 -- 172.21.15.43:0/4061477917 wait complete. 2024-09-11T23:26:34.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:33 smithi090 ceph-mon[35130]: pgmap v145: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 3.52k op/s 2024-09-11T23:26:34.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:33 smithi097 ceph-mon[35562]: pgmap v145: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 3.52k op/s 2024-09-11T23:26:34.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:33 smithi043 ceph-mon[30950]: pgmap v145: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 3.52k op/s 2024-09-11T23:26:36.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:35 smithi090 ceph-mon[35130]: pgmap v146: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 2.82k op/s 2024-09-11T23:26:36.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:35 smithi097 ceph-mon[35562]: pgmap v146: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 2.82k op/s 2024-09-11T23:26:36.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:35 smithi043 ceph-mon[30950]: pgmap v146: 97 pgs: 97 active+clean; 18 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 2.82k op/s 2024-09-11T23:26:38.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:37 smithi090 ceph-mon[35130]: pgmap v147: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 21 MiB/s wr, 3.31k op/s 2024-09-11T23:26:38.054 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:37 smithi097 ceph-mon[35562]: pgmap v147: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 21 MiB/s wr, 3.31k op/s 2024-09-11T23:26:38.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:37 smithi043 ceph-mon[30950]: pgmap v147: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 21 MiB/s wr, 3.31k op/s 2024-09-11T23:26:40.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:39 smithi090 ceph-mon[35130]: pgmap v148: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 16 MiB/s wr, 2.53k op/s 2024-09-11T23:26:40.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:39 smithi097 ceph-mon[35562]: pgmap v148: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 16 MiB/s wr, 2.53k op/s 2024-09-11T23:26:40.099 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:39 smithi043 ceph-mon[30950]: pgmap v148: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 16 MiB/s wr, 2.53k op/s 2024-09-11T23:26:41.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:40 smithi090 ceph-mon[35130]: scrub summary: idle 2024-09-11T23:26:41.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:40 smithi090 ceph-mon[35130]: scrub summary: idle 2024-09-11T23:26:41.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:40 smithi090 ceph-mon[35130]: scrub summary: idle 2024-09-11T23:26:41.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:40 smithi097 ceph-mon[35562]: scrub summary: idle 2024-09-11T23:26:41.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:40 smithi097 ceph-mon[35562]: scrub summary: idle 2024-09-11T23:26:41.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:40 smithi097 ceph-mon[35562]: scrub summary: idle 2024-09-11T23:26:41.074 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:40 smithi043 ceph-mon[30950]: scrub summary: idle 2024-09-11T23:26:41.074 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:40 smithi043 ceph-mon[30950]: scrub summary: idle 2024-09-11T23:26:41.074 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:40 smithi043 ceph-mon[30950]: scrub summary: idle 2024-09-11T23:26:41.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.073+0000 7f22277fe640 1 -- 172.21.15.43:0/3557255245 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564802c30d20 con 0x564802d6c7f0 2024-09-11T23:26:41.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.073+0000 7f22277fe640 10 client.16101 handle_command_reply: tid=0 2024-09-11T23:26:41.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.073+0000 7f225088f740 1 client.16101 shutdown 2024-09-11T23:26:41.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.073+0000 7f225088f740 1 -- 172.21.15.43:0/3557255245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc4080 msgr2=0x564802cd38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.073+0000 7f225088f740 1 --2- 172.21.15.43:0/3557255245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc4080 0x564802cd38b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2230009230 tx=0x7f2230009260 comp rx=0 tx=0).stop 2024-09-11T23:26:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.073+0000 7f225088f740 1 -- 172.21.15.43:0/3557255245 shutdown_connections 2024-09-11T23:26:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.073+0000 7f225088f740 1 --2- 172.21.15.43:0/3557255245 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x564802d6c7f0 0x564802d6ec10 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.073+0000 7f225088f740 1 --2- 172.21.15.43:0/3557255245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564802cce940 0x564802cd3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.073+0000 7f225088f740 1 --2- 172.21.15.43:0/3557255245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564802cc4080 0x564802cd38b0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.073+0000 7f22277fe640 0 client.16101 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.073+0000 7f225088f740 1 --2- 172.21.15.43:0/3557255245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564802cc3760 0x564802cd6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.073+0000 7f225088f740 1 -- 172.21.15.43:0/3557255245 >> 172.21.15.43:0/3557255245 conn(0x564802cbfcf0 msgr2=0x564802cd9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.074+0000 7f225088f740 1 -- 172.21.15.43:0/3557255245 shutdown_connections 2024-09-11T23:26:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.074+0000 7f225088f740 1 -- 172.21.15.43:0/3557255245 wait complete. 2024-09-11T23:26:41.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.074+0000 7f225088f740 20 client.16101 trim_cache size 0 max 16384 2024-09-11T23:26:41.076 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:26:41.076 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 30, 2024-09-11T23:26:41.076 INFO:teuthology.orchestra.run.smithi043.stdout: "leader": 24497, 2024-09-11T23:26:41.076 INFO:teuthology.orchestra.run.smithi043.stdout: "set_version": 10, 2024-09-11T23:26:41.076 INFO:teuthology.orchestra.run.smithi043.stdout: "sets": { 2024-09-11T23:26:41.076 INFO:teuthology.orchestra.run.smithi043.stdout: "f79b5b4b": { 2024-09-11T23:26:41.076 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 10, 2024-09-11T23:26:41.076 INFO:teuthology.orchestra.run.smithi043.stdout: "age_ref": 0.0, 2024-09-11T23:26:41.076 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:26:41.076 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "QUIESCED", 2024-09-11T23:26:41.077 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-09-11T23:26:41.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:26:41.077 INFO:teuthology.orchestra.run.smithi043.stdout: "timeout": 90.0, 2024-09-11T23:26:41.077 INFO:teuthology.orchestra.run.smithi043.stdout: "expiration": 144.7, 2024-09-11T23:26:41.077 INFO:teuthology.orchestra.run.smithi043.stdout: "members": { 2024-09-11T23:26:41.077 INFO:teuthology.orchestra.run.smithi043.stdout: "file:/": { 2024-09-11T23:26:41.077 INFO:teuthology.orchestra.run.smithi043.stdout: "excluded": false, 2024-09-11T23:26:41.077 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:26:41.077 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "QUIESCED", 2024-09-11T23:26:41.077 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-09-11T23:26:41.077 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:26:41.077 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:26:41.078 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:26:41.078 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:26:41.078 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:26:41.078 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:26:41.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.075+0000 7f224e4b6640 1 -- 172.21.15.43:0/905457447 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f22340a2cf0 msgr2=0x7f22340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:41.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.075+0000 7f224e4b6640 1 --2- 172.21.15.43:0/905457447 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f22340a2cf0 0x7f22340a51b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x564802cc0b50 tx=0x7f223801b6d0 comp rx=0 tx=0).stop 2024-09-11T23:26:41.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.075+0000 7f224e4b6640 1 -- 172.21.15.43:0/905457447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2248101de0 msgr2=0x7f2248198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:41.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.075+0000 7f224e4b6640 1 --2- 172.21.15.43:0/905457447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2248101de0 0x7f2248198490 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f223800ea60 tx=0x7f223800ef20 comp rx=0 tx=0).stop 2024-09-11T23:26:41.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.075+0000 7f224e4b6640 1 -- 172.21.15.43:0/905457447 shutdown_connections 2024-09-11T23:26:41.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.075+0000 7f224e4b6640 1 --2- 172.21.15.43:0/905457447 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f22340a2cf0 0x7f22340a51b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:41.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.075+0000 7f224e4b6640 1 --2- 172.21.15.43:0/905457447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2248106f10 0x7f224819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:41.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.075+0000 7f224e4b6640 1 --2- 172.21.15.43:0/905457447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2248102790 0x7f22481989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:41.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.075+0000 7f224e4b6640 1 --2- 172.21.15.43:0/905457447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2248101de0 0x7f2248198490 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:41.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.075+0000 7f224e4b6640 1 -- 172.21.15.43:0/905457447 >> 172.21.15.43:0/905457447 conn(0x7f22480fd8c0 msgr2=0x7f22480fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:41.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.075+0000 7f224e4b6640 1 -- 172.21.15.43:0/905457447 shutdown_connections 2024-09-11T23:26:41.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:41.075+0000 7f224e4b6640 1 -- 172.21.15.43:0/905457447 wait complete. 2024-09-11T23:26:41.830 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:41 smithi043 ceph-mon[30950]: pgmap v149: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 16 MiB/s wr, 2.46k op/s 2024-09-11T23:26:41.861 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f79b5b4b', quiesce took 80.2 seconds. Will release after: -55.5 2024-09-11T23:26:41.861 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:26:42.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:41 smithi090 ceph-mon[35130]: pgmap v149: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 16 MiB/s wr, 2.46k op/s 2024-09-11T23:26:42.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:41 smithi097 ceph-mon[35562]: pgmap v149: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 16 MiB/s wr, 2.46k op/s 2024-09-11T23:26:42.158 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:26:42.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.752+0000 7f6012933640 1 -- 172.21.15.43:0/92090769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c102e00 msgr2=0x7f600c1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:42.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.752+0000 7f6012933640 1 --2- 172.21.15.43:0/92090769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c102e00 0x7f600c1031e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009900 tx=0x7f5ffc02a480 comp rx=0 tx=0).stop 2024-09-11T23:26:42.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.752+0000 7f6012933640 1 -- 172.21.15.43:0/92090769 shutdown_connections 2024-09-11T23:26:42.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.752+0000 7f6012933640 1 --2- 172.21.15.43:0/92090769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f600c10c1a0 0x7f600c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:42.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.752+0000 7f6012933640 1 --2- 172.21.15.43:0/92090769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f600c103720 0x7f600c10bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:42.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.752+0000 7f6012933640 1 --2- 172.21.15.43:0/92090769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c102e00 0x7f600c1031e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:42.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.752+0000 7f6012933640 1 -- 172.21.15.43:0/92090769 >> 172.21.15.43:0/92090769 conn(0x7f600c0fc7a0 msgr2=0x7f600c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:42.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.752+0000 7f6012933640 1 -- 172.21.15.43:0/92090769 shutdown_connections 2024-09-11T23:26:42.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.752+0000 7f6012933640 1 -- 172.21.15.43:0/92090769 wait complete. 2024-09-11T23:26:42.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.753+0000 7f6012933640 1 Processor -- start 2024-09-11T23:26:42.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.753+0000 7f6012933640 1 -- start start 2024-09-11T23:26:42.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.753+0000 7f6012933640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c102e00 0x7f600c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:42.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.753+0000 7f6012933640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f600c103720 0x7f600c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:42.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.753+0000 7f6012933640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f600c10c1a0 0x7f600c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:42.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.753+0000 7f6012933640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f600c111f70 con 0x7f600c102e00 2024-09-11T23:26:42.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.753+0000 7f6012933640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f600c111df0 con 0x7f600c103720 2024-09-11T23:26:42.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.753+0000 7f6012933640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f600c1120f0 con 0x7f600c10c1a0 2024-09-11T23:26:42.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.753+0000 7f600bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c102e00 0x7f600c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:42.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.753+0000 7f600bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c102e00 0x7f600c1984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38998/0 (socket says 172.21.15.43:38998) 2024-09-11T23:26:42.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.753+0000 7f600bfff640 1 -- 172.21.15.43:0/1439292371 learned_addr learned my addr 172.21.15.43:0/1439292371 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:42.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.754+0000 7f600b7fe640 1 --2- 172.21.15.43:0/1439292371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f600c103720 0x7f600c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:42.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.754+0000 7f6010ed0640 1 --2- 172.21.15.43:0/1439292371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f600c10c1a0 0x7f600c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:42.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.754+0000 7f600bfff640 1 -- 172.21.15.43:0/1439292371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f600c10c1a0 msgr2=0x7f600c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:42.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.754+0000 7f600bfff640 1 --2- 172.21.15.43:0/1439292371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f600c10c1a0 0x7f600c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:42.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.754+0000 7f600bfff640 1 -- 172.21.15.43:0/1439292371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f600c103720 msgr2=0x7f600c1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:42.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.754+0000 7f600bfff640 1 --2- 172.21.15.43:0/1439292371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f600c103720 0x7f600c1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:42.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.754+0000 7f600bfff640 1 -- 172.21.15.43:0/1439292371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600c19d450 con 0x7f600c102e00 2024-09-11T23:26:42.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.754+0000 7f6010ed0640 1 --2- 172.21.15.43:0/1439292371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f600c10c1a0 0x7f600c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:26:42.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.754+0000 7f600bfff640 1 --2- 172.21.15.43:0/1439292371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c102e00 0x7f600c1984a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc006010 tx=0x7f5ffc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:42.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.754+0000 7f60097fa640 1 -- 172.21.15.43:0/1439292371 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc03f030 con 0x7f600c102e00 2024-09-11T23:26:42.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.754+0000 7f60097fa640 1 -- 172.21.15.43:0/1439292371 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ffc003ce0 con 0x7f600c102e00 2024-09-11T23:26:42.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.754+0000 7f6012933640 1 -- 172.21.15.43:0/1439292371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f600c19d6e0 con 0x7f600c102e00 2024-09-11T23:26:42.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.754+0000 7f60097fa640 1 -- 172.21.15.43:0/1439292371 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc035070 con 0x7f600c102e00 2024-09-11T23:26:42.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.754+0000 7f6012933640 1 -- 172.21.15.43:0/1439292371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f600c1a5020 con 0x7f600c102e00 2024-09-11T23:26:42.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.755+0000 7f6012933640 1 -- 172.21.15.43:0/1439292371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f600c1031e0 con 0x7f600c102e00 2024-09-11T23:26:42.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.757+0000 7f60097fa640 1 -- 172.21.15.43:0/1439292371 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5ffc042030 con 0x7f600c102e00 2024-09-11T23:26:42.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.757+0000 7f60097fa640 1 --2- 172.21.15.43:0/1439292371 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5ff40a2cf0 0x7f5ff40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:42.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.757+0000 7f600b7fe640 1 --2- 172.21.15.43:0/1439292371 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5ff40a2cf0 0x7f5ff40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:42.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.757+0000 7f60097fa640 1 -- 172.21.15.43:0/1439292371 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f5ffc0efe20 con 0x7f600c102e00 2024-09-11T23:26:42.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.758+0000 7f600b7fe640 1 --2- 172.21.15.43:0/1439292371 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5ff40a2cf0 0x7f5ff40a51b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6000005fd0 tx=0x7f6000005850 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:42.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.761+0000 7f60097fa640 1 -- 172.21.15.43:0/1439292371 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ffc0bcb00 con 0x7f600c102e00 2024-09-11T23:26:42.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.975+0000 7f6012933640 1 -- 172.21.15.43:0/1439292371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f600c107280 con 0x7f600c102e00 2024-09-11T23:26:42.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.975+0000 7f60097fa640 1 -- 172.21.15.43:0/1439292371 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11893 (secure 0 0 0) 0x7f5ffc0bc400 con 0x7f600c102e00 2024-09-11T23:26:42.976 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:26:42.976 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"btime":"2024-09-11T23:25:40:446496+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:25:39.727562+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25571":{"gid":25571,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/568491708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":568491708},{"type":"v1","addr":"172.21.15.90:6835","nonce":568491708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:26:42.976 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-09-11T23:26:42.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.977+0000 7f6012933640 1 -- 172.21.15.43:0/1439292371 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5ff40a2cf0 msgr2=0x7f5ff40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:42.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.977+0000 7f6012933640 1 --2- 172.21.15.43:0/1439292371 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5ff40a2cf0 0x7f5ff40a51b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6000005fd0 tx=0x7f6000005850 comp rx=0 tx=0).stop 2024-09-11T23:26:42.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.977+0000 7f6012933640 1 -- 172.21.15.43:0/1439292371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c102e00 msgr2=0x7f600c1984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:42.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.977+0000 7f6012933640 1 --2- 172.21.15.43:0/1439292371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c102e00 0x7f600c1984a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc006010 tx=0x7f5ffc0042f0 comp rx=0 tx=0).stop 2024-09-11T23:26:42.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.978+0000 7f6012933640 1 -- 172.21.15.43:0/1439292371 shutdown_connections 2024-09-11T23:26:42.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.978+0000 7f6012933640 1 --2- 172.21.15.43:0/1439292371 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5ff40a2cf0 0x7f5ff40a51b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:42.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.978+0000 7f6012933640 1 --2- 172.21.15.43:0/1439292371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f600c10c1a0 0x7f600c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:42.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.978+0000 7f6012933640 1 --2- 172.21.15.43:0/1439292371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f600c103720 0x7f600c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:42.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.978+0000 7f6012933640 1 --2- 172.21.15.43:0/1439292371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c102e00 0x7f600c1984a0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:42.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.978+0000 7f6012933640 1 -- 172.21.15.43:0/1439292371 >> 172.21.15.43:0/1439292371 conn(0x7f600c0fc7a0 msgr2=0x7f600c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:42.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.978+0000 7f6012933640 1 -- 172.21.15.43:0/1439292371 shutdown_connections 2024-09-11T23:26:42.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:42.978+0000 7f6012933640 1 -- 172.21.15.43:0/1439292371 wait complete. 2024-09-11T23:26:43.751 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id f79b5b4b --release --await' 2024-09-11T23:26:43.752 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id f79b5b4b --release --await 2024-09-11T23:26:44.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:43 smithi090 ceph-mon[35130]: pgmap v150: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 18 MiB/s wr, 2.75k op/s 2024-09-11T23:26:44.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:43 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1439292371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:26:44.023 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:26:44.043 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:43 smithi043 ceph-mon[30950]: pgmap v150: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 18 MiB/s wr, 2.75k op/s 2024-09-11T23:26:44.043 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:43 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1439292371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:26:44.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:43 smithi097 ceph-mon[35562]: pgmap v150: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 18 MiB/s wr, 2.75k op/s 2024-09-11T23:26:44.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:43 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1439292371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:26:44.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.512+0000 7f7752907640 1 -- 172.21.15.43:0/1981612695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f774c102570 msgr2=0x7f774c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:44.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.512+0000 7f7752907640 1 --2- 172.21.15.43:0/1981612695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f774c102570 0x7f774c1029b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7734009900 tx=0x7f773402a480 comp rx=0 tx=0).stop 2024-09-11T23:26:44.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.512+0000 7f7752907640 1 -- 172.21.15.43:0/1981612695 shutdown_connections 2024-09-11T23:26:44.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.513+0000 7f7752907640 1 --2- 172.21.15.43:0/1981612695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f774c102ef0 0x7f774c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.513+0000 7f7752907640 1 --2- 172.21.15.43:0/1981612695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f774c102570 0x7f774c1029b0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.513+0000 7f7752907640 1 --2- 172.21.15.43:0/1981612695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f774c100e20 0x7f774c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.513+0000 7f7752907640 1 -- 172.21.15.43:0/1981612695 >> 172.21.15.43:0/1981612695 conn(0x7f774c0fc7d0 msgr2=0x7f774c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:44.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.513+0000 7f7752907640 1 -- 172.21.15.43:0/1981612695 shutdown_connections 2024-09-11T23:26:44.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.513+0000 7f7752907640 1 -- 172.21.15.43:0/1981612695 wait complete. 2024-09-11T23:26:44.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.513+0000 7f7752907640 1 Processor -- start 2024-09-11T23:26:44.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.513+0000 7f7752907640 1 -- start start 2024-09-11T23:26:44.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.514+0000 7f7752907640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f774c100e20 0x7f774c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:44.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.514+0000 7f7752907640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f774c102570 0x7f774c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:44.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.514+0000 7f7752907640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f774c102ef0 0x7f774c19cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:44.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.514+0000 7f7752907640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f774c111020 con 0x7f774c100e20 2024-09-11T23:26:44.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.514+0000 7f7752907640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f774c110ea0 con 0x7f774c102ef0 2024-09-11T23:26:44.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.514+0000 7f7752907640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f774c1111a0 con 0x7f774c102570 2024-09-11T23:26:44.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.514+0000 7f774bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f774c100e20 0x7f774c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:44.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.514+0000 7f774bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f774c100e20 0x7f774c198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39016/0 (socket says 172.21.15.43:39016) 2024-09-11T23:26:44.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.514+0000 7f774bfff640 1 -- 172.21.15.43:0/2920647480 learned_addr learned my addr 172.21.15.43:0/2920647480 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:44.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.514+0000 7f7750ea4640 1 --2- 172.21.15.43:0/2920647480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f774c102ef0 0x7f774c19cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:44.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.514+0000 7f774b7fe640 1 --2- 172.21.15.43:0/2920647480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f774c102570 0x7f774c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:44.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.514+0000 7f774bfff640 1 -- 172.21.15.43:0/2920647480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f774c102570 msgr2=0x7f774c198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:44.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.514+0000 7f774bfff640 1 --2- 172.21.15.43:0/2920647480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f774c102570 0x7f774c198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.514+0000 7f774bfff640 1 -- 172.21.15.43:0/2920647480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f774c102ef0 msgr2=0x7f774c19cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:44.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.514+0000 7f774bfff640 1 --2- 172.21.15.43:0/2920647480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f774c102ef0 0x7f774c19cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.514+0000 7f774bfff640 1 -- 172.21.15.43:0/2920647480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774c19d6b0 con 0x7f774c100e20 2024-09-11T23:26:44.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.514+0000 7f774bfff640 1 --2- 172.21.15.43:0/2920647480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f774c100e20 0x7f774c198700 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f773c00b4a0 tx=0x7f773c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:44.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.514+0000 7f77497fa640 1 -- 172.21.15.43:0/2920647480 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c014070 con 0x7f774c100e20 2024-09-11T23:26:44.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.514+0000 7f77497fa640 1 -- 172.21.15.43:0/2920647480 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f773c010af0 con 0x7f774c100e20 2024-09-11T23:26:44.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.514+0000 7f7752907640 1 -- 172.21.15.43:0/2920647480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f774c19d9a0 con 0x7f774c100e20 2024-09-11T23:26:44.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.514+0000 7f77497fa640 1 -- 172.21.15.43:0/2920647480 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c017d10 con 0x7f774c100e20 2024-09-11T23:26:44.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.515+0000 7f7752907640 1 -- 172.21.15.43:0/2920647480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f774c1a5280 con 0x7f774c100e20 2024-09-11T23:26:44.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.518+0000 7f7754ce0740 1 Processor -- start 2024-09-11T23:26:44.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.518+0000 7f77497fa640 1 -- 172.21.15.43:0/2920647480 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f773c017070 con 0x7f774c100e20 2024-09-11T23:26:44.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.518+0000 7f7754ce0740 1 -- start start 2024-09-11T23:26:44.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.518+0000 7f7754ce0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b59e231060 0x55b59e231440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:44.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.518+0000 7f7754ce0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e231980 0x55b59e23bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:44.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.518+0000 7f7754ce0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e23c240 0x55b59e23e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:44.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.518+0000 7f7754ce0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b59e196620 con 0x55b59e23c240 2024-09-11T23:26:44.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.518+0000 7f7754ce0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b59e13de50 con 0x55b59e231980 2024-09-11T23:26:44.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.518+0000 7f7754ce0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b59e1925e0 con 0x55b59e231060 2024-09-11T23:26:44.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.518+0000 7f774b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e23c240 0x55b59e23e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:44.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.518+0000 7f774b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e23c240 0x55b59e23e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39020/0 (socket says 172.21.15.43:39020) 2024-09-11T23:26:44.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.518+0000 7f774b7fe640 1 -- 172.21.15.43:0/3967585877 learned_addr learned my addr 172.21.15.43:0/3967585877 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:44.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.518+0000 7f7750ea4640 1 --2- 172.21.15.43:0/3967585877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b59e231060 0x55b59e231440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:44.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.518+0000 7f774bfff640 1 --2- 172.21.15.43:0/3967585877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e231980 0x55b59e23bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:44.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.518+0000 7f774b7fe640 1 -- 172.21.15.43:0/3967585877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b59e231060 msgr2=0x55b59e231440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:44.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.519+0000 7f774b7fe640 1 --2- 172.21.15.43:0/3967585877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b59e231060 0x55b59e231440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.519+0000 7f774b7fe640 1 -- 172.21.15.43:0/3967585877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e231980 msgr2=0x55b59e23bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:44.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.519+0000 7f774b7fe640 1 --2- 172.21.15.43:0/3967585877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e231980 0x55b59e23bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.519+0000 7f774b7fe640 1 -- 172.21.15.43:0/3967585877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b59e12ed40 con 0x55b59e23c240 2024-09-11T23:26:44.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.519+0000 7f774b7fe640 1 --2- 172.21.15.43:0/3967585877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e23c240 0x55b59e23e630 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f77340314e0 tx=0x7f7734004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4fb7ce241be80a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:44.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.519+0000 7f77497fa640 1 --2- 172.21.15.43:0/2920647480 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f77380a2cf0 0x7f77380a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:44.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.519+0000 7f7750ea4640 1 --2- 172.21.15.43:0/2920647480 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f77380a2cf0 0x7f77380a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:44.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.519+0000 7f77497fa640 1 -- 172.21.15.43:0/2920647480 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f773c0ce710 con 0x7f774c100e20 2024-09-11T23:26:44.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.519+0000 7f77427fc640 1 -- 172.21.15.43:0/3967585877 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734035030 con 0x55b59e23c240 2024-09-11T23:26:44.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.519+0000 7f77427fc640 1 -- 172.21.15.43:0/3967585877 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7734003ad0 con 0x55b59e23c240 2024-09-11T23:26:44.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.519+0000 7f77427fc640 1 -- 172.21.15.43:0/3967585877 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734031c90 con 0x55b59e23c240 2024-09-11T23:26:44.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.520+0000 7f7750ea4640 1 --2- 172.21.15.43:0/2920647480 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f77380a2cf0 0x7f77380a51b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f774400a400 tx=0x7f774400d040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:44.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.520+0000 7f7754ce0740 1 -- 172.21.15.43:0/3967585877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e23c240 msgr2=0x55b59e23e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:44.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.520+0000 7f7754ce0740 1 --2- 172.21.15.43:0/3967585877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e23c240 0x55b59e23e630 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f77340314e0 tx=0x7f7734004170 comp rx=0 tx=0).stop 2024-09-11T23:26:44.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.520+0000 7f7754ce0740 1 -- 172.21.15.43:0/3967585877 shutdown_connections 2024-09-11T23:26:44.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.520+0000 7f7754ce0740 1 --2- 172.21.15.43:0/3967585877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e23c240 0x55b59e23e630 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.520+0000 7f7754ce0740 1 --2- 172.21.15.43:0/3967585877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e231980 0x55b59e23bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.520+0000 7f7754ce0740 1 --2- 172.21.15.43:0/3967585877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b59e231060 0x55b59e231440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.520+0000 7f7754ce0740 1 -- 172.21.15.43:0/3967585877 >> 172.21.15.43:0/3967585877 conn(0x55b59e22d5f0 msgr2=0x55b59e22d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:44.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.520+0000 7f7754ce0740 1 -- 172.21.15.43:0/3967585877 shutdown_connections 2024-09-11T23:26:44.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.520+0000 7f7754ce0740 1 -- 172.21.15.43:0/3967585877 wait complete. 2024-09-11T23:26:44.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.521+0000 7f7754ce0740 1 -- start start 2024-09-11T23:26:44.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.521+0000 7f7754ce0740 1 Processor -- start 2024-09-11T23:26:44.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f7754ce0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e231060 0x55b59e239180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:44.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f7754ce0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b59e231980 0x55b59e2396c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:44.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f7754ce0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e23c240 0x55b59e239c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:44.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f7754ce0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b59e23ee10 con 0x55b59e231060 2024-09-11T23:26:44.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f7754ce0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b59e11ef10 con 0x55b59e23c240 2024-09-11T23:26:44.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f7754ce0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b59e23ef90 con 0x55b59e231980 2024-09-11T23:26:44.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f774bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e231060 0x55b59e239180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:44.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f774bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e231060 0x55b59e239180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39024/0 (socket says 172.21.15.43:39024) 2024-09-11T23:26:44.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f774bfff640 1 -- 172.21.15.43:0/893004948 learned_addr learned my addr 172.21.15.43:0/893004948 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:44.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f7754ce0740 10 client.0 _ll_register_callbacks cb 0x55b59e142db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:44.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f774b7fe640 1 --2- 172.21.15.43:0/893004948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b59e231980 0x55b59e2396c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:44.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f7750ea4640 1 --2- 172.21.15.43:0/893004948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e23c240 0x55b59e239c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:44.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f774bfff640 1 -- 172.21.15.43:0/893004948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b59e231980 msgr2=0x55b59e2396c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:44.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f774bfff640 1 --2- 172.21.15.43:0/893004948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b59e231980 0x55b59e2396c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f774bfff640 1 -- 172.21.15.43:0/893004948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e23c240 msgr2=0x55b59e239c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:44.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f774bfff640 1 --2- 172.21.15.43:0/893004948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e23c240 0x55b59e239c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f774bfff640 1 -- 172.21.15.43:0/893004948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b59dea5580 con 0x55b59e231060 2024-09-11T23:26:44.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f774bfff640 1 --2- 172.21.15.43:0/893004948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e231060 0x55b59e239180 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f773c0cbfd0 tx=0x7f773c009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efc930434403b6d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:44.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f772b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:26:44.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f772b7fe640 1 -- 172.21.15.43:0/893004948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c01e070 con 0x55b59e231060 2024-09-11T23:26:44.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f7754ce0740 1 -- 172.21.15.43:0/893004948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7734003ad0 con 0x55b59e231060 2024-09-11T23:26:44.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.522+0000 7f772b7fe640 1 -- 172.21.15.43:0/893004948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f773c0cc0e0 con 0x55b59e231060 2024-09-11T23:26:44.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.523+0000 7f772b7fe640 1 -- 172.21.15.43:0/893004948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f773c09a6a0 con 0x55b59e231060 2024-09-11T23:26:44.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.523+0000 7f7754ce0740 10 client.16167 fetch_fsmap learned FSMap version 30 2024-09-11T23:26:44.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.523+0000 7f7754ce0740 1 -- 172.21.15.43:0/893004948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b59df6f960 con 0x55b59e231060 2024-09-11T23:26:44.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.523+0000 7f772b7fe640 1 -- 172.21.15.43:0/893004948 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7734003ad0 con 0x55b59e231060 2024-09-11T23:26:44.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.523+0000 7f772b7fe640 1 -- 172.21.15.43:0/893004948 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f773c0973b0 con 0x55b59e231060 2024-09-11T23:26:44.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.523+0000 7f7754ce0740 10 client.16167 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:26:44.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.523+0000 7f7754ce0740 10 client.16167 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:26:44.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.523+0000 7f7754ce0740 20 client.16167 populate_metadata read hostname 'smithi043' 2024-09-11T23:26:44.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.523+0000 7f7754ce0740 1 --2- 172.21.15.43:0/893004948 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b59e24be60 0x55b59e24e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:44.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.523+0000 7f7754ce0740 4 client.16167 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:26:44.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.523+0000 7f7754ce0740 1 -- 172.21.15.43:0/893004948 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b59e1a3b90 con 0x55b59e24be60 2024-09-11T23:26:44.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.523+0000 7f774b7fe640 1 --2- 172.21.15.43:0/893004948 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b59e24be60 0x55b59e24e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:44.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.524+0000 7f772b7fe640 10 client.16167 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:44.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.524+0000 7f774b7fe640 1 --2- 172.21.15.43:0/893004948 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b59e24be60 0x55b59e24e270 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0aa0422714549d1 server_cookie=b4b40ecb5fa12fdf in_seq=0 out_seq=0 2024-09-11T23:26:44.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.525+0000 7f772b7fe640 1 -- 172.21.15.43:0/893004948 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b59e1a3b90 con 0x55b59e24be60 2024-09-11T23:26:44.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.525+0000 7f772b7fe640 10 client.16167 handle_command_reply: tid=0 2024-09-11T23:26:44.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.525+0000 7f7754ce0740 1 client.16167 shutdown 2024-09-11T23:26:44.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.525+0000 7f7754ce0740 1 -- 172.21.15.43:0/893004948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e231060 msgr2=0x55b59e239180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:44.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.525+0000 7f7754ce0740 1 --2- 172.21.15.43:0/893004948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e231060 0x55b59e239180 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f773c0cbfd0 tx=0x7f773c009370 comp rx=0 tx=0).stop 2024-09-11T23:26:44.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.525+0000 7f7754ce0740 1 -- 172.21.15.43:0/893004948 shutdown_connections 2024-09-11T23:26:44.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.525+0000 7f7754ce0740 1 --2- 172.21.15.43:0/893004948 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b59e24be60 0x55b59e24e270 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.525+0000 7f772b7fe640 0 client.16167 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:44.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.525+0000 7f7754ce0740 1 --2- 172.21.15.43:0/893004948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e23c240 0x55b59e239c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.525+0000 7f7754ce0740 1 --2- 172.21.15.43:0/893004948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b59e231980 0x55b59e2396c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.525+0000 7f7754ce0740 1 --2- 172.21.15.43:0/893004948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e231060 0x55b59e239180 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.525+0000 7f7754ce0740 1 -- 172.21.15.43:0/893004948 >> 172.21.15.43:0/893004948 conn(0x55b59e22d5f0 msgr2=0x55b59e2332d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:44.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.525+0000 7f7754ce0740 1 -- 172.21.15.43:0/893004948 shutdown_connections 2024-09-11T23:26:44.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.525+0000 7f7754ce0740 1 -- 172.21.15.43:0/893004948 wait complete. 2024-09-11T23:26:44.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.525+0000 7f7754ce0740 20 client.16167 trim_cache size 0 max 16384 2024-09-11T23:26:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.542+0000 7f7754ce0740 1 Processor -- start 2024-09-11T23:26:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.542+0000 7f7754ce0740 1 -- start start 2024-09-11T23:26:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.543+0000 7f7754ce0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b59e231980 0x55b59e233ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.543+0000 7f7754ce0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e23c240 0x55b59e22c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.543+0000 7f7754ce0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e24be60 0x55b59e253ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.543+0000 7f7754ce0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f774c1ae190 con 0x55b59e24be60 2024-09-11T23:26:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.543+0000 7f7754ce0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f774c1aba60 con 0x55b59e23c240 2024-09-11T23:26:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.543+0000 7f7754ce0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f774c1a98b0 con 0x55b59e231980 2024-09-11T23:26:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.543+0000 7f7750ea4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e24be60 0x55b59e253ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.543+0000 7f7750ea4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e24be60 0x55b59e253ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39036/0 (socket says 172.21.15.43:39036) 2024-09-11T23:26:44.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.543+0000 7f7750ea4640 1 -- 172.21.15.43:0/4030449684 learned_addr learned my addr 172.21.15.43:0/4030449684 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.543+0000 7f774b7fe640 1 --2- 172.21.15.43:0/4030449684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e23c240 0x55b59e22c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.543+0000 7f774bfff640 1 --2- 172.21.15.43:0/4030449684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b59e231980 0x55b59e233ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.543+0000 7f7750ea4640 1 -- 172.21.15.43:0/4030449684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b59e231980 msgr2=0x55b59e233ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.543+0000 7f7750ea4640 1 --2- 172.21.15.43:0/4030449684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b59e231980 0x55b59e233ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.543+0000 7f7750ea4640 1 -- 172.21.15.43:0/4030449684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e23c240 msgr2=0x55b59e22c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.543+0000 7f7750ea4640 1 --2- 172.21.15.43:0/4030449684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e23c240 0x55b59e22c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.543+0000 7f7750ea4640 1 -- 172.21.15.43:0/4030449684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b59e22bbb0 con 0x55b59e24be60 2024-09-11T23:26:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.543+0000 7f774b7fe640 1 --2- 172.21.15.43:0/4030449684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e23c240 0x55b59e22c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:26:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.544+0000 7f7750ea4640 1 --2- 172.21.15.43:0/4030449684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e24be60 0x55b59e253ae0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f77440189a0 tx=0x7f7744014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2e06cc1ec8f91f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.544+0000 7f7741ffb640 1 -- 172.21.15.43:0/4030449684 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744018e60 con 0x55b59e24be60 2024-09-11T23:26:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.544+0000 7f7741ffb640 1 -- 172.21.15.43:0/4030449684 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7744029070 con 0x55b59e24be60 2024-09-11T23:26:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.544+0000 7f7741ffb640 1 -- 172.21.15.43:0/4030449684 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77440176f0 con 0x55b59e24be60 2024-09-11T23:26:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.545+0000 7f7754ce0740 1 -- 172.21.15.43:0/4030449684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e24be60 msgr2=0x55b59e253ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.545+0000 7f7754ce0740 1 --2- 172.21.15.43:0/4030449684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e24be60 0x55b59e253ae0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f77440189a0 tx=0x7f7744014000 comp rx=0 tx=0).stop 2024-09-11T23:26:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.545+0000 7f7754ce0740 1 -- 172.21.15.43:0/4030449684 shutdown_connections 2024-09-11T23:26:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.545+0000 7f7754ce0740 1 --2- 172.21.15.43:0/4030449684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e24be60 0x55b59e253ae0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.545+0000 7f7754ce0740 1 --2- 172.21.15.43:0/4030449684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e23c240 0x55b59e22c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.545+0000 7f7754ce0740 1 --2- 172.21.15.43:0/4030449684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b59e231980 0x55b59e233ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.545+0000 7f7754ce0740 1 -- 172.21.15.43:0/4030449684 >> 172.21.15.43:0/4030449684 conn(0x55b59e22d5f0 msgr2=0x55b59e22bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.545+0000 7f7754ce0740 1 -- 172.21.15.43:0/4030449684 shutdown_connections 2024-09-11T23:26:44.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.545+0000 7f7754ce0740 1 -- 172.21.15.43:0/4030449684 wait complete. 2024-09-11T23:26:44.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.545+0000 7f7754ce0740 1 -- start start 2024-09-11T23:26:44.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.545+0000 7f7754ce0740 1 Processor -- start 2024-09-11T23:26:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.546+0000 7f7754ce0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e231980 0x55b59e2406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.546+0000 7f7754ce0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b59e23c240 0x55b59e240c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.546+0000 7f7754ce0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e24be60 0x55b59e2b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.546+0000 7f7754ce0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f774c1a9f20 con 0x55b59e24be60 2024-09-11T23:26:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.546+0000 7f7754ce0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f774c1ab390 con 0x55b59e231980 2024-09-11T23:26:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.546+0000 7f7754ce0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f774c1aa5f0 con 0x55b59e23c240 2024-09-11T23:26:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.546+0000 7f7754ce0740 10 client.0 _ll_register_callbacks cb 0x7f773c017eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.546+0000 7f7750ea4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e24be60 0x55b59e2b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.546+0000 7f774bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e231980 0x55b59e2406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.546+0000 7f7750ea4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e24be60 0x55b59e2b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39046/0 (socket says 172.21.15.43:39046) 2024-09-11T23:26:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.546+0000 7f7750ea4640 1 -- 172.21.15.43:0/3611899683 learned_addr learned my addr 172.21.15.43:0/3611899683 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.546+0000 7f774b7fe640 1 --2- 172.21.15.43:0/3611899683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b59e23c240 0x55b59e240c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.546+0000 7f7750ea4640 1 -- 172.21.15.43:0/3611899683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b59e23c240 msgr2=0x55b59e240c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.546+0000 7f7750ea4640 1 --2- 172.21.15.43:0/3611899683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b59e23c240 0x55b59e240c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.546+0000 7f7750ea4640 1 -- 172.21.15.43:0/3611899683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e231980 msgr2=0x55b59e2406d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.546+0000 7f7750ea4640 1 --2- 172.21.15.43:0/3611899683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e231980 0x55b59e2406d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.546+0000 7f7750ea4640 1 -- 172.21.15.43:0/3611899683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b59e2b60a0 con 0x55b59e24be60 2024-09-11T23:26:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.546+0000 7f774bfff640 1 --2- 172.21.15.43:0/3611899683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e231980 0x55b59e2406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:26:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.547+0000 7f7750ea4640 1 --2- 172.21.15.43:0/3611899683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e24be60 0x55b59e2b5ab0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7744017010 tx=0x7f7744007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495a8c1e8a126a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.547+0000 7f772b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:26:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.547+0000 7f772b7fe640 1 -- 172.21.15.43:0/3611899683 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744018e60 con 0x55b59e24be60 2024-09-11T23:26:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.547+0000 7f772b7fe640 1 -- 172.21.15.43:0/3611899683 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f7744024860 con 0x55b59e24be60 2024-09-11T23:26:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.547+0000 7f7754ce0740 1 -- 172.21.15.43:0/3611899683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7744029070 con 0x55b59e24be60 2024-09-11T23:26:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.547+0000 7f772b7fe640 1 -- 172.21.15.43:0/3611899683 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7744024a60 con 0x55b59e24be60 2024-09-11T23:26:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.547+0000 7f7754ce0740 10 client.16173 fetch_fsmap learned FSMap version 30 2024-09-11T23:26:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.547+0000 7f7754ce0740 1 -- 172.21.15.43:0/3611899683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b59e2b7380 con 0x55b59e24be60 2024-09-11T23:26:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.547+0000 7f772b7fe640 1 -- 172.21.15.43:0/3611899683 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7744029070 con 0x55b59e24be60 2024-09-11T23:26:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.547+0000 7f772b7fe640 1 -- 172.21.15.43:0/3611899683 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f77440365a0 con 0x55b59e24be60 2024-09-11T23:26:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.547+0000 7f7754ce0740 10 client.16173 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:26:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.547+0000 7f7754ce0740 10 client.16173 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:26:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.547+0000 7f7754ce0740 20 client.16173 populate_metadata read hostname 'smithi043' 2024-09-11T23:26:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.547+0000 7f7754ce0740 1 --2- 172.21.15.43:0/3611899683 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b59e2b79f0 0x55b59e2b9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.547+0000 7f7754ce0740 4 client.16173 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f79b5b4b", "release": true, "await": true}] 2024-09-11T23:26:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.547+0000 7f7754ce0740 1 -- 172.21.15.43:0/3611899683 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f79b5b4b", "release": true, "await": true}) -- 0x55b59e19e0a0 con 0x55b59e2b79f0 2024-09-11T23:26:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.548+0000 7f774bfff640 1 --2- 172.21.15.43:0/3611899683 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b59e2b79f0 0x55b59e2b9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.548+0000 7f774bfff640 1 --2- 172.21.15.43:0/3611899683 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b59e2b79f0 0x55b59e2b9e10 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27b0b413e53af5cb server_cookie=9572d06cf3da6b59 in_seq=0 out_seq=0 2024-09-11T23:26:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.548+0000 7f772b7fe640 10 client.16173 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:44.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.580+0000 7f772b7fe640 1 -- 172.21.15.43:0/3611899683 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b59e19e0a0 con 0x55b59e2b79f0 2024-09-11T23:26:44.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.580+0000 7f772b7fe640 10 client.16173 handle_command_reply: tid=0 2024-09-11T23:26:44.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.582+0000 7f7754ce0740 1 client.16173 shutdown 2024-09-11T23:26:44.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.582+0000 7f7754ce0740 1 -- 172.21.15.43:0/3611899683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e24be60 msgr2=0x55b59e2b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:44.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.582+0000 7f7754ce0740 1 --2- 172.21.15.43:0/3611899683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e24be60 0x55b59e2b5ab0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7744017010 tx=0x7f7744007040 comp rx=0 tx=0).stop 2024-09-11T23:26:44.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.583+0000 7f7754ce0740 1 -- 172.21.15.43:0/3611899683 shutdown_connections 2024-09-11T23:26:44.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.583+0000 7f7754ce0740 1 --2- 172.21.15.43:0/3611899683 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b59e2b79f0 0x55b59e2b9e10 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.583+0000 7f7754ce0740 1 --2- 172.21.15.43:0/3611899683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b59e24be60 0x55b59e2b5ab0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.583+0000 7f7754ce0740 1 --2- 172.21.15.43:0/3611899683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b59e23c240 0x55b59e240c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.583+0000 7f7754ce0740 1 --2- 172.21.15.43:0/3611899683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b59e231980 0x55b59e2406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.583+0000 7f7754ce0740 1 -- 172.21.15.43:0/3611899683 >> 172.21.15.43:0/3611899683 conn(0x55b59e22d5f0 msgr2=0x55b59e239960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:44.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.587+0000 7f772b7fe640 0 client.16173 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:44.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.588+0000 7f7754ce0740 1 -- 172.21.15.43:0/3611899683 shutdown_connections 2024-09-11T23:26:44.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.589+0000 7f7754ce0740 1 -- 172.21.15.43:0/3611899683 wait complete. 2024-09-11T23:26:44.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.589+0000 7f7754ce0740 20 client.16173 trim_cache size 0 max 16384 2024-09-11T23:26:44.590 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:26:44.591 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 30, 2024-09-11T23:26:44.591 INFO:teuthology.orchestra.run.smithi043.stdout: "leader": 24497, 2024-09-11T23:26:44.591 INFO:teuthology.orchestra.run.smithi043.stdout: "set_version": 12, 2024-09-11T23:26:44.591 INFO:teuthology.orchestra.run.smithi043.stdout: "sets": { 2024-09-11T23:26:44.591 INFO:teuthology.orchestra.run.smithi043.stdout: "f79b5b4b": { 2024-09-11T23:26:44.591 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 12, 2024-09-11T23:26:44.591 INFO:teuthology.orchestra.run.smithi043.stdout: "age_ref": 0.0, 2024-09-11T23:26:44.591 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:26:44.591 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RELEASED", 2024-09-11T23:26:44.591 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-09-11T23:26:44.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:26:44.591 INFO:teuthology.orchestra.run.smithi043.stdout: "timeout": 90.0, 2024-09-11T23:26:44.591 INFO:teuthology.orchestra.run.smithi043.stdout: "expiration": 144.7, 2024-09-11T23:26:44.592 INFO:teuthology.orchestra.run.smithi043.stdout: "members": { 2024-09-11T23:26:44.592 INFO:teuthology.orchestra.run.smithi043.stdout: "file:/": { 2024-09-11T23:26:44.592 INFO:teuthology.orchestra.run.smithi043.stdout: "excluded": false, 2024-09-11T23:26:44.592 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:26:44.592 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RELEASED", 2024-09-11T23:26:44.592 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-09-11T23:26:44.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:26:44.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:26:44.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:26:44.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:26:44.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:26:44.592 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:26:44.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.591+0000 7f7752907640 1 -- 172.21.15.43:0/2920647480 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f77380a2cf0 msgr2=0x7f77380a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:44.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.591+0000 7f7752907640 1 --2- 172.21.15.43:0/2920647480 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f77380a2cf0 0x7f77380a51b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f774400a400 tx=0x7f774400d040 comp rx=0 tx=0).stop 2024-09-11T23:26:44.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.591+0000 7f7752907640 1 -- 172.21.15.43:0/2920647480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f774c100e20 msgr2=0x7f774c198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:44.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.591+0000 7f7752907640 1 --2- 172.21.15.43:0/2920647480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f774c100e20 0x7f774c198700 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f773c00b4a0 tx=0x7f773c00b960 comp rx=0 tx=0).stop 2024-09-11T23:26:44.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.591+0000 7f7752907640 1 -- 172.21.15.43:0/2920647480 shutdown_connections 2024-09-11T23:26:44.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.591+0000 7f7752907640 1 --2- 172.21.15.43:0/2920647480 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f77380a2cf0 0x7f77380a51b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.591+0000 7f7752907640 1 --2- 172.21.15.43:0/2920647480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f774c102ef0 0x7f774c19cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.592+0000 7f7752907640 1 --2- 172.21.15.43:0/2920647480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f774c102570 0x7f774c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.592+0000 7f7752907640 1 --2- 172.21.15.43:0/2920647480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f774c100e20 0x7f774c198700 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:44.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.592+0000 7f7752907640 1 -- 172.21.15.43:0/2920647480 >> 172.21.15.43:0/2920647480 conn(0x7f774c0fc7d0 msgr2=0x7f774c06a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:44.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.594+0000 7f7752907640 1 -- 172.21.15.43:0/2920647480 shutdown_connections 2024-09-11T23:26:44.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:44.594+0000 7f7752907640 1 -- 172.21.15.43:0/2920647480 wait complete. 2024-09-11T23:26:45.719 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f79b5b4b', total seconds elapsed: 84.1 2024-09-11T23:26:45.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:45 smithi043 ceph-mon[30950]: pgmap v151: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 16 MiB/s wr, 2.48k op/s 2024-09-11T23:26:46.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:45 smithi090 ceph-mon[35130]: pgmap v151: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 16 MiB/s wr, 2.48k op/s 2024-09-11T23:26:46.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:45 smithi097 ceph-mon[35562]: pgmap v151: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 16 MiB/s wr, 2.48k op/s 2024-09-11T23:26:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.444+0000 7f7452f52640 1 -- 172.21.15.43:0/839660982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f744c1095d0 msgr2=0x7f744c109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.444+0000 7f7452f52640 1 --2- 172.21.15.43:0/839660982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f744c1095d0 0x7f744c109a10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7438009890 tx=0x7f743802a390 comp rx=0 tx=0).stop 2024-09-11T23:26:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.446+0000 7f7452f52640 1 -- 172.21.15.43:0/839660982 shutdown_connections 2024-09-11T23:26:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.446+0000 7f7452f52640 1 --2- 172.21.15.43:0/839660982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f744c11b140 0x7f744c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.446+0000 7f7452f52640 1 --2- 172.21.15.43:0/839660982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f744c1095d0 0x7f744c109a10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.446+0000 7f7452f52640 1 --2- 172.21.15.43:0/839660982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744c1091f0 0x7f744c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.446+0000 7f7452f52640 1 -- 172.21.15.43:0/839660982 >> 172.21.15.43:0/839660982 conn(0x7f744c06cdf0 msgr2=0x7f744c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.453+0000 7f7452f52640 1 -- 172.21.15.43:0/839660982 shutdown_connections 2024-09-11T23:26:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.456+0000 7f7452f52640 1 -- 172.21.15.43:0/839660982 wait complete. 2024-09-11T23:26:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.458+0000 7f7452f52640 1 Processor -- start 2024-09-11T23:26:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.458+0000 7f7452f52640 1 -- start start 2024-09-11T23:26:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.459+0000 7f7452f52640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744c1091f0 0x7f744c1a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.459+0000 7f7452f52640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f744c1095d0 0x7f744c1a7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.459+0000 7f7452f52640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f744c11b140 0x7f744c1abba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.459+0000 7f7452f52640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f744c120cf0 con 0x7f744c1091f0 2024-09-11T23:26:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.459+0000 7f7452f52640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f744c120b70 con 0x7f744c1095d0 2024-09-11T23:26:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.459+0000 7f7452f52640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f744c120e70 con 0x7f744c11b140 2024-09-11T23:26:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.460+0000 7f7450cee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744c1091f0 0x7f744c1a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.460+0000 7f7450cee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744c1091f0 0x7f744c1a72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39062/0 (socket says 172.21.15.43:39062) 2024-09-11T23:26:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.460+0000 7f7450cee640 1 -- 172.21.15.43:0/1131830080 learned_addr learned my addr 172.21.15.43:0/1131830080 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.461+0000 7f744bfff640 1 --2- 172.21.15.43:0/1131830080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f744c1095d0 0x7f744c1a7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.461+0000 7f74514ef640 1 --2- 172.21.15.43:0/1131830080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f744c11b140 0x7f744c1abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.463+0000 7f7450cee640 1 -- 172.21.15.43:0/1131830080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f744c11b140 msgr2=0x7f744c1abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.463+0000 7f7450cee640 1 --2- 172.21.15.43:0/1131830080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f744c11b140 0x7f744c1abba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.463+0000 7f7450cee640 1 -- 172.21.15.43:0/1131830080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f744c1095d0 msgr2=0x7f744c1a7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.463+0000 7f7450cee640 1 --2- 172.21.15.43:0/1131830080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f744c1095d0 0x7f744c1a7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.463+0000 7f7450cee640 1 -- 172.21.15.43:0/1131830080 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744c1ac2a0 con 0x7f744c1091f0 2024-09-11T23:26:46.464 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.463+0000 7f7450cee640 1 --2- 172.21.15.43:0/1131830080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744c1091f0 0x7f744c1a72e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f744000d950 tx=0x7f744000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.465+0000 7f7449ffb640 1 -- 172.21.15.43:0/1131830080 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440017070 con 0x7f744c1091f0 2024-09-11T23:26:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.465+0000 7f7452f52640 1 -- 172.21.15.43:0/1131830080 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f744c1ac590 con 0x7f744c1091f0 2024-09-11T23:26:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.465+0000 7f7452f52640 1 -- 172.21.15.43:0/1131830080 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f744c110920 con 0x7f744c1091f0 2024-09-11T23:26:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.465+0000 7f7449ffb640 1 -- 172.21.15.43:0/1131830080 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7440009c30 con 0x7f744c1091f0 2024-09-11T23:26:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.465+0000 7f7449ffb640 1 -- 172.21.15.43:0/1131830080 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744000f690 con 0x7f744c1091f0 2024-09-11T23:26:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.469+0000 7f7455dcc740 1 Processor -- start 2024-09-11T23:26:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.471+0000 7f7455dcc740 1 -- start start 2024-09-11T23:26:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.471+0000 7f7455dcc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea62f1af0 0x555ea62f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.471+0000 7f7455dcc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62f2410 0x555ea62fc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.471+0000 7f7455dcc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62fccb0 0x555ea62ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.471+0000 7f7455dcc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555ea62562f0 con 0x555ea62f1af0 2024-09-11T23:26:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.471+0000 7f7455dcc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555ea61fe270 con 0x555ea62f2410 2024-09-11T23:26:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.471+0000 7f7455dcc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555ea62522b0 con 0x555ea62fccb0 2024-09-11T23:26:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.475+0000 7f74514ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea62f1af0 0x555ea62f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.476+0000 7f74514ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea62f1af0 0x555ea62f1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39068/0 (socket says 172.21.15.43:39068) 2024-09-11T23:26:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.476+0000 7f74514ef640 1 -- 172.21.15.43:0/187581400 learned_addr learned my addr 172.21.15.43:0/187581400 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.476+0000 7f744bfff640 1 --2- 172.21.15.43:0/187581400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62fccb0 0x555ea62ff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.476+0000 7f7449ffb640 1 -- 172.21.15.43:0/1131830080 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f744000f830 con 0x7f744c1091f0 2024-09-11T23:26:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.476+0000 7f7450cee640 1 --2- 172.21.15.43:0/187581400 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62f2410 0x555ea62fc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.476+0000 7f744bfff640 1 -- 172.21.15.43:0/187581400 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62f2410 msgr2=0x555ea62fc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.476+0000 7f744bfff640 1 --2- 172.21.15.43:0/187581400 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62f2410 0x555ea62fc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.476+0000 7f744bfff640 1 -- 172.21.15.43:0/187581400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea62f1af0 msgr2=0x555ea62f1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.476+0000 7f744bfff640 1 --2- 172.21.15.43:0/187581400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea62f1af0 0x555ea62f1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.476+0000 7f744bfff640 1 -- 172.21.15.43:0/187581400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ea61fd340 con 0x555ea62fccb0 2024-09-11T23:26:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.477+0000 7f744bfff640 1 --2- 172.21.15.43:0/187581400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62fccb0 0x555ea62ff0a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f744c11d3b0 tx=0x7f7438030340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20dc866a5887bda5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.477+0000 7f7436ffd640 1 -- 172.21.15.43:0/187581400 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438040020 con 0x555ea62fccb0 2024-09-11T23:26:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.477+0000 7f7436ffd640 1 -- 172.21.15.43:0/187581400 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7438038040 con 0x555ea62fccb0 2024-09-11T23:26:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.478+0000 7f7455dcc740 1 -- 172.21.15.43:0/187581400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62fccb0 msgr2=0x555ea62ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.478+0000 7f7455dcc740 1 --2- 172.21.15.43:0/187581400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62fccb0 0x555ea62ff0a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f744c11d3b0 tx=0x7f7438030340 comp rx=0 tx=0).stop 2024-09-11T23:26:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.480+0000 7f7455dcc740 1 -- 172.21.15.43:0/187581400 shutdown_connections 2024-09-11T23:26:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.480+0000 7f7455dcc740 1 --2- 172.21.15.43:0/187581400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62fccb0 0x555ea62ff0a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.480+0000 7f7455dcc740 1 --2- 172.21.15.43:0/187581400 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62f2410 0x555ea62fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.480+0000 7f7455dcc740 1 --2- 172.21.15.43:0/187581400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea62f1af0 0x555ea62f1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.480+0000 7f7455dcc740 1 -- 172.21.15.43:0/187581400 >> 172.21.15.43:0/187581400 conn(0x555ea62ee060 msgr2=0x555ea62ee440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.482+0000 7f7449ffb640 1 --2- 172.21.15.43:0/1131830080 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f743c0a2f00 0x7f743c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.482+0000 7f744bfff640 1 --2- 172.21.15.43:0/1131830080 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f743c0a2f00 0x7f743c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.483+0000 7f7449ffb640 1 -- 172.21.15.43:0/1131830080 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f74400cef60 con 0x7f744c1091f0 2024-09-11T23:26:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.483+0000 7f7455dcc740 1 -- 172.21.15.43:0/187581400 shutdown_connections 2024-09-11T23:26:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.483+0000 7f744bfff640 1 --2- 172.21.15.43:0/1131830080 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f743c0a2f00 0x7f743c0a53c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7438004650 tx=0x7f7438005b20 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.483+0000 7f7455dcc740 1 -- 172.21.15.43:0/187581400 wait complete. 2024-09-11T23:26:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.484+0000 7f7455dcc740 1 -- start start 2024-09-11T23:26:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.484+0000 7f7455dcc740 1 Processor -- start 2024-09-11T23:26:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.485+0000 7f7455dcc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62f1af0 0x555ea62f9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.485+0000 7f7455dcc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62f2410 0x555ea62f9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.485+0000 7f7455dcc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea62fccb0 0x555ea62fa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.485+0000 7f7455dcc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555ea62ff8d0 con 0x555ea62fccb0 2024-09-11T23:26:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.485+0000 7f744bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62f2410 0x555ea62f9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.485+0000 7f744bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62f2410 0x555ea62f9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:55578/0 (socket says 172.21.15.43:55578) 2024-09-11T23:26:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.485+0000 7f744bfff640 1 -- 172.21.15.43:0/4166622304 learned_addr learned my addr 172.21.15.43:0/4166622304 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.486+0000 7f7455dcc740 1 -- 172.21.15.43:0/4166622304 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555ea61df290 con 0x555ea62f1af0 2024-09-11T23:26:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.486+0000 7f7455dcc740 1 -- 172.21.15.43:0/4166622304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555ea62ffa50 con 0x555ea62f2410 2024-09-11T23:26:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.486+0000 7f7455dcc740 10 client.0 _ll_register_callbacks cb 0x555ea6210140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.486+0000 7f7450cee640 1 --2- 172.21.15.43:0/4166622304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62f1af0 0x555ea62f9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.486+0000 7f744bfff640 1 -- 172.21.15.43:0/4166622304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62f1af0 msgr2=0x555ea62f9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.486+0000 7f744bfff640 1 --2- 172.21.15.43:0/4166622304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62f1af0 0x555ea62f9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.486+0000 7f744bfff640 1 -- 172.21.15.43:0/4166622304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea62fccb0 msgr2=0x555ea62fa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.486+0000 7f744bfff640 1 --2- 172.21.15.43:0/4166622304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea62fccb0 0x555ea62fa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.486+0000 7f744bfff640 1 -- 172.21.15.43:0/4166622304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ea5f69210 con 0x555ea62f2410 2024-09-11T23:26:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.489+0000 7f744bfff640 1 --2- 172.21.15.43:0/4166622304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62f2410 0x555ea62f9f80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x555ea62f2ab0 tx=0x7f74380304b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e18d188c665c75f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.491+0000 7f742bfff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-11T23:26:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.497+0000 7f742bfff640 1 -- 172.21.15.43:0/4166622304 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743804b070 con 0x555ea62f2410 2024-09-11T23:26:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.497+0000 7f7455dcc740 1 -- 172.21.15.43:0/4166622304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7438038040 con 0x555ea62f2410 2024-09-11T23:26:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.498+0000 7f742bfff640 1 -- 172.21.15.43:0/4166622304 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f7438004c40 con 0x555ea62f2410 2024-09-11T23:26:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.501+0000 7f742bfff640 1 -- 172.21.15.43:0/4166622304 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7438040040 con 0x555ea62f2410 2024-09-11T23:26:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.503+0000 7f7455dcc740 10 client.25687 fetch_fsmap learned FSMap version 30 2024-09-11T23:26:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.503+0000 7f7455dcc740 1 -- 172.21.15.43:0/4166622304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555ea6035620 con 0x555ea62f2410 2024-09-11T23:26:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.510+0000 7f742bfff640 1 -- 172.21.15.43:0/4166622304 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7438038040 con 0x555ea62f2410 2024-09-11T23:26:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.510+0000 7f742bfff640 1 -- 172.21.15.43:0/4166622304 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f743805a430 con 0x555ea62f2410 2024-09-11T23:26:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.510+0000 7f7455dcc740 10 client.25687 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:26:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.510+0000 7f7455dcc740 10 client.25687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:26:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.510+0000 7f7455dcc740 20 client.25687 populate_metadata read hostname 'smithi043' 2024-09-11T23:26:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.510+0000 7f7455dcc740 1 --2- 172.21.15.43:0/4166622304 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x555ea630cd70 0x555ea630f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.510+0000 7f7455dcc740 4 client.25687 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:26:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.510+0000 7f7455dcc740 1 -- 172.21.15.43:0/4166622304 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ea6263a20 con 0x555ea630cd70 2024-09-11T23:26:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.515+0000 7f7450cee640 1 --2- 172.21.15.43:0/4166622304 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x555ea630cd70 0x555ea630f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.521+0000 7f7450cee640 1 --2- 172.21.15.43:0/4166622304 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x555ea630cd70 0x555ea630f230 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaa2b3c315077b96 server_cookie=73b655a3c0896c58 in_seq=0 out_seq=0 2024-09-11T23:26:46.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.522+0000 7f742bfff640 10 client.25687 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.525+0000 7f742bfff640 1 -- 172.21.15.43:0/4166622304 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ea6263a20 con 0x555ea630cd70 2024-09-11T23:26:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.525+0000 7f742bfff640 10 client.25687 handle_command_reply: tid=0 2024-09-11T23:26:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.525+0000 7f7455dcc740 1 client.25687 shutdown 2024-09-11T23:26:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.527+0000 7f7455dcc740 1 -- 172.21.15.43:0/4166622304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62f2410 msgr2=0x555ea62f9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.527+0000 7f7455dcc740 1 --2- 172.21.15.43:0/4166622304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62f2410 0x555ea62f9f80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x555ea62f2ab0 tx=0x7f74380304b0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.527+0000 7f7455dcc740 1 -- 172.21.15.43:0/4166622304 shutdown_connections 2024-09-11T23:26:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.527+0000 7f7455dcc740 1 --2- 172.21.15.43:0/4166622304 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x555ea630cd70 0x555ea630f230 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.527+0000 7f7455dcc740 1 --2- 172.21.15.43:0/4166622304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea62fccb0 0x555ea62fa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.527+0000 7f7455dcc740 1 --2- 172.21.15.43:0/4166622304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62f2410 0x555ea62f9f80 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x555ea62f2ab0 tx=0x7f74380304b0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.527+0000 7f7455dcc740 1 --2- 172.21.15.43:0/4166622304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62f1af0 0x555ea62f9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.527+0000 7f7455dcc740 1 -- 172.21.15.43:0/4166622304 >> 172.21.15.43:0/4166622304 conn(0x555ea62ee060 msgr2=0x555ea62f3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.527+0000 7f742bfff640 0 client.25687 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.528+0000 7f7455dcc740 1 -- 172.21.15.43:0/4166622304 shutdown_connections 2024-09-11T23:26:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.531+0000 7f7455dcc740 1 -- 172.21.15.43:0/4166622304 wait complete. 2024-09-11T23:26:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.532+0000 7f7455dcc740 20 client.25687 trim_cache size 0 max 16384 2024-09-11T23:26:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.563+0000 7f7455dcc740 1 Processor -- start 2024-09-11T23:26:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.563+0000 7f7455dcc740 1 -- start start 2024-09-11T23:26:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.563+0000 7f7455dcc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea62f1af0 0x555ea62fb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.564+0000 7f7450cee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea62f1af0 0x555ea62fb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.564+0000 7f7455dcc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62fccb0 0x555ea62fa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.564+0000 7f7455dcc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62f4560 0x555ea6314b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.564+0000 7f744bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62fccb0 0x555ea62fa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.564+0000 7f744bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62fccb0 0x555ea62fa2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:54016/0 (socket says 172.21.15.43:54016) 2024-09-11T23:26:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.564+0000 7f744bfff640 1 -- 172.21.15.43:0/455515040 learned_addr learned my addr 172.21.15.43:0/455515040 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.565+0000 7f74514ef640 1 --2- 172.21.15.43:0/455515040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62f4560 0x555ea6314b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.565+0000 7f7455dcc740 1 -- 172.21.15.43:0/455515040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7430011200 con 0x555ea62f1af0 2024-09-11T23:26:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.565+0000 7f7455dcc740 1 -- 172.21.15.43:0/455515040 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f743000ead0 con 0x555ea62fccb0 2024-09-11T23:26:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.565+0000 7f7455dcc740 1 -- 172.21.15.43:0/455515040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f743000c920 con 0x555ea62f4560 2024-09-11T23:26:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.565+0000 7f744bfff640 1 -- 172.21.15.43:0/455515040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62f4560 msgr2=0x555ea6314b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.565+0000 7f744bfff640 1 --2- 172.21.15.43:0/455515040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62f4560 0x555ea6314b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.565+0000 7f744bfff640 1 -- 172.21.15.43:0/455515040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea62f1af0 msgr2=0x555ea62fb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.565+0000 7f744bfff640 1 --2- 172.21.15.43:0/455515040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea62f1af0 0x555ea62fb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.565+0000 7f744bfff640 1 -- 172.21.15.43:0/455515040 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ea62fc150 con 0x555ea62fccb0 2024-09-11T23:26:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.566+0000 7f744bfff640 1 --2- 172.21.15.43:0/455515040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62fccb0 0x555ea62fa2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7438030520 tx=0x7f7438000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34216268270262d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.570+0000 7f74367fc640 1 -- 172.21.15.43:0/455515040 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743804b050 con 0x555ea62fccb0 2024-09-11T23:26:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.570+0000 7f74367fc640 1 -- 172.21.15.43:0/455515040 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f743805aa70 con 0x555ea62fccb0 2024-09-11T23:26:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.570+0000 7f74367fc640 1 -- 172.21.15.43:0/455515040 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438032920 con 0x555ea62fccb0 2024-09-11T23:26:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.572+0000 7f7455dcc740 1 -- 172.21.15.43:0/455515040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62fccb0 msgr2=0x555ea62fa2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.572+0000 7f7455dcc740 1 --2- 172.21.15.43:0/455515040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62fccb0 0x555ea62fa2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7438030520 tx=0x7f7438000bc0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.573+0000 7f7455dcc740 1 -- 172.21.15.43:0/455515040 shutdown_connections 2024-09-11T23:26:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.573+0000 7f7455dcc740 1 --2- 172.21.15.43:0/455515040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62f4560 0x555ea6314b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.573+0000 7f7455dcc740 1 --2- 172.21.15.43:0/455515040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62fccb0 0x555ea62fa2f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.573+0000 7f7455dcc740 1 --2- 172.21.15.43:0/455515040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea62f1af0 0x555ea62fb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.573+0000 7f7455dcc740 1 -- 172.21.15.43:0/455515040 >> 172.21.15.43:0/455515040 conn(0x555ea62ee060 msgr2=0x555ea62f89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.574+0000 7f7455dcc740 1 -- 172.21.15.43:0/455515040 shutdown_connections 2024-09-11T23:26:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.574+0000 7f7455dcc740 1 -- 172.21.15.43:0/455515040 wait complete. 2024-09-11T23:26:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.574+0000 7f7455dcc740 1 -- start start 2024-09-11T23:26:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.574+0000 7f7455dcc740 1 Processor -- start 2024-09-11T23:26:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.575+0000 7f7455dcc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea62f1af0 0x555ea6305150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.575+0000 7f7455dcc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62f4560 0x555ea6305690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.575+0000 7f7455dcc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62fccb0 0x555ea62f65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.575+0000 7f7455dcc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f743000cf90 con 0x555ea62f1af0 2024-09-11T23:26:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.575+0000 7f7455dcc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f743000e400 con 0x555ea62f4560 2024-09-11T23:26:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.575+0000 7f7455dcc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f743000d660 con 0x555ea62fccb0 2024-09-11T23:26:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.575+0000 7f7455dcc740 10 client.0 _ll_register_callbacks cb 0x7f74440093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.575+0000 7f7450cee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea62f1af0 0x555ea6305150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.576+0000 7f744bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62f4560 0x555ea6305690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.576+0000 7f744bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62f4560 0x555ea6305690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:54020/0 (socket says 172.21.15.43:54020) 2024-09-11T23:26:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.576+0000 7f744bfff640 1 -- 172.21.15.43:0/3786949017 learned_addr learned my addr 172.21.15.43:0/3786949017 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.576+0000 7f744bfff640 1 -- 172.21.15.43:0/3786949017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62fccb0 msgr2=0x555ea62f65f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:26:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.576+0000 7f744bfff640 1 --2- 172.21.15.43:0/3786949017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62fccb0 0x555ea62f65f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.576+0000 7f744bfff640 1 -- 172.21.15.43:0/3786949017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea62f1af0 msgr2=0x555ea6305150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.576+0000 7f744bfff640 1 --2- 172.21.15.43:0/3786949017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea62f1af0 0x555ea6305150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.576+0000 7f744bfff640 1 -- 172.21.15.43:0/3786949017 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ea62f6df0 con 0x555ea62f4560 2024-09-11T23:26:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.578+0000 7f744bfff640 1 --2- 172.21.15.43:0/3786949017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62f4560 0x555ea6305690 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f743805a970 tx=0x7f7438009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96e60b13d62cd61c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.578+0000 7f742bfff640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:26:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.578+0000 7f742bfff640 1 -- 172.21.15.43:0/3786949017 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743804b040 con 0x555ea62f4560 2024-09-11T23:26:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.578+0000 7f7455dcc740 1 -- 172.21.15.43:0/3786949017 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f743805aa70 con 0x555ea62f4560 2024-09-11T23:26:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.578+0000 7f742bfff640 1 -- 172.21.15.43:0/3786949017 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f743805a430 con 0x555ea62f4560 2024-09-11T23:26:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.578+0000 7f742bfff640 1 -- 172.21.15.43:0/3786949017 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7438030930 con 0x555ea62f4560 2024-09-11T23:26:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.579+0000 7f7455dcc740 10 client.25721 fetch_fsmap learned FSMap version 30 2024-09-11T23:26:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.579+0000 7f7455dcc740 1 -- 172.21.15.43:0/3786949017 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555ea6304be0 con 0x555ea62f4560 2024-09-11T23:26:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.579+0000 7f742bfff640 1 -- 172.21.15.43:0/3786949017 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f743805aa70 con 0x555ea62f4560 2024-09-11T23:26:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.579+0000 7f742bfff640 1 -- 172.21.15.43:0/3786949017 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7438069020 con 0x555ea62f4560 2024-09-11T23:26:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.579+0000 7f7455dcc740 10 client.25721 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:26:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.579+0000 7f7455dcc740 10 client.25721 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:26:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.579+0000 7f7455dcc740 20 client.25721 populate_metadata read hostname 'smithi043' 2024-09-11T23:26:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.579+0000 7f7455dcc740 1 --2- 172.21.15.43:0/3786949017 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x555ea6377e60 0x555ea637a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.579+0000 7f7455dcc740 4 client.25721 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:26:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.579+0000 7f7455dcc740 1 -- 172.21.15.43:0/3786949017 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555ea625dd70 con 0x555ea6377e60 2024-09-11T23:26:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.580+0000 7f7450cee640 1 --2- 172.21.15.43:0/3786949017 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x555ea6377e60 0x555ea637a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:46.582 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.581+0000 7f7450cee640 1 --2- 172.21.15.43:0/3786949017 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x555ea6377e60 0x555ea637a280 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c3aa4106ae9a5f1 server_cookie=588ec34ca7363cb2 in_seq=0 out_seq=0 2024-09-11T23:26:46.582 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:26:46.581+0000 7f742bfff640 10 client.25721 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:48.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:47 smithi090 ceph-mon[35130]: pgmap v152: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 15 MiB/s wr, 2.32k op/s 2024-09-11T23:26:48.050 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:47 smithi097 ceph-mon[35562]: pgmap v152: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 15 MiB/s wr, 2.32k op/s 2024-09-11T23:26:48.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:47 smithi043 ceph-mon[30950]: pgmap v152: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 15 MiB/s wr, 2.32k op/s 2024-09-11T23:26:50.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:49 smithi090 ceph-mon[35130]: pgmap v153: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 21 MiB/s wr, 3.35k op/s 2024-09-11T23:26:50.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:49 smithi097 ceph-mon[35562]: pgmap v153: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 21 MiB/s wr, 3.35k op/s 2024-09-11T23:26:50.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:49 smithi043 ceph-mon[30950]: pgmap v153: 97 pgs: 97 active+clean; 18 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 21 MiB/s wr, 3.35k op/s 2024-09-11T23:26:51.753 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:26:52.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:51 smithi090 ceph-mon[35130]: pgmap v154: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 29 MiB/s wr, 4.84k op/s 2024-09-11T23:26:52.050 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:51 smithi097 ceph-mon[35562]: pgmap v154: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 29 MiB/s wr, 4.84k op/s 2024-09-11T23:26:52.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:51 smithi043 ceph-mon[30950]: pgmap v154: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 29 MiB/s wr, 4.84k op/s 2024-09-11T23:26:52.356 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:26:54.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:53 smithi090 ceph-mon[35130]: pgmap v155: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 33 MiB/s wr, 5.62k op/s 2024-09-11T23:26:54.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:53 smithi097 ceph-mon[35562]: pgmap v155: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 33 MiB/s wr, 5.62k op/s 2024-09-11T23:26:54.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:53 smithi043 ceph-mon[30950]: pgmap v155: 97 pgs: 97 active+clean; 18 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 33 MiB/s wr, 5.62k op/s 2024-09-11T23:26:56.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:55 smithi090 ceph-mon[35130]: pgmap v156: 97 pgs: 97 active+clean; 18 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 37 MiB/s wr, 6.50k op/s 2024-09-11T23:26:56.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:55 smithi097 ceph-mon[35562]: pgmap v156: 97 pgs: 97 active+clean; 18 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 37 MiB/s wr, 6.50k op/s 2024-09-11T23:26:56.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:55 smithi043 ceph-mon[30950]: pgmap v156: 97 pgs: 97 active+clean; 18 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 37 MiB/s wr, 6.50k op/s 2024-09-11T23:26:58.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:57 smithi090 ceph-mon[35130]: pgmap v157: 97 pgs: 97 active+clean; 19 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 39 MiB/s wr, 6.72k op/s 2024-09-11T23:26:58.050 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:57 smithi097 ceph-mon[35562]: pgmap v157: 97 pgs: 97 active+clean; 19 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 39 MiB/s wr, 6.72k op/s 2024-09-11T23:26:58.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:57 smithi043 ceph-mon[30950]: pgmap v157: 97 pgs: 97 active+clean; 19 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 39 MiB/s wr, 6.72k op/s 2024-09-11T23:26:58.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.254+0000 7f5d0f99c640 1 -- 172.21.15.43:0/999573337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d08100a70 msgr2=0x7f5d08100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:58.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.254+0000 7f5d0f99c640 1 --2- 172.21.15.43:0/999573337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d08100a70 0x7f5d08100ed0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009930 tx=0x7f5cf802a420 comp rx=0 tx=0).stop 2024-09-11T23:26:58.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.255+0000 7f5d0f99c640 1 -- 172.21.15.43:0/999573337 shutdown_connections 2024-09-11T23:26:58.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.255+0000 7f5d0f99c640 1 --2- 172.21.15.43:0/999573337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d08101410 0x7f5d0810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.255+0000 7f5d0f99c640 1 --2- 172.21.15.43:0/999573337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d08100a70 0x7f5d08100ed0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.255+0000 7f5d0f99c640 1 --2- 172.21.15.43:0/999573337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d08106a70 0x7f5d08106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.255+0000 7f5d0f99c640 1 -- 172.21.15.43:0/999573337 >> 172.21.15.43:0/999573337 conn(0x7f5d080fc7d0 msgr2=0x7f5d080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:58.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.255+0000 7f5d0f99c640 1 -- 172.21.15.43:0/999573337 shutdown_connections 2024-09-11T23:26:58.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.255+0000 7f5d0f99c640 1 -- 172.21.15.43:0/999573337 wait complete. 2024-09-11T23:26:58.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.255+0000 7f5d0f99c640 1 Processor -- start 2024-09-11T23:26:58.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.255+0000 7f5d0f99c640 1 -- start start 2024-09-11T23:26:58.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.256+0000 7f5d0f99c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d08100a70 0x7f5d08198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:58.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.256+0000 7f5d0f99c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d08101410 0x7f5d08198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:58.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.256+0000 7f5d0f99c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d08106a70 0x7f5d0819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:58.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.256+0000 7f5d0f99c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d08110e30 con 0x7f5d08101410 2024-09-11T23:26:58.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.256+0000 7f5d0f99c640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5d08110cb0 con 0x7f5d08106a70 2024-09-11T23:26:58.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.256+0000 7f5d0f99c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5d08110fb0 con 0x7f5d08100a70 2024-09-11T23:26:58.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.256+0000 7f5d0cf37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d08101410 0x7f5d08198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:58.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.256+0000 7f5d0cf37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d08101410 0x7f5d08198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36322/0 (socket says 172.21.15.43:36322) 2024-09-11T23:26:58.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.256+0000 7f5d0cf37640 1 -- 172.21.15.43:0/1729575406 learned_addr learned my addr 172.21.15.43:0/1729575406 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:58.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.256+0000 7f5d0d738640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d08100a70 0x7f5d08198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:58.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.256+0000 7f5d0df39640 1 --2- 172.21.15.43:0/1729575406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d08106a70 0x7f5d0819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:58.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.256+0000 7f5d0cf37640 1 -- 172.21.15.43:0/1729575406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d08100a70 msgr2=0x7f5d08198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:58.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.256+0000 7f5d0cf37640 1 --2- 172.21.15.43:0/1729575406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d08100a70 0x7f5d08198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.256+0000 7f5d0cf37640 1 -- 172.21.15.43:0/1729575406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d08106a70 msgr2=0x7f5d0819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:58.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.256+0000 7f5d0cf37640 1 --2- 172.21.15.43:0/1729575406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d08106a70 0x7f5d0819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.256+0000 7f5d0cf37640 1 -- 172.21.15.43:0/1729575406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0819d3d0 con 0x7f5d08101410 2024-09-11T23:26:58.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.256+0000 7f5d0cf37640 1 --2- 172.21.15.43:0/1729575406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d08101410 0x7f5d08198960 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009230 tx=0x7f5cf8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:58.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.257+0000 7f5cfe7fc640 1 -- 172.21.15.43:0/1729575406 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf803f030 con 0x7f5d08101410 2024-09-11T23:26:58.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.257+0000 7f5cfe7fc640 1 -- 172.21.15.43:0/1729575406 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cf8003ad0 con 0x7f5d08101410 2024-09-11T23:26:58.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.257+0000 7f5d0f99c640 1 -- 172.21.15.43:0/1729575406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0819d660 con 0x7f5d08101410 2024-09-11T23:26:58.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.257+0000 7f5cfe7fc640 1 -- 172.21.15.43:0/1729575406 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8035070 con 0x7f5d08101410 2024-09-11T23:26:58.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.257+0000 7f5d0f99c640 1 -- 172.21.15.43:0/1729575406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d081a4fa0 con 0x7f5d08101410 2024-09-11T23:26:58.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.259+0000 7f5cfe7fc640 1 -- 172.21.15.43:0/1729575406 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5cf8042030 con 0x7f5d08101410 2024-09-11T23:26:58.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.259+0000 7f5d11d75740 1 Processor -- start 2024-09-11T23:26:58.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.260+0000 7f5d11d75740 1 -- start start 2024-09-11T23:26:58.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.260+0000 7f5d11d75740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560270cd44a0 0x560270cd4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:58.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.260+0000 7f5d11d75740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560270cd4dc0 0x560270cdf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:58.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.260+0000 7f5cfe7fc640 1 --2- 172.21.15.43:0/1729575406 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5cf40a2d70 0x7f5cf40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:58.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.260+0000 7f5d11d75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cdf680 0x560270ce1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:58.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.260+0000 7f5d11d75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560270c3a2e0 con 0x560270cdf680 2024-09-11T23:26:58.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.260+0000 7f5d11d75740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560270be1ce0 con 0x560270cd4dc0 2024-09-11T23:26:58.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.260+0000 7f5d11d75740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560270c362a0 con 0x560270cd44a0 2024-09-11T23:26:58.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.260+0000 7f5d0df39640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cdf680 0x560270ce1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:58.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.260+0000 7f5d0df39640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cdf680 0x560270ce1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36338/0 (socket says 172.21.15.43:36338) 2024-09-11T23:26:58.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.260+0000 7f5d0d738640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560270cd4dc0 0x560270cdf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:58.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.260+0000 7f5d0df39640 1 -- 172.21.15.43:0/342596332 learned_addr learned my addr 172.21.15.43:0/342596332 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:58.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.260+0000 7f5cfe7fc640 1 -- 172.21.15.43:0/1729575406 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f5cf80ef610 con 0x7f5d08101410 2024-09-11T23:26:58.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.260+0000 7f5d0cf37640 1 --2- 172.21.15.43:0/1729575406 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5cf40a2d70 0x7f5cf40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:58.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.260+0000 7f5d0df39640 1 --2- 172.21.15.43:0/342596332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560270cd44a0 0x560270cd4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:58.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.260+0000 7f5d0df39640 1 -- 172.21.15.43:0/342596332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560270cd44a0 msgr2=0x560270cd4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:58.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.260+0000 7f5d0df39640 1 --2- 172.21.15.43:0/342596332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560270cd44a0 0x560270cd4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.260+0000 7f5d0df39640 1 -- 172.21.15.43:0/342596332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560270cd4dc0 msgr2=0x560270cdf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:58.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.260+0000 7f5d0df39640 1 --2- 172.21.15.43:0/342596332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560270cd4dc0 0x560270cdf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.260+0000 7f5d0df39640 1 -- 172.21.15.43:0/342596332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560270bc6440 con 0x560270cdf680 2024-09-11T23:26:58.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.261+0000 7f5d0df39640 1 --2- 172.21.15.43:0/342596332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cdf680 0x560270ce1a70 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5d00005f40 tx=0x7f5d00009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a8d884779c615 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:58.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.261+0000 7f5d0cf37640 1 --2- 172.21.15.43:0/1729575406 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5cf40a2d70 0x7f5cf40a5230 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80c4270 tx=0x7f5cf808aa60 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:58.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.261+0000 7f5cf37fe640 1 -- 172.21.15.43:0/342596332 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00011070 con 0x560270cdf680 2024-09-11T23:26:58.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.261+0000 7f5cf37fe640 1 -- 172.21.15.43:0/342596332 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d00011210 con 0x560270cdf680 2024-09-11T23:26:58.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.261+0000 7f5cf37fe640 1 -- 172.21.15.43:0/342596332 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00010c50 con 0x560270cdf680 2024-09-11T23:26:58.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.261+0000 7f5d11d75740 1 -- 172.21.15.43:0/342596332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cdf680 msgr2=0x560270ce1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:58.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.261+0000 7f5d11d75740 1 --2- 172.21.15.43:0/342596332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cdf680 0x560270ce1a70 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5d00005f40 tx=0x7f5d00009b80 comp rx=0 tx=0).stop 2024-09-11T23:26:58.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.262+0000 7f5d11d75740 1 -- 172.21.15.43:0/342596332 shutdown_connections 2024-09-11T23:26:58.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.262+0000 7f5d11d75740 1 --2- 172.21.15.43:0/342596332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cdf680 0x560270ce1a70 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.262+0000 7f5d11d75740 1 --2- 172.21.15.43:0/342596332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560270cd4dc0 0x560270cdf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.262+0000 7f5d11d75740 1 --2- 172.21.15.43:0/342596332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560270cd44a0 0x560270cd4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.262+0000 7f5d11d75740 1 -- 172.21.15.43:0/342596332 >> 172.21.15.43:0/342596332 conn(0x560270cd0a30 msgr2=0x560270cd0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:58.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.262+0000 7f5d11d75740 1 -- 172.21.15.43:0/342596332 shutdown_connections 2024-09-11T23:26:58.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.262+0000 7f5d11d75740 1 -- 172.21.15.43:0/342596332 wait complete. 2024-09-11T23:26:58.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.262+0000 7f5d11d75740 1 -- start start 2024-09-11T23:26:58.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.262+0000 7f5d11d75740 1 Processor -- start 2024-09-11T23:26:58.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.262+0000 7f5d11d75740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560270cd44a0 0x560270cdef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:58.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.262+0000 7f5d11d75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cd4dc0 0x560270cdc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:58.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.262+0000 7f5d11d75740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560270cdf680 0x560270cdc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:58.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.262+0000 7f5d11d75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560270ce2250 con 0x560270cd4dc0 2024-09-11T23:26:58.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.262+0000 7f5d11d75740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560270bf5050 con 0x560270cdf680 2024-09-11T23:26:58.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.262+0000 7f5d11d75740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560270ce23d0 con 0x560270cd44a0 2024-09-11T23:26:58.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.263+0000 7f5d11d75740 10 client.0 _ll_register_callbacks cb 0x560270c35160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:58.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.263+0000 7f5d0d738640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cd4dc0 0x560270cdc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:58.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.263+0000 7f5d0d738640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cd4dc0 0x560270cdc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36340/0 (socket says 172.21.15.43:36340) 2024-09-11T23:26:58.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.263+0000 7f5d0d738640 1 -- 172.21.15.43:0/2885897329 learned_addr learned my addr 172.21.15.43:0/2885897329 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:58.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.263+0000 7f5d0df39640 1 --2- 172.21.15.43:0/2885897329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560270cd44a0 0x560270cdef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:58.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.263+0000 7f5d0df39640 1 --2- 172.21.15.43:0/2885897329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560270cdf680 0x560270cdc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:58.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.263+0000 7f5d0d738640 1 -- 172.21.15.43:0/2885897329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560270cd44a0 msgr2=0x560270cdef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:58.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.263+0000 7f5d0d738640 1 --2- 172.21.15.43:0/2885897329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560270cd44a0 0x560270cdef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.263+0000 7f5d0d738640 1 -- 172.21.15.43:0/2885897329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560270cdf680 msgr2=0x560270cdc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:58.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.263+0000 7f5d0d738640 1 --2- 172.21.15.43:0/2885897329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560270cdf680 0x560270cdc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.263+0000 7f5d0d738640 1 -- 172.21.15.43:0/2885897329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602709ab3c0 con 0x560270cd4dc0 2024-09-11T23:26:58.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.263+0000 7f5d0d738640 1 --2- 172.21.15.43:0/2885897329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cd4dc0 0x560270cdc200 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5d04010a50 tx=0x7f5d04010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4930e64b2e342bc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:58.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.263+0000 7f5cf0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:26:58.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.263+0000 7f5cf0ff9640 1 -- 172.21.15.43:0/2885897329 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0401e060 con 0x560270cd4dc0 2024-09-11T23:26:58.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.263+0000 7f5cf0ff9640 1 -- 172.21.15.43:0/2885897329 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f5d0400fbd0 con 0x560270cd4dc0 2024-09-11T23:26:58.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.263+0000 7f5d11d75740 1 -- 172.21.15.43:0/2885897329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d00011210 con 0x560270cd4dc0 2024-09-11T23:26:58.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.264+0000 7f5cf0ff9640 1 -- 172.21.15.43:0/2885897329 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d0400fdd0 con 0x560270cd4dc0 2024-09-11T23:26:58.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.264+0000 7f5d11d75740 10 client.16200 fetch_fsmap learned FSMap version 30 2024-09-11T23:26:58.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.264+0000 7f5d11d75740 1 -- 172.21.15.43:0/2885897329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560270cedc00 con 0x560270cd4dc0 2024-09-11T23:26:58.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.264+0000 7f5cf0ff9640 1 -- 172.21.15.43:0/2885897329 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d00011210 con 0x560270cd4dc0 2024-09-11T23:26:58.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.264+0000 7f5cf0ff9640 1 -- 172.21.15.43:0/2885897329 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f5d0400e5e0 con 0x560270cd4dc0 2024-09-11T23:26:58.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.264+0000 7f5d11d75740 10 client.16200 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:26:58.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.264+0000 7f5d11d75740 10 client.16200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:26:58.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.264+0000 7f5d11d75740 20 client.16200 populate_metadata read hostname 'smithi043' 2024-09-11T23:26:58.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.264+0000 7f5d11d75740 1 --2- 172.21.15.43:0/2885897329 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560270cef0c0 0x560270cf1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:58.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.264+0000 7f5d11d75740 4 client.16200 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:26:58.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.264+0000 7f5d11d75740 1 -- 172.21.15.43:0/2885897329 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560270c47850 con 0x560270cef0c0 2024-09-11T23:26:58.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.264+0000 7f5d0df39640 1 --2- 172.21.15.43:0/2885897329 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560270cef0c0 0x560270cf1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:58.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.265+0000 7f5d0df39640 1 --2- 172.21.15.43:0/2885897329 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560270cef0c0 0x560270cf1500 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b161787426d2f0ab server_cookie=63100bf3d8c2bcbe in_seq=0 out_seq=0 2024-09-11T23:26:58.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.265+0000 7f5cf0ff9640 10 client.16200 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:58.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.266+0000 7f5cf0ff9640 1 -- 172.21.15.43:0/2885897329 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560270c47850 con 0x560270cef0c0 2024-09-11T23:26:58.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.266+0000 7f5cf0ff9640 10 client.16200 handle_command_reply: tid=0 2024-09-11T23:26:58.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.266+0000 7f5d11d75740 1 client.16200 shutdown 2024-09-11T23:26:58.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.266+0000 7f5d11d75740 1 -- 172.21.15.43:0/2885897329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cd4dc0 msgr2=0x560270cdc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:58.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.266+0000 7f5d11d75740 1 --2- 172.21.15.43:0/2885897329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cd4dc0 0x560270cdc200 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5d04010a50 tx=0x7f5d04010f10 comp rx=0 tx=0).stop 2024-09-11T23:26:58.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.266+0000 7f5d11d75740 1 -- 172.21.15.43:0/2885897329 shutdown_connections 2024-09-11T23:26:58.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.266+0000 7f5d11d75740 1 --2- 172.21.15.43:0/2885897329 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560270cef0c0 0x560270cf1500 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.266+0000 7f5d11d75740 1 --2- 172.21.15.43:0/2885897329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560270cdf680 0x560270cdc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.266+0000 7f5d11d75740 1 --2- 172.21.15.43:0/2885897329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cd4dc0 0x560270cdc200 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.266+0000 7f5d11d75740 1 --2- 172.21.15.43:0/2885897329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560270cd44a0 0x560270cdef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.266+0000 7f5cf0ff9640 0 client.16200 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:58.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.266+0000 7f5d11d75740 1 -- 172.21.15.43:0/2885897329 >> 172.21.15.43:0/2885897329 conn(0x560270cd0a30 msgr2=0x560270cd6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:58.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.266+0000 7f5d11d75740 1 -- 172.21.15.43:0/2885897329 shutdown_connections 2024-09-11T23:26:58.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.266+0000 7f5d11d75740 1 -- 172.21.15.43:0/2885897329 wait complete. 2024-09-11T23:26:58.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.266+0000 7f5d11d75740 20 client.16200 trim_cache size 0 max 16384 2024-09-11T23:26:58.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.283+0000 7f5d11d75740 1 Processor -- start 2024-09-11T23:26:58.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.283+0000 7f5d11d75740 1 -- start start 2024-09-11T23:26:58.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.283+0000 7f5d11d75740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560270cd44a0 0x560270ceb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:58.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.283+0000 7f5d11d75740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560270cd4dc0 0x560270cf6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:58.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.283+0000 7f5d11d75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cdf680 0x560270cf7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:58.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.283+0000 7f5d11d75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d081b0170 con 0x560270cdf680 2024-09-11T23:26:58.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.283+0000 7f5d11d75740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5d081ada40 con 0x560270cd4dc0 2024-09-11T23:26:58.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.283+0000 7f5d11d75740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5d081ab890 con 0x560270cd44a0 2024-09-11T23:26:58.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.284+0000 7f5d0df39640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cdf680 0x560270cf7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:58.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.284+0000 7f5d0df39640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cdf680 0x560270cf7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36344/0 (socket says 172.21.15.43:36344) 2024-09-11T23:26:58.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.284+0000 7f5d0df39640 1 -- 172.21.15.43:0/1169900970 learned_addr learned my addr 172.21.15.43:0/1169900970 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:58.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.284+0000 7f5d0d738640 1 --2- 172.21.15.43:0/1169900970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560270cd4dc0 0x560270cf6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:58.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.284+0000 7f5d0df39640 1 --2- 172.21.15.43:0/1169900970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560270cd44a0 0x560270ceb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:58.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.284+0000 7f5d0df39640 1 -- 172.21.15.43:0/1169900970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560270cd44a0 msgr2=0x560270ceb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:58.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.284+0000 7f5d0df39640 1 --2- 172.21.15.43:0/1169900970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560270cd44a0 0x560270ceb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.284+0000 7f5d0df39640 1 -- 172.21.15.43:0/1169900970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560270cd4dc0 msgr2=0x560270cf6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:58.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.284+0000 7f5d0df39640 1 --2- 172.21.15.43:0/1169900970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560270cd4dc0 0x560270cf6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.284+0000 7f5d0df39640 1 -- 172.21.15.43:0/1169900970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560270cd0860 con 0x560270cdf680 2024-09-11T23:26:58.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.284+0000 7f5d0df39640 1 --2- 172.21.15.43:0/1169900970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cdf680 0x560270cf7310 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5d00011b30 tx=0x7f5d00009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e631fd821f475e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:58.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.284+0000 7f5cf2ffd640 1 -- 172.21.15.43:0/1169900970 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00004e50 con 0x560270cdf680 2024-09-11T23:26:58.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.284+0000 7f5cf2ffd640 1 -- 172.21.15.43:0/1169900970 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d0000fd30 con 0x560270cdf680 2024-09-11T23:26:58.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.284+0000 7f5cf2ffd640 1 -- 172.21.15.43:0/1169900970 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00005740 con 0x560270cdf680 2024-09-11T23:26:58.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.285+0000 7f5d11d75740 1 -- 172.21.15.43:0/1169900970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cdf680 msgr2=0x560270cf7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:58.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.285+0000 7f5d11d75740 1 --2- 172.21.15.43:0/1169900970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cdf680 0x560270cf7310 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5d00011b30 tx=0x7f5d00009b80 comp rx=0 tx=0).stop 2024-09-11T23:26:58.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.285+0000 7f5d11d75740 1 -- 172.21.15.43:0/1169900970 shutdown_connections 2024-09-11T23:26:58.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.285+0000 7f5d11d75740 1 --2- 172.21.15.43:0/1169900970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cdf680 0x560270cf7310 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.285+0000 7f5d11d75740 1 --2- 172.21.15.43:0/1169900970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560270cd4dc0 0x560270cf6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.285+0000 7f5d11d75740 1 --2- 172.21.15.43:0/1169900970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560270cd44a0 0x560270ceb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.285+0000 7f5d11d75740 1 -- 172.21.15.43:0/1169900970 >> 172.21.15.43:0/1169900970 conn(0x560270cd0a30 msgr2=0x560270cdb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:58.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.285+0000 7f5d11d75740 1 -- 172.21.15.43:0/1169900970 shutdown_connections 2024-09-11T23:26:58.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.285+0000 7f5d11d75740 1 -- 172.21.15.43:0/1169900970 wait complete. 2024-09-11T23:26:58.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.285+0000 7f5d11d75740 1 -- start start 2024-09-11T23:26:58.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.285+0000 7f5d11d75740 1 Processor -- start 2024-09-11T23:26:58.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.286+0000 7f5d11d75740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cd44a0 0x560270ce7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:58.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.286+0000 7f5d11d75740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560270cd4dc0 0x560270ce2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:58.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.286+0000 7f5d11d75740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560270cdf680 0x560270ce3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:58.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.286+0000 7f5d11d75740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d081abf00 con 0x560270cd44a0 2024-09-11T23:26:58.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.286+0000 7f5d11d75740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5d081ad370 con 0x560270cdf680 2024-09-11T23:26:58.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.286+0000 7f5d11d75740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5d081ac5d0 con 0x560270cd4dc0 2024-09-11T23:26:58.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.286+0000 7f5d11d75740 10 client.0 _ll_register_callbacks cb 0x7f5d0400a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:26:58.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.286+0000 7f5d0df39640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cd44a0 0x560270ce7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:58.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.286+0000 7f5d0df39640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cd44a0 0x560270ce7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36358/0 (socket says 172.21.15.43:36358) 2024-09-11T23:26:58.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.286+0000 7f5d0df39640 1 -- 172.21.15.43:0/1706513754 learned_addr learned my addr 172.21.15.43:0/1706513754 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:26:58.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.286+0000 7f5d0d738640 1 --2- 172.21.15.43:0/1706513754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560270cd4dc0 0x560270ce2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:58.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.286+0000 7f5d0df39640 1 --2- 172.21.15.43:0/1706513754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560270cdf680 0x560270ce3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:58.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.286+0000 7f5d0df39640 1 -- 172.21.15.43:0/1706513754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560270cd4dc0 msgr2=0x560270ce2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:58.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.286+0000 7f5d0df39640 1 --2- 172.21.15.43:0/1706513754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560270cd4dc0 0x560270ce2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.286+0000 7f5d0df39640 1 -- 172.21.15.43:0/1706513754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560270cdf680 msgr2=0x560270ce3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:26:58.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.286+0000 7f5d0df39640 1 --2- 172.21.15.43:0/1706513754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560270cdf680 0x560270ce3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.286+0000 7f5d0df39640 1 -- 172.21.15.43:0/1706513754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560270ce38b0 con 0x560270cd44a0 2024-09-11T23:26:58.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.287+0000 7f5d0df39640 1 --2- 172.21.15.43:0/1706513754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cd44a0 0x560270ce7ad0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009a70 tx=0x7f5d000106f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f7990bbae3c4f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:26:58.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.287+0000 7f5cf0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:26:58.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.287+0000 7f5cf0ff9640 1 -- 172.21.15.43:0/1706513754 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00040060 con 0x560270cd44a0 2024-09-11T23:26:58.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.287+0000 7f5cf0ff9640 1 -- 172.21.15.43:0/1706513754 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f5d0000f850 con 0x560270cd44a0 2024-09-11T23:26:58.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.287+0000 7f5d11d75740 1 -- 172.21.15.43:0/1706513754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d0000fd30 con 0x560270cd44a0 2024-09-11T23:26:58.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.287+0000 7f5cf0ff9640 1 -- 172.21.15.43:0/1706513754 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d0000fa50 con 0x560270cd44a0 2024-09-11T23:26:58.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.287+0000 7f5d11d75740 10 client.16206 fetch_fsmap learned FSMap version 30 2024-09-11T23:26:58.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.287+0000 7f5d11d75740 1 -- 172.21.15.43:0/1706513754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560270d59b10 con 0x560270cd44a0 2024-09-11T23:26:58.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.287+0000 7f5cf0ff9640 1 -- 172.21.15.43:0/1706513754 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d0000fd30 con 0x560270cd44a0 2024-09-11T23:26:58.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.287+0000 7f5cf0ff9640 1 -- 172.21.15.43:0/1706513754 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f5d0001d020 con 0x560270cd44a0 2024-09-11T23:26:58.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.287+0000 7f5d11d75740 10 client.16206 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:26:58.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.287+0000 7f5d11d75740 10 client.16206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:26:58.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.287+0000 7f5d11d75740 20 client.16206 populate_metadata read hostname 'smithi043' 2024-09-11T23:26:58.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.287+0000 7f5d11d75740 1 --2- 172.21.15.43:0/1706513754 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560270d59f90 0x560270d5c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:26:58.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.287+0000 7f5d11d75740 4 client.16206 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:26:58.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.287+0000 7f5d11d75740 1 -- 172.21.15.43:0/1706513754 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560270c41d60 con 0x560270d59f90 2024-09-11T23:26:58.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.288+0000 7f5d0d738640 1 --2- 172.21.15.43:0/1706513754 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560270d59f90 0x560270d5c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:26:58.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.288+0000 7f5d0d738640 1 --2- 172.21.15.43:0/1706513754 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560270d59f90 0x560270d5c380 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=282736722c32bf70 server_cookie=58579b32ddbf56f5 in_seq=0 out_seq=0 2024-09-11T23:26:58.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.288+0000 7f5cf0ff9640 10 client.16206 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:58.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.288+0000 7f5cf0ff9640 1 -- 172.21.15.43:0/1706513754 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560270c41d60 con 0x560270d59f90 2024-09-11T23:26:58.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.288+0000 7f5cf0ff9640 10 client.16206 handle_command_reply: tid=0 2024-09-11T23:26:58.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.288+0000 7f5d11d75740 1 client.16206 shutdown 2024-09-11T23:26:58.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.288+0000 7f5d11d75740 1 -- 172.21.15.43:0/1706513754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cd44a0 msgr2=0x560270ce7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:58.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.288+0000 7f5d11d75740 1 --2- 172.21.15.43:0/1706513754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cd44a0 0x560270ce7ad0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009a70 tx=0x7f5d000106f0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.288+0000 7f5d11d75740 1 -- 172.21.15.43:0/1706513754 shutdown_connections 2024-09-11T23:26:58.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.288+0000 7f5d11d75740 1 --2- 172.21.15.43:0/1706513754 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560270d59f90 0x560270d5c380 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.288+0000 7f5d11d75740 1 --2- 172.21.15.43:0/1706513754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560270cdf680 0x560270ce3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.288+0000 7f5d11d75740 1 --2- 172.21.15.43:0/1706513754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560270cd4dc0 0x560270ce2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.288+0000 7f5d11d75740 1 --2- 172.21.15.43:0/1706513754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560270cd44a0 0x560270ce7ad0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.288+0000 7f5d11d75740 1 -- 172.21.15.43:0/1706513754 >> 172.21.15.43:0/1706513754 conn(0x560270cd0a30 msgr2=0x560270cdda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:58.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.288+0000 7f5cf0ff9640 0 client.16206 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:26:58.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.289+0000 7f5d11d75740 1 -- 172.21.15.43:0/1706513754 shutdown_connections 2024-09-11T23:26:58.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.289+0000 7f5d11d75740 1 -- 172.21.15.43:0/1706513754 wait complete. 2024-09-11T23:26:58.644 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:26:58.644 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-11T23:26:58.644 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-11T23:26:58.644 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:26:58.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.289+0000 7f5d11d75740 20 client.16206 trim_cache size 0 max 16384 2024-09-11T23:26:58.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.289+0000 7f5d0f99c640 1 -- 172.21.15.43:0/1729575406 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5cf40a2d70 msgr2=0x7f5cf40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:58.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.289+0000 7f5d0f99c640 1 --2- 172.21.15.43:0/1729575406 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5cf40a2d70 0x7f5cf40a5230 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80c4270 tx=0x7f5cf808aa60 comp rx=0 tx=0).stop 2024-09-11T23:26:58.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.289+0000 7f5d0f99c640 1 -- 172.21.15.43:0/1729575406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d08101410 msgr2=0x7f5d08198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:26:58.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.289+0000 7f5d0f99c640 1 --2- 172.21.15.43:0/1729575406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d08101410 0x7f5d08198960 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009230 tx=0x7f5cf8004270 comp rx=0 tx=0).stop 2024-09-11T23:26:58.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.289+0000 7f5d0f99c640 1 -- 172.21.15.43:0/1729575406 shutdown_connections 2024-09-11T23:26:58.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.289+0000 7f5d0f99c640 1 --2- 172.21.15.43:0/1729575406 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5cf40a2d70 0x7f5cf40a5230 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.289+0000 7f5d0f99c640 1 --2- 172.21.15.43:0/1729575406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d08106a70 0x7f5d0819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.289+0000 7f5d0f99c640 1 --2- 172.21.15.43:0/1729575406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d08101410 0x7f5d08198960 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.289+0000 7f5d0f99c640 1 --2- 172.21.15.43:0/1729575406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d08100a70 0x7f5d08198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:26:58.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.289+0000 7f5d0f99c640 1 -- 172.21.15.43:0/1729575406 >> 172.21.15.43:0/1729575406 conn(0x7f5d080fc7d0 msgr2=0x7f5d0806aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:26:58.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.289+0000 7f5d0f99c640 1 -- 172.21.15.43:0/1729575406 shutdown_connections 2024-09-11T23:26:58.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:26:57.289+0000 7f5d0f99c640 1 -- 172.21.15.43:0/1729575406 wait complete. 2024-09-11T23:27:00.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:26:59 smithi090 ceph-mon[35130]: pgmap v158: 97 pgs: 97 active+clean; 19 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 47 MiB/s wr, 8.16k op/s 2024-09-11T23:27:00.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:26:59 smithi097 ceph-mon[35562]: pgmap v158: 97 pgs: 97 active+clean; 19 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 47 MiB/s wr, 8.16k op/s 2024-09-11T23:27:00.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:26:59 smithi043 ceph-mon[30950]: pgmap v158: 97 pgs: 97 active+clean; 19 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 47 MiB/s wr, 8.16k op/s 2024-09-11T23:27:01.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:01 smithi043 ceph-mon[30950]: pgmap v159: 97 pgs: 97 active+clean; 19 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 41 MiB/s wr, 7.09k op/s 2024-09-11T23:27:02.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:01 smithi090 ceph-mon[35130]: pgmap v159: 97 pgs: 97 active+clean; 19 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 41 MiB/s wr, 7.09k op/s 2024-09-11T23:27:02.050 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:01 smithi097 ceph-mon[35562]: pgmap v159: 97 pgs: 97 active+clean; 19 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 41 MiB/s wr, 7.09k op/s 2024-09-11T23:27:04.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:03 smithi090 ceph-mon[35130]: pgmap v160: 97 pgs: 97 active+clean; 19 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 34 MiB/s wr, 5.96k op/s 2024-09-11T23:27:04.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:03 smithi097 ceph-mon[35562]: pgmap v160: 97 pgs: 97 active+clean; 19 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 34 MiB/s wr, 5.96k op/s 2024-09-11T23:27:04.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:03 smithi043 ceph-mon[30950]: pgmap v160: 97 pgs: 97 active+clean; 19 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 34 MiB/s wr, 5.96k op/s 2024-09-11T23:27:05.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:05 smithi043 ceph-mon[30950]: pgmap v161: 97 pgs: 97 active+clean; 19 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 41 MiB/s wr, 7.08k op/s 2024-09-11T23:27:06.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:05 smithi090 ceph-mon[35130]: pgmap v161: 97 pgs: 97 active+clean; 19 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 41 MiB/s wr, 7.08k op/s 2024-09-11T23:27:06.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:05 smithi097 ceph-mon[35562]: pgmap v161: 97 pgs: 97 active+clean; 19 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 41 MiB/s wr, 7.08k op/s 2024-09-11T23:27:07.528 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:27:07.529 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:27:07.748 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:07 smithi043 ceph-mon[30950]: pgmap v162: 97 pgs: 97 active+clean; 19 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 MiB/s wr, 5.07k op/s 2024-09-11T23:27:08.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:07 smithi090 ceph-mon[35130]: pgmap v162: 97 pgs: 97 active+clean; 19 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 MiB/s wr, 5.07k op/s 2024-09-11T23:27:08.040 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:27:08.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:07 smithi097 ceph-mon[35562]: pgmap v162: 97 pgs: 97 active+clean; 19 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 MiB/s wr, 5.07k op/s 2024-09-11T23:27:09.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.705+0000 7f24b2998640 1 -- 172.21.15.43:0/2651556704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ac10c170 msgr2=0x7f24ac10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:09.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.705+0000 7f24b2998640 1 --2- 172.21.15.43:0/2651556704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ac10c170 0x7f24ac10e610 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009ca0 tx=0x7f24a402a950 comp rx=0 tx=0).stop 2024-09-11T23:27:09.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.705+0000 7f24b2998640 1 -- 172.21.15.43:0/2651556704 shutdown_connections 2024-09-11T23:27:09.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.705+0000 7f24b2998640 1 --2- 172.21.15.43:0/2651556704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ac10c170 0x7f24ac10e610 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:09.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.705+0000 7f24b2998640 1 --2- 172.21.15.43:0/2651556704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24ac103660 0x7f24ac10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:09.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.705+0000 7f24b2998640 1 --2- 172.21.15.43:0/2651556704 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24ac102d40 0x7f24ac103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:09.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.705+0000 7f24b2998640 1 -- 172.21.15.43:0/2651556704 >> 172.21.15.43:0/2651556704 conn(0x7f24ac0fc7a0 msgr2=0x7f24ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:09.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.705+0000 7f24b2998640 1 -- 172.21.15.43:0/2651556704 shutdown_connections 2024-09-11T23:27:09.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.705+0000 7f24b2998640 1 -- 172.21.15.43:0/2651556704 wait complete. 2024-09-11T23:27:09.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.705+0000 7f24b2998640 1 Processor -- start 2024-09-11T23:27:09.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.705+0000 7f24b2998640 1 -- start start 2024-09-11T23:27:09.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.706+0000 7f24b2998640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24ac102d40 0x7f24ac1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:09.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.706+0000 7f24b2998640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ac103660 0x7f24ac100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:09.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.706+0000 7f24b2998640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24ac10c170 0x7f24ac101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:09.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.706+0000 7f24b2998640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac111f70 con 0x7f24ac103660 2024-09-11T23:27:09.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.706+0000 7f24b2998640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac111df0 con 0x7f24ac102d40 2024-09-11T23:27:09.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.706+0000 7f24b2998640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac1120f0 con 0x7f24ac10c170 2024-09-11T23:27:09.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.706+0000 7f24ab7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ac103660 0x7f24ac100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:09.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.706+0000 7f24ab7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ac103660 0x7f24ac100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47756/0 (socket says 172.21.15.43:47756) 2024-09-11T23:27:09.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.706+0000 7f24ab7fe640 1 -- 172.21.15.43:0/367987006 learned_addr learned my addr 172.21.15.43:0/367987006 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:09.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.706+0000 7f24abfff640 1 --2- 172.21.15.43:0/367987006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24ac102d40 0x7f24ac1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:09.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.706+0000 7f24b0f35640 1 --2- 172.21.15.43:0/367987006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24ac10c170 0x7f24ac101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:09.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.706+0000 7f24ab7fe640 1 -- 172.21.15.43:0/367987006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24ac10c170 msgr2=0x7f24ac101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:09.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.706+0000 7f24ab7fe640 1 --2- 172.21.15.43:0/367987006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24ac10c170 0x7f24ac101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:09.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.706+0000 7f24ab7fe640 1 -- 172.21.15.43:0/367987006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24ac102d40 msgr2=0x7f24ac1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:09.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.706+0000 7f24ab7fe640 1 --2- 172.21.15.43:0/367987006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24ac102d40 0x7f24ac1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:09.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.706+0000 7f24ab7fe640 1 -- 172.21.15.43:0/367987006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ac1a67f0 con 0x7f24ac103660 2024-09-11T23:27:09.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.706+0000 7f24b0f35640 1 --2- 172.21.15.43:0/367987006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24ac10c170 0x7f24ac101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:27:09.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.706+0000 7f24ab7fe640 1 --2- 172.21.15.43:0/367987006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ac103660 0x7f24ac100ee0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f24a000c970 tx=0x7f24a000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:09.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.707+0000 7f24a97fa640 1 -- 172.21.15.43:0/367987006 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a0007bb0 con 0x7f24ac103660 2024-09-11T23:27:09.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.707+0000 7f24a97fa640 1 -- 172.21.15.43:0/367987006 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a0007d50 con 0x7f24ac103660 2024-09-11T23:27:09.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.707+0000 7f24b2998640 1 -- 172.21.15.43:0/367987006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ac1a6ae0 con 0x7f24ac103660 2024-09-11T23:27:09.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.707+0000 7f24a97fa640 1 -- 172.21.15.43:0/367987006 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a0018820 con 0x7f24ac103660 2024-09-11T23:27:09.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.707+0000 7f24b2998640 1 -- 172.21.15.43:0/367987006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ac1a7070 con 0x7f24ac103660 2024-09-11T23:27:09.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.708+0000 7f24b2998640 1 -- 172.21.15.43:0/367987006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24ac103120 con 0x7f24ac103660 2024-09-11T23:27:09.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.709+0000 7f24a97fa640 1 -- 172.21.15.43:0/367987006 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f24a00189c0 con 0x7f24ac103660 2024-09-11T23:27:09.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.710+0000 7f24a97fa640 1 --2- 172.21.15.43:0/367987006 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f24940a2d70 0x7f24940a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:09.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.710+0000 7f24a97fa640 1 -- 172.21.15.43:0/367987006 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f24a00cf400 con 0x7f24ac103660 2024-09-11T23:27:09.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.710+0000 7f24abfff640 1 --2- 172.21.15.43:0/367987006 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f24940a2d70 0x7f24940a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:09.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.710+0000 7f24abfff640 1 --2- 172.21.15.43:0/367987006 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f24940a2d70 0x7f24940a5230 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f249c00bbe0 tx=0x7f249c00b390 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:09.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.714+0000 7f24a97fa640 1 -- 172.21.15.43:0/367987006 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f24a0014030 con 0x7f24ac103660 2024-09-11T23:27:09.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.944+0000 7f24b2998640 1 -- 172.21.15.43:0/367987006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24ac1a7370 con 0x7f24ac103660 2024-09-11T23:27:09.944 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:09 smithi043 ceph-mon[30950]: pgmap v163: 97 pgs: 97 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 33 MiB/s wr, 5.90k op/s 2024-09-11T23:27:09.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.945+0000 7f24a97fa640 1 -- 172.21.15.43:0/367987006 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11893 (secure 0 0 0) 0x7f24a009c020 con 0x7f24ac103660 2024-09-11T23:27:09.945 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:27:09.945 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"btime":"2024-09-11T23:25:40:446496+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:25:39.727562+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25571":{"gid":25571,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/568491708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":568491708},{"type":"v1","addr":"172.21.15.90:6835","nonce":568491708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:27:09.946 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-09-11T23:27:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.948+0000 7f24b2998640 1 -- 172.21.15.43:0/367987006 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f24940a2d70 msgr2=0x7f24940a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.948+0000 7f24b2998640 1 --2- 172.21.15.43:0/367987006 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f24940a2d70 0x7f24940a5230 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f249c00bbe0 tx=0x7f249c00b390 comp rx=0 tx=0).stop 2024-09-11T23:27:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.948+0000 7f24b2998640 1 -- 172.21.15.43:0/367987006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ac103660 msgr2=0x7f24ac100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.948+0000 7f24b2998640 1 --2- 172.21.15.43:0/367987006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ac103660 0x7f24ac100ee0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f24a000c970 tx=0x7f24a000ce30 comp rx=0 tx=0).stop 2024-09-11T23:27:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.948+0000 7f24b2998640 1 -- 172.21.15.43:0/367987006 shutdown_connections 2024-09-11T23:27:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.948+0000 7f24b2998640 1 --2- 172.21.15.43:0/367987006 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f24940a2d70 0x7f24940a5230 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.948+0000 7f24b2998640 1 --2- 172.21.15.43:0/367987006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24ac10c170 0x7f24ac101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.948+0000 7f24b2998640 1 --2- 172.21.15.43:0/367987006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ac103660 0x7f24ac100ee0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.948+0000 7f24b2998640 1 --2- 172.21.15.43:0/367987006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24ac102d40 0x7f24ac1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.948+0000 7f24b2998640 1 -- 172.21.15.43:0/367987006 >> 172.21.15.43:0/367987006 conn(0x7f24ac0fc7a0 msgr2=0x7f24ac0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.948+0000 7f24b2998640 1 -- 172.21.15.43:0/367987006 shutdown_connections 2024-09-11T23:27:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:09.948+0000 7f24b2998640 1 -- 172.21.15.43:0/367987006 wait complete. 2024-09-11T23:27:10.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:09 smithi090 ceph-mon[35130]: pgmap v163: 97 pgs: 97 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 33 MiB/s wr, 5.90k op/s 2024-09-11T23:27:10.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:09 smithi097 ceph-mon[35562]: pgmap v163: 97 pgs: 97 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 33 MiB/s wr, 5.90k op/s 2024-09-11T23:27:11.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:10 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:27:11.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:10 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/367987006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:27:11.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:10 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:27:11.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:10 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/367987006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:27:11.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:10 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:27:11.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:10 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/367987006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:27:11.621 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:27:11.880 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:27:12.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:11 smithi090 ceph-mon[35130]: pgmap v164: 97 pgs: 97 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 5.09k op/s 2024-09-11T23:27:12.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:11 smithi097 ceph-mon[35562]: pgmap v164: 97 pgs: 97 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 5.09k op/s 2024-09-11T23:27:12.344 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:11 smithi043 ceph-mon[30950]: pgmap v164: 97 pgs: 97 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 5.09k op/s 2024-09-11T23:27:12.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.733+0000 7f26a2d10640 1 -- 172.21.15.43:0/2959431324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269c101450 msgr2=0x7f269c105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:12.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.733+0000 7f26a2d10640 1 --2- 172.21.15.43:0/2959431324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269c101450 0x7f269c105920 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2690009900 tx=0x7f269002a470 comp rx=0 tx=0).stop 2024-09-11T23:27:12.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.733+0000 7f26a2d10640 1 -- 172.21.15.43:0/2959431324 shutdown_connections 2024-09-11T23:27:12.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.733+0000 7f26a2d10640 1 --2- 172.21.15.43:0/2959431324 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f269c105f40 0x7f269c1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.733+0000 7f26a2d10640 1 --2- 172.21.15.43:0/2959431324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269c101450 0x7f269c105920 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.733+0000 7f26a2d10640 1 --2- 172.21.15.43:0/2959431324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f269c100b30 0x7f269c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.733+0000 7f26a2d10640 1 -- 172.21.15.43:0/2959431324 >> 172.21.15.43:0/2959431324 conn(0x7f269c0fc7a0 msgr2=0x7f269c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:12.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.733+0000 7f26a2d10640 1 -- 172.21.15.43:0/2959431324 shutdown_connections 2024-09-11T23:27:12.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.733+0000 7f26a2d10640 1 -- 172.21.15.43:0/2959431324 wait complete. 2024-09-11T23:27:12.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.734+0000 7f26a2d10640 1 Processor -- start 2024-09-11T23:27:12.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.734+0000 7f26a2d10640 1 -- start start 2024-09-11T23:27:12.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.734+0000 7f26a2d10640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f269c100b30 0x7f269c1a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:12.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.734+0000 7f26a2d10640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f269c101450 0x7f269c1a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:12.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.734+0000 7f26a2d10640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269c105f40 0x7f269c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:12.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.734+0000 7f26a2d10640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f269c118540 con 0x7f269c105f40 2024-09-11T23:27:12.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.734+0000 7f26a2d10640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f269c1183c0 con 0x7f269c100b30 2024-09-11T23:27:12.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.734+0000 7f26a2d10640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f269c1186c0 con 0x7f269c101450 2024-09-11T23:27:12.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.734+0000 7f26a12ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269c105f40 0x7f269c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:12.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.734+0000 7f26a0aac640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f269c100b30 0x7f269c1a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:12.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.734+0000 7f269bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f269c101450 0x7f269c1a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:12.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.734+0000 7f26a12ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269c105f40 0x7f269c19e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60058/0 (socket says 172.21.15.43:60058) 2024-09-11T23:27:12.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.734+0000 7f26a0aac640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f269c100b30 0x7f269c1a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:57068/0 (socket says 172.21.15.43:57068) 2024-09-11T23:27:12.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.734+0000 7f26a12ad640 1 -- 172.21.15.43:0/3808423157 learned_addr learned my addr 172.21.15.43:0/3808423157 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:12.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.735+0000 7f26a12ad640 1 -- 172.21.15.43:0/3808423157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f269c101450 msgr2=0x7f269c1a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:12.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.735+0000 7f26a12ad640 1 --2- 172.21.15.43:0/3808423157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f269c101450 0x7f269c1a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.735+0000 7f26a12ad640 1 -- 172.21.15.43:0/3808423157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f269c100b30 msgr2=0x7f269c1a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:12.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.735+0000 7f26a12ad640 1 --2- 172.21.15.43:0/3808423157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f269c100b30 0x7f269c1a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.735+0000 7f26a12ad640 1 -- 172.21.15.43:0/3808423157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269c19efd0 con 0x7f269c105f40 2024-09-11T23:27:12.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.735+0000 7f269bfff640 1 --2- 172.21.15.43:0/3808423157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f269c101450 0x7f269c1a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:27:12.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.735+0000 7f26a0aac640 1 --2- 172.21.15.43:0/3808423157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f269c100b30 0x7f269c1a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:27:12.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.735+0000 7f26a12ad640 1 --2- 172.21.15.43:0/3808423157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269c105f40 0x7f269c19e830 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f269400cce0 tx=0x7f269400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:12.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.735+0000 7f2699ffb640 1 -- 172.21.15.43:0/3808423157 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269400f450 con 0x7f269c105f40 2024-09-11T23:27:12.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.735+0000 7f26a2d10640 1 -- 172.21.15.43:0/3808423157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f269c19f2c0 con 0x7f269c105f40 2024-09-11T23:27:12.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.735+0000 7f2699ffb640 1 -- 172.21.15.43:0/3808423157 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f269400fad0 con 0x7f269c105f40 2024-09-11T23:27:12.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.735+0000 7f2699ffb640 1 -- 172.21.15.43:0/3808423157 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694018c10 con 0x7f269c105f40 2024-09-11T23:27:12.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.735+0000 7f26a2d10640 1 -- 172.21.15.43:0/3808423157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f269c1ab510 con 0x7f269c105f40 2024-09-11T23:27:12.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.738+0000 7f2699ffb640 1 -- 172.21.15.43:0/3808423157 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f269400f5f0 con 0x7f269c105f40 2024-09-11T23:27:12.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.738+0000 7f26a50e9740 1 Processor -- start 2024-09-11T23:27:12.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.738+0000 7f26a50e9740 1 -- start start 2024-09-11T23:27:12.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.738+0000 7f26a50e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4d0b40 0x557def4cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:12.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.738+0000 7f26a50e9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557def4cf5d0 0x557def4cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:12.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.738+0000 7f26a50e9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557def4da370 0x557def4dc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:12.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.738+0000 7f26a50e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557def4352f0 con 0x557def4d0b40 2024-09-11T23:27:12.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.738+0000 7f26a50e9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557def3dccf0 con 0x557def4cf5d0 2024-09-11T23:27:12.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.738+0000 7f26a50e9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557def4312b0 con 0x557def4da370 2024-09-11T23:27:12.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.738+0000 7f269bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4d0b40 0x557def4cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:12.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.738+0000 7f269bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4d0b40 0x557def4cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60064/0 (socket says 172.21.15.43:60064) 2024-09-11T23:27:12.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.738+0000 7f269bfff640 1 -- 172.21.15.43:0/219948179 learned_addr learned my addr 172.21.15.43:0/219948179 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:12.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.738+0000 7f26a0aac640 1 --2- 172.21.15.43:0/219948179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557def4cf5d0 0x557def4cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:12.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.739+0000 7f2699ffb640 1 --2- 172.21.15.43:0/3808423157 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f26840a2cf0 0x7f26840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:12.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.739+0000 7f269bfff640 1 --2- 172.21.15.43:0/219948179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557def4da370 0x557def4dc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:12.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.739+0000 7f269bfff640 1 -- 172.21.15.43:0/219948179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557def4da370 msgr2=0x557def4dc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:12.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.739+0000 7f269bfff640 1 --2- 172.21.15.43:0/219948179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557def4da370 0x557def4dc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.739+0000 7f26a12ad640 1 --2- 172.21.15.43:0/3808423157 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f26840a2cf0 0x7f26840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:12.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.739+0000 7f269bfff640 1 -- 172.21.15.43:0/219948179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557def4cf5d0 msgr2=0x557def4cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:12.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.739+0000 7f269bfff640 1 --2- 172.21.15.43:0/219948179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557def4cf5d0 0x557def4cfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.739+0000 7f269bfff640 1 -- 172.21.15.43:0/219948179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557def3c13a0 con 0x557def4d0b40 2024-09-11T23:27:12.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.739+0000 7f2699ffb640 1 -- 172.21.15.43:0/3808423157 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f26940cfa30 con 0x7f269c105f40 2024-09-11T23:27:12.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.739+0000 7f269bfff640 1 --2- 172.21.15.43:0/219948179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4d0b40 0x557def4cf090 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2690000c00 tx=0x7f26900314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf41daf69c5286c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:12.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.740+0000 7f2682ffd640 1 -- 172.21.15.43:0/219948179 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2690044070 con 0x557def4d0b40 2024-09-11T23:27:12.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.740+0000 7f2682ffd640 1 -- 172.21.15.43:0/219948179 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2690005500 con 0x557def4d0b40 2024-09-11T23:27:12.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.740+0000 7f2682ffd640 1 -- 172.21.15.43:0/219948179 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2690035040 con 0x557def4d0b40 2024-09-11T23:27:12.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.741+0000 7f26a50e9740 1 -- 172.21.15.43:0/219948179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4d0b40 msgr2=0x557def4cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:12.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.741+0000 7f26a12ad640 1 --2- 172.21.15.43:0/3808423157 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f26840a2cf0 0x7f26840a51b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f269409bbf0 tx=0x7f26940186b0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:12.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.741+0000 7f26a50e9740 1 --2- 172.21.15.43:0/219948179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4d0b40 0x557def4cf090 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2690000c00 tx=0x7f26900314e0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.741+0000 7f26a50e9740 1 -- 172.21.15.43:0/219948179 shutdown_connections 2024-09-11T23:27:12.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.741+0000 7f26a50e9740 1 --2- 172.21.15.43:0/219948179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557def4da370 0x557def4dc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.741+0000 7f26a50e9740 1 --2- 172.21.15.43:0/219948179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557def4cf5d0 0x557def4cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.741+0000 7f26a50e9740 1 --2- 172.21.15.43:0/219948179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4d0b40 0x557def4cf090 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.741+0000 7f26a50e9740 1 -- 172.21.15.43:0/219948179 >> 172.21.15.43:0/219948179 conn(0x557def498310 msgr2=0x557def4cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:12.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.741+0000 7f26a50e9740 1 -- 172.21.15.43:0/219948179 shutdown_connections 2024-09-11T23:27:12.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.741+0000 7f26a50e9740 1 -- 172.21.15.43:0/219948179 wait complete. 2024-09-11T23:27:12.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.741+0000 7f26a50e9740 1 -- start start 2024-09-11T23:27:12.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.741+0000 7f26a50e9740 1 Processor -- start 2024-09-11T23:27:12.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.742+0000 7f26a50e9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557def4cf5d0 0x557def4d9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:12.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.742+0000 7f26a50e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4d0b40 0x557def4d6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:12.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.742+0000 7f26a50e9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557def4da370 0x557def4d75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:12.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.742+0000 7f26a50e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557def4dd010 con 0x557def4d0b40 2024-09-11T23:27:12.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.742+0000 7f26a50e9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557def3f0060 con 0x557def4cf5d0 2024-09-11T23:27:12.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.742+0000 7f26a50e9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557def4dd190 con 0x557def4da370 2024-09-11T23:27:12.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.742+0000 7f26a50e9740 10 client.0 _ll_register_callbacks cb 0x557def1438c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:12.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.742+0000 7f26a0aac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4d0b40 0x557def4d6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:12.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.742+0000 7f269bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557def4cf5d0 0x557def4d9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:12.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.742+0000 7f26a0aac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4d0b40 0x557def4d6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60078/0 (socket says 172.21.15.43:60078) 2024-09-11T23:27:12.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.742+0000 7f26a0aac640 1 -- 172.21.15.43:0/4040445890 learned_addr learned my addr 172.21.15.43:0/4040445890 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:12.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.742+0000 7f269bfff640 1 --2- 172.21.15.43:0/4040445890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557def4da370 0x557def4d75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:12.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.742+0000 7f269bfff640 1 -- 172.21.15.43:0/4040445890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557def4da370 msgr2=0x557def4d75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:12.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.742+0000 7f269bfff640 1 --2- 172.21.15.43:0/4040445890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557def4da370 0x557def4d75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.742+0000 7f269bfff640 1 -- 172.21.15.43:0/4040445890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4d0b40 msgr2=0x557def4d6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:12.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.742+0000 7f269bfff640 1 --2- 172.21.15.43:0/4040445890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4d0b40 0x557def4d6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.742+0000 7f269bfff640 1 -- 172.21.15.43:0/4040445890 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557def1a63c0 con 0x557def4cf5d0 2024-09-11T23:27:12.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.743+0000 7f269bfff640 1 --2- 172.21.15.43:0/4040445890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557def4cf5d0 0x557def4d9d40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2690000c00 tx=0x7f2690004840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72236e581fb6aac2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:12.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.743+0000 7f267bfff640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:27:12.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.743+0000 7f267bfff640 1 -- 172.21.15.43:0/4040445890 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2690053060 con 0x557def4cf5d0 2024-09-11T23:27:12.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.743+0000 7f26a50e9740 1 -- 172.21.15.43:0/4040445890 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2690005500 con 0x557def4cf5d0 2024-09-11T23:27:12.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.743+0000 7f267bfff640 1 -- 172.21.15.43:0/4040445890 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f2690004ff0 con 0x557def4cf5d0 2024-09-11T23:27:12.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.743+0000 7f267bfff640 1 -- 172.21.15.43:0/4040445890 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2690044070 con 0x557def4cf5d0 2024-09-11T23:27:12.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.743+0000 7f26a50e9740 10 client.25754 fetch_fsmap learned FSMap version 30 2024-09-11T23:27:12.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.743+0000 7f26a50e9740 1 -- 172.21.15.43:0/4040445890 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557def4e8a80 con 0x557def4cf5d0 2024-09-11T23:27:12.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.743+0000 7f267bfff640 1 -- 172.21.15.43:0/4040445890 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2690005500 con 0x557def4cf5d0 2024-09-11T23:27:12.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.743+0000 7f267bfff640 1 -- 172.21.15.43:0/4040445890 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f2690057c60 con 0x557def4cf5d0 2024-09-11T23:27:12.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.744+0000 7f26a50e9740 10 client.25754 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:27:12.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.744+0000 7f26a50e9740 10 client.25754 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T23:27:12.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.744+0000 7f26a50e9740 20 client.25754 populate_metadata read hostname 'smithi043' 2024-09-11T23:27:12.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.744+0000 7f26a50e9740 1 --2- 172.21.15.43:0/4040445890 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x557def4e9dd0 0x557def4ec1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:12.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.744+0000 7f26a50e9740 4 client.25754 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:27:12.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.744+0000 7f26a50e9740 1 -- 172.21.15.43:0/4040445890 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557def442860 con 0x557def4e9dd0 2024-09-11T23:27:12.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.744+0000 7f26a0aac640 1 --2- 172.21.15.43:0/4040445890 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x557def4e9dd0 0x557def4ec1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:12.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.744+0000 7f267bfff640 10 client.25754 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:27:12.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.744+0000 7f26a0aac640 1 --2- 172.21.15.43:0/4040445890 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x557def4e9dd0 0x557def4ec1c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71ec415c53bef757 server_cookie=cbcd9cbf20a86dc0 in_seq=0 out_seq=0 2024-09-11T23:27:12.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.745+0000 7f267bfff640 1 -- 172.21.15.43:0/4040445890 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557def442860 con 0x557def4e9dd0 2024-09-11T23:27:12.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.745+0000 7f267bfff640 10 client.25754 handle_command_reply: tid=0 2024-09-11T23:27:12.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.745+0000 7f26a50e9740 1 client.25754 shutdown 2024-09-11T23:27:12.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.745+0000 7f26a50e9740 1 -- 172.21.15.43:0/4040445890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557def4cf5d0 msgr2=0x557def4d9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:12.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.745+0000 7f26a50e9740 1 --2- 172.21.15.43:0/4040445890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557def4cf5d0 0x557def4d9d40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2690000c00 tx=0x7f2690004840 comp rx=0 tx=0).stop 2024-09-11T23:27:12.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.745+0000 7f26a50e9740 1 -- 172.21.15.43:0/4040445890 shutdown_connections 2024-09-11T23:27:12.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.745+0000 7f26a50e9740 1 --2- 172.21.15.43:0/4040445890 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x557def4e9dd0 0x557def4ec1c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.745+0000 7f26a50e9740 1 --2- 172.21.15.43:0/4040445890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557def4da370 0x557def4d75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.745+0000 7f26a50e9740 1 --2- 172.21.15.43:0/4040445890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4d0b40 0x557def4d6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.745+0000 7f26a50e9740 1 --2- 172.21.15.43:0/4040445890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557def4cf5d0 0x557def4d9d40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.745+0000 7f267bfff640 0 client.25754 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:27:12.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.745+0000 7f26a50e9740 1 -- 172.21.15.43:0/4040445890 >> 172.21.15.43:0/4040445890 conn(0x557def498310 msgr2=0x557def4d1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:12.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.745+0000 7f26a50e9740 1 -- 172.21.15.43:0/4040445890 shutdown_connections 2024-09-11T23:27:12.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.745+0000 7f26a50e9740 1 -- 172.21.15.43:0/4040445890 wait complete. 2024-09-11T23:27:12.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.746+0000 7f26a50e9740 20 client.25754 trim_cache size 0 max 16384 2024-09-11T23:27:12.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.758+0000 7f26a50e9740 1 Processor -- start 2024-09-11T23:27:12.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.758+0000 7f26a50e9740 1 -- start start 2024-09-11T23:27:12.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.759+0000 7f26a50e9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557def4cf5d0 0x557def4d68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:12.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.759+0000 7f26a50e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4d0b40 0x557def4d5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:12.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.759+0000 7f26a50e9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557def4da370 0x557def4ef1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:12.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.759+0000 7f26a50e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f269c1b6560 con 0x557def4d0b40 2024-09-11T23:27:12.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.759+0000 7f26a50e9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f269c1b3e30 con 0x557def4da370 2024-09-11T23:27:12.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.759+0000 7f26a50e9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f269c1b1c80 con 0x557def4cf5d0 2024-09-11T23:27:12.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.759+0000 7f26a0aac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4d0b40 0x557def4d5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:12.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.759+0000 7f26a0aac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4d0b40 0x557def4d5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60086/0 (socket says 172.21.15.43:60086) 2024-09-11T23:27:12.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.759+0000 7f26a0aac640 1 -- 172.21.15.43:0/1864474844 learned_addr learned my addr 172.21.15.43:0/1864474844 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:12.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.759+0000 7f269bfff640 1 --2- 172.21.15.43:0/1864474844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557def4cf5d0 0x557def4d68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:12.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.759+0000 7f269bfff640 1 --2- 172.21.15.43:0/1864474844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557def4da370 0x557def4ef1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:12.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.759+0000 7f26a0aac640 1 -- 172.21.15.43:0/1864474844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557def4cf5d0 msgr2=0x557def4d68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:12.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.759+0000 7f26a0aac640 1 --2- 172.21.15.43:0/1864474844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557def4cf5d0 0x557def4d68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.759+0000 7f26a0aac640 1 -- 172.21.15.43:0/1864474844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557def4da370 msgr2=0x557def4ef1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:12.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.759+0000 7f26a0aac640 1 --2- 172.21.15.43:0/1864474844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557def4da370 0x557def4ef1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.759+0000 7f26a0aac640 1 -- 172.21.15.43:0/1864474844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557def4d2b30 con 0x557def4d0b40 2024-09-11T23:27:12.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.759+0000 7f26a0aac640 1 --2- 172.21.15.43:0/1864474844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4d0b40 0x557def4d5fd0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x557def4dd2b0 tx=0x7f268c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c806c8b9cab04fc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:12.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.759+0000 7f26827fc640 1 -- 172.21.15.43:0/1864474844 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c004220 con 0x557def4d0b40 2024-09-11T23:27:12.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.760+0000 7f26827fc640 1 -- 172.21.15.43:0/1864474844 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f268c011e30 con 0x557def4d0b40 2024-09-11T23:27:12.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.760+0000 7f26827fc640 1 -- 172.21.15.43:0/1864474844 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c0047a0 con 0x557def4d0b40 2024-09-11T23:27:12.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.760+0000 7f26a50e9740 1 -- 172.21.15.43:0/1864474844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4d0b40 msgr2=0x557def4d5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:12.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.760+0000 7f26a50e9740 1 --2- 172.21.15.43:0/1864474844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4d0b40 0x557def4d5fd0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x557def4dd2b0 tx=0x7f268c00b390 comp rx=0 tx=0).stop 2024-09-11T23:27:12.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.760+0000 7f26a50e9740 1 -- 172.21.15.43:0/1864474844 shutdown_connections 2024-09-11T23:27:12.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.760+0000 7f26a50e9740 1 --2- 172.21.15.43:0/1864474844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557def4da370 0x557def4ef1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.760+0000 7f26a50e9740 1 --2- 172.21.15.43:0/1864474844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4d0b40 0x557def4d5fd0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.760+0000 7f26a50e9740 1 --2- 172.21.15.43:0/1864474844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557def4cf5d0 0x557def4d68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.760+0000 7f26a50e9740 1 -- 172.21.15.43:0/1864474844 >> 172.21.15.43:0/1864474844 conn(0x557def498310 msgr2=0x557def4d58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:12.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.760+0000 7f26a50e9740 1 -- 172.21.15.43:0/1864474844 shutdown_connections 2024-09-11T23:27:12.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.760+0000 7f26a50e9740 1 -- 172.21.15.43:0/1864474844 wait complete. 2024-09-11T23:27:12.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.761+0000 7f26a50e9740 1 -- start start 2024-09-11T23:27:12.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.761+0000 7f26a50e9740 1 Processor -- start 2024-09-11T23:27:12.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.761+0000 7f26a50e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4cf5d0 0x557def4ef6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:12.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.761+0000 7f26a50e9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557def4d0b40 0x557def4eca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:12.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.761+0000 7f26a50e9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557def4da370 0x557def4ecf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:12.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.761+0000 7f26a50e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f269c1b22f0 con 0x557def4cf5d0 2024-09-11T23:27:12.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.761+0000 7f26a50e9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f269c1b3760 con 0x557def4d0b40 2024-09-11T23:27:12.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.761+0000 7f26a50e9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f269c1b29c0 con 0x557def4da370 2024-09-11T23:27:12.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.761+0000 7f26a50e9740 10 client.0 _ll_register_callbacks cb 0x7f268c00bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:12.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.761+0000 7f269bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4cf5d0 0x557def4ef6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:12.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.761+0000 7f269bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4cf5d0 0x557def4ef6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60096/0 (socket says 172.21.15.43:60096) 2024-09-11T23:27:12.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.761+0000 7f269bfff640 1 -- 172.21.15.43:0/1436883656 learned_addr learned my addr 172.21.15.43:0/1436883656 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:12.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.761+0000 7f26a0aac640 1 --2- 172.21.15.43:0/1436883656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557def4d0b40 0x557def4eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:12.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.762+0000 7f269bfff640 1 --2- 172.21.15.43:0/1436883656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557def4da370 0x557def4ecf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:12.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.762+0000 7f269bfff640 1 -- 172.21.15.43:0/1436883656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557def4da370 msgr2=0x557def4ecf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:12.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.762+0000 7f269bfff640 1 --2- 172.21.15.43:0/1436883656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557def4da370 0x557def4ecf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.762+0000 7f269bfff640 1 -- 172.21.15.43:0/1436883656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557def4d0b40 msgr2=0x557def4eca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:12.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.762+0000 7f269bfff640 1 --2- 172.21.15.43:0/1436883656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557def4d0b40 0x557def4eca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.762+0000 7f269bfff640 1 -- 172.21.15.43:0/1436883656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557def4ed520 con 0x557def4cf5d0 2024-09-11T23:27:12.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.762+0000 7f269bfff640 1 --2- 172.21.15.43:0/1436883656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4cf5d0 0x557def4ef6d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2690048400 tx=0x7f2690004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bfe71150b77c0b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:12.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.762+0000 7f267bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:27:12.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.762+0000 7f267bfff640 1 -- 172.21.15.43:0/1436883656 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2690053040 con 0x557def4cf5d0 2024-09-11T23:27:12.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.762+0000 7f267bfff640 1 -- 172.21.15.43:0/1436883656 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f2690031b80 con 0x557def4cf5d0 2024-09-11T23:27:12.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.762+0000 7f26a50e9740 1 -- 172.21.15.43:0/1436883656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f268c011e30 con 0x557def4cf5d0 2024-09-11T23:27:12.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.762+0000 7f267bfff640 1 -- 172.21.15.43:0/1436883656 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2690035040 con 0x557def4cf5d0 2024-09-11T23:27:12.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.762+0000 7f26a50e9740 10 client.16230 fetch_fsmap learned FSMap version 30 2024-09-11T23:27:12.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.762+0000 7f26a50e9740 1 -- 172.21.15.43:0/1436883656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557def5544a0 con 0x557def4cf5d0 2024-09-11T23:27:12.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.763+0000 7f267bfff640 1 -- 172.21.15.43:0/1436883656 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f268c011e30 con 0x557def4cf5d0 2024-09-11T23:27:12.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.763+0000 7f267bfff640 1 -- 172.21.15.43:0/1436883656 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f2690005420 con 0x557def4cf5d0 2024-09-11T23:27:12.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.763+0000 7f26a50e9740 10 client.16230 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:27:12.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.763+0000 7f26a50e9740 10 client.16230 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T23:27:12.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.763+0000 7f26a50e9740 20 client.16230 populate_metadata read hostname 'smithi043' 2024-09-11T23:27:12.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.763+0000 7f26a50e9740 1 --2- 172.21.15.43:0/1436883656 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x557def554aa0 0x557def556e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:12.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.763+0000 7f26a50e9740 4 client.16230 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:27:12.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.763+0000 7f26a50e9740 1 -- 172.21.15.43:0/1436883656 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557def43cd70 con 0x557def554aa0 2024-09-11T23:27:12.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.763+0000 7f26a0aac640 1 --2- 172.21.15.43:0/1436883656 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x557def554aa0 0x557def556e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:12.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.764+0000 7f26a0aac640 1 --2- 172.21.15.43:0/1436883656 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x557def554aa0 0x557def556e90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82823fef6060197f server_cookie=50b83ae19b63c314 in_seq=0 out_seq=0 2024-09-11T23:27:12.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.764+0000 7f267bfff640 10 client.16230 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:27:12.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.764+0000 7f267bfff640 1 -- 172.21.15.43:0/1436883656 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557def43cd70 con 0x557def554aa0 2024-09-11T23:27:12.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.764+0000 7f267bfff640 10 client.16230 handle_command_reply: tid=0 2024-09-11T23:27:12.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.764+0000 7f26a50e9740 1 client.16230 shutdown 2024-09-11T23:27:12.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.764+0000 7f26a50e9740 1 -- 172.21.15.43:0/1436883656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4cf5d0 msgr2=0x557def4ef6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:12.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.764+0000 7f26a50e9740 1 --2- 172.21.15.43:0/1436883656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4cf5d0 0x557def4ef6d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2690048400 tx=0x7f2690004840 comp rx=0 tx=0).stop 2024-09-11T23:27:12.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.764+0000 7f26a50e9740 1 -- 172.21.15.43:0/1436883656 shutdown_connections 2024-09-11T23:27:12.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.764+0000 7f26a50e9740 1 --2- 172.21.15.43:0/1436883656 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x557def554aa0 0x557def556e90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.764+0000 7f26a50e9740 1 --2- 172.21.15.43:0/1436883656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557def4da370 0x557def4ecf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.764+0000 7f26a50e9740 1 --2- 172.21.15.43:0/1436883656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557def4d0b40 0x557def4eca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.764+0000 7f26a50e9740 1 --2- 172.21.15.43:0/1436883656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557def4cf5d0 0x557def4ef6d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.764+0000 7f26a50e9740 1 -- 172.21.15.43:0/1436883656 >> 172.21.15.43:0/1436883656 conn(0x557def498310 msgr2=0x557def4ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:12.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.764+0000 7f267bfff640 0 client.16230 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:27:12.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.764+0000 7f26a50e9740 1 -- 172.21.15.43:0/1436883656 shutdown_connections 2024-09-11T23:27:12.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.764+0000 7f26a50e9740 1 -- 172.21.15.43:0/1436883656 wait complete. 2024-09-11T23:27:12.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.764+0000 7f26a50e9740 20 client.16230 trim_cache size 0 max 16384 2024-09-11T23:27:12.770 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:27:12.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.765+0000 7f26a2d10640 1 -- 172.21.15.43:0/3808423157 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f26840a2cf0 msgr2=0x7f26840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:12.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.765+0000 7f26a2d10640 1 --2- 172.21.15.43:0/3808423157 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f26840a2cf0 0x7f26840a51b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f269409bbf0 tx=0x7f26940186b0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.765+0000 7f26a2d10640 1 -- 172.21.15.43:0/3808423157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269c105f40 msgr2=0x7f269c19e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:12.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.765+0000 7f26a2d10640 1 --2- 172.21.15.43:0/3808423157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269c105f40 0x7f269c19e830 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f269400cce0 tx=0x7f269400a620 comp rx=0 tx=0).stop 2024-09-11T23:27:12.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.765+0000 7f26a2d10640 1 -- 172.21.15.43:0/3808423157 shutdown_connections 2024-09-11T23:27:12.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.765+0000 7f26a2d10640 1 --2- 172.21.15.43:0/3808423157 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f26840a2cf0 0x7f26840a51b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.765+0000 7f26a2d10640 1 --2- 172.21.15.43:0/3808423157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f269c105f40 0x7f269c19e830 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.765+0000 7f26a2d10640 1 --2- 172.21.15.43:0/3808423157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f269c101450 0x7f269c1a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.765+0000 7f26a2d10640 1 --2- 172.21.15.43:0/3808423157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f269c100b30 0x7f269c1a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:12.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.765+0000 7f26a2d10640 1 -- 172.21.15.43:0/3808423157 >> 172.21.15.43:0/3808423157 conn(0x7f269c0fc7a0 msgr2=0x7f269c10f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:12.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.765+0000 7f26a2d10640 1 -- 172.21.15.43:0/3808423157 shutdown_connections 2024-09-11T23:27:12.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:12.765+0000 7f26a2d10640 1 -- 172.21.15.43:0/3808423157 wait complete. 2024-09-11T23:27:13.612 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:27:13.884 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:27:14.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:13 smithi043 ceph-mon[30950]: pgmap v165: 97 pgs: 97 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 24 MiB/s wr, 4.36k op/s 2024-09-11T23:27:14.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:13 smithi090 ceph-mon[35130]: pgmap v165: 97 pgs: 97 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 24 MiB/s wr, 4.36k op/s 2024-09-11T23:27:14.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:13 smithi097 ceph-mon[35562]: pgmap v165: 97 pgs: 97 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 24 MiB/s wr, 4.36k op/s 2024-09-11T23:27:14.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.434+0000 7f3c1e749640 1 -- 172.21.15.43:0/474548244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c1810ef30 msgr2=0x7f3c1810d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:14.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.434+0000 7f3c1e749640 1 --2- 172.21.15.43:0/474548244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c1810ef30 0x7f3c1810d2a0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009930 tx=0x7f3c0c02a440 comp rx=0 tx=0).stop 2024-09-11T23:27:14.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.434+0000 7f3c1e749640 1 -- 172.21.15.43:0/474548244 shutdown_connections 2024-09-11T23:27:14.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.434+0000 7f3c1e749640 1 --2- 172.21.15.43:0/474548244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c1810d9b0 0x7f3c18115510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.434+0000 7f3c1e749640 1 --2- 172.21.15.43:0/474548244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c1810ef30 0x7f3c1810d2a0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.434+0000 7f3c1e749640 1 --2- 172.21.15.43:0/474548244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c1810e580 0x7f3c1810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.434+0000 7f3c1e749640 1 -- 172.21.15.43:0/474548244 >> 172.21.15.43:0/474548244 conn(0x7f3c1806e140 msgr2=0x7f3c1806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:14.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.434+0000 7f3c1e749640 1 -- 172.21.15.43:0/474548244 shutdown_connections 2024-09-11T23:27:14.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.434+0000 7f3c1e749640 1 -- 172.21.15.43:0/474548244 wait complete. 2024-09-11T23:27:14.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.435+0000 7f3c1e749640 1 Processor -- start 2024-09-11T23:27:14.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.435+0000 7f3c1e749640 1 -- start start 2024-09-11T23:27:14.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.436+0000 7f3c1e749640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c1810d9b0 0x7f3c181135d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:14.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.436+0000 7f3c1e749640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c1810e580 0x7f3c18113b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:14.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.436+0000 7f3c1e749640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c18115260 0x7f3c18114050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:14.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.436+0000 7f3c1e749640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3c18118640 con 0x7f3c18115260 2024-09-11T23:27:14.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.436+0000 7f3c1e749640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181184c0 con 0x7f3c1810e580 2024-09-11T23:27:14.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.436+0000 7f3c1e749640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181187c0 con 0x7f3c1810d9b0 2024-09-11T23:27:14.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.436+0000 7f3c1cce6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c18115260 0x7f3c18114050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:14.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.436+0000 7f3c1cce6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c18115260 0x7f3c18114050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60112/0 (socket says 172.21.15.43:60112) 2024-09-11T23:27:14.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.436+0000 7f3c1cce6640 1 -- 172.21.15.43:0/485831099 learned_addr learned my addr 172.21.15.43:0/485831099 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:14.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.441+0000 7f3c17fff640 1 --2- 172.21.15.43:0/485831099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c1810d9b0 0x7f3c181135d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:14.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.441+0000 7f3c177fe640 1 --2- 172.21.15.43:0/485831099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c1810e580 0x7f3c18113b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:14.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.441+0000 7f3c1cce6640 1 -- 172.21.15.43:0/485831099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c1810d9b0 msgr2=0x7f3c181135d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.441+0000 7f3c1cce6640 1 --2- 172.21.15.43:0/485831099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c1810d9b0 0x7f3c181135d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.442+0000 7f3c1cce6640 1 -- 172.21.15.43:0/485831099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c1810e580 msgr2=0x7f3c18113b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.442+0000 7f3c1cce6640 1 --2- 172.21.15.43:0/485831099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c1810e580 0x7f3c18113b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.442+0000 7f3c1cce6640 1 -- 172.21.15.43:0/485831099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c181a5220 con 0x7f3c18115260 2024-09-11T23:27:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.444+0000 7f3c1cce6640 1 --2- 172.21.15.43:0/485831099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c18115260 0x7f3c18114050 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000a660 tx=0x7f3c1000ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.447+0000 7f3c157fa640 1 -- 172.21.15.43:0/485831099 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c10018020 con 0x7f3c18115260 2024-09-11T23:27:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.447+0000 7f3c157fa640 1 -- 172.21.15.43:0/485831099 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c10004d20 con 0x7f3c18115260 2024-09-11T23:27:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.447+0000 7f3c157fa640 1 -- 172.21.15.43:0/485831099 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c10005580 con 0x7f3c18115260 2024-09-11T23:27:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.448+0000 7f3c1e749640 1 -- 172.21.15.43:0/485831099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c181a5510 con 0x7f3c18115260 2024-09-11T23:27:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.448+0000 7f3c1e749640 1 -- 172.21.15.43:0/485831099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c181a5a20 con 0x7f3c18115260 2024-09-11T23:27:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.452+0000 7f3c20b22740 1 Processor -- start 2024-09-11T23:27:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.456+0000 7f3c157fa640 1 -- 172.21.15.43:0/485831099 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3c10005720 con 0x7f3c18115260 2024-09-11T23:27:14.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.458+0000 7f3c20b22740 1 -- start start 2024-09-11T23:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.459+0000 7f3c20b22740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5573b83f2b40 0x5573b83f1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.459+0000 7f3c20b22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83f15d0 0x5573b83f1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.461+0000 7f3c20b22740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5573b83fc370 0x5573b83fe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.461+0000 7f3c20b22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5573b83572f0 con 0x5573b83f15d0 2024-09-11T23:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.461+0000 7f3c20b22740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5573b82fecf0 con 0x5573b83fc370 2024-09-11T23:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.461+0000 7f3c20b22740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5573b83532b0 con 0x5573b83f2b40 2024-09-11T23:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.461+0000 7f3c17fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83f15d0 0x5573b83f1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.461+0000 7f3c17fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83f15d0 0x5573b83f1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60126/0 (socket says 172.21.15.43:60126) 2024-09-11T23:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.461+0000 7f3c17fff640 1 -- 172.21.15.43:0/289020541 learned_addr learned my addr 172.21.15.43:0/289020541 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.461+0000 7f3c177fe640 1 --2- 172.21.15.43:0/289020541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5573b83f2b40 0x5573b83f1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.463+0000 7f3c157fa640 1 --2- 172.21.15.43:0/485831099 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f3c000a2f00 0x7f3c000a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.464+0000 7f3c157fa640 1 -- 172.21.15.43:0/485831099 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f3c100ceae0 con 0x7f3c18115260 2024-09-11T23:27:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.465+0000 7f3c177fe640 1 --2- 172.21.15.43:0/289020541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5573b83fc370 0x5573b83fe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.466+0000 7f3c17fff640 1 -- 172.21.15.43:0/289020541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5573b83f2b40 msgr2=0x5573b83f1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.466+0000 7f3c17fff640 1 --2- 172.21.15.43:0/289020541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5573b83f2b40 0x5573b83f1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.466+0000 7f3c17fff640 1 -- 172.21.15.43:0/289020541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5573b83fc370 msgr2=0x5573b83fe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.466+0000 7f3c17fff640 1 --2- 172.21.15.43:0/289020541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5573b83fc370 0x5573b83fe760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.466+0000 7f3c17fff640 1 -- 172.21.15.43:0/289020541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573b82e33a0 con 0x5573b83f15d0 2024-09-11T23:27:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.466+0000 7f3c1cce6640 1 --2- 172.21.15.43:0/485831099 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f3c000a2f00 0x7f3c000a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.467+0000 7f3c17fff640 1 --2- 172.21.15.43:0/289020541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83f15d0 0x5573b83f1a10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009a40 tx=0x7f3c08009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2a08bffca5918 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.467+0000 7f3c1cce6640 1 --2- 172.21.15.43:0/485831099 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f3c000a2f00 0x7f3c000a53c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3c100a36e0 tx=0x7f3c100093e0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.468+0000 7f3bfe7fc640 1 -- 172.21.15.43:0/289020541 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08017070 con 0x5573b83f15d0 2024-09-11T23:27:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.468+0000 7f3bfe7fc640 1 -- 172.21.15.43:0/289020541 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c08013470 con 0x5573b83f15d0 2024-09-11T23:27:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.469+0000 7f3c20b22740 1 -- 172.21.15.43:0/289020541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83f15d0 msgr2=0x5573b83f1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.469+0000 7f3c20b22740 1 --2- 172.21.15.43:0/289020541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83f15d0 0x5573b83f1a10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009a40 tx=0x7f3c08009f00 comp rx=0 tx=0).stop 2024-09-11T23:27:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.470+0000 7f3c20b22740 1 -- 172.21.15.43:0/289020541 shutdown_connections 2024-09-11T23:27:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.470+0000 7f3c20b22740 1 --2- 172.21.15.43:0/289020541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5573b83fc370 0x5573b83fe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.470+0000 7f3c20b22740 1 --2- 172.21.15.43:0/289020541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83f15d0 0x5573b83f1a10 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.470+0000 7f3c20b22740 1 --2- 172.21.15.43:0/289020541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5573b83f2b40 0x5573b83f1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.470+0000 7f3c20b22740 1 -- 172.21.15.43:0/289020541 >> 172.21.15.43:0/289020541 conn(0x5573b83ba310 msgr2=0x5573b83edc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.470+0000 7f3c20b22740 1 -- 172.21.15.43:0/289020541 shutdown_connections 2024-09-11T23:27:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.470+0000 7f3c20b22740 1 -- 172.21.15.43:0/289020541 wait complete. 2024-09-11T23:27:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.470+0000 7f3c20b22740 1 -- start start 2024-09-11T23:27:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.471+0000 7f3c20b22740 1 Processor -- start 2024-09-11T23:27:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.471+0000 7f3c20b22740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5573b83f15d0 0x5573b83fbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.471+0000 7f3c20b22740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5573b83f2b40 0x5573b83f8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.471+0000 7f3c20b22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83fc370 0x5573b83f93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:14.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.471+0000 7f3c20b22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5573b83ff110 con 0x5573b83fc370 2024-09-11T23:27:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.471+0000 7f3c20b22740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5573b8312060 con 0x5573b83f2b40 2024-09-11T23:27:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.471+0000 7f3c20b22740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5573b83ff290 con 0x5573b83f15d0 2024-09-11T23:27:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.471+0000 7f3c20b22740 10 client.0 _ll_register_callbacks cb 0x5573b80658c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.472+0000 7f3c177fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5573b83f15d0 0x5573b83fbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.472+0000 7f3c177fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83fc370 0x5573b83f93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.472+0000 7f3c177fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83fc370 0x5573b83f93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60136/0 (socket says 172.21.15.43:60136) 2024-09-11T23:27:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.472+0000 7f3c177fe640 1 -- 172.21.15.43:0/2583079055 learned_addr learned my addr 172.21.15.43:0/2583079055 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.473+0000 7f3c17fff640 1 --2- 172.21.15.43:0/2583079055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5573b83f2b40 0x5573b83f8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.473+0000 7f3c17fff640 1 -- 172.21.15.43:0/2583079055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5573b83f15d0 msgr2=0x5573b83fbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.473+0000 7f3c17fff640 1 --2- 172.21.15.43:0/2583079055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5573b83f15d0 0x5573b83fbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.473+0000 7f3c17fff640 1 -- 172.21.15.43:0/2583079055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83fc370 msgr2=0x5573b83f93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.473+0000 7f3c17fff640 1 --2- 172.21.15.43:0/2583079055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83fc370 0x5573b83f93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.473+0000 7f3c17fff640 1 -- 172.21.15.43:0/2583079055 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573b80c83c0 con 0x5573b83f2b40 2024-09-11T23:27:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.474+0000 7f3c17fff640 1 --2- 172.21.15.43:0/2583079055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5573b83f2b40 0x5573b83f8e70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3c08013e40 tx=0x7f3c080092b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbe7b884246a95dc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.474+0000 7f3bf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:27:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.474+0000 7f3bf77fe640 1 -- 172.21.15.43:0/2583079055 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08017050 con 0x5573b83f2b40 2024-09-11T23:27:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.474+0000 7f3bf77fe640 1 -- 172.21.15.43:0/2583079055 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f3c08013a30 con 0x5573b83f2b40 2024-09-11T23:27:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.476+0000 7f3c20b22740 1 -- 172.21.15.43:0/2583079055 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c08013470 con 0x5573b83f2b40 2024-09-11T23:27:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.477+0000 7f3bf77fe640 1 -- 172.21.15.43:0/2583079055 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3c08012810 con 0x5573b83f2b40 2024-09-11T23:27:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.477+0000 7f3c20b22740 10 client.25772 fetch_fsmap learned FSMap version 30 2024-09-11T23:27:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.477+0000 7f3c20b22740 1 -- 172.21.15.43:0/2583079055 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5573b840aad0 con 0x5573b83f2b40 2024-09-11T23:27:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.486+0000 7f3bf77fe640 1 -- 172.21.15.43:0/2583079055 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c08013470 con 0x5573b83f2b40 2024-09-11T23:27:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.486+0000 7f3bf77fe640 1 -- 172.21.15.43:0/2583079055 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f3c0801caa0 con 0x5573b83f2b40 2024-09-11T23:27:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.487+0000 7f3c20b22740 10 client.25772 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:27:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.487+0000 7f3c20b22740 10 client.25772 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T23:27:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.487+0000 7f3c20b22740 20 client.25772 populate_metadata read hostname 'smithi043' 2024-09-11T23:27:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.487+0000 7f3c20b22740 1 --2- 172.21.15.43:0/2583079055 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5573b840bf30 0x5573b840e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:14.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.487+0000 7f3c20b22740 4 client.25772 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:27:14.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.487+0000 7f3c20b22740 1 -- 172.21.15.43:0/2583079055 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573b8364860 con 0x5573b840bf30 2024-09-11T23:27:14.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.490+0000 7f3c177fe640 1 --2- 172.21.15.43:0/2583079055 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5573b840bf30 0x5573b840e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:14.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.491+0000 7f3c177fe640 1 --2- 172.21.15.43:0/2583079055 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5573b840bf30 0x5573b840e320 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f42e632f522dc6d6 server_cookie=f36eebd820d533 in_seq=0 out_seq=0 2024-09-11T23:27:14.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.491+0000 7f3bf77fe640 10 client.25772 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:27:14.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.492+0000 7f3bf77fe640 1 -- 172.21.15.43:0/2583079055 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5573b8364860 con 0x5573b840bf30 2024-09-11T23:27:14.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.492+0000 7f3bf77fe640 10 client.25772 handle_command_reply: tid=0 2024-09-11T23:27:14.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.492+0000 7f3c20b22740 1 client.25772 shutdown 2024-09-11T23:27:14.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.492+0000 7f3c20b22740 1 -- 172.21.15.43:0/2583079055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5573b83f2b40 msgr2=0x5573b83f8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:14.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.492+0000 7f3c20b22740 1 --2- 172.21.15.43:0/2583079055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5573b83f2b40 0x5573b83f8e70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3c08013e40 tx=0x7f3c080092b0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.492+0000 7f3c20b22740 1 -- 172.21.15.43:0/2583079055 shutdown_connections 2024-09-11T23:27:14.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.492+0000 7f3c20b22740 1 --2- 172.21.15.43:0/2583079055 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5573b840bf30 0x5573b840e320 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.492+0000 7f3c20b22740 1 --2- 172.21.15.43:0/2583079055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83fc370 0x5573b83f93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.492+0000 7f3c20b22740 1 --2- 172.21.15.43:0/2583079055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5573b83f2b40 0x5573b83f8e70 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3c08013e40 tx=0x7f3c080092b0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.492+0000 7f3c20b22740 1 --2- 172.21.15.43:0/2583079055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5573b83f15d0 0x5573b83fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.492+0000 7f3c20b22740 1 -- 172.21.15.43:0/2583079055 >> 172.21.15.43:0/2583079055 conn(0x5573b83ba310 msgr2=0x5573b83f3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:14.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.492+0000 7f3bf77fe640 0 client.25772 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:27:14.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.495+0000 7f3c20b22740 1 -- 172.21.15.43:0/2583079055 shutdown_connections 2024-09-11T23:27:14.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.499+0000 7f3c20b22740 1 -- 172.21.15.43:0/2583079055 wait complete. 2024-09-11T23:27:14.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.500+0000 7f3c20b22740 20 client.25772 trim_cache size 0 max 16384 2024-09-11T23:27:14.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.529+0000 7f3c20b22740 1 Processor -- start 2024-09-11T23:27:14.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.530+0000 7f3c20b22740 1 -- start start 2024-09-11T23:27:14.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.530+0000 7f3c20b22740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5573b83ba310 0x5573b83f4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:14.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.530+0000 7f3c20b22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83f15d0 0x5573b83fd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:14.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.530+0000 7f3c177fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5573b83ba310 0x5573b83f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:14.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.530+0000 7f3c177fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5573b83ba310 0x5573b83f4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:57148/0 (socket says 172.21.15.43:57148) 2024-09-11T23:27:14.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.530+0000 7f3c17fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83f15d0 0x5573b83fd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:14.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.530+0000 7f3c17fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83f15d0 0x5573b83fd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60146/0 (socket says 172.21.15.43:60146) 2024-09-11T23:27:14.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.530+0000 7f3c20b22740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5573b83fc370 0x5573b83fdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:14.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.531+0000 7f3c20b22740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181c7fb0 con 0x5573b83f15d0 2024-09-11T23:27:14.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.531+0000 7f3c177fe640 1 -- 172.21.15.43:0/1114182809 learned_addr learned my addr 172.21.15.43:0/1114182809 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:14.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.531+0000 7f3c20b22740 1 -- 172.21.15.43:0/1114182809 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181c5880 con 0x5573b83ba310 2024-09-11T23:27:14.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.531+0000 7f3c20b22740 1 -- 172.21.15.43:0/1114182809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181ac510 con 0x5573b83fc370 2024-09-11T23:27:14.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.531+0000 7f3c17fff640 1 -- 172.21.15.43:0/1114182809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5573b83fc370 msgr2=0x5573b83fdca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:27:14.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.531+0000 7f3c17fff640 1 --2- 172.21.15.43:0/1114182809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5573b83fc370 0x5573b83fdca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.531+0000 7f3c17fff640 1 -- 172.21.15.43:0/1114182809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5573b83ba310 msgr2=0x5573b83f4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:14.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.531+0000 7f3c17fff640 1 --2- 172.21.15.43:0/1114182809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5573b83ba310 0x5573b83f4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.531+0000 7f3c17fff640 1 -- 172.21.15.43:0/1114182809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573b83ee960 con 0x5573b83f15d0 2024-09-11T23:27:14.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.536+0000 7f3c17fff640 1 --2- 172.21.15.43:0/1114182809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83f15d0 0x5573b83fd760 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x5573b83fa4b0 tx=0x7f3c0800ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b5dd3382c403ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:14.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.536+0000 7f3bfdffb640 1 -- 172.21.15.43:0/1114182809 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08034030 con 0x5573b83f15d0 2024-09-11T23:27:14.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.536+0000 7f3bfdffb640 1 -- 172.21.15.43:0/1114182809 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c08013470 con 0x5573b83f15d0 2024-09-11T23:27:14.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.536+0000 7f3bfdffb640 1 -- 172.21.15.43:0/1114182809 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08017040 con 0x5573b83f15d0 2024-09-11T23:27:14.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.538+0000 7f3c20b22740 1 -- 172.21.15.43:0/1114182809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83f15d0 msgr2=0x5573b83fd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:14.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.538+0000 7f3c20b22740 1 --2- 172.21.15.43:0/1114182809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83f15d0 0x5573b83fd760 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x5573b83fa4b0 tx=0x7f3c0800ff80 comp rx=0 tx=0).stop 2024-09-11T23:27:14.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.538+0000 7f3c20b22740 1 -- 172.21.15.43:0/1114182809 shutdown_connections 2024-09-11T23:27:14.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.538+0000 7f3c20b22740 1 --2- 172.21.15.43:0/1114182809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5573b83fc370 0x5573b83fdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.539+0000 7f3c20b22740 1 --2- 172.21.15.43:0/1114182809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83f15d0 0x5573b83fd760 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.539+0000 7f3c20b22740 1 --2- 172.21.15.43:0/1114182809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5573b83ba310 0x5573b83f4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.539+0000 7f3c20b22740 1 -- 172.21.15.43:0/1114182809 >> 172.21.15.43:0/1114182809 conn(0x5573b83f9510 msgr2=0x5573b83f7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:14.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.542+0000 7f3c20b22740 1 -- 172.21.15.43:0/1114182809 shutdown_connections 2024-09-11T23:27:14.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.542+0000 7f3c20b22740 1 -- 172.21.15.43:0/1114182809 wait complete. 2024-09-11T23:27:14.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.543+0000 7f3c20b22740 1 -- start start 2024-09-11T23:27:14.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.543+0000 7f3c20b22740 1 Processor -- start 2024-09-11T23:27:14.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.544+0000 7f3c20b22740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83ba310 0x5573b840cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:14.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.544+0000 7f3c20b22740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5573b83f15d0 0x5573b840d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:14.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.544+0000 7f3c177fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83ba310 0x5573b840cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:14.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.544+0000 7f3c177fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83ba310 0x5573b840cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60148/0 (socket says 172.21.15.43:60148) 2024-09-11T23:27:14.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.544+0000 7f3c20b22740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5573b83fc370 0x5573b840da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:14.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.544+0000 7f3c177fe640 1 -- 172.21.15.43:0/3437231319 learned_addr learned my addr 172.21.15.43:0/3437231319 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:14.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.544+0000 7f3c20b22740 1 -- 172.21.15.43:0/3437231319 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181acb80 con 0x5573b83ba310 2024-09-11T23:27:14.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.544+0000 7f3c20b22740 1 -- 172.21.15.43:0/3437231319 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181c51b0 con 0x5573b83fc370 2024-09-11T23:27:14.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.544+0000 7f3c20b22740 1 -- 172.21.15.43:0/3437231319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181ad250 con 0x5573b83f15d0 2024-09-11T23:27:14.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.544+0000 7f3c17fff640 1 --2- 172.21.15.43:0/3437231319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5573b83f15d0 0x5573b840d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:14.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.544+0000 7f3c20b22740 10 client.0 _ll_register_callbacks cb 0x7f3c0c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:14.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.546+0000 7f3c17fff640 1 -- 172.21.15.43:0/3437231319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5573b83fc370 msgr2=0x5573b840da60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:27:14.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.546+0000 7f3c17fff640 1 --2- 172.21.15.43:0/3437231319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5573b83fc370 0x5573b840da60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.546+0000 7f3c17fff640 1 -- 172.21.15.43:0/3437231319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83ba310 msgr2=0x5573b840cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:14.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.546+0000 7f3c17fff640 1 --2- 172.21.15.43:0/3437231319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83ba310 0x5573b840cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.546+0000 7f3c17fff640 1 -- 172.21.15.43:0/3437231319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573b83f4120 con 0x5573b83f15d0 2024-09-11T23:27:14.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.548+0000 7f3c17fff640 1 --2- 172.21.15.43:0/3437231319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5573b83f15d0 0x5573b840d500 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x5573b83f75a0 tx=0x7f3c08013dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd011348af31c3c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:14.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.548+0000 7f3bf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-11T23:27:14.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.549+0000 7f3bf77fe640 1 -- 172.21.15.43:0/3437231319 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0803d060 con 0x5573b83f15d0 2024-09-11T23:27:14.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.549+0000 7f3c20b22740 1 -- 172.21.15.43:0/3437231319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c08013470 con 0x5573b83f15d0 2024-09-11T23:27:14.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.549+0000 7f3bf77fe640 1 -- 172.21.15.43:0/3437231319 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f3c08005b70 con 0x5573b83f15d0 2024-09-11T23:27:14.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.549+0000 7f3bf77fe640 1 -- 172.21.15.43:0/3437231319 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3c08034070 con 0x5573b83f15d0 2024-09-11T23:27:14.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.549+0000 7f3c20b22740 10 client.25738 fetch_fsmap learned FSMap version 30 2024-09-11T23:27:14.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.549+0000 7f3c20b22740 1 -- 172.21.15.43:0/3437231319 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5573b8476c60 con 0x5573b83f15d0 2024-09-11T23:27:14.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.550+0000 7f3bf77fe640 1 -- 172.21.15.43:0/3437231319 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c08013470 con 0x5573b83f15d0 2024-09-11T23:27:14.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.550+0000 7f3bf77fe640 1 -- 172.21.15.43:0/3437231319 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f3c08006140 con 0x5573b83f15d0 2024-09-11T23:27:14.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.550+0000 7f3c20b22740 10 client.25738 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:27:14.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.551+0000 7f3c20b22740 10 client.25738 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T23:27:14.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.551+0000 7f3c20b22740 20 client.25738 populate_metadata read hostname 'smithi043' 2024-09-11T23:27:14.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.551+0000 7f3c20b22740 1 --2- 172.21.15.43:0/3437231319 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5573b8477270 0x5573b8479690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:14.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.551+0000 7f3c20b22740 4 client.25738 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:27:14.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.551+0000 7f3c20b22740 1 -- 172.21.15.43:0/3437231319 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5573b835ed70 con 0x5573b8477270 2024-09-11T23:27:14.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.554+0000 7f3c177fe640 1 --2- 172.21.15.43:0/3437231319 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5573b8477270 0x5573b8479690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:14.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.556+0000 7f3c177fe640 1 --2- 172.21.15.43:0/3437231319 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5573b8477270 0x5573b8479690 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=295de262fffcc759 server_cookie=284b868d4bafb863 in_seq=0 out_seq=0 2024-09-11T23:27:14.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.556+0000 7f3bf77fe640 10 client.25738 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:27:14.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.560+0000 7f3bf77fe640 1 -- 172.21.15.43:0/3437231319 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5573b835ed70 con 0x5573b8477270 2024-09-11T23:27:14.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.560+0000 7f3bf77fe640 10 client.25738 handle_command_reply: tid=0 2024-09-11T23:27:14.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.560+0000 7f3c20b22740 1 client.25738 shutdown 2024-09-11T23:27:14.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.563+0000 7f3c20b22740 1 -- 172.21.15.43:0/3437231319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5573b83f15d0 msgr2=0x5573b840d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:14.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.563+0000 7f3c20b22740 1 --2- 172.21.15.43:0/3437231319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5573b83f15d0 0x5573b840d500 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x5573b83f75a0 tx=0x7f3c08013dd0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.563+0000 7f3c20b22740 1 -- 172.21.15.43:0/3437231319 shutdown_connections 2024-09-11T23:27:14.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.563+0000 7f3c20b22740 1 --2- 172.21.15.43:0/3437231319 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5573b8477270 0x5573b8479690 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.563+0000 7f3c20b22740 1 --2- 172.21.15.43:0/3437231319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5573b83fc370 0x5573b840da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.563+0000 7f3c20b22740 1 --2- 172.21.15.43:0/3437231319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5573b83f15d0 0x5573b840d500 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.563+0000 7f3c20b22740 1 --2- 172.21.15.43:0/3437231319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5573b83ba310 0x5573b840cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.563+0000 7f3c20b22740 1 -- 172.21.15.43:0/3437231319 >> 172.21.15.43:0/3437231319 conn(0x5573b83f9510 msgr2=0x5573b83f3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:14.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.563+0000 7f3bf77fe640 0 client.25738 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:27:14.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.563+0000 7f3c20b22740 1 -- 172.21.15.43:0/3437231319 shutdown_connections 2024-09-11T23:27:14.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.563+0000 7f3c20b22740 1 -- 172.21.15.43:0/3437231319 wait complete. 2024-09-11T23:27:14.564 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:27:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.563+0000 7f3c20b22740 20 client.25738 trim_cache size 0 max 16384 2024-09-11T23:27:14.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.566+0000 7f3c1e749640 1 -- 172.21.15.43:0/485831099 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f3c000a2f00 msgr2=0x7f3c000a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:14.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.566+0000 7f3c1e749640 1 --2- 172.21.15.43:0/485831099 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f3c000a2f00 0x7f3c000a53c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3c100a36e0 tx=0x7f3c100093e0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.566+0000 7f3c1e749640 1 -- 172.21.15.43:0/485831099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c18115260 msgr2=0x7f3c18114050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:14.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.566+0000 7f3c1e749640 1 --2- 172.21.15.43:0/485831099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c18115260 0x7f3c18114050 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000a660 tx=0x7f3c1000ab20 comp rx=0 tx=0).stop 2024-09-11T23:27:14.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.568+0000 7f3c1e749640 1 -- 172.21.15.43:0/485831099 shutdown_connections 2024-09-11T23:27:14.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.568+0000 7f3c1e749640 1 --2- 172.21.15.43:0/485831099 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f3c000a2f00 0x7f3c000a53c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.568+0000 7f3c1e749640 1 --2- 172.21.15.43:0/485831099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c18115260 0x7f3c18114050 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.568+0000 7f3c1e749640 1 --2- 172.21.15.43:0/485831099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c1810e580 0x7f3c18113b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.568+0000 7f3c1e749640 1 --2- 172.21.15.43:0/485831099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c1810d9b0 0x7f3c181135d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:14.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.568+0000 7f3c1e749640 1 -- 172.21.15.43:0/485831099 >> 172.21.15.43:0/485831099 conn(0x7f3c1806e140 msgr2=0x7f3c18072e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:14.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.568+0000 7f3c1e749640 1 -- 172.21.15.43:0/485831099 shutdown_connections 2024-09-11T23:27:14.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:14.568+0000 7f3c1e749640 1 -- 172.21.15.43:0/485831099 wait complete. 2024-09-11T23:27:16.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:15 smithi090 ceph-mon[35130]: pgmap v166: 97 pgs: 97 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 31 MiB/s wr, 5.56k op/s 2024-09-11T23:27:16.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:15 smithi097 ceph-mon[35562]: pgmap v166: 97 pgs: 97 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 31 MiB/s wr, 5.56k op/s 2024-09-11T23:27:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:15 smithi043 ceph-mon[30950]: pgmap v166: 97 pgs: 97 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 31 MiB/s wr, 5.56k op/s 2024-09-11T23:27:18.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:17 smithi090 ceph-mon[35130]: pgmap v167: 97 pgs: 97 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 22 MiB/s wr, 3.86k op/s 2024-09-11T23:27:18.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:17 smithi097 ceph-mon[35562]: pgmap v167: 97 pgs: 97 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 22 MiB/s wr, 3.86k op/s 2024-09-11T23:27:18.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:17 smithi043 ceph-mon[30950]: pgmap v167: 97 pgs: 97 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 22 MiB/s wr, 3.86k op/s 2024-09-11T23:27:19.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:18 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:27:19.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:18 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:27:19.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:18 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:27:19.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:18 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:27:19.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:18 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:27:19.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:18 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:27:19.342 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:18 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:27:19.342 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:18 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:27:19.342 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:18 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:27:20.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:19 smithi090 ceph-mon[35130]: pgmap v168: 97 pgs: 97 active+clean; 19 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 32 MiB/s wr, 5.65k op/s 2024-09-11T23:27:20.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:19 smithi097 ceph-mon[35562]: pgmap v168: 97 pgs: 97 active+clean; 19 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 32 MiB/s wr, 5.65k op/s 2024-09-11T23:27:20.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:19 smithi043 ceph-mon[30950]: pgmap v168: 97 pgs: 97 active+clean; 19 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 32 MiB/s wr, 5.65k op/s 2024-09-11T23:27:22.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:21 smithi090 ceph-mon[35130]: pgmap v169: 97 pgs: 97 active+clean; 19 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 MiB/s wr, 5.47k op/s 2024-09-11T23:27:22.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:21 smithi097 ceph-mon[35562]: pgmap v169: 97 pgs: 97 active+clean; 19 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 MiB/s wr, 5.47k op/s 2024-09-11T23:27:22.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:21 smithi043 ceph-mon[30950]: pgmap v169: 97 pgs: 97 active+clean; 19 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 MiB/s wr, 5.47k op/s 2024-09-11T23:27:24.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:23 smithi090 ceph-mon[35130]: pgmap v170: 97 pgs: 97 active+clean; 19 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 MiB/s wr, 5.19k op/s 2024-09-11T23:27:24.271 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:23 smithi043 ceph-mon[30950]: pgmap v170: 97 pgs: 97 active+clean; 19 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 MiB/s wr, 5.19k op/s 2024-09-11T23:27:24.300 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:23 smithi097 ceph-mon[35562]: pgmap v170: 97 pgs: 97 active+clean; 19 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 MiB/s wr, 5.19k op/s 2024-09-11T23:27:24.308 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:27:24.672 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:27:25.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.160+0000 7f7e787c7640 1 -- 172.21.15.43:0/2448970567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e70106f10 msgr2=0x7f7e7010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:25.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.160+0000 7f7e787c7640 1 --2- 172.21.15.43:0/2448970567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e70106f10 0x7f7e7010ea90 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009ca0 tx=0x7f7e6802a8c0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.160+0000 7f7e787c7640 1 -- 172.21.15.43:0/2448970567 shutdown_connections 2024-09-11T23:27:25.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.160+0000 7f7e787c7640 1 --2- 172.21.15.43:0/2448970567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e70106f10 0x7f7e7010ea90 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.160+0000 7f7e787c7640 1 --2- 172.21.15.43:0/2448970567 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e70102790 0x7f7e701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.160+0000 7f7e787c7640 1 --2- 172.21.15.43:0/2448970567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e70101de0 0x7f7e701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.160+0000 7f7e787c7640 1 -- 172.21.15.43:0/2448970567 >> 172.21.15.43:0/2448970567 conn(0x7f7e700fd8c0 msgr2=0x7f7e700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:25.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.160+0000 7f7e787c7640 1 -- 172.21.15.43:0/2448970567 shutdown_connections 2024-09-11T23:27:25.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.160+0000 7f7e787c7640 1 -- 172.21.15.43:0/2448970567 wait complete. 2024-09-11T23:27:25.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.161+0000 7f7e787c7640 1 Processor -- start 2024-09-11T23:27:25.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.161+0000 7f7e787c7640 1 -- start start 2024-09-11T23:27:25.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.161+0000 7f7e787c7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e70101de0 0x7f7e701983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:25.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.161+0000 7f7e787c7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e70102790 0x7f7e70198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:25.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.161+0000 7f7e787c7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e70106f10 0x7f7e7019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:25.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.161+0000 7f7e787c7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7e70111f70 con 0x7f7e70102790 2024-09-11T23:27:25.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.161+0000 7f7e787c7640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7e70111df0 con 0x7f7e70101de0 2024-09-11T23:27:25.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.161+0000 7f7e787c7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7e701120f0 con 0x7f7e70106f10 2024-09-11T23:27:25.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.161+0000 7f7e76563640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e70101de0 0x7f7e701983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:25.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.161+0000 7f7e76563640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e70101de0 0x7f7e701983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:49404/0 (socket says 172.21.15.43:49404) 2024-09-11T23:27:25.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.161+0000 7f7e76563640 1 -- 172.21.15.43:0/2126241545 learned_addr learned my addr 172.21.15.43:0/2126241545 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:25.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.161+0000 7f7e75d62640 1 --2- 172.21.15.43:0/2126241545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e70102790 0x7f7e70198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:25.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.161+0000 7f7e76d64640 1 --2- 172.21.15.43:0/2126241545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e70106f10 0x7f7e7019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:25.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.162+0000 7f7e76563640 1 -- 172.21.15.43:0/2126241545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e70106f10 msgr2=0x7f7e7019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:25.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.162+0000 7f7e76563640 1 --2- 172.21.15.43:0/2126241545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e70106f10 0x7f7e7019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.162+0000 7f7e76563640 1 -- 172.21.15.43:0/2126241545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e70102790 msgr2=0x7f7e70198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:25.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.162+0000 7f7e76563640 1 --2- 172.21.15.43:0/2126241545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e70102790 0x7f7e70198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.162+0000 7f7e76563640 1 -- 172.21.15.43:0/2126241545 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e7019d380 con 0x7f7e70101de0 2024-09-11T23:27:25.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.162+0000 7f7e75d62640 1 --2- 172.21.15.43:0/2126241545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e70102790 0x7f7e70198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:27:25.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.162+0000 7f7e76563640 1 --2- 172.21.15.43:0/2126241545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e70101de0 0x7f7e701983d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00e9f0 tx=0x7f7e6c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:25.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.162+0000 7f7e677fe640 1 -- 172.21.15.43:0/2126241545 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c00cd80 con 0x7f7e70101de0 2024-09-11T23:27:25.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.162+0000 7f7e677fe640 1 -- 172.21.15.43:0/2126241545 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e6c005500 con 0x7f7e70101de0 2024-09-11T23:27:25.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.162+0000 7f7e787c7640 1 -- 172.21.15.43:0/2126241545 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e7019d670 con 0x7f7e70101de0 2024-09-11T23:27:25.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.162+0000 7f7e677fe640 1 -- 172.21.15.43:0/2126241545 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c010690 con 0x7f7e70101de0 2024-09-11T23:27:25.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.162+0000 7f7e787c7640 1 -- 172.21.15.43:0/2126241545 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e701a4f50 con 0x7f7e70101de0 2024-09-11T23:27:25.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.164+0000 7f7e677fe640 1 -- 172.21.15.43:0/2126241545 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7e6c0046c0 con 0x7f7e70101de0 2024-09-11T23:27:25.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.165+0000 7f7e7aba0740 1 Processor -- start 2024-09-11T23:27:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.165+0000 7f7e7aba0740 1 -- start start 2024-09-11T23:27:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.165+0000 7f7e7aba0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c67b40 0x563648c66090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.165+0000 7f7e677fe640 1 --2- 172.21.15.43:0/2126241545 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7e5c0a2cf0 0x7f7e5c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.165+0000 7f7e7aba0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c665d0 0x563648c66a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.165+0000 7f7e7aba0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563648c71370 0x563648c73760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.165+0000 7f7e7aba0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563648bcc2f0 con 0x563648c67b40 2024-09-11T23:27:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.165+0000 7f7e7aba0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563648b73cf0 con 0x563648c665d0 2024-09-11T23:27:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.165+0000 7f7e7aba0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563648bc82b0 con 0x563648c71370 2024-09-11T23:27:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.165+0000 7f7e75d62640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c665d0 0x563648c66a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.165+0000 7f7e75d62640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c665d0 0x563648c66a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:49412/0 (socket says 172.21.15.43:49412) 2024-09-11T23:27:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.165+0000 7f7e75d62640 1 -- 172.21.15.43:0/1926749760 learned_addr learned my addr 172.21.15.43:0/1926749760 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.166+0000 7f7e76563640 1 --2- 172.21.15.43:0/2126241545 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7e5c0a2cf0 0x7f7e5c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.166+0000 7f7e76d64640 1 --2- 172.21.15.43:0/1926749760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c67b40 0x563648c66090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:25.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.166+0000 7f7e76d64640 1 --2- 172.21.15.43:0/1926749760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563648c71370 0x563648c73760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:25.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.166+0000 7f7e75d62640 1 -- 172.21.15.43:0/1926749760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563648c71370 msgr2=0x563648c73760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:25.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.166+0000 7f7e75d62640 1 --2- 172.21.15.43:0/1926749760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563648c71370 0x563648c73760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.166+0000 7f7e75d62640 1 -- 172.21.15.43:0/1926749760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c67b40 msgr2=0x563648c66090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:25.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.166+0000 7f7e75d62640 1 --2- 172.21.15.43:0/1926749760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c67b40 0x563648c66090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.166+0000 7f7e677fe640 1 -- 172.21.15.43:0/2126241545 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f7e6c0cf8b0 con 0x7f7e70101de0 2024-09-11T23:27:25.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.166+0000 7f7e75d62640 1 -- 172.21.15.43:0/1926749760 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563648b583a0 con 0x563648c665d0 2024-09-11T23:27:25.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.166+0000 7f7e76d64640 1 --2- 172.21.15.43:0/1926749760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c67b40 0x563648c66090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:27:25.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.166+0000 7f7e76563640 1 --2- 172.21.15.43:0/2126241545 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7e5c0a2cf0 0x7f7e5c0a51b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c09ba70 tx=0x7f7e6c06ddf0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:25.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.166+0000 7f7e75d62640 1 --2- 172.21.15.43:0/1926749760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c665d0 0x563648c66a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7e60011470 tx=0x7f7e60011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0cff72e679db608 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:25.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.166+0000 7f7e64ff9640 1 -- 172.21.15.43:0/1926749760 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6001ac30 con 0x563648c665d0 2024-09-11T23:27:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.166+0000 7f7e64ff9640 1 -- 172.21.15.43:0/1926749760 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e6001add0 con 0x563648c665d0 2024-09-11T23:27:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.166+0000 7f7e64ff9640 1 -- 172.21.15.43:0/1926749760 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e60022430 con 0x563648c665d0 2024-09-11T23:27:25.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.167+0000 7f7e7aba0740 1 -- 172.21.15.43:0/1926749760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c665d0 msgr2=0x563648c66a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:25.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.167+0000 7f7e7aba0740 1 --2- 172.21.15.43:0/1926749760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c665d0 0x563648c66a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7e60011470 tx=0x7f7e60011930 comp rx=0 tx=0).stop 2024-09-11T23:27:25.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.167+0000 7f7e7aba0740 1 -- 172.21.15.43:0/1926749760 shutdown_connections 2024-09-11T23:27:25.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.167+0000 7f7e7aba0740 1 --2- 172.21.15.43:0/1926749760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563648c71370 0x563648c73760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.167+0000 7f7e7aba0740 1 --2- 172.21.15.43:0/1926749760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c665d0 0x563648c66a10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.167+0000 7f7e7aba0740 1 --2- 172.21.15.43:0/1926749760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c67b40 0x563648c66090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.167+0000 7f7e7aba0740 1 -- 172.21.15.43:0/1926749760 >> 172.21.15.43:0/1926749760 conn(0x563648c2f310 msgr2=0x563648c62c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:25.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.168+0000 7f7e7aba0740 1 -- 172.21.15.43:0/1926749760 shutdown_connections 2024-09-11T23:27:25.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.168+0000 7f7e7aba0740 1 -- 172.21.15.43:0/1926749760 wait complete. 2024-09-11T23:27:25.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.168+0000 7f7e7aba0740 1 -- start start 2024-09-11T23:27:25.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.168+0000 7f7e7aba0740 1 Processor -- start 2024-09-11T23:27:25.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.169+0000 7f7e7aba0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c665d0 0x563648c70d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:25.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.169+0000 7f7e7aba0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c67b40 0x563648c6dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:25.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.169+0000 7f7e7aba0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563648c71370 0x563648c6e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:25.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.169+0000 7f7e7aba0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563648c74070 con 0x563648c67b40 2024-09-11T23:27:25.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.169+0000 7f7e7aba0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563648b87060 con 0x563648c665d0 2024-09-11T23:27:25.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.169+0000 7f7e7aba0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563648c741f0 con 0x563648c71370 2024-09-11T23:27:25.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.169+0000 7f7e76d64640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c665d0 0x563648c70d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:25.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.169+0000 7f7e76d64640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c665d0 0x563648c70d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:49416/0 (socket says 172.21.15.43:49416) 2024-09-11T23:27:25.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.169+0000 7f7e76d64640 1 -- 172.21.15.43:0/2992636674 learned_addr learned my addr 172.21.15.43:0/2992636674 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:25.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.169+0000 7f7e75d62640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c67b40 0x563648c6dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:25.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.169+0000 7f7e76d64640 1 --2- 172.21.15.43:0/2992636674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563648c71370 0x563648c6e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:25.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.169+0000 7f7e7aba0740 10 client.0 _ll_register_callbacks cb 0x5636488da8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:25.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.169+0000 7f7e76d64640 1 -- 172.21.15.43:0/2992636674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563648c71370 msgr2=0x563648c6e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:25.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.169+0000 7f7e76d64640 1 --2- 172.21.15.43:0/2992636674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563648c71370 0x563648c6e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.169+0000 7f7e76d64640 1 -- 172.21.15.43:0/2992636674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c67b40 msgr2=0x563648c6dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:25.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.169+0000 7f7e76d64640 1 --2- 172.21.15.43:0/2992636674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c67b40 0x563648c6dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.169+0000 7f7e76d64640 1 -- 172.21.15.43:0/2992636674 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56364893d3c0 con 0x563648c665d0 2024-09-11T23:27:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.169+0000 7f7e76d64640 1 --2- 172.21.15.43:0/2992636674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c665d0 0x563648c70d30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7e680093d0 tx=0x7f7e6802a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2347069d8b28738 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.169+0000 7f7e59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:27:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.170+0000 7f7e59ffb640 1 -- 172.21.15.43:0/2992636674 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6804d060 con 0x563648c665d0 2024-09-11T23:27:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.170+0000 7f7e59ffb640 1 -- 172.21.15.43:0/2992636674 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f7e6802f810 con 0x563648c665d0 2024-09-11T23:27:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.170+0000 7f7e7aba0740 1 -- 172.21.15.43:0/2992636674 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e6001add0 con 0x563648c665d0 2024-09-11T23:27:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.170+0000 7f7e59ffb640 1 -- 172.21.15.43:0/2992636674 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7e680319d0 con 0x563648c665d0 2024-09-11T23:27:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.170+0000 7f7e7aba0740 10 client.25787 fetch_fsmap learned FSMap version 30 2024-09-11T23:27:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.170+0000 7f7e7aba0740 1 -- 172.21.15.43:0/2992636674 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563648c7fab0 con 0x563648c665d0 2024-09-11T23:27:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.170+0000 7f7e59ffb640 1 -- 172.21.15.43:0/2992636674 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e6001add0 con 0x563648c665d0 2024-09-11T23:27:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.170+0000 7f7e59ffb640 1 -- 172.21.15.43:0/2992636674 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7e68030d50 con 0x563648c665d0 2024-09-11T23:27:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.170+0000 7f7e7aba0740 10 client.25787 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:27:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.170+0000 7f7e7aba0740 10 client.25787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:27:25.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.170+0000 7f7e7aba0740 20 client.25787 populate_metadata read hostname 'smithi043' 2024-09-11T23:27:25.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.170+0000 7f7e7aba0740 1 --2- 172.21.15.43:0/2992636674 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x563648c80e00 0x563648c83230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:25.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.170+0000 7f7e7aba0740 4 client.25787 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:27:25.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.170+0000 7f7e7aba0740 1 -- 172.21.15.43:0/2992636674 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563648bd9860 con 0x563648c80e00 2024-09-11T23:27:25.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.171+0000 7f7e75d62640 1 --2- 172.21.15.43:0/2992636674 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x563648c80e00 0x563648c83230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:25.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.171+0000 7f7e59ffb640 10 client.25787 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:27:25.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.171+0000 7f7e75d62640 1 --2- 172.21.15.43:0/2992636674 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x563648c80e00 0x563648c83230 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e09924fd4a504a7 server_cookie=942da4dbb0909f22 in_seq=0 out_seq=0 2024-09-11T23:27:25.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.172+0000 7f7e59ffb640 1 -- 172.21.15.43:0/2992636674 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563648bd9860 con 0x563648c80e00 2024-09-11T23:27:25.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.172+0000 7f7e59ffb640 10 client.25787 handle_command_reply: tid=0 2024-09-11T23:27:25.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.172+0000 7f7e7aba0740 1 client.25787 shutdown 2024-09-11T23:27:25.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.172+0000 7f7e7aba0740 1 -- 172.21.15.43:0/2992636674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c665d0 msgr2=0x563648c70d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:25.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.172+0000 7f7e7aba0740 1 --2- 172.21.15.43:0/2992636674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c665d0 0x563648c70d30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7e680093d0 tx=0x7f7e6802a8c0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.172+0000 7f7e7aba0740 1 -- 172.21.15.43:0/2992636674 shutdown_connections 2024-09-11T23:27:25.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.172+0000 7f7e7aba0740 1 --2- 172.21.15.43:0/2992636674 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x563648c80e00 0x563648c83230 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.172+0000 7f7e7aba0740 1 --2- 172.21.15.43:0/2992636674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563648c71370 0x563648c6e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.172+0000 7f7e7aba0740 1 --2- 172.21.15.43:0/2992636674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c67b40 0x563648c6dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.172+0000 7f7e59ffb640 0 client.25787 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:27:25.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.172+0000 7f7e7aba0740 1 --2- 172.21.15.43:0/2992636674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c665d0 0x563648c70d30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.172+0000 7f7e7aba0740 1 -- 172.21.15.43:0/2992636674 >> 172.21.15.43:0/2992636674 conn(0x563648c2f310 msgr2=0x563648c68960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:25.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.172+0000 7f7e7aba0740 1 -- 172.21.15.43:0/2992636674 shutdown_connections 2024-09-11T23:27:25.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.172+0000 7f7e7aba0740 1 -- 172.21.15.43:0/2992636674 wait complete. 2024-09-11T23:27:25.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.172+0000 7f7e7aba0740 20 client.25787 trim_cache size 0 max 16384 2024-09-11T23:27:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.186+0000 7f7e7aba0740 1 Processor -- start 2024-09-11T23:27:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.186+0000 7f7e7aba0740 1 -- start start 2024-09-11T23:27:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.187+0000 7f7e7aba0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c665d0 0x563648c7d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.187+0000 7f7e7aba0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c67b40 0x563648c6d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.187+0000 7f7e7aba0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563648c71370 0x563648c72760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.187+0000 7f7e7aba0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7e701adf20 con 0x563648c665d0 2024-09-11T23:27:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.187+0000 7f7e7aba0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7e701ab7f0 con 0x563648c67b40 2024-09-11T23:27:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.187+0000 7f7e7aba0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7e701a9640 con 0x563648c71370 2024-09-11T23:27:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.187+0000 7f7e76d64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c665d0 0x563648c7d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.187+0000 7f7e76d64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c665d0 0x563648c7d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41958/0 (socket says 172.21.15.43:41958) 2024-09-11T23:27:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.187+0000 7f7e76d64640 1 -- 172.21.15.43:0/3729772548 learned_addr learned my addr 172.21.15.43:0/3729772548 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:25.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.187+0000 7f7e75d62640 1 --2- 172.21.15.43:0/3729772548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c67b40 0x563648c6d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:25.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.187+0000 7f7e76d64640 1 --2- 172.21.15.43:0/3729772548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563648c71370 0x563648c72760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:25.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.187+0000 7f7e76d64640 1 -- 172.21.15.43:0/3729772548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563648c71370 msgr2=0x563648c72760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:25.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.187+0000 7f7e76d64640 1 --2- 172.21.15.43:0/3729772548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563648c71370 0x563648c72760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.187+0000 7f7e76d64640 1 -- 172.21.15.43:0/3729772548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c67b40 msgr2=0x563648c6d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:25.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.187+0000 7f7e76d64640 1 --2- 172.21.15.43:0/3729772548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c67b40 0x563648c6d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.187+0000 7f7e76d64640 1 -- 172.21.15.43:0/3729772548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563648c6c530 con 0x563648c665d0 2024-09-11T23:27:25.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.187+0000 7f7e75d62640 1 --2- 172.21.15.43:0/3729772548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c67b40 0x563648c6d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:27:25.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.187+0000 7f7e76d64640 1 --2- 172.21.15.43:0/3729772548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c665d0 0x563648c7d310 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7e68005d50 tx=0x7f7e68031450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55259a63dae12ad8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:25.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.188+0000 7f7e5bfff640 1 -- 172.21.15.43:0/3729772548 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e68004d40 con 0x563648c665d0 2024-09-11T23:27:25.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.188+0000 7f7e5bfff640 1 -- 172.21.15.43:0/3729772548 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e68036040 con 0x563648c665d0 2024-09-11T23:27:25.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.188+0000 7f7e5bfff640 1 -- 172.21.15.43:0/3729772548 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6802f870 con 0x563648c665d0 2024-09-11T23:27:25.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.188+0000 7f7e7aba0740 1 -- 172.21.15.43:0/3729772548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c665d0 msgr2=0x563648c7d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:25.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.188+0000 7f7e7aba0740 1 --2- 172.21.15.43:0/3729772548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c665d0 0x563648c7d310 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7e68005d50 tx=0x7f7e68031450 comp rx=0 tx=0).stop 2024-09-11T23:27:25.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e7aba0740 1 -- 172.21.15.43:0/3729772548 shutdown_connections 2024-09-11T23:27:25.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e7aba0740 1 --2- 172.21.15.43:0/3729772548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563648c71370 0x563648c72760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e7aba0740 1 --2- 172.21.15.43:0/3729772548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c67b40 0x563648c6d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e7aba0740 1 --2- 172.21.15.43:0/3729772548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c665d0 0x563648c7d310 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e7aba0740 1 -- 172.21.15.43:0/3729772548 >> 172.21.15.43:0/3729772548 conn(0x563648c2f310 msgr2=0x563648c69310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e7aba0740 1 -- 172.21.15.43:0/3729772548 shutdown_connections 2024-09-11T23:27:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e7aba0740 1 -- 172.21.15.43:0/3729772548 wait complete. 2024-09-11T23:27:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e7aba0740 1 -- start start 2024-09-11T23:27:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e7aba0740 1 Processor -- start 2024-09-11T23:27:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e7aba0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c665d0 0x563648c79be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e7aba0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c67b40 0x563648c884b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e7aba0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563648c71370 0x563648c889f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e7aba0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7e701a9cb0 con 0x563648c67b40 2024-09-11T23:27:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e7aba0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7e701ab120 con 0x563648c665d0 2024-09-11T23:27:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e7aba0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7e701aa380 con 0x563648c71370 2024-09-11T23:27:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e7aba0740 10 client.0 _ll_register_callbacks cb 0x7f7e6c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:25.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e75d62640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c67b40 0x563648c884b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:25.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e76d64640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c665d0 0x563648c79be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:25.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e76d64640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563648c71370 0x563648c889f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:25.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e75d62640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c67b40 0x563648c884b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41966/0 (socket says 172.21.15.43:41966) 2024-09-11T23:27:25.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e76d64640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563648c71370 0x563648c889f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:41974/0 (socket says 172.21.15.43:41974) 2024-09-11T23:27:25.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.189+0000 7f7e75d62640 1 -- 172.21.15.43:0/669090246 learned_addr learned my addr 172.21.15.43:0/669090246 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:25.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.190+0000 7f7e75d62640 1 -- 172.21.15.43:0/669090246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563648c71370 msgr2=0x563648c889f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:25.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.190+0000 7f7e75d62640 1 --2- 172.21.15.43:0/669090246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563648c71370 0x563648c889f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.190+0000 7f7e75d62640 1 -- 172.21.15.43:0/669090246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c665d0 msgr2=0x563648c79be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:25.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.190+0000 7f7e75d62640 1 --2- 172.21.15.43:0/669090246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c665d0 0x563648c79be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.190+0000 7f7e75d62640 1 -- 172.21.15.43:0/669090246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563648c89220 con 0x563648c67b40 2024-09-11T23:27:25.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.190+0000 7f7e75d62640 1 --2- 172.21.15.43:0/669090246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c67b40 0x563648c884b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009380 tx=0x7f7e600096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4bd41979761647 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:25.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.190+0000 7f7e59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:27:25.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.190+0000 7f7e59ffb640 1 -- 172.21.15.43:0/669090246 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e60003800 con 0x563648c67b40 2024-09-11T23:27:25.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.190+0000 7f7e7aba0740 1 -- 172.21.15.43:0/669090246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e68036040 con 0x563648c67b40 2024-09-11T23:27:25.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.190+0000 7f7e59ffb640 1 -- 172.21.15.43:0/669090246 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f7e60003b40 con 0x563648c67b40 2024-09-11T23:27:25.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.191+0000 7f7e59ffb640 1 -- 172.21.15.43:0/669090246 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7e60003d60 con 0x563648c67b40 2024-09-11T23:27:25.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.191+0000 7f7e7aba0740 10 client.16266 fetch_fsmap learned FSMap version 30 2024-09-11T23:27:25.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.191+0000 7f7e7aba0740 1 -- 172.21.15.43:0/669090246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563648cebb60 con 0x563648c67b40 2024-09-11T23:27:25.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.191+0000 7f7e59ffb640 1 -- 172.21.15.43:0/669090246 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e68036040 con 0x563648c67b40 2024-09-11T23:27:25.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.191+0000 7f7e59ffb640 1 -- 172.21.15.43:0/669090246 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7e60022790 con 0x563648c67b40 2024-09-11T23:27:25.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.191+0000 7f7e7aba0740 10 client.16266 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:27:25.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.191+0000 7f7e7aba0740 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:27:25.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.191+0000 7f7e7aba0740 20 client.16266 populate_metadata read hostname 'smithi043' 2024-09-11T23:27:25.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.191+0000 7f7e7aba0740 1 --2- 172.21.15.43:0/669090246 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x563648cebed0 0x563648cee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:25.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.191+0000 7f7e7aba0740 4 client.16266 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:27:25.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.191+0000 7f7e7aba0740 1 -- 172.21.15.43:0/669090246 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563648bd3d70 con 0x563648cebed0 2024-09-11T23:27:25.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.192+0000 7f7e76d64640 1 --2- 172.21.15.43:0/669090246 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x563648cebed0 0x563648cee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:25.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.192+0000 7f7e76d64640 1 --2- 172.21.15.43:0/669090246 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x563648cebed0 0x563648cee2c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcd92b3ab7e9858d server_cookie=dbc2cb783470ec3e in_seq=0 out_seq=0 2024-09-11T23:27:25.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.192+0000 7f7e59ffb640 10 client.16266 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:27:25.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.192+0000 7f7e59ffb640 1 -- 172.21.15.43:0/669090246 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563648bd3d70 con 0x563648cebed0 2024-09-11T23:27:25.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.192+0000 7f7e59ffb640 10 client.16266 handle_command_reply: tid=0 2024-09-11T23:27:25.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.192+0000 7f7e7aba0740 1 client.16266 shutdown 2024-09-11T23:27:25.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.192+0000 7f7e7aba0740 1 -- 172.21.15.43:0/669090246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c67b40 msgr2=0x563648c884b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:25.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.192+0000 7f7e7aba0740 1 --2- 172.21.15.43:0/669090246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c67b40 0x563648c884b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009380 tx=0x7f7e600096c0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.193+0000 7f7e7aba0740 1 -- 172.21.15.43:0/669090246 shutdown_connections 2024-09-11T23:27:25.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.193+0000 7f7e7aba0740 1 --2- 172.21.15.43:0/669090246 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x563648cebed0 0x563648cee2c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.193+0000 7f7e7aba0740 1 --2- 172.21.15.43:0/669090246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563648c71370 0x563648c889f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.193+0000 7f7e7aba0740 1 --2- 172.21.15.43:0/669090246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563648c67b40 0x563648c884b0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.193+0000 7f7e59ffb640 0 client.16266 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:27:25.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.193+0000 7f7e7aba0740 1 --2- 172.21.15.43:0/669090246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563648c665d0 0x563648c79be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.193+0000 7f7e7aba0740 1 -- 172.21.15.43:0/669090246 >> 172.21.15.43:0/669090246 conn(0x563648c2f310 msgr2=0x563648c6c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:25.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.193+0000 7f7e7aba0740 1 -- 172.21.15.43:0/669090246 shutdown_connections 2024-09-11T23:27:25.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.193+0000 7f7e7aba0740 1 -- 172.21.15.43:0/669090246 wait complete. 2024-09-11T23:27:25.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.193+0000 7f7e7aba0740 20 client.16266 trim_cache size 0 max 16384 2024-09-11T23:27:25.198 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:27:25.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.193+0000 7f7e787c7640 1 -- 172.21.15.43:0/2126241545 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7e5c0a2cf0 msgr2=0x7f7e5c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:25.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.193+0000 7f7e787c7640 1 --2- 172.21.15.43:0/2126241545 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7e5c0a2cf0 0x7f7e5c0a51b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c09ba70 tx=0x7f7e6c06ddf0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.193+0000 7f7e787c7640 1 -- 172.21.15.43:0/2126241545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e70101de0 msgr2=0x7f7e701983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:25.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.193+0000 7f7e787c7640 1 --2- 172.21.15.43:0/2126241545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e70101de0 0x7f7e701983d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00e9f0 tx=0x7f7e6c00edf0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.193+0000 7f7e787c7640 1 -- 172.21.15.43:0/2126241545 shutdown_connections 2024-09-11T23:27:25.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.193+0000 7f7e787c7640 1 --2- 172.21.15.43:0/2126241545 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7e5c0a2cf0 0x7f7e5c0a51b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.193+0000 7f7e787c7640 1 --2- 172.21.15.43:0/2126241545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e70106f10 0x7f7e7019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.193+0000 7f7e787c7640 1 --2- 172.21.15.43:0/2126241545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e70102790 0x7f7e70198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.193+0000 7f7e787c7640 1 --2- 172.21.15.43:0/2126241545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e70101de0 0x7f7e701983d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:25.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.193+0000 7f7e787c7640 1 -- 172.21.15.43:0/2126241545 >> 172.21.15.43:0/2126241545 conn(0x7f7e700fd8c0 msgr2=0x7f7e700feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:25.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.193+0000 7f7e787c7640 1 -- 172.21.15.43:0/2126241545 shutdown_connections 2024-09-11T23:27:25.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:25.194+0000 7f7e787c7640 1 -- 172.21.15.43:0/2126241545 wait complete. 2024-09-11T23:27:26.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:25 smithi043 ceph-mon[30950]: pgmap v171: 97 pgs: 97 active+clean; 19 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 MiB/s wr, 6.48k op/s 2024-09-11T23:27:26.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:25 smithi090 ceph-mon[35130]: pgmap v171: 97 pgs: 97 active+clean; 19 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 MiB/s wr, 6.48k op/s 2024-09-11T23:27:26.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:25 smithi097 ceph-mon[35562]: pgmap v171: 97 pgs: 97 active+clean; 19 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 MiB/s wr, 6.48k op/s 2024-09-11T23:27:26.996 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:27:26.996 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:27:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:27:28.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:27 smithi090 ceph-mon[35130]: pgmap v172: 97 pgs: 97 active+clean; 19 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 MiB/s wr, 5.21k op/s 2024-09-11T23:27:28.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:27 smithi097 ceph-mon[35562]: pgmap v172: 97 pgs: 97 active+clean; 19 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 MiB/s wr, 5.21k op/s 2024-09-11T23:27:28.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:27 smithi043 ceph-mon[30950]: pgmap v172: 97 pgs: 97 active+clean; 19 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 MiB/s wr, 5.21k op/s 2024-09-11T23:27:30.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:29 smithi090 ceph-mon[35130]: pgmap v173: 97 pgs: 97 active+clean; 19 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 36 MiB/s wr, 6.47k op/s 2024-09-11T23:27:30.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:29 smithi097 ceph-mon[35562]: pgmap v173: 97 pgs: 97 active+clean; 19 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 36 MiB/s wr, 6.47k op/s 2024-09-11T23:27:30.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:29 smithi043 ceph-mon[30950]: pgmap v173: 97 pgs: 97 active+clean; 19 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 36 MiB/s wr, 6.47k op/s 2024-09-11T23:27:31.591 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T23:27:31.592 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-11T23:27:31.592 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:27:32.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:31 smithi090 ceph-mon[35130]: pgmap v174: 97 pgs: 97 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 28 MiB/s wr, 5.03k op/s 2024-09-11T23:27:32.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:31 smithi090 ceph-mon[35130]: scrub queued for path: / 2024-09-11T23:27:32.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:31 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:27:32.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:31 smithi090 ceph-mon[35130]: scrub summary: active paths [/] 2024-09-11T23:27:32.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:31 smithi090 ceph-mon[35130]: scrub summary: active paths [/] 2024-09-11T23:27:32.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:31 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:27:32.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:31 smithi097 ceph-mon[35562]: pgmap v174: 97 pgs: 97 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 28 MiB/s wr, 5.03k op/s 2024-09-11T23:27:32.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:31 smithi097 ceph-mon[35562]: scrub queued for path: / 2024-09-11T23:27:32.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:31 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:27:32.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:31 smithi097 ceph-mon[35562]: scrub summary: active paths [/] 2024-09-11T23:27:32.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:31 smithi097 ceph-mon[35562]: scrub summary: active paths [/] 2024-09-11T23:27:32.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:31 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:27:32.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:31 smithi043 ceph-mon[30950]: pgmap v174: 97 pgs: 97 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 28 MiB/s wr, 5.03k op/s 2024-09-11T23:27:32.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:31 smithi043 ceph-mon[30950]: scrub queued for path: / 2024-09-11T23:27:32.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:31 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:27:32.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:31 smithi043 ceph-mon[30950]: scrub summary: active paths [/] 2024-09-11T23:27:32.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:31 smithi043 ceph-mon[30950]: scrub summary: active paths [/] 2024-09-11T23:27:32.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:31 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:27:32.419 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:27:32.419 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-11T23:27:32.419 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "2010b9bf-0fac-4632-b3c1-aa91af656687", 2024-09-11T23:27:32.419 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-11T23:27:32.419 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:27:32.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.685+0000 7f1521598640 1 -- 172.21.15.43:0/2319742619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c1009b0 msgr2=0x7f151c100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:32.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.685+0000 7f1521598640 1 --2- 172.21.15.43:0/2319742619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c1009b0 0x7f151c100d90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f150c009900 tx=0x7f150c02a410 comp rx=0 tx=0).stop 2024-09-11T23:27:32.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.685+0000 7f1521598640 1 -- 172.21.15.43:0/2319742619 shutdown_connections 2024-09-11T23:27:32.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.685+0000 7f1521598640 1 --2- 172.21.15.43:0/2319742619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f151c10e4d0 0x7f151c1108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.686+0000 7f1521598640 1 --2- 172.21.15.43:0/2319742619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f151c101360 0x7f151c10df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.686+0000 7f1521598640 1 --2- 172.21.15.43:0/2319742619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c1009b0 0x7f151c100d90 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.686+0000 7f1521598640 1 -- 172.21.15.43:0/2319742619 >> 172.21.15.43:0/2319742619 conn(0x7f151c0fc7d0 msgr2=0x7f151c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:32.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.686+0000 7f1521598640 1 -- 172.21.15.43:0/2319742619 shutdown_connections 2024-09-11T23:27:32.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.686+0000 7f1521598640 1 -- 172.21.15.43:0/2319742619 wait complete. 2024-09-11T23:27:32.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.687+0000 7f1521598640 1 Processor -- start 2024-09-11T23:27:32.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.687+0000 7f1521598640 1 -- start start 2024-09-11T23:27:32.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.687+0000 7f1521598640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f151c1009b0 0x7f151c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.687+0000 7f1521598640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c101360 0x7f151c19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.687+0000 7f1521598640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f151c10e4d0 0x7f151c19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.687+0000 7f1521598640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f151c114210 con 0x7f151c101360 2024-09-11T23:27:32.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.687+0000 7f1521598640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f151c114090 con 0x7f151c1009b0 2024-09-11T23:27:32.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.687+0000 7f1521598640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f151c114390 con 0x7f151c10e4d0 2024-09-11T23:27:32.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.687+0000 7f151a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c101360 0x7f151c19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.687+0000 7f151a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c101360 0x7f151c19ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40832/0 (socket says 172.21.15.43:40832) 2024-09-11T23:27:32.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.687+0000 7f151a7fc640 1 -- 172.21.15.43:0/3757683977 learned_addr learned my addr 172.21.15.43:0/3757683977 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:32.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.687+0000 7f151b7fe640 1 --2- 172.21.15.43:0/3757683977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f151c10e4d0 0x7f151c19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.688+0000 7f151affd640 1 --2- 172.21.15.43:0/3757683977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f151c1009b0 0x7f151c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.688+0000 7f151b7fe640 1 -- 172.21.15.43:0/3757683977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f151c1009b0 msgr2=0x7f151c19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:32.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.688+0000 7f151b7fe640 1 --2- 172.21.15.43:0/3757683977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f151c1009b0 0x7f151c19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.688+0000 7f151b7fe640 1 -- 172.21.15.43:0/3757683977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c101360 msgr2=0x7f151c19ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:32.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.688+0000 7f151b7fe640 1 --2- 172.21.15.43:0/3757683977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c101360 0x7f151c19ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.688+0000 7f151b7fe640 1 -- 172.21.15.43:0/3757683977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151c19f6d0 con 0x7f151c10e4d0 2024-09-11T23:27:32.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.688+0000 7f151a7fc640 1 --2- 172.21.15.43:0/3757683977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c101360 0x7f151c19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:27:32.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.688+0000 7f151b7fe640 1 --2- 172.21.15.43:0/3757683977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f151c10e4d0 0x7f151c19eff0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f151400e940 tx=0x7f151400ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:32.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.688+0000 7f1503fff640 1 -- 172.21.15.43:0/3757683977 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514004d20 con 0x7f151c10e4d0 2024-09-11T23:27:32.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.688+0000 7f1503fff640 1 -- 172.21.15.43:0/3757683977 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1514005020 con 0x7f151c10e4d0 2024-09-11T23:27:32.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.688+0000 7f1521598640 1 -- 172.21.15.43:0/3757683977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151c19f9c0 con 0x7f151c10e4d0 2024-09-11T23:27:32.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.688+0000 7f1503fff640 1 -- 172.21.15.43:0/3757683977 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15140116b0 con 0x7f151c10e4d0 2024-09-11T23:27:32.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.689+0000 7f1521598640 1 -- 172.21.15.43:0/3757683977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f151c1a72a0 con 0x7f151c10e4d0 2024-09-11T23:27:32.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.691+0000 7f1503fff640 1 -- 172.21.15.43:0/3757683977 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f15140054d0 con 0x7f151c10e4d0 2024-09-11T23:27:32.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.692+0000 7f1523971740 1 Processor -- start 2024-09-11T23:27:32.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.692+0000 7f1523971740 1 -- start start 2024-09-11T23:27:32.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.692+0000 7f1503fff640 1 --2- 172.21.15.43:0/3757683977 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f15040a2d70 0x7f15040a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.692+0000 7f1523971740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd3048ab60 0x55fd3048af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.692+0000 7f1523971740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd3048b480 0x55fd30495800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.693+0000 7f1523971740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd30495d40 0x55fd30498130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.693+0000 7f1503fff640 1 -- 172.21.15.43:0/3757683977 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f15140cf780 con 0x7f151c10e4d0 2024-09-11T23:27:32.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.693+0000 7f1523971740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fd303f0280 con 0x55fd3048b480 2024-09-11T23:27:32.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.693+0000 7f1523971740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fd30397b10 con 0x55fd3048ab60 2024-09-11T23:27:32.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.693+0000 7f1523971740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55fd303ec240 con 0x55fd30495d40 2024-09-11T23:27:32.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.693+0000 7f151a7fc640 1 --2- 172.21.15.43:0/3757683977 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f15040a2d70 0x7f15040a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.693+0000 7f151a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd3048b480 0x55fd30495800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.693+0000 7f151a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd3048b480 0x55fd30495800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40846/0 (socket says 172.21.15.43:40846) 2024-09-11T23:27:32.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.693+0000 7f151a7fc640 1 -- 172.21.15.43:0/3082037863 learned_addr learned my addr 172.21.15.43:0/3082037863 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:32.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.693+0000 7f151b7fe640 1 --2- 172.21.15.43:0/3082037863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd30495d40 0x55fd30498130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.693+0000 7f151affd640 1 --2- 172.21.15.43:0/3082037863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd3048ab60 0x55fd3048af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.693+0000 7f151b7fe640 1 -- 172.21.15.43:0/3082037863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd3048ab60 msgr2=0x55fd3048af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:32.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.693+0000 7f151b7fe640 1 --2- 172.21.15.43:0/3082037863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd3048ab60 0x55fd3048af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.693+0000 7f151b7fe640 1 -- 172.21.15.43:0/3082037863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd3048b480 msgr2=0x55fd30495800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:32.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.693+0000 7f151b7fe640 1 --2- 172.21.15.43:0/3082037863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd3048b480 0x55fd30495800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.694+0000 7f151b7fe640 1 -- 172.21.15.43:0/3082037863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd303a41f0 con 0x55fd30495d40 2024-09-11T23:27:32.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.694+0000 7f151a7fc640 1 --2- 172.21.15.43:0/3082037863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd3048b480 0x55fd30495800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:27:32.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.694+0000 7f151b7fe640 1 --2- 172.21.15.43:0/3082037863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd30495d40 0x55fd30498130 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f15140cce70 tx=0x7f151409b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dc5d7bb189717f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:32.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.694+0000 7f151a7fc640 1 --2- 172.21.15.43:0/3757683977 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f15040a2d70 0x7f15040a5230 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1510005cc0 tx=0x7f151000bf90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:32.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.694+0000 7f15017fa640 1 -- 172.21.15.43:0/3082037863 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151401f070 con 0x55fd30495d40 2024-09-11T23:27:32.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.694+0000 7f15017fa640 1 -- 172.21.15.43:0/3082037863 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15140a48b0 con 0x55fd30495d40 2024-09-11T23:27:32.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.694+0000 7f15017fa640 1 -- 172.21.15.43:0/3082037863 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15140a4680 con 0x55fd30495d40 2024-09-11T23:27:32.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.695+0000 7f1523971740 1 -- 172.21.15.43:0/3082037863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd30495d40 msgr2=0x55fd30498130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:32.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.695+0000 7f1523971740 1 --2- 172.21.15.43:0/3082037863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd30495d40 0x55fd30498130 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f15140cce70 tx=0x7f151409b760 comp rx=0 tx=0).stop 2024-09-11T23:27:32.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.695+0000 7f1523971740 1 -- 172.21.15.43:0/3082037863 shutdown_connections 2024-09-11T23:27:32.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.695+0000 7f1523971740 1 --2- 172.21.15.43:0/3082037863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd30495d40 0x55fd30498130 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.695+0000 7f1523971740 1 --2- 172.21.15.43:0/3082037863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd3048b480 0x55fd30495800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.695+0000 7f1523971740 1 --2- 172.21.15.43:0/3082037863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd3048ab60 0x55fd3048af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.695+0000 7f1523971740 1 -- 172.21.15.43:0/3082037863 >> 172.21.15.43:0/3082037863 conn(0x55fd304870f0 msgr2=0x55fd304874d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:32.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.695+0000 7f1523971740 1 -- 172.21.15.43:0/3082037863 shutdown_connections 2024-09-11T23:27:32.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.695+0000 7f1523971740 1 -- 172.21.15.43:0/3082037863 wait complete. 2024-09-11T23:27:32.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.696+0000 7f1523971740 1 -- start start 2024-09-11T23:27:32.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.696+0000 7f1523971740 1 Processor -- start 2024-09-11T23:27:32.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.696+0000 7f1523971740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd3048ab60 0x55fd30492c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.696+0000 7f1523971740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd3048b480 0x55fd30493190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.696+0000 7f1523971740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd30495d40 0x55fd304936d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.696+0000 7f151affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd3048ab60 0x55fd30492c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.696+0000 7f1523971740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fd30498910 con 0x55fd3048ab60 2024-09-11T23:27:32.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.696+0000 7f151a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd3048b480 0x55fd30493190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.696+0000 7f1523971740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fd303aaf10 con 0x55fd30495d40 2024-09-11T23:27:32.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.696+0000 7f1523971740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55fd30498a90 con 0x55fd3048b480 2024-09-11T23:27:32.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.696+0000 7f151affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd3048ab60 0x55fd30492c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40858/0 (socket says 172.21.15.43:40858) 2024-09-11T23:27:32.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.696+0000 7f151affd640 1 -- 172.21.15.43:0/4103695581 learned_addr learned my addr 172.21.15.43:0/4103695581 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:32.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.696+0000 7f151b7fe640 1 --2- 172.21.15.43:0/4103695581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd30495d40 0x55fd304936d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.697+0000 7f1523971740 10 client.0 _ll_register_callbacks cb 0x55fd30373a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:32.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.697+0000 7f151b7fe640 1 -- 172.21.15.43:0/4103695581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd3048b480 msgr2=0x55fd30493190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:32.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.697+0000 7f151b7fe640 1 --2- 172.21.15.43:0/4103695581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd3048b480 0x55fd30493190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.697+0000 7f151b7fe640 1 -- 172.21.15.43:0/4103695581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd3048ab60 msgr2=0x55fd30492c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:32.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.697+0000 7f151b7fe640 1 --2- 172.21.15.43:0/4103695581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd3048ab60 0x55fd30492c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.697+0000 7f151b7fe640 1 -- 172.21.15.43:0/4103695581 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd301616a0 con 0x55fd30495d40 2024-09-11T23:27:32.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.697+0000 7f151affd640 1 --2- 172.21.15.43:0/4103695581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd3048ab60 0x55fd30492c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:27:32.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.697+0000 7f151b7fe640 1 --2- 172.21.15.43:0/4103695581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd30495d40 0x55fd304936d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f15140ccea0 tx=0x7f1514098360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e465bdde15ae188 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:32.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.697+0000 7f14feff5640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:27:32.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.697+0000 7f14feff5640 1 -- 172.21.15.43:0/4103695581 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151401f040 con 0x55fd30495d40 2024-09-11T23:27:32.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.697+0000 7f14feff5640 1 -- 172.21.15.43:0/4103695581 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f151409b140 con 0x55fd30495d40 2024-09-11T23:27:32.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.697+0000 7f1523971740 1 -- 172.21.15.43:0/4103695581 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15140a48b0 con 0x55fd30495d40 2024-09-11T23:27:32.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.698+0000 7f14feff5640 1 -- 172.21.15.43:0/4103695581 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f151409b340 con 0x55fd30495d40 2024-09-11T23:27:32.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.698+0000 7f1523971740 10 client.25805 fetch_fsmap learned FSMap version 30 2024-09-11T23:27:32.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.698+0000 7f1523971740 1 -- 172.21.15.43:0/4103695581 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fd304a4520 con 0x55fd30495d40 2024-09-11T23:27:32.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.698+0000 7f14feff5640 1 -- 172.21.15.43:0/4103695581 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15140b9070 con 0x55fd30495d40 2024-09-11T23:27:32.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.698+0000 7f14feff5640 1 -- 172.21.15.43:0/4103695581 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f15140b4070 con 0x55fd30495d40 2024-09-11T23:27:32.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.698+0000 7f1523971740 10 client.25805 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:27:32.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.698+0000 7f1523971740 10 client.25805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:27:32.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.698+0000 7f1523971740 20 client.25805 populate_metadata read hostname 'smithi043' 2024-09-11T23:27:32.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.698+0000 7f1523971740 1 --2- 172.21.15.43:0/4103695581 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55fd304a5870 0x55fd304a7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.698+0000 7f1523971740 4 client.25805 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:27:32.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.698+0000 7f1523971740 1 -- 172.21.15.43:0/4103695581 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f15140a48b0 con 0x55fd304a5870 2024-09-11T23:27:32.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.698+0000 7f151affd640 1 --2- 172.21.15.43:0/4103695581 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55fd304a5870 0x55fd304a7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.699+0000 7f151affd640 1 --2- 172.21.15.43:0/4103695581 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55fd304a5870 0x55fd304a7c60 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=903932aff57b165b server_cookie=559ac13f6fa0b943 in_seq=0 out_seq=0 2024-09-11T23:27:32.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.699+0000 7f14feff5640 10 client.25805 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:27:32.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.701+0000 7f14feff5640 1 -- 172.21.15.43:0/4103695581 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f15140a48b0 con 0x55fd304a5870 2024-09-11T23:27:32.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.701+0000 7f14feff5640 10 client.25805 handle_command_reply: tid=0 2024-09-11T23:27:32.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.701+0000 7f1523971740 1 client.25805 shutdown 2024-09-11T23:27:32.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.702+0000 7f1523971740 1 -- 172.21.15.43:0/4103695581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd30495d40 msgr2=0x55fd304936d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:32.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.702+0000 7f1523971740 1 --2- 172.21.15.43:0/4103695581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd30495d40 0x55fd304936d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f15140ccea0 tx=0x7f1514098360 comp rx=0 tx=0).stop 2024-09-11T23:27:32.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.702+0000 7f1523971740 1 -- 172.21.15.43:0/4103695581 shutdown_connections 2024-09-11T23:27:32.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.702+0000 7f1523971740 1 --2- 172.21.15.43:0/4103695581 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55fd304a5870 0x55fd304a7c60 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.702+0000 7f1523971740 1 --2- 172.21.15.43:0/4103695581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd30495d40 0x55fd304936d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.702+0000 7f1523971740 1 --2- 172.21.15.43:0/4103695581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd3048b480 0x55fd30493190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.702+0000 7f14feff5640 0 client.25805 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:27:32.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.702+0000 7f1523971740 1 --2- 172.21.15.43:0/4103695581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd3048ab60 0x55fd30492c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.702+0000 7f1523971740 1 -- 172.21.15.43:0/4103695581 >> 172.21.15.43:0/4103695581 conn(0x55fd304870f0 msgr2=0x55fd3048ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:32.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.702+0000 7f1523971740 1 -- 172.21.15.43:0/4103695581 shutdown_connections 2024-09-11T23:27:32.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.702+0000 7f1523971740 1 -- 172.21.15.43:0/4103695581 wait complete. 2024-09-11T23:27:32.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.702+0000 7f1523971740 20 client.25805 trim_cache size 0 max 16384 2024-09-11T23:27:32.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.722+0000 7f1523971740 1 Processor -- start 2024-09-11T23:27:32.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.722+0000 7f1523971740 1 -- start start 2024-09-11T23:27:32.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.722+0000 7f1523971740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd3048ab60 0x55fd30492530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.722+0000 7f1523971740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd3048b480 0x55fd304aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.722+0000 7f1523971740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd30495d40 0x55fd304ab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.722+0000 7f1523971740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1b2460 con 0x55fd3048ab60 2024-09-11T23:27:32.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.722+0000 7f1523971740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1afd30 con 0x55fd30495d40 2024-09-11T23:27:32.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.722+0000 7f1523971740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1adb80 con 0x55fd3048b480 2024-09-11T23:27:32.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.723+0000 7f151affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd3048ab60 0x55fd30492530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.723+0000 7f151affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd3048ab60 0x55fd30492530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40862/0 (socket says 172.21.15.43:40862) 2024-09-11T23:27:32.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.723+0000 7f151affd640 1 -- 172.21.15.43:0/1586620531 learned_addr learned my addr 172.21.15.43:0/1586620531 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:32.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.723+0000 7f151a7fc640 1 --2- 172.21.15.43:0/1586620531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd3048b480 0x55fd304aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.723+0000 7f151b7fe640 1 --2- 172.21.15.43:0/1586620531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd30495d40 0x55fd304ab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.723+0000 7f151affd640 1 -- 172.21.15.43:0/1586620531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd3048b480 msgr2=0x55fd304aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:32.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.723+0000 7f151affd640 1 --2- 172.21.15.43:0/1586620531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd3048b480 0x55fd304aac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.723+0000 7f151affd640 1 -- 172.21.15.43:0/1586620531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd30495d40 msgr2=0x55fd304ab1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:32.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.723+0000 7f151affd640 1 --2- 172.21.15.43:0/1586620531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd30495d40 0x55fd304ab1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.723+0000 7f151affd640 1 -- 172.21.15.43:0/1586620531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd3048d4b0 con 0x55fd3048ab60 2024-09-11T23:27:32.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.723+0000 7f151b7fe640 1 --2- 172.21.15.43:0/1586620531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd30495d40 0x55fd304ab1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:27:32.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.723+0000 7f151affd640 1 --2- 172.21.15.43:0/1586620531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd3048ab60 0x55fd30492530 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f150c006010 tx=0x7f150c0319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a5b381f8c2c5fa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:32.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.723+0000 7f1500ff9640 1 -- 172.21.15.43:0/1586620531 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c035040 con 0x55fd3048ab60 2024-09-11T23:27:32.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.724+0000 7f1500ff9640 1 -- 172.21.15.43:0/1586620531 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f150c005510 con 0x55fd3048ab60 2024-09-11T23:27:32.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.724+0000 7f1500ff9640 1 -- 172.21.15.43:0/1586620531 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c004a40 con 0x55fd3048ab60 2024-09-11T23:27:32.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.724+0000 7f1523971740 1 -- 172.21.15.43:0/1586620531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd3048ab60 msgr2=0x55fd30492530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:32.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.724+0000 7f1523971740 1 --2- 172.21.15.43:0/1586620531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd3048ab60 0x55fd30492530 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f150c006010 tx=0x7f150c0319a0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.724+0000 7f1523971740 1 -- 172.21.15.43:0/1586620531 shutdown_connections 2024-09-11T23:27:32.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.724+0000 7f1523971740 1 --2- 172.21.15.43:0/1586620531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd30495d40 0x55fd304ab1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.724+0000 7f1523971740 1 --2- 172.21.15.43:0/1586620531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd3048b480 0x55fd304aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.724+0000 7f1523971740 1 --2- 172.21.15.43:0/1586620531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd3048ab60 0x55fd30492530 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.724+0000 7f1523971740 1 -- 172.21.15.43:0/1586620531 >> 172.21.15.43:0/1586620531 conn(0x55fd304870f0 msgr2=0x55fd304a1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:32.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.725+0000 7f1523971740 1 -- 172.21.15.43:0/1586620531 shutdown_connections 2024-09-11T23:27:32.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.725+0000 7f1523971740 1 -- 172.21.15.43:0/1586620531 wait complete. 2024-09-11T23:27:32.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.725+0000 7f1523971740 1 -- start start 2024-09-11T23:27:32.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.725+0000 7f1523971740 1 Processor -- start 2024-09-11T23:27:32.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.725+0000 7f1523971740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd3048ab60 0x55fd304a8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.725+0000 7f1523971740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd3048b480 0x55fd304a8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.725+0000 7f1523971740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd30495d40 0x55fd304a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.725+0000 7f1523971740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1ae1f0 con 0x55fd30495d40 2024-09-11T23:27:32.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.725+0000 7f1523971740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1af660 con 0x55fd3048b480 2024-09-11T23:27:32.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.725+0000 7f1523971740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1ae8c0 con 0x55fd3048ab60 2024-09-11T23:27:32.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.725+0000 7f1523971740 10 client.0 _ll_register_callbacks cb 0x7f1510012470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:32.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.726+0000 7f151b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd30495d40 0x55fd304a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.726+0000 7f151b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd30495d40 0x55fd304a93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40866/0 (socket says 172.21.15.43:40866) 2024-09-11T23:27:32.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.726+0000 7f151b7fe640 1 -- 172.21.15.43:0/2134336264 learned_addr learned my addr 172.21.15.43:0/2134336264 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:32.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.726+0000 7f151a7fc640 1 --2- 172.21.15.43:0/2134336264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd3048b480 0x55fd304a8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.726+0000 7f151affd640 1 --2- 172.21.15.43:0/2134336264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd3048ab60 0x55fd304a8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.726+0000 7f151b7fe640 1 -- 172.21.15.43:0/2134336264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd3048ab60 msgr2=0x55fd304a8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:32.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.726+0000 7f151b7fe640 1 --2- 172.21.15.43:0/2134336264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd3048ab60 0x55fd304a8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.726+0000 7f151b7fe640 1 -- 172.21.15.43:0/2134336264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd3048b480 msgr2=0x55fd304a8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:32.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.726+0000 7f151b7fe640 1 --2- 172.21.15.43:0/2134336264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd3048b480 0x55fd304a8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.726+0000 7f151b7fe640 1 -- 172.21.15.43:0/2134336264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd3050f1b0 con 0x55fd30495d40 2024-09-11T23:27:32.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.726+0000 7f151b7fe640 1 --2- 172.21.15.43:0/2134336264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd30495d40 0x55fd304a93a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1514097c90 tx=0x7f15140ccf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b9257243b080bd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:32.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.726+0000 7f14feff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:27:32.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.727+0000 7f14feff5640 1 -- 172.21.15.43:0/2134336264 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151401f070 con 0x55fd30495d40 2024-09-11T23:27:32.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.727+0000 7f14feff5640 1 -- 172.21.15.43:0/2134336264 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f1514099ff0 con 0x55fd30495d40 2024-09-11T23:27:32.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.727+0000 7f1523971740 1 -- 172.21.15.43:0/2134336264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f150c005510 con 0x55fd30495d40 2024-09-11T23:27:32.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.727+0000 7f14feff5640 1 -- 172.21.15.43:0/2134336264 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1514097f90 con 0x55fd30495d40 2024-09-11T23:27:32.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.727+0000 7f1523971740 10 client.16284 fetch_fsmap learned FSMap version 30 2024-09-11T23:27:32.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.727+0000 7f1523971740 1 -- 172.21.15.43:0/2134336264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fd305103f0 con 0x55fd30495d40 2024-09-11T23:27:32.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.727+0000 7f14feff5640 1 -- 172.21.15.43:0/2134336264 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f150c005510 con 0x55fd30495d40 2024-09-11T23:27:32.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.727+0000 7f14feff5640 1 -- 172.21.15.43:0/2134336264 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f15140b4040 con 0x55fd30495d40 2024-09-11T23:27:32.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.727+0000 7f1523971740 10 client.16284 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:27:32.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.727+0000 7f1523971740 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:27:32.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.727+0000 7f1523971740 20 client.16284 populate_metadata read hostname 'smithi043' 2024-09-11T23:27:32.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.727+0000 7f1523971740 1 --2- 172.21.15.43:0/2134336264 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55fd30510870 0x55fd30512c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:32.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.727+0000 7f1523971740 4 client.16284 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:27:32.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.727+0000 7f1523971740 1 -- 172.21.15.43:0/2134336264 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fd303fd7f0 con 0x55fd30510870 2024-09-11T23:27:32.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.728+0000 7f151affd640 1 --2- 172.21.15.43:0/2134336264 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55fd30510870 0x55fd30512c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:32.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.728+0000 7f14feff5640 10 client.16284 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:27:32.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.728+0000 7f151affd640 1 --2- 172.21.15.43:0/2134336264 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55fd30510870 0x55fd30512c60 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bbb612106dea6b2 server_cookie=40873a4e59ee0d55 in_seq=0 out_seq=0 2024-09-11T23:27:32.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.730+0000 7f14feff5640 1 -- 172.21.15.43:0/2134336264 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fd303fd7f0 con 0x55fd30510870 2024-09-11T23:27:32.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.730+0000 7f14feff5640 10 client.16284 handle_command_reply: tid=0 2024-09-11T23:27:32.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.730+0000 7f1523971740 1 client.16284 shutdown 2024-09-11T23:27:32.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.730+0000 7f1523971740 1 -- 172.21.15.43:0/2134336264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd30495d40 msgr2=0x55fd304a93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:32.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.730+0000 7f1523971740 1 --2- 172.21.15.43:0/2134336264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd30495d40 0x55fd304a93a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1514097c90 tx=0x7f15140ccf50 comp rx=0 tx=0).stop 2024-09-11T23:27:32.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.730+0000 7f1523971740 1 -- 172.21.15.43:0/2134336264 shutdown_connections 2024-09-11T23:27:32.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.730+0000 7f1523971740 1 --2- 172.21.15.43:0/2134336264 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55fd30510870 0x55fd30512c60 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.730+0000 7f14feff5640 0 client.16284 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:27:32.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.730+0000 7f1523971740 1 --2- 172.21.15.43:0/2134336264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd30495d40 0x55fd304a93a0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.730+0000 7f1523971740 1 --2- 172.21.15.43:0/2134336264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd3048b480 0x55fd304a8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.730+0000 7f1523971740 1 --2- 172.21.15.43:0/2134336264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fd3048ab60 0x55fd304a8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.730+0000 7f1523971740 1 -- 172.21.15.43:0/2134336264 >> 172.21.15.43:0/2134336264 conn(0x55fd304870f0 msgr2=0x55fd30485a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:32.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.731+0000 7f1523971740 1 -- 172.21.15.43:0/2134336264 shutdown_connections 2024-09-11T23:27:32.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.731+0000 7f1523971740 1 -- 172.21.15.43:0/2134336264 wait complete. 2024-09-11T23:27:32.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.731+0000 7f1523971740 20 client.16284 trim_cache size 0 max 16384 2024-09-11T23:27:32.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.731+0000 7f1521598640 1 -- 172.21.15.43:0/3757683977 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f15040a2d70 msgr2=0x7f15040a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:32.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.731+0000 7f1521598640 1 --2- 172.21.15.43:0/3757683977 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f15040a2d70 0x7f15040a5230 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1510005cc0 tx=0x7f151000bf90 comp rx=0 tx=0).stop 2024-09-11T23:27:32.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.731+0000 7f1521598640 1 -- 172.21.15.43:0/3757683977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f151c10e4d0 msgr2=0x7f151c19eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:32.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.731+0000 7f1521598640 1 --2- 172.21.15.43:0/3757683977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f151c10e4d0 0x7f151c19eff0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f151400e940 tx=0x7f151400ee00 comp rx=0 tx=0).stop 2024-09-11T23:27:32.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.731+0000 7f1521598640 1 -- 172.21.15.43:0/3757683977 shutdown_connections 2024-09-11T23:27:32.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.731+0000 7f1521598640 1 --2- 172.21.15.43:0/3757683977 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f15040a2d70 0x7f15040a5230 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.731+0000 7f1521598640 1 --2- 172.21.15.43:0/3757683977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f151c10e4d0 0x7f151c19eff0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.731+0000 7f1521598640 1 --2- 172.21.15.43:0/3757683977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c101360 0x7f151c19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.731+0000 7f1521598640 1 --2- 172.21.15.43:0/3757683977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f151c1009b0 0x7f151c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:32.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.731+0000 7f1521598640 1 -- 172.21.15.43:0/3757683977 >> 172.21.15.43:0/3757683977 conn(0x7f151c0fc7d0 msgr2=0x7f151c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:32.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.732+0000 7f1521598640 1 -- 172.21.15.43:0/3757683977 shutdown_connections 2024-09-11T23:27:32.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:31.732+0000 7f1521598640 1 -- 172.21.15.43:0/3757683977 wait complete. 2024-09-11T23:27:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.877+0000 7f742bfff640 1 -- 172.21.15.43:0/3786949017 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555ea625dd70 con 0x555ea6377e60 2024-09-11T23:27:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.877+0000 7f742bfff640 10 client.25721 handle_command_reply: tid=0 2024-09-11T23:27:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.877+0000 7f7455dcc740 1 client.25721 shutdown 2024-09-11T23:27:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.877+0000 7f7455dcc740 1 -- 172.21.15.43:0/3786949017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62f4560 msgr2=0x555ea6305690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.877+0000 7f7455dcc740 1 --2- 172.21.15.43:0/3786949017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62f4560 0x555ea6305690 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f743805a970 tx=0x7f7438009580 comp rx=0 tx=0).stop 2024-09-11T23:27:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.877+0000 7f7455dcc740 1 -- 172.21.15.43:0/3786949017 shutdown_connections 2024-09-11T23:27:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.877+0000 7f7455dcc740 1 --2- 172.21.15.43:0/3786949017 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x555ea6377e60 0x555ea637a280 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.877+0000 7f7455dcc740 1 --2- 172.21.15.43:0/3786949017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ea62fccb0 0x555ea62f65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.877+0000 7f742bfff640 0 client.25721 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:27:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.877+0000 7f7455dcc740 1 --2- 172.21.15.43:0/3786949017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555ea62f4560 0x555ea6305690 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.877+0000 7f7455dcc740 1 --2- 172.21.15.43:0/3786949017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555ea62f1af0 0x555ea6305150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.877+0000 7f7455dcc740 1 -- 172.21.15.43:0/3786949017 >> 172.21.15.43:0/3786949017 conn(0x555ea62ee060 msgr2=0x555ea62f81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.878+0000 7f7455dcc740 1 -- 172.21.15.43:0/3786949017 shutdown_connections 2024-09-11T23:27:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.878+0000 7f7455dcc740 1 -- 172.21.15.43:0/3786949017 wait complete. 2024-09-11T23:27:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.878+0000 7f7455dcc740 20 client.25721 trim_cache size 0 max 16384 2024-09-11T23:27:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:27:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:27:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:27:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:27:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.878+0000 7f7452f52640 1 -- 172.21.15.43:0/1131830080 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f743c0a2f00 msgr2=0x7f743c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.878+0000 7f7452f52640 1 --2- 172.21.15.43:0/1131830080 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f743c0a2f00 0x7f743c0a53c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7438004650 tx=0x7f7438005b20 comp rx=0 tx=0).stop 2024-09-11T23:27:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.878+0000 7f7452f52640 1 -- 172.21.15.43:0/1131830080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744c1091f0 msgr2=0x7f744c1a72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.878+0000 7f7452f52640 1 --2- 172.21.15.43:0/1131830080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744c1091f0 0x7f744c1a72e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f744000d950 tx=0x7f744000de10 comp rx=0 tx=0).stop 2024-09-11T23:27:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.878+0000 7f7452f52640 1 -- 172.21.15.43:0/1131830080 shutdown_connections 2024-09-11T23:27:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.878+0000 7f7452f52640 1 --2- 172.21.15.43:0/1131830080 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f743c0a2f00 0x7f743c0a53c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.878+0000 7f7452f52640 1 --2- 172.21.15.43:0/1131830080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f744c11b140 0x7f744c1abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.878+0000 7f7452f52640 1 --2- 172.21.15.43:0/1131830080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f744c1095d0 0x7f744c1a7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.878+0000 7f7452f52640 1 --2- 172.21.15.43:0/1131830080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744c1091f0 0x7f744c1a72e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.878+0000 7f7452f52640 1 -- 172.21.15.43:0/1131830080 >> 172.21.15.43:0/1131830080 conn(0x7f744c06cdf0 msgr2=0x7f744c11b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.878+0000 7f7452f52640 1 -- 172.21.15.43:0/1131830080 shutdown_connections 2024-09-11T23:27:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:33.878+0000 7f7452f52640 1 -- 172.21.15.43:0/1131830080 wait complete. 2024-09-11T23:27:34.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:33 smithi090 ceph-mon[35130]: pgmap v175: 97 pgs: 97 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 4.45k op/s 2024-09-11T23:27:34.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:33 smithi097 ceph-mon[35562]: pgmap v175: 97 pgs: 97 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 4.45k op/s 2024-09-11T23:27:34.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:33 smithi043 ceph-mon[30950]: pgmap v175: 97 pgs: 97 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 4.45k op/s 2024-09-11T23:27:36.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:35 smithi043 ceph-mon[30950]: pgmap v176: 97 pgs: 97 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 MiB/s wr, 5.42k op/s 2024-09-11T23:27:36.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:35 smithi090 ceph-mon[35130]: pgmap v176: 97 pgs: 97 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 MiB/s wr, 5.42k op/s 2024-09-11T23:27:36.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:35 smithi097 ceph-mon[35562]: pgmap v176: 97 pgs: 97 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 MiB/s wr, 5.42k op/s 2024-09-11T23:27:37.334 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:27:37.410 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:27:38.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.066+0000 7f47a27fc640 1 -- 172.21.15.43:0/2106734829 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798040880 con 0x7f47a4102ef0 2024-09-11T23:27:38.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.069+0000 7f47aa190640 1 -- 172.21.15.43:0/2106734829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a4102ef0 msgr2=0x7f47a410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:38.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.069+0000 7f47aa190640 1 --2- 172.21.15.43:0/2106734829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a4102ef0 0x7f47a410da40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f479801a910 tx=0x7f479803b4e0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.077+0000 7f47aa190640 1 -- 172.21.15.43:0/2106734829 shutdown_connections 2024-09-11T23:27:38.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.077+0000 7f47aa190640 1 --2- 172.21.15.43:0/2106734829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a4102ef0 0x7f47a410da40 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.077+0000 7f47aa190640 1 --2- 172.21.15.43:0/2106734829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47a4102570 0x7f47a41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.077+0000 7f47aa190640 1 --2- 172.21.15.43:0/2106734829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a4100e20 0x7f47a4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.077+0000 7f47aa190640 1 -- 172.21.15.43:0/2106734829 >> 172.21.15.43:0/2106734829 conn(0x7f47a40fc7d0 msgr2=0x7f47a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:38.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.079+0000 7f47aa190640 1 -- 172.21.15.43:0/2106734829 shutdown_connections 2024-09-11T23:27:38.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.080+0000 7f47aa190640 1 -- 172.21.15.43:0/2106734829 wait complete. 2024-09-11T23:27:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.080+0000 7f47aa190640 1 Processor -- start 2024-09-11T23:27:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.081+0000 7f47aa190640 1 -- start start 2024-09-11T23:27:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.081+0000 7f47aa190640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47a4100e20 0x7f47a4193340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.081+0000 7f47aa190640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a4102570 0x7f47a4193880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.081+0000 7f47aa190640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a4102ef0 0x7f47a4193dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.081+0000 7f47aa190640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f47a406e670 con 0x7f47a4102ef0 2024-09-11T23:27:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.081+0000 7f47aa190640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f47a406e4f0 con 0x7f47a4100e20 2024-09-11T23:27:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.081+0000 7f47aa190640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f47a406e7f0 con 0x7f47a4102570 2024-09-11T23:27:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.082+0000 7f47a37fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47a4100e20 0x7f47a4193340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.082+0000 7f47a37fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47a4100e20 0x7f47a4193340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:36786/0 (socket says 172.21.15.43:36786) 2024-09-11T23:27:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.082+0000 7f47a37fe640 1 -- 172.21.15.43:0/3581076560 learned_addr learned my addr 172.21.15.43:0/3581076560 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.082+0000 7f47a2ffd640 1 --2- 172.21.15.43:0/3581076560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a4102570 0x7f47a4193880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.083+0000 7f47a2ffd640 1 -- 172.21.15.43:0/3581076560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47a4100e20 msgr2=0x7f47a4193340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.083+0000 7f47a2ffd640 1 --2- 172.21.15.43:0/3581076560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47a4100e20 0x7f47a4193340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.083+0000 7f47a2ffd640 1 -- 172.21.15.43:0/3581076560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a4102ef0 msgr2=0x7f47a4193dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:38.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.083+0000 7f47a2ffd640 1 --2- 172.21.15.43:0/3581076560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a4102ef0 0x7f47a4193dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.083+0000 7f47a2ffd640 1 -- 172.21.15.43:0/3581076560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a410c6a0 con 0x7f47a4102570 2024-09-11T23:27:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.085+0000 7f47a2ffd640 1 --2- 172.21.15.43:0/3581076560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a4102570 0x7f47a4193880 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4794009fd0 tx=0x7f479400b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.086+0000 7f47a0ff9640 1 -- 172.21.15.43:0/3581076560 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4794015020 con 0x7f47a4102570 2024-09-11T23:27:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.086+0000 7f47aa190640 1 -- 172.21.15.43:0/3581076560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a410c990 con 0x7f47a4102570 2024-09-11T23:27:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.086+0000 7f47aa190640 1 -- 172.21.15.43:0/3581076560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a410cf50 con 0x7f47a4102570 2024-09-11T23:27:38.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.087+0000 7f47a0ff9640 1 -- 172.21.15.43:0/3581076560 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4794011d10 con 0x7f47a4102570 2024-09-11T23:27:38.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.087+0000 7f47a0ff9640 1 -- 172.21.15.43:0/3581076560 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479400f2e0 con 0x7f47a4102570 2024-09-11T23:27:38.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.092+0000 7f47a0ff9640 1 -- 172.21.15.43:0/3581076560 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4794018020 con 0x7f47a4102570 2024-09-11T23:27:38.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.094+0000 7f47a0ff9640 1 --2- 172.21.15.43:0/3581076560 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f478c0a2dc0 0x7f478c0a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.094+0000 7f47a0ff9640 1 -- 172.21.15.43:0/3581076560 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f47940cea60 con 0x7f47a4102570 2024-09-11T23:27:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.094+0000 7f47a37fe640 1 --2- 172.21.15.43:0/3581076560 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f478c0a2dc0 0x7f478c0a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.095+0000 7f47a37fe640 1 --2- 172.21.15.43:0/3581076560 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f478c0a2dc0 0x7f478c0a5280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f479c00e250 tx=0x7f479c00b040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:38.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.101+0000 7f47aa190640 1 -- 172.21.15.43:0/3581076560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4790005290 con 0x7f47a4102570 2024-09-11T23:27:38.119 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:27:38.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.110+0000 7f47a0ff9640 1 -- 172.21.15.43:0/3581076560 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f479401f020 con 0x7f47a4102570 2024-09-11T23:27:38.261 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:37 smithi043 ceph-mon[30950]: pgmap v177: 97 pgs: 97 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 4.14k op/s 2024-09-11T23:27:38.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:37 smithi090 ceph-mon[35130]: pgmap v177: 97 pgs: 97 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 4.14k op/s 2024-09-11T23:27:38.300 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:37 smithi097 ceph-mon[35562]: pgmap v177: 97 pgs: 97 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 4.14k op/s 2024-09-11T23:27:38.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.557+0000 7f47aa190640 1 -- 172.21.15.43:0/3581076560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4790005850 con 0x7f47a4102570 2024-09-11T23:27:38.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.568+0000 7f47a0ff9640 1 -- 172.21.15.43:0/3581076560 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11893 (secure 0 0 0) 0x7f479409b730 con 0x7f47a4102570 2024-09-11T23:27:38.570 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:27:38.570 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"btime":"2024-09-11T23:25:40:446496+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:25:39.727562+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25571":{"gid":25571,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/568491708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":568491708},{"type":"v1","addr":"172.21.15.90:6835","nonce":568491708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:27:38.570 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-09-11T23:27:38.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.575+0000 7f47aa190640 1 -- 172.21.15.43:0/3581076560 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f478c0a2dc0 msgr2=0x7f478c0a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:38.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.575+0000 7f47aa190640 1 --2- 172.21.15.43:0/3581076560 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f478c0a2dc0 0x7f478c0a5280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f479c00e250 tx=0x7f479c00b040 comp rx=0 tx=0).stop 2024-09-11T23:27:38.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.575+0000 7f47aa190640 1 -- 172.21.15.43:0/3581076560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a4102570 msgr2=0x7f47a4193880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:38.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.575+0000 7f47aa190640 1 --2- 172.21.15.43:0/3581076560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a4102570 0x7f47a4193880 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4794009fd0 tx=0x7f479400b870 comp rx=0 tx=0).stop 2024-09-11T23:27:38.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.575+0000 7f47aa190640 1 -- 172.21.15.43:0/3581076560 shutdown_connections 2024-09-11T23:27:38.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.575+0000 7f47aa190640 1 --2- 172.21.15.43:0/3581076560 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f478c0a2dc0 0x7f478c0a5280 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.575+0000 7f47aa190640 1 --2- 172.21.15.43:0/3581076560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47a4102ef0 0x7f47a4193dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.576+0000 7f47aa190640 1 --2- 172.21.15.43:0/3581076560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a4102570 0x7f47a4193880 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.576+0000 7f47aa190640 1 --2- 172.21.15.43:0/3581076560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47a4100e20 0x7f47a4193340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.576+0000 7f47aa190640 1 -- 172.21.15.43:0/3581076560 >> 172.21.15.43:0/3581076560 conn(0x7f47a40fc7d0 msgr2=0x7f47a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:38.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.579+0000 7f47aa190640 1 -- 172.21.15.43:0/3581076560 shutdown_connections 2024-09-11T23:27:38.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.579+0000 7f47aa190640 1 -- 172.21.15.43:0/3581076560 wait complete. 2024-09-11T23:27:38.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.864+0000 7f9784a27640 1 -- 172.21.15.43:0/1903354122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9780101270 msgr2=0x7f978010d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:38.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.864+0000 7f9784a27640 1 --2- 172.21.15.43:0/1903354122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9780101270 0x7f978010d7a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9778009cd0 tx=0x7f977802a8f0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.864+0000 7f9784a27640 1 -- 172.21.15.43:0/1903354122 shutdown_connections 2024-09-11T23:27:38.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.864+0000 7f9784a27640 1 --2- 172.21.15.43:0/1903354122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9780101270 0x7f978010d7a0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.864+0000 7f9784a27640 1 --2- 172.21.15.43:0/1903354122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97801008d0 0x7f9780100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.864+0000 7f9784a27640 1 --2- 172.21.15.43:0/1903354122 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97801068d0 0x7f9780106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.864+0000 7f9784a27640 1 -- 172.21.15.43:0/1903354122 >> 172.21.15.43:0/1903354122 conn(0x7f97800fc610 msgr2=0x7f97800fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:38.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.864+0000 7f9784a27640 1 -- 172.21.15.43:0/1903354122 shutdown_connections 2024-09-11T23:27:38.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.864+0000 7f9784a27640 1 -- 172.21.15.43:0/1903354122 wait complete. 2024-09-11T23:27:38.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.865+0000 7f9784a27640 1 Processor -- start 2024-09-11T23:27:38.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.865+0000 7f9784a27640 1 -- start start 2024-09-11T23:27:38.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.865+0000 7f9784a27640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97801008d0 0x7f9780073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.865+0000 7f9784a27640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97801068d0 0x7f9780073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.865+0000 7f9784a27640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f978006c710 0x7f978006cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.865+0000 7f9784a27640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f978010fcc0 con 0x7f97801068d0 2024-09-11T23:27:38.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.865+0000 7f9784a27640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f978010fb40 con 0x7f97801008d0 2024-09-11T23:27:38.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.865+0000 7f9784a27640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f978010fe40 con 0x7f978006c710 2024-09-11T23:27:38.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.866+0000 7f977effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97801068d0 0x7f9780073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.866+0000 7f977effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97801068d0 0x7f9780073bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40918/0 (socket says 172.21.15.43:40918) 2024-09-11T23:27:38.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.866+0000 7f977f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97801008d0 0x7f9780073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.866+0000 7f977f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97801008d0 0x7f9780073680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:36808/0 (socket says 172.21.15.43:36808) 2024-09-11T23:27:38.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.866+0000 7f977f7fe640 1 -- 172.21.15.43:0/2349011973 learned_addr learned my addr 172.21.15.43:0/2349011973 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:38.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.867+0000 7f977f7fe640 1 -- 172.21.15.43:0/2349011973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f978006c710 msgr2=0x7f978006cbc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:27:38.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.867+0000 7f977f7fe640 1 --2- 172.21.15.43:0/2349011973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f978006c710 0x7f978006cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.867+0000 7f977f7fe640 1 -- 172.21.15.43:0/2349011973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97801068d0 msgr2=0x7f9780073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:38.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.867+0000 7f977f7fe640 1 --2- 172.21.15.43:0/2349011973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97801068d0 0x7f9780073bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.867+0000 7f977f7fe640 1 -- 172.21.15.43:0/2349011973 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978006d340 con 0x7f97801008d0 2024-09-11T23:27:38.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.867+0000 7f977effd640 1 --2- 172.21.15.43:0/2349011973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97801068d0 0x7f9780073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:27:38.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.869+0000 7f977f7fe640 1 --2- 172.21.15.43:0/2349011973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97801008d0 0x7f9780073680 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f977000b4a0 tx=0x7f977000b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:38.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.871+0000 7f977cff9640 1 -- 172.21.15.43:0/2349011973 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770014070 con 0x7f97801008d0 2024-09-11T23:27:38.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.871+0000 7f977cff9640 1 -- 172.21.15.43:0/2349011973 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9770010af0 con 0x7f97801008d0 2024-09-11T23:27:38.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.871+0000 7f977cff9640 1 -- 172.21.15.43:0/2349011973 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770017c40 con 0x7f97801008d0 2024-09-11T23:27:38.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.871+0000 7f9784a27640 1 -- 172.21.15.43:0/2349011973 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97800711b0 con 0x7f97801008d0 2024-09-11T23:27:38.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.871+0000 7f9784a27640 1 -- 172.21.15.43:0/2349011973 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97800716a0 con 0x7f97801008d0 2024-09-11T23:27:38.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.876+0000 7f9788062740 1 Processor -- start 2024-09-11T23:27:38.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.877+0000 7f9788062740 1 -- start start 2024-09-11T23:27:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.878+0000 7f9788062740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e60670 0x560f23e60a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.878+0000 7f9788062740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60f90 0x560f23e6b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.878+0000 7f9788062740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e6b850 0x560f23e6dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.878+0000 7f9788062740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560f23dc62e0 con 0x560f23e60670 2024-09-11T23:27:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.878+0000 7f9788062740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560f23d6dce0 con 0x560f23e6b850 2024-09-11T23:27:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.878+0000 7f9788062740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560f23dc22a0 con 0x560f23e60f90 2024-09-11T23:27:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.879+0000 7f977effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e6b850 0x560f23e6dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.879+0000 7f977f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60f90 0x560f23e6b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.879+0000 7f977f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60f90 0x560f23e6b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:48726/0 (socket says 172.21.15.43:48726) 2024-09-11T23:27:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.879+0000 7f977f7fe640 1 -- 172.21.15.43:0/740551432 learned_addr learned my addr 172.21.15.43:0/740551432 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.879+0000 7f977cff9640 1 -- 172.21.15.43:0/2349011973 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9770017070 con 0x7f97801008d0 2024-09-11T23:27:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.879+0000 7f977ffff640 1 --2- 172.21.15.43:0/740551432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e60670 0x560f23e60a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.880+0000 7f977f7fe640 1 -- 172.21.15.43:0/740551432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e6b850 msgr2=0x560f23e6dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.880+0000 7f977f7fe640 1 --2- 172.21.15.43:0/740551432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e6b850 0x560f23e6dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.880+0000 7f977f7fe640 1 -- 172.21.15.43:0/740551432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e60670 msgr2=0x560f23e60a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.880+0000 7f977f7fe640 1 --2- 172.21.15.43:0/740551432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e60670 0x560f23e60a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.880+0000 7f977f7fe640 1 -- 172.21.15.43:0/740551432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f23d52440 con 0x560f23e60f90 2024-09-11T23:27:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.881+0000 7f977cff9640 1 --2- 172.21.15.43:0/2349011973 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f97680a2f00 0x7f97680a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.881+0000 7f977cff9640 1 -- 172.21.15.43:0/2349011973 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f97700ce5f0 con 0x7f97801008d0 2024-09-11T23:27:38.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.882+0000 7f977ffff640 1 --2- 172.21.15.43:0/2349011973 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f97680a2f00 0x7f97680a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.882+0000 7f977f7fe640 1 --2- 172.21.15.43:0/740551432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60f90 0x560f23e6b310 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f97700cbd30 tx=0x7f97700d62a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18dde102ba47ef07 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:38.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.883+0000 7f9765ffb640 1 -- 172.21.15.43:0/740551432 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977001e070 con 0x560f23e60f90 2024-09-11T23:27:38.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.883+0000 7f9765ffb640 1 -- 172.21.15.43:0/740551432 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f977009b2c0 con 0x560f23e60f90 2024-09-11T23:27:38.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.883+0000 7f9765ffb640 1 -- 172.21.15.43:0/740551432 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770009230 con 0x560f23e60f90 2024-09-11T23:27:38.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.883+0000 7f977ffff640 1 --2- 172.21.15.43:0/2349011973 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f97680a2f00 0x7f97680a53c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f977802ae60 tx=0x7f977802adf0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:38.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.885+0000 7f9788062740 1 -- 172.21.15.43:0/740551432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60f90 msgr2=0x560f23e6b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:38.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.885+0000 7f9788062740 1 --2- 172.21.15.43:0/740551432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60f90 0x560f23e6b310 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f97700cbd30 tx=0x7f97700d62a0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.885+0000 7f9788062740 1 -- 172.21.15.43:0/740551432 shutdown_connections 2024-09-11T23:27:38.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.885+0000 7f9788062740 1 --2- 172.21.15.43:0/740551432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e6b850 0x560f23e6dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.885+0000 7f9788062740 1 --2- 172.21.15.43:0/740551432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60f90 0x560f23e6b310 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.885+0000 7f9788062740 1 --2- 172.21.15.43:0/740551432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e60670 0x560f23e60a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.885+0000 7f9788062740 1 -- 172.21.15.43:0/740551432 >> 172.21.15.43:0/740551432 conn(0x560f23e5cc00 msgr2=0x560f23e5cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:38.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.889+0000 7f9788062740 1 -- 172.21.15.43:0/740551432 shutdown_connections 2024-09-11T23:27:38.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.889+0000 7f9788062740 1 -- 172.21.15.43:0/740551432 wait complete. 2024-09-11T23:27:38.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.890+0000 7f9788062740 1 -- start start 2024-09-11T23:27:38.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.890+0000 7f9788062740 1 Processor -- start 2024-09-11T23:27:38.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.893+0000 7f9788062740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e60670 0x560f23e6b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.893+0000 7f9788062740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60f90 0x560f23e68430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.893+0000 7f9788062740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e6b850 0x560f23e68970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.893+0000 7f9788062740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560f23e6e420 con 0x560f23e6b850 2024-09-11T23:27:38.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.893+0000 7f9788062740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560f23d81050 con 0x560f23e60670 2024-09-11T23:27:38.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.893+0000 7f9788062740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560f23e6e5a0 con 0x560f23e60f90 2024-09-11T23:27:38.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.893+0000 7f9788062740 10 client.0 _ll_register_callbacks cb 0x560f23dc1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:38.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.894+0000 7f977effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60f90 0x560f23e68430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.894+0000 7f977f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e60670 0x560f23e6b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.894+0000 7f977f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e60670 0x560f23e6b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:36824/0 (socket says 172.21.15.43:36824) 2024-09-11T23:27:38.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.894+0000 7f977f7fe640 1 -- 172.21.15.43:0/1422260821 learned_addr learned my addr 172.21.15.43:0/1422260821 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:38.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.895+0000 7f977ffff640 1 --2- 172.21.15.43:0/1422260821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e6b850 0x560f23e68970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.896+0000 7f977f7fe640 1 -- 172.21.15.43:0/1422260821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60f90 msgr2=0x560f23e68430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:38.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.896+0000 7f977f7fe640 1 --2- 172.21.15.43:0/1422260821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60f90 0x560f23e68430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.896+0000 7f977f7fe640 1 -- 172.21.15.43:0/1422260821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e6b850 msgr2=0x560f23e68970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:38.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.896+0000 7f977f7fe640 1 --2- 172.21.15.43:0/1422260821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e6b850 0x560f23e68970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.896+0000 7f977f7fe640 1 -- 172.21.15.43:0/1422260821 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f23b373c0 con 0x560f23e60670 2024-09-11T23:27:38.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.897+0000 7f977effd640 1 --2- 172.21.15.43:0/1422260821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60f90 0x560f23e68430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:27:38.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.897+0000 7f977f7fe640 1 --2- 172.21.15.43:0/1422260821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e60670 0x560f23e6b180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9770017770 tx=0x7f97700ce8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7ba3db6096fe68 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:38.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.897+0000 7f975effd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:27:38.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.897+0000 7f975effd640 1 -- 172.21.15.43:0/1422260821 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977001e030 con 0x560f23e60670 2024-09-11T23:27:38.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.897+0000 7f9788062740 1 -- 172.21.15.43:0/1422260821 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f977009b2c0 con 0x560f23e60670 2024-09-11T23:27:38.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.897+0000 7f975effd640 1 -- 172.21.15.43:0/1422260821 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f9770010ca0 con 0x560f23e60670 2024-09-11T23:27:38.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.898+0000 7f975effd640 1 -- 172.21.15.43:0/1422260821 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9770009230 con 0x560f23e60670 2024-09-11T23:27:38.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.898+0000 7f9788062740 10 client.25820 fetch_fsmap learned FSMap version 30 2024-09-11T23:27:38.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.898+0000 7f9788062740 1 -- 172.21.15.43:0/1422260821 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560f23e79e60 con 0x560f23e60670 2024-09-11T23:27:38.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.898+0000 7f975effd640 1 -- 172.21.15.43:0/1422260821 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f977009b2c0 con 0x560f23e60670 2024-09-11T23:27:38.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.899+0000 7f975effd640 1 -- 172.21.15.43:0/1422260821 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9770017360 con 0x560f23e60670 2024-09-11T23:27:38.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.899+0000 7f9788062740 10 client.25820 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:27:38.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.899+0000 7f9788062740 10 client.25820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:27:38.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.899+0000 7f9788062740 20 client.25820 populate_metadata read hostname 'smithi043' 2024-09-11T23:27:38.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.899+0000 7f9788062740 1 --2- 172.21.15.43:0/1422260821 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560f23e7b480 0x560f23e7d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.899+0000 7f9788062740 4 client.25820 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:27:38.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.899+0000 7f9788062740 1 -- 172.21.15.43:0/1422260821 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f23dd3850 con 0x560f23e7b480 2024-09-11T23:27:38.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.902+0000 7f977effd640 1 --2- 172.21.15.43:0/1422260821 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560f23e7b480 0x560f23e7d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.906+0000 7f977effd640 1 --2- 172.21.15.43:0/1422260821 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560f23e7b480 0x560f23e7d8e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f725a9d836d0b1bb server_cookie=9369e4784c881716 in_seq=0 out_seq=0 2024-09-11T23:27:38.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.906+0000 7f975effd640 10 client.25820 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:27:38.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.907+0000 7f975effd640 1 -- 172.21.15.43:0/1422260821 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f23dd3850 con 0x560f23e7b480 2024-09-11T23:27:38.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.907+0000 7f975effd640 10 client.25820 handle_command_reply: tid=0 2024-09-11T23:27:38.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.907+0000 7f9788062740 1 client.25820 shutdown 2024-09-11T23:27:38.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.910+0000 7f9788062740 1 -- 172.21.15.43:0/1422260821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e60670 msgr2=0x560f23e6b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:38.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.910+0000 7f9788062740 1 --2- 172.21.15.43:0/1422260821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e60670 0x560f23e6b180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9770017770 tx=0x7f97700ce8d0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.910+0000 7f9788062740 1 -- 172.21.15.43:0/1422260821 shutdown_connections 2024-09-11T23:27:38.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.910+0000 7f9788062740 1 --2- 172.21.15.43:0/1422260821 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560f23e7b480 0x560f23e7d8e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.910+0000 7f9788062740 1 --2- 172.21.15.43:0/1422260821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e6b850 0x560f23e68970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.910+0000 7f9788062740 1 --2- 172.21.15.43:0/1422260821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60f90 0x560f23e68430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.910+0000 7f9788062740 1 --2- 172.21.15.43:0/1422260821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e60670 0x560f23e6b180 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.910+0000 7f9788062740 1 -- 172.21.15.43:0/1422260821 >> 172.21.15.43:0/1422260821 conn(0x560f23e5cc00 msgr2=0x560f23e62930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:38.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.912+0000 7f975effd640 0 client.25820 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:27:38.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.913+0000 7f9788062740 1 -- 172.21.15.43:0/1422260821 shutdown_connections 2024-09-11T23:27:38.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.913+0000 7f9788062740 1 -- 172.21.15.43:0/1422260821 wait complete. 2024-09-11T23:27:38.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.913+0000 7f9788062740 20 client.25820 trim_cache size 0 max 16384 2024-09-11T23:27:38.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.951+0000 7f9788062740 1 Processor -- start 2024-09-11T23:27:38.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.952+0000 7f9788062740 1 -- start start 2024-09-11T23:27:38.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.952+0000 7f9788062740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60670 0x560f23e6a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.952+0000 7f9788062740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e60f90 0x560f23e831f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.952+0000 7f9788062740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e6b850 0x560f23e83730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.952+0000 7f9788062740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f976c010fc0 con 0x560f23e6b850 2024-09-11T23:27:38.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.952+0000 7f9788062740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f976c00e890 con 0x560f23e60f90 2024-09-11T23:27:38.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.952+0000 7f9788062740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f976c00c790 con 0x560f23e60670 2024-09-11T23:27:38.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.953+0000 7f977ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e6b850 0x560f23e83730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.953+0000 7f977ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e6b850 0x560f23e83730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40950/0 (socket says 172.21.15.43:40950) 2024-09-11T23:27:38.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.953+0000 7f977ffff640 1 -- 172.21.15.43:0/830551584 learned_addr learned my addr 172.21.15.43:0/830551584 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:38.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.953+0000 7f977effd640 1 --2- 172.21.15.43:0/830551584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e60f90 0x560f23e831f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.953+0000 7f977f7fe640 1 --2- 172.21.15.43:0/830551584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60670 0x560f23e6a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.954+0000 7f977effd640 1 -- 172.21.15.43:0/830551584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60670 msgr2=0x560f23e6a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:38.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.954+0000 7f977effd640 1 --2- 172.21.15.43:0/830551584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60670 0x560f23e6a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.954+0000 7f977effd640 1 -- 172.21.15.43:0/830551584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e6b850 msgr2=0x560f23e83730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:38.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.954+0000 7f977effd640 1 --2- 172.21.15.43:0/830551584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e6b850 0x560f23e83730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.954+0000 7f977effd640 1 -- 172.21.15.43:0/830551584 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f23e5ca30 con 0x560f23e60f90 2024-09-11T23:27:38.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.956+0000 7f977effd640 1 --2- 172.21.15.43:0/830551584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e60f90 0x560f23e831f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x560f23e69a70 tx=0x7f9774009a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1653ac4745901fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:38.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.956+0000 7f97657fa640 1 -- 172.21.15.43:0/830551584 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977400f040 con 0x560f23e60f90 2024-09-11T23:27:38.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.956+0000 7f97657fa640 1 -- 172.21.15.43:0/830551584 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9774024910 con 0x560f23e60f90 2024-09-11T23:27:38.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.956+0000 7f97657fa640 1 -- 172.21.15.43:0/830551584 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774014750 con 0x560f23e60f90 2024-09-11T23:27:38.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.958+0000 7f9788062740 1 -- 172.21.15.43:0/830551584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e60f90 msgr2=0x560f23e831f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:38.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.958+0000 7f9788062740 1 --2- 172.21.15.43:0/830551584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e60f90 0x560f23e831f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x560f23e69a70 tx=0x7f9774009a40 comp rx=0 tx=0).stop 2024-09-11T23:27:38.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.958+0000 7f9788062740 1 -- 172.21.15.43:0/830551584 shutdown_connections 2024-09-11T23:27:38.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.958+0000 7f9788062740 1 --2- 172.21.15.43:0/830551584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e6b850 0x560f23e83730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.958+0000 7f9788062740 1 --2- 172.21.15.43:0/830551584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e60f90 0x560f23e831f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.958+0000 7f9788062740 1 --2- 172.21.15.43:0/830551584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60670 0x560f23e6a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.958+0000 7f9788062740 1 -- 172.21.15.43:0/830551584 >> 172.21.15.43:0/830551584 conn(0x560f23e5cc00 msgr2=0x560f23e69c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:38.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.959+0000 7f9788062740 1 -- 172.21.15.43:0/830551584 shutdown_connections 2024-09-11T23:27:38.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.960+0000 7f9788062740 1 -- 172.21.15.43:0/830551584 wait complete. 2024-09-11T23:27:38.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.960+0000 7f9788062740 1 -- start start 2024-09-11T23:27:38.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.961+0000 7f9788062740 1 Processor -- start 2024-09-11T23:27:38.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.961+0000 7f9788062740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e60670 0x560f23e70650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.961+0000 7f9788062740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60f90 0x560f23e70b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.961+0000 7f9788062740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e6b850 0x560f23e710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.961+0000 7f9788062740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f976c00cd50 con 0x560f23e6b850 2024-09-11T23:27:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.961+0000 7f9788062740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f976c00e1c0 con 0x560f23e60670 2024-09-11T23:27:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.961+0000 7f9788062740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f976c00d420 con 0x560f23e60f90 2024-09-11T23:27:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.961+0000 7f9788062740 10 client.0 _ll_register_callbacks cb 0x7f97700d49b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.962+0000 7f977ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e6b850 0x560f23e710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.962+0000 7f977ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e6b850 0x560f23e710d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40954/0 (socket says 172.21.15.43:40954) 2024-09-11T23:27:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.962+0000 7f977ffff640 1 -- 172.21.15.43:0/3638445633 learned_addr learned my addr 172.21.15.43:0/3638445633 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.962+0000 7f977f7fe640 1 --2- 172.21.15.43:0/3638445633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e60670 0x560f23e70650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.966+0000 7f977effd640 1 --2- 172.21.15.43:0/3638445633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60f90 0x560f23e70b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.967+0000 7f977ffff640 1 -- 172.21.15.43:0/3638445633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60f90 msgr2=0x560f23e70b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.967+0000 7f977ffff640 1 --2- 172.21.15.43:0/3638445633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60f90 0x560f23e70b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.967+0000 7f977ffff640 1 -- 172.21.15.43:0/3638445633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e60670 msgr2=0x560f23e70650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:38.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.967+0000 7f977ffff640 1 --2- 172.21.15.43:0/3638445633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e60670 0x560f23e70650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.967+0000 7f977ffff640 1 -- 172.21.15.43:0/3638445633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f23ee41f0 con 0x560f23e6b850 2024-09-11T23:27:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.967+0000 7f977ffff640 1 --2- 172.21.15.43:0/3638445633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e6b850 0x560f23e710d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f97780097b0 tx=0x7f977802a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebc74ee3252fb58 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.968+0000 7f975effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:27:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.968+0000 7f975effd640 1 -- 172.21.15.43:0/3638445633 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778055060 con 0x560f23e6b850 2024-09-11T23:27:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.968+0000 7f975effd640 1 -- 172.21.15.43:0/3638445633 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f9778031b40 con 0x560f23e6b850 2024-09-11T23:27:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.968+0000 7f9788062740 1 -- 172.21.15.43:0/3638445633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9774024910 con 0x560f23e6b850 2024-09-11T23:27:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.969+0000 7f975effd640 1 -- 172.21.15.43:0/3638445633 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9778031d40 con 0x560f23e6b850 2024-09-11T23:27:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.975+0000 7f9788062740 10 client.16305 fetch_fsmap learned FSMap version 30 2024-09-11T23:27:38.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.975+0000 7f9788062740 1 -- 172.21.15.43:0/3638445633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560f23ee54c0 con 0x560f23e6b850 2024-09-11T23:27:38.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.979+0000 7f975effd640 1 -- 172.21.15.43:0/3638445633 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9774024910 con 0x560f23e6b850 2024-09-11T23:27:38.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.979+0000 7f975effd640 1 -- 172.21.15.43:0/3638445633 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f977804f030 con 0x560f23e6b850 2024-09-11T23:27:38.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.979+0000 7f9788062740 10 client.16305 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:27:38.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.979+0000 7f9788062740 10 client.16305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:27:38.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.979+0000 7f9788062740 20 client.16305 populate_metadata read hostname 'smithi043' 2024-09-11T23:27:38.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.979+0000 7f9788062740 1 --2- 172.21.15.43:0/3638445633 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560f23ee5bf0 0x560f23ee8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:38.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.979+0000 7f9788062740 4 client.16305 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:27:38.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.979+0000 7f9788062740 1 -- 172.21.15.43:0/3638445633 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560f23dcdd60 con 0x560f23ee5bf0 2024-09-11T23:27:38.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.985+0000 7f977f7fe640 1 --2- 172.21.15.43:0/3638445633 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560f23ee5bf0 0x560f23ee8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:38.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.992+0000 7f977f7fe640 1 --2- 172.21.15.43:0/3638445633 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560f23ee5bf0 0x560f23ee8010 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f85aaa502f603da8 server_cookie=5e4cf0eba187cf8c in_seq=0 out_seq=0 2024-09-11T23:27:38.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.992+0000 7f975effd640 10 client.16305 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:27:38.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.992+0000 7f975effd640 1 -- 172.21.15.43:0/3638445633 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x560f23dcdd60 con 0x560f23ee5bf0 2024-09-11T23:27:38.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.993+0000 7f975effd640 10 client.16305 handle_command_reply: tid=0 2024-09-11T23:27:38.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.994+0000 7f9788062740 1 client.16305 shutdown 2024-09-11T23:27:38.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.995+0000 7f9788062740 1 -- 172.21.15.43:0/3638445633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e6b850 msgr2=0x560f23e710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:38.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.995+0000 7f9788062740 1 --2- 172.21.15.43:0/3638445633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e6b850 0x560f23e710d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f97780097b0 tx=0x7f977802a8f0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.995+0000 7f9788062740 1 -- 172.21.15.43:0/3638445633 shutdown_connections 2024-09-11T23:27:38.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.995+0000 7f9788062740 1 --2- 172.21.15.43:0/3638445633 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560f23ee5bf0 0x560f23ee8010 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.995+0000 7f9788062740 1 --2- 172.21.15.43:0/3638445633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560f23e6b850 0x560f23e710d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.995+0000 7f9788062740 1 --2- 172.21.15.43:0/3638445633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f23e60f90 0x560f23e70b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.995+0000 7f9788062740 1 --2- 172.21.15.43:0/3638445633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560f23e60670 0x560f23e70650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:38.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.995+0000 7f9788062740 1 -- 172.21.15.43:0/3638445633 >> 172.21.15.43:0/3638445633 conn(0x560f23e5cc00 msgr2=0x560f23e668d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:38.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.995+0000 7f975effd640 0 client.16305 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:27:38.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.995+0000 7f9788062740 1 -- 172.21.15.43:0/3638445633 shutdown_connections 2024-09-11T23:27:38.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.995+0000 7f9788062740 1 -- 172.21.15.43:0/3638445633 wait complete. 2024-09-11T23:27:38.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:38.996+0000 7f9788062740 20 client.16305 trim_cache size 0 max 16384 2024-09-11T23:27:38.998 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:27:38.998 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "scrub active (58 inodes in the stack)", 2024-09-11T23:27:38.998 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-09-11T23:27:38.998 INFO:teuthology.orchestra.run.smithi043.stdout: "2010b9bf-0fac-4632-b3c1-aa91af656687": { 2024-09-11T23:27:38.998 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-09-11T23:27:38.998 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "2010b9bf-0fac-4632-b3c1-aa91af656687", 2024-09-11T23:27:38.998 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-09-11T23:27:38.998 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:27:38.998 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:27:38.999 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:27:39.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:39.002+0000 7f9784a27640 1 -- 172.21.15.43:0/2349011973 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f97680a2f00 msgr2=0x7f97680a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:39.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:39.002+0000 7f9784a27640 1 --2- 172.21.15.43:0/2349011973 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f97680a2f00 0x7f97680a53c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f977802ae60 tx=0x7f977802adf0 comp rx=0 tx=0).stop 2024-09-11T23:27:39.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:39.002+0000 7f9784a27640 1 -- 172.21.15.43:0/2349011973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97801008d0 msgr2=0x7f9780073680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:39.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:39.002+0000 7f9784a27640 1 --2- 172.21.15.43:0/2349011973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97801008d0 0x7f9780073680 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f977000b4a0 tx=0x7f977000b960 comp rx=0 tx=0).stop 2024-09-11T23:27:39.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:39.002+0000 7f9784a27640 1 -- 172.21.15.43:0/2349011973 shutdown_connections 2024-09-11T23:27:39.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:39.002+0000 7f9784a27640 1 --2- 172.21.15.43:0/2349011973 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f97680a2f00 0x7f97680a53c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:39.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:39.002+0000 7f9784a27640 1 --2- 172.21.15.43:0/2349011973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f978006c710 0x7f978006cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:39.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:39.002+0000 7f9784a27640 1 --2- 172.21.15.43:0/2349011973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97801068d0 0x7f9780073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:39.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:39.002+0000 7f9784a27640 1 --2- 172.21.15.43:0/2349011973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97801008d0 0x7f9780073680 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:39.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:39.002+0000 7f9784a27640 1 -- 172.21.15.43:0/2349011973 >> 172.21.15.43:0/2349011973 conn(0x7f97800fc610 msgr2=0x7f97800fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:39.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:39.002+0000 7f9784a27640 1 -- 172.21.15.43:0/2349011973 shutdown_connections 2024-09-11T23:27:39.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:39.002+0000 7f9784a27640 1 -- 172.21.15.43:0/2349011973 wait complete. 2024-09-11T23:27:39.108 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:38 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3581076560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:27:39.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:38 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3581076560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:27:39.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:38 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3581076560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:27:39.618 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-11T23:27:39.618 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-11T23:27:40.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:39 smithi043 ceph-mon[30950]: pgmap v178: 97 pgs: 97 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 MiB/s wr, 4.87k op/s 2024-09-11T23:27:40.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:39 smithi090 ceph-mon[35130]: pgmap v178: 97 pgs: 97 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 MiB/s wr, 4.87k op/s 2024-09-11T23:27:40.276 INFO:tasks.cephfs.filesystem:scrub status for tag:2010b9bf-0fac-4632-b3c1-aa91af656687 - {'path': '/', 'tag': '2010b9bf-0fac-4632-b3c1-aa91af656687', 'options': 'recursive,force'} 2024-09-11T23:27:40.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:39 smithi097 ceph-mon[35562]: pgmap v178: 97 pgs: 97 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 MiB/s wr, 4.87k op/s 2024-09-11T23:27:40.340 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:27:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.859+0000 7fe198f75640 1 -- 172.21.15.43:0/3960155155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe194100a80 msgr2=0x7fe194100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.859+0000 7fe198f75640 1 --2- 172.21.15.43:0/3960155155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe194100a80 0x7fe194100ee0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe188009930 tx=0x7fe18802a420 comp rx=0 tx=0).stop 2024-09-11T23:27:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.859+0000 7fe198f75640 1 -- 172.21.15.43:0/3960155155 shutdown_connections 2024-09-11T23:27:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.859+0000 7fe198f75640 1 --2- 172.21.15.43:0/3960155155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe194101420 0x7fe19410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.859+0000 7fe198f75640 1 --2- 172.21.15.43:0/3960155155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe194100a80 0x7fe194100ee0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.859+0000 7fe198f75640 1 --2- 172.21.15.43:0/3960155155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe194106a80 0x7fe194106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.859+0000 7fe198f75640 1 -- 172.21.15.43:0/3960155155 >> 172.21.15.43:0/3960155155 conn(0x7fe1940fc7a0 msgr2=0x7fe1940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.859+0000 7fe198f75640 1 -- 172.21.15.43:0/3960155155 shutdown_connections 2024-09-11T23:27:40.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.859+0000 7fe198f75640 1 -- 172.21.15.43:0/3960155155 wait complete. 2024-09-11T23:27:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.860+0000 7fe198f75640 1 Processor -- start 2024-09-11T23:27:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.860+0000 7fe198f75640 1 -- start start 2024-09-11T23:27:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.860+0000 7fe198f75640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe194100a80 0x7fe1940fcda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.860+0000 7fe198f75640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe194101420 0x7fe1940fd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.860+0000 7fe198f75640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe194106a80 0x7fe1940fd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.860+0000 7fe198f75640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe194110e40 con 0x7fe194106a80 2024-09-11T23:27:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.860+0000 7fe198f75640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe194110cc0 con 0x7fe194101420 2024-09-11T23:27:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.860+0000 7fe198f75640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe194110fc0 con 0x7fe194100a80 2024-09-11T23:27:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.860+0000 7fe192d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe194106a80 0x7fe1940fd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:40.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.860+0000 7fe192d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe194106a80 0x7fe1940fd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56036/0 (socket says 172.21.15.43:56036) 2024-09-11T23:27:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.861+0000 7fe192d9d640 1 -- 172.21.15.43:0/3798539695 learned_addr learned my addr 172.21.15.43:0/3798539695 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.861+0000 7fe191d9b640 1 --2- 172.21.15.43:0/3798539695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe194101420 0x7fe1940fd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.861+0000 7fe19259c640 1 --2- 172.21.15.43:0/3798539695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe194100a80 0x7fe1940fcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.861+0000 7fe192d9d640 1 -- 172.21.15.43:0/3798539695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe194100a80 msgr2=0x7fe1940fcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.861+0000 7fe192d9d640 1 --2- 172.21.15.43:0/3798539695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe194100a80 0x7fe1940fcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.861+0000 7fe192d9d640 1 -- 172.21.15.43:0/3798539695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe194101420 msgr2=0x7fe1940fd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.861+0000 7fe192d9d640 1 --2- 172.21.15.43:0/3798539695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe194101420 0x7fe1940fd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.861+0000 7fe192d9d640 1 -- 172.21.15.43:0/3798539695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1941a68b0 con 0x7fe194106a80 2024-09-11T23:27:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.861+0000 7fe192d9d640 1 --2- 172.21.15.43:0/3798539695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe194106a80 0x7fe1940fd820 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00eaa0 tx=0x7fe18c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.861+0000 7fe1837fe640 1 -- 172.21.15.43:0/3798539695 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c00cdb0 con 0x7fe194106a80 2024-09-11T23:27:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.861+0000 7fe198f75640 1 -- 172.21.15.43:0/3798539695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1941a6ba0 con 0x7fe194106a80 2024-09-11T23:27:40.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.861+0000 7fe1837fe640 1 -- 172.21.15.43:0/3798539695 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe18c005020 con 0x7fe194106a80 2024-09-11T23:27:40.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.861+0000 7fe1837fe640 1 -- 172.21.15.43:0/3798539695 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c0116b0 con 0x7fe194106a80 2024-09-11T23:27:40.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.861+0000 7fe198f75640 1 -- 172.21.15.43:0/3798539695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1941a70b0 con 0x7fe194106a80 2024-09-11T23:27:40.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.864+0000 7fe19b34e740 1 Processor -- start 2024-09-11T23:27:40.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.865+0000 7fe1837fe640 1 -- 172.21.15.43:0/3798539695 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe18c005560 con 0x7fe194106a80 2024-09-11T23:27:40.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.865+0000 7fe19b34e740 1 -- start start 2024-09-11T23:27:40.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.865+0000 7fe19b34e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20ef850 0x55cec20efc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:40.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.865+0000 7fe19b34e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cec20f0170 0x55cec20fa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:40.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.865+0000 7fe19b34e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cec20faa30 0x55cec20fce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:40.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.865+0000 7fe19b34e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cec2054160 con 0x55cec20ef850 2024-09-11T23:27:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.865+0000 7fe19b34e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55cec1ffb720 con 0x55cec20faa30 2024-09-11T23:27:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.865+0000 7fe19b34e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cec2050120 con 0x55cec20f0170 2024-09-11T23:27:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.865+0000 7fe191d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20ef850 0x55cec20efc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.865+0000 7fe191d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20ef850 0x55cec20efc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56042/0 (socket says 172.21.15.43:56042) 2024-09-11T23:27:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.865+0000 7fe191d9b640 1 -- 172.21.15.43:0/211500011 learned_addr learned my addr 172.21.15.43:0/211500011 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.865+0000 7fe19259c640 1 --2- 172.21.15.43:0/211500011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cec20f0170 0x55cec20fa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.865+0000 7fe191d9b640 1 --2- 172.21.15.43:0/211500011 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cec20faa30 0x55cec20fce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.866+0000 7fe191d9b640 1 -- 172.21.15.43:0/211500011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cec20f0170 msgr2=0x55cec20fa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.866+0000 7fe191d9b640 1 --2- 172.21.15.43:0/211500011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cec20f0170 0x55cec20fa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.866+0000 7fe191d9b640 1 -- 172.21.15.43:0/211500011 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cec20faa30 msgr2=0x55cec20fce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.866+0000 7fe191d9b640 1 --2- 172.21.15.43:0/211500011 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cec20faa30 0x55cec20fce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.866+0000 7fe1837fe640 1 --2- 172.21.15.43:0/3798539695 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fe17c0a2cf0 0x7fe17c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.866+0000 7fe191d9b640 1 -- 172.21.15.43:0/211500011 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec1dc5ba0 con 0x55cec20ef850 2024-09-11T23:27:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.866+0000 7fe19259c640 1 --2- 172.21.15.43:0/3798539695 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fe17c0a2cf0 0x7fe17c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:40.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.866+0000 7fe191d9b640 1 --2- 172.21.15.43:0/211500011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20ef850 0x55cec20efc30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe188000c00 tx=0x7fe188005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0ea415ae341770 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:40.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.866+0000 7fe1837fe640 1 -- 172.21.15.43:0/3798539695 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7fe18c0cf610 con 0x7fe194106a80 2024-09-11T23:27:40.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.866+0000 7fe180ff9640 1 -- 172.21.15.43:0/211500011 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188044030 con 0x55cec20ef850 2024-09-11T23:27:40.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.866+0000 7fe180ff9640 1 -- 172.21.15.43:0/211500011 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe188031a20 con 0x55cec20ef850 2024-09-11T23:27:40.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.866+0000 7fe180ff9640 1 -- 172.21.15.43:0/211500011 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188035040 con 0x55cec20ef850 2024-09-11T23:27:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.867+0000 7fe19b34e740 1 -- 172.21.15.43:0/211500011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20ef850 msgr2=0x55cec20efc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.867+0000 7fe19b34e740 1 --2- 172.21.15.43:0/211500011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20ef850 0x55cec20efc30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe188000c00 tx=0x7fe188005fd0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.867+0000 7fe19259c640 1 --2- 172.21.15.43:0/3798539695 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fe17c0a2cf0 0x7fe17c0a51b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55cec20ecc40 tx=0x7fe18400b410 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.867+0000 7fe19b34e740 1 -- 172.21.15.43:0/211500011 shutdown_connections 2024-09-11T23:27:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.867+0000 7fe19b34e740 1 --2- 172.21.15.43:0/211500011 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cec20faa30 0x55cec20fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.867+0000 7fe19b34e740 1 --2- 172.21.15.43:0/211500011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cec20f0170 0x55cec20fa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.867+0000 7fe19b34e740 1 --2- 172.21.15.43:0/211500011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20ef850 0x55cec20efc30 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.867+0000 7fe19b34e740 1 -- 172.21.15.43:0/211500011 >> 172.21.15.43:0/211500011 conn(0x55cec20ebde0 msgr2=0x55cec20ec1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.867+0000 7fe19b34e740 1 -- 172.21.15.43:0/211500011 shutdown_connections 2024-09-11T23:27:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.867+0000 7fe19b34e740 1 -- 172.21.15.43:0/211500011 wait complete. 2024-09-11T23:27:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.868+0000 7fe19b34e740 1 -- start start 2024-09-11T23:27:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.868+0000 7fe19b34e740 1 Processor -- start 2024-09-11T23:27:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.868+0000 7fe19b34e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20ef850 0x55cec20f77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.868+0000 7fe19b34e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cec20f0170 0x55cec20f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.868+0000 7fe19b34e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cec20faa30 0x55cec20f8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.868+0000 7fe19b34e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cec20fd600 con 0x55cec20ef850 2024-09-11T23:27:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.868+0000 7fe19b34e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55cec1fdc900 con 0x55cec20faa30 2024-09-11T23:27:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.868+0000 7fe19b34e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cec20fd780 con 0x55cec20f0170 2024-09-11T23:27:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.868+0000 7fe19b34e740 10 client.0 _ll_register_callbacks cb 0x55cec1fda780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.868+0000 7fe19259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20ef850 0x55cec20f77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.868+0000 7fe19259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20ef850 0x55cec20f77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56046/0 (socket says 172.21.15.43:56046) 2024-09-11T23:27:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.868+0000 7fe19259c640 1 -- 172.21.15.43:0/3579148334 learned_addr learned my addr 172.21.15.43:0/3579148334 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.868+0000 7fe192d9d640 1 --2- 172.21.15.43:0/3579148334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cec20faa30 0x55cec20f8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.868+0000 7fe191d9b640 1 --2- 172.21.15.43:0/3579148334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cec20f0170 0x55cec20f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.868+0000 7fe19259c640 1 -- 172.21.15.43:0/3579148334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cec20f0170 msgr2=0x55cec20f7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.868+0000 7fe19259c640 1 --2- 172.21.15.43:0/3579148334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cec20f0170 0x55cec20f7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.868+0000 7fe19259c640 1 -- 172.21.15.43:0/3579148334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cec20faa30 msgr2=0x55cec20f8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.868+0000 7fe19259c640 1 --2- 172.21.15.43:0/3579148334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cec20faa30 0x55cec20f8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.868+0000 7fe19259c640 1 -- 172.21.15.43:0/3579148334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec201cc50 con 0x55cec20ef850 2024-09-11T23:27:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.869+0000 7fe19259c640 1 --2- 172.21.15.43:0/3579148334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20ef850 0x55cec20f77d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe184005bc0 tx=0x7fe184004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb733b18a6a80a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.869+0000 7fe179ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:27:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.869+0000 7fe179ffb640 1 -- 172.21.15.43:0/3579148334 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18402c060 con 0x55cec20ef850 2024-09-11T23:27:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.869+0000 7fe179ffb640 1 -- 172.21.15.43:0/3579148334 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7fe184017030 con 0x55cec20ef850 2024-09-11T23:27:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.869+0000 7fe19b34e740 1 -- 172.21.15.43:0/3579148334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe188031a20 con 0x55cec20ef850 2024-09-11T23:27:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.869+0000 7fe179ffb640 1 -- 172.21.15.43:0/3579148334 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe18401c600 con 0x55cec20ef850 2024-09-11T23:27:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.869+0000 7fe19b34e740 10 client.16317 fetch_fsmap learned FSMap version 30 2024-09-11T23:27:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.869+0000 7fe19b34e740 1 -- 172.21.15.43:0/3579148334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cec2109230 con 0x55cec20ef850 2024-09-11T23:27:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.869+0000 7fe179ffb640 1 -- 172.21.15.43:0/3579148334 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe188031a20 con 0x55cec20ef850 2024-09-11T23:27:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.869+0000 7fe179ffb640 1 -- 172.21.15.43:0/3579148334 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fe184027970 con 0x55cec20ef850 2024-09-11T23:27:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.870+0000 7fe19b34e740 10 client.16317 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:27:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.870+0000 7fe19b34e740 10 client.16317 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:27:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.870+0000 7fe19b34e740 20 client.16317 populate_metadata read hostname 'smithi043' 2024-09-11T23:27:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.870+0000 7fe19b34e740 1 --2- 172.21.15.43:0/3579148334 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55cec210a6f0 0x55cec210cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.870+0000 7fe19b34e740 4 client.16317 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:27:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.870+0000 7fe19b34e740 1 -- 172.21.15.43:0/3579148334 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cec2061890 con 0x55cec210a6f0 2024-09-11T23:27:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.870+0000 7fe191d9b640 1 --2- 172.21.15.43:0/3579148334 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55cec210a6f0 0x55cec210cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.870+0000 7fe191d9b640 1 --2- 172.21.15.43:0/3579148334 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55cec210a6f0 0x55cec210cb50 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f9dcb9882cc97b9 server_cookie=227b27c77cf53cd6 in_seq=0 out_seq=0 2024-09-11T23:27:40.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.870+0000 7fe179ffb640 10 client.16317 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:27:40.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.871+0000 7fe179ffb640 1 -- 172.21.15.43:0/3579148334 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cec2061890 con 0x55cec210a6f0 2024-09-11T23:27:40.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.871+0000 7fe179ffb640 10 client.16317 handle_command_reply: tid=0 2024-09-11T23:27:40.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.871+0000 7fe19b34e740 1 client.16317 shutdown 2024-09-11T23:27:40.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.871+0000 7fe19b34e740 1 -- 172.21.15.43:0/3579148334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20ef850 msgr2=0x55cec20f77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:40.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.871+0000 7fe19b34e740 1 --2- 172.21.15.43:0/3579148334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20ef850 0x55cec20f77d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe184005bc0 tx=0x7fe184004730 comp rx=0 tx=0).stop 2024-09-11T23:27:40.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.871+0000 7fe19b34e740 1 -- 172.21.15.43:0/3579148334 shutdown_connections 2024-09-11T23:27:40.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.871+0000 7fe19b34e740 1 --2- 172.21.15.43:0/3579148334 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55cec210a6f0 0x55cec210cb50 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.871+0000 7fe19b34e740 1 --2- 172.21.15.43:0/3579148334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cec20faa30 0x55cec20f8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.871+0000 7fe19b34e740 1 --2- 172.21.15.43:0/3579148334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cec20f0170 0x55cec20f7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.871+0000 7fe19b34e740 1 --2- 172.21.15.43:0/3579148334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20ef850 0x55cec20f77d0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.871+0000 7fe179ffb640 0 client.16317 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:27:40.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.871+0000 7fe19b34e740 1 -- 172.21.15.43:0/3579148334 >> 172.21.15.43:0/3579148334 conn(0x55cec20ebde0 msgr2=0x55cec20f1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:40.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.871+0000 7fe19b34e740 1 -- 172.21.15.43:0/3579148334 shutdown_connections 2024-09-11T23:27:40.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.871+0000 7fe19b34e740 1 -- 172.21.15.43:0/3579148334 wait complete. 2024-09-11T23:27:40.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.871+0000 7fe19b34e740 20 client.16317 trim_cache size 0 max 16384 2024-09-11T23:27:40.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.888+0000 7fe19b34e740 1 Processor -- start 2024-09-11T23:27:40.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.888+0000 7fe19b34e740 1 -- start start 2024-09-11T23:27:40.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.888+0000 7fe19b34e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cec20ef850 0x55cec20f6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:40.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.888+0000 7fe19b34e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20f0170 0x55cec2112460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:40.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.888+0000 7fe19b34e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cec20faa30 0x55cec21129a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:40.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.888+0000 7fe19b34e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe1941b2300 con 0x55cec20f0170 2024-09-11T23:27:40.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.888+0000 7fe19b34e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe1941afbd0 con 0x55cec20faa30 2024-09-11T23:27:40.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.888+0000 7fe19b34e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe1941ada20 con 0x55cec20ef850 2024-09-11T23:27:40.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.888+0000 7fe191d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20f0170 0x55cec2112460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:40.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.889+0000 7fe191d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20f0170 0x55cec2112460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56050/0 (socket says 172.21.15.43:56050) 2024-09-11T23:27:40.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.889+0000 7fe191d9b640 1 -- 172.21.15.43:0/1684116503 learned_addr learned my addr 172.21.15.43:0/1684116503 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:40.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.889+0000 7fe19259c640 1 --2- 172.21.15.43:0/1684116503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cec20ef850 0x55cec20f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:40.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.889+0000 7fe191d9b640 1 -- 172.21.15.43:0/1684116503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cec20ef850 msgr2=0x55cec20f6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:40.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.889+0000 7fe191d9b640 1 --2- 172.21.15.43:0/1684116503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cec20ef850 0x55cec20f6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.889+0000 7fe191d9b640 1 -- 172.21.15.43:0/1684116503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cec20faa30 msgr2=0x55cec21129a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:40.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.889+0000 7fe191d9b640 1 --2- 172.21.15.43:0/1684116503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cec20faa30 0x55cec21129a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.889+0000 7fe191d9b640 1 -- 172.21.15.43:0/1684116503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec20effa0 con 0x55cec20f0170 2024-09-11T23:27:40.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.889+0000 7fe191d9b640 1 --2- 172.21.15.43:0/1684116503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20f0170 0x55cec2112460 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe1880055c0 tx=0x7fe1880055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cadd83adfe869c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:40.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.889+0000 7fe17bfff640 1 -- 172.21.15.43:0/1684116503 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188035040 con 0x55cec20f0170 2024-09-11T23:27:40.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.889+0000 7fe17bfff640 1 -- 172.21.15.43:0/1684116503 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1880315b0 con 0x55cec20f0170 2024-09-11T23:27:40.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.889+0000 7fe17bfff640 1 -- 172.21.15.43:0/1684116503 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18803e500 con 0x55cec20f0170 2024-09-11T23:27:40.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.890+0000 7fe19b34e740 1 -- 172.21.15.43:0/1684116503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20f0170 msgr2=0x55cec2112460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:40.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.890+0000 7fe19b34e740 1 --2- 172.21.15.43:0/1684116503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20f0170 0x55cec2112460 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe1880055c0 tx=0x7fe1880055f0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.890+0000 7fe19b34e740 1 -- 172.21.15.43:0/1684116503 shutdown_connections 2024-09-11T23:27:40.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.890+0000 7fe19b34e740 1 --2- 172.21.15.43:0/1684116503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cec20faa30 0x55cec21129a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.890+0000 7fe19b34e740 1 --2- 172.21.15.43:0/1684116503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20f0170 0x55cec2112460 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.890+0000 7fe19b34e740 1 --2- 172.21.15.43:0/1684116503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cec20ef850 0x55cec20f6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.890+0000 7fe19b34e740 1 -- 172.21.15.43:0/1684116503 >> 172.21.15.43:0/1684116503 conn(0x55cec20ebde0 msgr2=0x55cec20eaec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:40.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.890+0000 7fe19b34e740 1 -- 172.21.15.43:0/1684116503 shutdown_connections 2024-09-11T23:27:40.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.890+0000 7fe19b34e740 1 -- 172.21.15.43:0/1684116503 wait complete. 2024-09-11T23:27:40.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.890+0000 7fe19b34e740 1 -- start start 2024-09-11T23:27:40.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.890+0000 7fe19b34e740 1 Processor -- start 2024-09-11T23:27:40.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe19b34e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20ef850 0x55cec20fbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:40.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe19b34e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cec20f0170 0x55cec20fc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:40.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe19b34e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cec20faa30 0x55cec20fc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:40.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe19b34e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe1941ae090 con 0x55cec20ef850 2024-09-11T23:27:40.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe19b34e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe1941af500 con 0x55cec20f0170 2024-09-11T23:27:40.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe19b34e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe1941ae760 con 0x55cec20faa30 2024-09-11T23:27:40.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe19b34e740 10 client.0 _ll_register_callbacks cb 0x7fe18400bf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:40.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe19259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20ef850 0x55cec20fbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:40.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe19259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20ef850 0x55cec20fbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56060/0 (socket says 172.21.15.43:56060) 2024-09-11T23:27:40.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe19259c640 1 -- 172.21.15.43:0/1184614473 learned_addr learned my addr 172.21.15.43:0/1184614473 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:40.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe192d9d640 1 --2- 172.21.15.43:0/1184614473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cec20faa30 0x55cec20fc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:40.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe191d9b640 1 --2- 172.21.15.43:0/1184614473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cec20f0170 0x55cec20fc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:40.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe19259c640 1 -- 172.21.15.43:0/1184614473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cec20faa30 msgr2=0x55cec20fc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:40.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe19259c640 1 --2- 172.21.15.43:0/1184614473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cec20faa30 0x55cec20fc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe19259c640 1 -- 172.21.15.43:0/1184614473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cec20f0170 msgr2=0x55cec20fc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:40.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe19259c640 1 --2- 172.21.15.43:0/1184614473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cec20f0170 0x55cec20fc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:40.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe19259c640 1 -- 172.21.15.43:0/1184614473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec2101420 con 0x55cec20ef850 2024-09-11T23:27:40.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe19259c640 1 --2- 172.21.15.43:0/1184614473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20ef850 0x55cec20fbdf0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe18401ffd0 tx=0x7fe18401ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e130f5fb2625b51 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:40.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe179ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:27:40.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe179ffb640 1 -- 172.21.15.43:0/1184614473 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18401f220 con 0x55cec20ef850 2024-09-11T23:27:40.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe179ffb640 1 -- 172.21.15.43:0/1184614473 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7fe1840275f0 con 0x55cec20ef850 2024-09-11T23:27:40.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.891+0000 7fe19b34e740 1 -- 172.21.15.43:0/1184614473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1880315b0 con 0x55cec20ef850 2024-09-11T23:27:40.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.892+0000 7fe179ffb640 1 -- 172.21.15.43:0/1184614473 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe1840277f0 con 0x55cec20ef850 2024-09-11T23:27:40.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.892+0000 7fe19b34e740 10 client.16323 fetch_fsmap learned FSMap version 30 2024-09-11T23:27:40.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.892+0000 7fe19b34e740 1 -- 172.21.15.43:0/1184614473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cec21758d0 con 0x55cec20ef850 2024-09-11T23:27:40.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.892+0000 7fe179ffb640 1 -- 172.21.15.43:0/1184614473 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1880315b0 con 0x55cec20ef850 2024-09-11T23:27:40.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.892+0000 7fe179ffb640 1 -- 172.21.15.43:0/1184614473 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fe184042b00 con 0x55cec20ef850 2024-09-11T23:27:40.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.892+0000 7fe19b34e740 10 client.16323 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:27:40.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.892+0000 7fe19b34e740 10 client.16323 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:27:40.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.892+0000 7fe19b34e740 20 client.16323 populate_metadata read hostname 'smithi043' 2024-09-11T23:27:40.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.892+0000 7fe19b34e740 1 --2- 172.21.15.43:0/1184614473 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55cec2175fb0 0x55cec21783d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:40.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.892+0000 7fe19b34e740 4 client.16323 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-11T23:27:40.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.892+0000 7fe19b34e740 1 -- 172.21.15.43:0/1184614473 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x55cec205bbe0 con 0x55cec2175fb0 2024-09-11T23:27:40.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.892+0000 7fe191d9b640 1 --2- 172.21.15.43:0/1184614473 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55cec2175fb0 0x55cec21783d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:40.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.893+0000 7fe191d9b640 1 --2- 172.21.15.43:0/1184614473 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55cec2175fb0 0x55cec21783d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d532283acb193d3b server_cookie=f380c7664946e30a in_seq=0 out_seq=0 2024-09-11T23:27:40.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:27:40.893+0000 7fe179ffb640 10 client.16323 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:27:42.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:41 smithi090 ceph-mon[35130]: pgmap v179: 97 pgs: 97 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 23 MiB/s wr, 4.16k op/s 2024-09-11T23:27:42.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:41 smithi097 ceph-mon[35562]: pgmap v179: 97 pgs: 97 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 23 MiB/s wr, 4.16k op/s 2024-09-11T23:27:42.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:41 smithi043 ceph-mon[30950]: pgmap v179: 97 pgs: 97 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 23 MiB/s wr, 4.16k op/s 2024-09-11T23:27:44.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:44 smithi090 ceph-mon[35130]: pgmap v180: 97 pgs: 97 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 22 MiB/s wr, 3.90k op/s 2024-09-11T23:27:44.300 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:44 smithi097 ceph-mon[35562]: pgmap v180: 97 pgs: 97 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 22 MiB/s wr, 3.90k op/s 2024-09-11T23:27:44.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:44 smithi043 ceph-mon[30950]: pgmap v180: 97 pgs: 97 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 22 MiB/s wr, 3.90k op/s 2024-09-11T23:27:46.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:46 smithi090 ceph-mon[35130]: pgmap v181: 97 pgs: 97 active+clean; 20 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 38 MiB/s wr, 6.74k op/s 2024-09-11T23:27:46.300 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:46 smithi097 ceph-mon[35562]: pgmap v181: 97 pgs: 97 active+clean; 20 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 38 MiB/s wr, 6.74k op/s 2024-09-11T23:27:46.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:46 smithi043 ceph-mon[30950]: pgmap v181: 97 pgs: 97 active+clean; 20 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 38 MiB/s wr, 6.74k op/s 2024-09-11T23:27:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.033+0000 7f9c876ac640 1 -- 172.21.15.43:0/1080196574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c80100830 msgr2=0x7f9c80100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.033+0000 7f9c876ac640 1 --2- 172.21.15.43:0/1080196574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c80100830 0x7f9c80100c10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009900 tx=0x7f9c7c02a4b0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.033+0000 7f9c876ac640 1 -- 172.21.15.43:0/1080196574 shutdown_connections 2024-09-11T23:27:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.033+0000 7f9c876ac640 1 --2- 172.21.15.43:0/1080196574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c801080f0 0x7f9c8010ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.033+0000 7f9c876ac640 1 --2- 172.21.15.43:0/1080196574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c80101150 0x7f9c80107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.033+0000 7f9c876ac640 1 --2- 172.21.15.43:0/1080196574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c80100830 0x7f9c80100c10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.033+0000 7f9c876ac640 1 -- 172.21.15.43:0/1080196574 >> 172.21.15.43:0/1080196574 conn(0x7f9c80075f20 msgr2=0x7f9c80076330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.036+0000 7f9c876ac640 1 -- 172.21.15.43:0/1080196574 shutdown_connections 2024-09-11T23:27:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.036+0000 7f9c876ac640 1 -- 172.21.15.43:0/1080196574 wait complete. 2024-09-11T23:27:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.037+0000 7f9c876ac640 1 Processor -- start 2024-09-11T23:27:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.038+0000 7f9c876ac640 1 -- start start 2024-09-11T23:27:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.039+0000 7f9c876ac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c80100830 0x7f9c801170c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.039+0000 7f9c876ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c80101150 0x7f9c80117600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.039+0000 7f9c876ac640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c801080f0 0x7f9c80111130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.039+0000 7f9c876ac640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8010fcd0 con 0x7f9c80101150 2024-09-11T23:27:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.039+0000 7f9c876ac640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8010fb50 con 0x7f9c801080f0 2024-09-11T23:27:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.039+0000 7f9c876ac640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8010fe50 con 0x7f9c80100830 2024-09-11T23:27:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.040+0000 7f9c84c47640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c80101150 0x7f9c80117600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.041+0000 7f9c84c47640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c80101150 0x7f9c80117600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56078/0 (socket says 172.21.15.43:56078) 2024-09-11T23:27:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.041+0000 7f9c84c47640 1 -- 172.21.15.43:0/1236168323 learned_addr learned my addr 172.21.15.43:0/1236168323 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.042+0000 7f9c85c49640 1 --2- 172.21.15.43:0/1236168323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c801080f0 0x7f9c80111130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.042+0000 7f9c85448640 1 --2- 172.21.15.43:0/1236168323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c80100830 0x7f9c801170c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.046+0000 7f9c84c47640 1 -- 172.21.15.43:0/1236168323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c80100830 msgr2=0x7f9c801170c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.046+0000 7f9c84c47640 1 --2- 172.21.15.43:0/1236168323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c80100830 0x7f9c801170c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.046+0000 7f9c84c47640 1 -- 172.21.15.43:0/1236168323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c801080f0 msgr2=0x7f9c80111130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.046+0000 7f9c84c47640 1 --2- 172.21.15.43:0/1236168323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c801080f0 0x7f9c80111130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.046+0000 7f9c84c47640 1 -- 172.21.15.43:0/1236168323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c801119c0 con 0x7f9c80101150 2024-09-11T23:27:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.047+0000 7f9c85448640 1 --2- 172.21.15.43:0/1236168323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c80100830 0x7f9c801170c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:27:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.049+0000 7f9c84c47640 1 --2- 172.21.15.43:0/1236168323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c80101150 0x7f9c80117600 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000b470 tx=0x7f9c7000b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.053+0000 7f9c767fc640 1 -- 172.21.15.43:0/1236168323 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70015020 con 0x7f9c80101150 2024-09-11T23:27:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.053+0000 7f9c767fc640 1 -- 172.21.15.43:0/1236168323 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c70011d70 con 0x7f9c80101150 2024-09-11T23:27:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.053+0000 7f9c876ac640 1 -- 172.21.15.43:0/1236168323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c80111cb0 con 0x7f9c80101150 2024-09-11T23:27:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.053+0000 7f9c876ac640 1 -- 172.21.15.43:0/1236168323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c801b3da0 con 0x7f9c80101150 2024-09-11T23:27:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.053+0000 7f9c767fc640 1 -- 172.21.15.43:0/1236168323 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7000f340 con 0x7f9c80101150 2024-09-11T23:27:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.062+0000 7f9c8a44d740 1 Processor -- start 2024-09-11T23:27:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.062+0000 7f9c8a44d740 1 -- start start 2024-09-11T23:27:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.062+0000 7f9c8a44d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be979cbaf0 0x55be979cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.062+0000 7f9c8a44d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979cc410 0x55be979d6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.062+0000 7f9c8a44d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979d6cb0 0x55be979d90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.062+0000 7f9c8a44d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55be979302f0 con 0x55be979d6cb0 2024-09-11T23:27:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.062+0000 7f9c8a44d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55be978d8270 con 0x55be979cc410 2024-09-11T23:27:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.062+0000 7f9c8a44d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55be9792c2b0 con 0x55be979cbaf0 2024-09-11T23:27:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.063+0000 7f9c85448640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979cc410 0x55be979d6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.063+0000 7f9c85448640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979cc410 0x55be979d6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:40212/0 (socket says 172.21.15.43:40212) 2024-09-11T23:27:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.063+0000 7f9c85448640 1 -- 172.21.15.43:0/289013411 learned_addr learned my addr 172.21.15.43:0/289013411 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.063+0000 7f9c85c49640 1 --2- 172.21.15.43:0/289013411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be979cbaf0 0x55be979cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:47.065 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.064+0000 7f9c767fc640 1 -- 172.21.15.43:0/1236168323 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9c70018020 con 0x7f9c80101150 2024-09-11T23:27:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.066+0000 7f9c85c49640 1 -- 172.21.15.43:0/289013411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979cc410 msgr2=0x55be979d6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.066+0000 7f9c85c49640 1 --2- 172.21.15.43:0/289013411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979cc410 0x55be979d6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.066+0000 7f9c85c49640 1 -- 172.21.15.43:0/289013411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979d6cb0 msgr2=0x55be979d90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.066+0000 7f9c85c49640 1 --2- 172.21.15.43:0/289013411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979d6cb0 0x55be979d90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.066+0000 7f9c85c49640 1 -- 172.21.15.43:0/289013411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be978d7340 con 0x55be979cbaf0 2024-09-11T23:27:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.066+0000 7f9c767fc640 1 --2- 172.21.15.43:0/1236168323 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9c6c0a2ec0 0x7f9c6c0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.067+0000 7f9c85448640 1 --2- 172.21.15.43:0/1236168323 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9c6c0a2ec0 0x7f9c6c0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.067+0000 7f9c85c49640 1 --2- 172.21.15.43:0/289013411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be979cbaf0 0x55be979cbed0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9c78005fd0 tx=0x7f9c7800d360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0feb3daffacd7a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.067+0000 7f9c767fc640 1 -- 172.21.15.43:0/1236168323 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f9c700ced90 con 0x7f9c80101150 2024-09-11T23:27:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.067+0000 7f9c85448640 1 --2- 172.21.15.43:0/1236168323 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9c6c0a2ec0 0x7f9c6c0a5380 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0096c0 tx=0x7f9c7c00b040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.067+0000 7f9c6b7fe640 1 -- 172.21.15.43:0/289013411 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78015760 con 0x55be979cbaf0 2024-09-11T23:27:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.068+0000 7f9c6b7fe640 1 -- 172.21.15.43:0/289013411 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c7801b070 con 0x55be979cbaf0 2024-09-11T23:27:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.069+0000 7f9c8a44d740 1 -- 172.21.15.43:0/289013411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be979cbaf0 msgr2=0x55be979cbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.069+0000 7f9c8a44d740 1 --2- 172.21.15.43:0/289013411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be979cbaf0 0x55be979cbed0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9c78005fd0 tx=0x7f9c7800d360 comp rx=0 tx=0).stop 2024-09-11T23:27:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.069+0000 7f9c8a44d740 1 -- 172.21.15.43:0/289013411 shutdown_connections 2024-09-11T23:27:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.069+0000 7f9c8a44d740 1 --2- 172.21.15.43:0/289013411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979d6cb0 0x55be979d90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.069+0000 7f9c8a44d740 1 --2- 172.21.15.43:0/289013411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979cc410 0x55be979d6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.069+0000 7f9c8a44d740 1 --2- 172.21.15.43:0/289013411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be979cbaf0 0x55be979cbed0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.069+0000 7f9c8a44d740 1 -- 172.21.15.43:0/289013411 >> 172.21.15.43:0/289013411 conn(0x55be979c8060 msgr2=0x55be979c8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.069+0000 7f9c8a44d740 1 -- 172.21.15.43:0/289013411 shutdown_connections 2024-09-11T23:27:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.069+0000 7f9c8a44d740 1 -- 172.21.15.43:0/289013411 wait complete. 2024-09-11T23:27:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.070+0000 7f9c8a44d740 1 -- start start 2024-09-11T23:27:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.070+0000 7f9c8a44d740 1 Processor -- start 2024-09-11T23:27:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.070+0000 7f9c8a44d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be979cbaf0 0x55be979d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.070+0000 7f9c8a44d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979cc410 0x55be979d3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.070+0000 7f9c8a44d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979d6cb0 0x55be979d44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.070+0000 7f9c8a44d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55be979d98d0 con 0x55be979d6cb0 2024-09-11T23:27:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.070+0000 7f9c8a44d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55be978b9290 con 0x55be979cc410 2024-09-11T23:27:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.070+0000 7f9c8a44d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55be979d9a50 con 0x55be979cbaf0 2024-09-11T23:27:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.070+0000 7f9c8a44d740 10 client.0 _ll_register_callbacks cb 0x55be978ea140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.071+0000 7f9c84c47640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979cc410 0x55be979d3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.071+0000 7f9c84c47640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979cc410 0x55be979d3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:40222/0 (socket says 172.21.15.43:40222) 2024-09-11T23:27:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.071+0000 7f9c84c47640 1 -- 172.21.15.43:0/4206632823 learned_addr learned my addr 172.21.15.43:0/4206632823 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.071+0000 7f9c85448640 1 --2- 172.21.15.43:0/4206632823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be979cbaf0 0x55be979d3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.071+0000 7f9c85c49640 1 --2- 172.21.15.43:0/4206632823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979d6cb0 0x55be979d44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.071+0000 7f9c84c47640 1 -- 172.21.15.43:0/4206632823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be979cbaf0 msgr2=0x55be979d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.071+0000 7f9c84c47640 1 --2- 172.21.15.43:0/4206632823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be979cbaf0 0x55be979d3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.071+0000 7f9c84c47640 1 -- 172.21.15.43:0/4206632823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979d6cb0 msgr2=0x55be979d44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.071+0000 7f9c84c47640 1 --2- 172.21.15.43:0/4206632823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979d6cb0 0x55be979d44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.071+0000 7f9c84c47640 1 -- 172.21.15.43:0/4206632823 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be97643210 con 0x55be979cc410 2024-09-11T23:27:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.074+0000 7f9c84c47640 1 --2- 172.21.15.43:0/4206632823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979cc410 0x55be979d3fa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000be30 tx=0x7f9c7009aea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75e359a0ea970cb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.074+0000 7f9c68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:27:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.074+0000 7f9c68ff9640 1 -- 172.21.15.43:0/4206632823 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c700b4060 con 0x55be979cc410 2024-09-11T23:27:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.075+0000 7f9c8a44d740 1 -- 172.21.15.43:0/4206632823 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c7801b070 con 0x55be979cc410 2024-09-11T23:27:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.075+0000 7f9c68ff9640 1 -- 172.21.15.43:0/4206632823 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f9c700979d0 con 0x55be979cc410 2024-09-11T23:27:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.075+0000 7f9c68ff9640 1 -- 172.21.15.43:0/4206632823 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9c7009b360 con 0x55be979cc410 2024-09-11T23:27:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.075+0000 7f9c8a44d740 10 client.25853 fetch_fsmap learned FSMap version 30 2024-09-11T23:27:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.075+0000 7f9c8a44d740 1 -- 172.21.15.43:0/4206632823 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55be9770f620 con 0x55be979cc410 2024-09-11T23:27:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.075+0000 7f9c68ff9640 1 -- 172.21.15.43:0/4206632823 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c7801b070 con 0x55be979cc410 2024-09-11T23:27:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.076+0000 7f9c68ff9640 1 -- 172.21.15.43:0/4206632823 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9c700a7020 con 0x55be979cc410 2024-09-11T23:27:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.076+0000 7f9c8a44d740 10 client.25853 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:27:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.076+0000 7f9c8a44d740 10 client.25853 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:27:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.076+0000 7f9c8a44d740 20 client.25853 populate_metadata read hostname 'smithi043' 2024-09-11T23:27:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.076+0000 7f9c8a44d740 1 --2- 172.21.15.43:0/4206632823 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55be979e6fd0 0x55be979e93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.076+0000 7f9c8a44d740 4 client.25853 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:27:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.076+0000 7f9c8a44d740 1 -- 172.21.15.43:0/4206632823 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be9793da20 con 0x55be979e6fd0 2024-09-11T23:27:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.077+0000 7f9c85448640 1 --2- 172.21.15.43:0/4206632823 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55be979e6fd0 0x55be979e93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.079+0000 7f9c85448640 1 --2- 172.21.15.43:0/4206632823 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55be979e6fd0 0x55be979e93f0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ca12bca6000ffb5 server_cookie=edc031fd9326f1c9 in_seq=0 out_seq=0 2024-09-11T23:27:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.079+0000 7f9c68ff9640 10 client.25853 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:27:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.080+0000 7f9c68ff9640 1 -- 172.21.15.43:0/4206632823 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55be9793da20 con 0x55be979e6fd0 2024-09-11T23:27:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.080+0000 7f9c68ff9640 10 client.25853 handle_command_reply: tid=0 2024-09-11T23:27:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.080+0000 7f9c8a44d740 1 client.25853 shutdown 2024-09-11T23:27:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.083+0000 7f9c8a44d740 1 -- 172.21.15.43:0/4206632823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979cc410 msgr2=0x55be979d3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.083+0000 7f9c8a44d740 1 --2- 172.21.15.43:0/4206632823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979cc410 0x55be979d3fa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000be30 tx=0x7f9c7009aea0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.086+0000 7f9c8a44d740 1 -- 172.21.15.43:0/4206632823 shutdown_connections 2024-09-11T23:27:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.086+0000 7f9c8a44d740 1 --2- 172.21.15.43:0/4206632823 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55be979e6fd0 0x55be979e93f0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.086+0000 7f9c8a44d740 1 --2- 172.21.15.43:0/4206632823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979d6cb0 0x55be979d44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.086+0000 7f9c8a44d740 1 --2- 172.21.15.43:0/4206632823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979cc410 0x55be979d3fa0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.086+0000 7f9c8a44d740 1 --2- 172.21.15.43:0/4206632823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be979cbaf0 0x55be979d3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.086+0000 7f9c8a44d740 1 -- 172.21.15.43:0/4206632823 >> 172.21.15.43:0/4206632823 conn(0x55be979c8060 msgr2=0x55be979cdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.088+0000 7f9c68ff9640 0 client.25853 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:27:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.092+0000 7f9c8a44d740 1 -- 172.21.15.43:0/4206632823 shutdown_connections 2024-09-11T23:27:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.093+0000 7f9c8a44d740 1 -- 172.21.15.43:0/4206632823 wait complete. 2024-09-11T23:27:47.097 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.097+0000 7f9c8a44d740 20 client.25853 trim_cache size 0 max 16384 2024-09-11T23:27:47.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.132+0000 7f9c8a44d740 1 Processor -- start 2024-09-11T23:27:47.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.132+0000 7f9c8a44d740 1 -- start start 2024-09-11T23:27:47.133 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.132+0000 7f9c8a44d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be979cbaf0 0x55be979d4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.133+0000 7f9c8a44d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979cc410 0x55be979eece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.133+0000 7f9c85448640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be979cbaf0 0x55be979d4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.133+0000 7f9c85448640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be979cbaf0 0x55be979d4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:35608/0 (socket says 172.21.15.43:35608) 2024-09-11T23:27:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.133+0000 7f9c85448640 1 -- 172.21.15.43:0/2673697355 learned_addr learned my addr 172.21.15.43:0/2673697355 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.133+0000 7f9c8a44d740 1 --2- 172.21.15.43:0/2673697355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979d6cb0 0x55be979ef220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.133+0000 7f9c8a44d740 1 -- 172.21.15.43:0/2673697355 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9c801bf0a0 con 0x55be979d6cb0 2024-09-11T23:27:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.133+0000 7f9c85c49640 1 --2- 172.21.15.43:0/2673697355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979d6cb0 0x55be979ef220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.133+0000 7f9c8a44d740 1 -- 172.21.15.43:0/2673697355 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9c801bc970 con 0x55be979cc410 2024-09-11T23:27:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.134+0000 7f9c8a44d740 1 -- 172.21.15.43:0/2673697355 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9c801ba7c0 con 0x55be979cbaf0 2024-09-11T23:27:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.134+0000 7f9c85c49640 1 -- 172.21.15.43:0/2673697355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be979cbaf0 msgr2=0x55be979d4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.134+0000 7f9c85c49640 1 --2- 172.21.15.43:0/2673697355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be979cbaf0 0x55be979d4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.134+0000 7f9c85c49640 1 -- 172.21.15.43:0/2673697355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979cc410 msgr2=0x55be979eece0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:27:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.134+0000 7f9c85c49640 1 --2- 172.21.15.43:0/2673697355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979cc410 0x55be979eece0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.134+0000 7f9c85c49640 1 -- 172.21.15.43:0/2673697355 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be979cc240 con 0x55be979d6cb0 2024-09-11T23:27:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.136+0000 7f9c85c49640 1 --2- 172.21.15.43:0/2673697355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979d6cb0 0x55be979ef220 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9c78000f80 tx=0x7f9c78005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263153974809191f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.139+0000 7f9c6affd640 1 -- 172.21.15.43:0/2673697355 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78003970 con 0x55be979d6cb0 2024-09-11T23:27:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.140+0000 7f9c6affd640 1 -- 172.21.15.43:0/2673697355 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c7801b040 con 0x55be979d6cb0 2024-09-11T23:27:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.141+0000 7f9c8a44d740 1 -- 172.21.15.43:0/2673697355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979d6cb0 msgr2=0x55be979ef220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.141+0000 7f9c8a44d740 1 --2- 172.21.15.43:0/2673697355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979d6cb0 0x55be979ef220 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9c78000f80 tx=0x7f9c78005bd0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.141+0000 7f9c8a44d740 1 -- 172.21.15.43:0/2673697355 shutdown_connections 2024-09-11T23:27:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.141+0000 7f9c8a44d740 1 --2- 172.21.15.43:0/2673697355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979d6cb0 0x55be979ef220 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.141+0000 7f9c8a44d740 1 --2- 172.21.15.43:0/2673697355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979cc410 0x55be979eece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.141+0000 7f9c8a44d740 1 --2- 172.21.15.43:0/2673697355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be979cbaf0 0x55be979d4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.141+0000 7f9c8a44d740 1 -- 172.21.15.43:0/2673697355 >> 172.21.15.43:0/2673697355 conn(0x55be979c8060 msgr2=0x55be979d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.144+0000 7f9c8a44d740 1 -- 172.21.15.43:0/2673697355 shutdown_connections 2024-09-11T23:27:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.145+0000 7f9c8a44d740 1 -- 172.21.15.43:0/2673697355 wait complete. 2024-09-11T23:27:47.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.145+0000 7f9c8a44d740 1 -- start start 2024-09-11T23:27:47.146 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.145+0000 7f9c8a44d740 1 Processor -- start 2024-09-11T23:27:47.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.147+0000 7f9c8a44d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979cbaf0 0x55be979d0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.147+0000 7f9c8a44d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be979cc410 0x55be979d0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.147+0000 7f9c8a44d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979d6cb0 0x55be979ddd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.147+0000 7f9c8a44d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9c801bae30 con 0x55be979cbaf0 2024-09-11T23:27:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.147+0000 7f9c8a44d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9c801bc2a0 con 0x55be979d6cb0 2024-09-11T23:27:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.147+0000 7f9c8a44d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9c801bb500 con 0x55be979cc410 2024-09-11T23:27:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.147+0000 7f9c8a44d740 10 client.0 _ll_register_callbacks cb 0x7f9c7009b130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:27:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.148+0000 7f9c85448640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979cbaf0 0x55be979d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.148+0000 7f9c85448640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979cbaf0 0x55be979d0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56104/0 (socket says 172.21.15.43:56104) 2024-09-11T23:27:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.148+0000 7f9c85448640 1 -- 172.21.15.43:0/2516639343 learned_addr learned my addr 172.21.15.43:0/2516639343 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:27:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.149+0000 7f9c85448640 1 -- 172.21.15.43:0/2516639343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be979cc410 msgr2=0x55be979d0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.149+0000 7f9c85448640 1 --2- 172.21.15.43:0/2516639343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be979cc410 0x55be979d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.149+0000 7f9c85448640 1 -- 172.21.15.43:0/2516639343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979d6cb0 msgr2=0x55be979ddd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:27:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.149+0000 7f9c85c49640 1 --2- 172.21.15.43:0/2516639343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979d6cb0 0x55be979ddd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.149+0000 7f9c85448640 1 --2- 172.21.15.43:0/2516639343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979d6cb0 0x55be979ddd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.149+0000 7f9c85448640 1 -- 172.21.15.43:0/2516639343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be979de270 con 0x55be979cbaf0 2024-09-11T23:27:47.150 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.149+0000 7f9c85c49640 1 --2- 172.21.15.43:0/2516639343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979d6cb0 0x55be979ddd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:27:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.151+0000 7f9c85448640 1 --2- 172.21.15.43:0/2516639343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979cbaf0 0x55be979d0250 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009850 tx=0x7f9c7c03e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac52a5fa1be69 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:27:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.151+0000 7f9c68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:27:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.155+0000 7f9c68ff9640 1 -- 172.21.15.43:0/2516639343 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c035050 con 0x55be979cbaf0 2024-09-11T23:27:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.155+0000 7f9c68ff9640 1 -- 172.21.15.43:0/2516639343 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f9c7c005220 con 0x55be979cbaf0 2024-09-11T23:27:47.158 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.158+0000 7f9c8a44d740 1 -- 172.21.15.43:0/2516639343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c7801b040 con 0x55be979cbaf0 2024-09-11T23:27:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.159+0000 7f9c68ff9640 1 -- 172.21.15.43:0/2516639343 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9c7c005420 con 0x55be979cbaf0 2024-09-11T23:27:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.166+0000 7f9c8a44d740 10 client.16338 fetch_fsmap learned FSMap version 30 2024-09-11T23:27:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.166+0000 7f9c8a44d740 1 -- 172.21.15.43:0/2516639343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55be979df550 con 0x55be979cbaf0 2024-09-11T23:27:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.170+0000 7f9c68ff9640 1 -- 172.21.15.43:0/2516639343 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c7801b040 con 0x55be979cbaf0 2024-09-11T23:27:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.170+0000 7f9c68ff9640 1 -- 172.21.15.43:0/2516639343 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9c7c003990 con 0x55be979cbaf0 2024-09-11T23:27:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.171+0000 7f9c8a44d740 10 client.16338 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:27:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.171+0000 7f9c8a44d740 10 client.16338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:27:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.171+0000 7f9c8a44d740 20 client.16338 populate_metadata read hostname 'smithi043' 2024-09-11T23:27:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.171+0000 7f9c8a44d740 1 --2- 172.21.15.43:0/2516639343 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55be97a51d00 0x55be97a54140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:27:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.171+0000 7f9c8a44d740 4 client.16338 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:27:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.171+0000 7f9c8a44d740 1 -- 172.21.15.43:0/2516639343 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55be97937d70 con 0x55be97a51d00 2024-09-11T23:27:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.176+0000 7f9c84c47640 1 --2- 172.21.15.43:0/2516639343 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55be97a51d00 0x55be97a54140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:27:47.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.180+0000 7f9c84c47640 1 --2- 172.21.15.43:0/2516639343 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55be97a51d00 0x55be97a54140 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7a1772bb4aac045 server_cookie=8d2631afd879c934 in_seq=0 out_seq=0 2024-09-11T23:27:47.181 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:47.181+0000 7f9c68ff9640 10 client.16338 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:27:48.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:48 smithi090 ceph-mon[35130]: pgmap v182: 97 pgs: 97 active+clean; 20 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 38 MiB/s wr, 6.72k op/s 2024-09-11T23:27:48.300 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:48 smithi097 ceph-mon[35562]: pgmap v182: 97 pgs: 97 active+clean; 20 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 38 MiB/s wr, 6.72k op/s 2024-09-11T23:27:48.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:48 smithi043 ceph-mon[30950]: pgmap v182: 97 pgs: 97 active+clean; 20 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 38 MiB/s wr, 6.72k op/s 2024-09-11T23:27:50.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:50 smithi090 ceph-mon[35130]: pgmap v183: 97 pgs: 97 active+clean; 20 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 38 MiB/s wr, 6.61k op/s 2024-09-11T23:27:50.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:50 smithi097 ceph-mon[35562]: pgmap v183: 97 pgs: 97 active+clean; 20 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 38 MiB/s wr, 6.61k op/s 2024-09-11T23:27:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:50 smithi043 ceph-mon[30950]: pgmap v183: 97 pgs: 97 active+clean; 20 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 38 MiB/s wr, 6.61k op/s 2024-09-11T23:27:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.631+0000 7f9c68ff9640 1 -- 172.21.15.43:0/2516639343 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55be97937d70 con 0x55be97a51d00 2024-09-11T23:27:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.631+0000 7f9c68ff9640 10 client.16338 handle_command_reply: tid=0 2024-09-11T23:27:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.631+0000 7f9c8a44d740 1 client.16338 shutdown 2024-09-11T23:27:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.632+0000 7f9c8a44d740 1 -- 172.21.15.43:0/2516639343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979cbaf0 msgr2=0x55be979d0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.632+0000 7f9c8a44d740 1 --2- 172.21.15.43:0/2516639343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979cbaf0 0x55be979d0250 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009850 tx=0x7f9c7c03e070 comp rx=0 tx=0).stop 2024-09-11T23:27:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.632+0000 7f9c8a44d740 1 -- 172.21.15.43:0/2516639343 shutdown_connections 2024-09-11T23:27:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.632+0000 7f9c8a44d740 1 --2- 172.21.15.43:0/2516639343 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55be97a51d00 0x55be97a54140 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.632+0000 7f9c8a44d740 1 --2- 172.21.15.43:0/2516639343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be979d6cb0 0x55be979ddd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.632+0000 7f9c8a44d740 1 --2- 172.21.15.43:0/2516639343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55be979cc410 0x55be979d0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.632+0000 7f9c8a44d740 1 --2- 172.21.15.43:0/2516639343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be979cbaf0 0x55be979d0250 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.632+0000 7f9c8a44d740 1 -- 172.21.15.43:0/2516639343 >> 172.21.15.43:0/2516639343 conn(0x55be979c8060 msgr2=0x55be979c69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.632+0000 7f9c68ff9640 0 client.16338 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:27:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.632+0000 7f9c8a44d740 1 -- 172.21.15.43:0/2516639343 shutdown_connections 2024-09-11T23:27:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.632+0000 7f9c8a44d740 1 -- 172.21.15.43:0/2516639343 wait complete. 2024-09-11T23:27:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.632+0000 7f9c8a44d740 20 client.16338 trim_cache size 0 max 16384 2024-09-11T23:27:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:27:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:27:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:27:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:27:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.633+0000 7f9c876ac640 1 -- 172.21.15.43:0/1236168323 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9c6c0a2ec0 msgr2=0x7f9c6c0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.633+0000 7f9c876ac640 1 --2- 172.21.15.43:0/1236168323 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9c6c0a2ec0 0x7f9c6c0a5380 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0096c0 tx=0x7f9c7c00b040 comp rx=0 tx=0).stop 2024-09-11T23:27:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.633+0000 7f9c876ac640 1 -- 172.21.15.43:0/1236168323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c80101150 msgr2=0x7f9c80117600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:27:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.633+0000 7f9c876ac640 1 --2- 172.21.15.43:0/1236168323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c80101150 0x7f9c80117600 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000b470 tx=0x7f9c7000b930 comp rx=0 tx=0).stop 2024-09-11T23:27:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.633+0000 7f9c876ac640 1 -- 172.21.15.43:0/1236168323 shutdown_connections 2024-09-11T23:27:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.633+0000 7f9c876ac640 1 --2- 172.21.15.43:0/1236168323 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9c6c0a2ec0 0x7f9c6c0a5380 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.633+0000 7f9c876ac640 1 --2- 172.21.15.43:0/1236168323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c801080f0 0x7f9c80111130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.633+0000 7f9c876ac640 1 --2- 172.21.15.43:0/1236168323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c80101150 0x7f9c80117600 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.633+0000 7f9c876ac640 1 --2- 172.21.15.43:0/1236168323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c80100830 0x7f9c801170c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:27:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.633+0000 7f9c876ac640 1 -- 172.21.15.43:0/1236168323 >> 172.21.15.43:0/1236168323 conn(0x7f9c80075f20 msgr2=0x7f9c800ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:27:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.633+0000 7f9c876ac640 1 -- 172.21.15.43:0/1236168323 shutdown_connections 2024-09-11T23:27:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:27:51.634+0000 7f9c876ac640 1 -- 172.21.15.43:0/1236168323 wait complete. 2024-09-11T23:27:52.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:52 smithi097 ceph-mon[35562]: pgmap v184: 97 pgs: 97 active+clean; 20 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 MiB/s wr, 10.35k op/s 2024-09-11T23:27:52.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:52 smithi043 ceph-mon[30950]: pgmap v184: 97 pgs: 97 active+clean; 20 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 MiB/s wr, 10.35k op/s 2024-09-11T23:27:52.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:52 smithi090 ceph-mon[35130]: pgmap v184: 97 pgs: 97 active+clean; 20 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 MiB/s wr, 10.35k op/s 2024-09-11T23:27:54.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:54 smithi097 ceph-mon[35562]: pgmap v185: 97 pgs: 97 active+clean; 20 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 58 MiB/s wr, 10.31k op/s 2024-09-11T23:27:54.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:54 smithi043 ceph-mon[30950]: pgmap v185: 97 pgs: 97 active+clean; 20 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 58 MiB/s wr, 10.31k op/s 2024-09-11T23:27:54.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:54 smithi090 ceph-mon[35130]: pgmap v185: 97 pgs: 97 active+clean; 20 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 58 MiB/s wr, 10.31k op/s 2024-09-11T23:27:56.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:56 smithi097 ceph-mon[35562]: pgmap v186: 97 pgs: 97 active+clean; 21 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 MiB/s wr, 11.62k op/s 2024-09-11T23:27:56.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:56 smithi043 ceph-mon[30950]: pgmap v186: 97 pgs: 97 active+clean; 21 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 MiB/s wr, 11.62k op/s 2024-09-11T23:27:56.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:56 smithi090 ceph-mon[35130]: pgmap v186: 97 pgs: 97 active+clean; 21 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 MiB/s wr, 11.62k op/s 2024-09-11T23:27:58.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:27:58 smithi097 ceph-mon[35562]: pgmap v187: 97 pgs: 97 active+clean; 21 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 56 MiB/s wr, 10.03k op/s 2024-09-11T23:27:58.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:27:58 smithi043 ceph-mon[30950]: pgmap v187: 97 pgs: 97 active+clean; 21 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 56 MiB/s wr, 10.03k op/s 2024-09-11T23:27:58.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:27:58 smithi090 ceph-mon[35130]: pgmap v187: 97 pgs: 97 active+clean; 21 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 56 MiB/s wr, 10.03k op/s 2024-09-11T23:28:00.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:00 smithi097 ceph-mon[35562]: pgmap v188: 97 pgs: 97 active+clean; 21 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 55 MiB/s wr, 9.91k op/s 2024-09-11T23:28:00.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:00 smithi043 ceph-mon[30950]: pgmap v188: 97 pgs: 97 active+clean; 21 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 55 MiB/s wr, 9.91k op/s 2024-09-11T23:28:00.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:00 smithi090 ceph-mon[35130]: pgmap v188: 97 pgs: 97 active+clean; 21 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 55 MiB/s wr, 9.91k op/s 2024-09-11T23:28:02.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:02 smithi097 ceph-mon[35562]: pgmap v189: 97 pgs: 97 active+clean; 21 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 58 MiB/s wr, 10.49k op/s 2024-09-11T23:28:02.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:02 smithi043 ceph-mon[30950]: pgmap v189: 97 pgs: 97 active+clean; 21 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 58 MiB/s wr, 10.49k op/s 2024-09-11T23:28:02.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:02 smithi090 ceph-mon[35130]: pgmap v189: 97 pgs: 97 active+clean; 21 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 58 MiB/s wr, 10.49k op/s 2024-09-11T23:28:04.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:04 smithi097 ceph-mon[35562]: pgmap v190: 97 pgs: 97 active+clean; 21 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 37 MiB/s wr, 6.57k op/s 2024-09-11T23:28:04.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:04 smithi043 ceph-mon[30950]: pgmap v190: 97 pgs: 97 active+clean; 21 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 37 MiB/s wr, 6.57k op/s 2024-09-11T23:28:04.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:04 smithi090 ceph-mon[35130]: pgmap v190: 97 pgs: 97 active+clean; 21 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 37 MiB/s wr, 6.57k op/s 2024-09-11T23:28:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.742+0000 7fdbc57c4640 1 -- 172.21.15.43:0/2864282124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbc010e430 msgr2=0x7fdbc0110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.742+0000 7fdbc57c4640 1 --2- 172.21.15.43:0/2864282124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbc010e430 0x7fdbc0110820 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8009ca0 tx=0x7fdbb802a8c0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.742+0000 7fdbc57c4640 1 -- 172.21.15.43:0/2864282124 shutdown_connections 2024-09-11T23:28:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.742+0000 7fdbc57c4640 1 --2- 172.21.15.43:0/2864282124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbc010e430 0x7fdbc0110820 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.742+0000 7fdbc57c4640 1 --2- 172.21.15.43:0/2864282124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbc0101230 0x7fdbc010def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.742+0000 7fdbc57c4640 1 --2- 172.21.15.43:0/2864282124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbc0100880 0x7fdbc0100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.742+0000 7fdbc57c4640 1 -- 172.21.15.43:0/2864282124 >> 172.21.15.43:0/2864282124 conn(0x7fdbc00fc6b0 msgr2=0x7fdbc00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.742+0000 7fdbc57c4640 1 -- 172.21.15.43:0/2864282124 shutdown_connections 2024-09-11T23:28:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.743+0000 7fdbc57c4640 1 -- 172.21.15.43:0/2864282124 wait complete. 2024-09-11T23:28:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.743+0000 7fdbc57c4640 1 Processor -- start 2024-09-11T23:28:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.743+0000 7fdbc57c4640 1 -- start start 2024-09-11T23:28:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.743+0000 7fdbc57c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbc0100880 0x7fdbc019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.743+0000 7fdbc57c4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbc0101230 0x7fdbc019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.743+0000 7fdbc57c4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbc010e430 0x7fdbc019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.743+0000 7fdbc57c4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc0114080 con 0x7fdbc0100880 2024-09-11T23:28:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.743+0000 7fdbc57c4640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc0113f00 con 0x7fdbc010e430 2024-09-11T23:28:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.743+0000 7fdbc57c4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc0114200 con 0x7fdbc0101230 2024-09-11T23:28:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.744+0000 7fdbbeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbc0100880 0x7fdbc019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.744+0000 7fdbbeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbc0100880 0x7fdbc019a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40068/0 (socket says 172.21.15.43:40068) 2024-09-11T23:28:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.744+0000 7fdbbeffd640 1 -- 172.21.15.43:0/1099727895 learned_addr learned my addr 172.21.15.43:0/1099727895 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.744+0000 7fdbbe7fc640 1 --2- 172.21.15.43:0/1099727895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbc0101230 0x7fdbc019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.744+0000 7fdbbf7fe640 1 --2- 172.21.15.43:0/1099727895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbc010e430 0x7fdbc019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.744+0000 7fdbbf7fe640 1 -- 172.21.15.43:0/1099727895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbc0101230 msgr2=0x7fdbc019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.744+0000 7fdbbf7fe640 1 --2- 172.21.15.43:0/1099727895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbc0101230 0x7fdbc019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.744+0000 7fdbbf7fe640 1 -- 172.21.15.43:0/1099727895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbc0100880 msgr2=0x7fdbc019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.744+0000 7fdbbf7fe640 1 --2- 172.21.15.43:0/1099727895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbc0100880 0x7fdbc019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.744+0000 7fdbbf7fe640 1 -- 172.21.15.43:0/1099727895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc019f5a0 con 0x7fdbc010e430 2024-09-11T23:28:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.744+0000 7fdbbeffd640 1 --2- 172.21.15.43:0/1099727895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbc0100880 0x7fdbc019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.744+0000 7fdbbf7fe640 1 --2- 172.21.15.43:0/1099727895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbc010e430 0x7fdbc019ee70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdbb802ade0 tx=0x7fdbb80319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.744+0000 7fdba7fff640 1 -- 172.21.15.43:0/1099727895 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8003b70 con 0x7fdbc010e430 2024-09-11T23:28:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.744+0000 7fdba7fff640 1 -- 172.21.15.43:0/1099727895 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb8003d10 con 0x7fdbc010e430 2024-09-11T23:28:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.744+0000 7fdbc57c4640 1 -- 172.21.15.43:0/1099727895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc019f830 con 0x7fdbc010e430 2024-09-11T23:28:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.744+0000 7fdba7fff640 1 -- 172.21.15.43:0/1099727895 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb802fc40 con 0x7fdbc010e430 2024-09-11T23:28:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.744+0000 7fdbc57c4640 1 -- 172.21.15.43:0/1099727895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc01a7120 con 0x7fdbc010e430 2024-09-11T23:28:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.747+0000 7fdba7fff640 1 -- 172.21.15.43:0/1099727895 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fdbb8004050 con 0x7fdbc010e430 2024-09-11T23:28:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.747+0000 7fdbc863a740 1 Processor -- start 2024-09-11T23:28:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.747+0000 7fdbc863a740 1 -- start start 2024-09-11T23:28:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.747+0000 7fdbc863a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917ab9af0 0x557917ab9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.747+0000 7fdbc863a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917aba410 0x557917ac4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.747+0000 7fdbc863a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ac4cb0 0x557917ac70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.747+0000 7fdbc863a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557917a1e2f0 con 0x557917ab9af0 2024-09-11T23:28:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.747+0000 7fdbc863a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5579179c6270 con 0x557917aba410 2024-09-11T23:28:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.747+0000 7fdbc863a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557917a1a2b0 con 0x557917ac4cb0 2024-09-11T23:28:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.747+0000 7fdbbe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917ab9af0 0x557917ab9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.747+0000 7fdbbeffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917aba410 0x557917ac4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.747+0000 7fdbbe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917ab9af0 0x557917ab9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40076/0 (socket says 172.21.15.43:40076) 2024-09-11T23:28:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.747+0000 7fdbbe7fc640 1 -- 172.21.15.43:0/3999704776 learned_addr learned my addr 172.21.15.43:0/3999704776 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.747+0000 7fdbbeffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917aba410 0x557917ac4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:53326/0 (socket says 172.21.15.43:53326) 2024-09-11T23:28:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.747+0000 7fdba7fff640 1 --2- 172.21.15.43:0/1099727895 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdba80a2cf0 0x7fdba80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.747+0000 7fdbbe7fc640 1 --2- 172.21.15.43:0/3999704776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ac4cb0 0x557917ac70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.747+0000 7fdbbe7fc640 1 -- 172.21.15.43:0/3999704776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ac4cb0 msgr2=0x557917ac70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.748+0000 7fdbbe7fc640 1 --2- 172.21.15.43:0/3999704776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ac4cb0 0x557917ac70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.748+0000 7fdbbe7fc640 1 -- 172.21.15.43:0/3999704776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917aba410 msgr2=0x557917ac4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.748+0000 7fdbbe7fc640 1 --2- 172.21.15.43:0/3999704776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917aba410 0x557917ac4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.748+0000 7fdbbe7fc640 1 -- 172.21.15.43:0/3999704776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579179c5340 con 0x557917ab9af0 2024-09-11T23:28:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.748+0000 7fdbbf7fe640 1 --2- 172.21.15.43:0/1099727895 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdba80a2cf0 0x7fdba80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.748+0000 7fdbbe7fc640 1 --2- 172.21.15.43:0/3999704776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917ab9af0 0x557917ab9ed0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4005bc0 tx=0x7fdbb4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbe131730b85692 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.748+0000 7fdba7fff640 1 -- 172.21.15.43:0/1099727895 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7fdbb80f02f0 con 0x7fdbc010e430 2024-09-11T23:28:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.748+0000 7fdba57fa640 1 -- 172.21.15.43:0/3999704776 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4004050 con 0x557917ab9af0 2024-09-11T23:28:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.748+0000 7fdba57fa640 1 -- 172.21.15.43:0/3999704776 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb40041f0 con 0x557917ab9af0 2024-09-11T23:28:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.748+0000 7fdba57fa640 1 -- 172.21.15.43:0/3999704776 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb400f700 con 0x557917ab9af0 2024-09-11T23:28:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.749+0000 7fdbc863a740 1 -- 172.21.15.43:0/3999704776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917ab9af0 msgr2=0x557917ab9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.749+0000 7fdbc863a740 1 --2- 172.21.15.43:0/3999704776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917ab9af0 0x557917ab9ed0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4005bc0 tx=0x7fdbb4005850 comp rx=0 tx=0).stop 2024-09-11T23:28:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.749+0000 7fdbbf7fe640 1 --2- 172.21.15.43:0/1099727895 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdba80a2cf0 0x7fdba80a51b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80bc5c0 tx=0x7fdbb80874f0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.749+0000 7fdbc863a740 1 -- 172.21.15.43:0/3999704776 shutdown_connections 2024-09-11T23:28:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.749+0000 7fdbc863a740 1 --2- 172.21.15.43:0/3999704776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ac4cb0 0x557917ac70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.749+0000 7fdbc863a740 1 --2- 172.21.15.43:0/3999704776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917aba410 0x557917ac4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.749+0000 7fdbc863a740 1 --2- 172.21.15.43:0/3999704776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917ab9af0 0x557917ab9ed0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.749+0000 7fdbc863a740 1 -- 172.21.15.43:0/3999704776 >> 172.21.15.43:0/3999704776 conn(0x557917ab6060 msgr2=0x557917ab6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.749+0000 7fdbc863a740 1 -- 172.21.15.43:0/3999704776 shutdown_connections 2024-09-11T23:28:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.749+0000 7fdbc863a740 1 -- 172.21.15.43:0/3999704776 wait complete. 2024-09-11T23:28:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.749+0000 7fdbc863a740 1 -- start start 2024-09-11T23:28:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.749+0000 7fdbc863a740 1 Processor -- start 2024-09-11T23:28:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.750+0000 7fdbc863a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ab9af0 0x557917ac1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.750+0000 7fdbc863a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917aba410 0x557917ac1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.750+0000 7fdbc863a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917ac4cb0 0x557917ac2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.750+0000 7fdbc863a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557917ac7880 con 0x557917aba410 2024-09-11T23:28:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.750+0000 7fdbc863a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5579179a7290 con 0x557917ac4cb0 2024-09-11T23:28:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.750+0000 7fdbc863a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557917ac7a00 con 0x557917ab9af0 2024-09-11T23:28:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.750+0000 7fdbc863a740 10 client.0 _ll_register_callbacks cb 0x5579179d8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.750+0000 7fdbbe7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ab9af0 0x557917ac1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.750+0000 7fdbbeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917aba410 0x557917ac1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.750+0000 7fdbbe7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ab9af0 0x557917ac1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:36862/0 (socket says 172.21.15.43:36862) 2024-09-11T23:28:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.750+0000 7fdbbe7fc640 1 -- 172.21.15.43:0/2001609206 learned_addr learned my addr 172.21.15.43:0/2001609206 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.750+0000 7fdbbe7fc640 1 --2- 172.21.15.43:0/2001609206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917ac4cb0 0x557917ac2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.750+0000 7fdbbeffd640 1 -- 172.21.15.43:0/2001609206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ab9af0 msgr2=0x557917ac1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.750+0000 7fdbbeffd640 1 --2- 172.21.15.43:0/2001609206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ab9af0 0x557917ac1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.750+0000 7fdbbeffd640 1 -- 172.21.15.43:0/2001609206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917ac4cb0 msgr2=0x557917ac2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.750+0000 7fdbbeffd640 1 --2- 172.21.15.43:0/2001609206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917ac4cb0 0x557917ac2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.750+0000 7fdbbeffd640 1 -- 172.21.15.43:0/2001609206 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557917731210 con 0x557917aba410 2024-09-11T23:28:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.750+0000 7fdbbe7fc640 1 --2- 172.21.15.43:0/2001609206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ab9af0 0x557917ac1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.750+0000 7fdbbeffd640 1 --2- 172.21.15.43:0/2001609206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917aba410 0x557917ac1ff0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0012cf0 tx=0x7fdbb000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93f9993d7fc901d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.750+0000 7fdba2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:28:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.751+0000 7fdba2ff5640 1 -- 172.21.15.43:0/2001609206 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0005020 con 0x557917aba410 2024-09-11T23:28:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.751+0000 7fdba2ff5640 1 -- 172.21.15.43:0/2001609206 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7fdbb00051c0 con 0x557917aba410 2024-09-11T23:28:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.751+0000 7fdbc863a740 1 -- 172.21.15.43:0/2001609206 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbb40041f0 con 0x557917aba410 2024-09-11T23:28:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.751+0000 7fdba2ff5640 1 -- 172.21.15.43:0/2001609206 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdbb00053c0 con 0x557917aba410 2024-09-11T23:28:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.751+0000 7fdbc863a740 10 client.16350 fetch_fsmap learned FSMap version 30 2024-09-11T23:28:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.751+0000 7fdbc863a740 1 -- 172.21.15.43:0/2001609206 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5579177fd620 con 0x557917aba410 2024-09-11T23:28:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.751+0000 7fdba2ff5640 1 -- 172.21.15.43:0/2001609206 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb40041f0 con 0x557917aba410 2024-09-11T23:28:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.751+0000 7fdba2ff5640 1 -- 172.21.15.43:0/2001609206 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fdbb0025480 con 0x557917aba410 2024-09-11T23:28:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.751+0000 7fdbc863a740 10 client.16350 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:28:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.751+0000 7fdbc863a740 10 client.16350 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:28:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.752+0000 7fdbc863a740 20 client.16350 populate_metadata read hostname 'smithi043' 2024-09-11T23:28:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.752+0000 7fdbc863a740 1 --2- 172.21.15.43:0/2001609206 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x557917ad4b80 0x557917ad6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.752+0000 7fdbc863a740 4 client.16350 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:28:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.752+0000 7fdbc863a740 1 -- 172.21.15.43:0/2001609206 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557917a2ba20 con 0x557917ad4b80 2024-09-11T23:28:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.752+0000 7fdbbe7fc640 1 --2- 172.21.15.43:0/2001609206 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x557917ad4b80 0x557917ad6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.752+0000 7fdbbe7fc640 1 --2- 172.21.15.43:0/2001609206 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x557917ad4b80 0x557917ad6f70 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7e2909fd7831c2f server_cookie=90bc6b9fc18c5c42 in_seq=0 out_seq=0 2024-09-11T23:28:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.752+0000 7fdba2ff5640 10 client.16350 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.753+0000 7fdba2ff5640 1 -- 172.21.15.43:0/2001609206 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557917a2ba20 con 0x557917ad4b80 2024-09-11T23:28:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.753+0000 7fdba2ff5640 10 client.16350 handle_command_reply: tid=0 2024-09-11T23:28:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.753+0000 7fdbc863a740 1 client.16350 shutdown 2024-09-11T23:28:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.753+0000 7fdbc863a740 1 -- 172.21.15.43:0/2001609206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917aba410 msgr2=0x557917ac1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.753+0000 7fdbc863a740 1 --2- 172.21.15.43:0/2001609206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917aba410 0x557917ac1ff0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0012cf0 tx=0x7fdbb000f790 comp rx=0 tx=0).stop 2024-09-11T23:28:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.753+0000 7fdbc863a740 1 -- 172.21.15.43:0/2001609206 shutdown_connections 2024-09-11T23:28:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.753+0000 7fdbc863a740 1 --2- 172.21.15.43:0/2001609206 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x557917ad4b80 0x557917ad6f70 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.753+0000 7fdbc863a740 1 --2- 172.21.15.43:0/2001609206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917ac4cb0 0x557917ac2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.753+0000 7fdbc863a740 1 --2- 172.21.15.43:0/2001609206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917aba410 0x557917ac1ff0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.753+0000 7fdba2ff5640 0 client.16350 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.753+0000 7fdbc863a740 1 --2- 172.21.15.43:0/2001609206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ab9af0 0x557917ac1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.753+0000 7fdbc863a740 1 -- 172.21.15.43:0/2001609206 >> 172.21.15.43:0/2001609206 conn(0x557917ab6060 msgr2=0x557917abbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.753+0000 7fdbc863a740 1 -- 172.21.15.43:0/2001609206 shutdown_connections 2024-09-11T23:28:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.753+0000 7fdbc863a740 1 -- 172.21.15.43:0/2001609206 wait complete. 2024-09-11T23:28:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.753+0000 7fdbc863a740 20 client.16350 trim_cache size 0 max 16384 2024-09-11T23:28:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.768+0000 7fdbc863a740 1 Processor -- start 2024-09-11T23:28:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.768+0000 7fdbc863a740 1 -- start start 2024-09-11T23:28:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.768+0000 7fdbc863a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ab9af0 0x557917abc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.768+0000 7fdbc863a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917aba410 0x557917ac0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.768+0000 7fdbc863a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917ac4cb0 0x557917ac2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.768+0000 7fdbc863a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc01b23b0 con 0x557917ac4cb0 2024-09-11T23:28:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.768+0000 7fdbc863a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc01afc80 con 0x557917aba410 2024-09-11T23:28:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.769+0000 7fdbc863a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc01adad0 con 0x557917ab9af0 2024-09-11T23:28:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.769+0000 7fdbbe7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ab9af0 0x557917abc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.769+0000 7fdbbe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917ac4cb0 0x557917ac2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.769+0000 7fdbbeffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917aba410 0x557917ac0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.769+0000 7fdbbe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917ac4cb0 0x557917ac2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40104/0 (socket says 172.21.15.43:40104) 2024-09-11T23:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.769+0000 7fdbbeffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917aba410 0x557917ac0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:53344/0 (socket says 172.21.15.43:53344) 2024-09-11T23:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.769+0000 7fdbbeffd640 1 -- 172.21.15.43:0/3070257652 learned_addr learned my addr 172.21.15.43:0/3070257652 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.769+0000 7fdbbeffd640 1 -- 172.21.15.43:0/3070257652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ab9af0 msgr2=0x557917abc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.769+0000 7fdbbeffd640 1 --2- 172.21.15.43:0/3070257652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ab9af0 0x557917abc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.769+0000 7fdbbeffd640 1 -- 172.21.15.43:0/3070257652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917ac4cb0 msgr2=0x557917ac2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.769+0000 7fdbbeffd640 1 --2- 172.21.15.43:0/3070257652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917ac4cb0 0x557917ac2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.769+0000 7fdbbeffd640 1 -- 172.21.15.43:0/3070257652 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557917ab5e90 con 0x557917aba410 2024-09-11T23:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.769+0000 7fdbbe7fc640 1 --2- 172.21.15.43:0/3070257652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917ac4cb0 0x557917ac2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.769+0000 7fdbbeffd640 1 --2- 172.21.15.43:0/3070257652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917aba410 0x557917ac0360 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0012870 tx=0x7fdbb0005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27a3d56fcb946cb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.770+0000 7fdba4ff9640 1 -- 172.21.15.43:0/3070257652 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb002a030 con 0x557917aba410 2024-09-11T23:28:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.770+0000 7fdba4ff9640 1 -- 172.21.15.43:0/3070257652 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb00050c0 con 0x557917aba410 2024-09-11T23:28:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.770+0000 7fdba4ff9640 1 -- 172.21.15.43:0/3070257652 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0025ac0 con 0x557917aba410 2024-09-11T23:28:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.770+0000 7fdbc863a740 1 -- 172.21.15.43:0/3070257652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917aba410 msgr2=0x557917ac0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.770+0000 7fdbc863a740 1 --2- 172.21.15.43:0/3070257652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917aba410 0x557917ac0360 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0012870 tx=0x7fdbb0005770 comp rx=0 tx=0).stop 2024-09-11T23:28:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.770+0000 7fdbc863a740 1 -- 172.21.15.43:0/3070257652 shutdown_connections 2024-09-11T23:28:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.770+0000 7fdbc863a740 1 --2- 172.21.15.43:0/3070257652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917ac4cb0 0x557917ac2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.770+0000 7fdbc863a740 1 --2- 172.21.15.43:0/3070257652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917aba410 0x557917ac0360 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.770+0000 7fdbc863a740 1 --2- 172.21.15.43:0/3070257652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ab9af0 0x557917abc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.770+0000 7fdbc863a740 1 -- 172.21.15.43:0/3070257652 >> 172.21.15.43:0/3070257652 conn(0x557917ab6060 msgr2=0x557917ac3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.770+0000 7fdbc863a740 1 -- 172.21.15.43:0/3070257652 shutdown_connections 2024-09-11T23:28:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.770+0000 7fdbc863a740 1 -- 172.21.15.43:0/3070257652 wait complete. 2024-09-11T23:28:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.771+0000 7fdbc863a740 1 -- start start 2024-09-11T23:28:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.771+0000 7fdbc863a740 1 Processor -- start 2024-09-11T23:28:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.771+0000 7fdbc863a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ab9af0 0x557917acd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.771+0000 7fdbc863a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917aba410 0x557917acd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.771+0000 7fdbc863a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917ac4cb0 0x557917abe6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.771+0000 7fdbc863a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc01ae140 con 0x557917ac4cb0 2024-09-11T23:28:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.771+0000 7fdbc863a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc01af5b0 con 0x557917aba410 2024-09-11T23:28:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.771+0000 7fdbc863a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc01ae810 con 0x557917ab9af0 2024-09-11T23:28:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.771+0000 7fdbc863a740 10 client.0 _ll_register_callbacks cb 0x7fdbb0009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.771+0000 7fdbbeffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917aba410 0x557917acd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.771+0000 7fdbbe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917ac4cb0 0x557917abe6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.771+0000 7fdbbeffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917aba410 0x557917acd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:53346/0 (socket says 172.21.15.43:53346) 2024-09-11T23:28:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.771+0000 7fdbbeffd640 1 -- 172.21.15.43:0/58296394 learned_addr learned my addr 172.21.15.43:0/58296394 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.771+0000 7fdbbe7fc640 1 --2- 172.21.15.43:0/58296394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ab9af0 0x557917acd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.772+0000 7fdbbeffd640 1 -- 172.21.15.43:0/58296394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ab9af0 msgr2=0x557917acd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.772+0000 7fdbbeffd640 1 --2- 172.21.15.43:0/58296394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ab9af0 0x557917acd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.772+0000 7fdbbeffd640 1 -- 172.21.15.43:0/58296394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917ac4cb0 msgr2=0x557917abe6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.772+0000 7fdbbeffd640 1 --2- 172.21.15.43:0/58296394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917ac4cb0 0x557917abe6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.772+0000 7fdbbeffd640 1 -- 172.21.15.43:0/58296394 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557917ab7110 con 0x557917aba410 2024-09-11T23:28:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.772+0000 7fdbbe7fc640 1 --2- 172.21.15.43:0/58296394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917ac4cb0 0x557917abe6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.772+0000 7fdbbeffd640 1 --2- 172.21.15.43:0/58296394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917aba410 0x557917acd5a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000f7d0 tx=0x7fdbb0005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53118300f91abd18 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.772+0000 7fdba2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:28:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.772+0000 7fdba2ff5640 1 -- 172.21.15.43:0/58296394 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0033060 con 0x557917aba410 2024-09-11T23:28:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.772+0000 7fdbc863a740 1 -- 172.21.15.43:0/58296394 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbb00050c0 con 0x557917aba410 2024-09-11T23:28:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.772+0000 7fdba2ff5640 1 -- 172.21.15.43:0/58296394 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7fdbb0025c50 con 0x557917aba410 2024-09-11T23:28:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.772+0000 7fdba2ff5640 1 -- 172.21.15.43:0/58296394 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdbb002a070 con 0x557917aba410 2024-09-11T23:28:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.772+0000 7fdbc863a740 10 client.25871 fetch_fsmap learned FSMap version 30 2024-09-11T23:28:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.772+0000 7fdbc863a740 1 -- 172.21.15.43:0/58296394 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557917b3f320 con 0x557917aba410 2024-09-11T23:28:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.772+0000 7fdba2ff5640 1 -- 172.21.15.43:0/58296394 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb00050c0 con 0x557917aba410 2024-09-11T23:28:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.772+0000 7fdba2ff5640 1 -- 172.21.15.43:0/58296394 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fdbb001c610 con 0x557917aba410 2024-09-11T23:28:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.773+0000 7fdbc863a740 10 client.25871 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:28:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.773+0000 7fdbc863a740 10 client.25871 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:28:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.773+0000 7fdbc863a740 20 client.25871 populate_metadata read hostname 'smithi043' 2024-09-11T23:28:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.773+0000 7fdbc863a740 1 --2- 172.21.15.43:0/58296394 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x557917b3f9b0 0x557917b41dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.773+0000 7fdbc863a740 4 client.25871 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:28:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.773+0000 7fdbc863a740 1 -- 172.21.15.43:0/58296394 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557917a25d70 con 0x557917b3f9b0 2024-09-11T23:28:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.773+0000 7fdbbe7fc640 1 --2- 172.21.15.43:0/58296394 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x557917b3f9b0 0x557917b41dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:04.779 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.773+0000 7fdba2ff5640 10 client.25871 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:04.779 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:04.773+0000 7fdbbe7fc640 1 --2- 172.21.15.43:0/58296394 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x557917b3f9b0 0x557917b41dd0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37516b712784df5a server_cookie=58b020de43bcbd63 in_seq=0 out_seq=0 2024-09-11T23:28:06.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:06 smithi097 ceph-mon[35562]: pgmap v191: 97 pgs: 97 active+clean; 21 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 34 MiB/s wr, 6.19k op/s 2024-09-11T23:28:06.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:06 smithi043 ceph-mon[30950]: pgmap v191: 97 pgs: 97 active+clean; 21 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 34 MiB/s wr, 6.19k op/s 2024-09-11T23:28:06.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:06 smithi090 ceph-mon[35130]: pgmap v191: 97 pgs: 97 active+clean; 21 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 34 MiB/s wr, 6.19k op/s 2024-09-11T23:28:08.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:08 smithi097 ceph-mon[35562]: pgmap v192: 97 pgs: 97 active+clean; 21 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 26 MiB/s wr, 4.65k op/s 2024-09-11T23:28:08.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:08 smithi043 ceph-mon[30950]: pgmap v192: 97 pgs: 97 active+clean; 21 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 26 MiB/s wr, 4.65k op/s 2024-09-11T23:28:08.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:08 smithi090 ceph-mon[35130]: pgmap v192: 97 pgs: 97 active+clean; 21 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 26 MiB/s wr, 4.65k op/s 2024-09-11T23:28:10.277 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:28:10.300 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:10 smithi097 ceph-mon[35562]: pgmap v193: 97 pgs: 97 active+clean; 21 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 20 MiB/s wr, 3.59k op/s 2024-09-11T23:28:10.302 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:10 smithi043 ceph-mon[30950]: pgmap v193: 97 pgs: 97 active+clean; 21 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 20 MiB/s wr, 3.59k op/s 2024-09-11T23:28:10.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:10 smithi090 ceph-mon[35130]: pgmap v193: 97 pgs: 97 active+clean; 21 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 20 MiB/s wr, 3.59k op/s 2024-09-11T23:28:10.732 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:28:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.250+0000 7fdba2ff5640 1 -- 172.21.15.43:0/58296394 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557917a25d70 con 0x557917b3f9b0 2024-09-11T23:28:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.250+0000 7fdba2ff5640 10 client.25871 handle_command_reply: tid=0 2024-09-11T23:28:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.250+0000 7fdbc863a740 1 client.25871 shutdown 2024-09-11T23:28:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.250+0000 7fdbc863a740 1 -- 172.21.15.43:0/58296394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917aba410 msgr2=0x557917acd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.250+0000 7fdbc863a740 1 --2- 172.21.15.43:0/58296394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917aba410 0x557917acd5a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000f7d0 tx=0x7fdbb0005270 comp rx=0 tx=0).stop 2024-09-11T23:28:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.250+0000 7fdbc863a740 1 -- 172.21.15.43:0/58296394 shutdown_connections 2024-09-11T23:28:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.250+0000 7fdbc863a740 1 --2- 172.21.15.43:0/58296394 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x557917b3f9b0 0x557917b41dd0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.250+0000 7fdbc863a740 1 --2- 172.21.15.43:0/58296394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557917ac4cb0 0x557917abe6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.250+0000 7fdbc863a740 1 --2- 172.21.15.43:0/58296394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557917aba410 0x557917acd5a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.250+0000 7fdbc863a740 1 --2- 172.21.15.43:0/58296394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557917ab9af0 0x557917acd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.250+0000 7fdbc863a740 1 -- 172.21.15.43:0/58296394 >> 172.21.15.43:0/58296394 conn(0x557917ab6060 msgr2=0x557917ab49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.250+0000 7fdba2ff5640 0 client.25871 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.250+0000 7fdbc863a740 1 -- 172.21.15.43:0/58296394 shutdown_connections 2024-09-11T23:28:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.250+0000 7fdbc863a740 1 -- 172.21.15.43:0/58296394 wait complete. 2024-09-11T23:28:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.250+0000 7fdbc863a740 20 client.25871 trim_cache size 0 max 16384 2024-09-11T23:28:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:28:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:28:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:28:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:28:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.251+0000 7fdbc57c4640 1 -- 172.21.15.43:0/1099727895 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdba80a2cf0 msgr2=0x7fdba80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.251+0000 7fdbc57c4640 1 --2- 172.21.15.43:0/1099727895 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdba80a2cf0 0x7fdba80a51b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80bc5c0 tx=0x7fdbb80874f0 comp rx=0 tx=0).stop 2024-09-11T23:28:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.251+0000 7fdbc57c4640 1 -- 172.21.15.43:0/1099727895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbc010e430 msgr2=0x7fdbc019ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.251+0000 7fdbc57c4640 1 --2- 172.21.15.43:0/1099727895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbc010e430 0x7fdbc019ee70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdbb802ade0 tx=0x7fdbb80319f0 comp rx=0 tx=0).stop 2024-09-11T23:28:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.251+0000 7fdbc57c4640 1 -- 172.21.15.43:0/1099727895 shutdown_connections 2024-09-11T23:28:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.251+0000 7fdbc57c4640 1 --2- 172.21.15.43:0/1099727895 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdba80a2cf0 0x7fdba80a51b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.251+0000 7fdbc57c4640 1 --2- 172.21.15.43:0/1099727895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbc010e430 0x7fdbc019ee70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.251+0000 7fdbc57c4640 1 --2- 172.21.15.43:0/1099727895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbc0101230 0x7fdbc019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.251+0000 7fdbc57c4640 1 --2- 172.21.15.43:0/1099727895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbc0100880 0x7fdbc019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.251+0000 7fdbc57c4640 1 -- 172.21.15.43:0/1099727895 >> 172.21.15.43:0/1099727895 conn(0x7fdbc00fc6b0 msgr2=0x7fdbc00feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.251+0000 7fdbc57c4640 1 -- 172.21.15.43:0/1099727895 shutdown_connections 2024-09-11T23:28:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:11.252+0000 7fdbc57c4640 1 -- 172.21.15.43:0/1099727895 wait complete. 2024-09-11T23:28:12.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:12 smithi097 ceph-mon[35562]: pgmap v194: 97 pgs: 97 active+clean; 21 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 18 MiB/s wr, 3.29k op/s 2024-09-11T23:28:12.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:12 smithi043 ceph-mon[30950]: pgmap v194: 97 pgs: 97 active+clean; 21 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 18 MiB/s wr, 3.29k op/s 2024-09-11T23:28:12.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:12 smithi090 ceph-mon[35130]: pgmap v194: 97 pgs: 97 active+clean; 21 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 18 MiB/s wr, 3.29k op/s 2024-09-11T23:28:14.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:14 smithi097 ceph-mon[35562]: pgmap v195: 97 pgs: 97 active+clean; 21 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 14 MiB/s wr, 2.60k op/s 2024-09-11T23:28:14.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:14 smithi043 ceph-mon[30950]: pgmap v195: 97 pgs: 97 active+clean; 21 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 14 MiB/s wr, 2.60k op/s 2024-09-11T23:28:14.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:14 smithi090 ceph-mon[35130]: pgmap v195: 97 pgs: 97 active+clean; 21 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 14 MiB/s wr, 2.60k op/s 2024-09-11T23:28:15.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.551+0000 7f69ffb53640 1 -- 172.21.15.43:0/2074095800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f8102d40 msgr2=0x7f69f8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:15.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.551+0000 7f69ffb53640 1 --2- 172.21.15.43:0/2074095800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f8102d40 0x7f69f8103120 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009900 tx=0x7f69f402a480 comp rx=0 tx=0).stop 2024-09-11T23:28:15.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.551+0000 7f69ffb53640 1 -- 172.21.15.43:0/2074095800 shutdown_connections 2024-09-11T23:28:15.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.551+0000 7f69ffb53640 1 --2- 172.21.15.43:0/2074095800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69f810c170 0x7f69f810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.551+0000 7f69ffb53640 1 --2- 172.21.15.43:0/2074095800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69f8103660 0x7f69f810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.551+0000 7f69ffb53640 1 --2- 172.21.15.43:0/2074095800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f8102d40 0x7f69f8103120 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.551+0000 7f69ffb53640 1 -- 172.21.15.43:0/2074095800 >> 172.21.15.43:0/2074095800 conn(0x7f69f80fc7a0 msgr2=0x7f69f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.551+0000 7f69ffb53640 1 -- 172.21.15.43:0/2074095800 shutdown_connections 2024-09-11T23:28:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.551+0000 7f69ffb53640 1 -- 172.21.15.43:0/2074095800 wait complete. 2024-09-11T23:28:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.552+0000 7f69ffb53640 1 Processor -- start 2024-09-11T23:28:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.552+0000 7f69ffb53640 1 -- start start 2024-09-11T23:28:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.552+0000 7f69ffb53640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69f8102d40 0x7f69f81984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.552+0000 7f69ffb53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f8103660 0x7f69f8198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.552+0000 7f69ffb53640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69f810c170 0x7f69f819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.552+0000 7f69ffb53640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f69f8111f70 con 0x7f69f8103660 2024-09-11T23:28:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.552+0000 7f69ffb53640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f69f8111df0 con 0x7f69f810c170 2024-09-11T23:28:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.552+0000 7f69ffb53640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f69f81120f0 con 0x7f69f8102d40 2024-09-11T23:28:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.552+0000 7f69fd8ef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69f8102d40 0x7f69f81984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.552+0000 7f69fd8ef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69f8102d40 0x7f69f81984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:52346/0 (socket says 172.21.15.43:52346) 2024-09-11T23:28:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.552+0000 7f69fd0ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f8103660 0x7f69f8198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.552+0000 7f69fe0f0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69f810c170 0x7f69f819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.552+0000 7f69fd8ef640 1 -- 172.21.15.43:0/1955727655 learned_addr learned my addr 172.21.15.43:0/1955727655 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.553+0000 7f69fe0f0640 1 -- 172.21.15.43:0/1955727655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69f8102d40 msgr2=0x7f69f81984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.553+0000 7f69fe0f0640 1 --2- 172.21.15.43:0/1955727655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69f8102d40 0x7f69f81984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.553+0000 7f69fe0f0640 1 -- 172.21.15.43:0/1955727655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f8103660 msgr2=0x7f69f8198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.553+0000 7f69fe0f0640 1 --2- 172.21.15.43:0/1955727655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f8103660 0x7f69f8198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.553+0000 7f69fe0f0640 1 -- 172.21.15.43:0/1955727655 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f819d490 con 0x7f69f810c170 2024-09-11T23:28:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.553+0000 7f69fe0f0640 1 --2- 172.21.15.43:0/1955727655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69f810c170 0x7f69f819cdb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f69f0009fd0 tx=0x7f69f000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.553+0000 7f69eeffd640 1 -- 172.21.15.43:0/1955727655 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f000ce20 con 0x7f69f810c170 2024-09-11T23:28:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.553+0000 7f69eeffd640 1 -- 172.21.15.43:0/1955727655 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69f0005020 con 0x7f69f810c170 2024-09-11T23:28:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.553+0000 7f69ffb53640 1 -- 172.21.15.43:0/1955727655 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f819d780 con 0x7f69f810c170 2024-09-11T23:28:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.553+0000 7f69eeffd640 1 -- 172.21.15.43:0/1955727655 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f0010720 con 0x7f69f810c170 2024-09-11T23:28:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.553+0000 7f69ffb53640 1 -- 172.21.15.43:0/1955727655 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f81a5060 con 0x7f69f810c170 2024-09-11T23:28:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.555+0000 7f69eeffd640 1 -- 172.21.15.43:0/1955727655 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f69f0010900 con 0x7f69f810c170 2024-09-11T23:28:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.556+0000 7f6a01f2c740 1 Processor -- start 2024-09-11T23:28:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.556+0000 7f6a01f2c740 1 -- start start 2024-09-11T23:28:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.556+0000 7f6a01f2c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bac4a0 0x55c5f5bac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.556+0000 7f6a01f2c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bacdc0 0x55c5f5bb7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.556+0000 7f6a01f2c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c5f5bb7680 0x55c5f5bb9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.556+0000 7f6a01f2c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c5f5b122e0 con 0x55c5f5bacdc0 2024-09-11T23:28:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.556+0000 7f6a01f2c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c5f5ab9ce0 con 0x55c5f5bac4a0 2024-09-11T23:28:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.556+0000 7f6a01f2c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c5f5b0e2a0 con 0x55c5f5bb7680 2024-09-11T23:28:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.556+0000 7f69fd8ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bacdc0 0x55c5f5bb7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.556+0000 7f69fd8ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bacdc0 0x55c5f5bb7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51328/0 (socket says 172.21.15.43:51328) 2024-09-11T23:28:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.556+0000 7f69fd8ef640 1 -- 172.21.15.43:0/1565728684 learned_addr learned my addr 172.21.15.43:0/1565728684 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.556+0000 7f69fd0ee640 1 --2- 172.21.15.43:0/1565728684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bac4a0 0x55c5f5bac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.557+0000 7f69fd0ee640 1 --2- 172.21.15.43:0/1565728684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c5f5bb7680 0x55c5f5bb9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.557+0000 7f69eeffd640 1 --2- 172.21.15.43:0/1955727655 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f69e40a2cf0 0x7f69e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.557+0000 7f69fd8ef640 1 -- 172.21.15.43:0/1565728684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c5f5bb7680 msgr2=0x55c5f5bb9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.557+0000 7f69fd8ef640 1 --2- 172.21.15.43:0/1565728684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c5f5bb7680 0x55c5f5bb9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.557+0000 7f69fd8ef640 1 -- 172.21.15.43:0/1565728684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bac4a0 msgr2=0x55c5f5bac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.557+0000 7f69fd8ef640 1 --2- 172.21.15.43:0/1565728684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bac4a0 0x55c5f5bac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.557+0000 7f69fd8ef640 1 -- 172.21.15.43:0/1565728684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5f5a9e440 con 0x55c5f5bacdc0 2024-09-11T23:28:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.557+0000 7f69fd0ee640 1 --2- 172.21.15.43:0/1565728684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bac4a0 0x55c5f5bac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.557+0000 7f69fe0f0640 1 --2- 172.21.15.43:0/1955727655 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f69e40a2cf0 0x7f69e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.557+0000 7f69eeffd640 1 -- 172.21.15.43:0/1955727655 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f69f00cfb20 con 0x7f69f810c170 2024-09-11T23:28:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.557+0000 7f69fd8ef640 1 --2- 172.21.15.43:0/1565728684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bacdc0 0x55c5f5bb7140 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009a20 tx=0x7f69f4005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaba2a059d53b267 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.557+0000 7f69e3fff640 1 -- 172.21.15.43:0/1565728684 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4040070 con 0x55c5f5bacdc0 2024-09-11T23:28:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.558+0000 7f69e3fff640 1 -- 172.21.15.43:0/1565728684 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69f4004030 con 0x55c5f5bacdc0 2024-09-11T23:28:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.558+0000 7f69e3fff640 1 -- 172.21.15.43:0/1565728684 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4035040 con 0x55c5f5bacdc0 2024-09-11T23:28:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.558+0000 7f6a01f2c740 1 -- 172.21.15.43:0/1565728684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bacdc0 msgr2=0x55c5f5bb7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.558+0000 7f69fe0f0640 1 --2- 172.21.15.43:0/1955727655 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f69e40a2cf0 0x7f69e40a51b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f69f00cd320 tx=0x7f69f00bc040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.558+0000 7f6a01f2c740 1 --2- 172.21.15.43:0/1565728684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bacdc0 0x55c5f5bb7140 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009a20 tx=0x7f69f4005dd0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.558+0000 7f6a01f2c740 1 -- 172.21.15.43:0/1565728684 shutdown_connections 2024-09-11T23:28:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.558+0000 7f6a01f2c740 1 --2- 172.21.15.43:0/1565728684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c5f5bb7680 0x55c5f5bb9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.558+0000 7f6a01f2c740 1 --2- 172.21.15.43:0/1565728684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bacdc0 0x55c5f5bb7140 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.558+0000 7f6a01f2c740 1 --2- 172.21.15.43:0/1565728684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bac4a0 0x55c5f5bac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.558+0000 7f6a01f2c740 1 -- 172.21.15.43:0/1565728684 >> 172.21.15.43:0/1565728684 conn(0x55c5f5ba8a30 msgr2=0x55c5f5ba8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.559+0000 7f6a01f2c740 1 -- 172.21.15.43:0/1565728684 shutdown_connections 2024-09-11T23:28:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.559+0000 7f6a01f2c740 1 -- 172.21.15.43:0/1565728684 wait complete. 2024-09-11T23:28:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.559+0000 7f6a01f2c740 1 -- start start 2024-09-11T23:28:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.559+0000 7f6a01f2c740 1 Processor -- start 2024-09-11T23:28:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.560+0000 7f6a01f2c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bac4a0 0x55c5f5bb6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.560+0000 7f6a01f2c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bacdc0 0x55c5f5bb4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.560+0000 7f6a01f2c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c5f5bb7680 0x55c5f5bb4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.560+0000 7f6a01f2c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c5f5bba250 con 0x55c5f5bacdc0 2024-09-11T23:28:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.560+0000 7f6a01f2c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c5f5acd050 con 0x55c5f5bac4a0 2024-09-11T23:28:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.560+0000 7f69fd0ee640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bac4a0 0x55c5f5bb6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.560+0000 7f6a01f2c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c5f5bba3d0 con 0x55c5f5bb7680 2024-09-11T23:28:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.560+0000 7f69fd8ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bacdc0 0x55c5f5bb4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.560+0000 7f69fd8ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bacdc0 0x55c5f5bb4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51338/0 (socket says 172.21.15.43:51338) 2024-09-11T23:28:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.560+0000 7f69fd8ef640 1 -- 172.21.15.43:0/499536321 learned_addr learned my addr 172.21.15.43:0/499536321 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.560+0000 7f6a01f2c740 10 client.0 _ll_register_callbacks cb 0x55c5f5b0d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.560+0000 7f69fd0ee640 1 --2- 172.21.15.43:0/499536321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c5f5bb7680 0x55c5f5bb4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.560+0000 7f69fd8ef640 1 -- 172.21.15.43:0/499536321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c5f5bb7680 msgr2=0x55c5f5bb4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.560+0000 7f69fd8ef640 1 --2- 172.21.15.43:0/499536321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c5f5bb7680 0x55c5f5bb4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.560+0000 7f69fd8ef640 1 -- 172.21.15.43:0/499536321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bac4a0 msgr2=0x55c5f5bb6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.560+0000 7f69fd8ef640 1 --2- 172.21.15.43:0/499536321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bac4a0 0x55c5f5bb6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.560+0000 7f69fd8ef640 1 -- 172.21.15.43:0/499536321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5f58833c0 con 0x55c5f5bacdc0 2024-09-11T23:28:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.560+0000 7f69fd0ee640 1 --2- 172.21.15.43:0/499536321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c5f5bb7680 0x55c5f5bb4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:28:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.560+0000 7f69fd8ef640 1 --2- 172.21.15.43:0/499536321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bacdc0 0x55c5f5bb4220 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f69f4031550 tx=0x7f69f4009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af961d8a1284e930 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.560+0000 7f69e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:28:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.561+0000 7f69e17fa640 1 -- 172.21.15.43:0/499536321 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4040070 con 0x55c5f5bacdc0 2024-09-11T23:28:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.561+0000 7f69e17fa640 1 -- 172.21.15.43:0/499536321 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f69f40317b0 con 0x55c5f5bacdc0 2024-09-11T23:28:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.561+0000 7f6a01f2c740 1 -- 172.21.15.43:0/499536321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69f4004030 con 0x55c5f5bacdc0 2024-09-11T23:28:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.561+0000 7f69e17fa640 1 -- 172.21.15.43:0/499536321 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f69f4056050 con 0x55c5f5bacdc0 2024-09-11T23:28:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.561+0000 7f6a01f2c740 10 client.16368 fetch_fsmap learned FSMap version 30 2024-09-11T23:28:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.561+0000 7f6a01f2c740 1 -- 172.21.15.43:0/499536321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c5f5bc5ba0 con 0x55c5f5bacdc0 2024-09-11T23:28:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.561+0000 7f69e17fa640 1 -- 172.21.15.43:0/499536321 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69f4004030 con 0x55c5f5bacdc0 2024-09-11T23:28:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.561+0000 7f69e17fa640 1 -- 172.21.15.43:0/499536321 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f69f40035f0 con 0x55c5f5bacdc0 2024-09-11T23:28:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.561+0000 7f6a01f2c740 10 client.16368 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:28:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.561+0000 7f6a01f2c740 10 client.16368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:28:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.562+0000 7f6a01f2c740 20 client.16368 populate_metadata read hostname 'smithi043' 2024-09-11T23:28:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.562+0000 7f6a01f2c740 1 --2- 172.21.15.43:0/499536321 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c5f5bc6ef0 0x55c5f5bc92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.562+0000 7f6a01f2c740 4 client.16368 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:28:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.562+0000 7f6a01f2c740 1 -- 172.21.15.43:0/499536321 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5f5b1f850 con 0x55c5f5bc6ef0 2024-09-11T23:28:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.562+0000 7f69fd0ee640 1 --2- 172.21.15.43:0/499536321 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c5f5bc6ef0 0x55c5f5bc92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.562+0000 7f69e17fa640 10 client.16368 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.562+0000 7f69fd0ee640 1 --2- 172.21.15.43:0/499536321 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c5f5bc6ef0 0x55c5f5bc92e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8596c434c43446c server_cookie=4d29b188a3e780b4 in_seq=0 out_seq=0 2024-09-11T23:28:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.563+0000 7f69e17fa640 1 -- 172.21.15.43:0/499536321 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c5f5b1f850 con 0x55c5f5bc6ef0 2024-09-11T23:28:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.563+0000 7f69e17fa640 10 client.16368 handle_command_reply: tid=0 2024-09-11T23:28:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.563+0000 7f6a01f2c740 1 client.16368 shutdown 2024-09-11T23:28:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.563+0000 7f6a01f2c740 1 -- 172.21.15.43:0/499536321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bacdc0 msgr2=0x55c5f5bb4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.563+0000 7f6a01f2c740 1 --2- 172.21.15.43:0/499536321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bacdc0 0x55c5f5bb4220 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f69f4031550 tx=0x7f69f4009630 comp rx=0 tx=0).stop 2024-09-11T23:28:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.563+0000 7f6a01f2c740 1 -- 172.21.15.43:0/499536321 shutdown_connections 2024-09-11T23:28:15.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.563+0000 7f6a01f2c740 1 --2- 172.21.15.43:0/499536321 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c5f5bc6ef0 0x55c5f5bc92e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.563+0000 7f6a01f2c740 1 --2- 172.21.15.43:0/499536321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c5f5bb7680 0x55c5f5bb4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.563+0000 7f6a01f2c740 1 --2- 172.21.15.43:0/499536321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bacdc0 0x55c5f5bb4220 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.563+0000 7f6a01f2c740 1 --2- 172.21.15.43:0/499536321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bac4a0 0x55c5f5bb6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.563+0000 7f6a01f2c740 1 -- 172.21.15.43:0/499536321 >> 172.21.15.43:0/499536321 conn(0x55c5f5ba8a30 msgr2=0x55c5f5ba7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:15.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.563+0000 7f69e17fa640 0 client.16368 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:15.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.563+0000 7f6a01f2c740 1 -- 172.21.15.43:0/499536321 shutdown_connections 2024-09-11T23:28:15.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.563+0000 7f6a01f2c740 1 -- 172.21.15.43:0/499536321 wait complete. 2024-09-11T23:28:15.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.563+0000 7f6a01f2c740 20 client.16368 trim_cache size 0 max 16384 2024-09-11T23:28:15.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.584+0000 7f6a01f2c740 1 Processor -- start 2024-09-11T23:28:15.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.584+0000 7f6a01f2c740 1 -- start start 2024-09-11T23:28:15.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.584+0000 7f6a01f2c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c5f5bac4a0 0x55c5f5bc34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:15.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.584+0000 7f6a01f2c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bacdc0 0x55c5f5bb6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:15.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.584+0000 7f6a01f2c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bb7680 0x55c5f5bce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:15.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.584+0000 7f6a01f2c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f69f81b0180 con 0x55c5f5bb7680 2024-09-11T23:28:15.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.584+0000 7f6a01f2c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f69f81ada50 con 0x55c5f5bacdc0 2024-09-11T23:28:15.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.584+0000 7f6a01f2c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f69f81ab8a0 con 0x55c5f5bac4a0 2024-09-11T23:28:15.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.584+0000 7f69fd0ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bb7680 0x55c5f5bce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:15.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.584+0000 7f69fd0ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bb7680 0x55c5f5bce910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51342/0 (socket says 172.21.15.43:51342) 2024-09-11T23:28:15.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.584+0000 7f69fd0ee640 1 -- 172.21.15.43:0/475180843 learned_addr learned my addr 172.21.15.43:0/475180843 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:15.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.584+0000 7f69fd0ee640 1 --2- 172.21.15.43:0/475180843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c5f5bac4a0 0x55c5f5bc34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:15.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.584+0000 7f69fd8ef640 1 --2- 172.21.15.43:0/475180843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bacdc0 0x55c5f5bb6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:15.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.584+0000 7f69fd0ee640 1 -- 172.21.15.43:0/475180843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c5f5bac4a0 msgr2=0x55c5f5bc34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:15.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.584+0000 7f69fd0ee640 1 --2- 172.21.15.43:0/475180843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c5f5bac4a0 0x55c5f5bc34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.584+0000 7f69fd0ee640 1 -- 172.21.15.43:0/475180843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bacdc0 msgr2=0x55c5f5bb6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:15.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.584+0000 7f69fd0ee640 1 --2- 172.21.15.43:0/475180843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bacdc0 0x55c5f5bb6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.584+0000 7f69fd0ee640 1 -- 172.21.15.43:0/475180843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5f5ba84d0 con 0x55c5f5bb7680 2024-09-11T23:28:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.585+0000 7f69fd0ee640 1 --2- 172.21.15.43:0/475180843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bb7680 0x55c5f5bce910 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x55c5f5bc98b0 tx=0x7f69e8013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f16c3c782c505e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.585+0000 7f69e37fe640 1 -- 172.21.15.43:0/475180843 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8011050 con 0x55c5f5bb7680 2024-09-11T23:28:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.585+0000 7f69e37fe640 1 -- 172.21.15.43:0/475180843 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69e8003f20 con 0x55c5f5bb7680 2024-09-11T23:28:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.585+0000 7f69e37fe640 1 -- 172.21.15.43:0/475180843 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8005440 con 0x55c5f5bb7680 2024-09-11T23:28:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.586+0000 7f6a01f2c740 1 -- 172.21.15.43:0/475180843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bb7680 msgr2=0x55c5f5bce910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.586+0000 7f6a01f2c740 1 --2- 172.21.15.43:0/475180843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bb7680 0x55c5f5bce910 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x55c5f5bc98b0 tx=0x7f69e8013040 comp rx=0 tx=0).stop 2024-09-11T23:28:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.586+0000 7f6a01f2c740 1 -- 172.21.15.43:0/475180843 shutdown_connections 2024-09-11T23:28:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.586+0000 7f6a01f2c740 1 --2- 172.21.15.43:0/475180843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bb7680 0x55c5f5bce910 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.586+0000 7f6a01f2c740 1 --2- 172.21.15.43:0/475180843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bacdc0 0x55c5f5bb6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.586+0000 7f6a01f2c740 1 --2- 172.21.15.43:0/475180843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c5f5bac4a0 0x55c5f5bc34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.586+0000 7f6a01f2c740 1 -- 172.21.15.43:0/475180843 >> 172.21.15.43:0/475180843 conn(0x55c5f5ba8a30 msgr2=0x55c5f5bb5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.586+0000 7f6a01f2c740 1 -- 172.21.15.43:0/475180843 shutdown_connections 2024-09-11T23:28:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.586+0000 7f6a01f2c740 1 -- 172.21.15.43:0/475180843 wait complete. 2024-09-11T23:28:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.586+0000 7f6a01f2c740 1 -- start start 2024-09-11T23:28:15.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.586+0000 7f6a01f2c740 1 Processor -- start 2024-09-11T23:28:15.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.586+0000 7f6a01f2c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bac4a0 0x55c5f5bbb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:15.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.586+0000 7f6a01f2c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bacdc0 0x55c5f5bbb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:15.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.586+0000 7f6a01f2c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c5f5bb7680 0x55c5f5bbbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:15.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.586+0000 7f6a01f2c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f69f81abf10 con 0x55c5f5bacdc0 2024-09-11T23:28:15.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.586+0000 7f6a01f2c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f69f81ad380 con 0x55c5f5bac4a0 2024-09-11T23:28:15.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.586+0000 7f6a01f2c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f69f81ac5e0 con 0x55c5f5bb7680 2024-09-11T23:28:15.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.587+0000 7f6a01f2c740 10 client.0 _ll_register_callbacks cb 0x7f69e8009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:15.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.587+0000 7f69fd8ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bacdc0 0x55c5f5bbb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:15.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.587+0000 7f69fd0ee640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bac4a0 0x55c5f5bbb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:15.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.587+0000 7f69fd0ee640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bac4a0 0x55c5f5bbb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:37794/0 (socket says 172.21.15.43:37794) 2024-09-11T23:28:15.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.587+0000 7f69fd0ee640 1 -- 172.21.15.43:0/2572461842 learned_addr learned my addr 172.21.15.43:0/2572461842 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:15.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.587+0000 7f69fd0ee640 1 --2- 172.21.15.43:0/2572461842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c5f5bb7680 0x55c5f5bbbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:15.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.587+0000 7f69fd8ef640 1 -- 172.21.15.43:0/2572461842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c5f5bb7680 msgr2=0x55c5f5bbbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.587+0000 7f69fd8ef640 1 --2- 172.21.15.43:0/2572461842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c5f5bb7680 0x55c5f5bbbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.587+0000 7f69fd8ef640 1 -- 172.21.15.43:0/2572461842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bac4a0 msgr2=0x55c5f5bbb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.587+0000 7f69fd8ef640 1 --2- 172.21.15.43:0/2572461842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bac4a0 0x55c5f5bbb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.587+0000 7f69fd8ef640 1 -- 172.21.15.43:0/2572461842 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5f5c34cf0 con 0x55c5f5bacdc0 2024-09-11T23:28:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.587+0000 7f69fd0ee640 1 --2- 172.21.15.43:0/2572461842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bac4a0 0x55c5f5bbb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.587+0000 7f69fd8ef640 1 --2- 172.21.15.43:0/2572461842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bacdc0 0x55c5f5bbb860 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f69f4003a40 tx=0x7f69f4031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86155d9431121ddc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.587+0000 7f69e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:28:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.587+0000 7f69e17fa640 1 -- 172.21.15.43:0/2572461842 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4040030 con 0x55c5f5bacdc0 2024-09-11T23:28:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.587+0000 7f69e17fa640 1 -- 172.21.15.43:0/2572461842 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f69f4003e30 con 0x55c5f5bacdc0 2024-09-11T23:28:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.587+0000 7f6a01f2c740 1 -- 172.21.15.43:0/2572461842 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c5f5b19d60 con 0x55c5f5bacdc0 2024-09-11T23:28:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.588+0000 7f69e17fa640 1 -- 172.21.15.43:0/2572461842 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f69f4067050 con 0x55c5f5bacdc0 2024-09-11T23:28:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.588+0000 7f6a01f2c740 10 client.16374 fetch_fsmap learned FSMap version 30 2024-09-11T23:28:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.588+0000 7f6a01f2c740 1 -- 172.21.15.43:0/2572461842 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c5f5c35f80 con 0x55c5f5bacdc0 2024-09-11T23:28:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.588+0000 7f69e17fa640 1 -- 172.21.15.43:0/2572461842 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c5f5b19d60 con 0x55c5f5bacdc0 2024-09-11T23:28:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.588+0000 7f69e17fa640 1 -- 172.21.15.43:0/2572461842 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f69f4062030 con 0x55c5f5bacdc0 2024-09-11T23:28:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.588+0000 7f6a01f2c740 10 client.16374 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:28:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.588+0000 7f6a01f2c740 10 client.16374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:28:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.588+0000 7f6a01f2c740 20 client.16374 populate_metadata read hostname 'smithi043' 2024-09-11T23:28:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.588+0000 7f6a01f2c740 1 --2- 172.21.15.43:0/2572461842 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c5f5c366c0 0x55c5f5c38ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.588+0000 7f6a01f2c740 4 client.16374 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:28:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.588+0000 7f6a01f2c740 1 -- 172.21.15.43:0/2572461842 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c5f5a96210 con 0x55c5f5c366c0 2024-09-11T23:28:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.588+0000 7f69fd0ee640 1 --2- 172.21.15.43:0/2572461842 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c5f5c366c0 0x55c5f5c38ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.589+0000 7f69e17fa640 10 client.16374 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.589+0000 7f69fd0ee640 1 --2- 172.21.15.43:0/2572461842 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c5f5c366c0 0x55c5f5c38ae0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fe54a308d3fee21 server_cookie=fff00b55d1342186 in_seq=0 out_seq=0 2024-09-11T23:28:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.589+0000 7f69e17fa640 1 -- 172.21.15.43:0/2572461842 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55c5f5a96210 con 0x55c5f5c366c0 2024-09-11T23:28:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.589+0000 7f69e17fa640 10 client.16374 handle_command_reply: tid=0 2024-09-11T23:28:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.589+0000 7f6a01f2c740 1 client.16374 shutdown 2024-09-11T23:28:15.875 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:28:15.875 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "scrub active (51 inodes in the stack)", 2024-09-11T23:28:15.875 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-09-11T23:28:15.875 INFO:teuthology.orchestra.run.smithi043.stdout: "2010b9bf-0fac-4632-b3c1-aa91af656687": { 2024-09-11T23:28:15.875 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-09-11T23:28:15.875 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "2010b9bf-0fac-4632-b3c1-aa91af656687", 2024-09-11T23:28:15.875 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-09-11T23:28:15.875 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:28:15.875 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:28:15.875 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:28:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.589+0000 7f6a01f2c740 1 -- 172.21.15.43:0/2572461842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bacdc0 msgr2=0x55c5f5bbb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.589+0000 7f6a01f2c740 1 --2- 172.21.15.43:0/2572461842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bacdc0 0x55c5f5bbb860 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f69f4003a40 tx=0x7f69f4031860 comp rx=0 tx=0).stop 2024-09-11T23:28:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.589+0000 7f6a01f2c740 1 -- 172.21.15.43:0/2572461842 shutdown_connections 2024-09-11T23:28:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.589+0000 7f6a01f2c740 1 --2- 172.21.15.43:0/2572461842 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55c5f5c366c0 0x55c5f5c38ae0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.589+0000 7f6a01f2c740 1 --2- 172.21.15.43:0/2572461842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c5f5bb7680 0x55c5f5bbbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.589+0000 7f6a01f2c740 1 --2- 172.21.15.43:0/2572461842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5f5bacdc0 0x55c5f5bbb860 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.589+0000 7f69e17fa640 0 client.16374 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.589+0000 7f6a01f2c740 1 --2- 172.21.15.43:0/2572461842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c5f5bac4a0 0x55c5f5bbb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.589+0000 7f6a01f2c740 1 -- 172.21.15.43:0/2572461842 >> 172.21.15.43:0/2572461842 conn(0x55c5f5ba8a30 msgr2=0x55c5f5bb62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.590+0000 7f6a01f2c740 1 -- 172.21.15.43:0/2572461842 shutdown_connections 2024-09-11T23:28:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.591+0000 7f6a01f2c740 1 -- 172.21.15.43:0/2572461842 wait complete. 2024-09-11T23:28:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.591+0000 7f6a01f2c740 20 client.16374 trim_cache size 0 max 16384 2024-09-11T23:28:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.592+0000 7f69ffb53640 1 -- 172.21.15.43:0/1955727655 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f69e40a2cf0 msgr2=0x7f69e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.592+0000 7f69ffb53640 1 --2- 172.21.15.43:0/1955727655 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f69e40a2cf0 0x7f69e40a51b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f69f00cd320 tx=0x7f69f00bc040 comp rx=0 tx=0).stop 2024-09-11T23:28:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.592+0000 7f69ffb53640 1 -- 172.21.15.43:0/1955727655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69f810c170 msgr2=0x7f69f819cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.592+0000 7f69ffb53640 1 --2- 172.21.15.43:0/1955727655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69f810c170 0x7f69f819cdb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f69f0009fd0 tx=0x7f69f000ef10 comp rx=0 tx=0).stop 2024-09-11T23:28:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.592+0000 7f69ffb53640 1 -- 172.21.15.43:0/1955727655 shutdown_connections 2024-09-11T23:28:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.592+0000 7f69ffb53640 1 --2- 172.21.15.43:0/1955727655 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f69e40a2cf0 0x7f69e40a51b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.592+0000 7f69ffb53640 1 --2- 172.21.15.43:0/1955727655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69f810c170 0x7f69f819cdb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.592+0000 7f69ffb53640 1 --2- 172.21.15.43:0/1955727655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f8103660 0x7f69f8198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.592+0000 7f69ffb53640 1 --2- 172.21.15.43:0/1955727655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69f8102d40 0x7f69f81984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:15.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.592+0000 7f69ffb53640 1 -- 172.21.15.43:0/1955727655 >> 172.21.15.43:0/1955727655 conn(0x7f69f80fc7a0 msgr2=0x7f69f80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:15.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.592+0000 7f69ffb53640 1 -- 172.21.15.43:0/1955727655 shutdown_connections 2024-09-11T23:28:15.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:15.592+0000 7f69ffb53640 1 -- 172.21.15.43:0/1955727655 wait complete. 2024-09-11T23:28:16.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:16 smithi043 ceph-mon[30950]: pgmap v196: 97 pgs: 97 active+clean; 21 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 17 MiB/s wr, 3.08k op/s 2024-09-11T23:28:16.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:16 smithi097 ceph-mon[35562]: pgmap v196: 97 pgs: 97 active+clean; 21 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 17 MiB/s wr, 3.08k op/s 2024-09-11T23:28:16.321 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:16 smithi090 ceph-mon[35130]: pgmap v196: 97 pgs: 97 active+clean; 21 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 17 MiB/s wr, 3.08k op/s 2024-09-11T23:28:18.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:18 smithi097 ceph-mon[35562]: pgmap v197: 97 pgs: 97 active+clean; 21 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 2.81k op/s 2024-09-11T23:28:18.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:18 smithi043 ceph-mon[30950]: pgmap v197: 97 pgs: 97 active+clean; 21 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 2.81k op/s 2024-09-11T23:28:18.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:18 smithi090 ceph-mon[35130]: pgmap v197: 97 pgs: 97 active+clean; 21 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 2.81k op/s 2024-09-11T23:28:19.769 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:28:19 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-b[65178]: 2024-09-11T23:28:19.509+0000 7fa495419640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000215 [...4,head] /volumes/qa/sv_1/2b469744-514b-43ef-9c93-a53c45581893/client.0/tmp/ffsb/.git/refs/ auth{1=1,2=1} v405 ap=3 RANDEPHEMERALPIN f(v1 m2024-09-11T23:23:15.144864+0000 3=0+3) n(v3 rc2024-09-11T23:23:15.250863+0000 b73 7=2+5) old_inodes=1 (inest sync->mix g=1,2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:1184 nref=3)) caps={24820=pAsLsXs/-@18} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x55b6eefef600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000215:[<0x10000000008/refs v360>,<0x10000000007/.git v1663>,<0x10000000006/ffsb v1323>,<0x10000000005/tmp v15>,<0x100000001fb/client.0 v16>,<0x100000001fa/2b469744-514b-43ef-9c93-a53c45581893 v32>,<0x10000000001/sv_1 v54>,<0x10000000000/qa v47>,<0x1/volumes v44>]//[]","memoryvalue":"(2)0x10000000215:[<0x10000000008/refs v405>,<0x10000000007/.git v3113>,<0x10000000006/ffsb v2831>,<0x10000000005/tmp v426>,<0x100000001fb/client.0 v418>,<0x100000001fa/2b469744-514b-43ef-9c93-a53c45581893 v424>,<0x10000000001/sv_1 v429>,<0x10000000000/qa v409>,<0x1/volumes v403>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-11T23:23:14.524871+0000 2=0+2)","ondisk_value.rstat":"n(v0 rc2024-09-11T23:23:15.250863+0000 b73 7=2+5)","memory_value.dirstat":"f(v1 m2024-09-11T23:23:15.144864+0000 3=0+3)","memory_value.rstat":"n(v3 rc2024-09-11T23:23:15.250863+0000 b73 7=2+5)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-11T23:28:20.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:20 smithi043 ceph-mon[30950]: pgmap v198: 97 pgs: 97 active+clean; 21 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 21 MiB/s wr, 3.78k op/s 2024-09-11T23:28:20.499 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:20 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:28:20.499 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:20 smithi043 ceph-mon[30950]: scrub summary: active paths [/] 2024-09-11T23:28:20.500 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:20 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:28:20.500 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:20 smithi043 ceph-mon[30950]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/2b469744-514b-43ef-9c93-a53c45581893/client.0/tmp/ffsb/.git/refs) see mds.b log and `damage ls` output for details 2024-09-11T23:28:20.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:20 smithi090 ceph-mon[35130]: pgmap v198: 97 pgs: 97 active+clean; 21 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 21 MiB/s wr, 3.78k op/s 2024-09-11T23:28:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:20 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:28:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:20 smithi090 ceph-mon[35130]: scrub summary: active paths [/] 2024-09-11T23:28:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:20 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:28:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:20 smithi090 ceph-mon[35130]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/2b469744-514b-43ef-9c93-a53c45581893/client.0/tmp/ffsb/.git/refs) see mds.b log and `damage ls` output for details 2024-09-11T23:28:20.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:20 smithi097 ceph-mon[35562]: pgmap v198: 97 pgs: 97 active+clean; 21 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 21 MiB/s wr, 3.78k op/s 2024-09-11T23:28:20.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:20 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:28:20.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:20 smithi097 ceph-mon[35562]: scrub summary: active paths [/] 2024-09-11T23:28:20.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:20 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:28:20.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:20 smithi097 ceph-mon[35562]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/2b469744-514b-43ef-9c93-a53c45581893/client.0/tmp/ffsb/.git/refs) see mds.b log and `damage ls` output for details 2024-09-11T23:28:22.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:22 smithi043 ceph-mon[30950]: pgmap v199: 97 pgs: 97 active+clean; 21 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 22 MiB/s wr, 4.01k op/s 2024-09-11T23:28:22.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:22 smithi090 ceph-mon[35130]: pgmap v199: 97 pgs: 97 active+clean; 21 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 22 MiB/s wr, 4.01k op/s 2024-09-11T23:28:22.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:22 smithi097 ceph-mon[35562]: pgmap v199: 97 pgs: 97 active+clean; 21 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 22 MiB/s wr, 4.01k op/s 2024-09-11T23:28:23.050 INFO:tasks.cephfs.filesystem:scrub status for tag:2010b9bf-0fac-4632-b3c1-aa91af656687 - {'path': '/', 'tag': '2010b9bf-0fac-4632-b3c1-aa91af656687', 'options': 'recursive,force'} 2024-09-11T23:28:23.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:23 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:28:23.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:23 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:28:23.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:23 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:28:23.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:23 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:28:23.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:23 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:28:23.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:23 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:28:23.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:23 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:28:23.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:23 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:28:23.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:23 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:28:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.356+0000 7f02ff9c4640 1 -- 172.21.15.43:0/2101688347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02f81069b0 msgr2=0x7f02f8106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.356+0000 7f02ff9c4640 1 --2- 172.21.15.43:0/2101688347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02f81069b0 0x7f02f8106d90 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009930 tx=0x7f02f402a420 comp rx=0 tx=0).stop 2024-09-11T23:28:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.356+0000 7f02ff9c4640 1 -- 172.21.15.43:0/2101688347 shutdown_connections 2024-09-11T23:28:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.356+0000 7f02ff9c4640 1 --2- 172.21.15.43:0/2101688347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02f8101340 0x7f02f810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.356+0000 7f02ff9c4640 1 --2- 172.21.15.43:0/2101688347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02f81009a0 0x7f02f8100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.356+0000 7f02ff9c4640 1 --2- 172.21.15.43:0/2101688347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02f81069b0 0x7f02f8106d90 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.356+0000 7f02ff9c4640 1 -- 172.21.15.43:0/2101688347 >> 172.21.15.43:0/2101688347 conn(0x7f02f80779a0 msgr2=0x7f02f80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.356+0000 7f02ff9c4640 1 -- 172.21.15.43:0/2101688347 shutdown_connections 2024-09-11T23:28:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.356+0000 7f02ff9c4640 1 -- 172.21.15.43:0/2101688347 wait complete. 2024-09-11T23:28:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.357+0000 7f02ff9c4640 1 Processor -- start 2024-09-11T23:28:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.357+0000 7f02ff9c4640 1 -- start start 2024-09-11T23:28:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.357+0000 7f02ff9c4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02f81009a0 0x7f02f81992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.357+0000 7f02ff9c4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02f8101340 0x7f02f81997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.357+0000 7f02ff9c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02f81069b0 0x7f02f81a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.357+0000 7f02ff9c4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f02f8110df0 con 0x7f02f81069b0 2024-09-11T23:28:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.357+0000 7f02ff9c4640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f02f8110c70 con 0x7f02f8101340 2024-09-11T23:28:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.357+0000 7f02ff9c4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f02f8110f70 con 0x7f02f81009a0 2024-09-11T23:28:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.357+0000 7f02fd760640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02f81009a0 0x7f02f81992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.358+0000 7f02fd760640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02f81009a0 0x7f02f81992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:41820/0 (socket says 172.21.15.43:41820) 2024-09-11T23:28:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.358+0000 7f02fd760640 1 -- 172.21.15.43:0/165938542 learned_addr learned my addr 172.21.15.43:0/165938542 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.358+0000 7f02fdf61640 1 --2- 172.21.15.43:0/165938542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02f81069b0 0x7f02f81a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.358+0000 7f02fdf61640 1 -- 172.21.15.43:0/165938542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02f81009a0 msgr2=0x7f02f81992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.358+0000 7f02fdf61640 1 --2- 172.21.15.43:0/165938542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02f81009a0 0x7f02f81992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.358+0000 7f02fdf61640 1 -- 172.21.15.43:0/165938542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02f8101340 msgr2=0x7f02f81997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.358+0000 7f02fdf61640 1 --2- 172.21.15.43:0/165938542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02f8101340 0x7f02f81997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.358+0000 7f02fdf61640 1 -- 172.21.15.43:0/165938542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f81a4860 con 0x7f02f81069b0 2024-09-11T23:28:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.358+0000 7f02fd760640 1 --2- 172.21.15.43:0/165938542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02f81009a0 0x7f02f81992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.358+0000 7f02fdf61640 1 --2- 172.21.15.43:0/165938542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02f81069b0 0x7f02f81a4320 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f02f000e940 tx=0x7f02f000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.358+0000 7f02ee7fc640 1 -- 172.21.15.43:0/165938542 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0004d20 con 0x7f02f81069b0 2024-09-11T23:28:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.358+0000 7f02ff9c4640 1 -- 172.21.15.43:0/165938542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f81a4a30 con 0x7f02f81069b0 2024-09-11T23:28:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.358+0000 7f02ee7fc640 1 -- 172.21.15.43:0/165938542 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f0005020 con 0x7f02f81069b0 2024-09-11T23:28:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.358+0000 7f02ee7fc640 1 -- 172.21.15.43:0/165938542 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f00116d0 con 0x7f02f81069b0 2024-09-11T23:28:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.358+0000 7f02ff9c4640 1 -- 172.21.15.43:0/165938542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f81a4fa0 con 0x7f02f81069b0 2024-09-11T23:28:24.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:24 smithi043 ceph-mon[30950]: pgmap v200: 97 pgs: 97 active+clean; 21 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 21 MiB/s wr, 3.85k op/s 2024-09-11T23:28:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.361+0000 7f030283f740 1 Processor -- start 2024-09-11T23:28:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.362+0000 7f02ee7fc640 1 -- 172.21.15.43:0/165938542 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f02f00054d0 con 0x7f02f81069b0 2024-09-11T23:28:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.362+0000 7f030283f740 1 -- start start 2024-09-11T23:28:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.363+0000 7f030283f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db67af0 0x56356db67ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.363+0000 7f030283f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56356db68410 0x56356db72770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.363+0000 7f030283f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db72cb0 0x56356db750a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.363+0000 7f030283f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56356dacc2f0 con 0x56356db72cb0 2024-09-11T23:28:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.363+0000 7f030283f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56356da74270 con 0x56356db68410 2024-09-11T23:28:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.363+0000 7f030283f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56356dac82b0 con 0x56356db67af0 2024-09-11T23:28:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.363+0000 7f02fcf5f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db67af0 0x56356db67ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.363+0000 7f02fcf5f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db67af0 0x56356db67ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:41826/0 (socket says 172.21.15.43:41826) 2024-09-11T23:28:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.363+0000 7f02fcf5f640 1 -- 172.21.15.43:0/3013794284 learned_addr learned my addr 172.21.15.43:0/3013794284 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.363+0000 7f02fd760640 1 --2- 172.21.15.43:0/3013794284 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56356db68410 0x56356db72770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.363+0000 7f02fcf5f640 1 --2- 172.21.15.43:0/3013794284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db72cb0 0x56356db750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.363+0000 7f02fcf5f640 1 -- 172.21.15.43:0/3013794284 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56356db68410 msgr2=0x56356db72770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.363+0000 7f02fcf5f640 1 --2- 172.21.15.43:0/3013794284 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56356db68410 0x56356db72770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.363+0000 7f02fcf5f640 1 -- 172.21.15.43:0/3013794284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db72cb0 msgr2=0x56356db750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.363+0000 7f02fcf5f640 1 --2- 172.21.15.43:0/3013794284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db72cb0 0x56356db750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.363+0000 7f02fcf5f640 1 -- 172.21.15.43:0/3013794284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56356da73340 con 0x56356db67af0 2024-09-11T23:28:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.363+0000 7f02ee7fc640 1 --2- 172.21.15.43:0/165938542 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f02e40a2cf0 0x7f02e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.363+0000 7f02fcf5f640 1 --2- 172.21.15.43:0/3013794284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db67af0 0x56356db67ed0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f02e800dbb0 tx=0x7f02e800d3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18978ae8a7e3bd3b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.364+0000 7f02fd760640 1 --2- 172.21.15.43:0/165938542 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f02e40a2cf0 0x7f02e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.364+0000 7f02ee7fc640 1 -- 172.21.15.43:0/165938542 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f02f00cf7f0 con 0x7f02f81069b0 2024-09-11T23:28:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.364+0000 7f02e37fe640 1 -- 172.21.15.43:0/3013794284 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e800ab40 con 0x56356db67af0 2024-09-11T23:28:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.364+0000 7f02e37fe640 1 -- 172.21.15.43:0/3013794284 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e800ace0 con 0x56356db67af0 2024-09-11T23:28:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.364+0000 7f02e37fe640 1 -- 172.21.15.43:0/3013794284 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8005020 con 0x56356db67af0 2024-09-11T23:28:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.364+0000 7f030283f740 1 -- 172.21.15.43:0/3013794284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db67af0 msgr2=0x56356db67ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.364+0000 7f02fd760640 1 --2- 172.21.15.43:0/165938542 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f02e40a2cf0 0x7f02e40a51b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f02f40314b0 tx=0x7f02f4005d90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.364+0000 7f030283f740 1 --2- 172.21.15.43:0/3013794284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db67af0 0x56356db67ed0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f02e800dbb0 tx=0x7f02e800d3c0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.365+0000 7f030283f740 1 -- 172.21.15.43:0/3013794284 shutdown_connections 2024-09-11T23:28:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.365+0000 7f030283f740 1 --2- 172.21.15.43:0/3013794284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db72cb0 0x56356db750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.365+0000 7f030283f740 1 --2- 172.21.15.43:0/3013794284 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56356db68410 0x56356db72770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.365+0000 7f030283f740 1 --2- 172.21.15.43:0/3013794284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db67af0 0x56356db67ed0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.365+0000 7f030283f740 1 -- 172.21.15.43:0/3013794284 >> 172.21.15.43:0/3013794284 conn(0x56356db64060 msgr2=0x56356db64440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.365+0000 7f030283f740 1 -- 172.21.15.43:0/3013794284 shutdown_connections 2024-09-11T23:28:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.365+0000 7f030283f740 1 -- 172.21.15.43:0/3013794284 wait complete. 2024-09-11T23:28:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.365+0000 7f030283f740 1 -- start start 2024-09-11T23:28:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.365+0000 7f030283f740 1 Processor -- start 2024-09-11T23:28:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.365+0000 7f030283f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db67af0 0x56356db6fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.365+0000 7f030283f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56356db68410 0x56356db70080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.365+0000 7f030283f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db72cb0 0x56356db705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.365+0000 7f030283f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56356db75880 con 0x56356db67af0 2024-09-11T23:28:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.365+0000 7f030283f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56356da55290 con 0x56356db68410 2024-09-11T23:28:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.365+0000 7f030283f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56356db75a00 con 0x56356db72cb0 2024-09-11T23:28:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.366+0000 7f030283f740 10 client.0 _ll_register_callbacks cb 0x56356da86140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.366+0000 7f02fd760640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db67af0 0x56356db6fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.366+0000 7f02fd760640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db67af0 0x56356db6fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46598/0 (socket says 172.21.15.43:46598) 2024-09-11T23:28:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.366+0000 7f02fd760640 1 -- 172.21.15.43:0/2080800391 learned_addr learned my addr 172.21.15.43:0/2080800391 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.366+0000 7f02fdf61640 1 --2- 172.21.15.43:0/2080800391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db72cb0 0x56356db705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.366+0000 7f02fcf5f640 1 --2- 172.21.15.43:0/2080800391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56356db68410 0x56356db70080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.366+0000 7f02fdf61640 1 -- 172.21.15.43:0/2080800391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56356db68410 msgr2=0x56356db70080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.366+0000 7f02fdf61640 1 --2- 172.21.15.43:0/2080800391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56356db68410 0x56356db70080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.366+0000 7f02fdf61640 1 -- 172.21.15.43:0/2080800391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db67af0 msgr2=0x56356db6fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.366+0000 7f02fdf61640 1 --2- 172.21.15.43:0/2080800391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db67af0 0x56356db6fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.366+0000 7f02fdf61640 1 -- 172.21.15.43:0/2080800391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56356d7df210 con 0x56356db72cb0 2024-09-11T23:28:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.366+0000 7f02fd760640 1 --2- 172.21.15.43:0/2080800391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db67af0 0x56356db6fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.366+0000 7f02fdf61640 1 --2- 172.21.15.43:0/2080800391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db72cb0 0x56356db705c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f02f00ccf40 tx=0x7f02f009b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12742e804961873a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.366+0000 7f02e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-11T23:28:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.366+0000 7f02e0ff9640 1 -- 172.21.15.43:0/2080800391 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f00a00c0 con 0x56356db72cb0 2024-09-11T23:28:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.366+0000 7f02e0ff9640 1 -- 172.21.15.43:0/2080800391 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f02f00a44f0 con 0x56356db72cb0 2024-09-11T23:28:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.366+0000 7f030283f740 1 -- 172.21.15.43:0/2080800391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02e800ace0 con 0x56356db72cb0 2024-09-11T23:28:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.367+0000 7f02e0ff9640 1 -- 172.21.15.43:0/2080800391 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f02f00a4980 con 0x56356db72cb0 2024-09-11T23:28:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.367+0000 7f030283f740 10 client.25861 fetch_fsmap learned FSMap version 30 2024-09-11T23:28:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.367+0000 7f030283f740 1 -- 172.21.15.43:0/2080800391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56356d8ab620 con 0x56356db72cb0 2024-09-11T23:28:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.367+0000 7f02e0ff9640 1 -- 172.21.15.43:0/2080800391 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e800ace0 con 0x56356db72cb0 2024-09-11T23:28:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.367+0000 7f02e0ff9640 1 -- 172.21.15.43:0/2080800391 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f02f000f040 con 0x56356db72cb0 2024-09-11T23:28:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.367+0000 7f030283f740 10 client.25861 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:28:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.367+0000 7f030283f740 10 client.25861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:28:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.367+0000 7f030283f740 20 client.25861 populate_metadata read hostname 'smithi043' 2024-09-11T23:28:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.367+0000 7f030283f740 1 --2- 172.21.15.43:0/2080800391 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56356db82c40 0x56356db85030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.367+0000 7f030283f740 4 client.25861 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:28:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.367+0000 7f030283f740 1 -- 172.21.15.43:0/2080800391 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56356dad9a20 con 0x56356db82c40 2024-09-11T23:28:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.367+0000 7f02fd760640 1 --2- 172.21.15.43:0/2080800391 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56356db82c40 0x56356db85030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.368+0000 7f02fd760640 1 --2- 172.21.15.43:0/2080800391 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56356db82c40 0x56356db85030 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51309361d3771a35 server_cookie=e9e6e5db4461445d in_seq=0 out_seq=0 2024-09-11T23:28:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.368+0000 7f02e0ff9640 10 client.25861 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.369+0000 7f02e0ff9640 1 -- 172.21.15.43:0/2080800391 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56356dad9a20 con 0x56356db82c40 2024-09-11T23:28:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.369+0000 7f02e0ff9640 10 client.25861 handle_command_reply: tid=0 2024-09-11T23:28:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.369+0000 7f030283f740 1 client.25861 shutdown 2024-09-11T23:28:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.369+0000 7f030283f740 1 -- 172.21.15.43:0/2080800391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db72cb0 msgr2=0x56356db705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.369+0000 7f030283f740 1 --2- 172.21.15.43:0/2080800391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db72cb0 0x56356db705c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f02f00ccf40 tx=0x7f02f009b670 comp rx=0 tx=0).stop 2024-09-11T23:28:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.369+0000 7f030283f740 1 -- 172.21.15.43:0/2080800391 shutdown_connections 2024-09-11T23:28:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.369+0000 7f030283f740 1 --2- 172.21.15.43:0/2080800391 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56356db82c40 0x56356db85030 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.369+0000 7f030283f740 1 --2- 172.21.15.43:0/2080800391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db72cb0 0x56356db705c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.369+0000 7f030283f740 1 --2- 172.21.15.43:0/2080800391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56356db68410 0x56356db70080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.369+0000 7f030283f740 1 --2- 172.21.15.43:0/2080800391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db67af0 0x56356db6fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.369+0000 7f030283f740 1 -- 172.21.15.43:0/2080800391 >> 172.21.15.43:0/2080800391 conn(0x56356db64060 msgr2=0x56356db69d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.369+0000 7f02e0ff9640 0 client.25861 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.369+0000 7f030283f740 1 -- 172.21.15.43:0/2080800391 shutdown_connections 2024-09-11T23:28:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.369+0000 7f030283f740 1 -- 172.21.15.43:0/2080800391 wait complete. 2024-09-11T23:28:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.369+0000 7f030283f740 20 client.25861 trim_cache size 0 max 16384 2024-09-11T23:28:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.387+0000 7f030283f740 1 Processor -- start 2024-09-11T23:28:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.387+0000 7f030283f740 1 -- start start 2024-09-11T23:28:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.387+0000 7f030283f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db67af0 0x56356db6a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.387+0000 7f030283f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db68410 0x56356db70410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.387+0000 7f030283f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56356db72cb0 0x56356db71ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.387+0000 7f030283f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f02f81adfb0 con 0x56356db67af0 2024-09-11T23:28:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.388+0000 7f030283f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f02f81ab880 con 0x56356db72cb0 2024-09-11T23:28:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.388+0000 7f030283f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f02f81a96d0 con 0x56356db68410 2024-09-11T23:28:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.388+0000 7f02fd760640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db67af0 0x56356db6a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.388+0000 7f02fd760640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db67af0 0x56356db6a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46608/0 (socket says 172.21.15.43:46608) 2024-09-11T23:28:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.388+0000 7f02fd760640 1 -- 172.21.15.43:0/3417847290 learned_addr learned my addr 172.21.15.43:0/3417847290 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.388+0000 7f02fcf5f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db68410 0x56356db70410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.388+0000 7f02fdf61640 1 --2- 172.21.15.43:0/3417847290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56356db72cb0 0x56356db71ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.388+0000 7f02fd760640 1 -- 172.21.15.43:0/3417847290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db68410 msgr2=0x56356db70410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.388+0000 7f02fd760640 1 --2- 172.21.15.43:0/3417847290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db68410 0x56356db70410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.388+0000 7f02fd760640 1 -- 172.21.15.43:0/3417847290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56356db72cb0 msgr2=0x56356db71ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.388+0000 7f02fd760640 1 --2- 172.21.15.43:0/3417847290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56356db72cb0 0x56356db71ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.388+0000 7f02fd760640 1 -- 172.21.15.43:0/3417847290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56356db72690 con 0x56356db67af0 2024-09-11T23:28:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.388+0000 7f02fd760640 1 --2- 172.21.15.43:0/3417847290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db67af0 0x56356db6a560 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009a50 tx=0x7f02f403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8195476658b225f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.388+0000 7f02e2ffd640 1 -- 172.21.15.43:0/3417847290 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4058070 con 0x56356db67af0 2024-09-11T23:28:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.388+0000 7f02e2ffd640 1 -- 172.21.15.43:0/3417847290 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f4005510 con 0x56356db67af0 2024-09-11T23:28:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.388+0000 7f02e2ffd640 1 -- 172.21.15.43:0/3417847290 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4043070 con 0x56356db67af0 2024-09-11T23:28:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.389+0000 7f030283f740 1 -- 172.21.15.43:0/3417847290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db67af0 msgr2=0x56356db6a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.389+0000 7f030283f740 1 --2- 172.21.15.43:0/3417847290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db67af0 0x56356db6a560 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009a50 tx=0x7f02f403f000 comp rx=0 tx=0).stop 2024-09-11T23:28:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.389+0000 7f030283f740 1 -- 172.21.15.43:0/3417847290 shutdown_connections 2024-09-11T23:28:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.389+0000 7f030283f740 1 --2- 172.21.15.43:0/3417847290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56356db72cb0 0x56356db71ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.389+0000 7f030283f740 1 --2- 172.21.15.43:0/3417847290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db68410 0x56356db70410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.389+0000 7f030283f740 1 --2- 172.21.15.43:0/3417847290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db67af0 0x56356db6a560 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.389+0000 7f030283f740 1 -- 172.21.15.43:0/3417847290 >> 172.21.15.43:0/3417847290 conn(0x56356db64060 msgr2=0x56356db713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.389+0000 7f030283f740 1 -- 172.21.15.43:0/3417847290 shutdown_connections 2024-09-11T23:28:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.389+0000 7f030283f740 1 -- 172.21.15.43:0/3417847290 wait complete. 2024-09-11T23:28:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.390+0000 7f030283f740 1 -- start start 2024-09-11T23:28:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.390+0000 7f030283f740 1 Processor -- start 2024-09-11T23:28:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.390+0000 7f030283f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db67af0 0x56356db7ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.390+0000 7f030283f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56356db68410 0x56356db7b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.390+0000 7f030283f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db72cb0 0x56356db6c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.390+0000 7f030283f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f02f81a9d40 con 0x56356db67af0 2024-09-11T23:28:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.390+0000 7f030283f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f02f81ab1b0 con 0x56356db68410 2024-09-11T23:28:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.390+0000 7f030283f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f02f81aa410 con 0x56356db72cb0 2024-09-11T23:28:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.390+0000 7f030283f740 10 client.0 _ll_register_callbacks cb 0x7f02f0005f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.390+0000 7f02fd760640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db67af0 0x56356db7ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.390+0000 7f02fd760640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db67af0 0x56356db7ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46622/0 (socket says 172.21.15.43:46622) 2024-09-11T23:28:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.390+0000 7f02fd760640 1 -- 172.21.15.43:0/1088892758 learned_addr learned my addr 172.21.15.43:0/1088892758 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.390+0000 7f02fcf5f640 1 --2- 172.21.15.43:0/1088892758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56356db68410 0x56356db7b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.390+0000 7f02fdf61640 1 --2- 172.21.15.43:0/1088892758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db72cb0 0x56356db6c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.391+0000 7f02fd760640 1 -- 172.21.15.43:0/1088892758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db72cb0 msgr2=0x56356db6c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.391+0000 7f02fd760640 1 --2- 172.21.15.43:0/1088892758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db72cb0 0x56356db6c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.391+0000 7f02fd760640 1 -- 172.21.15.43:0/1088892758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56356db68410 msgr2=0x56356db7b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.391+0000 7f02fd760640 1 --2- 172.21.15.43:0/1088892758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56356db68410 0x56356db7b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.391+0000 7f02fd760640 1 -- 172.21.15.43:0/1088892758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56356db6cac0 con 0x56356db67af0 2024-09-11T23:28:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.391+0000 7f02fcf5f640 1 --2- 172.21.15.43:0/1088892758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56356db68410 0x56356db7b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.391+0000 7f02fd760640 1 --2- 172.21.15.43:0/1088892758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db67af0 0x56356db7ae80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f02f4004530 tx=0x7f02f40318c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf73b5658d52b50 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.391+0000 7f02e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:28:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.391+0000 7f02e0ff9640 1 -- 172.21.15.43:0/1088892758 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4058070 con 0x56356db67af0 2024-09-11T23:28:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.391+0000 7f030283f740 1 -- 172.21.15.43:0/1088892758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02f4005510 con 0x56356db67af0 2024-09-11T23:28:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.391+0000 7f02e0ff9640 1 -- 172.21.15.43:0/1088892758 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f02f4004980 con 0x56356db67af0 2024-09-11T23:28:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.391+0000 7f02e0ff9640 1 -- 172.21.15.43:0/1088892758 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f02f4035040 con 0x56356db67af0 2024-09-11T23:28:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.391+0000 7f030283f740 10 client.16392 fetch_fsmap learned FSMap version 30 2024-09-11T23:28:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.391+0000 7f030283f740 1 -- 172.21.15.43:0/1088892758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56356dbed220 con 0x56356db67af0 2024-09-11T23:28:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.391+0000 7f02e0ff9640 1 -- 172.21.15.43:0/1088892758 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f4005510 con 0x56356db67af0 2024-09-11T23:28:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.392+0000 7f02e0ff9640 1 -- 172.21.15.43:0/1088892758 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f02f403d450 con 0x56356db67af0 2024-09-11T23:28:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.392+0000 7f030283f740 10 client.16392 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:28:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.392+0000 7f030283f740 10 client.16392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:28:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.392+0000 7f030283f740 20 client.16392 populate_metadata read hostname 'smithi043' 2024-09-11T23:28:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.392+0000 7f030283f740 1 --2- 172.21.15.43:0/1088892758 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56356dbed970 0x56356dbefd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.392+0000 7f030283f740 4 client.16392 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:28:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.392+0000 7f030283f740 1 -- 172.21.15.43:0/1088892758 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56356dad3d70 con 0x56356dbed970 2024-09-11T23:28:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.392+0000 7f02fcf5f640 1 --2- 172.21.15.43:0/1088892758 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56356dbed970 0x56356dbefd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.392+0000 7f02e0ff9640 10 client.16392 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.392+0000 7f02fcf5f640 1 --2- 172.21.15.43:0/1088892758 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56356dbed970 0x56356dbefd90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da5ff9d43c99656a server_cookie=5b6401aa97c460c6 in_seq=0 out_seq=0 2024-09-11T23:28:24.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:24 smithi090 ceph-mon[35130]: pgmap v200: 97 pgs: 97 active+clean; 21 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 21 MiB/s wr, 3.85k op/s 2024-09-11T23:28:24.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:24 smithi097 ceph-mon[35562]: pgmap v200: 97 pgs: 97 active+clean; 21 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 21 MiB/s wr, 3.85k op/s 2024-09-11T23:28:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.722+0000 7f02e0ff9640 1 -- 172.21.15.43:0/1088892758 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56356dad3d70 con 0x56356dbed970 2024-09-11T23:28:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.722+0000 7f02e0ff9640 10 client.16392 handle_command_reply: tid=0 2024-09-11T23:28:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.722+0000 7f030283f740 1 client.16392 shutdown 2024-09-11T23:28:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.722+0000 7f030283f740 1 -- 172.21.15.43:0/1088892758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db67af0 msgr2=0x56356db7ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.722+0000 7f030283f740 1 --2- 172.21.15.43:0/1088892758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db67af0 0x56356db7ae80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f02f4004530 tx=0x7f02f40318c0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.722+0000 7f030283f740 1 -- 172.21.15.43:0/1088892758 shutdown_connections 2024-09-11T23:28:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.722+0000 7f030283f740 1 --2- 172.21.15.43:0/1088892758 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56356dbed970 0x56356dbefd90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.722+0000 7f030283f740 1 --2- 172.21.15.43:0/1088892758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56356db72cb0 0x56356db6c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.722+0000 7f030283f740 1 --2- 172.21.15.43:0/1088892758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56356db68410 0x56356db7b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.722+0000 7f02e0ff9640 0 client.16392 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.722+0000 7f030283f740 1 --2- 172.21.15.43:0/1088892758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56356db67af0 0x56356db7ae80 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.722+0000 7f030283f740 1 -- 172.21.15.43:0/1088892758 >> 172.21.15.43:0/1088892758 conn(0x56356db64060 msgr2=0x56356db631c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.723+0000 7f030283f740 1 -- 172.21.15.43:0/1088892758 shutdown_connections 2024-09-11T23:28:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.723+0000 7f030283f740 1 -- 172.21.15.43:0/1088892758 wait complete. 2024-09-11T23:28:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.723+0000 7f030283f740 20 client.16392 trim_cache size 0 max 16384 2024-09-11T23:28:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:28:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:28:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:28:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:28:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.723+0000 7f02ff9c4640 1 -- 172.21.15.43:0/165938542 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f02e40a2cf0 msgr2=0x7f02e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.723+0000 7f02ff9c4640 1 --2- 172.21.15.43:0/165938542 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f02e40a2cf0 0x7f02e40a51b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f02f40314b0 tx=0x7f02f4005d90 comp rx=0 tx=0).stop 2024-09-11T23:28:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.723+0000 7f02ff9c4640 1 -- 172.21.15.43:0/165938542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02f81069b0 msgr2=0x7f02f81a4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.723+0000 7f02ff9c4640 1 --2- 172.21.15.43:0/165938542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02f81069b0 0x7f02f81a4320 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f02f000e940 tx=0x7f02f000ee00 comp rx=0 tx=0).stop 2024-09-11T23:28:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.723+0000 7f02ff9c4640 1 -- 172.21.15.43:0/165938542 shutdown_connections 2024-09-11T23:28:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.723+0000 7f02ff9c4640 1 --2- 172.21.15.43:0/165938542 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f02e40a2cf0 0x7f02e40a51b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.723+0000 7f02ff9c4640 1 --2- 172.21.15.43:0/165938542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02f81069b0 0x7f02f81a4320 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.723+0000 7f02ff9c4640 1 --2- 172.21.15.43:0/165938542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02f8101340 0x7f02f81997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.723+0000 7f02ff9c4640 1 --2- 172.21.15.43:0/165938542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02f81009a0 0x7f02f81992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.723+0000 7f02ff9c4640 1 -- 172.21.15.43:0/165938542 >> 172.21.15.43:0/165938542 conn(0x7f02f80779a0 msgr2=0x7f02f8103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.724+0000 7f02ff9c4640 1 -- 172.21.15.43:0/165938542 shutdown_connections 2024-09-11T23:28:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:24.724+0000 7f02ff9c4640 1 -- 172.21.15.43:0/165938542 wait complete. 2024-09-11T23:28:25.899 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:28:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 30, 2024-09-11T23:28:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "leader": 24497, 2024-09-11T23:28:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "set_version": 13, 2024-09-11T23:28:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "sets": { 2024-09-11T23:28:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "733f7a10": { 2024-09-11T23:28:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 13, 2024-09-11T23:28:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "age_ref": 0.0, 2024-09-11T23:28:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:28:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "QUIESCING", 2024-09-11T23:28:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.0 2024-09-11T23:28:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:28:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "timeout": 90.0, 2024-09-11T23:28:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "expiration": 140.2, 2024-09-11T23:28:25.900 INFO:teuthology.orchestra.run.smithi043.stdout: "members": { 2024-09-11T23:28:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: "file:/": { 2024-09-11T23:28:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: "excluded": false, 2024-09-11T23:28:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:28:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "QUIESCING", 2024-09-11T23:28:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.0 2024-09-11T23:28:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:28:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:28:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:28:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:28:25.901 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:28:25.901 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:28:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:25.895+0000 7fe179ffb640 1 -- 172.21.15.43:0/1184614473 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: -115 ) ==== 8+0+595 (crc 0 0 0) 0x55cec205bbe0 con 0x55cec2175fb0 2024-09-11T23:28:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:25.895+0000 7fe179ffb640 10 client.16323 handle_command_reply: tid=0 2024-09-11T23:28:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:25.895+0000 7fe19b34e740 1 client.16323 shutdown 2024-09-11T23:28:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:25.896+0000 7fe19b34e740 1 -- 172.21.15.43:0/1184614473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20ef850 msgr2=0x55cec20fbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:25.896+0000 7fe19b34e740 1 --2- 172.21.15.43:0/1184614473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20ef850 0x55cec20fbdf0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe18401ffd0 tx=0x7fe18401ced0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:25.896+0000 7fe19b34e740 1 -- 172.21.15.43:0/1184614473 shutdown_connections 2024-09-11T23:28:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:25.896+0000 7fe19b34e740 1 --2- 172.21.15.43:0/1184614473 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55cec2175fb0 0x55cec21783d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:25.896+0000 7fe19b34e740 1 --2- 172.21.15.43:0/1184614473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cec20faa30 0x55cec20fc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:25.896+0000 7fe179ffb640 0 client.16323 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:25.896+0000 7fe19b34e740 1 --2- 172.21.15.43:0/1184614473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cec20f0170 0x55cec20fc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:25.896+0000 7fe19b34e740 1 --2- 172.21.15.43:0/1184614473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cec20ef850 0x55cec20fbdf0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:25.896+0000 7fe19b34e740 1 -- 172.21.15.43:0/1184614473 >> 172.21.15.43:0/1184614473 conn(0x55cec20ebde0 msgr2=0x55cec20ea6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:25.896+0000 7fe19b34e740 1 -- 172.21.15.43:0/1184614473 shutdown_connections 2024-09-11T23:28:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:25.896+0000 7fe19b34e740 1 -- 172.21.15.43:0/1184614473 wait complete. 2024-09-11T23:28:25.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:25.896+0000 7fe19b34e740 20 client.16323 trim_cache size 0 max 16384 2024-09-11T23:28:25.903 INFO:teuthology.orchestra.run.smithi043.stderr:Error EINPROGRESS: 2024-09-11T23:28:26.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:26 smithi043 ceph-mon[30950]: pgmap v201: 97 pgs: 97 active+clean; 21 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 22 MiB/s wr, 4.12k op/s 2024-09-11T23:28:26.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:26 smithi090 ceph-mon[35130]: pgmap v201: 97 pgs: 97 active+clean; 21 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 22 MiB/s wr, 4.12k op/s 2024-09-11T23:28:26.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:26 smithi097 ceph-mon[35562]: pgmap v201: 97 pgs: 97 active+clean; 21 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 22 MiB/s wr, 4.12k op/s 2024-09-11T23:28:27.364 DEBUG:teuthology.orchestra.run:got remote process result: 115 2024-09-11T23:28:27.365 WARNING:tasks.quiescer.fs.[cephfs]:Set '733f7a10' hasn't quiesced after 55.8 seconds (timeout: 90). Dumping ops with locks from all ranks. 2024-09-11T23:28:27.366 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:28:27.631 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:28:28.323 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:28 smithi043 ceph-mon[30950]: pgmap v202: 97 pgs: 97 active+clean; 21 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 16 MiB/s wr, 2.87k op/s 2024-09-11T23:28:28.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.331+0000 7fd285c13640 1 -- 172.21.15.43:0/2139788604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2801002f0 msgr2=0x7fd2801006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:28.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.331+0000 7fd285c13640 1 --2- 172.21.15.43:0/2139788604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2801002f0 0x7fd2801006d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd270009930 tx=0x7fd27002a420 comp rx=0 tx=0).stop 2024-09-11T23:28:28.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.331+0000 7fd285c13640 1 -- 172.21.15.43:0/2139788604 shutdown_connections 2024-09-11T23:28:28.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.331+0000 7fd285c13640 1 --2- 172.21.15.43:0/2139788604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd280106f50 0x7fd28010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:28.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.331+0000 7fd285c13640 1 --2- 172.21.15.43:0/2139788604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd280100ca0 0x7fd280106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:28.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.331+0000 7fd285c13640 1 --2- 172.21.15.43:0/2139788604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2801002f0 0x7fd2801006d0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:28.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.331+0000 7fd285c13640 1 -- 172.21.15.43:0/2139788604 >> 172.21.15.43:0/2139788604 conn(0x7fd280077220 msgr2=0x7fd2800fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:28.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.331+0000 7fd285c13640 1 -- 172.21.15.43:0/2139788604 shutdown_connections 2024-09-11T23:28:28.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.331+0000 7fd285c13640 1 -- 172.21.15.43:0/2139788604 wait complete. 2024-09-11T23:28:28.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.332+0000 7fd285c13640 1 Processor -- start 2024-09-11T23:28:28.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.332+0000 7fd285c13640 1 -- start start 2024-09-11T23:28:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.332+0000 7fd285c13640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2801002f0 0x7fd280101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.332+0000 7fd285c13640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd280100ca0 0x7fd2801017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.332+0000 7fd285c13640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd280106f50 0x7fd280101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.332+0000 7fd285c13640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd280111fa0 con 0x7fd280100ca0 2024-09-11T23:28:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.332+0000 7fd285c13640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd280111e20 con 0x7fd280106f50 2024-09-11T23:28:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.332+0000 7fd285c13640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd280112120 con 0x7fd2801002f0 2024-09-11T23:28:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.332+0000 7fd27effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd280100ca0 0x7fd2801017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.332+0000 7fd27f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2801002f0 0x7fd280101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.332+0000 7fd27effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd280100ca0 0x7fd2801017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46644/0 (socket says 172.21.15.43:46644) 2024-09-11T23:28:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.332+0000 7fd27effd640 1 -- 172.21.15.43:0/521885399 learned_addr learned my addr 172.21.15.43:0/521885399 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.332+0000 7fd27ffff640 1 --2- 172.21.15.43:0/521885399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd280106f50 0x7fd280101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.333+0000 7fd27effd640 1 -- 172.21.15.43:0/521885399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2801002f0 msgr2=0x7fd280101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.333+0000 7fd27effd640 1 --2- 172.21.15.43:0/521885399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2801002f0 0x7fd280101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.333+0000 7fd27effd640 1 -- 172.21.15.43:0/521885399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd280106f50 msgr2=0x7fd280101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.333+0000 7fd27effd640 1 --2- 172.21.15.43:0/521885399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd280106f50 0x7fd280101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.333+0000 7fd27effd640 1 -- 172.21.15.43:0/521885399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2801a6970 con 0x7fd280100ca0 2024-09-11T23:28:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.333+0000 7fd27f7fe640 1 --2- 172.21.15.43:0/521885399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2801002f0 0x7fd280101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.333+0000 7fd27effd640 1 --2- 172.21.15.43:0/521885399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd280100ca0 0x7fd2801017b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd27400e9f0 tx=0x7fd27400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.333+0000 7fd27cff9640 1 -- 172.21.15.43:0/521885399 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27400ccc0 con 0x7fd280100ca0 2024-09-11T23:28:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.333+0000 7fd27cff9640 1 -- 172.21.15.43:0/521885399 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd274005500 con 0x7fd280100ca0 2024-09-11T23:28:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.333+0000 7fd285c13640 1 -- 172.21.15.43:0/521885399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2801a6c60 con 0x7fd280100ca0 2024-09-11T23:28:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.333+0000 7fd27cff9640 1 -- 172.21.15.43:0/521885399 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274010690 con 0x7fd280100ca0 2024-09-11T23:28:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.333+0000 7fd285c13640 1 -- 172.21.15.43:0/521885399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2801a7150 con 0x7fd280100ca0 2024-09-11T23:28:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.335+0000 7fd285c13640 1 -- 172.21.15.43:0/521885399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2801006d0 con 0x7fd280100ca0 2024-09-11T23:28:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.336+0000 7fd27cff9640 1 -- 172.21.15.43:0/521885399 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd2740046c0 con 0x7fd280100ca0 2024-09-11T23:28:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.337+0000 7fd27cff9640 1 --2- 172.21.15.43:0/521885399 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fd2680a2cf0 0x7fd2680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.337+0000 7fd27f7fe640 1 --2- 172.21.15.43:0/521885399 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fd2680a2cf0 0x7fd2680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.337+0000 7fd27cff9640 1 -- 172.21.15.43:0/521885399 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7fd2740cf870 con 0x7fd280100ca0 2024-09-11T23:28:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.338+0000 7fd27f7fe640 1 --2- 172.21.15.43:0/521885399 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fd2680a2cf0 0x7fd2680a51b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd270009900 tx=0x7fd2700058d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.341+0000 7fd27cff9640 1 -- 172.21.15.43:0/521885399 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd27409c490 con 0x7fd280100ca0 2024-09-11T23:28:28.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:28 smithi090 ceph-mon[35130]: pgmap v202: 97 pgs: 97 active+clean; 21 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 16 MiB/s wr, 2.87k op/s 2024-09-11T23:28:28.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:28 smithi097 ceph-mon[35562]: pgmap v202: 97 pgs: 97 active+clean; 21 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 16 MiB/s wr, 2.87k op/s 2024-09-11T23:28:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.566+0000 7fd285c13640 1 -- 172.21.15.43:0/521885399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd28002d7c0 con 0x7fd280100ca0 2024-09-11T23:28:28.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.566+0000 7fd27cff9640 1 -- 172.21.15.43:0/521885399 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11893 (secure 0 0 0) 0x7fd27409bd90 con 0x7fd280100ca0 2024-09-11T23:28:28.567 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:28:28.567 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"btime":"2024-09-11T23:25:40:446496+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:25:39.727562+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25571":{"gid":25571,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/568491708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":568491708},{"type":"v1","addr":"172.21.15.90:6835","nonce":568491708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:28:28.567 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-09-11T23:28:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.569+0000 7fd285c13640 1 -- 172.21.15.43:0/521885399 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fd2680a2cf0 msgr2=0x7fd2680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.569+0000 7fd285c13640 1 --2- 172.21.15.43:0/521885399 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fd2680a2cf0 0x7fd2680a51b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd270009900 tx=0x7fd2700058d0 comp rx=0 tx=0).stop 2024-09-11T23:28:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.570+0000 7fd285c13640 1 -- 172.21.15.43:0/521885399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd280100ca0 msgr2=0x7fd2801017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.570+0000 7fd285c13640 1 --2- 172.21.15.43:0/521885399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd280100ca0 0x7fd2801017b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd27400e9f0 tx=0x7fd27400eeb0 comp rx=0 tx=0).stop 2024-09-11T23:28:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.570+0000 7fd285c13640 1 -- 172.21.15.43:0/521885399 shutdown_connections 2024-09-11T23:28:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.570+0000 7fd285c13640 1 --2- 172.21.15.43:0/521885399 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fd2680a2cf0 0x7fd2680a51b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.570+0000 7fd285c13640 1 --2- 172.21.15.43:0/521885399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd280106f50 0x7fd280101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.570+0000 7fd285c13640 1 --2- 172.21.15.43:0/521885399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd280100ca0 0x7fd2801017b0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:28.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.570+0000 7fd285c13640 1 --2- 172.21.15.43:0/521885399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd2801002f0 0x7fd280101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:28.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.570+0000 7fd285c13640 1 -- 172.21.15.43:0/521885399 >> 172.21.15.43:0/521885399 conn(0x7fd280077220 msgr2=0x7fd280105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:28.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.570+0000 7fd285c13640 1 -- 172.21.15.43:0/521885399 shutdown_connections 2024-09-11T23:28:28.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:28.570+0000 7fd285c13640 1 -- 172.21.15.43:0/521885399 wait complete. 2024-09-11T23:28:29.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:29 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/521885399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:28:29.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:29 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/521885399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:28:29.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:29 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/521885399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:28:30.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:30 smithi043 ceph-mon[30950]: pgmap v203: 97 pgs: 97 active+clean; 21 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 21 MiB/s wr, 3.89k op/s 2024-09-11T23:28:30.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:30 smithi090 ceph-mon[35130]: pgmap v203: 97 pgs: 97 active+clean; 21 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 21 MiB/s wr, 3.89k op/s 2024-09-11T23:28:30.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:30 smithi097 ceph-mon[35562]: pgmap v203: 97 pgs: 97 active+clean; 21 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 21 MiB/s wr, 3.89k op/s 2024-09-11T23:28:32.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:32 smithi043 ceph-mon[30950]: pgmap v204: 97 pgs: 97 active+clean; 21 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 20 MiB/s wr, 3.63k op/s 2024-09-11T23:28:32.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:32 smithi090 ceph-mon[35130]: pgmap v204: 97 pgs: 97 active+clean; 21 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 20 MiB/s wr, 3.63k op/s 2024-09-11T23:28:32.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:32 smithi097 ceph-mon[35562]: pgmap v204: 97 pgs: 97 active+clean; 21 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 20 MiB/s wr, 3.63k op/s 2024-09-11T23:28:32.612 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 2 (i) to a remote file /var/run/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ops-733f7a10-1-mds.i.json 2024-09-11T23:28:32.613 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 ops --flags=locks --path=/var/run/ceph/ops-733f7a10-1-mds.i.json 2024-09-11T23:28:32.892 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:28:33.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.451+0000 7f11db011640 1 -- 172.21.15.43:0/3990807190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d4106f10 msgr2=0x7f11d410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:33.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.451+0000 7f11db011640 1 --2- 172.21.15.43:0/3990807190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d4106f10 0x7f11d410ea90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009ca0 tx=0x7f11cc02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.452+0000 7f11db011640 1 -- 172.21.15.43:0/3990807190 shutdown_connections 2024-09-11T23:28:33.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.452+0000 7f11db011640 1 --2- 172.21.15.43:0/3990807190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d4106f10 0x7f11d410ea90 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.452+0000 7f11db011640 1 --2- 172.21.15.43:0/3990807190 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11d4102790 0x7f11d41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.452+0000 7f11db011640 1 --2- 172.21.15.43:0/3990807190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11d4101de0 0x7f11d41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.452+0000 7f11db011640 1 -- 172.21.15.43:0/3990807190 >> 172.21.15.43:0/3990807190 conn(0x7f11d40fd8c0 msgr2=0x7f11d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:33.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.452+0000 7f11db011640 1 -- 172.21.15.43:0/3990807190 shutdown_connections 2024-09-11T23:28:33.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.452+0000 7f11db011640 1 -- 172.21.15.43:0/3990807190 wait complete. 2024-09-11T23:28:33.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.452+0000 7f11db011640 1 Processor -- start 2024-09-11T23:28:33.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.452+0000 7f11db011640 1 -- start start 2024-09-11T23:28:33.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.452+0000 7f11db011640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11d4101de0 0x7f11d4198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:33.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.452+0000 7f11db011640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11d4102790 0x7f11d4198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:33.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.452+0000 7f11db011640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d4106f10 0x7f11d419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:33.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.452+0000 7f11db011640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11d4111f70 con 0x7f11d4106f10 2024-09-11T23:28:33.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.452+0000 7f11db011640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f11d4111df0 con 0x7f11d4101de0 2024-09-11T23:28:33.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.452+0000 7f11db011640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41120f0 con 0x7f11d4102790 2024-09-11T23:28:33.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.453+0000 7f11d95ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d4106f10 0x7f11d419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:33.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.453+0000 7f11d8dad640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11d4101de0 0x7f11d4198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:33.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.453+0000 7f11d95ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d4106f10 0x7f11d419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37346/0 (socket says 172.21.15.43:37346) 2024-09-11T23:28:33.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.453+0000 7f11d95ae640 1 -- 172.21.15.43:0/610154561 learned_addr learned my addr 172.21.15.43:0/610154561 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:33.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.453+0000 7f11d3fff640 1 --2- 172.21.15.43:0/610154561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11d4102790 0x7f11d4198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:33.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.453+0000 7f11d95ae640 1 -- 172.21.15.43:0/610154561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11d4102790 msgr2=0x7f11d4198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:33.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.453+0000 7f11d95ae640 1 --2- 172.21.15.43:0/610154561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11d4102790 0x7f11d4198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.453+0000 7f11d95ae640 1 -- 172.21.15.43:0/610154561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11d4101de0 msgr2=0x7f11d4198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:33.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.453+0000 7f11d95ae640 1 --2- 172.21.15.43:0/610154561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11d4101de0 0x7f11d4198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.453+0000 7f11d95ae640 1 -- 172.21.15.43:0/610154561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d419d2e0 con 0x7f11d4106f10 2024-09-11T23:28:33.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.453+0000 7f11d8dad640 1 --2- 172.21.15.43:0/610154561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11d4101de0 0x7f11d4198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:33.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.453+0000 7f11d95ae640 1 --2- 172.21.15.43:0/610154561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d4106f10 0x7f11d419cc00 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f11cc031a20 tx=0x7f11cc031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:33.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.453+0000 7f11d1ffb640 1 -- 172.21.15.43:0/610154561 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc003b70 con 0x7f11d4106f10 2024-09-11T23:28:33.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.453+0000 7f11db011640 1 -- 172.21.15.43:0/610154561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d419d570 con 0x7f11d4106f10 2024-09-11T23:28:33.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.453+0000 7f11d1ffb640 1 -- 172.21.15.43:0/610154561 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11cc003d10 con 0x7f11d4106f10 2024-09-11T23:28:33.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.453+0000 7f11d1ffb640 1 -- 172.21.15.43:0/610154561 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc02fc80 con 0x7f11d4106f10 2024-09-11T23:28:33.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.454+0000 7f11db011640 1 -- 172.21.15.43:0/610154561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d41a4eb0 con 0x7f11d4106f10 2024-09-11T23:28:33.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.456+0000 7f11d1ffb640 1 -- 172.21.15.43:0/610154561 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f11cc0046a0 con 0x7f11d4106f10 2024-09-11T23:28:33.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.456+0000 7f11dd3ea740 1 Processor -- start 2024-09-11T23:28:33.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.456+0000 7f11dd3ea740 1 -- start start 2024-09-11T23:28:33.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11dd3ea740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615611918f0 0x561561191cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:33.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11dd3ea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561561192210 0x56156119c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:33.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11d1ffb640 1 --2- 172.21.15.43:0/610154561 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f11c40a2cf0 0x7f11c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:33.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11dd3ea740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56156119cad0 0x56156119eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:33.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11dd3ea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5615610f7c10 con 0x561561192210 2024-09-11T23:28:33.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11dd3ea740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56156109f520 con 0x5615611918f0 2024-09-11T23:28:33.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11dd3ea740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5615610f3bd0 con 0x56156119cad0 2024-09-11T23:28:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11d8dad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561561192210 0x56156119c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11d8dad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561561192210 0x56156119c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37362/0 (socket says 172.21.15.43:37362) 2024-09-11T23:28:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11d8dad640 1 -- 172.21.15.43:0/1788428620 learned_addr learned my addr 172.21.15.43:0/1788428620 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11d3fff640 1 --2- 172.21.15.43:0/1788428620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615611918f0 0x561561191cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11d95ae640 1 --2- 172.21.15.43:0/1788428620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56156119cad0 0x56156119eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11d3fff640 1 --2- 172.21.15.43:0/610154561 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f11c40a2cf0 0x7f11c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11d1ffb640 1 -- 172.21.15.43:0/610154561 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f11cc0f05b0 con 0x7f11d4106f10 2024-09-11T23:28:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11d3fff640 1 -- 172.21.15.43:0/1788428620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56156119cad0 msgr2=0x56156119eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11d3fff640 1 --2- 172.21.15.43:0/1788428620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56156119cad0 0x56156119eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11d3fff640 1 -- 172.21.15.43:0/1788428620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561561192210 msgr2=0x56156119c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11d3fff640 1 --2- 172.21.15.43:0/1788428620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561561192210 0x56156119c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11d3fff640 1 -- 172.21.15.43:0/1788428620 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561560ecc490 con 0x5615611918f0 2024-09-11T23:28:33.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11d8dad640 1 --2- 172.21.15.43:0/1788428620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561561192210 0x56156119c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11d95ae640 1 --2- 172.21.15.43:0/1788428620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56156119cad0 0x56156119eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11d3fff640 1 --2- 172.21.15.43:0/1788428620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615611918f0 0x561561191cd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f11c0014b70 tx=0x7f11c00117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72506f21341fc00a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.457+0000 7f11d3fff640 1 --2- 172.21.15.43:0/610154561 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f11c40a2cf0 0x7f11c40a51b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f11c0011cb0 tx=0x7f11c0009000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.458+0000 7f11beffd640 1 -- 172.21.15.43:0/1788428620 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c0009d60 con 0x5615611918f0 2024-09-11T23:28:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.458+0000 7f11beffd640 1 -- 172.21.15.43:0/1788428620 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11c0014590 con 0x5615611918f0 2024-09-11T23:28:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.458+0000 7f11beffd640 1 -- 172.21.15.43:0/1788428620 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c0003a70 con 0x5615611918f0 2024-09-11T23:28:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.458+0000 7f11dd3ea740 1 -- 172.21.15.43:0/1788428620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615611918f0 msgr2=0x561561191cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.458+0000 7f11dd3ea740 1 --2- 172.21.15.43:0/1788428620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615611918f0 0x561561191cd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f11c0014b70 tx=0x7f11c00117b0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.458+0000 7f11dd3ea740 1 -- 172.21.15.43:0/1788428620 shutdown_connections 2024-09-11T23:28:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.458+0000 7f11dd3ea740 1 --2- 172.21.15.43:0/1788428620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56156119cad0 0x56156119eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.458+0000 7f11dd3ea740 1 --2- 172.21.15.43:0/1788428620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561561192210 0x56156119c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.458+0000 7f11dd3ea740 1 --2- 172.21.15.43:0/1788428620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5615611918f0 0x561561191cd0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.458+0000 7f11dd3ea740 1 -- 172.21.15.43:0/1788428620 >> 172.21.15.43:0/1788428620 conn(0x56156118de80 msgr2=0x56156118e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.458+0000 7f11dd3ea740 1 -- 172.21.15.43:0/1788428620 shutdown_connections 2024-09-11T23:28:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.458+0000 7f11dd3ea740 1 -- 172.21.15.43:0/1788428620 wait complete. 2024-09-11T23:28:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.459+0000 7f11dd3ea740 1 -- start start 2024-09-11T23:28:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.459+0000 7f11dd3ea740 1 Processor -- start 2024-09-11T23:28:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.459+0000 7f11dd3ea740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615611918f0 0x561561199a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.459+0000 7f11dd3ea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561561192210 0x561561199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.459+0000 7f11dd3ea740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56156119cad0 0x56156119a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.459+0000 7f11dd3ea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56156119f6a0 con 0x561561192210 2024-09-11T23:28:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.459+0000 7f11dd3ea740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561561080700 con 0x56156119cad0 2024-09-11T23:28:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.459+0000 7f11dd3ea740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56156119f820 con 0x5615611918f0 2024-09-11T23:28:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.459+0000 7f11dd3ea740 10 client.0 _ll_register_callbacks cb 0x5615610a1ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.459+0000 7f11d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561561192210 0x561561199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.459+0000 7f11d95ae640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56156119cad0 0x56156119a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.459+0000 7f11d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561561192210 0x561561199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37370/0 (socket says 172.21.15.43:37370) 2024-09-11T23:28:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.459+0000 7f11d3fff640 1 -- 172.21.15.43:0/2275138957 learned_addr learned my addr 172.21.15.43:0/2275138957 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.459+0000 7f11d8dad640 1 --2- 172.21.15.43:0/2275138957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615611918f0 0x561561199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.460+0000 7f11d95ae640 1 -- 172.21.15.43:0/2275138957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615611918f0 msgr2=0x561561199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.460+0000 7f11d95ae640 1 --2- 172.21.15.43:0/2275138957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615611918f0 0x561561199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.460+0000 7f11d95ae640 1 -- 172.21.15.43:0/2275138957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561561192210 msgr2=0x561561199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.460+0000 7f11d95ae640 1 --2- 172.21.15.43:0/2275138957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561561192210 0x561561199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.460+0000 7f11d95ae640 1 -- 172.21.15.43:0/2275138957 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615611a8c70 con 0x56156119cad0 2024-09-11T23:28:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.460+0000 7f11d8dad640 1 --2- 172.21.15.43:0/2275138957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615611918f0 0x561561199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:28:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.460+0000 7f11d95ae640 1 --2- 172.21.15.43:0/2275138957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56156119cad0 0x56156119a500 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0ede10 tx=0x7f11cc02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94e1adfa8545de2b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.460+0000 7f11b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:28:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.460+0000 7f11b3fff640 1 -- 172.21.15.43:0/2275138957 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc0cf720 con 0x56156119cad0 2024-09-11T23:28:33.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.460+0000 7f11b3fff640 1 -- 172.21.15.43:0/2275138957 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f11cc0baf70 con 0x56156119cad0 2024-09-11T23:28:33.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.460+0000 7f11dd3ea740 1 -- 172.21.15.43:0/2275138957 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561561105340 con 0x56156119cad0 2024-09-11T23:28:33.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.460+0000 7f11b3fff640 1 -- 172.21.15.43:0/2275138957 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f11cc082070 con 0x56156119cad0 2024-09-11T23:28:33.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.460+0000 7f11dd3ea740 10 client.25922 fetch_fsmap learned FSMap version 30 2024-09-11T23:28:33.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.460+0000 7f11dd3ea740 1 -- 172.21.15.43:0/2275138957 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56156119f6a0 con 0x56156119cad0 2024-09-11T23:28:33.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.460+0000 7f11b3fff640 1 -- 172.21.15.43:0/2275138957 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561561105340 con 0x56156119cad0 2024-09-11T23:28:33.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.461+0000 7f11b3fff640 1 -- 172.21.15.43:0/2275138957 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f11cc0edf80 con 0x56156119cad0 2024-09-11T23:28:33.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.461+0000 7f11dd3ea740 10 client.25922 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:28:33.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.461+0000 7f11dd3ea740 10 client.25922 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T23:28:33.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.461+0000 7f11dd3ea740 20 client.25922 populate_metadata read hostname 'smithi043' 2024-09-11T23:28:33.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.461+0000 7f11dd3ea740 1 --2- 172.21.15.43:0/2275138957 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5615611acd90 0x5615611af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:33.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.461+0000 7f11dd3ea740 4 client.25922 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:28:33.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.461+0000 7f11dd3ea740 1 -- 172.21.15.43:0/2275138957 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615610ff690 con 0x5615611acd90 2024-09-11T23:28:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.461+0000 7f11d8dad640 1 --2- 172.21.15.43:0/2275138957 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5615611acd90 0x5615611af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.461+0000 7f11d8dad640 1 --2- 172.21.15.43:0/2275138957 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5615611acd90 0x5615611af1a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a43f08023b540c3e server_cookie=b5984fd16ee356d in_seq=0 out_seq=0 2024-09-11T23:28:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.461+0000 7f11b3fff640 10 client.25922 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:28:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.462+0000 7f11b3fff640 1 -- 172.21.15.43:0/2275138957 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5615610ff690 con 0x5615611acd90 2024-09-11T23:28:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.462+0000 7f11b3fff640 10 client.25922 handle_command_reply: tid=0 2024-09-11T23:28:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.462+0000 7f11dd3ea740 1 client.25922 shutdown 2024-09-11T23:28:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.462+0000 7f11dd3ea740 1 -- 172.21.15.43:0/2275138957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56156119cad0 msgr2=0x56156119a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.462+0000 7f11dd3ea740 1 --2- 172.21.15.43:0/2275138957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56156119cad0 0x56156119a500 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0ede10 tx=0x7f11cc02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.462+0000 7f11dd3ea740 1 -- 172.21.15.43:0/2275138957 shutdown_connections 2024-09-11T23:28:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.462+0000 7f11dd3ea740 1 --2- 172.21.15.43:0/2275138957 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5615611acd90 0x5615611af1a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.462+0000 7f11dd3ea740 1 --2- 172.21.15.43:0/2275138957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56156119cad0 0x56156119a500 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.462+0000 7f11dd3ea740 1 --2- 172.21.15.43:0/2275138957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561561192210 0x561561199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.462+0000 7f11dd3ea740 1 --2- 172.21.15.43:0/2275138957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615611918f0 0x561561199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.462+0000 7f11dd3ea740 1 -- 172.21.15.43:0/2275138957 >> 172.21.15.43:0/2275138957 conn(0x56156118de80 msgr2=0x56156118cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.462+0000 7f11b3fff640 0 client.25922 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:28:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.462+0000 7f11dd3ea740 1 -- 172.21.15.43:0/2275138957 shutdown_connections 2024-09-11T23:28:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.462+0000 7f11dd3ea740 1 -- 172.21.15.43:0/2275138957 wait complete. 2024-09-11T23:28:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.463+0000 7f11dd3ea740 20 client.25922 trim_cache size 0 max 16384 2024-09-11T23:28:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.479+0000 7f11dd3ea740 1 Processor -- start 2024-09-11T23:28:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.479+0000 7f11dd3ea740 1 -- start start 2024-09-11T23:28:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.479+0000 7f11dd3ea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56156118de80 0x561561198910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.479+0000 7f11dd3ea740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615611918f0 0x56156119a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.479+0000 7f11dd3ea740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561561192210 0x561561194350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.479+0000 7f11dd3ea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41ade10 con 0x56156118de80 2024-09-11T23:28:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.479+0000 7f11dd3ea740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41ab6e0 con 0x561561192210 2024-09-11T23:28:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.479+0000 7f11dd3ea740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41a9530 con 0x5615611918f0 2024-09-11T23:28:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.479+0000 7f11d8dad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56156118de80 0x561561198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:33.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.479+0000 7f11d3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615611918f0 0x56156119a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:33.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.479+0000 7f11d8dad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56156118de80 0x561561198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37378/0 (socket says 172.21.15.43:37378) 2024-09-11T23:28:33.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.479+0000 7f11d8dad640 1 -- 172.21.15.43:0/806682651 learned_addr learned my addr 172.21.15.43:0/806682651 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:33.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.479+0000 7f11d95ae640 1 --2- 172.21.15.43:0/806682651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561561192210 0x561561194350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:33.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.479+0000 7f11d3fff640 1 -- 172.21.15.43:0/806682651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561561192210 msgr2=0x561561194350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:33.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.479+0000 7f11d3fff640 1 --2- 172.21.15.43:0/806682651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561561192210 0x561561194350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.480+0000 7f11d3fff640 1 -- 172.21.15.43:0/806682651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56156118de80 msgr2=0x561561198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:33.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.480+0000 7f11d3fff640 1 --2- 172.21.15.43:0/806682651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56156118de80 0x561561198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.480+0000 7f11d3fff640 1 -- 172.21.15.43:0/806682651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56156118eb10 con 0x5615611918f0 2024-09-11T23:28:33.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.480+0000 7f11d3fff640 1 --2- 172.21.15.43:0/806682651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615611918f0 0x56156119a0b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f11c0003630 tx=0x7f11c0003690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=262309b8f41b24e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:33.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.480+0000 7f11be7fc640 1 -- 172.21.15.43:0/806682651 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c0034070 con 0x5615611918f0 2024-09-11T23:28:33.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.480+0000 7f11be7fc640 1 -- 172.21.15.43:0/806682651 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11c000c8e0 con 0x5615611918f0 2024-09-11T23:28:33.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.480+0000 7f11be7fc640 1 -- 172.21.15.43:0/806682651 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c0005610 con 0x5615611918f0 2024-09-11T23:28:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.481+0000 7f11dd3ea740 1 -- 172.21.15.43:0/806682651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615611918f0 msgr2=0x56156119a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.481+0000 7f11dd3ea740 1 --2- 172.21.15.43:0/806682651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615611918f0 0x56156119a0b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f11c0003630 tx=0x7f11c0003690 comp rx=0 tx=0).stop 2024-09-11T23:28:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.481+0000 7f11dd3ea740 1 -- 172.21.15.43:0/806682651 shutdown_connections 2024-09-11T23:28:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.481+0000 7f11dd3ea740 1 --2- 172.21.15.43:0/806682651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561561192210 0x561561194350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.481+0000 7f11dd3ea740 1 --2- 172.21.15.43:0/806682651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615611918f0 0x56156119a0b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.481+0000 7f11dd3ea740 1 --2- 172.21.15.43:0/806682651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56156118de80 0x561561198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.481+0000 7f11dd3ea740 1 -- 172.21.15.43:0/806682651 >> 172.21.15.43:0/806682651 conn(0x56156119a660 msgr2=0x56156118cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.481+0000 7f11dd3ea740 1 -- 172.21.15.43:0/806682651 shutdown_connections 2024-09-11T23:28:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.481+0000 7f11dd3ea740 1 -- 172.21.15.43:0/806682651 wait complete. 2024-09-11T23:28:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.481+0000 7f11dd3ea740 1 -- start start 2024-09-11T23:28:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.481+0000 7f11dd3ea740 1 Processor -- start 2024-09-11T23:28:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.482+0000 7f11dd3ea740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56156118de80 0x5615611a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.482+0000 7f11dd3ea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615611918f0 0x5615611a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.482+0000 7f11dd3ea740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561561192210 0x5615611a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.482+0000 7f11dd3ea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41a9ba0 con 0x5615611918f0 2024-09-11T23:28:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.482+0000 7f11dd3ea740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41ab010 con 0x561561192210 2024-09-11T23:28:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.482+0000 7f11dd3ea740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41aa270 con 0x56156118de80 2024-09-11T23:28:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.482+0000 7f11dd3ea740 10 client.0 _ll_register_callbacks cb 0x7f11cc0b8fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.482+0000 7f11d95ae640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561561192210 0x5615611a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.482+0000 7f11d95ae640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561561192210 0x5615611a1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:44758/0 (socket says 172.21.15.43:44758) 2024-09-11T23:28:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.482+0000 7f11d95ae640 1 -- 172.21.15.43:0/2472145760 learned_addr learned my addr 172.21.15.43:0/2472145760 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.482+0000 7f11d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615611918f0 0x5615611a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.482+0000 7f11d8dad640 1 --2- 172.21.15.43:0/2472145760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56156118de80 0x5615611a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.482+0000 7f11d95ae640 1 -- 172.21.15.43:0/2472145760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56156118de80 msgr2=0x5615611a4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.482+0000 7f11d95ae640 1 --2- 172.21.15.43:0/2472145760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56156118de80 0x5615611a4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.482+0000 7f11d95ae640 1 -- 172.21.15.43:0/2472145760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615611918f0 msgr2=0x5615611a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.482+0000 7f11d95ae640 1 --2- 172.21.15.43:0/2472145760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615611918f0 0x5615611a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.482+0000 7f11d95ae640 1 -- 172.21.15.43:0/2472145760 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615611a24c0 con 0x561561192210 2024-09-11T23:28:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.483+0000 7f11d95ae640 1 --2- 172.21.15.43:0/2472145760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561561192210 0x5615611a1f80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0baf20 tx=0x7f11cc0bc5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73ea7ca27f92779 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.483+0000 7f11b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:28:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.483+0000 7f11b3fff640 1 -- 172.21.15.43:0/2472145760 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc08f000 con 0x561561192210 2024-09-11T23:28:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.483+0000 7f11b3fff640 1 -- 172.21.15.43:0/2472145760 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f11cc0edec0 con 0x561561192210 2024-09-11T23:28:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.483+0000 7f11dd3ea740 1 -- 172.21.15.43:0/2472145760 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11c000c8e0 con 0x561561192210 2024-09-11T23:28:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.483+0000 7f11b3fff640 1 -- 172.21.15.43:0/2472145760 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f11cc0bd370 con 0x561561192210 2024-09-11T23:28:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.483+0000 7f11dd3ea740 10 client.25928 fetch_fsmap learned FSMap version 30 2024-09-11T23:28:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.483+0000 7f11dd3ea740 1 -- 172.21.15.43:0/2472145760 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5615611a24c0 con 0x561561192210 2024-09-11T23:28:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.483+0000 7f11b3fff640 1 -- 172.21.15.43:0/2472145760 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11c000c8e0 con 0x561561192210 2024-09-11T23:28:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.484+0000 7f11b3fff640 1 -- 172.21.15.43:0/2472145760 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f11cc0c53c0 con 0x561561192210 2024-09-11T23:28:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.484+0000 7f11dd3ea740 10 client.25928 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:28:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.484+0000 7f11dd3ea740 10 client.25928 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T23:28:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.484+0000 7f11dd3ea740 20 client.25928 populate_metadata read hostname 'smithi043' 2024-09-11T23:28:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.484+0000 7f11dd3ea740 1 --2- 172.21.15.43:0/2472145760 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x56156119cad0 0x56156121ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.484+0000 7f11dd3ea740 4 client.25928 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-733f7a10-1-mds.i.json"}] 2024-09-11T23:28:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.484+0000 7f11dd3ea740 1 -- 172.21.15.43:0/2472145760 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-733f7a10-1-mds.i.json"}) -- 0x56156107c210 con 0x56156119cad0 2024-09-11T23:28:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.484+0000 7f11d8dad640 1 --2- 172.21.15.43:0/2472145760 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x56156119cad0 0x56156121ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.484+0000 7f11d8dad640 1 --2- 172.21.15.43:0/2472145760 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x56156119cad0 0x56156121ac50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=603c994253f2be09 server_cookie=1218366b9ed911c9 in_seq=0 out_seq=0 2024-09-11T23:28:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.484+0000 7f11b3fff640 10 client.25928 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:28:33.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.491+0000 7f11b3fff640 1 -- 172.21.15.43:0/2472145760 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x56156107c210 con 0x56156119cad0 2024-09-11T23:28:33.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.491+0000 7f11b3fff640 10 client.25928 handle_command_reply: tid=0 2024-09-11T23:28:33.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.491+0000 7f11dd3ea740 1 client.25928 shutdown 2024-09-11T23:28:33.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.491+0000 7f11dd3ea740 1 -- 172.21.15.43:0/2472145760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561561192210 msgr2=0x5615611a1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:33.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.491+0000 7f11dd3ea740 1 --2- 172.21.15.43:0/2472145760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561561192210 0x5615611a1f80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0baf20 tx=0x7f11cc0bc5d0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.491+0000 7f11dd3ea740 1 -- 172.21.15.43:0/2472145760 shutdown_connections 2024-09-11T23:28:33.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.491+0000 7f11dd3ea740 1 --2- 172.21.15.43:0/2472145760 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x56156119cad0 0x56156121ac50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.491+0000 7f11dd3ea740 1 --2- 172.21.15.43:0/2472145760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561561192210 0x5615611a1f80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.491+0000 7f11b3fff640 0 client.25928 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:28:33.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.491+0000 7f11dd3ea740 1 --2- 172.21.15.43:0/2472145760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5615611918f0 0x5615611a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.491+0000 7f11dd3ea740 1 --2- 172.21.15.43:0/2472145760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56156118de80 0x5615611a4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.491+0000 7f11dd3ea740 1 -- 172.21.15.43:0/2472145760 >> 172.21.15.43:0/2472145760 conn(0x56156119a660 msgr2=0x56156119b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:33.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.491+0000 7f11dd3ea740 1 -- 172.21.15.43:0/2472145760 shutdown_connections 2024-09-11T23:28:33.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.491+0000 7f11dd3ea740 1 -- 172.21.15.43:0/2472145760 wait complete. 2024-09-11T23:28:33.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.492+0000 7f11dd3ea740 20 client.25928 trim_cache size 0 max 16384 2024-09-11T23:28:33.497 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:28:33.497 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/var/run/ceph/ops-733f7a10-1-mds.i.json" 2024-09-11T23:28:33.497 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:28:33.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.492+0000 7f11db011640 1 -- 172.21.15.43:0/610154561 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f11c40a2cf0 msgr2=0x7f11c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:33.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.492+0000 7f11db011640 1 --2- 172.21.15.43:0/610154561 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f11c40a2cf0 0x7f11c40a51b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f11c0011cb0 tx=0x7f11c0009000 comp rx=0 tx=0).stop 2024-09-11T23:28:33.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.492+0000 7f11db011640 1 -- 172.21.15.43:0/610154561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d4106f10 msgr2=0x7f11d419cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:33.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.492+0000 7f11db011640 1 --2- 172.21.15.43:0/610154561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d4106f10 0x7f11d419cc00 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f11cc031a20 tx=0x7f11cc031a50 comp rx=0 tx=0).stop 2024-09-11T23:28:33.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.492+0000 7f11db011640 1 -- 172.21.15.43:0/610154561 shutdown_connections 2024-09-11T23:28:33.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.492+0000 7f11db011640 1 --2- 172.21.15.43:0/610154561 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f11c40a2cf0 0x7f11c40a51b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.492+0000 7f11db011640 1 --2- 172.21.15.43:0/610154561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d4106f10 0x7f11d419cc00 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.492+0000 7f11db011640 1 --2- 172.21.15.43:0/610154561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11d4102790 0x7f11d4198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.492+0000 7f11db011640 1 --2- 172.21.15.43:0/610154561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11d4101de0 0x7f11d4198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:33.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.492+0000 7f11db011640 1 -- 172.21.15.43:0/610154561 >> 172.21.15.43:0/610154561 conn(0x7f11d40fd8c0 msgr2=0x7f11d40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:33.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.493+0000 7f11db011640 1 -- 172.21.15.43:0/610154561 shutdown_connections 2024-09-11T23:28:33.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:33.493+0000 7f11db011640 1 -- 172.21.15.43:0/610154561 wait complete. 2024-09-11T23:28:34.220 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 1 (h) to a remote file /var/run/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ops-733f7a10-1-mds.h.json 2024-09-11T23:28:34.221 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 ops --flags=locks --path=/var/run/ceph/ops-733f7a10-1-mds.h.json 2024-09-11T23:28:34.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:34 smithi043 ceph-mon[30950]: pgmap v205: 97 pgs: 97 active+clean; 21 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 22 MiB/s wr, 3.99k op/s 2024-09-11T23:28:34.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:34 smithi043 ceph-mon[30950]: scrub summary: active paths [/] 2024-09-11T23:28:34.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:34 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:28:34.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:34 smithi043 ceph-mon[30950]: scrub summary: active paths [/] 2024-09-11T23:28:34.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:34 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:28:34.477 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:28:34.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:34 smithi090 ceph-mon[35130]: pgmap v205: 97 pgs: 97 active+clean; 21 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 22 MiB/s wr, 3.99k op/s 2024-09-11T23:28:34.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:34 smithi090 ceph-mon[35130]: scrub summary: active paths [/] 2024-09-11T23:28:34.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:34 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:28:34.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:34 smithi090 ceph-mon[35130]: scrub summary: active paths [/] 2024-09-11T23:28:34.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:34 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:28:34.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:34 smithi097 ceph-mon[35562]: pgmap v205: 97 pgs: 97 active+clean; 21 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 22 MiB/s wr, 3.99k op/s 2024-09-11T23:28:34.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:34 smithi097 ceph-mon[35562]: scrub summary: active paths [/] 2024-09-11T23:28:34.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:34 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:28:34.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:34 smithi097 ceph-mon[35562]: scrub summary: active paths [/] 2024-09-11T23:28:34.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:34 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:28:35.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.107+0000 7efd77d15640 1 -- 172.21.15.43:0/4089025589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd70101430 msgr2=0x7efd7010d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:35.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.107+0000 7efd77d15640 1 --2- 172.21.15.43:0/4089025589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd70101430 0x7efd7010d960 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7efd68009cd0 tx=0x7efd6802a930 comp rx=0 tx=0).stop 2024-09-11T23:28:35.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.109+0000 7efd77d15640 1 -- 172.21.15.43:0/4089025589 shutdown_connections 2024-09-11T23:28:35.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.109+0000 7efd77d15640 1 --2- 172.21.15.43:0/4089025589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd70101430 0x7efd7010d960 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.109+0000 7efd77d15640 1 --2- 172.21.15.43:0/4089025589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd70100a90 0x7efd70100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.109+0000 7efd77d15640 1 --2- 172.21.15.43:0/4089025589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd70106a90 0x7efd70106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.109+0000 7efd77d15640 1 -- 172.21.15.43:0/4089025589 >> 172.21.15.43:0/4089025589 conn(0x7efd700fc7d0 msgr2=0x7efd700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:35.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.111+0000 7efd77d15640 1 -- 172.21.15.43:0/4089025589 shutdown_connections 2024-09-11T23:28:35.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.111+0000 7efd77d15640 1 -- 172.21.15.43:0/4089025589 wait complete. 2024-09-11T23:28:35.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.112+0000 7efd77d15640 1 Processor -- start 2024-09-11T23:28:35.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.112+0000 7efd77d15640 1 -- start start 2024-09-11T23:28:35.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.112+0000 7efd77d15640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd70100a90 0x7efd70073c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:35.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.112+0000 7efd77d15640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd70101430 0x7efd7006ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:35.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.112+0000 7efd77d15640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd70106a90 0x7efd7006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:35.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.112+0000 7efd77d15640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efd7006c660 con 0x7efd70100a90 2024-09-11T23:28:35.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.112+0000 7efd77d15640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efd7006c4e0 con 0x7efd70101430 2024-09-11T23:28:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.112+0000 7efd77d15640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efd7006c7e0 con 0x7efd70106a90 2024-09-11T23:28:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.113+0000 7efd752b0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd70101430 0x7efd7006ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.113+0000 7efd762b2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd70106a90 0x7efd7006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.113+0000 7efd762b2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd70106a90 0x7efd7006d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:56568/0 (socket says 172.21.15.43:56568) 2024-09-11T23:28:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.113+0000 7efd762b2640 1 -- 172.21.15.43:0/2327930887 learned_addr learned my addr 172.21.15.43:0/2327930887 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.113+0000 7efd762b2640 1 -- 172.21.15.43:0/2327930887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd70101430 msgr2=0x7efd7006ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.113+0000 7efd762b2640 1 --2- 172.21.15.43:0/2327930887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd70101430 0x7efd7006ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.113+0000 7efd762b2640 1 -- 172.21.15.43:0/2327930887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd70100a90 msgr2=0x7efd70073c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.113+0000 7efd762b2640 1 --2- 172.21.15.43:0/2327930887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd70100a90 0x7efd70073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.113+0000 7efd762b2640 1 -- 172.21.15.43:0/2327930887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7006dc20 con 0x7efd70106a90 2024-09-11T23:28:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.114+0000 7efd762b2640 1 --2- 172.21.15.43:0/2327930887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd70106a90 0x7efd7006d360 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efd680316b0 tx=0x7efd680316e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.114+0000 7efd66ffd640 1 -- 172.21.15.43:0/2327930887 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68004040 con 0x7efd70106a90 2024-09-11T23:28:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.114+0000 7efd66ffd640 1 -- 172.21.15.43:0/2327930887 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd6802d070 con 0x7efd70106a90 2024-09-11T23:28:35.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.115+0000 7efd66ffd640 1 -- 172.21.15.43:0/2327930887 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6802fc60 con 0x7efd70106a90 2024-09-11T23:28:35.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.116+0000 7efd77d15640 1 -- 172.21.15.43:0/2327930887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd70071840 con 0x7efd70106a90 2024-09-11T23:28:35.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.116+0000 7efd77d15640 1 -- 172.21.15.43:0/2327930887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd70071d00 con 0x7efd70106a90 2024-09-11T23:28:35.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.120+0000 7efd7a0ee740 1 Processor -- start 2024-09-11T23:28:35.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.121+0000 7efd66ffd640 1 -- 172.21.15.43:0/2327930887 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7efd680041e0 con 0x7efd70106a90 2024-09-11T23:28:35.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.122+0000 7efd66ffd640 1 --2- 172.21.15.43:0/2327930887 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7efd5c0a2d60 0x7efd5c0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:35.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.122+0000 7efd66ffd640 1 -- 172.21.15.43:0/2327930887 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7efd680f08a0 con 0x7efd70106a90 2024-09-11T23:28:35.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.123+0000 7efd7a0ee740 1 -- start start 2024-09-11T23:28:35.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.123+0000 7efd7a0ee740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc7c8fa8e0 0x55cc7c8facc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:35.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.123+0000 7efd7a0ee740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c8fb200 0x55cc7c905580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:35.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.123+0000 7efd7a0ee740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc7c905ac0 0x55cc7c907eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:35.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.123+0000 7efd7a0ee740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cc7c860c10 con 0x55cc7c8fa8e0 2024-09-11T23:28:35.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.123+0000 7efd7a0ee740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55cc7c808520 con 0x55cc7c8fb200 2024-09-11T23:28:35.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.123+0000 7efd7a0ee740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cc7c85cbd0 con 0x55cc7c905ac0 2024-09-11T23:28:35.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.124+0000 7efd752b0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c8fb200 0x55cc7c905580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:35.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.124+0000 7efd752b0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c8fb200 0x55cc7c905580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:44788/0 (socket says 172.21.15.43:44788) 2024-09-11T23:28:35.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.124+0000 7efd752b0640 1 -- 172.21.15.43:0/1132956413 learned_addr learned my addr 172.21.15.43:0/1132956413 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:35.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.124+0000 7efd762b2640 1 --2- 172.21.15.43:0/1132956413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc7c905ac0 0x55cc7c907eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:35.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.124+0000 7efd75ab1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc7c8fa8e0 0x55cc7c8facc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:35.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.124+0000 7efd752b0640 1 -- 172.21.15.43:0/1132956413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc7c905ac0 msgr2=0x55cc7c907eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:35.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.124+0000 7efd752b0640 1 --2- 172.21.15.43:0/1132956413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc7c905ac0 0x55cc7c907eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.124+0000 7efd752b0640 1 -- 172.21.15.43:0/1132956413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc7c8fa8e0 msgr2=0x55cc7c8facc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:35.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.124+0000 7efd752b0640 1 --2- 172.21.15.43:0/1132956413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc7c8fa8e0 0x55cc7c8facc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.124+0000 7efd752b0640 1 -- 172.21.15.43:0/1132956413 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc7c635490 con 0x55cc7c8fb200 2024-09-11T23:28:35.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.124+0000 7efd752b0640 1 --2- 172.21.15.43:0/1132956413 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c8fb200 0x55cc7c905580 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efd60009b20 tx=0x7efd60010250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13905cc407337260 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:35.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.125+0000 7efd5bfff640 1 -- 172.21.15.43:0/1132956413 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60018020 con 0x55cc7c8fb200 2024-09-11T23:28:35.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.125+0000 7efd5bfff640 1 -- 172.21.15.43:0/1132956413 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd60010c90 con 0x55cc7c8fb200 2024-09-11T23:28:35.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.126+0000 7efd752b0640 1 --2- 172.21.15.43:0/2327930887 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7efd5c0a2d60 0x7efd5c0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:35.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.126+0000 7efd7a0ee740 1 -- 172.21.15.43:0/1132956413 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c8fb200 msgr2=0x55cc7c905580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:35.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.126+0000 7efd7a0ee740 1 --2- 172.21.15.43:0/1132956413 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c8fb200 0x55cc7c905580 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efd60009b20 tx=0x7efd60010250 comp rx=0 tx=0).stop 2024-09-11T23:28:35.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.128+0000 7efd7a0ee740 1 -- 172.21.15.43:0/1132956413 shutdown_connections 2024-09-11T23:28:35.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.128+0000 7efd7a0ee740 1 --2- 172.21.15.43:0/1132956413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc7c905ac0 0x55cc7c907eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.128+0000 7efd7a0ee740 1 --2- 172.21.15.43:0/1132956413 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c8fb200 0x55cc7c905580 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.128+0000 7efd7a0ee740 1 --2- 172.21.15.43:0/1132956413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc7c8fa8e0 0x55cc7c8facc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.128+0000 7efd7a0ee740 1 -- 172.21.15.43:0/1132956413 >> 172.21.15.43:0/1132956413 conn(0x55cc7c8f6e70 msgr2=0x55cc7c8f7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:35.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.130+0000 7efd7a0ee740 1 -- 172.21.15.43:0/1132956413 shutdown_connections 2024-09-11T23:28:35.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.131+0000 7efd7a0ee740 1 -- 172.21.15.43:0/1132956413 wait complete. 2024-09-11T23:28:35.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.131+0000 7efd7a0ee740 1 -- start start 2024-09-11T23:28:35.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.131+0000 7efd7a0ee740 1 Processor -- start 2024-09-11T23:28:35.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.133+0000 7efd7a0ee740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c8fa8e0 0x55cc7c902a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:35.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.133+0000 7efd7a0ee740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc7c8fb200 0x55cc7c902f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:35.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.133+0000 7efd7a0ee740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc7c905ac0 0x55cc7c9034d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:35.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.133+0000 7efd7a0ee740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cc7c9086e0 con 0x55cc7c905ac0 2024-09-11T23:28:35.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.133+0000 7efd7a0ee740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55cc7c7e9700 con 0x55cc7c8fa8e0 2024-09-11T23:28:35.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.133+0000 7efd7a0ee740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cc7c908860 con 0x55cc7c8fb200 2024-09-11T23:28:35.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.133+0000 7efd7a0ee740 10 client.0 _ll_register_callbacks cb 0x55cc7c80aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:35.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.133+0000 7efd752b0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc7c8fb200 0x55cc7c902f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:35.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.133+0000 7efd752b0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc7c8fb200 0x55cc7c902f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:56592/0 (socket says 172.21.15.43:56592) 2024-09-11T23:28:35.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.133+0000 7efd752b0640 1 -- 172.21.15.43:0/3010709869 learned_addr learned my addr 172.21.15.43:0/3010709869 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:35.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.134+0000 7efd762b2640 1 --2- 172.21.15.43:0/3010709869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc7c905ac0 0x55cc7c9034d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:35.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.135+0000 7efd752b0640 1 -- 172.21.15.43:0/3010709869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c8fa8e0 msgr2=0x55cc7c902a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:28:35.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.135+0000 7efd752b0640 1 --2- 172.21.15.43:0/3010709869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c8fa8e0 0x55cc7c902a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.135+0000 7efd752b0640 1 -- 172.21.15.43:0/3010709869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc7c905ac0 msgr2=0x55cc7c9034d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:35.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.135+0000 7efd752b0640 1 --2- 172.21.15.43:0/3010709869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc7c905ac0 0x55cc7c9034d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.135+0000 7efd752b0640 1 -- 172.21.15.43:0/3010709869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc7c911d00 con 0x55cc7c8fb200 2024-09-11T23:28:35.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.136+0000 7efd752b0640 1 --2- 172.21.15.43:0/3010709869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc7c8fb200 0x55cc7c902f90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efd60010750 tx=0x7efd60013b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88d7397db0a3ab7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:35.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.136+0000 7efd752b0640 1 --2- 172.21.15.43:0/2327930887 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7efd5c0a2d60 0x7efd5c0a5220 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efd60009b20 tx=0x7efd60010250 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:35.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.136+0000 7efd597fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-11T23:28:35.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.136+0000 7efd597fa640 1 -- 172.21.15.43:0/3010709869 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60018030 con 0x55cc7c8fb200 2024-09-11T23:28:35.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.137+0000 7efd7a0ee740 1 -- 172.21.15.43:0/3010709869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cc7c86e340 con 0x55cc7c8fb200 2024-09-11T23:28:35.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.137+0000 7efd597fa640 1 -- 172.21.15.43:0/3010709869 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7efd60003b00 con 0x55cc7c8fb200 2024-09-11T23:28:35.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.137+0000 7efd597fa640 1 -- 172.21.15.43:0/3010709869 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efd60038050 con 0x55cc7c8fb200 2024-09-11T23:28:35.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.137+0000 7efd7a0ee740 10 client.25894 fetch_fsmap learned FSMap version 30 2024-09-11T23:28:35.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.138+0000 7efd7a0ee740 1 -- 172.21.15.43:0/3010709869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cc7c914ac0 con 0x55cc7c8fb200 2024-09-11T23:28:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.138+0000 7efd597fa640 1 -- 172.21.15.43:0/3010709869 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cc7c86e340 con 0x55cc7c8fb200 2024-09-11T23:28:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.138+0000 7efd597fa640 1 -- 172.21.15.43:0/3010709869 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7efd600344e0 con 0x55cc7c8fb200 2024-09-11T23:28:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.138+0000 7efd7a0ee740 10 client.25894 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:28:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.139+0000 7efd7a0ee740 10 client.25894 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T23:28:35.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.139+0000 7efd7a0ee740 20 client.25894 populate_metadata read hostname 'smithi043' 2024-09-11T23:28:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.139+0000 7efd7a0ee740 1 --2- 172.21.15.43:0/3010709869 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55cc7c9163a0 0x55cc7c918860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.139+0000 7efd7a0ee740 4 client.25894 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:28:35.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.139+0000 7efd7a0ee740 1 -- 172.21.15.43:0/3010709869 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc7c868690 con 0x55cc7c9163a0 2024-09-11T23:28:35.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.143+0000 7efd75ab1640 1 --2- 172.21.15.43:0/3010709869 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55cc7c9163a0 0x55cc7c918860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.160+0000 7efd75ab1640 1 --2- 172.21.15.43:0/3010709869 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55cc7c9163a0 0x55cc7c918860 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c07fbab30178e6b0 server_cookie=f460eda53da42f26 in_seq=0 out_seq=0 2024-09-11T23:28:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.160+0000 7efd597fa640 10 client.25894 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:28:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.161+0000 7efd597fa640 1 -- 172.21.15.43:0/3010709869 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc7c868690 con 0x55cc7c9163a0 2024-09-11T23:28:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.161+0000 7efd597fa640 10 client.25894 handle_command_reply: tid=0 2024-09-11T23:28:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.161+0000 7efd7a0ee740 1 client.25894 shutdown 2024-09-11T23:28:35.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.168+0000 7efd7a0ee740 1 -- 172.21.15.43:0/3010709869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc7c8fb200 msgr2=0x55cc7c902f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:35.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.168+0000 7efd7a0ee740 1 --2- 172.21.15.43:0/3010709869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc7c8fb200 0x55cc7c902f90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efd60010750 tx=0x7efd60013b10 comp rx=0 tx=0).stop 2024-09-11T23:28:35.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.168+0000 7efd7a0ee740 1 -- 172.21.15.43:0/3010709869 shutdown_connections 2024-09-11T23:28:35.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.168+0000 7efd7a0ee740 1 --2- 172.21.15.43:0/3010709869 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55cc7c9163a0 0x55cc7c918860 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.168+0000 7efd7a0ee740 1 --2- 172.21.15.43:0/3010709869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc7c905ac0 0x55cc7c9034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.168+0000 7efd7a0ee740 1 --2- 172.21.15.43:0/3010709869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc7c8fb200 0x55cc7c902f90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.168+0000 7efd7a0ee740 1 --2- 172.21.15.43:0/3010709869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c8fa8e0 0x55cc7c902a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.168+0000 7efd7a0ee740 1 -- 172.21.15.43:0/3010709869 >> 172.21.15.43:0/3010709869 conn(0x55cc7c8f6e70 msgr2=0x55cc7c8fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:35.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.168+0000 7efd597fa640 0 client.25894 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:28:35.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.169+0000 7efd7a0ee740 1 -- 172.21.15.43:0/3010709869 shutdown_connections 2024-09-11T23:28:35.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.169+0000 7efd7a0ee740 1 -- 172.21.15.43:0/3010709869 wait complete. 2024-09-11T23:28:35.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.169+0000 7efd7a0ee740 20 client.25894 trim_cache size 0 max 16384 2024-09-11T23:28:35.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.201+0000 7efd7a0ee740 1 Processor -- start 2024-09-11T23:28:35.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.204+0000 7efd7a0ee740 1 -- start start 2024-09-11T23:28:35.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.204+0000 7efd7a0ee740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c8fa8e0 0x55cc7c91e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:35.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.204+0000 7efd7a0ee740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc7c8fb200 0x55cc7c91e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:35.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.205+0000 7efd7a0ee740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc7c905ac0 0x55cc7c8ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:35.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.205+0000 7efd75ab1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c8fa8e0 0x55cc7c91e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:35.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.205+0000 7efd75ab1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c8fa8e0 0x55cc7c91e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:44794/0 (socket says 172.21.15.43:44794) 2024-09-11T23:28:35.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.205+0000 7efd75ab1640 1 -- 172.21.15.43:0/145769801 learned_addr learned my addr 172.21.15.43:0/145769801 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:35.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.205+0000 7efd7a0ee740 1 -- 172.21.15.43:0/145769801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efd701c76f0 con 0x55cc7c8fb200 2024-09-11T23:28:35.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.205+0000 7efd7a0ee740 1 -- 172.21.15.43:0/145769801 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efd701c4fc0 con 0x55cc7c8fa8e0 2024-09-11T23:28:35.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.205+0000 7efd7a0ee740 1 -- 172.21.15.43:0/145769801 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efd701c2e10 con 0x55cc7c905ac0 2024-09-11T23:28:35.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.206+0000 7efd75ab1640 1 -- 172.21.15.43:0/145769801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc7c905ac0 msgr2=0x55cc7c8ffcd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:28:35.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.206+0000 7efd75ab1640 1 --2- 172.21.15.43:0/145769801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc7c905ac0 0x55cc7c8ffcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.206+0000 7efd75ab1640 1 -- 172.21.15.43:0/145769801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc7c8fb200 msgr2=0x55cc7c91e6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:28:35.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.206+0000 7efd75ab1640 1 --2- 172.21.15.43:0/145769801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc7c8fb200 0x55cc7c91e6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.206+0000 7efd75ab1640 1 -- 172.21.15.43:0/145769801 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc7c918ee0 con 0x55cc7c8fa8e0 2024-09-11T23:28:35.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.206+0000 7efd75ab1640 1 --2- 172.21.15.43:0/145769801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c8fa8e0 0x55cc7c91e170 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55cc7c9190e0 tx=0x7efd6c00bd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5340cf98baf8a939 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:35.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.207+0000 7efd5b7fe640 1 -- 172.21.15.43:0/145769801 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c022070 con 0x55cc7c8fa8e0 2024-09-11T23:28:35.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.207+0000 7efd5b7fe640 1 -- 172.21.15.43:0/145769801 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd6c014dc0 con 0x55cc7c8fa8e0 2024-09-11T23:28:35.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.207+0000 7efd5b7fe640 1 -- 172.21.15.43:0/145769801 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c01dcd0 con 0x55cc7c8fa8e0 2024-09-11T23:28:35.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.208+0000 7efd7a0ee740 1 -- 172.21.15.43:0/145769801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c8fa8e0 msgr2=0x55cc7c91e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:35.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.208+0000 7efd7a0ee740 1 --2- 172.21.15.43:0/145769801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c8fa8e0 0x55cc7c91e170 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55cc7c9190e0 tx=0x7efd6c00bd40 comp rx=0 tx=0).stop 2024-09-11T23:28:35.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.209+0000 7efd7a0ee740 1 -- 172.21.15.43:0/145769801 shutdown_connections 2024-09-11T23:28:35.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.209+0000 7efd7a0ee740 1 --2- 172.21.15.43:0/145769801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc7c905ac0 0x55cc7c8ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.209+0000 7efd7a0ee740 1 --2- 172.21.15.43:0/145769801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc7c8fb200 0x55cc7c91e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.209+0000 7efd7a0ee740 1 --2- 172.21.15.43:0/145769801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c8fa8e0 0x55cc7c91e170 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.209+0000 7efd7a0ee740 1 -- 172.21.15.43:0/145769801 >> 172.21.15.43:0/145769801 conn(0x55cc7c8f6e70 msgr2=0x55cc7c904ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:35.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.210+0000 7efd7a0ee740 1 -- 172.21.15.43:0/145769801 shutdown_connections 2024-09-11T23:28:35.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.210+0000 7efd7a0ee740 1 -- 172.21.15.43:0/145769801 wait complete. 2024-09-11T23:28:35.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.211+0000 7efd7a0ee740 1 -- start start 2024-09-11T23:28:35.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.211+0000 7efd7a0ee740 1 Processor -- start 2024-09-11T23:28:35.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.212+0000 7efd7a0ee740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc7c8fa8e0 0x55cc7c90e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:35.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.212+0000 7efd7a0ee740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc7c8fb200 0x55cc7c8ff5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:35.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.212+0000 7efd7a0ee740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c905ac0 0x55cc7c8ffb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:35.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.212+0000 7efd7a0ee740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efd701c3480 con 0x55cc7c8fa8e0 2024-09-11T23:28:35.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.212+0000 7efd762b2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c905ac0 0x55cc7c8ffb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:35.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.212+0000 7efd762b2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c905ac0 0x55cc7c8ffb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:44802/0 (socket says 172.21.15.43:44802) 2024-09-11T23:28:35.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.212+0000 7efd762b2640 1 -- 172.21.15.43:0/4021026487 learned_addr learned my addr 172.21.15.43:0/4021026487 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:35.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.212+0000 7efd7a0ee740 1 -- 172.21.15.43:0/4021026487 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efd701c48f0 con 0x55cc7c905ac0 2024-09-11T23:28:35.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.212+0000 7efd7a0ee740 1 -- 172.21.15.43:0/4021026487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efd701c3b50 con 0x55cc7c8fb200 2024-09-11T23:28:35.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.213+0000 7efd7a0ee740 10 client.0 _ll_register_callbacks cb 0x7efd60009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:35.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.213+0000 7efd762b2640 1 -- 172.21.15.43:0/4021026487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc7c8fb200 msgr2=0x55cc7c8ff5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:28:35.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.213+0000 7efd762b2640 1 --2- 172.21.15.43:0/4021026487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc7c8fb200 0x55cc7c8ff5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.213+0000 7efd762b2640 1 -- 172.21.15.43:0/4021026487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc7c8fa8e0 msgr2=0x55cc7c90e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:35.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.213+0000 7efd762b2640 1 --2- 172.21.15.43:0/4021026487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc7c8fa8e0 0x55cc7c90e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.213+0000 7efd762b2640 1 -- 172.21.15.43:0/4021026487 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc7c900150 con 0x55cc7c905ac0 2024-09-11T23:28:35.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.213+0000 7efd7a0ee740 1 -- 172.21.15.43:0/4021026487 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd6c014dc0 con 0x55cc7c905ac0 2024-09-11T23:28:35.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.213+0000 7efd762b2640 1 --2- 172.21.15.43:0/4021026487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c905ac0 0x55cc7c8ffb10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efd680b92d0 tx=0x7efd680bcc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12ddafea651a2e4a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:35.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.214+0000 7efd597fa640 10 client.25940 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:28:35.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.218+0000 7efd597fa640 1 -- 172.21.15.43:0/4021026487 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd680d9020 con 0x55cc7c905ac0 2024-09-11T23:28:35.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.218+0000 7efd597fa640 1 -- 172.21.15.43:0/4021026487 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7efd680b94e0 con 0x55cc7c905ac0 2024-09-11T23:28:35.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.218+0000 7efd597fa640 1 -- 172.21.15.43:0/4021026487 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efd680bd570 con 0x55cc7c905ac0 2024-09-11T23:28:35.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.218+0000 7efd7a0ee740 10 client.25940 fetch_fsmap learned FSMap version 30 2024-09-11T23:28:35.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.218+0000 7efd7a0ee740 1 -- 172.21.15.43:0/4021026487 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55cc7c980ef0 con 0x55cc7c905ac0 2024-09-11T23:28:35.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.223+0000 7efd597fa640 1 -- 172.21.15.43:0/4021026487 <== mon.2 v2:172.21.15.90:3300/0 4 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7efd680ee210 con 0x55cc7c905ac0 2024-09-11T23:28:35.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.223+0000 7efd7a0ee740 10 client.25940 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:28:35.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.223+0000 7efd7a0ee740 10 client.25940 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T23:28:35.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.223+0000 7efd7a0ee740 20 client.25940 populate_metadata read hostname 'smithi043' 2024-09-11T23:28:35.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.223+0000 7efd7a0ee740 1 --2- 172.21.15.43:0/4021026487 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55cc7c9813c0 0x55cc7c983820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:35.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.223+0000 7efd7a0ee740 4 client.25940 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-733f7a10-1-mds.h.json"}] 2024-09-11T23:28:35.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.223+0000 7efd7a0ee740 1 -- 172.21.15.43:0/4021026487 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-733f7a10-1-mds.h.json"}) -- 0x7efd6c014dc0 con 0x55cc7c9813c0 2024-09-11T23:28:35.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.224+0000 7efd75ab1640 1 --2- 172.21.15.43:0/4021026487 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55cc7c9813c0 0x55cc7c983820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:35.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.225+0000 7efd75ab1640 1 --2- 172.21.15.43:0/4021026487 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55cc7c9813c0 0x55cc7c983820 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64213129edc22172 server_cookie=3bcb8c8a71c48f1 in_seq=0 out_seq=0 2024-09-11T23:28:35.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.225+0000 7efd597fa640 10 client.25940 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:28:35.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.237+0000 7efd597fa640 1 -- 172.21.15.43:0/4021026487 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x7efd6c014dc0 con 0x55cc7c9813c0 2024-09-11T23:28:35.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.237+0000 7efd597fa640 10 client.25940 handle_command_reply: tid=0 2024-09-11T23:28:35.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.237+0000 7efd7a0ee740 1 client.25940 shutdown 2024-09-11T23:28:35.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.242+0000 7efd7a0ee740 1 -- 172.21.15.43:0/4021026487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c905ac0 msgr2=0x55cc7c8ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:35.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.242+0000 7efd7a0ee740 1 --2- 172.21.15.43:0/4021026487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c905ac0 0x55cc7c8ffb10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efd680b92d0 tx=0x7efd680bcc40 comp rx=0 tx=0).stop 2024-09-11T23:28:35.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.242+0000 7efd7a0ee740 1 -- 172.21.15.43:0/4021026487 shutdown_connections 2024-09-11T23:28:35.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.243+0000 7efd7a0ee740 1 --2- 172.21.15.43:0/4021026487 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55cc7c9813c0 0x55cc7c983820 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.243+0000 7efd7a0ee740 1 --2- 172.21.15.43:0/4021026487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc7c905ac0 0x55cc7c8ffb10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.243+0000 7efd597fa640 0 client.25940 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:28:35.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.243+0000 7efd7a0ee740 1 --2- 172.21.15.43:0/4021026487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cc7c8fb200 0x55cc7c8ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.243+0000 7efd7a0ee740 1 --2- 172.21.15.43:0/4021026487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cc7c8fa8e0 0x55cc7c90e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.243+0000 7efd7a0ee740 1 -- 172.21.15.43:0/4021026487 >> 172.21.15.43:0/4021026487 conn(0x55cc7c8f6e70 msgr2=0x55cc7c9029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:35.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.244+0000 7efd7a0ee740 1 -- 172.21.15.43:0/4021026487 shutdown_connections 2024-09-11T23:28:35.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.245+0000 7efd7a0ee740 1 -- 172.21.15.43:0/4021026487 wait complete. 2024-09-11T23:28:35.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.245+0000 7efd7a0ee740 20 client.25940 trim_cache size 0 max 16384 2024-09-11T23:28:35.245 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:28:35.245 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/var/run/ceph/ops-733f7a10-1-mds.h.json" 2024-09-11T23:28:35.245 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:28:35.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.246+0000 7efd77d15640 1 -- 172.21.15.43:0/2327930887 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7efd5c0a2d60 msgr2=0x7efd5c0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:35.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.246+0000 7efd77d15640 1 --2- 172.21.15.43:0/2327930887 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7efd5c0a2d60 0x7efd5c0a5220 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efd60009b20 tx=0x7efd60010250 comp rx=0 tx=0).stop 2024-09-11T23:28:35.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.246+0000 7efd77d15640 1 -- 172.21.15.43:0/2327930887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd70106a90 msgr2=0x7efd7006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:35.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.246+0000 7efd77d15640 1 --2- 172.21.15.43:0/2327930887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd70106a90 0x7efd7006d360 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efd680316b0 tx=0x7efd680316e0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.246+0000 7efd77d15640 1 -- 172.21.15.43:0/2327930887 shutdown_connections 2024-09-11T23:28:35.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.246+0000 7efd77d15640 1 --2- 172.21.15.43:0/2327930887 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7efd5c0a2d60 0x7efd5c0a5220 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.246+0000 7efd77d15640 1 --2- 172.21.15.43:0/2327930887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd70106a90 0x7efd7006d360 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.246+0000 7efd77d15640 1 --2- 172.21.15.43:0/2327930887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd70101430 0x7efd7006ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.246+0000 7efd77d15640 1 --2- 172.21.15.43:0/2327930887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd70100a90 0x7efd70073c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:35.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.247+0000 7efd77d15640 1 -- 172.21.15.43:0/2327930887 >> 172.21.15.43:0/2327930887 conn(0x7efd700fc7d0 msgr2=0x7efd700fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:35.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.247+0000 7efd77d15640 1 -- 172.21.15.43:0/2327930887 shutdown_connections 2024-09-11T23:28:35.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:35.247+0000 7efd77d15640 1 -- 172.21.15.43:0/2327930887 wait complete. 2024-09-11T23:28:35.941 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 0 (b) to a remote file /var/run/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ops-733f7a10-1-mds.b.json 2024-09-11T23:28:35.942 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 ops --flags=locks --path=/var/run/ceph/ops-733f7a10-1-mds.b.json 2024-09-11T23:28:36.204 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:28:36.325 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:36 smithi043 ceph-mon[30950]: pgmap v206: 97 pgs: 97 active+clean; 21 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 18 MiB/s wr, 3.23k op/s 2024-09-11T23:28:36.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:36 smithi090 ceph-mon[35130]: pgmap v206: 97 pgs: 97 active+clean; 21 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 18 MiB/s wr, 3.23k op/s 2024-09-11T23:28:36.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:36 smithi097 ceph-mon[35562]: pgmap v206: 97 pgs: 97 active+clean; 21 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 18 MiB/s wr, 3.23k op/s 2024-09-11T23:28:36.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.714+0000 7f7f71afe640 1 -- 172.21.15.43:0/703835232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c102790 msgr2=0x7f7f6c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:36.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.714+0000 7f7f71afe640 1 --2- 172.21.15.43:0/703835232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c102790 0x7f7f6c1067e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009900 tx=0x7f7f6002a3f0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.714+0000 7f7f71afe640 1 -- 172.21.15.43:0/703835232 shutdown_connections 2024-09-11T23:28:36.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.714+0000 7f7f71afe640 1 --2- 172.21.15.43:0/703835232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f6c106f10 0x7f7f6c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.714+0000 7f7f71afe640 1 --2- 172.21.15.43:0/703835232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c102790 0x7f7f6c1067e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.714+0000 7f7f71afe640 1 --2- 172.21.15.43:0/703835232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f6c101de0 0x7f7f6c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.714+0000 7f7f71afe640 1 -- 172.21.15.43:0/703835232 >> 172.21.15.43:0/703835232 conn(0x7f7f6c0fd8c0 msgr2=0x7f7f6c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:36.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.714+0000 7f7f71afe640 1 -- 172.21.15.43:0/703835232 shutdown_connections 2024-09-11T23:28:36.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.714+0000 7f7f71afe640 1 -- 172.21.15.43:0/703835232 wait complete. 2024-09-11T23:28:36.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.715+0000 7f7f71afe640 1 Processor -- start 2024-09-11T23:28:36.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.715+0000 7f7f71afe640 1 -- start start 2024-09-11T23:28:36.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.715+0000 7f7f71afe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f6c101de0 0x7f7f6c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:36.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.715+0000 7f7f71afe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f6c102790 0x7f7f6c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:36.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.715+0000 7f7f71afe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c106f10 0x7f7f6c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:36.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.715+0000 7f7f71afe640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c111f90 con 0x7f7f6c106f10 2024-09-11T23:28:36.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.715+0000 7f7f71afe640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c111e10 con 0x7f7f6c101de0 2024-09-11T23:28:36.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.715+0000 7f7f71afe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c112110 con 0x7f7f6c102790 2024-09-11T23:28:36.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.715+0000 7f7f6bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c106f10 0x7f7f6c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:36.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.716+0000 7f7f6affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f6c102790 0x7f7f6c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:36.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.716+0000 7f7f6bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c106f10 0x7f7f6c19eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37446/0 (socket says 172.21.15.43:37446) 2024-09-11T23:28:36.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.716+0000 7f7f6bfff640 1 -- 172.21.15.43:0/3584368084 learned_addr learned my addr 172.21.15.43:0/3584368084 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:36.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.716+0000 7f7f6b7fe640 1 --2- 172.21.15.43:0/3584368084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f6c101de0 0x7f7f6c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:36.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.716+0000 7f7f6bfff640 1 -- 172.21.15.43:0/3584368084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f6c102790 msgr2=0x7f7f6c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:36.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.716+0000 7f7f6bfff640 1 --2- 172.21.15.43:0/3584368084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f6c102790 0x7f7f6c19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.716+0000 7f7f6bfff640 1 -- 172.21.15.43:0/3584368084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f6c101de0 msgr2=0x7f7f6c19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:36.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.716+0000 7f7f6bfff640 1 --2- 172.21.15.43:0/3584368084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f6c101de0 0x7f7f6c19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.716+0000 7f7f6bfff640 1 -- 172.21.15.43:0/3584368084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c19f590 con 0x7f7f6c106f10 2024-09-11T23:28:36.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.716+0000 7f7f6bfff640 1 --2- 172.21.15.43:0/3584368084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c106f10 0x7f7f6c19eeb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400b970 tx=0x7f7f6400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:36.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.716+0000 7f7f68ff9640 1 -- 172.21.15.43:0/3584368084 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64019070 con 0x7f7f6c106f10 2024-09-11T23:28:36.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.716+0000 7f7f68ff9640 1 -- 172.21.15.43:0/3584368084 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f64014440 con 0x7f7f6c106f10 2024-09-11T23:28:36.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.716+0000 7f7f71afe640 1 -- 172.21.15.43:0/3584368084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6c19f880 con 0x7f7f6c106f10 2024-09-11T23:28:36.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.716+0000 7f7f68ff9640 1 -- 172.21.15.43:0/3584368084 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f640075d0 con 0x7f7f6c106f10 2024-09-11T23:28:36.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.716+0000 7f7f71afe640 1 -- 172.21.15.43:0/3584368084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6c1a7160 con 0x7f7f6c106f10 2024-09-11T23:28:36.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.719+0000 7f7f68ff9640 1 -- 172.21.15.43:0/3584368084 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7f6401c070 con 0x7f7f6c106f10 2024-09-11T23:28:36.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.719+0000 7f7f73ed7740 1 Processor -- start 2024-09-11T23:28:36.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.719+0000 7f7f73ed7740 1 -- start start 2024-09-11T23:28:36.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.719+0000 7f7f73ed7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e254b0f8f0 0x55e254b0fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:36.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.719+0000 7f7f73ed7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e254b10210 0x55e254b1a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:36.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.719+0000 7f7f73ed7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b1aad0 0x55e254b1cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:36.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.719+0000 7f7f73ed7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e254a75c10 con 0x55e254b1aad0 2024-09-11T23:28:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.719+0000 7f7f73ed7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e254a1d520 con 0x55e254b10210 2024-09-11T23:28:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.719+0000 7f7f73ed7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e254a71bd0 con 0x55e254b0f8f0 2024-09-11T23:28:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.719+0000 7f7f6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b1aad0 0x55e254b1cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.719+0000 7f7f6b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e254b10210 0x55e254b1a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.719+0000 7f7f6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b1aad0 0x55e254b1cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37450/0 (socket says 172.21.15.43:37450) 2024-09-11T23:28:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.719+0000 7f7f6affd640 1 -- 172.21.15.43:0/924513162 learned_addr learned my addr 172.21.15.43:0/924513162 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.719+0000 7f7f6affd640 1 --2- 172.21.15.43:0/924513162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e254b0f8f0 0x55e254b0fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.719+0000 7f7f68ff9640 1 --2- 172.21.15.43:0/3584368084 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7f540a2cf0 0x7f7f540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.720+0000 7f7f6b7fe640 1 -- 172.21.15.43:0/924513162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e254b0f8f0 msgr2=0x55e254b0fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.720+0000 7f7f6b7fe640 1 --2- 172.21.15.43:0/924513162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e254b0f8f0 0x55e254b0fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.720+0000 7f7f6bfff640 1 --2- 172.21.15.43:0/3584368084 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7f540a2cf0 0x7f7f540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:36.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.720+0000 7f7f6b7fe640 1 -- 172.21.15.43:0/924513162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b1aad0 msgr2=0x55e254b1cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:36.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.720+0000 7f7f6b7fe640 1 --2- 172.21.15.43:0/924513162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b1aad0 0x55e254b1cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.720+0000 7f7f6b7fe640 1 -- 172.21.15.43:0/924513162 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e25484a490 con 0x55e254b10210 2024-09-11T23:28:36.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.720+0000 7f7f68ff9640 1 -- 172.21.15.43:0/3584368084 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f7f640ce8e0 con 0x7f7f6c106f10 2024-09-11T23:28:36.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.720+0000 7f7f6b7fe640 1 --2- 172.21.15.43:0/924513162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e254b10210 0x55e254b1a590 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00abf0 tx=0x7f7f5c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee5f5d93b4774dc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:36.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.720+0000 7f7f6bfff640 1 --2- 172.21.15.43:0/3584368084 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7f540a2cf0 0x7f7f540a51b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7f640cc1f0 tx=0x7f7f64004a40 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:36.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.720+0000 7f7f51ffb640 1 -- 172.21.15.43:0/924513162 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c005220 con 0x55e254b10210 2024-09-11T23:28:36.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.720+0000 7f7f51ffb640 1 -- 172.21.15.43:0/924513162 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f5c004c80 con 0x55e254b10210 2024-09-11T23:28:36.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.720+0000 7f7f51ffb640 1 -- 172.21.15.43:0/924513162 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c018810 con 0x55e254b10210 2024-09-11T23:28:36.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.721+0000 7f7f73ed7740 1 -- 172.21.15.43:0/924513162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e254b10210 msgr2=0x55e254b1a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:36.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.721+0000 7f7f73ed7740 1 --2- 172.21.15.43:0/924513162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e254b10210 0x55e254b1a590 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00abf0 tx=0x7f7f5c00f790 comp rx=0 tx=0).stop 2024-09-11T23:28:36.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.721+0000 7f7f73ed7740 1 -- 172.21.15.43:0/924513162 shutdown_connections 2024-09-11T23:28:36.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.721+0000 7f7f73ed7740 1 --2- 172.21.15.43:0/924513162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b1aad0 0x55e254b1cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.721+0000 7f7f73ed7740 1 --2- 172.21.15.43:0/924513162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e254b10210 0x55e254b1a590 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.721+0000 7f7f73ed7740 1 --2- 172.21.15.43:0/924513162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e254b0f8f0 0x55e254b0fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.721+0000 7f7f73ed7740 1 -- 172.21.15.43:0/924513162 >> 172.21.15.43:0/924513162 conn(0x55e254b0be80 msgr2=0x55e254b0c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:36.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.721+0000 7f7f73ed7740 1 -- 172.21.15.43:0/924513162 shutdown_connections 2024-09-11T23:28:36.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.721+0000 7f7f73ed7740 1 -- 172.21.15.43:0/924513162 wait complete. 2024-09-11T23:28:36.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.721+0000 7f7f73ed7740 1 -- start start 2024-09-11T23:28:36.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.722+0000 7f7f73ed7740 1 Processor -- start 2024-09-11T23:28:36.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.722+0000 7f7f73ed7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b0f8f0 0x55e254b17910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:36.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.722+0000 7f7f73ed7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e254b10210 0x55e254b17e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:36.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.722+0000 7f7f73ed7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e254b1aad0 0x55e254b18390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:36.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.722+0000 7f7f73ed7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e254b1d6a0 con 0x55e254b0f8f0 2024-09-11T23:28:36.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.722+0000 7f7f73ed7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e2549fe700 con 0x55e254b10210 2024-09-11T23:28:36.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.722+0000 7f7f73ed7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e254b1d820 con 0x55e254b1aad0 2024-09-11T23:28:36.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.722+0000 7f7f6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b0f8f0 0x55e254b17910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:36.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.722+0000 7f7f73ed7740 10 client.0 _ll_register_callbacks cb 0x55e254a1fae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:36.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.722+0000 7f7f6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b0f8f0 0x55e254b17910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37458/0 (socket says 172.21.15.43:37458) 2024-09-11T23:28:36.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.722+0000 7f7f6affd640 1 -- 172.21.15.43:0/146210224 learned_addr learned my addr 172.21.15.43:0/146210224 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:36.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.722+0000 7f7f6b7fe640 1 --2- 172.21.15.43:0/146210224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e254b10210 0x55e254b17e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:36.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.722+0000 7f7f6affd640 1 --2- 172.21.15.43:0/146210224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e254b1aad0 0x55e254b18390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:36.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.722+0000 7f7f6affd640 1 -- 172.21.15.43:0/146210224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e254b1aad0 msgr2=0x55e254b18390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:36.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.722+0000 7f7f6affd640 1 --2- 172.21.15.43:0/146210224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e254b1aad0 0x55e254b18390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.722+0000 7f7f6affd640 1 -- 172.21.15.43:0/146210224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e254b10210 msgr2=0x55e254b17e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.722+0000 7f7f6affd640 1 --2- 172.21.15.43:0/146210224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e254b10210 0x55e254b17e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.722+0000 7f7f6affd640 1 -- 172.21.15.43:0/146210224 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e254b26b10 con 0x55e254b0f8f0 2024-09-11T23:28:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.722+0000 7f7f6affd640 1 --2- 172.21.15.43:0/146210224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b0f8f0 0x55e254b17910 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7f60000c00 tx=0x7f7f600058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875c90b8fed2d6de server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.722+0000 7f7f4affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:28:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.723+0000 7f7f4affd640 1 -- 172.21.15.43:0/146210224 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60041070 con 0x55e254b0f8f0 2024-09-11T23:28:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.723+0000 7f7f4affd640 1 -- 172.21.15.43:0/146210224 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f7f60005220 con 0x55e254b0f8f0 2024-09-11T23:28:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.723+0000 7f7f73ed7740 1 -- 172.21.15.43:0/146210224 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f5c004c80 con 0x55e254b0f8f0 2024-09-11T23:28:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.723+0000 7f7f4affd640 1 -- 172.21.15.43:0/146210224 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7f60051050 con 0x55e254b0f8f0 2024-09-11T23:28:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.723+0000 7f7f73ed7740 10 client.16437 fetch_fsmap learned FSMap version 30 2024-09-11T23:28:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.723+0000 7f7f73ed7740 1 -- 172.21.15.43:0/146210224 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e254b29800 con 0x55e254b0f8f0 2024-09-11T23:28:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.723+0000 7f7f4affd640 1 -- 172.21.15.43:0/146210224 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f5c004c80 con 0x55e254b0f8f0 2024-09-11T23:28:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.723+0000 7f7f4affd640 1 -- 172.21.15.43:0/146210224 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7f600048c0 con 0x55e254b0f8f0 2024-09-11T23:28:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.723+0000 7f7f73ed7740 10 client.16437 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:28:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.723+0000 7f7f73ed7740 10 client.16437 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:28:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.723+0000 7f7f73ed7740 20 client.16437 populate_metadata read hostname 'smithi043' 2024-09-11T23:28:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.723+0000 7f7f73ed7740 1 --2- 172.21.15.43:0/146210224 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55e254b2acc0 0x55e254b2d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.723+0000 7f7f73ed7740 4 client.16437 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:28:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.723+0000 7f7f73ed7740 1 -- 172.21.15.43:0/146210224 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e254a83340 con 0x55e254b2acc0 2024-09-11T23:28:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.724+0000 7f7f6b7fe640 1 --2- 172.21.15.43:0/146210224 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55e254b2acc0 0x55e254b2d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.724+0000 7f7f6b7fe640 1 --2- 172.21.15.43:0/146210224 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55e254b2acc0 0x55e254b2d0b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11fd09e8af372acf server_cookie=935a772b632417c4 in_seq=0 out_seq=0 2024-09-11T23:28:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.724+0000 7f7f4affd640 10 client.16437 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.725+0000 7f7f4affd640 1 -- 172.21.15.43:0/146210224 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e254a83340 con 0x55e254b2acc0 2024-09-11T23:28:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.725+0000 7f7f4affd640 10 client.16437 handle_command_reply: tid=0 2024-09-11T23:28:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.725+0000 7f7f73ed7740 1 client.16437 shutdown 2024-09-11T23:28:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.725+0000 7f7f73ed7740 1 -- 172.21.15.43:0/146210224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b0f8f0 msgr2=0x55e254b17910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.725+0000 7f7f73ed7740 1 --2- 172.21.15.43:0/146210224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b0f8f0 0x55e254b17910 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7f60000c00 tx=0x7f7f600058d0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.725+0000 7f7f73ed7740 1 -- 172.21.15.43:0/146210224 shutdown_connections 2024-09-11T23:28:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.725+0000 7f7f73ed7740 1 --2- 172.21.15.43:0/146210224 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55e254b2acc0 0x55e254b2d0b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.725+0000 7f7f73ed7740 1 --2- 172.21.15.43:0/146210224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e254b1aad0 0x55e254b18390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.725+0000 7f7f73ed7740 1 --2- 172.21.15.43:0/146210224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e254b10210 0x55e254b17e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.725+0000 7f7f4affd640 0 client.16437 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.725+0000 7f7f73ed7740 1 --2- 172.21.15.43:0/146210224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b0f8f0 0x55e254b17910 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.725+0000 7f7f73ed7740 1 -- 172.21.15.43:0/146210224 >> 172.21.15.43:0/146210224 conn(0x55e254b0be80 msgr2=0x55e254b11b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.725+0000 7f7f73ed7740 1 -- 172.21.15.43:0/146210224 shutdown_connections 2024-09-11T23:28:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.725+0000 7f7f73ed7740 1 -- 172.21.15.43:0/146210224 wait complete. 2024-09-11T23:28:36.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.725+0000 7f7f73ed7740 20 client.16437 trim_cache size 0 max 16384 2024-09-11T23:28:36.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.741+0000 7f7f73ed7740 1 Processor -- start 2024-09-11T23:28:36.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.741+0000 7f7f73ed7740 1 -- start start 2024-09-11T23:28:36.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.741+0000 7f7f73ed7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e254b0f8f0 0x55e254b12350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:36.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.741+0000 7f7f73ed7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e254b10210 0x55e254b16ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:36.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.741+0000 7f7f73ed7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b1aad0 0x55e254b32960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:36.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.741+0000 7f7f73ed7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1b2340 con 0x55e254b1aad0 2024-09-11T23:28:36.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.741+0000 7f7f73ed7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1afc10 con 0x55e254b0f8f0 2024-09-11T23:28:36.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.741+0000 7f7f73ed7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1ada60 con 0x55e254b10210 2024-09-11T23:28:36.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.741+0000 7f7f6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b1aad0 0x55e254b32960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:36.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.742+0000 7f7f6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b1aad0 0x55e254b32960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37470/0 (socket says 172.21.15.43:37470) 2024-09-11T23:28:36.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.742+0000 7f7f6affd640 1 -- 172.21.15.43:0/3520780049 learned_addr learned my addr 172.21.15.43:0/3520780049 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:36.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.742+0000 7f7f6b7fe640 1 --2- 172.21.15.43:0/3520780049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e254b10210 0x55e254b16ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:36.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.742+0000 7f7f6affd640 1 --2- 172.21.15.43:0/3520780049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e254b0f8f0 0x55e254b12350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:36.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.742+0000 7f7f6affd640 1 -- 172.21.15.43:0/3520780049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e254b10210 msgr2=0x55e254b16ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:36.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.742+0000 7f7f6affd640 1 --2- 172.21.15.43:0/3520780049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e254b10210 0x55e254b16ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.742+0000 7f7f6affd640 1 -- 172.21.15.43:0/3520780049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e254b0f8f0 msgr2=0x55e254b12350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:36.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.742+0000 7f7f6affd640 1 --2- 172.21.15.43:0/3520780049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e254b0f8f0 0x55e254b12350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.742+0000 7f7f6affd640 1 -- 172.21.15.43:0/3520780049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e254b0bcb0 con 0x55e254b1aad0 2024-09-11T23:28:36.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.742+0000 7f7f6b7fe640 1 --2- 172.21.15.43:0/3520780049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e254b10210 0x55e254b16ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:36.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.742+0000 7f7f6affd640 1 --2- 172.21.15.43:0/3520780049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b1aad0 0x55e254b32960 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7f6002a910 tx=0x7f7f600314d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac1fccc807222e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:36.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.742+0000 7f7f517fa640 1 -- 172.21.15.43:0/3520780049 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60041070 con 0x55e254b1aad0 2024-09-11T23:28:36.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.742+0000 7f7f517fa640 1 -- 172.21.15.43:0/3520780049 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f60003ce0 con 0x55e254b1aad0 2024-09-11T23:28:36.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.742+0000 7f7f517fa640 1 -- 172.21.15.43:0/3520780049 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60035070 con 0x55e254b1aad0 2024-09-11T23:28:36.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.743+0000 7f7f73ed7740 1 -- 172.21.15.43:0/3520780049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b1aad0 msgr2=0x55e254b32960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:36.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.743+0000 7f7f73ed7740 1 --2- 172.21.15.43:0/3520780049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b1aad0 0x55e254b32960 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7f6002a910 tx=0x7f7f600314d0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.743+0000 7f7f73ed7740 1 -- 172.21.15.43:0/3520780049 shutdown_connections 2024-09-11T23:28:36.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.743+0000 7f7f73ed7740 1 --2- 172.21.15.43:0/3520780049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b1aad0 0x55e254b32960 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.743+0000 7f7f73ed7740 1 --2- 172.21.15.43:0/3520780049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e254b10210 0x55e254b16ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.743+0000 7f7f73ed7740 1 --2- 172.21.15.43:0/3520780049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e254b0f8f0 0x55e254b12350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.743+0000 7f7f73ed7740 1 -- 172.21.15.43:0/3520780049 >> 172.21.15.43:0/3520780049 conn(0x55e254b0be80 msgr2=0x55e254b19990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:36.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.743+0000 7f7f73ed7740 1 -- 172.21.15.43:0/3520780049 shutdown_connections 2024-09-11T23:28:36.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.743+0000 7f7f73ed7740 1 -- 172.21.15.43:0/3520780049 wait complete. 2024-09-11T23:28:36.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.743+0000 7f7f73ed7740 1 -- start start 2024-09-11T23:28:36.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.744+0000 7f7f73ed7740 1 Processor -- start 2024-09-11T23:28:36.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.744+0000 7f7f73ed7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b0f8f0 0x55e254b23010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:36.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.744+0000 7f7f73ed7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e254b10210 0x55e254b14550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:36.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.744+0000 7f7f73ed7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e254b1aad0 0x55e254b14a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:36.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.744+0000 7f7f73ed7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1ae0d0 con 0x55e254b0f8f0 2024-09-11T23:28:36.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.744+0000 7f7f73ed7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1af540 con 0x55e254b10210 2024-09-11T23:28:36.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.744+0000 7f7f73ed7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1ae7a0 con 0x55e254b1aad0 2024-09-11T23:28:36.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.744+0000 7f7f73ed7740 10 client.0 _ll_register_callbacks cb 0x7f7f600040b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:36.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.744+0000 7f7f6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b0f8f0 0x55e254b23010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:36.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.744+0000 7f7f6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b0f8f0 0x55e254b23010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37486/0 (socket says 172.21.15.43:37486) 2024-09-11T23:28:36.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.744+0000 7f7f6affd640 1 -- 172.21.15.43:0/28459787 learned_addr learned my addr 172.21.15.43:0/28459787 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:36.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.744+0000 7f7f6b7fe640 1 --2- 172.21.15.43:0/28459787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e254b10210 0x55e254b14550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:36.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.744+0000 7f7f6affd640 1 --2- 172.21.15.43:0/28459787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e254b1aad0 0x55e254b14a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:36.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.744+0000 7f7f6affd640 1 -- 172.21.15.43:0/28459787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e254b1aad0 msgr2=0x55e254b14a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:36.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.744+0000 7f7f6affd640 1 --2- 172.21.15.43:0/28459787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e254b1aad0 0x55e254b14a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.744+0000 7f7f6affd640 1 -- 172.21.15.43:0/28459787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e254b10210 msgr2=0x55e254b14550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:36.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.744+0000 7f7f6affd640 1 --2- 172.21.15.43:0/28459787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e254b10210 0x55e254b14550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.744+0000 7f7f6affd640 1 -- 172.21.15.43:0/28459787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e254b15100 con 0x55e254b0f8f0 2024-09-11T23:28:36.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.745+0000 7f7f6affd640 1 --2- 172.21.15.43:0/28459787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b0f8f0 0x55e254b23010 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7f60005260 tx=0x7f7f60005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b675f0b91cdf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:36.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.745+0000 7f7f4affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:28:36.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.745+0000 7f7f4affd640 1 -- 172.21.15.43:0/28459787 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60035070 con 0x55e254b0f8f0 2024-09-11T23:28:36.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.745+0000 7f7f73ed7740 1 -- 172.21.15.43:0/28459787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f60003ce0 con 0x55e254b0f8f0 2024-09-11T23:28:36.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.745+0000 7f7f4affd640 1 -- 172.21.15.43:0/28459787 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f7f600049e0 con 0x55e254b0f8f0 2024-09-11T23:28:36.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.745+0000 7f7f4affd640 1 -- 172.21.15.43:0/28459787 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7f60041040 con 0x55e254b0f8f0 2024-09-11T23:28:36.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.745+0000 7f7f73ed7740 10 client.16443 fetch_fsmap learned FSMap version 30 2024-09-11T23:28:36.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.745+0000 7f7f73ed7740 1 -- 172.21.15.43:0/28459787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e254b957a0 con 0x55e254b0f8f0 2024-09-11T23:28:36.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.745+0000 7f7f4affd640 1 -- 172.21.15.43:0/28459787 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f60003ce0 con 0x55e254b0f8f0 2024-09-11T23:28:36.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.745+0000 7f7f4affd640 1 -- 172.21.15.43:0/28459787 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7f6005f070 con 0x55e254b0f8f0 2024-09-11T23:28:36.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.745+0000 7f7f73ed7740 10 client.16443 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:28:36.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.745+0000 7f7f73ed7740 10 client.16443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:28:36.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.745+0000 7f7f73ed7740 20 client.16443 populate_metadata read hostname 'smithi043' 2024-09-11T23:28:36.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.745+0000 7f7f73ed7740 1 --2- 172.21.15.43:0/28459787 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55e254b95f70 0x55e254b98390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:36.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.745+0000 7f7f73ed7740 4 client.16443 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-733f7a10-1-mds.b.json"}] 2024-09-11T23:28:36.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.745+0000 7f7f73ed7740 1 -- 172.21.15.43:0/28459787 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-733f7a10-1-mds.b.json"}) -- 0x55e254a7d690 con 0x55e254b95f70 2024-09-11T23:28:36.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.746+0000 7f7f6b7fe640 1 --2- 172.21.15.43:0/28459787 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55e254b95f70 0x55e254b98390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:36.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.746+0000 7f7f4affd640 10 client.16443 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:36.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.746+0000 7f7f6b7fe640 1 --2- 172.21.15.43:0/28459787 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55e254b95f70 0x55e254b98390 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e46e8ea19af59ba2 server_cookie=b1b2e68769b0bd22 in_seq=0 out_seq=0 2024-09-11T23:28:36.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.794+0000 7f7f4affd640 1 -- 172.21.15.43:0/28459787 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x55e254a7d690 con 0x55e254b95f70 2024-09-11T23:28:36.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.794+0000 7f7f4affd640 10 client.16443 handle_command_reply: tid=0 2024-09-11T23:28:36.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.794+0000 7f7f73ed7740 1 client.16443 shutdown 2024-09-11T23:28:36.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.794+0000 7f7f73ed7740 1 -- 172.21.15.43:0/28459787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b0f8f0 msgr2=0x55e254b23010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:36.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.794+0000 7f7f73ed7740 1 --2- 172.21.15.43:0/28459787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b0f8f0 0x55e254b23010 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7f60005260 tx=0x7f7f60005160 comp rx=0 tx=0).stop 2024-09-11T23:28:36.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.794+0000 7f7f73ed7740 1 -- 172.21.15.43:0/28459787 shutdown_connections 2024-09-11T23:28:36.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.794+0000 7f7f73ed7740 1 --2- 172.21.15.43:0/28459787 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55e254b95f70 0x55e254b98390 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.794+0000 7f7f73ed7740 1 --2- 172.21.15.43:0/28459787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e254b1aad0 0x55e254b14a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.794+0000 7f7f73ed7740 1 --2- 172.21.15.43:0/28459787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e254b10210 0x55e254b14550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.794+0000 7f7f4affd640 0 client.16443 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:36.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.794+0000 7f7f73ed7740 1 --2- 172.21.15.43:0/28459787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e254b0f8f0 0x55e254b23010 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.794+0000 7f7f73ed7740 1 -- 172.21.15.43:0/28459787 >> 172.21.15.43:0/28459787 conn(0x55e254b0be80 msgr2=0x55e254b19170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:36.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.794+0000 7f7f73ed7740 1 -- 172.21.15.43:0/28459787 shutdown_connections 2024-09-11T23:28:36.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.795+0000 7f7f73ed7740 1 -- 172.21.15.43:0/28459787 wait complete. 2024-09-11T23:28:36.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.795+0000 7f7f73ed7740 20 client.16443 trim_cache size 0 max 16384 2024-09-11T23:28:36.796 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:28:36.796 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/var/run/ceph/ops-733f7a10-1-mds.b.json" 2024-09-11T23:28:36.796 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:28:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.795+0000 7f7f71afe640 1 -- 172.21.15.43:0/3584368084 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7f540a2cf0 msgr2=0x7f7f540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.795+0000 7f7f71afe640 1 --2- 172.21.15.43:0/3584368084 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7f540a2cf0 0x7f7f540a51b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7f640cc1f0 tx=0x7f7f64004a40 comp rx=0 tx=0).stop 2024-09-11T23:28:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.795+0000 7f7f71afe640 1 -- 172.21.15.43:0/3584368084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c106f10 msgr2=0x7f7f6c19eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.795+0000 7f7f71afe640 1 --2- 172.21.15.43:0/3584368084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c106f10 0x7f7f6c19eeb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400b970 tx=0x7f7f6400bd70 comp rx=0 tx=0).stop 2024-09-11T23:28:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.795+0000 7f7f71afe640 1 -- 172.21.15.43:0/3584368084 shutdown_connections 2024-09-11T23:28:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.795+0000 7f7f71afe640 1 --2- 172.21.15.43:0/3584368084 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7f540a2cf0 0x7f7f540a51b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.795+0000 7f7f71afe640 1 --2- 172.21.15.43:0/3584368084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f6c106f10 0x7f7f6c19eeb0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.795+0000 7f7f71afe640 1 --2- 172.21.15.43:0/3584368084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f6c102790 0x7f7f6c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.795+0000 7f7f71afe640 1 --2- 172.21.15.43:0/3584368084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f6c101de0 0x7f7f6c19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.795+0000 7f7f71afe640 1 -- 172.21.15.43:0/3584368084 >> 172.21.15.43:0/3584368084 conn(0x7f7f6c0fd8c0 msgr2=0x7f7f6c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.796+0000 7f7f71afe640 1 -- 172.21.15.43:0/3584368084 shutdown_connections 2024-09-11T23:28:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:36.796+0000 7f7f71afe640 1 -- 172.21.15.43:0/3584368084 wait complete. 2024-09-11T23:28:37.651 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-09-11T23:28:37.700 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.3Rptbt6vK9 2024-09-11T23:28:37.700 DEBUG:teuthology.orchestra.run.smithi097:> sudo cp /var/run/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ops-733f7a10-1-mds.i.json /tmp/tmp.3Rptbt6vK9 2024-09-11T23:28:37.791 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.3Rptbt6vK9 2024-09-11T23:28:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.823+0000 7f6ba2a8d640 1 -- 172.21.15.43:0/3863181858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b9c10c0c0 msgr2=0x7f6b9c10e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.823+0000 7f6ba2a8d640 1 --2- 172.21.15.43:0/3863181858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b9c10c0c0 0x7f6b9c10e580 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6b94009ca0 tx=0x7f6b9402a950 comp rx=0 tx=0).stop 2024-09-11T23:28:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.823+0000 7f6ba2a8d640 1 -- 172.21.15.43:0/3863181858 shutdown_connections 2024-09-11T23:28:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.823+0000 7f6ba2a8d640 1 --2- 172.21.15.43:0/3863181858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b9c10c0c0 0x7f6b9c10e580 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.823+0000 7f6ba2a8d640 1 --2- 172.21.15.43:0/3863181858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b9c1035c0 0x7f6b9c10baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.823+0000 7f6ba2a8d640 1 --2- 172.21.15.43:0/3863181858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b9c102ca0 0x7f6b9c103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.823+0000 7f6ba2a8d640 1 -- 172.21.15.43:0/3863181858 >> 172.21.15.43:0/3863181858 conn(0x7f6b9c0fc690 msgr2=0x7f6b9c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.823+0000 7f6ba2a8d640 1 -- 172.21.15.43:0/3863181858 shutdown_connections 2024-09-11T23:28:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.823+0000 7f6ba2a8d640 1 -- 172.21.15.43:0/3863181858 wait complete. 2024-09-11T23:28:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.824+0000 7f6ba2a8d640 1 Processor -- start 2024-09-11T23:28:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.824+0000 7f6ba2a8d640 1 -- start start 2024-09-11T23:28:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.824+0000 7f6ba2a8d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b9c102ca0 0x7f6b9c1008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.824+0000 7f6ba2a8d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b9c1035c0 0x7f6b9c100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.824+0000 7f6ba2a8d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b9c10c0c0 0x7f6b9c101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.824+0000 7f6ba2a8d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c111ee0 con 0x7f6b9c10c0c0 2024-09-11T23:28:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.824+0000 7f6ba2a8d640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c111d60 con 0x7f6b9c102ca0 2024-09-11T23:28:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.824+0000 7f6ba2a8d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c112060 con 0x7f6b9c1035c0 2024-09-11T23:28:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.825+0000 7f6ba102a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b9c10c0c0 0x7f6b9c101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.825+0000 7f6ba102a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b9c10c0c0 0x7f6b9c101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37508/0 (socket says 172.21.15.43:37508) 2024-09-11T23:28:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.825+0000 7f6ba102a640 1 -- 172.21.15.43:0/1051570801 learned_addr learned my addr 172.21.15.43:0/1051570801 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.825+0000 7f6ba0829640 1 --2- 172.21.15.43:0/1051570801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b9c102ca0 0x7f6b9c1008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.825+0000 7f6b9bfff640 1 --2- 172.21.15.43:0/1051570801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b9c1035c0 0x7f6b9c100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.825+0000 7f6ba102a640 1 -- 172.21.15.43:0/1051570801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b9c1035c0 msgr2=0x7f6b9c100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.825+0000 7f6ba102a640 1 --2- 172.21.15.43:0/1051570801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b9c1035c0 0x7f6b9c100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.825+0000 7f6ba102a640 1 -- 172.21.15.43:0/1051570801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b9c102ca0 msgr2=0x7f6b9c1008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.825+0000 7f6ba102a640 1 --2- 172.21.15.43:0/1051570801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b9c102ca0 0x7f6b9c1008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.825+0000 7f6ba102a640 1 -- 172.21.15.43:0/1051570801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9c1a68d0 con 0x7f6b9c10c0c0 2024-09-11T23:28:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.825+0000 7f6ba0829640 1 --2- 172.21.15.43:0/1051570801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b9c102ca0 0x7f6b9c1008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.825+0000 7f6ba102a640 1 --2- 172.21.15.43:0/1051570801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b9c10c0c0 0x7f6b9c101370 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6b94004af0 tx=0x7f6b9402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.825+0000 7f6b99ffb640 1 -- 172.21.15.43:0/1051570801 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b94034040 con 0x7f6b9c10c0c0 2024-09-11T23:28:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.825+0000 7f6ba2a8d640 1 -- 172.21.15.43:0/1051570801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9c1a6b60 con 0x7f6b9c10c0c0 2024-09-11T23:28:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.825+0000 7f6b99ffb640 1 -- 172.21.15.43:0/1051570801 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b94030920 con 0x7f6b9c10c0c0 2024-09-11T23:28:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.825+0000 7f6b99ffb640 1 -- 172.21.15.43:0/1051570801 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b94004820 con 0x7f6b9c10c0c0 2024-09-11T23:28:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.826+0000 7f6ba2a8d640 1 -- 172.21.15.43:0/1051570801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b9c1a70a0 con 0x7f6b9c10c0c0 2024-09-11T23:28:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.828+0000 7f6b99ffb640 1 -- 172.21.15.43:0/1051570801 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6b94037020 con 0x7f6b9c10c0c0 2024-09-11T23:28:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.828+0000 7f6ba5828740 1 Processor -- start 2024-09-11T23:28:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.828+0000 7f6ba5828740 1 -- start start 2024-09-11T23:28:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.828+0000 7f6ba5828740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cfa50af0 0x55b8cfa50ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.828+0000 7f6ba5828740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8cfa51410 0x55b8cfa5b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.828+0000 7f6ba5828740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa5bcb0 0x55b8cfa5e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.828+0000 7f6ba5828740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b8cf9b52f0 con 0x55b8cfa5bcb0 2024-09-11T23:28:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.828+0000 7f6ba5828740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b8cf95d270 con 0x55b8cfa50af0 2024-09-11T23:28:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.828+0000 7f6ba5828740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b8cf9b12b0 con 0x55b8cfa51410 2024-09-11T23:28:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.829+0000 7f6b9bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa5bcb0 0x55b8cfa5e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.829+0000 7f6b9bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa5bcb0 0x55b8cfa5e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37524/0 (socket says 172.21.15.43:37524) 2024-09-11T23:28:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.829+0000 7f6b9bfff640 1 -- 172.21.15.43:0/2226935933 learned_addr learned my addr 172.21.15.43:0/2226935933 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.829+0000 7f6b9bfff640 1 --2- 172.21.15.43:0/2226935933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cfa50af0 0x55b8cfa50ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.829+0000 7f6b99ffb640 1 --2- 172.21.15.43:0/1051570801 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6b8c0a2cf0 0x7f6b8c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.829+0000 7f6ba0829640 1 --2- 172.21.15.43:0/2226935933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8cfa51410 0x55b8cfa5b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.829+0000 7f6ba102a640 1 --2- 172.21.15.43:0/1051570801 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6b8c0a2cf0 0x7f6b8c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.829+0000 7f6b9bfff640 1 -- 172.21.15.43:0/2226935933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8cfa51410 msgr2=0x55b8cfa5b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.829+0000 7f6b9bfff640 1 --2- 172.21.15.43:0/2226935933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8cfa51410 0x55b8cfa5b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.829+0000 7f6b9bfff640 1 -- 172.21.15.43:0/2226935933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cfa50af0 msgr2=0x55b8cfa50ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.829+0000 7f6b9bfff640 1 --2- 172.21.15.43:0/2226935933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cfa50af0 0x55b8cfa50ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.829+0000 7f6b9bfff640 1 -- 172.21.15.43:0/2226935933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8cf95c340 con 0x55b8cfa5bcb0 2024-09-11T23:28:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.829+0000 7f6b99ffb640 1 -- 172.21.15.43:0/1051570801 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f6b940f00f0 con 0x7f6b9c10c0c0 2024-09-11T23:28:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.829+0000 7f6b9bfff640 1 --2- 172.21.15.43:0/2226935933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa5bcb0 0x55b8cfa5e0a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f6b88011780 tx=0x7f6b88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224c5a3e7c40b80c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.829+0000 7f6b86ffd640 1 -- 172.21.15.43:0/2226935933 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88004050 con 0x55b8cfa5bcb0 2024-09-11T23:28:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.829+0000 7f6ba102a640 1 --2- 172.21.15.43:0/1051570801 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6b8c0a2cf0 0x7f6b8c0a51b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6b940edaa0 tx=0x7f6b9402ff80 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.829+0000 7f6b86ffd640 1 -- 172.21.15.43:0/2226935933 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b8800fca0 con 0x55b8cfa5bcb0 2024-09-11T23:28:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.829+0000 7f6b86ffd640 1 -- 172.21.15.43:0/2226935933 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b880039f0 con 0x55b8cfa5bcb0 2024-09-11T23:28:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.830+0000 7f6ba5828740 1 -- 172.21.15.43:0/2226935933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa5bcb0 msgr2=0x55b8cfa5e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.830+0000 7f6ba5828740 1 --2- 172.21.15.43:0/2226935933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa5bcb0 0x55b8cfa5e0a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f6b88011780 tx=0x7f6b88004730 comp rx=0 tx=0).stop 2024-09-11T23:28:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.830+0000 7f6ba5828740 1 -- 172.21.15.43:0/2226935933 shutdown_connections 2024-09-11T23:28:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.830+0000 7f6ba5828740 1 --2- 172.21.15.43:0/2226935933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa5bcb0 0x55b8cfa5e0a0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.830+0000 7f6ba5828740 1 --2- 172.21.15.43:0/2226935933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8cfa51410 0x55b8cfa5b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.830+0000 7f6ba5828740 1 --2- 172.21.15.43:0/2226935933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cfa50af0 0x55b8cfa50ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.830+0000 7f6ba5828740 1 -- 172.21.15.43:0/2226935933 >> 172.21.15.43:0/2226935933 conn(0x55b8cfa4d060 msgr2=0x55b8cfa4d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.830+0000 7f6ba5828740 1 -- 172.21.15.43:0/2226935933 shutdown_connections 2024-09-11T23:28:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.830+0000 7f6ba5828740 1 -- 172.21.15.43:0/2226935933 wait complete. 2024-09-11T23:28:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.830+0000 7f6ba5828740 1 -- start start 2024-09-11T23:28:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.830+0000 7f6ba5828740 1 Processor -- start 2024-09-11T23:28:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.831+0000 7f6ba5828740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8cfa50af0 0x55b8cfa58bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.831+0000 7f6ba5828740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa51410 0x55b8cfa590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.831+0000 7f6ba5828740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cfa5bcb0 0x55b8cfa59630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.831+0000 7f6ba5828740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b8cfa5e880 con 0x55b8cfa51410 2024-09-11T23:28:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.831+0000 7f6ba5828740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b8cf93e290 con 0x55b8cfa5bcb0 2024-09-11T23:28:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.831+0000 7f6ba5828740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b8cfa5ea00 con 0x55b8cfa50af0 2024-09-11T23:28:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.831+0000 7f6ba5828740 10 client.0 _ll_register_callbacks cb 0x55b8cf96f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.831+0000 7f6ba0829640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa51410 0x55b8cfa590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.831+0000 7f6ba0829640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa51410 0x55b8cfa590f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37536/0 (socket says 172.21.15.43:37536) 2024-09-11T23:28:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.831+0000 7f6b9bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8cfa50af0 0x55b8cfa58bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.831+0000 7f6ba0829640 1 -- 172.21.15.43:0/3449092195 learned_addr learned my addr 172.21.15.43:0/3449092195 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.831+0000 7f6b9bfff640 1 --2- 172.21.15.43:0/3449092195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cfa5bcb0 0x55b8cfa59630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.831+0000 7f6ba0829640 1 -- 172.21.15.43:0/3449092195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8cfa50af0 msgr2=0x55b8cfa58bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.831+0000 7f6ba0829640 1 --2- 172.21.15.43:0/3449092195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8cfa50af0 0x55b8cfa58bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.831+0000 7f6ba0829640 1 -- 172.21.15.43:0/3449092195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cfa5bcb0 msgr2=0x55b8cfa59630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.831+0000 7f6ba0829640 1 --2- 172.21.15.43:0/3449092195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cfa5bcb0 0x55b8cfa59630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.831+0000 7f6ba0829640 1 -- 172.21.15.43:0/3449092195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8cf6c8210 con 0x55b8cfa51410 2024-09-11T23:28:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.831+0000 7f6b9bfff640 1 --2- 172.21.15.43:0/3449092195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cfa5bcb0 0x55b8cfa59630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:28:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.832+0000 7f6ba0829640 1 --2- 172.21.15.43:0/3449092195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa51410 0x55b8cfa590f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6b90004490 tx=0x7f6b900039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde27c1a48fc3f20 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.832+0000 7f6b7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:28:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.832+0000 7f6b7bfff640 1 -- 172.21.15.43:0/3449092195 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b90011770 con 0x55b8cfa51410 2024-09-11T23:28:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.832+0000 7f6b7bfff640 1 -- 172.21.15.43:0/3449092195 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f6b90011910 con 0x55b8cfa51410 2024-09-11T23:28:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.832+0000 7f6ba5828740 1 -- 172.21.15.43:0/3449092195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b8800fca0 con 0x55b8cfa51410 2024-09-11T23:28:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.832+0000 7f6b7bfff640 1 -- 172.21.15.43:0/3449092195 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6b90011b10 con 0x55b8cfa51410 2024-09-11T23:28:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.832+0000 7f6ba5828740 10 client.16455 fetch_fsmap learned FSMap version 30 2024-09-11T23:28:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.832+0000 7f6ba5828740 1 -- 172.21.15.43:0/3449092195 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b8cf794620 con 0x55b8cfa51410 2024-09-11T23:28:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.832+0000 7f6b7bfff640 1 -- 172.21.15.43:0/3449092195 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b8800fca0 con 0x55b8cfa51410 2024-09-11T23:28:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.832+0000 7f6b7bfff640 1 -- 172.21.15.43:0/3449092195 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f6b90023400 con 0x55b8cfa51410 2024-09-11T23:28:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.832+0000 7f6ba5828740 10 client.16455 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:28:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.832+0000 7f6ba5828740 10 client.16455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:28:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.832+0000 7f6ba5828740 20 client.16455 populate_metadata read hostname 'smithi043' 2024-09-11T23:28:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.832+0000 7f6ba5828740 1 --2- 172.21.15.43:0/3449092195 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b8cfa6bd50 0x55b8cfa6e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.832+0000 7f6ba5828740 4 client.16455 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:28:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.832+0000 7f6ba5828740 1 -- 172.21.15.43:0/3449092195 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8cf9c2a20 con 0x55b8cfa6bd50 2024-09-11T23:28:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.833+0000 7f6b9bfff640 1 --2- 172.21.15.43:0/3449092195 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b8cfa6bd50 0x55b8cfa6e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.833+0000 7f6b9bfff640 1 --2- 172.21.15.43:0/3449092195 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b8cfa6bd50 0x55b8cfa6e140 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e03f269b230b115 server_cookie=b788d0df107146e9 in_seq=0 out_seq=0 2024-09-11T23:28:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.833+0000 7f6b7bfff640 10 client.16455 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.834+0000 7f6b7bfff640 1 -- 172.21.15.43:0/3449092195 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b8cf9c2a20 con 0x55b8cfa6bd50 2024-09-11T23:28:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.834+0000 7f6b7bfff640 10 client.16455 handle_command_reply: tid=0 2024-09-11T23:28:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.834+0000 7f6ba5828740 1 client.16455 shutdown 2024-09-11T23:28:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.834+0000 7f6ba5828740 1 -- 172.21.15.43:0/3449092195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa51410 msgr2=0x55b8cfa590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.834+0000 7f6ba5828740 1 --2- 172.21.15.43:0/3449092195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa51410 0x55b8cfa590f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6b90004490 tx=0x7f6b900039c0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.834+0000 7f6ba5828740 1 -- 172.21.15.43:0/3449092195 shutdown_connections 2024-09-11T23:28:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.834+0000 7f6ba5828740 1 --2- 172.21.15.43:0/3449092195 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b8cfa6bd50 0x55b8cfa6e140 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.834+0000 7f6ba5828740 1 --2- 172.21.15.43:0/3449092195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cfa5bcb0 0x55b8cfa59630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.834+0000 7f6b7bfff640 0 client.16455 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.834+0000 7f6ba5828740 1 --2- 172.21.15.43:0/3449092195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa51410 0x55b8cfa590f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.834+0000 7f6ba5828740 1 --2- 172.21.15.43:0/3449092195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8cfa50af0 0x55b8cfa58bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.834+0000 7f6ba5828740 1 -- 172.21.15.43:0/3449092195 >> 172.21.15.43:0/3449092195 conn(0x55b8cfa4d060 msgr2=0x55b8cfa52d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.834+0000 7f6ba5828740 1 -- 172.21.15.43:0/3449092195 shutdown_connections 2024-09-11T23:28:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.834+0000 7f6ba5828740 1 -- 172.21.15.43:0/3449092195 wait complete. 2024-09-11T23:28:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.834+0000 7f6ba5828740 20 client.16455 trim_cache size 0 max 16384 2024-09-11T23:28:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.848+0000 7f6ba5828740 1 Processor -- start 2024-09-11T23:28:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.848+0000 7f6ba5828740 1 -- start start 2024-09-11T23:28:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.849+0000 7f6ba5828740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8cfa50af0 0x55b8cfa52c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.849+0000 7f6ba5828740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa51410 0x55b8cfa59460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.849+0000 7f6ba5828740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cfa5bcb0 0x55b8cfa73a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.849+0000 7f6ba5828740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c1b2310 con 0x55b8cfa51410 2024-09-11T23:28:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.849+0000 7f6ba5828740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c1afbe0 con 0x55b8cfa5bcb0 2024-09-11T23:28:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.849+0000 7f6ba5828740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c1ada30 con 0x55b8cfa50af0 2024-09-11T23:28:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.849+0000 7f6ba0829640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa51410 0x55b8cfa59460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.849+0000 7f6ba0829640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa51410 0x55b8cfa59460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37552/0 (socket says 172.21.15.43:37552) 2024-09-11T23:28:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.849+0000 7f6ba0829640 1 -- 172.21.15.43:0/3661190896 learned_addr learned my addr 172.21.15.43:0/3661190896 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.849+0000 7f6b9bfff640 1 --2- 172.21.15.43:0/3661190896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cfa5bcb0 0x55b8cfa73a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.849+0000 7f6b9bfff640 1 --2- 172.21.15.43:0/3661190896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8cfa50af0 0x55b8cfa52c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.849+0000 7f6ba0829640 1 -- 172.21.15.43:0/3661190896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8cfa50af0 msgr2=0x55b8cfa52c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.849+0000 7f6ba0829640 1 --2- 172.21.15.43:0/3661190896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8cfa50af0 0x55b8cfa52c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.849+0000 7f6ba0829640 1 -- 172.21.15.43:0/3661190896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cfa5bcb0 msgr2=0x55b8cfa73a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.849+0000 7f6ba0829640 1 --2- 172.21.15.43:0/3661190896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cfa5bcb0 0x55b8cfa73a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.849+0000 7f6ba0829640 1 -- 172.21.15.43:0/3661190896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8cfa53a80 con 0x55b8cfa51410 2024-09-11T23:28:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.849+0000 7f6b9bfff640 1 --2- 172.21.15.43:0/3661190896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8cfa50af0 0x55b8cfa52c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:28:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.849+0000 7f6ba0829640 1 --2- 172.21.15.43:0/3661190896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa51410 0x55b8cfa59460 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6b900119f0 tx=0x7f6b90011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eeda7240d3b6606 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.849+0000 7f6b867fc640 1 -- 172.21.15.43:0/3661190896 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b90023400 con 0x55b8cfa51410 2024-09-11T23:28:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.849+0000 7f6b867fc640 1 -- 172.21.15.43:0/3661190896 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b900235a0 con 0x55b8cfa51410 2024-09-11T23:28:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.849+0000 7f6b867fc640 1 -- 172.21.15.43:0/3661190896 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b900048c0 con 0x55b8cfa51410 2024-09-11T23:28:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.850+0000 7f6ba5828740 1 -- 172.21.15.43:0/3661190896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa51410 msgr2=0x55b8cfa59460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.850+0000 7f6ba5828740 1 --2- 172.21.15.43:0/3661190896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa51410 0x55b8cfa59460 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6b900119f0 tx=0x7f6b90011a20 comp rx=0 tx=0).stop 2024-09-11T23:28:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.850+0000 7f6ba5828740 1 -- 172.21.15.43:0/3661190896 shutdown_connections 2024-09-11T23:28:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.850+0000 7f6ba5828740 1 --2- 172.21.15.43:0/3661190896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cfa5bcb0 0x55b8cfa73a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.850+0000 7f6ba5828740 1 --2- 172.21.15.43:0/3661190896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa51410 0x55b8cfa59460 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.850+0000 7f6ba5828740 1 --2- 172.21.15.43:0/3661190896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8cfa50af0 0x55b8cfa52c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.850+0000 7f6ba5828740 1 -- 172.21.15.43:0/3661190896 >> 172.21.15.43:0/3661190896 conn(0x55b8cfa4d060 msgr2=0x55b8cfa52550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.850+0000 7f6ba5828740 1 -- 172.21.15.43:0/3661190896 shutdown_connections 2024-09-11T23:28:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.850+0000 7f6ba5828740 1 -- 172.21.15.43:0/3661190896 wait complete. 2024-09-11T23:28:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.850+0000 7f6ba5828740 1 -- start start 2024-09-11T23:28:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.850+0000 7f6ba5828740 1 Processor -- start 2024-09-11T23:28:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.851+0000 7f6ba5828740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cfa50af0 0x55b8cfa55bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.851+0000 7f6ba5828740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8cfa51410 0x55b8cfa56100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.851+0000 7f6ba5828740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa5bcb0 0x55b8cfa56640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.851+0000 7f6ba5828740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c1ae0a0 con 0x55b8cfa5bcb0 2024-09-11T23:28:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.851+0000 7f6ba5828740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c1af510 con 0x55b8cfa50af0 2024-09-11T23:28:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.851+0000 7f6ba5828740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c1ae770 con 0x55b8cfa51410 2024-09-11T23:28:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.851+0000 7f6ba5828740 10 client.0 _ll_register_callbacks cb 0x7f6b9000bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.851+0000 7f6b9bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa5bcb0 0x55b8cfa56640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.851+0000 7f6b9bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa5bcb0 0x55b8cfa56640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37554/0 (socket says 172.21.15.43:37554) 2024-09-11T23:28:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.851+0000 7f6b9bfff640 1 -- 172.21.15.43:0/3269197084 learned_addr learned my addr 172.21.15.43:0/3269197084 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.851+0000 7f6ba0829640 1 --2- 172.21.15.43:0/3269197084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8cfa51410 0x55b8cfa56100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.851+0000 7f6b9bfff640 1 --2- 172.21.15.43:0/3269197084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cfa50af0 0x55b8cfa55bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.851+0000 7f6b9bfff640 1 -- 172.21.15.43:0/3269197084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8cfa51410 msgr2=0x55b8cfa56100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.851+0000 7f6b9bfff640 1 --2- 172.21.15.43:0/3269197084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8cfa51410 0x55b8cfa56100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.851+0000 7f6b9bfff640 1 -- 172.21.15.43:0/3269197084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cfa50af0 msgr2=0x55b8cfa55bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.851+0000 7f6b9bfff640 1 --2- 172.21.15.43:0/3269197084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cfa50af0 0x55b8cfa55bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.851+0000 7f6b9bfff640 1 -- 172.21.15.43:0/3269197084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8cfad59d0 con 0x55b8cfa5bcb0 2024-09-11T23:28:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.851+0000 7f6b9bfff640 1 --2- 172.21.15.43:0/3269197084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa5bcb0 0x55b8cfa56640 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x55b8cfa53690 tx=0x7f6b88021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2015207115b5d7df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.851+0000 7f6b7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:28:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.852+0000 7f6b7bfff640 1 -- 172.21.15.43:0/3269197084 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88033070 con 0x55b8cfa5bcb0 2024-09-11T23:28:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.852+0000 7f6b7bfff640 1 -- 172.21.15.43:0/3269197084 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f6b8802ee80 con 0x55b8cfa5bcb0 2024-09-11T23:28:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.852+0000 7f6ba5828740 1 -- 172.21.15.43:0/3269197084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b900235a0 con 0x55b8cfa5bcb0 2024-09-11T23:28:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.852+0000 7f6b7bfff640 1 -- 172.21.15.43:0/3269197084 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6b8800fab0 con 0x55b8cfa5bcb0 2024-09-11T23:28:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.852+0000 7f6ba5828740 10 client.16461 fetch_fsmap learned FSMap version 30 2024-09-11T23:28:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.852+0000 7f6ba5828740 1 -- 172.21.15.43:0/3269197084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b8cfad6c30 con 0x55b8cfa5bcb0 2024-09-11T23:28:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.852+0000 7f6b7bfff640 1 -- 172.21.15.43:0/3269197084 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b900235a0 con 0x55b8cfa5bcb0 2024-09-11T23:28:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.852+0000 7f6b7bfff640 1 -- 172.21.15.43:0/3269197084 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f6b8801f440 con 0x55b8cfa5bcb0 2024-09-11T23:28:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.852+0000 7f6ba5828740 10 client.16461 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:28:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.852+0000 7f6ba5828740 10 client.16461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:28:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.852+0000 7f6ba5828740 20 client.16461 populate_metadata read hostname 'smithi043' 2024-09-11T23:28:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.852+0000 7f6ba5828740 1 --2- 172.21.15.43:0/3269197084 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b8cfa6bd50 0x55b8cfad9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.852+0000 7f6ba5828740 4 client.16461 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:28:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.852+0000 7f6ba5828740 1 -- 172.21.15.43:0/3269197084 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b8cf9bcd70 con 0x55b8cfa6bd50 2024-09-11T23:28:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.852+0000 7f6ba0829640 1 --2- 172.21.15.43:0/3269197084 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b8cfa6bd50 0x55b8cfad9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.853+0000 7f6ba0829640 1 --2- 172.21.15.43:0/3269197084 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b8cfa6bd50 0x55b8cfad9160 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fb17107b62288f2 server_cookie=fc448e8080c0c78b in_seq=0 out_seq=0 2024-09-11T23:28:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:37.853+0000 7f6b7bfff640 10 client.16461 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:37.895 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.3Rptbt6vK9 is 217KB 2024-09-11T23:28:37.990 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.3Rptbt6vK9 2024-09-11T23:28:38.005 DEBUG:tasks.quiescer.fs.[cephfs]:read 222352B of ops from '/var/run/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ops-733f7a10-1-mds.i.json' on mds.2 (i) 2024-09-11T23:28:38.078 INFO:tasks.quiescer.fs.[cephfs]:Pulled 87 ops from rank 2 (i) into home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901461/ops-733f7a10-1-mds.i.json 2024-09-11T23:28:38.078 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -- /var/run/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ops-733f7a10-1-mds.i.json 2024-09-11T23:28:38.104 DEBUG:teuthology.orchestra.run.smithi090:> mktemp 2024-09-11T23:28:38.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:38 smithi043 ceph-mon[30950]: pgmap v207: 97 pgs: 97 active+clean; 21 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 17 MiB/s wr, 3.02k op/s 2024-09-11T23:28:38.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:38 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:28:38.495 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/tmp.3tFYwQyHFx 2024-09-11T23:28:38.495 DEBUG:teuthology.orchestra.run.smithi090:> sudo cp /var/run/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ops-733f7a10-1-mds.h.json /tmp/tmp.3tFYwQyHFx 2024-09-11T23:28:38.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:38 smithi090 ceph-mon[35130]: pgmap v207: 97 pgs: 97 active+clean; 21 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 17 MiB/s wr, 3.02k op/s 2024-09-11T23:28:38.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:38 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:28:38.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:38 smithi097 ceph-mon[35562]: pgmap v207: 97 pgs: 97 active+clean; 21 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 17 MiB/s wr, 3.02k op/s 2024-09-11T23:28:38.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:38 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:28:38.917 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0666 /tmp/tmp.3tFYwQyHFx 2024-09-11T23:28:39.130 DEBUG:teuthology.orchestra.remote:smithi090:/tmp/tmp.3tFYwQyHFx is 262KB 2024-09-11T23:28:39.225 DEBUG:teuthology.orchestra.run.smithi090:> rm -fr /tmp/tmp.3tFYwQyHFx 2024-09-11T23:28:39.462 DEBUG:tasks.quiescer.fs.[cephfs]:read 268088B of ops from '/var/run/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ops-733f7a10-1-mds.h.json' on mds.1 (h) 2024-09-11T23:28:39.495 INFO:tasks.quiescer.fs.[cephfs]:Pulled 104 ops from rank 1 (h) into home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901461/ops-733f7a10-1-mds.h.json 2024-09-11T23:28:39.496 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -- /var/run/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ops-733f7a10-1-mds.h.json 2024-09-11T23:28:39.534 DEBUG:teuthology.orchestra.run.smithi090:> mktemp 2024-09-11T23:28:39.597 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/tmp.LlZcP9LtRQ 2024-09-11T23:28:39.597 DEBUG:teuthology.orchestra.run.smithi090:> sudo cp /var/run/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ops-733f7a10-1-mds.b.json /tmp/tmp.LlZcP9LtRQ 2024-09-11T23:28:39.775 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0666 /tmp/tmp.LlZcP9LtRQ 2024-09-11T23:28:39.863 DEBUG:teuthology.orchestra.remote:smithi090:/tmp/tmp.LlZcP9LtRQ is 1MB 2024-09-11T23:28:39.968 DEBUG:teuthology.orchestra.run.smithi090:> rm -fr /tmp/tmp.LlZcP9LtRQ 2024-09-11T23:28:39.991 DEBUG:tasks.quiescer.fs.[cephfs]:read 1114902B of ops from '/var/run/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ops-733f7a10-1-mds.b.json' on mds.0 (b) 2024-09-11T23:28:40.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:936 fp=#0x100000004cf' 2024-09-11T23:28:40.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:1033 fp=#0x100000004f7' 2024-09-11T23:28:40.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:1131 fp=#0x10000000535' 2024-09-11T23:28:40.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:1070 fp=#0x100000004d9' 2024-09-11T23:28:40.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:1074 fp=#0x10000000505' 2024-09-11T23:28:40.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:1077 fp=#0x10000000530' 2024-09-11T23:28:40.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:1141 fp=#0x100000004d4' 2024-09-11T23:28:40.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_path:mds.0:919 fp=#0x1' 2024-09-11T23:28:40.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:920 fp=#0x1' 2024-09-11T23:28:40.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:1112 fp=#0x10000000519' 2024-09-11T23:28:40.060 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:921 fp=#0x10000000000' 2024-09-11T23:28:40.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:1114 fp=#0x100000004e8' 2024-09-11T23:28:40.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:923 fp=#0x10000000001' 2024-09-11T23:28:40.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:1147 fp=#0x10000000508' 2024-09-11T23:28:40.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:1148 fp=#0x10000000539' 2024-09-11T23:28:40.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:925 fp=#0x100000001fa' 2024-09-11T23:28:40.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:1053 fp=#0x10000000516' 2024-09-11T23:28:40.081 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:927 fp=#0x100000001fb' 2024-09-11T23:28:40.083 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:1055 fp=#0x100000004ef' 2024-09-11T23:28:40.083 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:1119 fp=#0x100000004dc' 2024-09-11T23:28:40.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:1151 fp=#0x1000000050a' 2024-09-11T23:28:40.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:1120 fp=#0x100000004d7' 2024-09-11T23:28:40.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:1057 fp=#0x10000000517' 2024-09-11T23:28:40.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:1153 fp=#0x10000000543' 2024-09-11T23:28:40.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:931 fp=#0x10000000005' 2024-09-11T23:28:40.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:932 fp=#0x10000000006' 2024-09-11T23:28:40.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:933 fp=#0x100000004ce' 2024-09-11T23:28:40.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:1093 fp=#0x10000000540' 2024-09-11T23:28:40.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:1030 fp=#0x100000004f5' 2024-09-11T23:28:40.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 733f7a10-1: 'internal op quiesce_inode:mds.0:1095 fp=#0x1000000054b' 2024-09-11T23:28:40.117 INFO:tasks.quiescer.fs.[cephfs]:Pulled 408 ops from rank 0 (b) into home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901461/ops-733f7a10-1-mds.b.json 2024-09-11T23:28:40.118 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -- /var/run/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ops-733f7a10-1-mds.b.json 2024-09-11T23:28:40.145 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:28:40.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:40 smithi043 ceph-mon[30950]: pgmap v208: 97 pgs: 97 active+clean; 21 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 43 KiB/s rd, 21 MiB/s wr, 3.90k op/s 2024-09-11T23:28:40.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:40 smithi043 ceph-mon[30950]: scrub summary: idle 2024-09-11T23:28:40.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:40 smithi043 ceph-mon[30950]: scrub summary: idle 2024-09-11T23:28:40.417 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:28:40.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:40 smithi090 ceph-mon[35130]: pgmap v208: 97 pgs: 97 active+clean; 21 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 43 KiB/s rd, 21 MiB/s wr, 3.90k op/s 2024-09-11T23:28:40.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:40 smithi090 ceph-mon[35130]: scrub summary: idle 2024-09-11T23:28:40.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:40 smithi090 ceph-mon[35130]: scrub summary: idle 2024-09-11T23:28:40.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:40 smithi097 ceph-mon[35562]: pgmap v208: 97 pgs: 97 active+clean; 21 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 43 KiB/s rd, 21 MiB/s wr, 3.90k op/s 2024-09-11T23:28:40.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:40 smithi097 ceph-mon[35562]: scrub summary: idle 2024-09-11T23:28:40.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:40 smithi097 ceph-mon[35562]: scrub summary: idle 2024-09-11T23:28:40.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.921+0000 7f7f84f63640 1 -- 172.21.15.43:0/2827896102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f80102570 msgr2=0x7f7f801029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:40.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.921+0000 7f7f84f63640 1 --2- 172.21.15.43:0/2827896102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f80102570 0x7f7f801029b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009900 tx=0x7f7f6802a480 comp rx=0 tx=0).stop 2024-09-11T23:28:40.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.921+0000 7f7f84f63640 1 -- 172.21.15.43:0/2827896102 shutdown_connections 2024-09-11T23:28:40.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.921+0000 7f7f84f63640 1 --2- 172.21.15.43:0/2827896102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f80102ef0 0x7f7f8010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:40.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.921+0000 7f7f84f63640 1 --2- 172.21.15.43:0/2827896102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f80102570 0x7f7f801029b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:40.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.921+0000 7f7f84f63640 1 --2- 172.21.15.43:0/2827896102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f80100e20 0x7f7f80101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:40.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.921+0000 7f7f84f63640 1 -- 172.21.15.43:0/2827896102 >> 172.21.15.43:0/2827896102 conn(0x7f7f800fc7d0 msgr2=0x7f7f800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:40.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.921+0000 7f7f84f63640 1 -- 172.21.15.43:0/2827896102 shutdown_connections 2024-09-11T23:28:40.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.921+0000 7f7f84f63640 1 -- 172.21.15.43:0/2827896102 wait complete. 2024-09-11T23:28:40.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.922+0000 7f7f84f63640 1 Processor -- start 2024-09-11T23:28:40.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.922+0000 7f7f84f63640 1 -- start start 2024-09-11T23:28:40.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.922+0000 7f7f84f63640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f80100e20 0x7f7f80199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:40.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.922+0000 7f7f84f63640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f80102570 0x7f7f801998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:40.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.922+0000 7f7f84f63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f80102ef0 0x7f7f801a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:40.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.922+0000 7f7f84f63640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80110f80 con 0x7f7f80102ef0 2024-09-11T23:28:40.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.922+0000 7f7f84f63640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80110e00 con 0x7f7f80102570 2024-09-11T23:28:40.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.922+0000 7f7f84f63640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80111100 con 0x7f7f80100e20 2024-09-11T23:28:40.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.923+0000 7f7f7ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f80102ef0 0x7f7f801a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:40.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.923+0000 7f7f7ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f80102ef0 0x7f7f801a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41150/0 (socket says 172.21.15.43:41150) 2024-09-11T23:28:40.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.923+0000 7f7f7e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f80100e20 0x7f7f80199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:40.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.923+0000 7f7f7ed9d640 1 -- 172.21.15.43:0/4239411910 learned_addr learned my addr 172.21.15.43:0/4239411910 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:40.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.923+0000 7f7f7dd9b640 1 --2- 172.21.15.43:0/4239411910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f80102570 0x7f7f801998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:40.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.923+0000 7f7f7ed9d640 1 -- 172.21.15.43:0/4239411910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f80100e20 msgr2=0x7f7f80199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:40.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.923+0000 7f7f7ed9d640 1 --2- 172.21.15.43:0/4239411910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f80100e20 0x7f7f80199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:40.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.923+0000 7f7f7ed9d640 1 -- 172.21.15.43:0/4239411910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f80102570 msgr2=0x7f7f801998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:40.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.923+0000 7f7f7ed9d640 1 --2- 172.21.15.43:0/4239411910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f80102570 0x7f7f801998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:40.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.923+0000 7f7f7ed9d640 1 -- 172.21.15.43:0/4239411910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f801a4930 con 0x7f7f80102ef0 2024-09-11T23:28:40.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.923+0000 7f7f7e59c640 1 --2- 172.21.15.43:0/4239411910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f80100e20 0x7f7f80199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:40.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.923+0000 7f7f7ed9d640 1 --2- 172.21.15.43:0/4239411910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f80102ef0 0x7f7f801a43f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800eb10 tx=0x7f7f7800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:40.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.923+0000 7f7f777fe640 1 -- 172.21.15.43:0/4239411910 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78004b20 con 0x7f7f80102ef0 2024-09-11T23:28:40.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.923+0000 7f7f84f63640 1 -- 172.21.15.43:0/4239411910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f801a4b00 con 0x7f7f80102ef0 2024-09-11T23:28:40.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.923+0000 7f7f84f63640 1 -- 172.21.15.43:0/4239411910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f801a4fc0 con 0x7f7f80102ef0 2024-09-11T23:28:40.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.923+0000 7f7f777fe640 1 -- 172.21.15.43:0/4239411910 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f78004cc0 con 0x7f7f80102ef0 2024-09-11T23:28:40.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.924+0000 7f7f777fe640 1 -- 172.21.15.43:0/4239411910 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78010700 con 0x7f7f80102ef0 2024-09-11T23:28:40.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.925+0000 7f7f84f63640 1 -- 172.21.15.43:0/4239411910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f801021b0 con 0x7f7f80102ef0 2024-09-11T23:28:40.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.927+0000 7f7f777fe640 1 -- 172.21.15.43:0/4239411910 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7f780108a0 con 0x7f7f80102ef0 2024-09-11T23:28:40.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.928+0000 7f7f777fe640 1 --2- 172.21.15.43:0/4239411910 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7f6c0a2cf0 0x7f7f6c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:40.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.928+0000 7f7f7e59c640 1 --2- 172.21.15.43:0/4239411910 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7f6c0a2cf0 0x7f7f6c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:40.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.928+0000 7f7f777fe640 1 -- 172.21.15.43:0/4239411910 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f7f780cfdc0 con 0x7f7f80102ef0 2024-09-11T23:28:40.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.928+0000 7f7f7e59c640 1 --2- 172.21.15.43:0/4239411910 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7f6c0a2cf0 0x7f7f6c0a51b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009fc0 tx=0x7f7f70005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:40.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:40.930+0000 7f7f777fe640 1 -- 172.21.15.43:0/4239411910 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7f7809c9e0 con 0x7f7f80102ef0 2024-09-11T23:28:41.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:41.163+0000 7f7f84f63640 1 -- 172.21.15.43:0/4239411910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f80109090 con 0x7f7f80102ef0 2024-09-11T23:28:41.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:41.165+0000 7f7f777fe640 1 -- 172.21.15.43:0/4239411910 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11893 (secure 0 0 0) 0x7f7f7809c2e0 con 0x7f7f80102ef0 2024-09-11T23:28:41.165 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:28:41.165 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"btime":"2024-09-11T23:25:40:446496+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:25:39.727562+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25571":{"gid":25571,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/568491708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":568491708},{"type":"v1","addr":"172.21.15.90:6835","nonce":568491708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:28:41.165 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-09-11T23:28:41.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:41.168+0000 7f7f84f63640 1 -- 172.21.15.43:0/4239411910 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7f6c0a2cf0 msgr2=0x7f7f6c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:41.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:41.168+0000 7f7f84f63640 1 --2- 172.21.15.43:0/4239411910 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7f6c0a2cf0 0x7f7f6c0a51b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009fc0 tx=0x7f7f70005f90 comp rx=0 tx=0).stop 2024-09-11T23:28:41.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:41.168+0000 7f7f84f63640 1 -- 172.21.15.43:0/4239411910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f80102ef0 msgr2=0x7f7f801a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:41.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:41.168+0000 7f7f84f63640 1 --2- 172.21.15.43:0/4239411910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f80102ef0 0x7f7f801a43f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800eb10 tx=0x7f7f7800c2d0 comp rx=0 tx=0).stop 2024-09-11T23:28:41.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:41.168+0000 7f7f84f63640 1 -- 172.21.15.43:0/4239411910 shutdown_connections 2024-09-11T23:28:41.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:41.168+0000 7f7f84f63640 1 --2- 172.21.15.43:0/4239411910 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7f6c0a2cf0 0x7f7f6c0a51b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:41.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:41.168+0000 7f7f84f63640 1 --2- 172.21.15.43:0/4239411910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f80102ef0 0x7f7f801a43f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:41.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:41.168+0000 7f7f84f63640 1 --2- 172.21.15.43:0/4239411910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f80102570 0x7f7f801998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:41.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:41.168+0000 7f7f84f63640 1 --2- 172.21.15.43:0/4239411910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f80100e20 0x7f7f80199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:41.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:41.168+0000 7f7f84f63640 1 -- 172.21.15.43:0/4239411910 >> 172.21.15.43:0/4239411910 conn(0x7f7f800fc7d0 msgr2=0x7f7f800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:41.168+0000 7f7f84f63640 1 -- 172.21.15.43:0/4239411910 shutdown_connections 2024-09-11T23:28:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:41.168+0000 7f7f84f63640 1 -- 172.21.15.43:0/4239411910 wait complete. 2024-09-11T23:28:41.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:41 smithi043 ceph-mon[30950]: scrub summary: idle 2024-09-11T23:28:41.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:41 smithi090 ceph-mon[35130]: scrub summary: idle 2024-09-11T23:28:41.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:41 smithi097 ceph-mon[35562]: scrub summary: idle 2024-09-11T23:28:41.846 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 733f7a10 --await-for 45.0' 2024-09-11T23:28:41.846 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 733f7a10 --await-for 45.0 2024-09-11T23:28:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.992+0000 7f6b7bfff640 1 -- 172.21.15.43:0/3269197084 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b8cf9bcd70 con 0x55b8cfa6bd50 2024-09-11T23:28:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.992+0000 7f6b7bfff640 10 client.16461 handle_command_reply: tid=0 2024-09-11T23:28:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.992+0000 7f6ba5828740 1 client.16461 shutdown 2024-09-11T23:28:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.992+0000 7f6ba5828740 1 -- 172.21.15.43:0/3269197084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa5bcb0 msgr2=0x55b8cfa56640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.992+0000 7f6ba5828740 1 --2- 172.21.15.43:0/3269197084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa5bcb0 0x55b8cfa56640 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x55b8cfa53690 tx=0x7f6b88021040 comp rx=0 tx=0).stop 2024-09-11T23:28:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.992+0000 7f6ba5828740 1 -- 172.21.15.43:0/3269197084 shutdown_connections 2024-09-11T23:28:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.992+0000 7f6ba5828740 1 --2- 172.21.15.43:0/3269197084 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b8cfa6bd50 0x55b8cfad9160 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.992+0000 7f6ba5828740 1 --2- 172.21.15.43:0/3269197084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b8cfa5bcb0 0x55b8cfa56640 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.992+0000 7f6ba5828740 1 --2- 172.21.15.43:0/3269197084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b8cfa51410 0x55b8cfa56100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.992+0000 7f6ba5828740 1 --2- 172.21.15.43:0/3269197084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cfa50af0 0x55b8cfa55bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.992+0000 7f6ba5828740 1 -- 172.21.15.43:0/3269197084 >> 172.21.15.43:0/3269197084 conn(0x55b8cfa4d060 msgr2=0x55b8cfa52550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.992+0000 7f6b7bfff640 0 client.16461 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.992+0000 7f6ba5828740 1 -- 172.21.15.43:0/3269197084 shutdown_connections 2024-09-11T23:28:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.992+0000 7f6ba5828740 1 -- 172.21.15.43:0/3269197084 wait complete. 2024-09-11T23:28:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.992+0000 7f6ba5828740 20 client.16461 trim_cache size 0 max 16384 2024-09-11T23:28:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:28:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:28:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:28:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:28:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.993+0000 7f6ba2a8d640 1 -- 172.21.15.43:0/1051570801 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6b8c0a2cf0 msgr2=0x7f6b8c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.993+0000 7f6ba2a8d640 1 --2- 172.21.15.43:0/1051570801 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6b8c0a2cf0 0x7f6b8c0a51b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6b940edaa0 tx=0x7f6b9402ff80 comp rx=0 tx=0).stop 2024-09-11T23:28:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.993+0000 7f6ba2a8d640 1 -- 172.21.15.43:0/1051570801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b9c10c0c0 msgr2=0x7f6b9c101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.993+0000 7f6ba2a8d640 1 --2- 172.21.15.43:0/1051570801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b9c10c0c0 0x7f6b9c101370 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6b94004af0 tx=0x7f6b9402f5d0 comp rx=0 tx=0).stop 2024-09-11T23:28:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.993+0000 7f6ba2a8d640 1 -- 172.21.15.43:0/1051570801 shutdown_connections 2024-09-11T23:28:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.993+0000 7f6ba2a8d640 1 --2- 172.21.15.43:0/1051570801 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f6b8c0a2cf0 0x7f6b8c0a51b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.993+0000 7f6ba2a8d640 1 --2- 172.21.15.43:0/1051570801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b9c10c0c0 0x7f6b9c101370 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.993+0000 7f6ba2a8d640 1 --2- 172.21.15.43:0/1051570801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b9c1035c0 0x7f6b9c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.993+0000 7f6ba2a8d640 1 --2- 172.21.15.43:0/1051570801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b9c102ca0 0x7f6b9c1008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.993+0000 7f6ba2a8d640 1 -- 172.21.15.43:0/1051570801 >> 172.21.15.43:0/1051570801 conn(0x7f6b9c0fc690 msgr2=0x7f6b9c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.993+0000 7f6ba2a8d640 1 -- 172.21.15.43:0/1051570801 shutdown_connections 2024-09-11T23:28:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:41.993+0000 7f6ba2a8d640 1 -- 172.21.15.43:0/1051570801 wait complete. 2024-09-11T23:28:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:28:42.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:42 smithi043 ceph-mon[30950]: pgmap v209: 97 pgs: 97 active+clean; 21 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 79 KiB/s rd, 17 MiB/s wr, 3.18k op/s 2024-09-11T23:28:42.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:42 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/4239411910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:28:42.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:42 smithi090 ceph-mon[35130]: pgmap v209: 97 pgs: 97 active+clean; 21 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 79 KiB/s rd, 17 MiB/s wr, 3.18k op/s 2024-09-11T23:28:42.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:42 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/4239411910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:28:42.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:42 smithi097 ceph-mon[35562]: pgmap v209: 97 pgs: 97 active+clean; 21 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 79 KiB/s rd, 17 MiB/s wr, 3.18k op/s 2024-09-11T23:28:42.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:42 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/4239411910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:28:42.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.932+0000 7fa55dc58640 1 -- 172.21.15.43:0/657243967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558102790 msgr2=0x7fa5581067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:42.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.932+0000 7fa55dc58640 1 --2- 172.21.15.43:0/657243967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558102790 0x7fa5581067e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa540009900 tx=0x7fa54002a410 comp rx=0 tx=0).stop 2024-09-11T23:28:42.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.932+0000 7fa55dc58640 1 -- 172.21.15.43:0/657243967 shutdown_connections 2024-09-11T23:28:42.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.932+0000 7fa55dc58640 1 --2- 172.21.15.43:0/657243967 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa558106f10 0x7fa55810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.932+0000 7fa55dc58640 1 --2- 172.21.15.43:0/657243967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558102790 0x7fa5581067e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.932+0000 7fa55dc58640 1 --2- 172.21.15.43:0/657243967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa558101de0 0x7fa5581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.932+0000 7fa55dc58640 1 -- 172.21.15.43:0/657243967 >> 172.21.15.43:0/657243967 conn(0x7fa5580fd8c0 msgr2=0x7fa5580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:42.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.932+0000 7fa55dc58640 1 -- 172.21.15.43:0/657243967 shutdown_connections 2024-09-11T23:28:42.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.932+0000 7fa55dc58640 1 -- 172.21.15.43:0/657243967 wait complete. 2024-09-11T23:28:42.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.932+0000 7fa55dc58640 1 Processor -- start 2024-09-11T23:28:42.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.932+0000 7fa55dc58640 1 -- start start 2024-09-11T23:28:42.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.933+0000 7fa55dc58640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558101de0 0x7fa5581983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:42.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.933+0000 7fa55dc58640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa558102790 0x7fa558198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:42.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.933+0000 7fa55dc58640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa558106f10 0x7fa55819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.933+0000 7fa55dc58640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa558111f70 con 0x7fa558101de0 2024-09-11T23:28:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.933+0000 7fa55dc58640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa558111df0 con 0x7fa558106f10 2024-09-11T23:28:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.933+0000 7fa55dc58640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa5581120f0 con 0x7fa558102790 2024-09-11T23:28:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.933+0000 7fa5577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558101de0 0x7fa5581983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.933+0000 7fa5577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558101de0 0x7fa5581983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41164/0 (socket says 172.21.15.43:41164) 2024-09-11T23:28:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.933+0000 7fa5577fe640 1 -- 172.21.15.43:0/4106299817 learned_addr learned my addr 172.21.15.43:0/4106299817 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.933+0000 7fa54effd640 1 --2- 172.21.15.43:0/4106299817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa558102790 0x7fa558198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.933+0000 7fa557fff640 1 --2- 172.21.15.43:0/4106299817 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa558106f10 0x7fa55819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.933+0000 7fa5577fe640 1 -- 172.21.15.43:0/4106299817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa558102790 msgr2=0x7fa558198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.933+0000 7fa5577fe640 1 --2- 172.21.15.43:0/4106299817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa558102790 0x7fa558198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.933+0000 7fa5577fe640 1 -- 172.21.15.43:0/4106299817 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa558106f10 msgr2=0x7fa55819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.933+0000 7fa5577fe640 1 --2- 172.21.15.43:0/4106299817 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa558106f10 0x7fa55819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.933+0000 7fa5577fe640 1 -- 172.21.15.43:0/4106299817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55819d380 con 0x7fa558101de0 2024-09-11T23:28:42.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.933+0000 7fa557fff640 1 --2- 172.21.15.43:0/4106299817 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa558106f10 0x7fa55819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:28:42.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.933+0000 7fa5577fe640 1 --2- 172.21.15.43:0/4106299817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558101de0 0x7fa5581983d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa54800d770 tx=0x7fa54800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:42.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.934+0000 7fa5557fa640 1 -- 172.21.15.43:0/4106299817 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548017070 con 0x7fa558101de0 2024-09-11T23:28:42.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.934+0000 7fa55dc58640 1 -- 172.21.15.43:0/4106299817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55819d670 con 0x7fa558101de0 2024-09-11T23:28:42.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.934+0000 7fa5557fa640 1 -- 172.21.15.43:0/4106299817 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa548005500 con 0x7fa558101de0 2024-09-11T23:28:42.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.934+0000 7fa5557fa640 1 -- 172.21.15.43:0/4106299817 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548004e00 con 0x7fa558101de0 2024-09-11T23:28:42.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.934+0000 7fa55dc58640 1 -- 172.21.15.43:0/4106299817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5581a4f50 con 0x7fa558101de0 2024-09-11T23:28:42.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.936+0000 7fa5557fa640 1 -- 172.21.15.43:0/4106299817 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa548004fa0 con 0x7fa558101de0 2024-09-11T23:28:42.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.936+0000 7fa560031740 1 Processor -- start 2024-09-11T23:28:42.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.936+0000 7fa560031740 1 -- start start 2024-09-11T23:28:42.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.937+0000 7fa560031740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eab93f2000 0x55eab93f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:42.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.937+0000 7fa560031740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2920 0x55eab93fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:42.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.937+0000 7fa560031740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93fd1e0 0x55eab93ff5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:42.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.937+0000 7fa560031740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55eab9358410 con 0x55eab93f2920 2024-09-11T23:28:42.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.937+0000 7fa560031740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55eab92fee10 con 0x55eab93f2000 2024-09-11T23:28:42.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.937+0000 7fa560031740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55eab93543d0 con 0x55eab93fd1e0 2024-09-11T23:28:42.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.937+0000 7fa5577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2920 0x55eab93fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:42.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.937+0000 7fa557fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eab93f2000 0x55eab93f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:42.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.937+0000 7fa5577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2920 0x55eab93fcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41168/0 (socket says 172.21.15.43:41168) 2024-09-11T23:28:42.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.937+0000 7fa5577fe640 1 -- 172.21.15.43:0/2135605487 learned_addr learned my addr 172.21.15.43:0/2135605487 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:42.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.937+0000 7fa54effd640 1 --2- 172.21.15.43:0/2135605487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93fd1e0 0x55eab93ff5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:42.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.937+0000 7fa557fff640 1 -- 172.21.15.43:0/2135605487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93fd1e0 msgr2=0x55eab93ff5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:42.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.937+0000 7fa557fff640 1 --2- 172.21.15.43:0/2135605487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93fd1e0 0x55eab93ff5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.937+0000 7fa557fff640 1 -- 172.21.15.43:0/2135605487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2920 msgr2=0x55eab93fcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:42.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.937+0000 7fa557fff640 1 --2- 172.21.15.43:0/2135605487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2920 0x55eab93fcca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.937+0000 7fa557fff640 1 -- 172.21.15.43:0/2135605487 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eab90c9870 con 0x55eab93f2000 2024-09-11T23:28:42.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.937+0000 7fa5557fa640 1 --2- 172.21.15.43:0/4106299817 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa5440a2cf0 0x7fa5440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:42.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.937+0000 7fa5577fe640 1 --2- 172.21.15.43:0/4106299817 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa5440a2cf0 0x7fa5440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:42.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.938+0000 7fa557fff640 1 --2- 172.21.15.43:0/2135605487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eab93f2000 0x55eab93f23e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa550008d20 tx=0x7fa55000d710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bd3c1f0c7767bfd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:42.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.938+0000 7fa5557fa640 1 -- 172.21.15.43:0/4106299817 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7fa5480ceed0 con 0x7fa558101de0 2024-09-11T23:28:42.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.938+0000 7fa54dffb640 1 -- 172.21.15.43:0/2135605487 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55000f830 con 0x55eab93f2000 2024-09-11T23:28:42.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.938+0000 7fa54dffb640 1 -- 172.21.15.43:0/2135605487 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa550008e10 con 0x55eab93f2000 2024-09-11T23:28:42.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.938+0000 7fa54dffb640 1 -- 172.21.15.43:0/2135605487 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55000e3b0 con 0x55eab93f2000 2024-09-11T23:28:42.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.939+0000 7fa5577fe640 1 --2- 172.21.15.43:0/4106299817 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa5440a2cf0 0x7fa5440a51b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa548097850 tx=0x7fa5480d2a20 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:42.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.939+0000 7fa560031740 1 -- 172.21.15.43:0/2135605487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eab93f2000 msgr2=0x55eab93f23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:42.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.939+0000 7fa560031740 1 --2- 172.21.15.43:0/2135605487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eab93f2000 0x55eab93f23e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa550008d20 tx=0x7fa55000d710 comp rx=0 tx=0).stop 2024-09-11T23:28:42.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.939+0000 7fa560031740 1 -- 172.21.15.43:0/2135605487 shutdown_connections 2024-09-11T23:28:42.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.939+0000 7fa560031740 1 --2- 172.21.15.43:0/2135605487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93fd1e0 0x55eab93ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.939+0000 7fa560031740 1 --2- 172.21.15.43:0/2135605487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2920 0x55eab93fcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.939+0000 7fa560031740 1 --2- 172.21.15.43:0/2135605487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eab93f2000 0x55eab93f23e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.939+0000 7fa560031740 1 -- 172.21.15.43:0/2135605487 >> 172.21.15.43:0/2135605487 conn(0x55eab93ee590 msgr2=0x55eab93ee970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:42.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.939+0000 7fa560031740 1 -- 172.21.15.43:0/2135605487 shutdown_connections 2024-09-11T23:28:42.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.939+0000 7fa560031740 1 -- 172.21.15.43:0/2135605487 wait complete. 2024-09-11T23:28:42.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.939+0000 7fa560031740 1 -- start start 2024-09-11T23:28:42.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.939+0000 7fa560031740 1 Processor -- start 2024-09-11T23:28:42.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.939+0000 7fa560031740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93f2000 0x55eab93fa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:42.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.939+0000 7fa560031740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2920 0x55eab93fa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:42.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.939+0000 7fa560031740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eab93fd1e0 0x55eab93fab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:42.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.940+0000 7fa560031740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55eab93ffdb0 con 0x55eab93f2920 2024-09-11T23:28:42.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.940+0000 7fa560031740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55eab92e0c50 con 0x55eab93fd1e0 2024-09-11T23:28:42.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.940+0000 7fa560031740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55eab93fff30 con 0x55eab93f2000 2024-09-11T23:28:42.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.940+0000 7fa560031740 10 client.0 _ll_register_callbacks cb 0x55eab9066d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:42.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.940+0000 7fa54effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2920 0x55eab93fa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:42.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.940+0000 7fa557fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93f2000 0x55eab93fa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:42.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.940+0000 7fa557fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93f2000 0x55eab93fa0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:48908/0 (socket says 172.21.15.43:48908) 2024-09-11T23:28:42.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.940+0000 7fa54effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2920 0x55eab93fa600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41182/0 (socket says 172.21.15.43:41182) 2024-09-11T23:28:42.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.940+0000 7fa557fff640 1 -- 172.21.15.43:0/3545110370 learned_addr learned my addr 172.21.15.43:0/3545110370 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:42.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.940+0000 7fa557fff640 1 --2- 172.21.15.43:0/3545110370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eab93fd1e0 0x55eab93fab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:42.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.940+0000 7fa54effd640 1 -- 172.21.15.43:0/3545110370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93f2000 msgr2=0x55eab93fa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:42.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.940+0000 7fa54effd640 1 --2- 172.21.15.43:0/3545110370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93f2000 0x55eab93fa0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.940+0000 7fa54effd640 1 -- 172.21.15.43:0/3545110370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eab93fd1e0 msgr2=0x55eab93fab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:42.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.940+0000 7fa54effd640 1 --2- 172.21.15.43:0/3545110370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eab93fd1e0 0x55eab93fab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.940+0000 7fa54effd640 1 -- 172.21.15.43:0/3545110370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eab92e13c0 con 0x55eab93f2920 2024-09-11T23:28:42.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.940+0000 7fa557fff640 1 --2- 172.21.15.43:0/3545110370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93f2000 0x55eab93fa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:42.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.940+0000 7fa54effd640 1 --2- 172.21.15.43:0/3545110370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2920 0x55eab93fa600 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa540005940 tx=0x7fa540031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33068d0e1a13a7af server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.940+0000 7fa536ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:28:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.940+0000 7fa536ffd640 1 -- 172.21.15.43:0/3545110370 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540035040 con 0x55eab93f2920 2024-09-11T23:28:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.941+0000 7fa536ffd640 1 -- 172.21.15.43:0/3545110370 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7fa540004240 con 0x55eab93f2920 2024-09-11T23:28:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.941+0000 7fa560031740 1 -- 172.21.15.43:0/3545110370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa550008e10 con 0x55eab93f2920 2024-09-11T23:28:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.941+0000 7fa536ffd640 1 -- 172.21.15.43:0/3545110370 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa540004440 con 0x55eab93f2920 2024-09-11T23:28:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.941+0000 7fa560031740 10 client.16479 fetch_fsmap learned FSMap version 30 2024-09-11T23:28:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.941+0000 7fa560031740 1 -- 172.21.15.43:0/3545110370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eab940ba00 con 0x55eab93f2920 2024-09-11T23:28:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.941+0000 7fa536ffd640 1 -- 172.21.15.43:0/3545110370 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa550008e10 con 0x55eab93f2920 2024-09-11T23:28:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.941+0000 7fa536ffd640 1 -- 172.21.15.43:0/3545110370 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fa54004f7b0 con 0x55eab93f2920 2024-09-11T23:28:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.941+0000 7fa560031740 10 client.16479 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:28:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.941+0000 7fa560031740 10 client.16479 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:28:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.941+0000 7fa560031740 20 client.16479 populate_metadata read hostname 'smithi043' 2024-09-11T23:28:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.941+0000 7fa560031740 1 --2- 172.21.15.43:0/3545110370 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55eab940ce10 0x55eab940f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:42.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.941+0000 7fa560031740 4 client.16479 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:28:42.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.941+0000 7fa560031740 1 -- 172.21.15.43:0/3545110370 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eab9365980 con 0x55eab940ce10 2024-09-11T23:28:42.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.941+0000 7fa557fff640 1 --2- 172.21.15.43:0/3545110370 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55eab940ce10 0x55eab940f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:42.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.942+0000 7fa557fff640 1 --2- 172.21.15.43:0/3545110370 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55eab940ce10 0x55eab940f200 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a9f7cce6ef86e36 server_cookie=c646412143a61268 in_seq=0 out_seq=0 2024-09-11T23:28:42.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.942+0000 7fa536ffd640 10 client.16479 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:42.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.942+0000 7fa536ffd640 1 -- 172.21.15.43:0/3545110370 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eab9365980 con 0x55eab940ce10 2024-09-11T23:28:42.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.942+0000 7fa536ffd640 10 client.16479 handle_command_reply: tid=0 2024-09-11T23:28:42.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.942+0000 7fa560031740 1 client.16479 shutdown 2024-09-11T23:28:42.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.943+0000 7fa560031740 1 -- 172.21.15.43:0/3545110370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2920 msgr2=0x55eab93fa600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:42.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.943+0000 7fa560031740 1 --2- 172.21.15.43:0/3545110370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2920 0x55eab93fa600 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa540005940 tx=0x7fa540031920 comp rx=0 tx=0).stop 2024-09-11T23:28:42.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.943+0000 7fa560031740 1 -- 172.21.15.43:0/3545110370 shutdown_connections 2024-09-11T23:28:42.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.943+0000 7fa560031740 1 --2- 172.21.15.43:0/3545110370 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55eab940ce10 0x55eab940f200 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.943+0000 7fa560031740 1 --2- 172.21.15.43:0/3545110370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eab93fd1e0 0x55eab93fab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.943+0000 7fa560031740 1 --2- 172.21.15.43:0/3545110370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2920 0x55eab93fa600 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.943+0000 7fa560031740 1 --2- 172.21.15.43:0/3545110370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93f2000 0x55eab93fa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.943+0000 7fa560031740 1 -- 172.21.15.43:0/3545110370 >> 172.21.15.43:0/3545110370 conn(0x55eab93ee590 msgr2=0x55eab93ed6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:42.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.943+0000 7fa536ffd640 0 client.16479 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:42.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.943+0000 7fa560031740 1 -- 172.21.15.43:0/3545110370 shutdown_connections 2024-09-11T23:28:42.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.943+0000 7fa560031740 1 -- 172.21.15.43:0/3545110370 wait complete. 2024-09-11T23:28:42.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.943+0000 7fa560031740 20 client.16479 trim_cache size 0 max 16384 2024-09-11T23:28:42.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.960+0000 7fa560031740 1 Processor -- start 2024-09-11T23:28:42.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.960+0000 7fa560031740 1 -- start start 2024-09-11T23:28:42.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.960+0000 7fa560031740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2000 0x55eab93fa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:42.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.960+0000 7fa560031740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93f2920 0x55eab9414ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:42.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.960+0000 7fa560031740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eab93fd1e0 0x55eab9415010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:42.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.960+0000 7fa560031740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa5581b00b0 con 0x55eab93f2000 2024-09-11T23:28:42.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.960+0000 7fa560031740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa5581ad980 con 0x55eab93fd1e0 2024-09-11T23:28:42.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.960+0000 7fa560031740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa5581ab7d0 con 0x55eab93f2920 2024-09-11T23:28:42.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.960+0000 7fa557fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2000 0x55eab93fa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:42.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.960+0000 7fa557fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2000 0x55eab93fa990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41184/0 (socket says 172.21.15.43:41184) 2024-09-11T23:28:42.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.960+0000 7fa557fff640 1 -- 172.21.15.43:0/3880871308 learned_addr learned my addr 172.21.15.43:0/3880871308 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:42.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.960+0000 7fa54effd640 1 --2- 172.21.15.43:0/3880871308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93f2920 0x55eab9414ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:42.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.960+0000 7fa557fff640 1 -- 172.21.15.43:0/3880871308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93f2920 msgr2=0x55eab9414ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:42.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.960+0000 7fa557fff640 1 --2- 172.21.15.43:0/3880871308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93f2920 0x55eab9414ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.960+0000 7fa557fff640 1 -- 172.21.15.43:0/3880871308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eab93fd1e0 msgr2=0x55eab9415010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:42.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.960+0000 7fa557fff640 1 --2- 172.21.15.43:0/3880871308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eab93fd1e0 0x55eab9415010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.960+0000 7fa557fff640 1 -- 172.21.15.43:0/3880871308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eab93ee030 con 0x55eab93f2000 2024-09-11T23:28:42.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.961+0000 7fa54effd640 1 --2- 172.21.15.43:0/3880871308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93f2920 0x55eab9414ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:28:42.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.961+0000 7fa557fff640 1 --2- 172.21.15.43:0/3880871308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2000 0x55eab93fa990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa550008d20 tx=0x7fa550018920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11daadd8102c1c67 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:42.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.961+0000 7fa54d7fa640 1 -- 172.21.15.43:0/3880871308 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550021040 con 0x55eab93f2000 2024-09-11T23:28:42.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.961+0000 7fa54d7fa640 1 -- 172.21.15.43:0/3880871308 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa550008e10 con 0x55eab93f2000 2024-09-11T23:28:42.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.961+0000 7fa54d7fa640 1 -- 172.21.15.43:0/3880871308 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5500052a0 con 0x55eab93f2000 2024-09-11T23:28:42.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.961+0000 7fa560031740 1 -- 172.21.15.43:0/3880871308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2000 msgr2=0x55eab93fa990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:42.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.961+0000 7fa560031740 1 --2- 172.21.15.43:0/3880871308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2000 0x55eab93fa990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa550008d20 tx=0x7fa550018920 comp rx=0 tx=0).stop 2024-09-11T23:28:42.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.961+0000 7fa560031740 1 -- 172.21.15.43:0/3880871308 shutdown_connections 2024-09-11T23:28:42.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.961+0000 7fa560031740 1 --2- 172.21.15.43:0/3880871308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eab93fd1e0 0x55eab9415010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.961+0000 7fa560031740 1 --2- 172.21.15.43:0/3880871308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93f2920 0x55eab9414ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.962+0000 7fa560031740 1 --2- 172.21.15.43:0/3880871308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2000 0x55eab93fa990 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.962+0000 7fa560031740 1 -- 172.21.15.43:0/3880871308 >> 172.21.15.43:0/3880871308 conn(0x55eab93ee590 msgr2=0x55eab93f9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:42.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.962+0000 7fa560031740 1 -- 172.21.15.43:0/3880871308 shutdown_connections 2024-09-11T23:28:42.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.962+0000 7fa560031740 1 -- 172.21.15.43:0/3880871308 wait complete. 2024-09-11T23:28:42.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.962+0000 7fa560031740 1 -- start start 2024-09-11T23:28:42.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.962+0000 7fa560031740 1 Processor -- start 2024-09-11T23:28:42.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.962+0000 7fa560031740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93f2000 0x55eab94055a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:42.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.962+0000 7fa560031740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2920 0x55eab94008d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:42.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.962+0000 7fa560031740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eab93fd1e0 0x55eab9400e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.962+0000 7fa560031740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa5581abe40 con 0x55eab93f2920 2024-09-11T23:28:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.962+0000 7fa560031740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa5581ad2b0 con 0x55eab93fd1e0 2024-09-11T23:28:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.962+0000 7fa560031740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa5581ac510 con 0x55eab93f2000 2024-09-11T23:28:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.962+0000 7fa560031740 10 client.0 _ll_register_callbacks cb 0x7fa5400315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.963+0000 7fa54effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2920 0x55eab94008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.963+0000 7fa54effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2920 0x55eab94008d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41188/0 (socket says 172.21.15.43:41188) 2024-09-11T23:28:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.963+0000 7fa54effd640 1 -- 172.21.15.43:0/3746758102 learned_addr learned my addr 172.21.15.43:0/3746758102 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.963+0000 7fa557fff640 1 --2- 172.21.15.43:0/3746758102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93f2000 0x55eab94055a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.963+0000 7fa557fff640 1 --2- 172.21.15.43:0/3746758102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eab93fd1e0 0x55eab9400e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.963+0000 7fa54effd640 1 -- 172.21.15.43:0/3746758102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93f2000 msgr2=0x55eab94055a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.963+0000 7fa54effd640 1 --2- 172.21.15.43:0/3746758102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93f2000 0x55eab94055a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.963+0000 7fa54effd640 1 -- 172.21.15.43:0/3746758102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eab93fd1e0 msgr2=0x55eab9400e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.963+0000 7fa54effd640 1 --2- 172.21.15.43:0/3746758102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eab93fd1e0 0x55eab9400e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.963+0000 7fa54effd640 1 -- 172.21.15.43:0/3746758102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eab9401350 con 0x55eab93f2920 2024-09-11T23:28:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.963+0000 7fa54effd640 1 --2- 172.21.15.43:0/3746758102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2920 0x55eab94008d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa54002a930 tx=0x7fa540009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec324115fe1289b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.963+0000 7fa536ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:28:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.963+0000 7fa536ffd640 1 -- 172.21.15.43:0/3746758102 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540035050 con 0x55eab93f2920 2024-09-11T23:28:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.963+0000 7fa536ffd640 1 -- 172.21.15.43:0/3746758102 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7fa54004f890 con 0x55eab93f2920 2024-09-11T23:28:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.963+0000 7fa560031740 1 -- 172.21.15.43:0/3746758102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa550008e10 con 0x55eab93f2920 2024-09-11T23:28:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.964+0000 7fa536ffd640 1 -- 172.21.15.43:0/3746758102 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa540005040 con 0x55eab93f2920 2024-09-11T23:28:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.964+0000 7fa560031740 10 client.16485 fetch_fsmap learned FSMap version 30 2024-09-11T23:28:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.964+0000 7fa560031740 1 -- 172.21.15.43:0/3746758102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eab9477710 con 0x55eab93f2920 2024-09-11T23:28:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.964+0000 7fa536ffd640 1 -- 172.21.15.43:0/3746758102 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa550008e10 con 0x55eab93f2920 2024-09-11T23:28:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.964+0000 7fa536ffd640 1 -- 172.21.15.43:0/3746758102 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fa540004aa0 con 0x55eab93f2920 2024-09-11T23:28:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.964+0000 7fa560031740 10 client.16485 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:28:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.964+0000 7fa560031740 10 client.16485 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:28:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.964+0000 7fa560031740 20 client.16485 populate_metadata read hostname 'smithi043' 2024-09-11T23:28:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.964+0000 7fa560031740 1 --2- 172.21.15.43:0/3746758102 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55eab9477e80 0x55eab947a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.964+0000 7fa560031740 4 client.16485 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "733f7a10", "await_for": 45.0}] 2024-09-11T23:28:42.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.964+0000 7fa560031740 1 -- 172.21.15.43:0/3746758102 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "quiesce db", "set_id": "733f7a10", "await_for": 45.0}) -- 0x55eab935fe90 con 0x55eab9477e80 2024-09-11T23:28:42.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.964+0000 7fa557fff640 1 --2- 172.21.15.43:0/3746758102 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55eab9477e80 0x55eab947a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:42.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.965+0000 7fa557fff640 1 --2- 172.21.15.43:0/3746758102 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55eab9477e80 0x55eab947a2a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9903a1f316ee91bd server_cookie=2d4df641fd8da6d9 in_seq=0 out_seq=0 2024-09-11T23:28:42.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:42.965+0000 7fa536ffd640 10 client.16485 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:44.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:44 smithi043 ceph-mon[30950]: pgmap v210: 97 pgs: 97 active+clean; 21 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 59 KiB/s rd, 14 MiB/s wr, 2.48k op/s 2024-09-11T23:28:44.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:44 smithi090 ceph-mon[35130]: pgmap v210: 97 pgs: 97 active+clean; 21 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 59 KiB/s rd, 14 MiB/s wr, 2.48k op/s 2024-09-11T23:28:44.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:44 smithi097 ceph-mon[35562]: pgmap v210: 97 pgs: 97 active+clean; 21 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 59 KiB/s rd, 14 MiB/s wr, 2.48k op/s 2024-09-11T23:28:46.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:46 smithi043 ceph-mon[30950]: pgmap v211: 97 pgs: 97 active+clean; 22 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 74 KiB/s rd, 15 MiB/s wr, 2.60k op/s 2024-09-11T23:28:46.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:46 smithi090 ceph-mon[35130]: pgmap v211: 97 pgs: 97 active+clean; 22 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 74 KiB/s rd, 15 MiB/s wr, 2.60k op/s 2024-09-11T23:28:46.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:46 smithi097 ceph-mon[35562]: pgmap v211: 97 pgs: 97 active+clean; 22 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 74 KiB/s rd, 15 MiB/s wr, 2.60k op/s 2024-09-11T23:28:48.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:48 smithi090 ceph-mon[35130]: pgmap v212: 97 pgs: 97 active+clean; 22 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 60 KiB/s rd, 15 MiB/s wr, 2.62k op/s 2024-09-11T23:28:48.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:48 smithi097 ceph-mon[35562]: pgmap v212: 97 pgs: 97 active+clean; 22 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 60 KiB/s rd, 15 MiB/s wr, 2.62k op/s 2024-09-11T23:28:48.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:48 smithi043 ceph-mon[30950]: pgmap v212: 97 pgs: 97 active+clean; 22 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 60 KiB/s rd, 15 MiB/s wr, 2.62k op/s 2024-09-11T23:28:49.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.739+0000 7fa536ffd640 1 -- 172.21.15.43:0/3746758102 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55eab935fe90 con 0x55eab9477e80 2024-09-11T23:28:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.739+0000 7fa536ffd640 10 client.16485 handle_command_reply: tid=0 2024-09-11T23:28:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.739+0000 7fa560031740 1 client.16485 shutdown 2024-09-11T23:28:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.739+0000 7fa560031740 1 -- 172.21.15.43:0/3746758102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2920 msgr2=0x55eab94008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.739+0000 7fa560031740 1 --2- 172.21.15.43:0/3746758102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2920 0x55eab94008d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa54002a930 tx=0x7fa540009830 comp rx=0 tx=0).stop 2024-09-11T23:28:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.739+0000 7fa560031740 1 -- 172.21.15.43:0/3746758102 shutdown_connections 2024-09-11T23:28:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.739+0000 7fa560031740 1 --2- 172.21.15.43:0/3746758102 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55eab9477e80 0x55eab947a2a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.739+0000 7fa560031740 1 --2- 172.21.15.43:0/3746758102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eab93fd1e0 0x55eab9400e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:49.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.739+0000 7fa560031740 1 --2- 172.21.15.43:0/3746758102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eab93f2920 0x55eab94008d0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.739+0000 7fa560031740 1 --2- 172.21.15.43:0/3746758102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eab93f2000 0x55eab94055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.740+0000 7fa536ffd640 0 client.16485 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.740+0000 7fa560031740 1 -- 172.21.15.43:0/3746758102 >> 172.21.15.43:0/3746758102 conn(0x55eab93ee590 msgr2=0x55eab93eced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.740+0000 7fa560031740 1 -- 172.21.15.43:0/3746758102 shutdown_connections 2024-09-11T23:28:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.740+0000 7fa560031740 1 -- 172.21.15.43:0/3746758102 wait complete. 2024-09-11T23:28:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.740+0000 7fa560031740 20 client.16485 trim_cache size 0 max 16384 2024-09-11T23:28:49.741 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:28:49.741 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 30, 2024-09-11T23:28:49.742 INFO:teuthology.orchestra.run.smithi043.stdout: "leader": 24497, 2024-09-11T23:28:49.742 INFO:teuthology.orchestra.run.smithi043.stdout: "set_version": 14, 2024-09-11T23:28:49.742 INFO:teuthology.orchestra.run.smithi043.stdout: "sets": { 2024-09-11T23:28:49.742 INFO:teuthology.orchestra.run.smithi043.stdout: "733f7a10": { 2024-09-11T23:28:49.742 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 14, 2024-09-11T23:28:49.742 INFO:teuthology.orchestra.run.smithi043.stdout: "age_ref": 0.0, 2024-09-11T23:28:49.742 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:28:49.742 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "QUIESCED", 2024-09-11T23:28:49.742 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-09-11T23:28:49.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:28:49.742 INFO:teuthology.orchestra.run.smithi043.stdout: "timeout": 90.0, 2024-09-11T23:28:49.742 INFO:teuthology.orchestra.run.smithi043.stdout: "expiration": 140.2, 2024-09-11T23:28:49.742 INFO:teuthology.orchestra.run.smithi043.stdout: "members": { 2024-09-11T23:28:49.743 INFO:teuthology.orchestra.run.smithi043.stdout: "file:/": { 2024-09-11T23:28:49.743 INFO:teuthology.orchestra.run.smithi043.stdout: "excluded": false, 2024-09-11T23:28:49.743 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:28:49.743 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "QUIESCED", 2024-09-11T23:28:49.743 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-09-11T23:28:49.743 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:28:49.743 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:28:49.743 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:28:49.743 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:28:49.743 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:28:49.743 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:28:49.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.741+0000 7fa55dc58640 1 -- 172.21.15.43:0/4106299817 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa5440a2cf0 msgr2=0x7fa5440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:49.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.741+0000 7fa55dc58640 1 --2- 172.21.15.43:0/4106299817 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa5440a2cf0 0x7fa5440a51b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa548097850 tx=0x7fa5480d2a20 comp rx=0 tx=0).stop 2024-09-11T23:28:49.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.741+0000 7fa55dc58640 1 -- 172.21.15.43:0/4106299817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558101de0 msgr2=0x7fa5581983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:49.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.741+0000 7fa55dc58640 1 --2- 172.21.15.43:0/4106299817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558101de0 0x7fa5581983d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa54800d770 tx=0x7fa54800dc30 comp rx=0 tx=0).stop 2024-09-11T23:28:49.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.741+0000 7fa55dc58640 1 -- 172.21.15.43:0/4106299817 shutdown_connections 2024-09-11T23:28:49.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.741+0000 7fa55dc58640 1 --2- 172.21.15.43:0/4106299817 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa5440a2cf0 0x7fa5440a51b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:49.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.741+0000 7fa55dc58640 1 --2- 172.21.15.43:0/4106299817 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa558106f10 0x7fa55819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:49.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.741+0000 7fa55dc58640 1 --2- 172.21.15.43:0/4106299817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa558102790 0x7fa558198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:49.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.741+0000 7fa55dc58640 1 --2- 172.21.15.43:0/4106299817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558101de0 0x7fa5581983d0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:49.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.741+0000 7fa55dc58640 1 -- 172.21.15.43:0/4106299817 >> 172.21.15.43:0/4106299817 conn(0x7fa5580fd8c0 msgr2=0x7fa5580feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:49.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.741+0000 7fa55dc58640 1 -- 172.21.15.43:0/4106299817 shutdown_connections 2024-09-11T23:28:49.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:49.741+0000 7fa55dc58640 1 -- 172.21.15.43:0/4106299817 wait complete. 2024-09-11T23:28:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:50 smithi043 ceph-mon[30950]: pgmap v213: 97 pgs: 97 active+clean; 22 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 94 KiB/s rd, 17 MiB/s wr, 3.01k op/s 2024-09-11T23:28:50.427 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '733f7a10', quiesce took 78.8 seconds. Will release after: -58.65 2024-09-11T23:28:50.427 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:28:50.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:50 smithi090 ceph-mon[35130]: pgmap v213: 97 pgs: 97 active+clean; 22 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 94 KiB/s rd, 17 MiB/s wr, 3.01k op/s 2024-09-11T23:28:50.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:50 smithi097 ceph-mon[35562]: pgmap v213: 97 pgs: 97 active+clean; 22 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 94 KiB/s rd, 17 MiB/s wr, 3.01k op/s 2024-09-11T23:28:50.686 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:28:51.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.226+0000 7fcbe214f640 1 -- 172.21.15.43:0/575706261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbdc102790 msgr2=0x7fcbdc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:51.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.226+0000 7fcbe214f640 1 --2- 172.21.15.43:0/575706261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbdc102790 0x7fcbdc1067e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009900 tx=0x7fcbd002a410 comp rx=0 tx=0).stop 2024-09-11T23:28:51.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.227+0000 7fcbe214f640 1 -- 172.21.15.43:0/575706261 shutdown_connections 2024-09-11T23:28:51.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.227+0000 7fcbe214f640 1 --2- 172.21.15.43:0/575706261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbdc106f10 0x7fcbdc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.227+0000 7fcbe214f640 1 --2- 172.21.15.43:0/575706261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbdc102790 0x7fcbdc1067e0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.227+0000 7fcbe214f640 1 --2- 172.21.15.43:0/575706261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbdc101de0 0x7fcbdc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.227+0000 7fcbe214f640 1 -- 172.21.15.43:0/575706261 >> 172.21.15.43:0/575706261 conn(0x7fcbdc0fd8c0 msgr2=0x7fcbdc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:51.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.227+0000 7fcbe214f640 1 -- 172.21.15.43:0/575706261 shutdown_connections 2024-09-11T23:28:51.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.227+0000 7fcbe214f640 1 -- 172.21.15.43:0/575706261 wait complete. 2024-09-11T23:28:51.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.227+0000 7fcbe214f640 1 Processor -- start 2024-09-11T23:28:51.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.227+0000 7fcbe214f640 1 -- start start 2024-09-11T23:28:51.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.228+0000 7fcbe214f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbdc101de0 0x7fcbdc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.228+0000 7fcbe214f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbdc102790 0x7fcbdc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.228+0000 7fcbe214f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbdc106f10 0x7fcbdc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.228+0000 7fcbe214f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc111f70 con 0x7fcbdc106f10 2024-09-11T23:28:51.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.228+0000 7fcbe214f640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc111df0 con 0x7fcbdc102790 2024-09-11T23:28:51.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.228+0000 7fcbe214f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcbdc1120f0 con 0x7fcbdc101de0 2024-09-11T23:28:51.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.228+0000 7fcbdbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbdc106f10 0x7fcbdc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:51.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.228+0000 7fcbdbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbdc106f10 0x7fcbdc074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36194/0 (socket says 172.21.15.43:36194) 2024-09-11T23:28:51.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.228+0000 7fcbdbfff640 1 -- 172.21.15.43:0/20902974 learned_addr learned my addr 172.21.15.43:0/20902974 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:51.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.228+0000 7fcbdaffd640 1 --2- 172.21.15.43:0/20902974 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbdc102790 0x7fcbdc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:51.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.228+0000 7fcbdb7fe640 1 --2- 172.21.15.43:0/20902974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbdc101de0 0x7fcbdc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:51.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.228+0000 7fcbdbfff640 1 -- 172.21.15.43:0/20902974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbdc101de0 msgr2=0x7fcbdc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:51.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.228+0000 7fcbdbfff640 1 --2- 172.21.15.43:0/20902974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbdc101de0 0x7fcbdc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.228+0000 7fcbdbfff640 1 -- 172.21.15.43:0/20902974 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbdc102790 msgr2=0x7fcbdc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:51.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.228+0000 7fcbdbfff640 1 --2- 172.21.15.43:0/20902974 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbdc102790 0x7fcbdc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.228+0000 7fcbdbfff640 1 -- 172.21.15.43:0/20902974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbdc1a68f0 con 0x7fcbdc106f10 2024-09-11T23:28:51.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.228+0000 7fcbdbfff640 1 --2- 172.21.15.43:0/20902974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbdc106f10 0x7fcbdc074f70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400eaa0 tx=0x7fcbd400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:51.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.228+0000 7fcbd8ff9640 1 -- 172.21.15.43:0/20902974 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd400cdb0 con 0x7fcbdc106f10 2024-09-11T23:28:51.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.229+0000 7fcbd8ff9640 1 -- 172.21.15.43:0/20902974 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbd4005020 con 0x7fcbdc106f10 2024-09-11T23:28:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.229+0000 7fcbe214f640 1 -- 172.21.15.43:0/20902974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbdc1a6be0 con 0x7fcbdc106f10 2024-09-11T23:28:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.229+0000 7fcbd8ff9640 1 -- 172.21.15.43:0/20902974 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd40116b0 con 0x7fcbdc106f10 2024-09-11T23:28:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.229+0000 7fcbe214f640 1 -- 172.21.15.43:0/20902974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbdc1a70f0 con 0x7fcbdc106f10 2024-09-11T23:28:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.230+0000 7fcbe214f640 1 -- 172.21.15.43:0/20902974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbdc1021c0 con 0x7fcbdc106f10 2024-09-11T23:28:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.231+0000 7fcbd8ff9640 1 -- 172.21.15.43:0/20902974 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fcbd4005560 con 0x7fcbdc106f10 2024-09-11T23:28:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.232+0000 7fcbd8ff9640 1 --2- 172.21.15.43:0/20902974 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fcbc40a2d70 0x7fcbc40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.232+0000 7fcbdb7fe640 1 --2- 172.21.15.43:0/20902974 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fcbc40a2d70 0x7fcbc40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.232+0000 7fcbd8ff9640 1 -- 172.21.15.43:0/20902974 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7fcbd40cf850 con 0x7fcbdc106f10 2024-09-11T23:28:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.232+0000 7fcbdb7fe640 1 --2- 172.21.15.43:0/20902974 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fcbc40a2d70 0x7fcbc40a5230 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc005fd0 tx=0x7fcbcc00b390 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.235+0000 7fcbd8ff9640 1 -- 172.21.15.43:0/20902974 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcbd4016030 con 0x7fcbdc106f10 2024-09-11T23:28:51.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.447+0000 7fcbe214f640 1 -- 172.21.15.43:0/20902974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbdc02d7c0 con 0x7fcbdc106f10 2024-09-11T23:28:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.448+0000 7fcbd8ff9640 1 -- 172.21.15.43:0/20902974 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11893 (secure 0 0 0) 0x7fcbd409bea0 con 0x7fcbdc106f10 2024-09-11T23:28:51.449 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:28:51.449 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"btime":"2024-09-11T23:25:40:446496+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:25:39.727562+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25571":{"gid":25571,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/568491708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":568491708},{"type":"v1","addr":"172.21.15.90:6835","nonce":568491708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:28:51.450 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-09-11T23:28:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.451+0000 7fcbe214f640 1 -- 172.21.15.43:0/20902974 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fcbc40a2d70 msgr2=0x7fcbc40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.451+0000 7fcbe214f640 1 --2- 172.21.15.43:0/20902974 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fcbc40a2d70 0x7fcbc40a5230 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc005fd0 tx=0x7fcbcc00b390 comp rx=0 tx=0).stop 2024-09-11T23:28:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.451+0000 7fcbe214f640 1 -- 172.21.15.43:0/20902974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbdc106f10 msgr2=0x7fcbdc074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.451+0000 7fcbe214f640 1 --2- 172.21.15.43:0/20902974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbdc106f10 0x7fcbdc074f70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400eaa0 tx=0x7fcbd400ef60 comp rx=0 tx=0).stop 2024-09-11T23:28:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.451+0000 7fcbe214f640 1 -- 172.21.15.43:0/20902974 shutdown_connections 2024-09-11T23:28:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.451+0000 7fcbe214f640 1 --2- 172.21.15.43:0/20902974 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fcbc40a2d70 0x7fcbc40a5230 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.451+0000 7fcbe214f640 1 --2- 172.21.15.43:0/20902974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbdc106f10 0x7fcbdc074f70 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.451+0000 7fcbe214f640 1 --2- 172.21.15.43:0/20902974 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbdc102790 0x7fcbdc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.451+0000 7fcbe214f640 1 --2- 172.21.15.43:0/20902974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbdc101de0 0x7fcbdc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.451+0000 7fcbe214f640 1 -- 172.21.15.43:0/20902974 >> 172.21.15.43:0/20902974 conn(0x7fcbdc0fd8c0 msgr2=0x7fcbdc0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.451+0000 7fcbe214f640 1 -- 172.21.15.43:0/20902974 shutdown_connections 2024-09-11T23:28:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:51.451+0000 7fcbe214f640 1 -- 172.21.15.43:0/20902974 wait complete. 2024-09-11T23:28:52.166 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 733f7a10 --release --await' 2024-09-11T23:28:52.167 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 733f7a10 --release --await 2024-09-11T23:28:52.426 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:28:52.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:52 smithi090 ceph-mon[35130]: pgmap v214: 97 pgs: 97 active+clean; 22 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 90 KiB/s rd, 11 MiB/s wr, 2.06k op/s 2024-09-11T23:28:52.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:52 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/20902974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:28:52.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:52 smithi097 ceph-mon[35562]: pgmap v214: 97 pgs: 97 active+clean; 22 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 90 KiB/s rd, 11 MiB/s wr, 2.06k op/s 2024-09-11T23:28:52.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:52 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/20902974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:28:52.552 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:52 smithi043 ceph-mon[30950]: pgmap v214: 97 pgs: 97 active+clean; 22 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 90 KiB/s rd, 11 MiB/s wr, 2.06k op/s 2024-09-11T23:28:52.552 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:52 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/20902974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:28:52.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.932+0000 7f8b9b165640 1 -- 172.21.15.43:0/3146060713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b94102ef0 msgr2=0x7f8b9410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:52.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.932+0000 7f8b9b165640 1 --2- 172.21.15.43:0/3146060713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b94102ef0 0x7f8b9410da40 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009ca0 tx=0x7f8b8c02a950 comp rx=0 tx=0).stop 2024-09-11T23:28:52.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.932+0000 7f8b9b165640 1 -- 172.21.15.43:0/3146060713 shutdown_connections 2024-09-11T23:28:52.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.932+0000 7f8b9b165640 1 --2- 172.21.15.43:0/3146060713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b94102ef0 0x7f8b9410da40 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.932+0000 7f8b9b165640 1 --2- 172.21.15.43:0/3146060713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b94102570 0x7f8b941029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.932+0000 7f8b9b165640 1 --2- 172.21.15.43:0/3146060713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b94100e20 0x7f8b94101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.932+0000 7f8b9b165640 1 -- 172.21.15.43:0/3146060713 >> 172.21.15.43:0/3146060713 conn(0x7f8b940fc7d0 msgr2=0x7f8b940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:52.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.932+0000 7f8b9b165640 1 -- 172.21.15.43:0/3146060713 shutdown_connections 2024-09-11T23:28:52.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.932+0000 7f8b9b165640 1 -- 172.21.15.43:0/3146060713 wait complete. 2024-09-11T23:28:52.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.933+0000 7f8b9b165640 1 Processor -- start 2024-09-11T23:28:52.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.933+0000 7f8b9b165640 1 -- start start 2024-09-11T23:28:52.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.933+0000 7f8b9b165640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b94100e20 0x7f8b9419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.933+0000 7f8b9b165640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b94102570 0x7f8b9419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.933+0000 7f8b9b165640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b94102ef0 0x7f8b9419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.933+0000 7f8b9b165640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8b94110f80 con 0x7f8b94102570 2024-09-11T23:28:52.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.933+0000 7f8b9b165640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8b94110e00 con 0x7f8b94100e20 2024-09-11T23:28:52.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.933+0000 7f8b9b165640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8b94111100 con 0x7f8b94102ef0 2024-09-11T23:28:52.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.933+0000 7f8b93fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b94102570 0x7f8b9419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.933+0000 7f8b93fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b94102570 0x7f8b9419aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36204/0 (socket says 172.21.15.43:36204) 2024-09-11T23:28:52.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.933+0000 7f8b93fff640 1 -- 172.21.15.43:0/767556038 learned_addr learned my addr 172.21.15.43:0/767556038 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:52.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.933+0000 7f8b99702640 1 --2- 172.21.15.43:0/767556038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b94102ef0 0x7f8b9419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.933+0000 7f8b98f01640 1 --2- 172.21.15.43:0/767556038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b94100e20 0x7f8b9419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.934+0000 7f8b93fff640 1 -- 172.21.15.43:0/767556038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b94102ef0 msgr2=0x7f8b9419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:52.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.934+0000 7f8b93fff640 1 --2- 172.21.15.43:0/767556038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b94102ef0 0x7f8b9419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.934+0000 7f8b93fff640 1 -- 172.21.15.43:0/767556038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b94100e20 msgr2=0x7f8b9419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:52.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.934+0000 7f8b93fff640 1 --2- 172.21.15.43:0/767556038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b94100e20 0x7f8b9419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.934+0000 7f8b93fff640 1 -- 172.21.15.43:0/767556038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9419f4e0 con 0x7f8b94102570 2024-09-11T23:28:52.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.934+0000 7f8b93fff640 1 --2- 172.21.15.43:0/767556038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b94102570 0x7f8b9419aa70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000c920 tx=0x7f8b8000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:52.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.934+0000 7f8b91ffb640 1 -- 172.21.15.43:0/767556038 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b80007c70 con 0x7f8b94102570 2024-09-11T23:28:52.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.934+0000 7f8b9b165640 1 -- 172.21.15.43:0/767556038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b9419f7d0 con 0x7f8b94102570 2024-09-11T23:28:52.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.934+0000 7f8b91ffb640 1 -- 172.21.15.43:0/767556038 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b80007e10 con 0x7f8b94102570 2024-09-11T23:28:52.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.934+0000 7f8b91ffb640 1 -- 172.21.15.43:0/767556038 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b80018890 con 0x7f8b94102570 2024-09-11T23:28:52.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.934+0000 7f8b9b165640 1 -- 172.21.15.43:0/767556038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b941a70b0 con 0x7f8b94102570 2024-09-11T23:28:52.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.937+0000 7f8b91ffb640 1 -- 172.21.15.43:0/767556038 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8b80018b30 con 0x7f8b94102570 2024-09-11T23:28:52.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.937+0000 7f8b9d53e740 1 Processor -- start 2024-09-11T23:28:52.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.937+0000 7f8b9d53e740 1 -- start start 2024-09-11T23:28:52.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.937+0000 7f8b9d53e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a6b8a1060 0x559a6b8a1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.937+0000 7f8b9d53e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b8a1980 0x559a6b8abd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.937+0000 7f8b9d53e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8ac240 0x559a6b8ae630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.937+0000 7f8b9d53e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559a6b806620 con 0x559a6b8ac240 2024-09-11T23:28:52.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.937+0000 7f8b9d53e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559a6b7ade50 con 0x559a6b8a1980 2024-09-11T23:28:52.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.937+0000 7f8b9d53e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559a6b8025e0 con 0x559a6b8a1060 2024-09-11T23:28:52.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.937+0000 7f8b91ffb640 1 --2- 172.21.15.43:0/767556038 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f8b840a2cf0 0x7f8b840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.937+0000 7f8b93fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8ac240 0x559a6b8ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.937+0000 7f8b93fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8ac240 0x559a6b8ae630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36208/0 (socket says 172.21.15.43:36208) 2024-09-11T23:28:52.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.937+0000 7f8b93fff640 1 -- 172.21.15.43:0/1664899516 learned_addr learned my addr 172.21.15.43:0/1664899516 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:52.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.938+0000 7f8b99702640 1 --2- 172.21.15.43:0/767556038 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f8b840a2cf0 0x7f8b840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.938+0000 7f8b98f01640 1 --2- 172.21.15.43:0/1664899516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b8a1980 0x559a6b8abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.938+0000 7f8b99702640 1 --2- 172.21.15.43:0/1664899516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a6b8a1060 0x559a6b8a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.938+0000 7f8b91ffb640 1 -- 172.21.15.43:0/767556038 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f8b800cf600 con 0x7f8b94102570 2024-09-11T23:28:52.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.938+0000 7f8b98f01640 1 -- 172.21.15.43:0/1664899516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a6b8a1060 msgr2=0x559a6b8a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:52.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.938+0000 7f8b98f01640 1 --2- 172.21.15.43:0/1664899516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a6b8a1060 0x559a6b8a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.938+0000 7f8b98f01640 1 -- 172.21.15.43:0/1664899516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8ac240 msgr2=0x559a6b8ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:52.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.938+0000 7f8b98f01640 1 --2- 172.21.15.43:0/1664899516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8ac240 0x559a6b8ae630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.938+0000 7f8b98f01640 1 -- 172.21.15.43:0/1664899516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a6b79ed40 con 0x559a6b8a1980 2024-09-11T23:28:52.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.938+0000 7f8b93fff640 1 --2- 172.21.15.43:0/1664899516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8ac240 0x559a6b8ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:52.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.938+0000 7f8b99702640 1 --2- 172.21.15.43:0/767556038 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f8b840a2cf0 0x7f8b840a51b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0098d0 tx=0x7f8b8c0099a0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:52.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.938+0000 7f8b98f01640 1 --2- 172.21.15.43:0/1664899516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b8a1980 0x559a6b8abd00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800cba0 tx=0x7f8b8800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bf604afd984bf64 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:52.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.938+0000 7f8b7effd640 1 -- 172.21.15.43:0/1664899516 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88004220 con 0x559a6b8a1980 2024-09-11T23:28:52.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.938+0000 7f8b7effd640 1 -- 172.21.15.43:0/1664899516 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b880043c0 con 0x559a6b8a1980 2024-09-11T23:28:52.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.938+0000 7f8b7effd640 1 -- 172.21.15.43:0/1664899516 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88004e00 con 0x559a6b8a1980 2024-09-11T23:28:52.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.939+0000 7f8b9d53e740 1 -- 172.21.15.43:0/1664899516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b8a1980 msgr2=0x559a6b8abd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:52.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.939+0000 7f8b9d53e740 1 --2- 172.21.15.43:0/1664899516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b8a1980 0x559a6b8abd00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800cba0 tx=0x7f8b8800e720 comp rx=0 tx=0).stop 2024-09-11T23:28:52.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.939+0000 7f8b9d53e740 1 -- 172.21.15.43:0/1664899516 shutdown_connections 2024-09-11T23:28:52.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.939+0000 7f8b9d53e740 1 --2- 172.21.15.43:0/1664899516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8ac240 0x559a6b8ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.939+0000 7f8b9d53e740 1 --2- 172.21.15.43:0/1664899516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b8a1980 0x559a6b8abd00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.939+0000 7f8b9d53e740 1 --2- 172.21.15.43:0/1664899516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a6b8a1060 0x559a6b8a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.939+0000 7f8b9d53e740 1 -- 172.21.15.43:0/1664899516 >> 172.21.15.43:0/1664899516 conn(0x559a6b89d5f0 msgr2=0x559a6b89d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:52.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.939+0000 7f8b9d53e740 1 -- 172.21.15.43:0/1664899516 shutdown_connections 2024-09-11T23:28:52.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.939+0000 7f8b9d53e740 1 -- 172.21.15.43:0/1664899516 wait complete. 2024-09-11T23:28:52.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.939+0000 7f8b9d53e740 1 -- start start 2024-09-11T23:28:52.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.940+0000 7f8b9d53e740 1 Processor -- start 2024-09-11T23:28:52.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.940+0000 7f8b9d53e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a6b8a1060 0x559a6b8a90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.940+0000 7f8b9d53e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b8a1980 0x559a6b8a9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.940+0000 7f8b9d53e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8ac240 0x559a6b8a9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.940+0000 7f8b9d53e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559a6b8aee10 con 0x559a6b8ac240 2024-09-11T23:28:52.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.940+0000 7f8b9d53e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559a6b78ef10 con 0x559a6b8a1980 2024-09-11T23:28:52.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.940+0000 7f8b9d53e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559a6b8aef90 con 0x559a6b8a1060 2024-09-11T23:28:52.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.940+0000 7f8b9d53e740 10 client.0 _ll_register_callbacks cb 0x559a6b7b2db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:52.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.940+0000 7f8b99702640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8ac240 0x559a6b8a9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.940+0000 7f8b99702640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8ac240 0x559a6b8a9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36220/0 (socket says 172.21.15.43:36220) 2024-09-11T23:28:52.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.940+0000 7f8b99702640 1 -- 172.21.15.43:0/3502675480 learned_addr learned my addr 172.21.15.43:0/3502675480 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:52.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.940+0000 7f8b93fff640 1 --2- 172.21.15.43:0/3502675480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b8a1980 0x559a6b8a9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.940+0000 7f8b98f01640 1 --2- 172.21.15.43:0/3502675480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a6b8a1060 0x559a6b8a90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.940+0000 7f8b99702640 1 -- 172.21.15.43:0/3502675480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a6b8a1060 msgr2=0x559a6b8a90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:52.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.940+0000 7f8b99702640 1 --2- 172.21.15.43:0/3502675480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a6b8a1060 0x559a6b8a90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.940+0000 7f8b99702640 1 -- 172.21.15.43:0/3502675480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b8a1980 msgr2=0x559a6b8a9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:52.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.940+0000 7f8b99702640 1 --2- 172.21.15.43:0/3502675480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b8a1980 0x559a6b8a9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.940+0000 7f8b99702640 1 -- 172.21.15.43:0/3502675480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a6b515580 con 0x559a6b8ac240 2024-09-11T23:28:52.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.940+0000 7f8b98f01640 1 --2- 172.21.15.43:0/3502675480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a6b8a1060 0x559a6b8a90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:28:52.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.941+0000 7f8b99702640 1 --2- 172.21.15.43:0/3502675480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8ac240 0x559a6b8a9b50 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c003ab0 tx=0x7f8b8c02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346c7814a8cdec7f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:52.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.941+0000 7f8b73fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:28:52.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.941+0000 7f8b73fff640 1 -- 172.21.15.43:0/3502675480 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c04f030 con 0x559a6b8ac240 2024-09-11T23:28:52.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.941+0000 7f8b9d53e740 1 -- 172.21.15.43:0/3502675480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b880043c0 con 0x559a6b8ac240 2024-09-11T23:28:52.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.941+0000 7f8b73fff640 1 -- 172.21.15.43:0/3502675480 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f8b8c045040 con 0x559a6b8ac240 2024-09-11T23:28:52.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.941+0000 7f8b73fff640 1 -- 172.21.15.43:0/3502675480 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8b8c034040 con 0x559a6b8ac240 2024-09-11T23:28:52.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.941+0000 7f8b9d53e740 10 client.16503 fetch_fsmap learned FSMap version 30 2024-09-11T23:28:52.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.941+0000 7f8b9d53e740 1 -- 172.21.15.43:0/3502675480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a6b5df960 con 0x559a6b8ac240 2024-09-11T23:28:52.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.941+0000 7f8b73fff640 1 -- 172.21.15.43:0/3502675480 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b880043c0 con 0x559a6b8ac240 2024-09-11T23:28:52.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.942+0000 7f8b73fff640 1 -- 172.21.15.43:0/3502675480 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f8b8c04b9f0 con 0x559a6b8ac240 2024-09-11T23:28:52.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.942+0000 7f8b9d53e740 10 client.16503 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:28:52.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.942+0000 7f8b9d53e740 10 client.16503 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:28:52.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.942+0000 7f8b9d53e740 20 client.16503 populate_metadata read hostname 'smithi043' 2024-09-11T23:28:52.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.942+0000 7f8b9d53e740 1 --2- 172.21.15.43:0/3502675480 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x559a6b8bbce0 0x559a6b8be150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.942+0000 7f8b9d53e740 4 client.16503 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:28:52.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.942+0000 7f8b9d53e740 1 -- 172.21.15.43:0/3502675480 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a6b813b90 con 0x559a6b8bbce0 2024-09-11T23:28:52.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.942+0000 7f8b98f01640 1 --2- 172.21.15.43:0/3502675480 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x559a6b8bbce0 0x559a6b8be150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.943+0000 7f8b98f01640 1 --2- 172.21.15.43:0/3502675480 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x559a6b8bbce0 0x559a6b8be150 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=435516b38137b60d server_cookie=94ab3e1c9a4aedc7 in_seq=0 out_seq=0 2024-09-11T23:28:52.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.943+0000 7f8b73fff640 10 client.16503 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:52.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.943+0000 7f8b73fff640 1 -- 172.21.15.43:0/3502675480 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a6b813b90 con 0x559a6b8bbce0 2024-09-11T23:28:52.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.943+0000 7f8b73fff640 10 client.16503 handle_command_reply: tid=0 2024-09-11T23:28:52.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.943+0000 7f8b9d53e740 1 client.16503 shutdown 2024-09-11T23:28:52.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.943+0000 7f8b9d53e740 1 -- 172.21.15.43:0/3502675480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8ac240 msgr2=0x559a6b8a9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:52.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.943+0000 7f8b9d53e740 1 --2- 172.21.15.43:0/3502675480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8ac240 0x559a6b8a9b50 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c003ab0 tx=0x7f8b8c02a950 comp rx=0 tx=0).stop 2024-09-11T23:28:52.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.943+0000 7f8b9d53e740 1 -- 172.21.15.43:0/3502675480 shutdown_connections 2024-09-11T23:28:52.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.943+0000 7f8b9d53e740 1 --2- 172.21.15.43:0/3502675480 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x559a6b8bbce0 0x559a6b8be150 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.943+0000 7f8b9d53e740 1 --2- 172.21.15.43:0/3502675480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8ac240 0x559a6b8a9b50 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.943+0000 7f8b9d53e740 1 --2- 172.21.15.43:0/3502675480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b8a1980 0x559a6b8a9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.943+0000 7f8b9d53e740 1 --2- 172.21.15.43:0/3502675480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a6b8a1060 0x559a6b8a90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.943+0000 7f8b9d53e740 1 -- 172.21.15.43:0/3502675480 >> 172.21.15.43:0/3502675480 conn(0x559a6b89d5f0 msgr2=0x559a6b8a32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.944+0000 7f8b73fff640 0 client.16503 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.944+0000 7f8b9d53e740 1 -- 172.21.15.43:0/3502675480 shutdown_connections 2024-09-11T23:28:52.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.944+0000 7f8b9d53e740 1 -- 172.21.15.43:0/3502675480 wait complete. 2024-09-11T23:28:52.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.944+0000 7f8b9d53e740 20 client.16503 trim_cache size 0 max 16384 2024-09-11T23:28:52.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.961+0000 7f8b9d53e740 1 Processor -- start 2024-09-11T23:28:52.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.961+0000 7f8b9d53e740 1 -- start start 2024-09-11T23:28:52.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.961+0000 7f8b9d53e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b89d5f0 0x559a6b8a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.961+0000 7f8b9d53e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a6b8a1060 0x559a6b8a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.961+0000 7f8b9d53e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8a1980 0x559a6b89c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.961+0000 7f8b9d53e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8b941b2280 con 0x559a6b8a1980 2024-09-11T23:28:52.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.961+0000 7f8b9d53e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8b941afb50 con 0x559a6b89d5f0 2024-09-11T23:28:52.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.961+0000 7f8b9d53e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8b941ad9a0 con 0x559a6b8a1060 2024-09-11T23:28:52.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.961+0000 7f8b99702640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8a1980 0x559a6b89c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.961+0000 7f8b99702640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8a1980 0x559a6b89c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36228/0 (socket says 172.21.15.43:36228) 2024-09-11T23:28:52.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.961+0000 7f8b99702640 1 -- 172.21.15.43:0/2740109313 learned_addr learned my addr 172.21.15.43:0/2740109313 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:52.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.961+0000 7f8b93fff640 1 --2- 172.21.15.43:0/2740109313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a6b8a1060 0x559a6b8a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.961+0000 7f8b98f01640 1 --2- 172.21.15.43:0/2740109313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b89d5f0 0x559a6b8a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.961+0000 7f8b99702640 1 -- 172.21.15.43:0/2740109313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a6b8a1060 msgr2=0x559a6b8a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:52.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.961+0000 7f8b99702640 1 --2- 172.21.15.43:0/2740109313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a6b8a1060 0x559a6b8a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.961+0000 7f8b99702640 1 -- 172.21.15.43:0/2740109313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b89d5f0 msgr2=0x559a6b8a3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:52.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.961+0000 7f8b99702640 1 --2- 172.21.15.43:0/2740109313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b89d5f0 0x559a6b8a3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.961+0000 7f8b99702640 1 -- 172.21.15.43:0/2740109313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a6b8ab010 con 0x559a6b8a1980 2024-09-11T23:28:52.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.962+0000 7f8b99702640 1 --2- 172.21.15.43:0/2740109313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8a1980 0x559a6b89c640 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009ca0 tx=0x7f8b8c02b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1baa15a988405d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:52.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.962+0000 7f8b7e7fc640 1 -- 172.21.15.43:0/2740109313 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c04f070 con 0x559a6b8a1980 2024-09-11T23:28:52.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.962+0000 7f8b7e7fc640 1 -- 172.21.15.43:0/2740109313 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b8c04b450 con 0x559a6b8a1980 2024-09-11T23:28:52.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.962+0000 7f8b7e7fc640 1 -- 172.21.15.43:0/2740109313 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c03b6a0 con 0x559a6b8a1980 2024-09-11T23:28:52.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.962+0000 7f8b9d53e740 1 -- 172.21.15.43:0/2740109313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8a1980 msgr2=0x559a6b89c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:52.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.962+0000 7f8b9d53e740 1 --2- 172.21.15.43:0/2740109313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8a1980 0x559a6b89c640 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009ca0 tx=0x7f8b8c02b0d0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.962+0000 7f8b9d53e740 1 -- 172.21.15.43:0/2740109313 shutdown_connections 2024-09-11T23:28:52.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.962+0000 7f8b9d53e740 1 --2- 172.21.15.43:0/2740109313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8a1980 0x559a6b89c640 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.962+0000 7f8b9d53e740 1 --2- 172.21.15.43:0/2740109313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a6b8a1060 0x559a6b8a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.962+0000 7f8b9d53e740 1 --2- 172.21.15.43:0/2740109313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b89d5f0 0x559a6b8a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.962+0000 7f8b9d53e740 1 -- 172.21.15.43:0/2740109313 >> 172.21.15.43:0/2740109313 conn(0x559a6b8a4ac0 msgr2=0x559a6b89bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:52.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.962+0000 7f8b9d53e740 1 -- 172.21.15.43:0/2740109313 shutdown_connections 2024-09-11T23:28:52.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.962+0000 7f8b9d53e740 1 -- 172.21.15.43:0/2740109313 wait complete. 2024-09-11T23:28:52.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.963+0000 7f8b9d53e740 1 -- start start 2024-09-11T23:28:52.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.963+0000 7f8b9d53e740 1 Processor -- start 2024-09-11T23:28:52.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.963+0000 7f8b9d53e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a6b89d5f0 0x559a6b8b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.963+0000 7f8b9d53e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b8a1060 0x559a6b8b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.963+0000 7f8b9d53e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8a1980 0x559a6b8b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.963+0000 7f8b9d53e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8b941ae010 con 0x559a6b8a1980 2024-09-11T23:28:52.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.963+0000 7f8b9d53e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8b941af480 con 0x559a6b8a1060 2024-09-11T23:28:52.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.963+0000 7f8b9d53e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8b941ae6e0 con 0x559a6b89d5f0 2024-09-11T23:28:52.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.963+0000 7f8b9d53e740 10 client.0 _ll_register_callbacks cb 0x7f8b8c003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:52.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.963+0000 7f8b99702640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8a1980 0x559a6b8b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.963+0000 7f8b93fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b8a1060 0x559a6b8b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.963+0000 7f8b99702640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8a1980 0x559a6b8b0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36232/0 (socket says 172.21.15.43:36232) 2024-09-11T23:28:52.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.963+0000 7f8b99702640 1 -- 172.21.15.43:0/1691713551 learned_addr learned my addr 172.21.15.43:0/1691713551 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:52.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.964+0000 7f8b98f01640 1 --2- 172.21.15.43:0/1691713551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a6b89d5f0 0x559a6b8b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.964+0000 7f8b99702640 1 -- 172.21.15.43:0/1691713551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a6b89d5f0 msgr2=0x559a6b8b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:52.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.964+0000 7f8b99702640 1 --2- 172.21.15.43:0/1691713551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a6b89d5f0 0x559a6b8b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.964+0000 7f8b99702640 1 -- 172.21.15.43:0/1691713551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b8a1060 msgr2=0x559a6b8b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:52.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.964+0000 7f8b99702640 1 --2- 172.21.15.43:0/1691713551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b8a1060 0x559a6b8b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:52.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.964+0000 7f8b99702640 1 -- 172.21.15.43:0/1691713551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a6b925bf0 con 0x559a6b8a1980 2024-09-11T23:28:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.964+0000 7f8b93fff640 1 --2- 172.21.15.43:0/1691713551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b8a1060 0x559a6b8b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.964+0000 7f8b99702640 1 --2- 172.21.15.43:0/1691713551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8a1980 0x559a6b8b0b70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c04b8b0 tx=0x7f8b8c04b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5e3faa77d46fa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.964+0000 7f8b73fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:28:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.964+0000 7f8b73fff640 1 -- 172.21.15.43:0/1691713551 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c061030 con 0x559a6b8a1980 2024-09-11T23:28:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.964+0000 7f8b73fff640 1 -- 172.21.15.43:0/1691713551 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f8b8c045040 con 0x559a6b8a1980 2024-09-11T23:28:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.964+0000 7f8b9d53e740 1 -- 172.21.15.43:0/1691713551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b8c04b450 con 0x559a6b8a1980 2024-09-11T23:28:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.964+0000 7f8b73fff640 1 -- 172.21.15.43:0/1691713551 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8b8c06b050 con 0x559a6b8a1980 2024-09-11T23:28:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.964+0000 7f8b9d53e740 10 client.16509 fetch_fsmap learned FSMap version 30 2024-09-11T23:28:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.964+0000 7f8b9d53e740 1 -- 172.21.15.43:0/1691713551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a6b926eb0 con 0x559a6b8a1980 2024-09-11T23:28:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.965+0000 7f8b73fff640 1 -- 172.21.15.43:0/1691713551 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b8c04b450 con 0x559a6b8a1980 2024-09-11T23:28:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.965+0000 7f8b73fff640 1 -- 172.21.15.43:0/1691713551 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f8b8c030bf0 con 0x559a6b8a1980 2024-09-11T23:28:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.965+0000 7f8b9d53e740 10 client.16509 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:28:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.965+0000 7f8b9d53e740 10 client.16509 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:28:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.965+0000 7f8b9d53e740 20 client.16509 populate_metadata read hostname 'smithi043' 2024-09-11T23:28:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.965+0000 7f8b9d53e740 1 --2- 172.21.15.43:0/1691713551 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x559a6b8ac240 0x559a6b929520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.965+0000 7f8b9d53e740 4 client.16509 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "733f7a10", "release": true, "await": true}] 2024-09-11T23:28:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.965+0000 7f8b9d53e740 1 -- 172.21.15.43:0/1691713551 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "quiesce db", "set_id": "733f7a10", "release": true, "await": true}) -- 0x559a6b80e0a0 con 0x559a6b8ac240 2024-09-11T23:28:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.965+0000 7f8b98f01640 1 --2- 172.21.15.43:0/1691713551 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x559a6b8ac240 0x559a6b929520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.965+0000 7f8b98f01640 1 --2- 172.21.15.43:0/1691713551 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x559a6b8ac240 0x559a6b929520 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a764e22702efcbd server_cookie=2c1aed4c87376e27 in_seq=0 out_seq=0 2024-09-11T23:28:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.965+0000 7f8b73fff640 10 client.16509 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:53.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.999+0000 7f8b73fff640 1 -- 172.21.15.43:0/1691713551 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559a6b80e0a0 con 0x559a6b8ac240 2024-09-11T23:28:53.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:52.999+0000 7f8b73fff640 10 client.16509 handle_command_reply: tid=0 2024-09-11T23:28:53.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.000+0000 7f8b9d53e740 1 client.16509 shutdown 2024-09-11T23:28:53.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.002+0000 7f8b9d53e740 1 -- 172.21.15.43:0/1691713551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8a1980 msgr2=0x559a6b8b0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:53.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.002+0000 7f8b9d53e740 1 --2- 172.21.15.43:0/1691713551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8a1980 0x559a6b8b0b70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c04b8b0 tx=0x7f8b8c04b8e0 comp rx=0 tx=0).stop 2024-09-11T23:28:53.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.002+0000 7f8b9d53e740 1 -- 172.21.15.43:0/1691713551 shutdown_connections 2024-09-11T23:28:53.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.002+0000 7f8b9d53e740 1 --2- 172.21.15.43:0/1691713551 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x559a6b8ac240 0x559a6b929520 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:53.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.002+0000 7f8b73fff640 0 client.16509 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:53.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.002+0000 7f8b9d53e740 1 --2- 172.21.15.43:0/1691713551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a6b8a1980 0x559a6b8b0b70 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:53.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.002+0000 7f8b9d53e740 1 --2- 172.21.15.43:0/1691713551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a6b8a1060 0x559a6b8b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:53.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.002+0000 7f8b9d53e740 1 --2- 172.21.15.43:0/1691713551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a6b89d5f0 0x559a6b8b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:53.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.002+0000 7f8b9d53e740 1 -- 172.21.15.43:0/1691713551 >> 172.21.15.43:0/1691713551 conn(0x559a6b8a4ac0 msgr2=0x559a6b8a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:53.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.007+0000 7f8b9d53e740 1 -- 172.21.15.43:0/1691713551 shutdown_connections 2024-09-11T23:28:53.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.007+0000 7f8b9d53e740 1 -- 172.21.15.43:0/1691713551 wait complete. 2024-09-11T23:28:53.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.007+0000 7f8b9d53e740 20 client.16509 trim_cache size 0 max 16384 2024-09-11T23:28:53.007 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:28:53.007 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 30, 2024-09-11T23:28:53.007 INFO:teuthology.orchestra.run.smithi043.stdout: "leader": 24497, 2024-09-11T23:28:53.007 INFO:teuthology.orchestra.run.smithi043.stdout: "set_version": 16, 2024-09-11T23:28:53.007 INFO:teuthology.orchestra.run.smithi043.stdout: "sets": { 2024-09-11T23:28:53.008 INFO:teuthology.orchestra.run.smithi043.stdout: "733f7a10": { 2024-09-11T23:28:53.008 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 16, 2024-09-11T23:28:53.008 INFO:teuthology.orchestra.run.smithi043.stdout: "age_ref": 0.0, 2024-09-11T23:28:53.008 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:28:53.008 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RELEASED", 2024-09-11T23:28:53.008 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-09-11T23:28:53.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:28:53.008 INFO:teuthology.orchestra.run.smithi043.stdout: "timeout": 90.0, 2024-09-11T23:28:53.008 INFO:teuthology.orchestra.run.smithi043.stdout: "expiration": 140.2, 2024-09-11T23:28:53.008 INFO:teuthology.orchestra.run.smithi043.stdout: "members": { 2024-09-11T23:28:53.008 INFO:teuthology.orchestra.run.smithi043.stdout: "file:/": { 2024-09-11T23:28:53.008 INFO:teuthology.orchestra.run.smithi043.stdout: "excluded": false, 2024-09-11T23:28:53.009 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:28:53.009 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RELEASED", 2024-09-11T23:28:53.009 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-09-11T23:28:53.009 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:28:53.009 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:28:53.009 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:28:53.009 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:28:53.009 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:28:53.009 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:28:53.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.008+0000 7f8b9b165640 1 -- 172.21.15.43:0/767556038 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f8b840a2cf0 msgr2=0x7f8b840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:53.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.008+0000 7f8b9b165640 1 --2- 172.21.15.43:0/767556038 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f8b840a2cf0 0x7f8b840a51b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0098d0 tx=0x7f8b8c0099a0 comp rx=0 tx=0).stop 2024-09-11T23:28:53.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.008+0000 7f8b9b165640 1 -- 172.21.15.43:0/767556038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b94102570 msgr2=0x7f8b9419aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:53.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.008+0000 7f8b9b165640 1 --2- 172.21.15.43:0/767556038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b94102570 0x7f8b9419aa70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000c920 tx=0x7f8b8000cde0 comp rx=0 tx=0).stop 2024-09-11T23:28:53.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.008+0000 7f8b9b165640 1 -- 172.21.15.43:0/767556038 shutdown_connections 2024-09-11T23:28:53.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.008+0000 7f8b9b165640 1 --2- 172.21.15.43:0/767556038 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f8b840a2cf0 0x7f8b840a51b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:53.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.008+0000 7f8b9b165640 1 --2- 172.21.15.43:0/767556038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b94102ef0 0x7f8b9419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:53.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.008+0000 7f8b9b165640 1 --2- 172.21.15.43:0/767556038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b94102570 0x7f8b9419aa70 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:53.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.008+0000 7f8b9b165640 1 --2- 172.21.15.43:0/767556038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b94100e20 0x7f8b9419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:53.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.008+0000 7f8b9b165640 1 -- 172.21.15.43:0/767556038 >> 172.21.15.43:0/767556038 conn(0x7f8b940fc7d0 msgr2=0x7f8b9406a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:53.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.009+0000 7f8b9b165640 1 -- 172.21.15.43:0/767556038 shutdown_connections 2024-09-11T23:28:53.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:28:53.011+0000 7f8b9b165640 1 -- 172.21.15.43:0/767556038 wait complete. 2024-09-11T23:28:53.051 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:28:54.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:54 smithi090 ceph-mon[35130]: pgmap v215: 97 pgs: 97 active+clean; 22 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 55 KiB/s rd, 9.4 MiB/s wr, 1.70k op/s 2024-09-11T23:28:54.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:54 smithi097 ceph-mon[35562]: pgmap v215: 97 pgs: 97 active+clean; 22 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 55 KiB/s rd, 9.4 MiB/s wr, 1.70k op/s 2024-09-11T23:28:54.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:54 smithi043 ceph-mon[30950]: pgmap v215: 97 pgs: 97 active+clean; 22 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 55 KiB/s rd, 9.4 MiB/s wr, 1.70k op/s 2024-09-11T23:28:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.147+0000 7f9e9fe27640 1 -- 172.21.15.43:0/3970690238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e981091f0 msgr2=0x7f9e98110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.147+0000 7f9e9fe27640 1 --2- 172.21.15.43:0/3970690238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e981091f0 0x7f9e98110250 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009930 tx=0x7f9e8802a480 comp rx=0 tx=0).stop 2024-09-11T23:28:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.151+0000 7f9e9fe27640 1 -- 172.21.15.43:0/3970690238 shutdown_connections 2024-09-11T23:28:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.151+0000 7f9e9fe27640 1 --2- 172.21.15.43:0/3970690238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e9811b140 0x7f9e9811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.151+0000 7f9e9fe27640 1 --2- 172.21.15.43:0/3970690238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e981095d0 0x7f9e98109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.151+0000 7f9e9fe27640 1 --2- 172.21.15.43:0/3970690238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e981091f0 0x7f9e98110250 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.151+0000 7f9e9fe27640 1 -- 172.21.15.43:0/3970690238 >> 172.21.15.43:0/3970690238 conn(0x7f9e9806cdf0 msgr2=0x7f9e9806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.159+0000 7f9e9fe27640 1 -- 172.21.15.43:0/3970690238 shutdown_connections 2024-09-11T23:28:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.160+0000 7f9e9fe27640 1 -- 172.21.15.43:0/3970690238 wait complete. 2024-09-11T23:28:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.161+0000 7f9e9fe27640 1 Processor -- start 2024-09-11T23:28:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.163+0000 7f9e9fe27640 1 -- start start 2024-09-11T23:28:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.163+0000 7f9e9fe27640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e981091f0 0x7f9e981186e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.163+0000 7f9e9fe27640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e981095d0 0x7f9e98111760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.163+0000 7f9e9fe27640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e9811b140 0x7f9e98111ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.163+0000 7f9e9fe27640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e98120d30 con 0x7f9e981095d0 2024-09-11T23:28:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.163+0000 7f9e9fe27640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9e98120bb0 con 0x7f9e9811b140 2024-09-11T23:28:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.163+0000 7f9e9fe27640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e98120eb0 con 0x7f9e981091f0 2024-09-11T23:28:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.164+0000 7f9e9e3c4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e9811b140 0x7f9e98111ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.165+0000 7f9e9e3c4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e9811b140 0x7f9e98111ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:54372/0 (socket says 172.21.15.43:54372) 2024-09-11T23:28:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.165+0000 7f9e9e3c4640 1 -- 172.21.15.43:0/1140266612 learned_addr learned my addr 172.21.15.43:0/1140266612 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.166+0000 7f9e9dbc3640 1 --2- 172.21.15.43:0/1140266612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e981091f0 0x7f9e981186e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.167+0000 7f9e9e3c4640 1 -- 172.21.15.43:0/1140266612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e981091f0 msgr2=0x7f9e981186e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.167+0000 7f9e9e3c4640 1 --2- 172.21.15.43:0/1140266612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e981091f0 0x7f9e981186e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.167+0000 7f9e9e3c4640 1 -- 172.21.15.43:0/1140266612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e981095d0 msgr2=0x7f9e98111760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.167+0000 7f9e9e3c4640 1 --2- 172.21.15.43:0/1140266612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e981095d0 0x7f9e98111760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.167+0000 7f9e9e3c4640 1 -- 172.21.15.43:0/1140266612 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e98112230 con 0x7f9e9811b140 2024-09-11T23:28:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.169+0000 7f9e9dbc3640 1 --2- 172.21.15.43:0/1140266612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e981091f0 0x7f9e981186e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:28:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.169+0000 7f9e9e3c4640 1 --2- 172.21.15.43:0/1140266612 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e9811b140 0x7f9e98111ca0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000eaa0 tx=0x7f9e9000ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.170+0000 7f9e8effd640 1 -- 172.21.15.43:0/1140266612 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9000cdb0 con 0x7f9e9811b140 2024-09-11T23:28:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.173+0000 7f9e9fe27640 1 -- 172.21.15.43:0/1140266612 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e98112520 con 0x7f9e9811b140 2024-09-11T23:28:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.173+0000 7f9e9fe27640 1 -- 172.21.15.43:0/1140266612 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e98116520 con 0x7f9e9811b140 2024-09-11T23:28:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.176+0000 7f9ea2bc3740 1 Processor -- start 2024-09-11T23:28:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.176+0000 7f9e8effd640 1 -- 172.21.15.43:0/1140266612 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e90005020 con 0x7f9e9811b140 2024-09-11T23:28:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.176+0000 7f9e8effd640 1 -- 172.21.15.43:0/1140266612 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e900116b0 con 0x7f9e9811b140 2024-09-11T23:28:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.185+0000 7f9ea2bc3740 1 -- start start 2024-09-11T23:28:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.185+0000 7f9ea2bc3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b6caf0 0x562010b6ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.185+0000 7f9ea2bc3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010b6d410 0x562010b77770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.186+0000 7f9ea2bc3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562010b77cb0 0x562010b7a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.186+0000 7f9ea2bc3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562010ad12f0 con 0x562010b6caf0 2024-09-11T23:28:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.186+0000 7f9ea2bc3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562010a79270 con 0x562010b77cb0 2024-09-11T23:28:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.186+0000 7f9ea2bc3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562010acd2b0 con 0x562010b6d410 2024-09-11T23:28:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.186+0000 7f9e9d3c2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b6caf0 0x562010b6ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.186+0000 7f9e9d3c2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b6caf0 0x562010b6ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36254/0 (socket says 172.21.15.43:36254) 2024-09-11T23:28:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.186+0000 7f9e9d3c2640 1 -- 172.21.15.43:0/1431381896 learned_addr learned my addr 172.21.15.43:0/1431381896 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.186+0000 7f9e9d3c2640 1 --2- 172.21.15.43:0/1431381896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562010b77cb0 0x562010b7a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.187+0000 7f9e9dbc3640 1 --2- 172.21.15.43:0/1431381896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010b6d410 0x562010b77770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.188+0000 7f9e9d3c2640 1 -- 172.21.15.43:0/1431381896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010b6d410 msgr2=0x562010b77770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.189+0000 7f9e9d3c2640 1 --2- 172.21.15.43:0/1431381896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010b6d410 0x562010b77770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.189+0000 7f9e9d3c2640 1 -- 172.21.15.43:0/1431381896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b6caf0 msgr2=0x562010b6ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.189+0000 7f9e9d3c2640 1 --2- 172.21.15.43:0/1431381896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b6caf0 0x562010b6ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.189+0000 7f9e9d3c2640 1 -- 172.21.15.43:0/1431381896 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562010a78340 con 0x562010b77cb0 2024-09-11T23:28:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.189+0000 7f9e9dbc3640 1 --2- 172.21.15.43:0/1431381896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010b6d410 0x562010b77770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:28:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.191+0000 7f9e8effd640 1 -- 172.21.15.43:0/1140266612 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9e900118d0 con 0x7f9e9811b140 2024-09-11T23:28:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.193+0000 7f9e8effd640 1 --2- 172.21.15.43:0/1140266612 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e840a2e40 0x7f9e840a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.193+0000 7f9e8effd640 1 -- 172.21.15.43:0/1140266612 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f9e900cf620 con 0x7f9e9811b140 2024-09-11T23:28:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.195+0000 7f9e9d3c2640 1 --2- 172.21.15.43:0/1431381896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562010b77cb0 0x562010b7a0a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9e94014400 tx=0x7f9e94004840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a12e95276dd6837 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.195+0000 7f9e9dbc3640 1 --2- 172.21.15.43:0/1140266612 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e840a2e40 0x7f9e840a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.198+0000 7f9e83fff640 1 -- 172.21.15.43:0/1431381896 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9401c070 con 0x562010b77cb0 2024-09-11T23:28:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.198+0000 7f9e83fff640 1 -- 172.21.15.43:0/1431381896 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e94012960 con 0x562010b77cb0 2024-09-11T23:28:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.199+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/1431381896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562010b77cb0 msgr2=0x562010b7a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.199+0000 7f9ea2bc3740 1 --2- 172.21.15.43:0/1431381896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562010b77cb0 0x562010b7a0a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9e94014400 tx=0x7f9e94004840 comp rx=0 tx=0).stop 2024-09-11T23:28:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.203+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/1431381896 shutdown_connections 2024-09-11T23:28:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.203+0000 7f9ea2bc3740 1 --2- 172.21.15.43:0/1431381896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562010b77cb0 0x562010b7a0a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.203+0000 7f9ea2bc3740 1 --2- 172.21.15.43:0/1431381896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010b6d410 0x562010b77770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.203+0000 7f9ea2bc3740 1 --2- 172.21.15.43:0/1431381896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b6caf0 0x562010b6ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.203+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/1431381896 >> 172.21.15.43:0/1431381896 conn(0x562010b69060 msgr2=0x562010b69440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.203+0000 7f9e9dbc3640 1 --2- 172.21.15.43:0/1140266612 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e840a2e40 0x7f9e840a5300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9e88000c00 tx=0x7f9e88005a00 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.206+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/1431381896 shutdown_connections 2024-09-11T23:28:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.208+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/1431381896 wait complete. 2024-09-11T23:28:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.208+0000 7f9ea2bc3740 1 -- start start 2024-09-11T23:28:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.208+0000 7f9ea2bc3740 1 Processor -- start 2024-09-11T23:28:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.209+0000 7f9ea2bc3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b6caf0 0x562010b74b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.209+0000 7f9ea2bc3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562010b6d410 0x562010b75040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.209+0000 7f9ea2bc3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010b77cb0 0x562010b75580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.209+0000 7f9ea2bc3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562010b7a8d0 con 0x562010b6caf0 2024-09-11T23:28:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.209+0000 7f9ea2bc3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562010a5a290 con 0x562010b6d410 2024-09-11T23:28:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.209+0000 7f9ea2bc3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562010b7aa50 con 0x562010b77cb0 2024-09-11T23:28:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.209+0000 7f9ea2bc3740 10 client.0 _ll_register_callbacks cb 0x562010a8b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.210+0000 7f9e9dbc3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b6caf0 0x562010b74b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.210+0000 7f9e9dbc3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b6caf0 0x562010b74b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36260/0 (socket says 172.21.15.43:36260) 2024-09-11T23:28:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.210+0000 7f9e9dbc3640 1 -- 172.21.15.43:0/4103346572 learned_addr learned my addr 172.21.15.43:0/4103346572 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.210+0000 7f9e9e3c4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010b77cb0 0x562010b75580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.212+0000 7f9e9d3c2640 1 --2- 172.21.15.43:0/4103346572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562010b6d410 0x562010b75040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.215+0000 7f9e9dbc3640 1 -- 172.21.15.43:0/4103346572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010b77cb0 msgr2=0x562010b75580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.215+0000 7f9e9dbc3640 1 --2- 172.21.15.43:0/4103346572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010b77cb0 0x562010b75580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.215+0000 7f9e9dbc3640 1 -- 172.21.15.43:0/4103346572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562010b6d410 msgr2=0x562010b75040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.215+0000 7f9e9dbc3640 1 --2- 172.21.15.43:0/4103346572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562010b6d410 0x562010b75040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.215+0000 7f9e9dbc3640 1 -- 172.21.15.43:0/4103346572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620107e4210 con 0x562010b6caf0 2024-09-11T23:28:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.223+0000 7f9e9dbc3640 1 --2- 172.21.15.43:0/4103346572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b6caf0 0x562010b74b00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9e88033040 tx=0x7f9e88009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe83fab51b8f585 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.224+0000 7f9e817fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:28:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.228+0000 7f9e817fa640 1 -- 172.21.15.43:0/4103346572 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e88005650 con 0x562010b6caf0 2024-09-11T23:28:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.228+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/4103346572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e94012960 con 0x562010b6caf0 2024-09-11T23:28:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.228+0000 7f9e817fa640 1 -- 172.21.15.43:0/4103346572 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f9e88046680 con 0x562010b6caf0 2024-09-11T23:28:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.232+0000 7f9e817fa640 1 -- 172.21.15.43:0/4103346572 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9e88046880 con 0x562010b6caf0 2024-09-11T23:28:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.234+0000 7f9ea2bc3740 10 client.16515 fetch_fsmap learned FSMap version 30 2024-09-11T23:28:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.234+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/4103346572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5620108b0620 con 0x562010b6caf0 2024-09-11T23:28:55.236 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.236+0000 7f9e817fa640 1 -- 172.21.15.43:0/4103346572 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e94012960 con 0x562010b6caf0 2024-09-11T23:28:55.237 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.237+0000 7f9e817fa640 1 -- 172.21.15.43:0/4103346572 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9e8804f020 con 0x562010b6caf0 2024-09-11T23:28:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.241+0000 7f9ea2bc3740 10 client.16515 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:28:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.241+0000 7f9ea2bc3740 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:28:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.241+0000 7f9ea2bc3740 20 client.16515 populate_metadata read hostname 'smithi043' 2024-09-11T23:28:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.241+0000 7f9ea2bc3740 1 --2- 172.21.15.43:0/4103346572 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562010ba9bd0 0x562010babfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.241+0000 7f9ea2bc3740 4 client.16515 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:28:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.241+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/4103346572 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562010adea20 con 0x562010ba9bd0 2024-09-11T23:28:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.245+0000 7f9e9d3c2640 1 --2- 172.21.15.43:0/4103346572 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562010ba9bd0 0x562010babfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.255+0000 7f9e9d3c2640 1 --2- 172.21.15.43:0/4103346572 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562010ba9bd0 0x562010babfc0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35158f79e0a7a104 server_cookie=8dfd34b59d2c7a27 in_seq=0 out_seq=0 2024-09-11T23:28:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.258+0000 7f9e817fa640 10 client.16515 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.258+0000 7f9e817fa640 1 -- 172.21.15.43:0/4103346572 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562010adea20 con 0x562010ba9bd0 2024-09-11T23:28:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.258+0000 7f9e817fa640 10 client.16515 handle_command_reply: tid=0 2024-09-11T23:28:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.258+0000 7f9ea2bc3740 1 client.16515 shutdown 2024-09-11T23:28:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.259+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/4103346572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b6caf0 msgr2=0x562010b74b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.259+0000 7f9ea2bc3740 1 --2- 172.21.15.43:0/4103346572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b6caf0 0x562010b74b00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9e88033040 tx=0x7f9e88009260 comp rx=0 tx=0).stop 2024-09-11T23:28:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.260+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/4103346572 shutdown_connections 2024-09-11T23:28:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.260+0000 7f9ea2bc3740 1 --2- 172.21.15.43:0/4103346572 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562010ba9bd0 0x562010babfc0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.260+0000 7f9ea2bc3740 1 --2- 172.21.15.43:0/4103346572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010b77cb0 0x562010b75580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.260+0000 7f9ea2bc3740 1 --2- 172.21.15.43:0/4103346572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562010b6d410 0x562010b75040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.260+0000 7f9ea2bc3740 1 --2- 172.21.15.43:0/4103346572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b6caf0 0x562010b74b00 secure :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9e88033040 tx=0x7f9e88009260 comp rx=0 tx=0).stop 2024-09-11T23:28:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.260+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/4103346572 >> 172.21.15.43:0/4103346572 conn(0x562010b69060 msgr2=0x562010b6edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.260+0000 7f9e817fa640 0 client.16515 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.260+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/4103346572 shutdown_connections 2024-09-11T23:28:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.261+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/4103346572 wait complete. 2024-09-11T23:28:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.262+0000 7f9ea2bc3740 20 client.16515 trim_cache size 0 max 16384 2024-09-11T23:28:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.300+0000 7f9ea2bc3740 1 Processor -- start 2024-09-11T23:28:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.301+0000 7f9ea2bc3740 1 -- start start 2024-09-11T23:28:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.301+0000 7f9ea2bc3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b6d410 0x562010b753b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.301+0000 7f9ea2bc3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562010b77cb0 0x562010bb18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.301+0000 7f9ea2bc3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010bb1e10 0x562010b71d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.301+0000 7f9ea2bc3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c0111b0 con 0x562010b6d410 2024-09-11T23:28:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.301+0000 7f9ea2bc3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c00ea80 con 0x562010b77cb0 2024-09-11T23:28:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.301+0000 7f9ea2bc3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c00c8d0 con 0x562010bb1e10 2024-09-11T23:28:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.301+0000 7f9e9e3c4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010bb1e10 0x562010b71d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.301+0000 7f9e9d3c2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562010b77cb0 0x562010bb18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.302+0000 7f9e9e3c4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010bb1e10 0x562010b71d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:48676/0 (socket says 172.21.15.43:48676) 2024-09-11T23:28:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.302+0000 7f9e9e3c4640 1 -- 172.21.15.43:0/649155463 learned_addr learned my addr 172.21.15.43:0/649155463 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.302+0000 7f9e9dbc3640 1 --2- 172.21.15.43:0/649155463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b6d410 0x562010b753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.303+0000 7f9e9e3c4640 1 -- 172.21.15.43:0/649155463 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562010b77cb0 msgr2=0x562010bb18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.303+0000 7f9e9e3c4640 1 --2- 172.21.15.43:0/649155463 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562010b77cb0 0x562010bb18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.303+0000 7f9e9e3c4640 1 -- 172.21.15.43:0/649155463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b6d410 msgr2=0x562010b753b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.303+0000 7f9e9e3c4640 1 --2- 172.21.15.43:0/649155463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b6d410 0x562010b753b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.303+0000 7f9e9e3c4640 1 -- 172.21.15.43:0/649155463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562010b68e90 con 0x562010bb1e10 2024-09-11T23:28:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.304+0000 7f9e9e3c4640 1 --2- 172.21.15.43:0/649155463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010bb1e10 0x562010b71d40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9e9009b7e0 tx=0x7f9e9009b570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=236f44ee2e705e71 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.305+0000 7f9e837fe640 1 -- 172.21.15.43:0/649155463 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e900b3070 con 0x562010bb1e10 2024-09-11T23:28:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.305+0000 7f9e837fe640 1 -- 172.21.15.43:0/649155463 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e900cd190 con 0x562010bb1e10 2024-09-11T23:28:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.305+0000 7f9e837fe640 1 -- 172.21.15.43:0/649155463 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9001f040 con 0x562010bb1e10 2024-09-11T23:28:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.307+0000 7f9e9dbc3640 1 --2- 172.21.15.43:0/649155463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b6d410 0x562010b753b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:28:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.307+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/649155463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010bb1e10 msgr2=0x562010b71d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.307+0000 7f9ea2bc3740 1 --2- 172.21.15.43:0/649155463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010bb1e10 0x562010b71d40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9e9009b7e0 tx=0x7f9e9009b570 comp rx=0 tx=0).stop 2024-09-11T23:28:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.309+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/649155463 shutdown_connections 2024-09-11T23:28:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.309+0000 7f9ea2bc3740 1 --2- 172.21.15.43:0/649155463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010bb1e10 0x562010b71d40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.309+0000 7f9ea2bc3740 1 --2- 172.21.15.43:0/649155463 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562010b77cb0 0x562010bb18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.309+0000 7f9ea2bc3740 1 --2- 172.21.15.43:0/649155463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b6d410 0x562010b753b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.309+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/649155463 >> 172.21.15.43:0/649155463 conn(0x562010b69060 msgr2=0x562010b76b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.313+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/649155463 shutdown_connections 2024-09-11T23:28:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.313+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/649155463 wait complete. 2024-09-11T23:28:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.314+0000 7f9ea2bc3740 1 -- start start 2024-09-11T23:28:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.314+0000 7f9ea2bc3740 1 Processor -- start 2024-09-11T23:28:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.314+0000 7f9ea2bc3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562010b6d410 0x562010b7ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.314+0000 7f9ea2bc3740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b77cb0 0x562010b804d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.314+0000 7f9ea2bc3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010bb1e10 0x562010b7ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.314+0000 7f9ea2bc3740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c00cf40 con 0x562010b77cb0 2024-09-11T23:28:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.314+0000 7f9ea2bc3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c00e3b0 con 0x562010b6d410 2024-09-11T23:28:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.314+0000 7f9ea2bc3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c00d610 con 0x562010bb1e10 2024-09-11T23:28:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.314+0000 7f9ea2bc3740 10 client.0 _ll_register_callbacks cb 0x7f9e880037c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:28:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.315+0000 7f9e9e3c4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010bb1e10 0x562010b7ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.315+0000 7f9e9e3c4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010bb1e10 0x562010b7ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:48690/0 (socket says 172.21.15.43:48690) 2024-09-11T23:28:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.315+0000 7f9e9e3c4640 1 -- 172.21.15.43:0/2617197914 learned_addr learned my addr 172.21.15.43:0/2617197914 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:28:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.315+0000 7f9e9dbc3640 1 --2- 172.21.15.43:0/2617197914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562010b6d410 0x562010b7ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.316+0000 7f9e9e3c4640 1 -- 172.21.15.43:0/2617197914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562010b6d410 msgr2=0x562010b7ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.316+0000 7f9e9e3c4640 1 --2- 172.21.15.43:0/2617197914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562010b6d410 0x562010b7ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.316+0000 7f9e9e3c4640 1 -- 172.21.15.43:0/2617197914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b77cb0 msgr2=0x562010b804d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:28:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.316+0000 7f9e9e3c4640 1 --2- 172.21.15.43:0/2617197914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b77cb0 0x562010b804d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.316+0000 7f9e9e3c4640 1 -- 172.21.15.43:0/2617197914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562010b642d0 con 0x562010bb1e10 2024-09-11T23:28:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.319+0000 7f9e9e3c4640 1 --2- 172.21.15.43:0/2617197914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010bb1e10 0x562010b7ad50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9e90005a70 tx=0x7f9e9000c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38fd9d92c0f28340 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:28:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.319+0000 7f9e817fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-11T23:28:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.319+0000 7f9e817fa640 1 -- 172.21.15.43:0/2617197914 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e900b3070 con 0x562010bb1e10 2024-09-11T23:28:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.319+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/2617197914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e900cd190 con 0x562010bb1e10 2024-09-11T23:28:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.319+0000 7f9e817fa640 1 -- 172.21.15.43:0/2617197914 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f9e900ccf90 con 0x562010bb1e10 2024-09-11T23:28:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.319+0000 7f9e817fa640 1 -- 172.21.15.43:0/2617197914 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9e9001f050 con 0x562010bb1e10 2024-09-11T23:28:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.321+0000 7f9ea2bc3740 10 client.25981 fetch_fsmap learned FSMap version 30 2024-09-11T23:28:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.321+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/2617197914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562010c15110 con 0x562010bb1e10 2024-09-11T23:28:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.326+0000 7f9e817fa640 1 -- 172.21.15.43:0/2617197914 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e900cd190 con 0x562010bb1e10 2024-09-11T23:28:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.326+0000 7f9e817fa640 1 -- 172.21.15.43:0/2617197914 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9e900b4ce0 con 0x562010bb1e10 2024-09-11T23:28:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.327+0000 7f9ea2bc3740 10 client.25981 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:28:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.327+0000 7f9ea2bc3740 10 client.25981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:28:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.327+0000 7f9ea2bc3740 20 client.25981 populate_metadata read hostname 'smithi043' 2024-09-11T23:28:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.327+0000 7f9ea2bc3740 1 --2- 172.21.15.43:0/2617197914 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562010c15760 0x562010c17b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:28:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.327+0000 7f9ea2bc3740 4 client.25981 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:28:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.327+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/2617197914 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562010ad8d70 con 0x562010c15760 2024-09-11T23:28:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.329+0000 7f9e9dbc3640 1 --2- 172.21.15.43:0/2617197914 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562010c15760 0x562010c17b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:28:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.338+0000 7f9e9dbc3640 1 --2- 172.21.15.43:0/2617197914 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562010c15760 0x562010c17b80 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25e8c87813e004df server_cookie=a253af6e3ef8b37e in_seq=0 out_seq=0 2024-09-11T23:28:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:55.338+0000 7f9e817fa640 10 client.25981 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:56.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:56 smithi090 ceph-mon[35130]: pgmap v216: 97 pgs: 97 active+clean; 22 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 66 KiB/s rd, 9.6 MiB/s wr, 1.75k op/s 2024-09-11T23:28:56.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:56 smithi097 ceph-mon[35562]: pgmap v216: 97 pgs: 97 active+clean; 22 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 66 KiB/s rd, 9.6 MiB/s wr, 1.75k op/s 2024-09-11T23:28:56.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:56 smithi043 ceph-mon[30950]: pgmap v216: 97 pgs: 97 active+clean; 22 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 66 KiB/s rd, 9.6 MiB/s wr, 1.75k op/s 2024-09-11T23:28:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.519+0000 7f9e817fa640 1 -- 172.21.15.43:0/2617197914 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562010ad8d70 con 0x562010c15760 2024-09-11T23:28:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.519+0000 7f9e817fa640 10 client.25981 handle_command_reply: tid=0 2024-09-11T23:28:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.519+0000 7f9ea2bc3740 1 client.25981 shutdown 2024-09-11T23:28:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.519+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/2617197914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010bb1e10 msgr2=0x562010b7ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.519+0000 7f9ea2bc3740 1 --2- 172.21.15.43:0/2617197914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010bb1e10 0x562010b7ad50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9e90005a70 tx=0x7f9e9000c870 comp rx=0 tx=0).stop 2024-09-11T23:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.519+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/2617197914 shutdown_connections 2024-09-11T23:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.519+0000 7f9ea2bc3740 1 --2- 172.21.15.43:0/2617197914 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562010c15760 0x562010c17b80 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.519+0000 7f9ea2bc3740 1 --2- 172.21.15.43:0/2617197914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562010bb1e10 0x562010b7ad50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.519+0000 7f9ea2bc3740 1 --2- 172.21.15.43:0/2617197914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562010b77cb0 0x562010b804d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.519+0000 7f9e817fa640 0 client.25981 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.519+0000 7f9ea2bc3740 1 --2- 172.21.15.43:0/2617197914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562010b6d410 0x562010b7ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.519+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/2617197914 >> 172.21.15.43:0/2617197914 conn(0x562010b69060 msgr2=0x562010b679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.519+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/2617197914 shutdown_connections 2024-09-11T23:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.519+0000 7f9ea2bc3740 1 -- 172.21.15.43:0/2617197914 wait complete. 2024-09-11T23:28:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.520+0000 7f9ea2bc3740 20 client.25981 trim_cache size 0 max 16384 2024-09-11T23:28:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:28:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:28:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:28:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.520+0000 7f9e9fe27640 1 -- 172.21.15.43:0/1140266612 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e840a2e40 msgr2=0x7f9e840a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.520+0000 7f9e9fe27640 1 --2- 172.21.15.43:0/1140266612 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e840a2e40 0x7f9e840a5300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9e88000c00 tx=0x7f9e88005a00 comp rx=0 tx=0).stop 2024-09-11T23:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.520+0000 7f9e9fe27640 1 -- 172.21.15.43:0/1140266612 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e9811b140 msgr2=0x7f9e98111ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.520+0000 7f9e9fe27640 1 --2- 172.21.15.43:0/1140266612 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e9811b140 0x7f9e98111ca0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000eaa0 tx=0x7f9e9000ef60 comp rx=0 tx=0).stop 2024-09-11T23:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.520+0000 7f9e9fe27640 1 -- 172.21.15.43:0/1140266612 shutdown_connections 2024-09-11T23:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.520+0000 7f9e9fe27640 1 --2- 172.21.15.43:0/1140266612 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e840a2e40 0x7f9e840a5300 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.520+0000 7f9e9fe27640 1 --2- 172.21.15.43:0/1140266612 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e9811b140 0x7f9e98111ca0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.520+0000 7f9e9fe27640 1 --2- 172.21.15.43:0/1140266612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e981095d0 0x7f9e98111760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.520+0000 7f9e9fe27640 1 --2- 172.21.15.43:0/1140266612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e981091f0 0x7f9e981186e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:28:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.520+0000 7f9e9fe27640 1 -- 172.21.15.43:0/1140266612 >> 172.21.15.43:0/1140266612 conn(0x7f9e9806cdf0 msgr2=0x7f9e9811b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:28:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.520+0000 7f9e9fe27640 1 -- 172.21.15.43:0/1140266612 shutdown_connections 2024-09-11T23:28:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:28:57.520+0000 7f9e9fe27640 1 -- 172.21.15.43:0/1140266612 wait complete. 2024-09-11T23:28:58.138 INFO:tasks.workunit.client.0.smithi043.stdout:Syncing()...5 sec 2024-09-11T23:28:58.138 INFO:tasks.workunit.client.0.smithi043.stdout:FFSB benchmark finished at: Wed Sep 11 23:28:58 2024 2024-09-11T23:28:58.139 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-11T23:28:58.139 INFO:tasks.workunit.client.0.smithi043.stdout:Results: 2024-09-11T23:28:58.139 INFO:tasks.workunit.client.0.smithi043.stdout:Benchmark took 315.35 sec 2024-09-11T23:28:58.139 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-11T23:28:58.139 INFO:tasks.workunit.client.0.smithi043.stdout:Total Results 2024-09-11T23:28:58.139 INFO:tasks.workunit.client.0.smithi043.stdout:=============== 2024-09-11T23:28:58.139 INFO:tasks.workunit.client.0.smithi043.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-09-11T23:28:58.140 INFO:tasks.workunit.client.0.smithi043.stdout: ======= ============ ========= ======= =========== ========== 2024-09-11T23:28:58.140 INFO:tasks.workunit.client.0.smithi043.stdout: write : 6270720 19885.14 100.000% 100.000% 77.7MB/sec 2024-09-11T23:28:58.140 INFO:tasks.workunit.client.0.smithi043.stdout:- 2024-09-11T23:28:58.140 INFO:tasks.workunit.client.0.smithi043.stdout:19885.14 Transactions per Second 2024-09-11T23:28:58.140 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-11T23:28:58.140 INFO:tasks.workunit.client.0.smithi043.stdout:Throughput Results 2024-09-11T23:28:58.141 INFO:tasks.workunit.client.0.smithi043.stdout:=================== 2024-09-11T23:28:58.141 INFO:tasks.workunit.client.0.smithi043.stdout:Write Throughput: 77.7MB/sec 2024-09-11T23:28:58.169 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-11T23:28:58.170 INFO:tasks.workunit.client.0.smithi043.stdout:System Call Latency statistics in millisecs 2024-09-11T23:28:58.170 INFO:tasks.workunit.client.0.smithi043.stdout:===== 2024-09-11T23:28:58.170 INFO:tasks.workunit.client.0.smithi043.stdout: Min Avg Max Total Calls 2024-09-11T23:28:58.170 INFO:tasks.workunit.client.0.smithi043.stdout: ======== ======== ======== ============ 2024-09-11T23:28:58.170 INFO:tasks.workunit.client.0.smithi043.stdout:[ open] 0.005000 0.751465 47.119999 4899 2024-09-11T23:28:58.170 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-11T23:28:58.170 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-11T23:28:58.170 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-11T23:28:58.170 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-11T23:28:58.170 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-11T23:28:58.170 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-09-11T23:28:58.171 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-09-11T23:28:58.171 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[7] 20.000000 - 50.000000 : 68 2024-09-11T23:28:58.171 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[8] 10.000000 - 20.000000 : 83 2024-09-11T23:28:58.171 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[9] 5.000000 - 10.000000 : 45 2024-09-11T23:28:58.171 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[10] 2.000000 - 5.000000 : 48 2024-09-11T23:28:58.171 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[11] 1.000000 - 2.000000 : 30 2024-09-11T23:28:58.171 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[12] 0.500000 - 1.000000 : 13 2024-09-11T23:28:58.171 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[13] 0.200000 - 0.500000 : 5 2024-09-11T23:28:58.171 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[14] 0.100000 - 0.200000 : 13 2024-09-11T23:28:58.171 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[15] 0.050000 - 0.100000 : 29 2024-09-11T23:28:58.171 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[16] 0.020000 - 0.050000 : 1009 2024-09-11T23:28:58.171 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[17] 0.010000 - 0.020000 : 2727 2024-09-11T23:28:58.171 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[18] 0.000000 - 0.010000 : 829 2024-09-11T23:28:58.171 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-11T23:28:58.172 INFO:tasks.workunit.client.0.smithi043.stdout:[ write] 0.001000 1.579658 127505.281250 6270720 2024-09-11T23:28:58.172 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[0] 5000.000000 - 10000.000000 : 99 2024-09-11T23:28:58.172 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[1] 2000.000000 - 5000.000000 : 176 2024-09-11T23:28:58.172 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[2] 1000.000000 - 2000.000000 : 179 2024-09-11T23:28:58.172 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[3] 500.000000 - 1000.000000 : 210 2024-09-11T23:28:58.172 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[4] 200.000000 - 500.000000 : 297 2024-09-11T23:28:58.172 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[5] 100.000000 - 200.000000 : 224 2024-09-11T23:28:58.172 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[6] 50.000000 - 100.000000 : 386 2024-09-11T23:28:58.172 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[7] 20.000000 - 50.000000 : 1760 2024-09-11T23:28:58.172 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[8] 10.000000 - 20.000000 : 2193 2024-09-11T23:28:58.172 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[9] 5.000000 - 10.000000 : 1169 2024-09-11T23:28:58.172 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[10] 2.000000 - 5.000000 : 1525 2024-09-11T23:28:58.172 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[11] 1.000000 - 2.000000 : 893 2024-09-11T23:28:58.173 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[12] 0.500000 - 1.000000 : 643 2024-09-11T23:28:58.173 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[13] 0.200000 - 0.500000 : 663 2024-09-11T23:28:58.173 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[14] 0.100000 - 0.200000 : 986 2024-09-11T23:28:58.173 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[15] 0.050000 - 0.100000 : 2958 2024-09-11T23:28:58.173 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[16] 0.020000 - 0.050000 : 13627 2024-09-11T23:28:58.173 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[17] 0.010000 - 0.020000 : 28880 2024-09-11T23:28:58.173 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[18] 0.000000 - 0.010000 : 6213678 2024-09-11T23:28:58.173 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-11T23:28:58.173 INFO:tasks.workunit.client.0.smithi043.stdout:[ lseek] 0.000000 0.001563 68.267998 6270720 2024-09-11T23:28:58.173 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-11T23:28:58.173 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-11T23:28:58.173 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-11T23:28:58.174 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-11T23:28:58.174 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-11T23:28:58.174 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-09-11T23:28:58.174 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[6] 50.000000 - 100.000000 : 4 2024-09-11T23:28:58.174 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[7] 20.000000 - 50.000000 : 94 2024-09-11T23:28:58.174 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[8] 10.000000 - 20.000000 : 109 2024-09-11T23:28:58.174 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[9] 5.000000 - 10.000000 : 91 2024-09-11T23:28:58.174 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[10] 2.000000 - 5.000000 : 118 2024-09-11T23:28:58.174 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[11] 1.000000 - 2.000000 : 85 2024-09-11T23:28:58.174 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[12] 0.500000 - 1.000000 : 69 2024-09-11T23:28:58.174 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[13] 0.200000 - 0.500000 : 89 2024-09-11T23:28:58.174 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[14] 0.100000 - 0.200000 : 101 2024-09-11T23:28:58.174 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[15] 0.050000 - 0.100000 : 266 2024-09-11T23:28:58.174 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[16] 0.020000 - 0.050000 : 1149 2024-09-11T23:28:58.174 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[17] 0.010000 - 0.020000 : 2320 2024-09-11T23:28:58.175 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[18] 0.000000 - 0.010000 : 6266225 2024-09-11T23:28:58.175 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-11T23:28:58.175 INFO:tasks.workunit.client.0.smithi043.stdout:[ close] 0.001000 0.008438 12.222000 4899 2024-09-11T23:28:58.175 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-11T23:28:58.175 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-11T23:28:58.175 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-11T23:28:58.175 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-11T23:28:58.175 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-11T23:28:58.175 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-09-11T23:28:58.175 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-09-11T23:28:58.175 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2024-09-11T23:28:58.175 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[8] 10.000000 - 20.000000 : 1 2024-09-11T23:28:58.175 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[9] 5.000000 - 10.000000 : 0 2024-09-11T23:28:58.175 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[10] 2.000000 - 5.000000 : 2 2024-09-11T23:28:58.175 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[11] 1.000000 - 2.000000 : 0 2024-09-11T23:28:58.176 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[12] 0.500000 - 1.000000 : 0 2024-09-11T23:28:58.176 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[13] 0.200000 - 0.500000 : 0 2024-09-11T23:28:58.176 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[14] 0.100000 - 0.200000 : 0 2024-09-11T23:28:58.176 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[15] 0.050000 - 0.100000 : 1 2024-09-11T23:28:58.176 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[16] 0.020000 - 0.050000 : 9 2024-09-11T23:28:58.176 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[17] 0.010000 - 0.020000 : 19 2024-09-11T23:28:58.176 INFO:tasks.workunit.client.0.smithi043.stdout: msec_range[18] 0.000000 - 0.010000 : 4867 2024-09-11T23:28:58.176 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-11T23:28:58.176 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-11T23:28:58.176 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-11T23:28:58.176 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-09-11T23:28:58.176 INFO:tasks.workunit.client.0.smithi043.stdout:1.9% User Time 2024-09-11T23:28:58.176 INFO:tasks.workunit.client.0.smithi043.stdout:6.6% System Time 2024-09-11T23:28:58.177 INFO:tasks.workunit.client.0.smithi043.stdout:8.6% CPU Utilization 2024-09-11T23:28:58.177 INFO:tasks.workunit.client.0.smithi043.stderr:+ cd .. 2024-09-11T23:28:58.180 INFO:tasks.workunit.client.0.smithi043.stderr:+ rm -r tmp ffsb 2024-09-11T23:28:58.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:28:58 smithi090 ceph-mon[35130]: pgmap v217: 97 pgs: 97 active+clean; 22 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 51 KiB/s rd, 6.6 MiB/s wr, 1.46k op/s 2024-09-11T23:28:58.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:28:58 smithi097 ceph-mon[35562]: pgmap v217: 97 pgs: 97 active+clean; 22 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 51 KiB/s rd, 6.6 MiB/s wr, 1.46k op/s 2024-09-11T23:28:58.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:28:58 smithi043 ceph-mon[30950]: pgmap v217: 97 pgs: 97 active+clean; 22 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 51 KiB/s rd, 6.6 MiB/s wr, 1.46k op/s 2024-09-11T23:28:59.662 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '733f7a10', total seconds elapsed: 88.1 2024-09-11T23:28:59.663 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-11T23:28:59.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:28:59.789 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:29:00.232 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:00 smithi043 ceph-mon[30950]: pgmap v218: 97 pgs: 97 active+clean; 22 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 51 KiB/s rd, 9.0 MiB/s wr, 2.03k op/s 2024-09-11T23:29:00.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.324+0000 7f9e71509640 1 -- 172.21.15.43:0/2480240712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c101420 msgr2=0x7f9e6c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:00.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.324+0000 7f9e71509640 1 --2- 172.21.15.43:0/2480240712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c101420 0x7f9e6c10d950 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009cd0 tx=0x7f9e6402a8e0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.324+0000 7f9e71509640 1 -- 172.21.15.43:0/2480240712 shutdown_connections 2024-09-11T23:29:00.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.324+0000 7f9e71509640 1 --2- 172.21.15.43:0/2480240712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c101420 0x7f9e6c10d950 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.324+0000 7f9e71509640 1 --2- 172.21.15.43:0/2480240712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e6c100a80 0x7f9e6c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.324+0000 7f9e71509640 1 --2- 172.21.15.43:0/2480240712 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e6c106a80 0x7f9e6c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.324+0000 7f9e71509640 1 -- 172.21.15.43:0/2480240712 >> 172.21.15.43:0/2480240712 conn(0x7f9e6c0fc7a0 msgr2=0x7f9e6c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:00.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.324+0000 7f9e71509640 1 -- 172.21.15.43:0/2480240712 shutdown_connections 2024-09-11T23:29:00.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.324+0000 7f9e71509640 1 -- 172.21.15.43:0/2480240712 wait complete. 2024-09-11T23:29:00.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.325+0000 7f9e71509640 1 Processor -- start 2024-09-11T23:29:00.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.325+0000 7f9e71509640 1 -- start start 2024-09-11T23:29:00.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.325+0000 7f9e71509640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c100a80 0x7f9e6c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:00.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.326+0000 7f9e71509640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e6c101420 0x7f9e6c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:00.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.326+0000 7f9e71509640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e6c106a80 0x7f9e6c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:00.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.326+0000 7f9e71509640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c110ed0 con 0x7f9e6c100a80 2024-09-11T23:29:00.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.326+0000 7f9e71509640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c110d50 con 0x7f9e6c106a80 2024-09-11T23:29:00.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.326+0000 7f9e71509640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c111050 con 0x7f9e6c101420 2024-09-11T23:29:00.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.326+0000 7f9e6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c100a80 0x7f9e6c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:00.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.326+0000 7f9e6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c100a80 0x7f9e6c198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58868/0 (socket says 172.21.15.43:58868) 2024-09-11T23:29:00.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.326+0000 7f9e6affd640 1 -- 172.21.15.43:0/3770864027 learned_addr learned my addr 172.21.15.43:0/3770864027 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:00.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.326+0000 7f9e6a7fc640 1 --2- 172.21.15.43:0/3770864027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e6c101420 0x7f9e6c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:00.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.326+0000 7f9e6b7fe640 1 --2- 172.21.15.43:0/3770864027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e6c106a80 0x7f9e6c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:00.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.326+0000 7f9e6affd640 1 -- 172.21.15.43:0/3770864027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e6c101420 msgr2=0x7f9e6c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:00.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.326+0000 7f9e6affd640 1 --2- 172.21.15.43:0/3770864027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e6c101420 0x7f9e6c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.326+0000 7f9e6affd640 1 -- 172.21.15.43:0/3770864027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e6c106a80 msgr2=0x7f9e6c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:00.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.326+0000 7f9e6affd640 1 --2- 172.21.15.43:0/3770864027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e6c106a80 0x7f9e6c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.326+0000 7f9e6affd640 1 -- 172.21.15.43:0/3770864027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6c19d3d0 con 0x7f9e6c100a80 2024-09-11T23:29:00.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.326+0000 7f9e6affd640 1 --2- 172.21.15.43:0/3770864027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c100a80 0x7f9e6c198420 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00e9f0 tx=0x7f9e5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:00.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.326+0000 7f9e53fff640 1 -- 172.21.15.43:0/3770864027 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c00cd80 con 0x7f9e6c100a80 2024-09-11T23:29:00.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.326+0000 7f9e71509640 1 -- 172.21.15.43:0/3770864027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e6c19d6c0 con 0x7f9e6c100a80 2024-09-11T23:29:00.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.326+0000 7f9e53fff640 1 -- 172.21.15.43:0/3770864027 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e5c010910 con 0x7f9e6c100a80 2024-09-11T23:29:00.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.327+0000 7f9e53fff640 1 -- 172.21.15.43:0/3770864027 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c018990 con 0x7f9e6c100a80 2024-09-11T23:29:00.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.327+0000 7f9e71509640 1 -- 172.21.15.43:0/3770864027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e6c1a4fa0 con 0x7f9e6c100a80 2024-09-11T23:29:00.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.329+0000 7f9e738e2740 1 Processor -- start 2024-09-11T23:29:00.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.329+0000 7f9e738e2740 1 -- start start 2024-09-11T23:29:00.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.330+0000 7f9e738e2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9dd4a0 0x561ccf9dd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:00.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.330+0000 7f9e738e2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dddc0 0x561ccf9e8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:00.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.330+0000 7f9e6b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9dd4a0 0x561ccf9dd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:00.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.330+0000 7f9e6b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9dd4a0 0x561ccf9dd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:40924/0 (socket says 172.21.15.43:40924) 2024-09-11T23:29:00.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.330+0000 7f9e738e2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9e8680 0x561ccf9eaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:00.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.330+0000 7f9e6b7fe640 1 -- 172.21.15.43:0/4029677605 learned_addr learned my addr 172.21.15.43:0/4029677605 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:00.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.330+0000 7f9e738e2740 1 -- 172.21.15.43:0/4029677605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561ccf9432e0 con 0x561ccf9dddc0 2024-09-11T23:29:00.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.330+0000 7f9e738e2740 1 -- 172.21.15.43:0/4029677605 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561ccf8eace0 con 0x561ccf9e8680 2024-09-11T23:29:00.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.330+0000 7f9e738e2740 1 -- 172.21.15.43:0/4029677605 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561ccf93f2a0 con 0x561ccf9dd4a0 2024-09-11T23:29:00.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.330+0000 7f9e53fff640 1 -- 172.21.15.43:0/3770864027 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9e5c010ac0 con 0x7f9e6c100a80 2024-09-11T23:29:00.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.330+0000 7f9e6a7fc640 1 --2- 172.21.15.43:0/4029677605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9e8680 0x561ccf9eaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:00.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.330+0000 7f9e6affd640 1 --2- 172.21.15.43:0/4029677605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dddc0 0x561ccf9e8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:00.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.331+0000 7f9e6b7fe640 1 -- 172.21.15.43:0/4029677605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9e8680 msgr2=0x561ccf9eaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:00.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.331+0000 7f9e6b7fe640 1 --2- 172.21.15.43:0/4029677605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9e8680 0x561ccf9eaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.331+0000 7f9e6b7fe640 1 -- 172.21.15.43:0/4029677605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dddc0 msgr2=0x561ccf9e8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:00.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.331+0000 7f9e6b7fe640 1 --2- 172.21.15.43:0/4029677605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dddc0 0x561ccf9e8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.331+0000 7f9e6b7fe640 1 -- 172.21.15.43:0/4029677605 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ccf8cf440 con 0x561ccf9dd4a0 2024-09-11T23:29:00.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.331+0000 7f9e6affd640 1 --2- 172.21.15.43:0/4029677605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dddc0 0x561ccf9e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:29:00.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.331+0000 7f9e6b7fe640 1 --2- 172.21.15.43:0/4029677605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9dd4a0 0x561ccf9dd880 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009cd0 tx=0x7f9e64009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=442763f2db89481a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:00.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.331+0000 7f9e517fa640 1 -- 172.21.15.43:0/4029677605 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64049070 con 0x561ccf9dd4a0 2024-09-11T23:29:00.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.331+0000 7f9e53fff640 1 --2- 172.21.15.43:0/3770864027 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e540a2cf0 0x7f9e540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:00.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.331+0000 7f9e517fa640 1 -- 172.21.15.43:0/4029677605 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e64036040 con 0x561ccf9dd4a0 2024-09-11T23:29:00.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.331+0000 7f9e517fa640 1 -- 172.21.15.43:0/4029677605 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64004260 con 0x561ccf9dd4a0 2024-09-11T23:29:00.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.332+0000 7f9e53fff640 1 -- 172.21.15.43:0/3770864027 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f9e5c0cf6e0 con 0x7f9e6c100a80 2024-09-11T23:29:00.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.332+0000 7f9e738e2740 1 -- 172.21.15.43:0/4029677605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9dd4a0 msgr2=0x561ccf9dd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:00.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.333+0000 7f9e738e2740 1 --2- 172.21.15.43:0/4029677605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9dd4a0 0x561ccf9dd880 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009cd0 tx=0x7f9e64009a20 comp rx=0 tx=0).stop 2024-09-11T23:29:00.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.332+0000 7f9e6affd640 1 --2- 172.21.15.43:0/3770864027 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e540a2cf0 0x7f9e540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:00.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.333+0000 7f9e738e2740 1 -- 172.21.15.43:0/4029677605 shutdown_connections 2024-09-11T23:29:00.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.333+0000 7f9e738e2740 1 --2- 172.21.15.43:0/4029677605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9e8680 0x561ccf9eaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.333+0000 7f9e738e2740 1 --2- 172.21.15.43:0/4029677605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dddc0 0x561ccf9e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.333+0000 7f9e738e2740 1 --2- 172.21.15.43:0/4029677605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9dd4a0 0x561ccf9dd880 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.333+0000 7f9e738e2740 1 -- 172.21.15.43:0/4029677605 >> 172.21.15.43:0/4029677605 conn(0x561ccf9d9a30 msgr2=0x561ccf9d9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:00.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.333+0000 7f9e738e2740 1 -- 172.21.15.43:0/4029677605 shutdown_connections 2024-09-11T23:29:00.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.333+0000 7f9e738e2740 1 -- 172.21.15.43:0/4029677605 wait complete. 2024-09-11T23:29:00.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.333+0000 7f9e6affd640 1 --2- 172.21.15.43:0/3770864027 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e540a2cf0 0x7f9e540a51b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0ccee0 tx=0x7f9e5c000bc0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:00.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.333+0000 7f9e738e2740 1 -- start start 2024-09-11T23:29:00.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.333+0000 7f9e738e2740 1 Processor -- start 2024-09-11T23:29:00.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.334+0000 7f9e738e2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dd4a0 0x561ccf9e7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:00.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.334+0000 7f9e738e2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9dddc0 0x561ccf9e5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:00.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.334+0000 7f9e738e2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9e8680 0x561ccf9e57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:00.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.334+0000 7f9e738e2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561ccf9eb250 con 0x561ccf9dd4a0 2024-09-11T23:29:00.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.334+0000 7f9e738e2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561ccf8fe050 con 0x561ccf9e8680 2024-09-11T23:29:00.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.334+0000 7f9e738e2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561ccf9eb3d0 con 0x561ccf9dddc0 2024-09-11T23:29:00.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.334+0000 7f9e738e2740 10 client.0 _ll_register_callbacks cb 0x561ccf93e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:00.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.334+0000 7f9e6a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9dddc0 0x561ccf9e5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:00.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.334+0000 7f9e6b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dd4a0 0x561ccf9e7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:00.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.334+0000 7f9e6a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9dddc0 0x561ccf9e5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:40932/0 (socket says 172.21.15.43:40932) 2024-09-11T23:29:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.334+0000 7f9e6a7fc640 1 -- 172.21.15.43:0/3820153483 learned_addr learned my addr 172.21.15.43:0/3820153483 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.334+0000 7f9e6b7fe640 1 --2- 172.21.15.43:0/3820153483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9e8680 0x561ccf9e57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.334+0000 7f9e6b7fe640 1 -- 172.21.15.43:0/3820153483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9dddc0 msgr2=0x561ccf9e5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.334+0000 7f9e6b7fe640 1 --2- 172.21.15.43:0/3820153483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9dddc0 0x561ccf9e5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.334+0000 7f9e6b7fe640 1 -- 172.21.15.43:0/3820153483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9e8680 msgr2=0x561ccf9e57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.334+0000 7f9e6b7fe640 1 --2- 172.21.15.43:0/3820153483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9e8680 0x561ccf9e57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.334+0000 7f9e6b7fe640 1 -- 172.21.15.43:0/3820153483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ccf6b43c0 con 0x561ccf9dd4a0 2024-09-11T23:29:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.334+0000 7f9e6b7fe640 1 --2- 172.21.15.43:0/3820153483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dd4a0 0x561ccf9e7fe0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9e64047040 tx=0x7f9e64003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74da209b65ebe1b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.335+0000 7f9e4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:29:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.335+0000 7f9e4eff5640 1 -- 172.21.15.43:0/3820153483 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64057060 con 0x561ccf9dd4a0 2024-09-11T23:29:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.335+0000 7f9e4eff5640 1 -- 172.21.15.43:0/3820153483 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f9e64041040 con 0x561ccf9dd4a0 2024-09-11T23:29:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.335+0000 7f9e738e2740 1 -- 172.21.15.43:0/3820153483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e64036040 con 0x561ccf9dd4a0 2024-09-11T23:29:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.335+0000 7f9e4eff5640 1 -- 172.21.15.43:0/3820153483 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9e64049040 con 0x561ccf9dd4a0 2024-09-11T23:29:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.335+0000 7f9e738e2740 10 client.16527 fetch_fsmap learned FSMap version 30 2024-09-11T23:29:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.335+0000 7f9e738e2740 1 -- 172.21.15.43:0/3820153483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561ccf9eb250 con 0x561ccf9dd4a0 2024-09-11T23:29:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.335+0000 7f9e4eff5640 1 -- 172.21.15.43:0/3820153483 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e64036040 con 0x561ccf9dd4a0 2024-09-11T23:29:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.335+0000 7f9e4eff5640 1 -- 172.21.15.43:0/3820153483 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9e6404f070 con 0x561ccf9dd4a0 2024-09-11T23:29:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.335+0000 7f9e738e2740 10 client.16527 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:29:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.335+0000 7f9e738e2740 10 client.16527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.335+0000 7f9e738e2740 20 client.16527 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.336+0000 7f9e738e2740 1 --2- 172.21.15.43:0/3820153483 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561ccf9f7d10 0x561ccf9fa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.336+0000 7f9e738e2740 4 client.16527 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:29:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.336+0000 7f9e738e2740 1 -- 172.21.15.43:0/3820153483 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ccf950850 con 0x561ccf9f7d10 2024-09-11T23:29:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.336+0000 7f9e6a7fc640 1 --2- 172.21.15.43:0/3820153483 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561ccf9f7d10 0x561ccf9fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.336+0000 7f9e4eff5640 10 client.16527 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.336+0000 7f9e6a7fc640 1 --2- 172.21.15.43:0/3820153483 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561ccf9f7d10 0x561ccf9fa100 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76f976de8bf1fa19 server_cookie=b7f6020b902b43b6 in_seq=0 out_seq=0 2024-09-11T23:29:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.337+0000 7f9e4eff5640 1 -- 172.21.15.43:0/3820153483 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ccf950850 con 0x561ccf9f7d10 2024-09-11T23:29:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.337+0000 7f9e4eff5640 10 client.16527 handle_command_reply: tid=0 2024-09-11T23:29:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.337+0000 7f9e738e2740 1 client.16527 shutdown 2024-09-11T23:29:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.337+0000 7f9e738e2740 1 -- 172.21.15.43:0/3820153483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dd4a0 msgr2=0x561ccf9e7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.337+0000 7f9e738e2740 1 --2- 172.21.15.43:0/3820153483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dd4a0 0x561ccf9e7fe0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9e64047040 tx=0x7f9e64003d10 comp rx=0 tx=0).stop 2024-09-11T23:29:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.337+0000 7f9e738e2740 1 -- 172.21.15.43:0/3820153483 shutdown_connections 2024-09-11T23:29:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.337+0000 7f9e738e2740 1 --2- 172.21.15.43:0/3820153483 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561ccf9f7d10 0x561ccf9fa100 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.338+0000 7f9e738e2740 1 --2- 172.21.15.43:0/3820153483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9e8680 0x561ccf9e57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.338+0000 7f9e738e2740 1 --2- 172.21.15.43:0/3820153483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9dddc0 0x561ccf9e5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.338+0000 7f9e738e2740 1 --2- 172.21.15.43:0/3820153483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dd4a0 0x561ccf9e7fe0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.338+0000 7f9e738e2740 1 -- 172.21.15.43:0/3820153483 >> 172.21.15.43:0/3820153483 conn(0x561ccf9d9a30 msgr2=0x561ccf9df760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.338+0000 7f9e4eff5640 0 client.16527 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.338+0000 7f9e738e2740 1 -- 172.21.15.43:0/3820153483 shutdown_connections 2024-09-11T23:29:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.338+0000 7f9e738e2740 1 -- 172.21.15.43:0/3820153483 wait complete. 2024-09-11T23:29:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.338+0000 7f9e738e2740 20 client.16527 trim_cache size 0 max 16384 2024-09-11T23:29:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.365+0000 7f9e738e2740 1 Processor -- start 2024-09-11T23:29:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.365+0000 7f9e738e2740 1 -- start start 2024-09-11T23:29:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.365+0000 7f9e738e2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dd4a0 0x561ccf9f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.365+0000 7f9e738e2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9dddc0 0x561ccf9ff730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.365+0000 7f9e738e2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9e8680 0x561ccf9ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.365+0000 7f9e738e2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c1b0190 con 0x561ccf9dd4a0 2024-09-11T23:29:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.365+0000 7f9e738e2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c1ada60 con 0x561ccf9e8680 2024-09-11T23:29:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.365+0000 7f9e738e2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c1ab8b0 con 0x561ccf9dddc0 2024-09-11T23:29:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.365+0000 7f9e6b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dd4a0 0x561ccf9f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.365+0000 7f9e6b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dd4a0 0x561ccf9f44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58900/0 (socket says 172.21.15.43:58900) 2024-09-11T23:29:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.365+0000 7f9e6b7fe640 1 -- 172.21.15.43:0/2196045893 learned_addr learned my addr 172.21.15.43:0/2196045893 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.365+0000 7f9e6a7fc640 1 --2- 172.21.15.43:0/2196045893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9dddc0 0x561ccf9ff730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.366+0000 7f9e6b7fe640 1 --2- 172.21.15.43:0/2196045893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9e8680 0x561ccf9ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.366+0000 7f9e6b7fe640 1 -- 172.21.15.43:0/2196045893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9dddc0 msgr2=0x561ccf9ff730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.366+0000 7f9e6b7fe640 1 --2- 172.21.15.43:0/2196045893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9dddc0 0x561ccf9ff730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.366+0000 7f9e6b7fe640 1 -- 172.21.15.43:0/2196045893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9e8680 msgr2=0x561ccf9ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.366+0000 7f9e6b7fe640 1 --2- 172.21.15.43:0/2196045893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9e8680 0x561ccf9ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.366+0000 7f9e6b7fe640 1 -- 172.21.15.43:0/2196045893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ccf9dd290 con 0x561ccf9dd4a0 2024-09-11T23:29:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.366+0000 7f9e6b7fe640 1 --2- 172.21.15.43:0/2196045893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dd4a0 0x561ccf9f44b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009620 tx=0x7f9e640021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de0659ab6dc7293 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.366+0000 7f9e50ff9640 1 -- 172.21.15.43:0/2196045893 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64057040 con 0x561ccf9dd4a0 2024-09-11T23:29:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.367+0000 7f9e50ff9640 1 -- 172.21.15.43:0/2196045893 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e64036050 con 0x561ccf9dd4a0 2024-09-11T23:29:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.367+0000 7f9e50ff9640 1 -- 172.21.15.43:0/2196045893 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6405b460 con 0x561ccf9dd4a0 2024-09-11T23:29:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.367+0000 7f9e738e2740 1 -- 172.21.15.43:0/2196045893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dd4a0 msgr2=0x561ccf9f44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.367+0000 7f9e738e2740 1 --2- 172.21.15.43:0/2196045893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dd4a0 0x561ccf9f44b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009620 tx=0x7f9e640021f0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.368+0000 7f9e738e2740 1 -- 172.21.15.43:0/2196045893 shutdown_connections 2024-09-11T23:29:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.368+0000 7f9e738e2740 1 --2- 172.21.15.43:0/2196045893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9e8680 0x561ccf9ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.368+0000 7f9e738e2740 1 --2- 172.21.15.43:0/2196045893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9dddc0 0x561ccf9ff730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.368+0000 7f9e738e2740 1 --2- 172.21.15.43:0/2196045893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dd4a0 0x561ccf9f44b0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.368+0000 7f9e738e2740 1 -- 172.21.15.43:0/2196045893 >> 172.21.15.43:0/2196045893 conn(0x561ccf9d9a30 msgr2=0x561ccf9e0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:00.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.368+0000 7f9e738e2740 1 -- 172.21.15.43:0/2196045893 shutdown_connections 2024-09-11T23:29:00.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.368+0000 7f9e738e2740 1 -- 172.21.15.43:0/2196045893 wait complete. 2024-09-11T23:29:00.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.368+0000 7f9e738e2740 1 -- start start 2024-09-11T23:29:00.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.368+0000 7f9e738e2740 1 Processor -- start 2024-09-11T23:29:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.369+0000 7f9e738e2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9dd4a0 0x561ccfa5ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.369+0000 7f9e738e2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dddc0 0x561ccfa5b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.369+0000 7f9e6b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9dd4a0 0x561ccfa5ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.369+0000 7f9e6b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9dd4a0 0x561ccfa5ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:57082/0 (socket says 172.21.15.43:57082) 2024-09-11T23:29:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.369+0000 7f9e6b7fe640 1 -- 172.21.15.43:0/3677641 learned_addr learned my addr 172.21.15.43:0/3677641 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.369+0000 7f9e738e2740 1 --2- 172.21.15.43:0/3677641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9e8680 0x561ccfa5b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.369+0000 7f9e738e2740 1 -- 172.21.15.43:0/3677641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c1abf20 con 0x561ccf9dddc0 2024-09-11T23:29:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.369+0000 7f9e738e2740 1 -- 172.21.15.43:0/3677641 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c1ad390 con 0x561ccf9dd4a0 2024-09-11T23:29:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.369+0000 7f9e738e2740 1 -- 172.21.15.43:0/3677641 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c1ac5f0 con 0x561ccf9e8680 2024-09-11T23:29:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.369+0000 7f9e6a7fc640 1 --2- 172.21.15.43:0/3677641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dddc0 0x561ccfa5b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.369+0000 7f9e738e2740 10 client.0 _ll_register_callbacks cb 0x7f9e64047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.369+0000 7f9e6b7fe640 1 --2- 172.21.15.43:0/3677641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9e8680 0x561ccfa5b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.369+0000 7f9e6b7fe640 1 -- 172.21.15.43:0/3677641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9e8680 msgr2=0x561ccfa5b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.369+0000 7f9e6b7fe640 1 --2- 172.21.15.43:0/3677641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9e8680 0x561ccfa5b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.369+0000 7f9e6b7fe640 1 -- 172.21.15.43:0/3677641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dddc0 msgr2=0x561ccfa5b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.369+0000 7f9e6b7fe640 1 --2- 172.21.15.43:0/3677641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dddc0 0x561ccfa5b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.369+0000 7f9e6b7fe640 1 -- 172.21.15.43:0/3677641 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ccf9ef710 con 0x561ccf9dd4a0 2024-09-11T23:29:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.370+0000 7f9e6b7fe640 1 --2- 172.21.15.43:0/3677641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9dd4a0 0x561ccfa5ac60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9e6405e040 tx=0x7f9e64030da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d0aa05e75246ee9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:00.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.370+0000 7f9e4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:29:00.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.370+0000 7f9e4eff5640 1 -- 172.21.15.43:0/3677641 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64057040 con 0x561ccf9dd4a0 2024-09-11T23:29:00.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.370+0000 7f9e4eff5640 1 -- 172.21.15.43:0/3677641 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f9e64052a80 con 0x561ccf9dd4a0 2024-09-11T23:29:00.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.370+0000 7f9e738e2740 1 -- 172.21.15.43:0/3677641 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e64036050 con 0x561ccf9dd4a0 2024-09-11T23:29:00.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.370+0000 7f9e4eff5640 1 -- 172.21.15.43:0/3677641 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9e64004b40 con 0x561ccf9dd4a0 2024-09-11T23:29:00.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.371+0000 7f9e738e2740 10 client.26045 fetch_fsmap learned FSMap version 30 2024-09-11T23:29:00.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.371+0000 7f9e738e2740 1 -- 172.21.15.43:0/3677641 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561ccf9f0980 con 0x561ccf9dd4a0 2024-09-11T23:29:00.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.371+0000 7f9e4eff5640 1 -- 172.21.15.43:0/3677641 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e64004ce0 con 0x561ccf9dd4a0 2024-09-11T23:29:00.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.372+0000 7f9e4eff5640 1 -- 172.21.15.43:0/3677641 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9e6404f040 con 0x561ccf9dd4a0 2024-09-11T23:29:00.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.372+0000 7f9e738e2740 10 client.26045 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T23:29:00.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.372+0000 7f9e738e2740 10 client.26045 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:00.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.372+0000 7f9e738e2740 20 client.26045 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:00.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.372+0000 7f9e738e2740 1 --2- 172.21.15.43:0/3677641 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561ccf9f0e40 0x561ccfa6d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:00.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.372+0000 7f9e738e2740 4 client.26045 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:29:00.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.372+0000 7f9e738e2740 1 -- 172.21.15.43:0/3677641 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f9e64036050 con 0x561ccf9f0e40 2024-09-11T23:29:00.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.372+0000 7f9e6a7fc640 1 --2- 172.21.15.43:0/3677641 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561ccf9f0e40 0x561ccfa6d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:00.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.373+0000 7f9e6a7fc640 1 --2- 172.21.15.43:0/3677641 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561ccf9f0e40 0x561ccfa6d310 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21736ad4bb084489 server_cookie=2b5fc5600bafdd8a in_seq=0 out_seq=0 2024-09-11T23:29:00.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.373+0000 7f9e4eff5640 10 client.26045 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:00.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.373+0000 7f9e4eff5640 1 -- 172.21.15.43:0/3677641 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f9e64036050 con 0x561ccf9f0e40 2024-09-11T23:29:00.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.373+0000 7f9e4eff5640 10 client.26045 handle_command_reply: tid=0 2024-09-11T23:29:00.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.374+0000 7f9e738e2740 1 client.26045 shutdown 2024-09-11T23:29:00.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.374+0000 7f9e738e2740 1 -- 172.21.15.43:0/3677641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9dd4a0 msgr2=0x561ccfa5ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:00.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.374+0000 7f9e738e2740 1 --2- 172.21.15.43:0/3677641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9dd4a0 0x561ccfa5ac60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9e6405e040 tx=0x7f9e64030da0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.374+0000 7f9e738e2740 1 -- 172.21.15.43:0/3677641 shutdown_connections 2024-09-11T23:29:00.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.374+0000 7f9e738e2740 1 --2- 172.21.15.43:0/3677641 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561ccf9f0e40 0x561ccfa6d310 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.374+0000 7f9e738e2740 1 --2- 172.21.15.43:0/3677641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561ccf9e8680 0x561ccfa5b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.376 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:29:00.376 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-11T23:29:00.376 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-11T23:29:00.376 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:29:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.374+0000 7f9e738e2740 1 --2- 172.21.15.43:0/3677641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561ccf9dddc0 0x561ccfa5b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.374+0000 7f9e738e2740 1 --2- 172.21.15.43:0/3677641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ccf9dd4a0 0x561ccfa5ac60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.374+0000 7f9e738e2740 1 -- 172.21.15.43:0/3677641 >> 172.21.15.43:0/3677641 conn(0x561ccf9d9a30 msgr2=0x561ccf9d8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.374+0000 7f9e4eff5640 0 client.26045 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.374+0000 7f9e738e2740 1 -- 172.21.15.43:0/3677641 shutdown_connections 2024-09-11T23:29:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.374+0000 7f9e738e2740 1 -- 172.21.15.43:0/3677641 wait complete. 2024-09-11T23:29:00.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.374+0000 7f9e738e2740 20 client.26045 trim_cache size 0 max 16384 2024-09-11T23:29:00.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.375+0000 7f9e71509640 1 -- 172.21.15.43:0/3770864027 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e540a2cf0 msgr2=0x7f9e540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:00.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.375+0000 7f9e71509640 1 --2- 172.21.15.43:0/3770864027 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e540a2cf0 0x7f9e540a51b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0ccee0 tx=0x7f9e5c000bc0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.375+0000 7f9e71509640 1 -- 172.21.15.43:0/3770864027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c100a80 msgr2=0x7f9e6c198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:00.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.375+0000 7f9e71509640 1 --2- 172.21.15.43:0/3770864027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c100a80 0x7f9e6c198420 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00e9f0 tx=0x7f9e5c00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.378+0000 7f9e71509640 1 -- 172.21.15.43:0/3770864027 shutdown_connections 2024-09-11T23:29:00.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.378+0000 7f9e71509640 1 --2- 172.21.15.43:0/3770864027 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e540a2cf0 0x7f9e540a51b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.378+0000 7f9e71509640 1 --2- 172.21.15.43:0/3770864027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e6c106a80 0x7f9e6c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.378+0000 7f9e71509640 1 --2- 172.21.15.43:0/3770864027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e6c101420 0x7f9e6c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.378+0000 7f9e71509640 1 --2- 172.21.15.43:0/3770864027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e6c100a80 0x7f9e6c198420 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.378+0000 7f9e71509640 1 -- 172.21.15.43:0/3770864027 >> 172.21.15.43:0/3770864027 conn(0x7f9e6c0fc7a0 msgr2=0x7f9e6c1047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.379+0000 7f9e71509640 1 -- 172.21.15.43:0/3770864027 shutdown_connections 2024-09-11T23:29:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:00.379+0000 7f9e71509640 1 -- 172.21.15.43:0/3770864027 wait complete. 2024-09-11T23:29:00.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:00 smithi090 ceph-mon[35130]: pgmap v218: 97 pgs: 97 active+clean; 22 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 51 KiB/s rd, 9.0 MiB/s wr, 2.03k op/s 2024-09-11T23:29:00.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:00 smithi097 ceph-mon[35562]: pgmap v218: 97 pgs: 97 active+clean; 22 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 51 KiB/s rd, 9.0 MiB/s wr, 2.03k op/s 2024-09-11T23:29:00.581 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:29:01.091 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:29:01.091 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:29:01.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.633+0000 7efd1be64640 1 -- 172.21.15.43:0/736962923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd1410e3c0 msgr2=0x7efd14110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.633+0000 7efd1be64640 1 --2- 172.21.15.43:0/736962923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd1410e3c0 0x7efd14110810 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009900 tx=0x7efd0c02a480 comp rx=0 tx=0).stop 2024-09-11T23:29:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.634+0000 7efd1be64640 1 -- 172.21.15.43:0/736962923 shutdown_connections 2024-09-11T23:29:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.634+0000 7efd1be64640 1 --2- 172.21.15.43:0/736962923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd1410e3c0 0x7efd14110810 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.634+0000 7efd1be64640 1 --2- 172.21.15.43:0/736962923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd14074f40 0x7efd1410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.634+0000 7efd1be64640 1 --2- 172.21.15.43:0/736962923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd14074620 0x7efd14074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.634+0000 7efd1be64640 1 -- 172.21.15.43:0/736962923 >> 172.21.15.43:0/736962923 conn(0x7efd140fd8c0 msgr2=0x7efd140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.634+0000 7efd1be64640 1 -- 172.21.15.43:0/736962923 shutdown_connections 2024-09-11T23:29:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.634+0000 7efd1be64640 1 -- 172.21.15.43:0/736962923 wait complete. 2024-09-11T23:29:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.634+0000 7efd1be64640 1 Processor -- start 2024-09-11T23:29:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.634+0000 7efd1be64640 1 -- start start 2024-09-11T23:29:01.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.635+0000 7efd1be64640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd14074620 0x7efd1419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:01.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.635+0000 7efd1be64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd14074f40 0x7efd1419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:01.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.635+0000 7efd1be64640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1410e3c0 0x7efd1419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:01.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.635+0000 7efd1be64640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efd14114170 con 0x7efd14074f40 2024-09-11T23:29:01.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.635+0000 7efd1be64640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efd14113ff0 con 0x7efd14074620 2024-09-11T23:29:01.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.635+0000 7efd1be64640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efd141142f0 con 0x7efd1410e3c0 2024-09-11T23:29:01.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.635+0000 7efd193ff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd14074f40 0x7efd1419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:01.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.635+0000 7efd193ff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd14074f40 0x7efd1419ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58920/0 (socket says 172.21.15.43:58920) 2024-09-11T23:29:01.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.635+0000 7efd193ff640 1 -- 172.21.15.43:0/2793304894 learned_addr learned my addr 172.21.15.43:0/2793304894 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:01.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.635+0000 7efd1a401640 1 --2- 172.21.15.43:0/2793304894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1410e3c0 0x7efd1419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:01.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.635+0000 7efd19c00640 1 --2- 172.21.15.43:0/2793304894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd14074620 0x7efd1419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:01.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.635+0000 7efd193ff640 1 -- 172.21.15.43:0/2793304894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1410e3c0 msgr2=0x7efd1419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:01.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.635+0000 7efd193ff640 1 --2- 172.21.15.43:0/2793304894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1410e3c0 0x7efd1419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:01.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.635+0000 7efd193ff640 1 -- 172.21.15.43:0/2793304894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd14074620 msgr2=0x7efd1419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:01.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.635+0000 7efd193ff640 1 --2- 172.21.15.43:0/2793304894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd14074620 0x7efd1419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:01.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.635+0000 7efd193ff640 1 -- 172.21.15.43:0/2793304894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1419f680 con 0x7efd14074f40 2024-09-11T23:29:01.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.635+0000 7efd19c00640 1 --2- 172.21.15.43:0/2793304894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd14074620 0x7efd1419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:01.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.635+0000 7efd193ff640 1 --2- 172.21.15.43:0/2793304894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd14074f40 0x7efd1419ac10 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7efd0800e9f0 tx=0x7efd0800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:01.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.635+0000 7efd06ffd640 1 -- 172.21.15.43:0/2793304894 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0800cd80 con 0x7efd14074f40 2024-09-11T23:29:01.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.636+0000 7efd1be64640 1 -- 172.21.15.43:0/2793304894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1419f970 con 0x7efd14074f40 2024-09-11T23:29:01.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.636+0000 7efd06ffd640 1 -- 172.21.15.43:0/2793304894 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd08005500 con 0x7efd14074f40 2024-09-11T23:29:01.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.636+0000 7efd06ffd640 1 -- 172.21.15.43:0/2793304894 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08010690 con 0x7efd14074f40 2024-09-11T23:29:01.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.636+0000 7efd1be64640 1 -- 172.21.15.43:0/2793304894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd141a7250 con 0x7efd14074f40 2024-09-11T23:29:01.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.637+0000 7efd1be64640 1 -- 172.21.15.43:0/2793304894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd14074a00 con 0x7efd14074f40 2024-09-11T23:29:01.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.638+0000 7efd06ffd640 1 -- 172.21.15.43:0/2793304894 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7efd080046c0 con 0x7efd14074f40 2024-09-11T23:29:01.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.638+0000 7efd06ffd640 1 --2- 172.21.15.43:0/2793304894 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7efd000a2cf0 0x7efd000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:01.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.638+0000 7efd06ffd640 1 -- 172.21.15.43:0/2793304894 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7efd080cf8a0 con 0x7efd14074f40 2024-09-11T23:29:01.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.638+0000 7efd19c00640 1 --2- 172.21.15.43:0/2793304894 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7efd000a2cf0 0x7efd000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:01.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.639+0000 7efd19c00640 1 --2- 172.21.15.43:0/2793304894 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7efd000a2cf0 0x7efd000a51b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd10005fd0 tx=0x7efd10009500 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:01.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.643+0000 7efd06ffd640 1 -- 172.21.15.43:0/2793304894 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efd0809c4c0 con 0x7efd14074f40 2024-09-11T23:29:01.682 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:29:01.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.877+0000 7efd1be64640 1 -- 172.21.15.43:0/2793304894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd14109570 con 0x7efd14074f40 2024-09-11T23:29:01.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.878+0000 7efd06ffd640 1 -- 172.21.15.43:0/2793304894 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11893 (secure 0 0 0) 0x7efd0809bdc0 con 0x7efd14074f40 2024-09-11T23:29:01.878 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:29:01.879 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":30,"btime":"2024-09-11T23:25:40:446496+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:25:39.727562+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25571":{"gid":25571,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/568491708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":568491708},{"type":"v1","addr":"172.21.15.90:6835","nonce":568491708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:29:01.879 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 30 2024-09-11T23:29:01.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.883+0000 7efd1be64640 1 -- 172.21.15.43:0/2793304894 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7efd000a2cf0 msgr2=0x7efd000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.883+0000 7efd1be64640 1 --2- 172.21.15.43:0/2793304894 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7efd000a2cf0 0x7efd000a51b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd10005fd0 tx=0x7efd10009500 comp rx=0 tx=0).stop 2024-09-11T23:29:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.883+0000 7efd1be64640 1 -- 172.21.15.43:0/2793304894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd14074f40 msgr2=0x7efd1419ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.883+0000 7efd1be64640 1 --2- 172.21.15.43:0/2793304894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd14074f40 0x7efd1419ac10 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7efd0800e9f0 tx=0x7efd0800edf0 comp rx=0 tx=0).stop 2024-09-11T23:29:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.887+0000 7efd1be64640 1 -- 172.21.15.43:0/2793304894 shutdown_connections 2024-09-11T23:29:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.887+0000 7efd1be64640 1 --2- 172.21.15.43:0/2793304894 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7efd000a2cf0 0x7efd000a51b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.887+0000 7efd1be64640 1 --2- 172.21.15.43:0/2793304894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1410e3c0 0x7efd1419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.887+0000 7efd1be64640 1 --2- 172.21.15.43:0/2793304894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd14074f40 0x7efd1419ac10 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.887+0000 7efd1be64640 1 --2- 172.21.15.43:0/2793304894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd14074620 0x7efd1419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.887+0000 7efd1be64640 1 -- 172.21.15.43:0/2793304894 >> 172.21.15.43:0/2793304894 conn(0x7efd140fd8c0 msgr2=0x7efd140fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.888+0000 7efd1be64640 1 -- 172.21.15.43:0/2793304894 shutdown_connections 2024-09-11T23:29:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:01.888+0000 7efd1be64640 1 -- 172.21.15.43:0/2793304894 wait complete. 2024-09-11T23:29:02.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:02 smithi090 ceph-mon[35130]: pgmap v219: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 123 KiB/s rd, 11 MiB/s wr, 2.68k op/s 2024-09-11T23:29:02.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:02 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2793304894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:29:02.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:02 smithi097 ceph-mon[35562]: pgmap v219: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 123 KiB/s rd, 11 MiB/s wr, 2.68k op/s 2024-09-11T23:29:02.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:02 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2793304894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:29:02.602 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:02 smithi043 ceph-mon[30950]: pgmap v219: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 123 KiB/s rd, 11 MiB/s wr, 2.68k op/s 2024-09-11T23:29:02.602 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:02 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2793304894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:29:02.632 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-11T23:29:02.633 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-11T23:29:02.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.902+0000 7f83682a0640 1 -- 172.21.15.43:0/626528460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8360102570 msgr2=0x7f83601029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:02.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.902+0000 7f83682a0640 1 --2- 172.21.15.43:0/626528460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8360102570 0x7f83601029b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8350009900 tx=0x7f835002a480 comp rx=0 tx=0).stop 2024-09-11T23:29:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.902+0000 7f83682a0640 1 -- 172.21.15.43:0/626528460 shutdown_connections 2024-09-11T23:29:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.902+0000 7f83682a0640 1 --2- 172.21.15.43:0/626528460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8360102ef0 0x7f836010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.902+0000 7f83682a0640 1 --2- 172.21.15.43:0/626528460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8360102570 0x7f83601029b0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.902+0000 7f83682a0640 1 --2- 172.21.15.43:0/626528460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8360100e20 0x7f8360101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.902+0000 7f83682a0640 1 -- 172.21.15.43:0/626528460 >> 172.21.15.43:0/626528460 conn(0x7f83600fc7d0 msgr2=0x7f83600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.902+0000 7f83682a0640 1 -- 172.21.15.43:0/626528460 shutdown_connections 2024-09-11T23:29:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.902+0000 7f83682a0640 1 -- 172.21.15.43:0/626528460 wait complete. 2024-09-11T23:29:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.903+0000 7f83682a0640 1 Processor -- start 2024-09-11T23:29:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.903+0000 7f83682a0640 1 -- start start 2024-09-11T23:29:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.903+0000 7f83682a0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8360100e20 0x7f83601984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.903+0000 7f83682a0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8360102570 0x7f8360198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:02.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.903+0000 7f83682a0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8360102ef0 0x7f836019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:02.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.903+0000 7f83682a0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f83601120a0 con 0x7f8360102570 2024-09-11T23:29:02.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.903+0000 7f83682a0640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8360111f20 con 0x7f8360102ef0 2024-09-11T23:29:02.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.903+0000 7f83682a0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8360112220 con 0x7f8360100e20 2024-09-11T23:29:02.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.903+0000 7f836583b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8360102570 0x7f8360198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:02.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.903+0000 7f836583b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8360102570 0x7f8360198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58944/0 (socket says 172.21.15.43:58944) 2024-09-11T23:29:02.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.903+0000 7f836583b640 1 -- 172.21.15.43:0/4293669427 learned_addr learned my addr 172.21.15.43:0/4293669427 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:02.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.903+0000 7f836603c640 1 --2- 172.21.15.43:0/4293669427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8360100e20 0x7f83601984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:02.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.903+0000 7f836683d640 1 --2- 172.21.15.43:0/4293669427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8360102ef0 0x7f836019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:02.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.903+0000 7f836583b640 1 -- 172.21.15.43:0/4293669427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8360100e20 msgr2=0x7f83601984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:02.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.903+0000 7f836583b640 1 --2- 172.21.15.43:0/4293669427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8360100e20 0x7f83601984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:02.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.904+0000 7f836583b640 1 -- 172.21.15.43:0/4293669427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8360102ef0 msgr2=0x7f836019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:02.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.904+0000 7f836583b640 1 --2- 172.21.15.43:0/4293669427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8360102ef0 0x7f836019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:02.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.904+0000 7f836583b640 1 -- 172.21.15.43:0/4293669427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836019d490 con 0x7f8360102570 2024-09-11T23:29:02.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.904+0000 7f836583b640 1 --2- 172.21.15.43:0/4293669427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8360102570 0x7f8360198a20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8350006010 tx=0x7f8350003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:02.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.904+0000 7f83577fe640 1 -- 172.21.15.43:0/4293669427 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835003f070 con 0x7f8360102570 2024-09-11T23:29:02.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.904+0000 7f83682a0640 1 -- 172.21.15.43:0/4293669427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f836019d720 con 0x7f8360102570 2024-09-11T23:29:02.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.904+0000 7f83577fe640 1 -- 172.21.15.43:0/4293669427 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8350031c30 con 0x7f8360102570 2024-09-11T23:29:02.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.904+0000 7f83577fe640 1 -- 172.21.15.43:0/4293669427 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350035030 con 0x7f8360102570 2024-09-11T23:29:02.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.904+0000 7f83682a0640 1 -- 172.21.15.43:0/4293669427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83601a5060 con 0x7f8360102570 2024-09-11T23:29:02.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.905+0000 7f83682a0640 1 -- 172.21.15.43:0/4293669427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83601021b0 con 0x7f8360102570 2024-09-11T23:29:02.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.906+0000 7f83577fe640 1 -- 172.21.15.43:0/4293669427 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8350042070 con 0x7f8360102570 2024-09-11T23:29:02.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.907+0000 7f83577fe640 1 --2- 172.21.15.43:0/4293669427 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f834c0a2cf0 0x7f834c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:02.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.907+0000 7f836603c640 1 --2- 172.21.15.43:0/4293669427 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f834c0a2cf0 0x7f834c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:02.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.907+0000 7f83577fe640 1 -- 172.21.15.43:0/4293669427 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f83500f01d0 con 0x7f8360102570 2024-09-11T23:29:02.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.907+0000 7f836603c640 1 --2- 172.21.15.43:0/4293669427 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f834c0a2cf0 0x7f834c0a51b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f835c009b60 tx=0x7f835c005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:02.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:02.910+0000 7f83577fe640 1 -- 172.21.15.43:0/4293669427 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f83500bcdf0 con 0x7f8360102570 2024-09-11T23:29:03.018 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:29:03.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.213+0000 7f83682a0640 1 -- 172.21.15.43:0/4293669427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8360109090 con 0x7f8360102570 2024-09-11T23:29:03.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.216+0000 7f83577fe640 1 -- 172.21.15.43:0/4293669427 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11894 (secure 0 0 0) 0x7f8350031de0 con 0x7f8360102570 2024-09-11T23:29:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:29:03.216 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":31,"btime":"2024-09-11T23:29:03:134233+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:29:02.865047+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25571":{"gid":25571,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/568491708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":568491708},{"type":"v1","addr":"172.21.15.90:6835","nonce":568491708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24457,24470,24497]},"id":1}]} 2024-09-11T23:29:03.217 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 31 2024-09-11T23:29:03.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.220+0000 7f83682a0640 1 -- 172.21.15.43:0/4293669427 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f834c0a2cf0 msgr2=0x7f834c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:03.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.220+0000 7f83682a0640 1 --2- 172.21.15.43:0/4293669427 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f834c0a2cf0 0x7f834c0a51b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f835c009b60 tx=0x7f835c005f90 comp rx=0 tx=0).stop 2024-09-11T23:29:03.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.220+0000 7f83682a0640 1 -- 172.21.15.43:0/4293669427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8360102570 msgr2=0x7f8360198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:03.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.220+0000 7f83682a0640 1 --2- 172.21.15.43:0/4293669427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8360102570 0x7f8360198a20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8350006010 tx=0x7f8350003c10 comp rx=0 tx=0).stop 2024-09-11T23:29:03.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.220+0000 7f83682a0640 1 -- 172.21.15.43:0/4293669427 shutdown_connections 2024-09-11T23:29:03.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.220+0000 7f83682a0640 1 --2- 172.21.15.43:0/4293669427 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f834c0a2cf0 0x7f834c0a51b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.220+0000 7f83682a0640 1 --2- 172.21.15.43:0/4293669427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8360102ef0 0x7f836019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.220+0000 7f83682a0640 1 --2- 172.21.15.43:0/4293669427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8360102570 0x7f8360198a20 secure :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8350006010 tx=0x7f8350003c10 comp rx=0 tx=0).stop 2024-09-11T23:29:03.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.220+0000 7f83682a0640 1 --2- 172.21.15.43:0/4293669427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8360100e20 0x7f83601984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.220+0000 7f83682a0640 1 -- 172.21.15.43:0/4293669427 >> 172.21.15.43:0/4293669427 conn(0x7f83600fc7d0 msgr2=0x7f83600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:03.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.221+0000 7f83682a0640 1 -- 172.21.15.43:0/4293669427 shutdown_connections 2024-09-11T23:29:03.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.221+0000 7f83682a0640 1 -- 172.21.15.43:0/4293669427 wait complete. 2024-09-11T23:29:03.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.756+0000 7f56e1518640 1 -- 172.21.15.43:0/157216977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56dc1138b0 msgr2=0x7f56dc115ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:03.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.756+0000 7f56e1518640 1 --2- 172.21.15.43:0/157216977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56dc1138b0 0x7f56dc115ca0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f56d400bc00 tx=0x7f56d4009260 comp rx=0 tx=0).stop 2024-09-11T23:29:03.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.757+0000 7f56e1518640 1 -- 172.21.15.43:0/157216977 shutdown_connections 2024-09-11T23:29:03.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.757+0000 7f56e1518640 1 --2- 172.21.15.43:0/157216977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56dc1138b0 0x7f56dc115ca0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.757+0000 7f56e1518640 1 --2- 172.21.15.43:0/157216977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56dc10ac20 0x7f56dc113370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.757+0000 7f56e1518640 1 --2- 172.21.15.43:0/157216977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56dc10a300 0x7f56dc10a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.757+0000 7f56e1518640 1 -- 172.21.15.43:0/157216977 >> 172.21.15.43:0/157216977 conn(0x7f56dc0735f0 msgr2=0x7f56dc073a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:03.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.758+0000 7f56e1518640 1 -- 172.21.15.43:0/157216977 shutdown_connections 2024-09-11T23:29:03.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.758+0000 7f56e1518640 1 -- 172.21.15.43:0/157216977 wait complete. 2024-09-11T23:29:03.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.759+0000 7f56e1518640 1 Processor -- start 2024-09-11T23:29:03.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.759+0000 7f56e1518640 1 -- start start 2024-09-11T23:29:03.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.760+0000 7f56e1518640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56dc10a300 0x7f56dc19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:03.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.760+0000 7f56e1518640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56dc10ac20 0x7f56dc19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:03.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.760+0000 7f56e1518640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56dc1138b0 0x7f56dc1a2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:03.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.761+0000 7f56da7fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56dc10ac20 0x7f56dc19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:03.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.761+0000 7f56da7fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56dc10ac20 0x7f56dc19ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:57138/0 (socket says 172.21.15.43:57138) 2024-09-11T23:29:03.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.761+0000 7f56da7fc640 1 -- 172.21.15.43:0/1632444560 learned_addr learned my addr 172.21.15.43:0/1632444560 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:03.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.761+0000 7f56db7fe640 1 --2- 172.21.15.43:0/1632444560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56dc1138b0 0x7f56dc1a2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:03.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.761+0000 7f56e1518640 1 -- 172.21.15.43:0/1632444560 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc118400 con 0x7f56dc10a300 2024-09-11T23:29:03.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.761+0000 7f56e1518640 1 -- 172.21.15.43:0/1632444560 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc118280 con 0x7f56dc10ac20 2024-09-11T23:29:03.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.761+0000 7f56e1518640 1 -- 172.21.15.43:0/1632444560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc118580 con 0x7f56dc1138b0 2024-09-11T23:29:03.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.761+0000 7f56daffd640 1 --2- 172.21.15.43:0/1632444560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56dc10a300 0x7f56dc19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:03.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.762+0000 7f56db7fe640 1 -- 172.21.15.43:0/1632444560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56dc10ac20 msgr2=0x7f56dc19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:03.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.762+0000 7f56db7fe640 1 --2- 172.21.15.43:0/1632444560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56dc10ac20 0x7f56dc19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.762+0000 7f56db7fe640 1 -- 172.21.15.43:0/1632444560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56dc10a300 msgr2=0x7f56dc19e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:03.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.762+0000 7f56db7fe640 1 --2- 172.21.15.43:0/1632444560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56dc10a300 0x7f56dc19e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.762+0000 7f56db7fe640 1 -- 172.21.15.43:0/1632444560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56dc1a3650 con 0x7f56dc1138b0 2024-09-11T23:29:03.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.762+0000 7f56da7fc640 1 --2- 172.21.15.43:0/1632444560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56dc10ac20 0x7f56dc19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:03.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.763+0000 7f56db7fe640 1 --2- 172.21.15.43:0/1632444560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56dc1138b0 0x7f56dc1a2f70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f56d402a830 tx=0x7f56d400ae80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:03.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.763+0000 7f56c3fff640 1 -- 172.21.15.43:0/1632444560 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d4041020 con 0x7f56dc1138b0 2024-09-11T23:29:03.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.763+0000 7f56e1518640 1 -- 172.21.15.43:0/1632444560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56dc1a38e0 con 0x7f56dc1138b0 2024-09-11T23:29:03.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.763+0000 7f56e1518640 1 -- 172.21.15.43:0/1632444560 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56dc1ab220 con 0x7f56dc1138b0 2024-09-11T23:29:03.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.763+0000 7f56c3fff640 1 -- 172.21.15.43:0/1632444560 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56d40326c0 con 0x7f56dc1138b0 2024-09-11T23:29:03.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.764+0000 7f56c3fff640 1 -- 172.21.15.43:0/1632444560 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d403d660 con 0x7f56dc1138b0 2024-09-11T23:29:03.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.766+0000 7f56e38f1740 1 Processor -- start 2024-09-11T23:29:03.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.768+0000 7f56e38f1740 1 -- start start 2024-09-11T23:29:03.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.768+0000 7f56e38f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151a1a20 0x561e151a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:03.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.768+0000 7f56e38f1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e151a2340 0x561e151ac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:03.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.768+0000 7f56e38f1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151acc00 0x561e151aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:03.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.768+0000 7f56e38f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561e15106160 con 0x561e151a1a20 2024-09-11T23:29:03.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.768+0000 7f56e38f1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561e150ad720 con 0x561e151a2340 2024-09-11T23:29:03.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.768+0000 7f56e38f1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561e15102120 con 0x561e151acc00 2024-09-11T23:29:03.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.768+0000 7f56c3fff640 1 -- 172.21.15.43:0/1632444560 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f56d4044020 con 0x7f56dc1138b0 2024-09-11T23:29:03.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.769+0000 7f56daffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e151a2340 0x561e151ac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:03.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.769+0000 7f56daffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e151a2340 0x561e151ac6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:57140/0 (socket says 172.21.15.43:57140) 2024-09-11T23:29:03.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.769+0000 7f56daffd640 1 -- 172.21.15.43:0/2826663902 learned_addr learned my addr 172.21.15.43:0/2826663902 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:03.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.769+0000 7f56da7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151a1a20 0x561e151a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:03.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.770+0000 7f56da7fc640 1 --2- 172.21.15.43:0/2826663902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151acc00 0x561e151aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:03.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.770+0000 7f56daffd640 1 -- 172.21.15.43:0/2826663902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151acc00 msgr2=0x561e151aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:03.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.770+0000 7f56daffd640 1 --2- 172.21.15.43:0/2826663902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151acc00 0x561e151aeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.770+0000 7f56daffd640 1 -- 172.21.15.43:0/2826663902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151a1a20 msgr2=0x561e151a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:03.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.770+0000 7f56daffd640 1 --2- 172.21.15.43:0/2826663902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151a1a20 0x561e151a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.770+0000 7f56daffd640 1 -- 172.21.15.43:0/2826663902 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e14e77ba0 con 0x561e151a2340 2024-09-11T23:29:03.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.771+0000 7f56da7fc640 1 --2- 172.21.15.43:0/2826663902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151acc00 0x561e151aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:29:03.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.771+0000 7f56daffd640 1 --2- 172.21.15.43:0/2826663902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e151a2340 0x561e151ac6c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f56cc00fbd0 tx=0x7f56cc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65209a22ea4de572 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:03.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.771+0000 7f56c3fff640 1 --2- 172.21.15.43:0/1632444560 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56c40a2df0 0x7f56c40a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:03.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.771+0000 7f56c17fa640 1 -- 172.21.15.43:0/2826663902 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc00fcc0 con 0x561e151a2340 2024-09-11T23:29:03.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.771+0000 7f56c3fff640 1 -- 172.21.15.43:0/1632444560 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f56d40f02b0 con 0x7f56dc1138b0 2024-09-11T23:29:03.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.771+0000 7f56c17fa640 1 -- 172.21.15.43:0/2826663902 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56cc0035f0 con 0x561e151a2340 2024-09-11T23:29:03.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.772+0000 7f56da7fc640 1 --2- 172.21.15.43:0/1632444560 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56c40a2df0 0x7f56c40a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:03.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.772+0000 7f56e38f1740 1 -- 172.21.15.43:0/2826663902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e151a2340 msgr2=0x561e151ac6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:03.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.772+0000 7f56e38f1740 1 --2- 172.21.15.43:0/2826663902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e151a2340 0x561e151ac6c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f56cc00fbd0 tx=0x7f56cc009260 comp rx=0 tx=0).stop 2024-09-11T23:29:03.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.772+0000 7f56da7fc640 1 --2- 172.21.15.43:0/1632444560 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56c40a2df0 0x7f56c40a52b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x561e1519f330 tx=0x7f56d000fb50 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:03.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.773+0000 7f56e38f1740 1 -- 172.21.15.43:0/2826663902 shutdown_connections 2024-09-11T23:29:03.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.773+0000 7f56e38f1740 1 --2- 172.21.15.43:0/2826663902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151acc00 0x561e151aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.773+0000 7f56e38f1740 1 --2- 172.21.15.43:0/2826663902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e151a2340 0x561e151ac6c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.773+0000 7f56e38f1740 1 --2- 172.21.15.43:0/2826663902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151a1a20 0x561e151a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.773+0000 7f56e38f1740 1 -- 172.21.15.43:0/2826663902 >> 172.21.15.43:0/2826663902 conn(0x561e1519dfb0 msgr2=0x561e1519e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:03.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.774+0000 7f56e38f1740 1 -- 172.21.15.43:0/2826663902 shutdown_connections 2024-09-11T23:29:03.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.774+0000 7f56e38f1740 1 -- 172.21.15.43:0/2826663902 wait complete. 2024-09-11T23:29:03.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.774+0000 7f56e38f1740 1 -- start start 2024-09-11T23:29:03.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.774+0000 7f56e38f1740 1 Processor -- start 2024-09-11T23:29:03.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.775+0000 7f56e38f1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e151a1a20 0x561e151ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:03.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.775+0000 7f56e38f1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151a2340 0x561e151a9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:03.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.776+0000 7f56e38f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151acc00 0x561e151a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:03.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.776+0000 7f56e38f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x561e151af870 con 0x561e151acc00 2024-09-11T23:29:03.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.776+0000 7f56e38f1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561e1508e900 con 0x561e151a1a20 2024-09-11T23:29:03.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.776+0000 7f56e38f1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561e151af9f0 con 0x561e151a2340 2024-09-11T23:29:03.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.776+0000 7f56e38f1740 10 client.0 _ll_register_callbacks cb 0x561e1508c780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:03.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.776+0000 7f56db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151acc00 0x561e151a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:03.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.776+0000 7f56db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151acc00 0x561e151a9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58986/0 (socket says 172.21.15.43:58986) 2024-09-11T23:29:03.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.776+0000 7f56db7fe640 1 -- 172.21.15.43:0/1829068971 learned_addr learned my addr 172.21.15.43:0/1829068971 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.776+0000 7f56db7fe640 1 -- 172.21.15.43:0/1829068971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151a2340 msgr2=0x561e151a9850 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:29:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.776+0000 7f56db7fe640 1 --2- 172.21.15.43:0/1829068971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151a2340 0x561e151a9850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.776+0000 7f56db7fe640 1 -- 172.21.15.43:0/1829068971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e151a1a20 msgr2=0x561e151ac5a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:29:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.776+0000 7f56db7fe640 1 --2- 172.21.15.43:0/1829068971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e151a1a20 0x561e151ac5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.776+0000 7f56db7fe640 1 -- 172.21.15.43:0/1829068971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e150cec50 con 0x561e151acc00 2024-09-11T23:29:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.777+0000 7f56db7fe640 1 --2- 172.21.15.43:0/1829068971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151acc00 0x561e151a9df0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f56d402a760 tx=0x7f56d40bc480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6e855a3c32a91c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.777+0000 7f56beff5640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:29:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.778+0000 7f56beff5640 1 -- 172.21.15.43:0/1829068971 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d40bcf80 con 0x561e151acc00 2024-09-11T23:29:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.778+0000 7f56e38f1740 1 -- 172.21.15.43:0/1829068971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56cc0035f0 con 0x561e151acc00 2024-09-11T23:29:03.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.778+0000 7f56beff5640 1 -- 172.21.15.43:0/1829068971 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f56d40edc20 con 0x561e151acc00 2024-09-11T23:29:03.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.779+0000 7f56beff5640 1 -- 172.21.15.43:0/1829068971 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f56d40ede20 con 0x561e151acc00 2024-09-11T23:29:03.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.779+0000 7f56e38f1740 10 client.16557 fetch_fsmap learned FSMap version 31 2024-09-11T23:29:03.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.779+0000 7f56e38f1740 1 -- 172.21.15.43:0/1829068971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561e151cc2e0 con 0x561e151acc00 2024-09-11T23:29:03.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.780+0000 7f56beff5640 1 -- 172.21.15.43:0/1829068971 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56cc0035f0 con 0x561e151acc00 2024-09-11T23:29:03.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.780+0000 7f56beff5640 1 -- 172.21.15.43:0/1829068971 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f56d40c5060 con 0x561e151acc00 2024-09-11T23:29:03.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.780+0000 7f56e38f1740 10 client.16557 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:29:03.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.780+0000 7f56e38f1740 10 client.16557 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:29:03.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.780+0000 7f56e38f1740 20 client.16557 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:03.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.780+0000 7f56e38f1740 1 --2- 172.21.15.43:0/1829068971 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561e151cdaf0 0x561e151cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:03.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.780+0000 7f56e38f1740 4 client.16557 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:29:03.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.780+0000 7f56e38f1740 1 -- 172.21.15.43:0/1829068971 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e15113890 con 0x561e151cdaf0 2024-09-11T23:29:03.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.781+0000 7f56daffd640 1 --2- 172.21.15.43:0/1829068971 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561e151cdaf0 0x561e151cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:03.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.782+0000 7f56beff5640 10 client.16557 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:03.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.782+0000 7f56daffd640 1 --2- 172.21.15.43:0/1829068971 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561e151cdaf0 0x561e151cffb0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d10162161fb52b6 server_cookie=b5e88cbb153f26d6 in_seq=0 out_seq=0 2024-09-11T23:29:03.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.782+0000 7f56beff5640 1 -- 172.21.15.43:0/1829068971 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e15113890 con 0x561e151cdaf0 2024-09-11T23:29:03.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.782+0000 7f56beff5640 10 client.16557 handle_command_reply: tid=0 2024-09-11T23:29:03.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.782+0000 7f56e38f1740 1 client.16557 shutdown 2024-09-11T23:29:03.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.783+0000 7f56e38f1740 1 -- 172.21.15.43:0/1829068971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151acc00 msgr2=0x561e151a9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:03.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.783+0000 7f56e38f1740 1 --2- 172.21.15.43:0/1829068971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151acc00 0x561e151a9df0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f56d402a760 tx=0x7f56d40bc480 comp rx=0 tx=0).stop 2024-09-11T23:29:03.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.783+0000 7f56e38f1740 1 -- 172.21.15.43:0/1829068971 shutdown_connections 2024-09-11T23:29:03.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.783+0000 7f56e38f1740 1 --2- 172.21.15.43:0/1829068971 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561e151cdaf0 0x561e151cffb0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.783+0000 7f56e38f1740 1 --2- 172.21.15.43:0/1829068971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151acc00 0x561e151a9df0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.783+0000 7f56e38f1740 1 --2- 172.21.15.43:0/1829068971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151a2340 0x561e151a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.783+0000 7f56e38f1740 1 --2- 172.21.15.43:0/1829068971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e151a1a20 0x561e151ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.783+0000 7f56e38f1740 1 -- 172.21.15.43:0/1829068971 >> 172.21.15.43:0/1829068971 conn(0x561e1519dfb0 msgr2=0x561e151a3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:03.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.783+0000 7f56beff5640 0 client.16557 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:03.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.785+0000 7f56e38f1740 1 -- 172.21.15.43:0/1829068971 shutdown_connections 2024-09-11T23:29:03.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.785+0000 7f56e38f1740 1 -- 172.21.15.43:0/1829068971 wait complete. 2024-09-11T23:29:03.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.785+0000 7f56e38f1740 20 client.16557 trim_cache size 0 max 16384 2024-09-11T23:29:03.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.818+0000 7f56e38f1740 1 Processor -- start 2024-09-11T23:29:03.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.818+0000 7f56e38f1740 1 -- start start 2024-09-11T23:29:03.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.819+0000 7f56e38f1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151a1a20 0x561e151a9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:03.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.819+0000 7f56e38f1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e151a2340 0x561e151a9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:03.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.819+0000 7f56e38f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151acc00 0x561e151a8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.819+0000 7f56e38f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f56c80110e0 con 0x561e151acc00 2024-09-11T23:29:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.819+0000 7f56e38f1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f56c800e980 con 0x561e151a2340 2024-09-11T23:29:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.819+0000 7f56e38f1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f56c800c7d0 con 0x561e151a1a20 2024-09-11T23:29:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.819+0000 7f56db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151acc00 0x561e151a8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.819+0000 7f56daffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151a1a20 0x561e151a9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.819+0000 7f56db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151acc00 0x561e151a8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58998/0 (socket says 172.21.15.43:58998) 2024-09-11T23:29:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.819+0000 7f56db7fe640 1 -- 172.21.15.43:0/3898355103 learned_addr learned my addr 172.21.15.43:0/3898355103 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.819+0000 7f56da7fc640 1 --2- 172.21.15.43:0/3898355103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e151a2340 0x561e151a9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.820+0000 7f56db7fe640 1 -- 172.21.15.43:0/3898355103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151a1a20 msgr2=0x561e151a9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.820+0000 7f56db7fe640 1 --2- 172.21.15.43:0/3898355103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151a1a20 0x561e151a9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.820+0000 7f56db7fe640 1 -- 172.21.15.43:0/3898355103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e151a2340 msgr2=0x561e151a9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.820+0000 7f56db7fe640 1 --2- 172.21.15.43:0/3898355103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e151a2340 0x561e151a9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.820+0000 7f56db7fe640 1 -- 172.21.15.43:0/3898355103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e151a2170 con 0x561e151acc00 2024-09-11T23:29:03.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.820+0000 7f56db7fe640 1 --2- 172.21.15.43:0/3898355103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151acc00 0x561e151a8120 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f56d40bc580 tx=0x7f56d40bad90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78e834bdc6555fa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:03.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.820+0000 7f56c0ff9640 1 -- 172.21.15.43:0/3898355103 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d40db2a0 con 0x561e151acc00 2024-09-11T23:29:03.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.820+0000 7f56c0ff9640 1 -- 172.21.15.43:0/3898355103 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56d40bc880 con 0x561e151acc00 2024-09-11T23:29:03.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.820+0000 7f56c0ff9640 1 -- 172.21.15.43:0/3898355103 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d40edc20 con 0x561e151acc00 2024-09-11T23:29:03.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.821+0000 7f56e38f1740 1 -- 172.21.15.43:0/3898355103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151acc00 msgr2=0x561e151a8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:03.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.821+0000 7f56e38f1740 1 --2- 172.21.15.43:0/3898355103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151acc00 0x561e151a8120 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f56d40bc580 tx=0x7f56d40bad90 comp rx=0 tx=0).stop 2024-09-11T23:29:03.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.822+0000 7f56e38f1740 1 -- 172.21.15.43:0/3898355103 shutdown_connections 2024-09-11T23:29:03.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.822+0000 7f56e38f1740 1 --2- 172.21.15.43:0/3898355103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151acc00 0x561e151a8120 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.822+0000 7f56e38f1740 1 --2- 172.21.15.43:0/3898355103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e151a2340 0x561e151a9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.822+0000 7f56e38f1740 1 --2- 172.21.15.43:0/3898355103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151a1a20 0x561e151a9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.822+0000 7f56e38f1740 1 -- 172.21.15.43:0/3898355103 >> 172.21.15.43:0/3898355103 conn(0x561e1519dfb0 msgr2=0x561e151a4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:03.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.822+0000 7f56e38f1740 1 -- 172.21.15.43:0/3898355103 shutdown_connections 2024-09-11T23:29:03.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.822+0000 7f56e38f1740 1 -- 172.21.15.43:0/3898355103 wait complete. 2024-09-11T23:29:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.823+0000 7f56e38f1740 1 -- start start 2024-09-11T23:29:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.823+0000 7f56e38f1740 1 Processor -- start 2024-09-11T23:29:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.823+0000 7f56e38f1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151a1a20 0x561e151b5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.823+0000 7f56e38f1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151a2340 0x561e151ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.823+0000 7f56e38f1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e151acc00 0x561e151adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.823+0000 7f56e38f1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f56c800ce40 con 0x561e151a1a20 2024-09-11T23:29:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.823+0000 7f56e38f1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f56c800e2b0 con 0x561e151acc00 2024-09-11T23:29:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.823+0000 7f56e38f1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f56c800d510 con 0x561e151a2340 2024-09-11T23:29:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.823+0000 7f56e38f1740 10 client.0 _ll_register_callbacks cb 0x7f56d40b8bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.823+0000 7f56daffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151a1a20 0x561e151b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.823+0000 7f56da7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151a2340 0x561e151ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.823+0000 7f56daffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151a1a20 0x561e151b5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59000/0 (socket says 172.21.15.43:59000) 2024-09-11T23:29:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.823+0000 7f56daffd640 1 -- 172.21.15.43:0/2885637536 learned_addr learned my addr 172.21.15.43:0/2885637536 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.823+0000 7f56da7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151a2340 0x561e151ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:41064/0 (socket says 172.21.15.43:41064) 2024-09-11T23:29:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.824+0000 7f56db7fe640 1 --2- 172.21.15.43:0/2885637536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e151acc00 0x561e151adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.824+0000 7f56da7fc640 1 -- 172.21.15.43:0/2885637536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e151acc00 msgr2=0x561e151adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.824+0000 7f56da7fc640 1 --2- 172.21.15.43:0/2885637536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e151acc00 0x561e151adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.824+0000 7f56da7fc640 1 -- 172.21.15.43:0/2885637536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151a1a20 msgr2=0x561e151b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.824+0000 7f56da7fc640 1 --2- 172.21.15.43:0/2885637536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151a1a20 0x561e151b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.824+0000 7f56da7fc640 1 -- 172.21.15.43:0/2885637536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e151ae5d0 con 0x561e151a2340 2024-09-11T23:29:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.824+0000 7f56da7fc640 1 --2- 172.21.15.43:0/2885637536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151a2340 0x561e151ad860 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x561e151a90e0 tx=0x7f56d0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59e8685dea97a7fe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.824+0000 7f56beff5640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-11T23:29:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.824+0000 7f56beff5640 1 -- 172.21.15.43:0/2885637536 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d001a040 con 0x561e151a2340 2024-09-11T23:29:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.825+0000 7f56beff5640 1 -- 172.21.15.43:0/2885637536 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7f56d000f650 con 0x561e151a2340 2024-09-11T23:29:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.825+0000 7f56e38f1740 1 -- 172.21.15.43:0/2885637536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56d40bc880 con 0x561e151a2340 2024-09-11T23:29:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.825+0000 7f56beff5640 1 -- 172.21.15.43:0/2885637536 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f56d001a9a0 con 0x561e151a2340 2024-09-11T23:29:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.825+0000 7f56e38f1740 10 client.26023 fetch_fsmap learned FSMap version 31 2024-09-11T23:29:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.825+0000 7f56e38f1740 1 -- 172.21.15.43:0/2885637536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561e15238850 con 0x561e151a2340 2024-09-11T23:29:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.825+0000 7f56beff5640 1 -- 172.21.15.43:0/2885637536 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56d40bc880 con 0x561e151a2340 2024-09-11T23:29:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.825+0000 7f56beff5640 1 -- 172.21.15.43:0/2885637536 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f56d00047a0 con 0x561e151a2340 2024-09-11T23:29:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.826+0000 7f56e38f1740 10 client.26023 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:29:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.826+0000 7f56e38f1740 10 client.26023 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:29:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.826+0000 7f56e38f1740 20 client.26023 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.826+0000 7f56e38f1740 1 --2- 172.21.15.43:0/2885637536 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561e15238f20 0x561e1523b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.826+0000 7f56e38f1740 4 client.26023 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-11T23:29:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.826+0000 7f56e38f1740 1 -- 172.21.15.43:0/2885637536 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x561e1510dbe0 con 0x561e15238f20 2024-09-11T23:29:03.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.826+0000 7f56daffd640 1 --2- 172.21.15.43:0/2885637536 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561e15238f20 0x561e1523b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:03.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.827+0000 7f56beff5640 10 client.26023 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:03.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:03.827+0000 7f56daffd640 1 --2- 172.21.15.43:0/2885637536 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561e15238f20 0x561e1523b340 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=615ae5d2a7e2d274 server_cookie=b6ac138fdbc32031 in_seq=0 out_seq=0 2024-09-11T23:29:04.260 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:04 smithi043 ceph-mon[30950]: pgmap v220: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 124 KiB/s rd, 13 MiB/s wr, 2.93k op/s 2024-09-11T23:29:04.260 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:04 smithi043 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:29:04.261 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:04 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/4293669427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:29:04.290 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:29:04.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:04 smithi090 ceph-mon[35130]: pgmap v220: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 124 KiB/s rd, 13 MiB/s wr, 2.93k op/s 2024-09-11T23:29:04.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:04 smithi090 ceph-mon[35130]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:29:04.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:04 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/4293669427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:29:04.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:04 smithi097 ceph-mon[35562]: pgmap v220: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 124 KiB/s rd, 13 MiB/s wr, 2.93k op/s 2024-09-11T23:29:04.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:04 smithi097 ceph-mon[35562]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:29:04.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:04 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/4293669427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:29:04.589 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:29:04.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.742+0000 7f56beff5640 1 -- 172.21.15.43:0/2885637536 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561e1510dbe0 con 0x561e15238f20 2024-09-11T23:29:04.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.742+0000 7f56beff5640 10 client.26023 handle_command_reply: tid=0 2024-09-11T23:29:04.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.742+0000 7f56e38f1740 1 client.26023 shutdown 2024-09-11T23:29:04.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.742+0000 7f56e38f1740 1 -- 172.21.15.43:0/2885637536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151a2340 msgr2=0x561e151ad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:04.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.742+0000 7f56e38f1740 1 --2- 172.21.15.43:0/2885637536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151a2340 0x561e151ad860 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x561e151a90e0 tx=0x7f56d0004730 comp rx=0 tx=0).stop 2024-09-11T23:29:04.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.742+0000 7f56e38f1740 1 -- 172.21.15.43:0/2885637536 shutdown_connections 2024-09-11T23:29:04.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.742+0000 7f56e38f1740 1 --2- 172.21.15.43:0/2885637536 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x561e15238f20 0x561e1523b340 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:04.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.742+0000 7f56e38f1740 1 --2- 172.21.15.43:0/2885637536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e151acc00 0x561e151adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:04.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.742+0000 7f56e38f1740 1 --2- 172.21.15.43:0/2885637536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e151a2340 0x561e151ad860 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:04.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.742+0000 7f56e38f1740 1 --2- 172.21.15.43:0/2885637536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561e151a1a20 0x561e151b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:04.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.742+0000 7f56beff5640 0 client.26023 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:04.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.742+0000 7f56e38f1740 1 -- 172.21.15.43:0/2885637536 >> 172.21.15.43:0/2885637536 conn(0x561e1519dfb0 msgr2=0x561e151a9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:04.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.742+0000 7f56e38f1740 1 -- 172.21.15.43:0/2885637536 shutdown_connections 2024-09-11T23:29:04.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.742+0000 7f56e38f1740 1 -- 172.21.15.43:0/2885637536 wait complete. 2024-09-11T23:29:04.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.742+0000 7f56e38f1740 20 client.26023 trim_cache size 0 max 16384 2024-09-11T23:29:04.744 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:29:04.744 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 31, 2024-09-11T23:29:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: "leader": 24497, 2024-09-11T23:29:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: "set_version": 18, 2024-09-11T23:29:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: "sets": { 2024-09-11T23:29:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: "59d4cefc": { 2024-09-11T23:29:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 18, 2024-09-11T23:29:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: "age_ref": 0.0, 2024-09-11T23:29:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:29:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "QUIESCED", 2024-09-11T23:29:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-09-11T23:29:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:29:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: "timeout": 90.0, 2024-09-11T23:29:04.745 INFO:teuthology.orchestra.run.smithi043.stdout: "expiration": 140.2, 2024-09-11T23:29:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: "members": { 2024-09-11T23:29:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: "file:/": { 2024-09-11T23:29:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: "excluded": false, 2024-09-11T23:29:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:29:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "QUIESCED", 2024-09-11T23:29:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-09-11T23:29:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:29:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:29:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:29:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:29:04.746 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:29:04.746 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:29:04.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.743+0000 7f56e1518640 1 -- 172.21.15.43:0/1632444560 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56c40a2df0 msgr2=0x7f56c40a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:04.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.743+0000 7f56e1518640 1 --2- 172.21.15.43:0/1632444560 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56c40a2df0 0x7f56c40a52b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x561e1519f330 tx=0x7f56d000fb50 comp rx=0 tx=0).stop 2024-09-11T23:29:04.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.743+0000 7f56e1518640 1 -- 172.21.15.43:0/1632444560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56dc1138b0 msgr2=0x7f56dc1a2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:04.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.743+0000 7f56e1518640 1 --2- 172.21.15.43:0/1632444560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56dc1138b0 0x7f56dc1a2f70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f56d402a830 tx=0x7f56d400ae80 comp rx=0 tx=0).stop 2024-09-11T23:29:04.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.743+0000 7f56e1518640 1 -- 172.21.15.43:0/1632444560 shutdown_connections 2024-09-11T23:29:04.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.743+0000 7f56e1518640 1 --2- 172.21.15.43:0/1632444560 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56c40a2df0 0x7f56c40a52b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:04.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.743+0000 7f56e1518640 1 --2- 172.21.15.43:0/1632444560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56dc1138b0 0x7f56dc1a2f70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:04.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.743+0000 7f56e1518640 1 --2- 172.21.15.43:0/1632444560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56dc10ac20 0x7f56dc19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:04.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.743+0000 7f56e1518640 1 --2- 172.21.15.43:0/1632444560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56dc10a300 0x7f56dc19e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:04.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.743+0000 7f56e1518640 1 -- 172.21.15.43:0/1632444560 >> 172.21.15.43:0/1632444560 conn(0x7f56dc0735f0 msgr2=0x7f56dc06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:04.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.743+0000 7f56e1518640 1 -- 172.21.15.43:0/1632444560 shutdown_connections 2024-09-11T23:29:04.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:04.743+0000 7f56e1518640 1 -- 172.21.15.43:0/1632444560 wait complete. 2024-09-11T23:29:06.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:06 smithi090 ceph-mon[35130]: pgmap v221: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 244 KiB/s rd, 13 MiB/s wr, 2.94k op/s 2024-09-11T23:29:06.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:06 smithi097 ceph-mon[35562]: pgmap v221: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 244 KiB/s rd, 13 MiB/s wr, 2.94k op/s 2024-09-11T23:29:06.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:06 smithi043 ceph-mon[30950]: pgmap v221: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 244 KiB/s rd, 13 MiB/s wr, 2.94k op/s 2024-09-11T23:29:08.318 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:08 smithi043 ceph-mon[30950]: pgmap v222: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 255 KiB/s rd, 12 MiB/s wr, 2.92k op/s 2024-09-11T23:29:08.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:08 smithi090 ceph-mon[35130]: pgmap v222: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 255 KiB/s rd, 12 MiB/s wr, 2.92k op/s 2024-09-11T23:29:08.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:08 smithi097 ceph-mon[35562]: pgmap v222: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 255 KiB/s rd, 12 MiB/s wr, 2.92k op/s 2024-09-11T23:29:10.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:10 smithi090 ceph-mon[35130]: pgmap v223: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 253 KiB/s rd, 8.5 MiB/s wr, 2k op/s 2024-09-11T23:29:10.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:10 smithi097 ceph-mon[35562]: pgmap v223: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 253 KiB/s rd, 8.5 MiB/s wr, 2k op/s 2024-09-11T23:29:10.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:10 smithi043 ceph-mon[30950]: pgmap v223: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 253 KiB/s rd, 8.5 MiB/s wr, 2k op/s 2024-09-11T23:29:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.651+0000 7fa887f5f640 1 -- 172.21.15.43:0/4056827868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa880107b70 msgr2=0x7fa880107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.651+0000 7fa887f5f640 1 --2- 172.21.15.43:0/4056827868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa880107b70 0x7fa880107f50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa874009930 tx=0x7fa87402a420 comp rx=0 tx=0).stop 2024-09-11T23:29:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.652+0000 7fa887f5f640 1 -- 172.21.15.43:0/4056827868 shutdown_connections 2024-09-11T23:29:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.652+0000 7fa887f5f640 1 --2- 172.21.15.43:0/4056827868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa880102470 0x7fa88010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.652+0000 7fa887f5f640 1 --2- 172.21.15.43:0/4056827868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa880101ad0 0x7fa880101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.653 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.652+0000 7fa887f5f640 1 --2- 172.21.15.43:0/4056827868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa880107b70 0x7fa880107f50 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.652+0000 7fa887f5f640 1 -- 172.21.15.43:0/4056827868 >> 172.21.15.43:0/4056827868 conn(0x7fa8800fd7d0 msgr2=0x7fa8800ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.652+0000 7fa887f5f640 1 -- 172.21.15.43:0/4056827868 shutdown_connections 2024-09-11T23:29:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.652+0000 7fa887f5f640 1 -- 172.21.15.43:0/4056827868 wait complete. 2024-09-11T23:29:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.653+0000 7fa887f5f640 1 Processor -- start 2024-09-11T23:29:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.653+0000 7fa887f5f640 1 -- start start 2024-09-11T23:29:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.653+0000 7fa887f5f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa880101ad0 0x7fa88019d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.653+0000 7fa887f5f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa880102470 0x7fa880195360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.653+0000 7fa887f5f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa880107b70 0x7fa8801958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.653+0000 7fa887f5f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa88010fdc0 con 0x7fa880101ad0 2024-09-11T23:29:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.653+0000 7fa887f5f640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa88010fc40 con 0x7fa880107b70 2024-09-11T23:29:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.653+0000 7fa887f5f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa88010ff40 con 0x7fa880102470 2024-09-11T23:29:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.654+0000 7fa885cfb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa880101ad0 0x7fa88019d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.654+0000 7fa885cfb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa880101ad0 0x7fa88019d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55162/0 (socket says 172.21.15.43:55162) 2024-09-11T23:29:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.654+0000 7fa885cfb640 1 -- 172.21.15.43:0/2148970730 learned_addr learned my addr 172.21.15.43:0/2148970730 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.654+0000 7fa885cfb640 1 -- 172.21.15.43:0/2148970730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa880102470 msgr2=0x7fa880195360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.654+0000 7fa885cfb640 1 --2- 172.21.15.43:0/2148970730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa880102470 0x7fa880195360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.654+0000 7fa885cfb640 1 -- 172.21.15.43:0/2148970730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa880107b70 msgr2=0x7fa8801958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.654+0000 7fa885cfb640 1 --2- 172.21.15.43:0/2148970730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa880107b70 0x7fa8801958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.654+0000 7fa885cfb640 1 -- 172.21.15.43:0/2148970730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8801960c0 con 0x7fa880101ad0 2024-09-11T23:29:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.654+0000 7fa885cfb640 1 --2- 172.21.15.43:0/2148970730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa880101ad0 0x7fa88019d3a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa87402a990 tx=0x7fa874003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:10.657 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.657+0000 7fa872ffd640 1 -- 172.21.15.43:0/2148970730 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87403f070 con 0x7fa880101ad0 2024-09-11T23:29:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.657+0000 7fa887f5f640 1 -- 172.21.15.43:0/2148970730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8801b3900 con 0x7fa880101ad0 2024-09-11T23:29:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.657+0000 7fa887f5f640 1 -- 172.21.15.43:0/2148970730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8801b3df0 con 0x7fa880101ad0 2024-09-11T23:29:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.658+0000 7fa872ffd640 1 -- 172.21.15.43:0/2148970730 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa87403b460 con 0x7fa880101ad0 2024-09-11T23:29:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.658+0000 7fa872ffd640 1 -- 172.21.15.43:0/2148970730 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874035030 con 0x7fa880101ad0 2024-09-11T23:29:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.661+0000 7fa88add7740 1 Processor -- start 2024-09-11T23:29:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.661+0000 7fa872ffd640 1 -- 172.21.15.43:0/2148970730 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa874042070 con 0x7fa880101ad0 2024-09-11T23:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.662+0000 7fa88add7740 1 -- start start 2024-09-11T23:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.662+0000 7fa88add7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee5af0 0x55effaee5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.662+0000 7fa88add7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55effaee6410 0x55effaef0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.662+0000 7fa88add7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaef0cb0 0x55effaef30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.662+0000 7fa88add7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55effae4a2f0 con 0x55effaef0cb0 2024-09-11T23:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.662+0000 7fa88add7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55effadf2270 con 0x55effaee6410 2024-09-11T23:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.662+0000 7fa88add7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55effae462b0 con 0x55effaee5af0 2024-09-11T23:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.662+0000 7fa8864fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee5af0 0x55effaee5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.662+0000 7fa8864fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee5af0 0x55effaee5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:55836/0 (socket says 172.21.15.43:55836) 2024-09-11T23:29:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.662+0000 7fa8864fc640 1 -- 172.21.15.43:0/2925568638 learned_addr learned my addr 172.21.15.43:0/2925568638 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.663+0000 7fa8864fc640 1 -- 172.21.15.43:0/2925568638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55effaee6410 msgr2=0x55effaef0770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:29:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.663+0000 7fa8864fc640 1 --2- 172.21.15.43:0/2925568638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55effaee6410 0x55effaef0770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.663+0000 7fa8864fc640 1 -- 172.21.15.43:0/2925568638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaef0cb0 msgr2=0x55effaef30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.663+0000 7fa8864fc640 1 --2- 172.21.15.43:0/2925568638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaef0cb0 0x55effaef30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.663+0000 7fa8864fc640 1 -- 172.21.15.43:0/2925568638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55effadf1340 con 0x55effaee5af0 2024-09-11T23:29:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.663+0000 7fa872ffd640 1 --2- 172.21.15.43:0/2148970730 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa86c0a2f20 0x7fa86c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.664+0000 7fa872ffd640 1 -- 172.21.15.43:0/2148970730 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7fa8740f0a40 con 0x7fa880101ad0 2024-09-11T23:29:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.664+0000 7fa8864fc640 1 --2- 172.21.15.43:0/2925568638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee5af0 0x55effaee5ed0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa878009d00 tx=0x7fa87800b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64c32061f9fb31ee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.665+0000 7fa86bfff640 1 -- 172.21.15.43:0/2925568638 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87800bd40 con 0x55effaee5af0 2024-09-11T23:29:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.665+0000 7fa86bfff640 1 -- 172.21.15.43:0/2925568638 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa878017070 con 0x55effaee5af0 2024-09-11T23:29:10.666 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.665+0000 7fa86bfff640 1 -- 172.21.15.43:0/2925568638 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878011720 con 0x55effaee5af0 2024-09-11T23:29:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.665+0000 7fa885cfb640 1 --2- 172.21.15.43:0/2148970730 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa86c0a2f20 0x7fa86c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.666+0000 7fa88add7740 1 -- 172.21.15.43:0/2925568638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee5af0 msgr2=0x55effaee5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.666+0000 7fa88add7740 1 --2- 172.21.15.43:0/2925568638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee5af0 0x55effaee5ed0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa878009d00 tx=0x7fa87800b420 comp rx=0 tx=0).stop 2024-09-11T23:29:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.666+0000 7fa885cfb640 1 --2- 172.21.15.43:0/2148970730 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa86c0a2f20 0x7fa86c0a53e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55effaee2ec0 tx=0x7fa874031420 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.666+0000 7fa88add7740 1 -- 172.21.15.43:0/2925568638 shutdown_connections 2024-09-11T23:29:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.666+0000 7fa88add7740 1 --2- 172.21.15.43:0/2925568638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaef0cb0 0x55effaef30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.666+0000 7fa88add7740 1 --2- 172.21.15.43:0/2925568638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55effaee6410 0x55effaef0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.666+0000 7fa88add7740 1 --2- 172.21.15.43:0/2925568638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee5af0 0x55effaee5ed0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.666+0000 7fa88add7740 1 -- 172.21.15.43:0/2925568638 >> 172.21.15.43:0/2925568638 conn(0x55effaee2060 msgr2=0x55effaee2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.667+0000 7fa88add7740 1 -- 172.21.15.43:0/2925568638 shutdown_connections 2024-09-11T23:29:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.667+0000 7fa88add7740 1 -- 172.21.15.43:0/2925568638 wait complete. 2024-09-11T23:29:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.667+0000 7fa88add7740 1 -- start start 2024-09-11T23:29:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.667+0000 7fa88add7740 1 Processor -- start 2024-09-11T23:29:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.668+0000 7fa88add7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55effaee5af0 0x55effaeedb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.668+0000 7fa88add7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee6410 0x55effaeee070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.668+0000 7fa88add7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaef0cb0 0x55effaeee5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.668+0000 7fa88add7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55effaef3880 con 0x55effaef0cb0 2024-09-11T23:29:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.668+0000 7fa88add7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55effadd3290 con 0x55effaee5af0 2024-09-11T23:29:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.668+0000 7fa88add7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55effaef3a00 con 0x55effaee6410 2024-09-11T23:29:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.668+0000 7fa88add7740 10 client.0 _ll_register_callbacks cb 0x55effae04140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.668+0000 7fa8854fa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee6410 0x55effaeee070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.668+0000 7fa8854fa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee6410 0x55effaeee070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:55850/0 (socket says 172.21.15.43:55850) 2024-09-11T23:29:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.668+0000 7fa8854fa640 1 -- 172.21.15.43:0/2641382072 learned_addr learned my addr 172.21.15.43:0/2641382072 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.668+0000 7fa8864fc640 1 --2- 172.21.15.43:0/2641382072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaef0cb0 0x55effaeee5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.668+0000 7fa8864fc640 1 --2- 172.21.15.43:0/2641382072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55effaee5af0 0x55effaeedb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.669+0000 7fa8854fa640 1 -- 172.21.15.43:0/2641382072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55effaee5af0 msgr2=0x55effaeedb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.669+0000 7fa8854fa640 1 --2- 172.21.15.43:0/2641382072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55effaee5af0 0x55effaeedb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.669+0000 7fa8854fa640 1 -- 172.21.15.43:0/2641382072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaef0cb0 msgr2=0x55effaeee5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.669+0000 7fa8854fa640 1 --2- 172.21.15.43:0/2641382072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaef0cb0 0x55effaeee5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.669+0000 7fa8854fa640 1 -- 172.21.15.43:0/2641382072 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55effab5d210 con 0x55effaee6410 2024-09-11T23:29:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.669+0000 7fa8854fa640 1 --2- 172.21.15.43:0/2641382072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee6410 0x55effaeee070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009dc0 tx=0x7fa87c00b4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e4a1fb08fa6d804 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.669+0000 7fa8697fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-11T23:29:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.669+0000 7fa8697fa640 1 -- 172.21.15.43:0/2641382072 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c015020 con 0x55effaee6410 2024-09-11T23:29:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.669+0000 7fa88add7740 1 -- 172.21.15.43:0/2641382072 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa878017070 con 0x55effaee6410 2024-09-11T23:29:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.670+0000 7fa8697fa640 1 -- 172.21.15.43:0/2641382072 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7fa87c011420 con 0x55effaee6410 2024-09-11T23:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.670+0000 7fa8697fa640 1 -- 172.21.15.43:0/2641382072 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa87c0116a0 con 0x55effaee6410 2024-09-11T23:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.670+0000 7fa88add7740 10 client.26029 fetch_fsmap learned FSMap version 31 2024-09-11T23:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.670+0000 7fa88add7740 1 -- 172.21.15.43:0/2641382072 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55effac29620 con 0x55effaee6410 2024-09-11T23:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.670+0000 7fa8697fa640 1 -- 172.21.15.43:0/2641382072 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa878017070 con 0x55effaee6410 2024-09-11T23:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.671+0000 7fa8697fa640 1 -- 172.21.15.43:0/2641382072 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fa87c01a990 con 0x55effaee6410 2024-09-11T23:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.671+0000 7fa88add7740 10 client.26029 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.671+0000 7fa88add7740 10 client.26029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.671+0000 7fa88add7740 20 client.26029 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.671+0000 7fa88add7740 1 --2- 172.21.15.43:0/2641382072 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55effaf01160 0x55effaf035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.671+0000 7fa88add7740 4 client.26029 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.671+0000 7fa88add7740 1 -- 172.21.15.43:0/2641382072 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55effae57a20 con 0x55effaf01160 2024-09-11T23:29:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.671+0000 7fa8864fc640 1 --2- 172.21.15.43:0/2641382072 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55effaf01160 0x55effaf035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.671+0000 7fa8864fc640 1 --2- 172.21.15.43:0/2641382072 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55effaf01160 0x55effaf035a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=170b92f01589a2aa server_cookie=fdd737cd81c8206a in_seq=0 out_seq=0 2024-09-11T23:29:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.671+0000 7fa8697fa640 10 client.26029 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.672+0000 7fa8697fa640 1 -- 172.21.15.43:0/2641382072 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55effae57a20 con 0x55effaf01160 2024-09-11T23:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.672+0000 7fa8697fa640 10 client.26029 handle_command_reply: tid=0 2024-09-11T23:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.673+0000 7fa88add7740 1 client.26029 shutdown 2024-09-11T23:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.673+0000 7fa88add7740 1 -- 172.21.15.43:0/2641382072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee6410 msgr2=0x55effaeee070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.673+0000 7fa88add7740 1 --2- 172.21.15.43:0/2641382072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee6410 0x55effaeee070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009dc0 tx=0x7fa87c00b4b0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.673+0000 7fa88add7740 1 -- 172.21.15.43:0/2641382072 shutdown_connections 2024-09-11T23:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.673+0000 7fa88add7740 1 --2- 172.21.15.43:0/2641382072 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55effaf01160 0x55effaf035a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.673+0000 7fa88add7740 1 --2- 172.21.15.43:0/2641382072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaef0cb0 0x55effaeee5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.673+0000 7fa88add7740 1 --2- 172.21.15.43:0/2641382072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee6410 0x55effaeee070 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.673+0000 7fa88add7740 1 --2- 172.21.15.43:0/2641382072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55effaee5af0 0x55effaeedb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.673+0000 7fa88add7740 1 -- 172.21.15.43:0/2641382072 >> 172.21.15.43:0/2641382072 conn(0x55effaee2060 msgr2=0x55effaee7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.673+0000 7fa8697fa640 0 client.26029 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.673+0000 7fa88add7740 1 -- 172.21.15.43:0/2641382072 shutdown_connections 2024-09-11T23:29:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.673+0000 7fa88add7740 1 -- 172.21.15.43:0/2641382072 wait complete. 2024-09-11T23:29:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.673+0000 7fa88add7740 20 client.26029 trim_cache size 0 max 16384 2024-09-11T23:29:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.688+0000 7fa88add7740 1 Processor -- start 2024-09-11T23:29:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.688+0000 7fa88add7740 1 -- start start 2024-09-11T23:29:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.688+0000 7fa88add7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaee5af0 0x55effaee8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.688+0000 7fa88add7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee6410 0x55effaeee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.688+0000 7fa8864fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaee5af0 0x55effaee8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.689+0000 7fa8864fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaee5af0 0x55effaee8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55180/0 (socket says 172.21.15.43:55180) 2024-09-11T23:29:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.689+0000 7fa8864fc640 1 -- 172.21.15.43:0/43191318 learned_addr learned my addr 172.21.15.43:0/43191318 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.689+0000 7fa88add7740 1 --2- 172.21.15.43:0/43191318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55effaef0cb0 0x55effaf08eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.689+0000 7fa88add7740 1 -- 172.21.15.43:0/43191318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa8640110c0 con 0x55effaee5af0 2024-09-11T23:29:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.689+0000 7fa88add7740 1 -- 172.21.15.43:0/43191318 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa86400e960 con 0x55effaef0cb0 2024-09-11T23:29:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.689+0000 7fa88add7740 1 -- 172.21.15.43:0/43191318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa86400c7b0 con 0x55effaee6410 2024-09-11T23:29:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.689+0000 7fa8864fc640 1 -- 172.21.15.43:0/43191318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee6410 msgr2=0x55effaeee3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.689+0000 7fa8854fa640 1 --2- 172.21.15.43:0/43191318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee6410 0x55effaeee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.689+0000 7fa8864fc640 1 --2- 172.21.15.43:0/43191318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee6410 0x55effaeee3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.689+0000 7fa8864fc640 1 -- 172.21.15.43:0/43191318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55effaef0cb0 msgr2=0x55effaf08eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.689+0000 7fa8864fc640 1 --2- 172.21.15.43:0/43191318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55effaef0cb0 0x55effaf08eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.689+0000 7fa8864fc640 1 -- 172.21.15.43:0/43191318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55effaee3110 con 0x55effaee5af0 2024-09-11T23:29:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.689+0000 7fa8854fa640 1 --2- 172.21.15.43:0/43191318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee6410 0x55effaeee3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:29:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.689+0000 7fa8864fc640 1 --2- 172.21.15.43:0/43191318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaee5af0 0x55effaee8df0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa878009ca0 tx=0x7fa878009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20da5cf63cb1661b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.689+0000 7fa86b7fe640 1 -- 172.21.15.43:0/43191318 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878012860 con 0x55effaee5af0 2024-09-11T23:29:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.689+0000 7fa86b7fe640 1 -- 172.21.15.43:0/43191318 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa878017040 con 0x55effaee5af0 2024-09-11T23:29:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.689+0000 7fa86b7fe640 1 -- 172.21.15.43:0/43191318 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87801c660 con 0x55effaee5af0 2024-09-11T23:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.690+0000 7fa88add7740 1 -- 172.21.15.43:0/43191318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaee5af0 msgr2=0x55effaee8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.690+0000 7fa88add7740 1 --2- 172.21.15.43:0/43191318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaee5af0 0x55effaee8df0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa878009ca0 tx=0x7fa878009690 comp rx=0 tx=0).stop 2024-09-11T23:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.690+0000 7fa88add7740 1 -- 172.21.15.43:0/43191318 shutdown_connections 2024-09-11T23:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.690+0000 7fa88add7740 1 --2- 172.21.15.43:0/43191318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55effaef0cb0 0x55effaf08eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.690+0000 7fa88add7740 1 --2- 172.21.15.43:0/43191318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee6410 0x55effaeee3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.690+0000 7fa88add7740 1 --2- 172.21.15.43:0/43191318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaee5af0 0x55effaee8df0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.690+0000 7fa88add7740 1 -- 172.21.15.43:0/43191318 >> 172.21.15.43:0/43191318 conn(0x55effaee2060 msgr2=0x55effaeefbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.690+0000 7fa88add7740 1 -- 172.21.15.43:0/43191318 shutdown_connections 2024-09-11T23:29:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.690+0000 7fa88add7740 1 -- 172.21.15.43:0/43191318 wait complete. 2024-09-11T23:29:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.691+0000 7fa88add7740 1 -- start start 2024-09-11T23:29:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.691+0000 7fa88add7740 1 Processor -- start 2024-09-11T23:29:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.691+0000 7fa88add7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaee5af0 0x55effaef4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.691+0000 7fa88add7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee6410 0x55effaef4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.691+0000 7fa88add7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55effaef0cb0 0x55effaef5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.691+0000 7fa88add7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa86400ce20 con 0x55effaee5af0 2024-09-11T23:29:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.691+0000 7fa88add7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa86400e290 con 0x55effaef0cb0 2024-09-11T23:29:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.691+0000 7fa88add7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa86400d4f0 con 0x55effaee6410 2024-09-11T23:29:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.691+0000 7fa88add7740 10 client.0 _ll_register_callbacks cb 0x7fa87c006e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.691+0000 7fa8864fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaee5af0 0x55effaef4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.692+0000 7fa8864fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaee5af0 0x55effaef4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55188/0 (socket says 172.21.15.43:55188) 2024-09-11T23:29:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.692+0000 7fa8864fc640 1 -- 172.21.15.43:0/2046488018 learned_addr learned my addr 172.21.15.43:0/2046488018 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.692+0000 7fa8854fa640 1 --2- 172.21.15.43:0/2046488018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee6410 0x55effaef4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.692+0000 7fa8864fc640 1 --2- 172.21.15.43:0/2046488018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55effaef0cb0 0x55effaef5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.692+0000 7fa8864fc640 1 -- 172.21.15.43:0/2046488018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee6410 msgr2=0x55effaef4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.692+0000 7fa8864fc640 1 --2- 172.21.15.43:0/2046488018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee6410 0x55effaef4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.692+0000 7fa8864fc640 1 -- 172.21.15.43:0/2046488018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55effaef0cb0 msgr2=0x55effaef5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.692+0000 7fa8864fc640 1 --2- 172.21.15.43:0/2046488018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55effaef0cb0 0x55effaef5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.692+0000 7fa8864fc640 1 -- 172.21.15.43:0/2046488018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55effaee1e90 con 0x55effaee5af0 2024-09-11T23:29:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.692+0000 7fa8864fc640 1 --2- 172.21.15.43:0/2046488018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaee5af0 0x55effaef4a80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa878012cb0 tx=0x7fa878004b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16be7db10d624125 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.692+0000 7fa8697fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:29:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.692+0000 7fa8697fa640 1 -- 172.21.15.43:0/2046488018 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878036060 con 0x55effaee5af0 2024-09-11T23:29:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.692+0000 7fa88add7740 1 -- 172.21.15.43:0/2046488018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa878017040 con 0x55effaee5af0 2024-09-11T23:29:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.692+0000 7fa8697fa640 1 -- 172.21.15.43:0/2046488018 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8527+0+0 (secure 0 0 0) 0x7fa8780273b0 con 0x55effaee5af0 2024-09-11T23:29:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.692+0000 7fa8697fa640 1 -- 172.21.15.43:0/2046488018 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa8780275b0 con 0x55effaee5af0 2024-09-11T23:29:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.692+0000 7fa88add7740 10 client.16578 fetch_fsmap learned FSMap version 31 2024-09-11T23:29:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.692+0000 7fa88add7740 1 -- 172.21.15.43:0/2046488018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55effaee5920 con 0x55effaee5af0 2024-09-11T23:29:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.693+0000 7fa8697fa640 1 -- 172.21.15.43:0/2046488018 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa878017040 con 0x55effaee5af0 2024-09-11T23:29:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.693+0000 7fa8697fa640 1 -- 172.21.15.43:0/2046488018 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fa878011070 con 0x55effaee5af0 2024-09-11T23:29:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.693+0000 7fa88add7740 10 client.16578 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:29:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.693+0000 7fa88add7740 10 client.16578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.693+0000 7fa88add7740 20 client.16578 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.693+0000 7fa88add7740 1 --2- 172.21.15.43:0/2046488018 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55effaf6c1d0 0x55effaf6e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.693+0000 7fa88add7740 4 client.16578 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:29:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.694+0000 7fa88add7740 1 -- 172.21.15.43:0/2046488018 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55effae51d70 con 0x55effaf6c1d0 2024-09-11T23:29:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.694+0000 7fa8854fa640 1 --2- 172.21.15.43:0/2046488018 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55effaf6c1d0 0x55effaf6e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.694+0000 7fa8854fa640 1 --2- 172.21.15.43:0/2046488018 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55effaf6c1d0 0x55effaf6e5c0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d647e236b98cf530 server_cookie=e39e404e46aa3c29 in_seq=0 out_seq=0 2024-09-11T23:29:10.699 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:10.694+0000 7fa8697fa640 10 client.16578 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.367+0000 7fa8697fa640 1 -- 172.21.15.43:0/2046488018 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55effae51d70 con 0x55effaf6c1d0 2024-09-11T23:29:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.367+0000 7fa8697fa640 10 client.16578 handle_command_reply: tid=0 2024-09-11T23:29:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.367+0000 7fa88add7740 1 client.16578 shutdown 2024-09-11T23:29:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.367+0000 7fa88add7740 1 -- 172.21.15.43:0/2046488018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaee5af0 msgr2=0x55effaef4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.367+0000 7fa88add7740 1 --2- 172.21.15.43:0/2046488018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaee5af0 0x55effaef4a80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa878012cb0 tx=0x7fa878004b70 comp rx=0 tx=0).stop 2024-09-11T23:29:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.367+0000 7fa88add7740 1 -- 172.21.15.43:0/2046488018 shutdown_connections 2024-09-11T23:29:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.367+0000 7fa88add7740 1 --2- 172.21.15.43:0/2046488018 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55effaf6c1d0 0x55effaf6e5c0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.367+0000 7fa88add7740 1 --2- 172.21.15.43:0/2046488018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55effaef0cb0 0x55effaef5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.367+0000 7fa88add7740 1 --2- 172.21.15.43:0/2046488018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55effaee6410 0x55effaef4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.367+0000 7fa8697fa640 0 client.16578 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.367+0000 7fa88add7740 1 --2- 172.21.15.43:0/2046488018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55effaee5af0 0x55effaef4a80 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.367+0000 7fa88add7740 1 -- 172.21.15.43:0/2046488018 >> 172.21.15.43:0/2046488018 conn(0x55effaee2060 msgr2=0x55effaee09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.367+0000 7fa88add7740 1 -- 172.21.15.43:0/2046488018 shutdown_connections 2024-09-11T23:29:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.367+0000 7fa88add7740 1 -- 172.21.15.43:0/2046488018 wait complete. 2024-09-11T23:29:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.368+0000 7fa88add7740 20 client.16578 trim_cache size 0 max 16384 2024-09-11T23:29:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:29:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:29:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:29:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:29:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.368+0000 7fa887f5f640 1 -- 172.21.15.43:0/2148970730 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa86c0a2f20 msgr2=0x7fa86c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.368+0000 7fa887f5f640 1 --2- 172.21.15.43:0/2148970730 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa86c0a2f20 0x7fa86c0a53e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55effaee2ec0 tx=0x7fa874031420 comp rx=0 tx=0).stop 2024-09-11T23:29:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.368+0000 7fa887f5f640 1 -- 172.21.15.43:0/2148970730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa880101ad0 msgr2=0x7fa88019d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.368+0000 7fa887f5f640 1 --2- 172.21.15.43:0/2148970730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa880101ad0 0x7fa88019d3a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa87402a990 tx=0x7fa874003cd0 comp rx=0 tx=0).stop 2024-09-11T23:29:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.368+0000 7fa887f5f640 1 -- 172.21.15.43:0/2148970730 shutdown_connections 2024-09-11T23:29:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.368+0000 7fa887f5f640 1 --2- 172.21.15.43:0/2148970730 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fa86c0a2f20 0x7fa86c0a53e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.368+0000 7fa887f5f640 1 --2- 172.21.15.43:0/2148970730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa880107b70 0x7fa8801958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.368+0000 7fa887f5f640 1 --2- 172.21.15.43:0/2148970730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa880102470 0x7fa880195360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.368+0000 7fa887f5f640 1 --2- 172.21.15.43:0/2148970730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa880101ad0 0x7fa88019d3a0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.368+0000 7fa887f5f640 1 -- 172.21.15.43:0/2148970730 >> 172.21.15.43:0/2148970730 conn(0x7fa8800fd7d0 msgr2=0x7fa8800fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.368+0000 7fa887f5f640 1 -- 172.21.15.43:0/2148970730 shutdown_connections 2024-09-11T23:29:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:11.368+0000 7fa887f5f640 1 -- 172.21.15.43:0/2148970730 wait complete. 2024-09-11T23:29:12.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:12 smithi090 ceph-mon[35130]: pgmap v224: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 265 KiB/s rd, 6.0 MiB/s wr, 1.42k op/s 2024-09-11T23:29:12.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:12 smithi097 ceph-mon[35562]: pgmap v224: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 265 KiB/s rd, 6.0 MiB/s wr, 1.42k op/s 2024-09-11T23:29:12.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:12 smithi043 ceph-mon[30950]: pgmap v224: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 265 KiB/s rd, 6.0 MiB/s wr, 1.42k op/s 2024-09-11T23:29:14.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:14 smithi090 ceph-mon[35130]: pgmap v225: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 157 KiB/s rd, 1.3 MiB/s wr, 343 op/s 2024-09-11T23:29:14.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:14 smithi097 ceph-mon[35562]: pgmap v225: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 157 KiB/s rd, 1.3 MiB/s wr, 343 op/s 2024-09-11T23:29:14.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:14 smithi043 ceph-mon[30950]: pgmap v225: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 157 KiB/s rd, 1.3 MiB/s wr, 343 op/s 2024-09-11T23:29:15.450 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '59d4cefc', quiesce took 15.8 seconds. Will release after: 4.349999999999998 2024-09-11T23:29:16.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:16 smithi090 ceph-mon[35130]: pgmap v226: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 154 KiB/s rd, 143 KiB/s wr, 84 op/s 2024-09-11T23:29:16.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:16 smithi090 ceph-mon[35130]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T23:29:16.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:16 smithi097 ceph-mon[35562]: pgmap v226: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 154 KiB/s rd, 143 KiB/s wr, 84 op/s 2024-09-11T23:29:16.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:16 smithi097 ceph-mon[35562]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T23:29:16.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:16 smithi043 ceph-mon[30950]: pgmap v226: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 154 KiB/s rd, 143 KiB/s wr, 84 op/s 2024-09-11T23:29:16.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:16 smithi043 ceph-mon[30950]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T23:29:18.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.040+0000 7f15fffb1640 1 -- 172.21.15.43:0/4266763302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f81037a0 msgr2=0x7f15f810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:18.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.040+0000 7f15fffb1640 1 --2- 172.21.15.43:0/4266763302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f81037a0 0x7f15f810bc00 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009ca0 tx=0x7f15ec02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.040+0000 7f15fffb1640 1 -- 172.21.15.43:0/4266763302 shutdown_connections 2024-09-11T23:29:18.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.040+0000 7f15fffb1640 1 --2- 172.21.15.43:0/4266763302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15f810c2f0 0x7f15f810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.040+0000 7f15fffb1640 1 --2- 172.21.15.43:0/4266763302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f81037a0 0x7f15f810bc00 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.040+0000 7f15fffb1640 1 --2- 172.21.15.43:0/4266763302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15f8102df0 0x7f15f81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.040+0000 7f15fffb1640 1 -- 172.21.15.43:0/4266763302 >> 172.21.15.43:0/4266763302 conn(0x7f15f80fc7d0 msgr2=0x7f15f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:18.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.040+0000 7f15fffb1640 1 -- 172.21.15.43:0/4266763302 shutdown_connections 2024-09-11T23:29:18.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.040+0000 7f15fffb1640 1 -- 172.21.15.43:0/4266763302 wait complete. 2024-09-11T23:29:18.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.041+0000 7f15fffb1640 1 Processor -- start 2024-09-11T23:29:18.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.041+0000 7f15fffb1640 1 -- start start 2024-09-11T23:29:18.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.041+0000 7f15fffb1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15f8102df0 0x7f15f8100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:18.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.041+0000 7f15fffb1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15f81037a0 0x7f15f8100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:18.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.041+0000 7f15fffb1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f810c2f0 0x7f15f81014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:18.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.041+0000 7f15fffb1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81120d0 con 0x7f15f810c2f0 2024-09-11T23:29:18.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.041+0000 7f15fffb1640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f15f8111f50 con 0x7f15f8102df0 2024-09-11T23:29:18.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.041+0000 7f15fffb1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f15f8112250 con 0x7f15f81037a0 2024-09-11T23:29:18.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.041+0000 7f15fe54e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f810c2f0 0x7f15f81014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:18.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.041+0000 7f15fe54e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f810c2f0 0x7f15f81014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55204/0 (socket says 172.21.15.43:55204) 2024-09-11T23:29:18.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.041+0000 7f15fe54e640 1 -- 172.21.15.43:0/2584655544 learned_addr learned my addr 172.21.15.43:0/2584655544 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:18.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.041+0000 7f15fd54c640 1 --2- 172.21.15.43:0/2584655544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15f81037a0 0x7f15f8100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:18.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.041+0000 7f15fdd4d640 1 --2- 172.21.15.43:0/2584655544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15f8102df0 0x7f15f8100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:18.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.041+0000 7f15fe54e640 1 -- 172.21.15.43:0/2584655544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15f81037a0 msgr2=0x7f15f8100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:18.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.041+0000 7f15fe54e640 1 --2- 172.21.15.43:0/2584655544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15f81037a0 0x7f15f8100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.041+0000 7f15fe54e640 1 -- 172.21.15.43:0/2584655544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15f8102df0 msgr2=0x7f15f8100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:18.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.041+0000 7f15fe54e640 1 --2- 172.21.15.43:0/2584655544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15f8102df0 0x7f15f8100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.041+0000 7f15fe54e640 1 -- 172.21.15.43:0/2584655544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f81a6b00 con 0x7f15f810c2f0 2024-09-11T23:29:18.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.041+0000 7f15fdd4d640 1 --2- 172.21.15.43:0/2584655544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15f8102df0 0x7f15f8100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:18.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.042+0000 7f15fe54e640 1 --2- 172.21.15.43:0/2584655544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f810c2f0 0x7f15f81014d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f15f000e970 tx=0x7f15f000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:18.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.042+0000 7f15f6ffd640 1 -- 172.21.15.43:0/2584655544 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f000ccc0 con 0x7f15f810c2f0 2024-09-11T23:29:18.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.042+0000 7f15f6ffd640 1 -- 172.21.15.43:0/2584655544 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15f000ce60 con 0x7f15f810c2f0 2024-09-11T23:29:18.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.042+0000 7f15fffb1640 1 -- 172.21.15.43:0/2584655544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f81a6df0 con 0x7f15f810c2f0 2024-09-11T23:29:18.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.042+0000 7f15f6ffd640 1 -- 172.21.15.43:0/2584655544 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0010490 con 0x7f15f810c2f0 2024-09-11T23:29:18.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.042+0000 7f15fffb1640 1 -- 172.21.15.43:0/2584655544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f81a72b0 con 0x7f15f810c2f0 2024-09-11T23:29:18.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.044+0000 7f15f6ffd640 1 -- 172.21.15.43:0/2584655544 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f15f0005020 con 0x7f15f810c2f0 2024-09-11T23:29:18.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.044+0000 7f160238a740 1 Processor -- start 2024-09-11T23:29:18.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.045+0000 7f160238a740 1 -- start start 2024-09-11T23:29:18.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.045+0000 7f160238a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd514eb40 0x55fdd514d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:18.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.045+0000 7f160238a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514d5d0 0x55fdd514da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:18.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.045+0000 7f160238a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd5158370 0x55fdd515a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:18.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.045+0000 7f160238a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fdd50b32f0 con 0x55fdd514eb40 2024-09-11T23:29:18.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.045+0000 7f160238a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fdd505acf0 con 0x55fdd5158370 2024-09-11T23:29:18.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.045+0000 7f160238a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55fdd50af2b0 con 0x55fdd514d5d0 2024-09-11T23:29:18.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.045+0000 7f15fd54c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd514eb40 0x55fdd514d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:18.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.045+0000 7f15fd54c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd514eb40 0x55fdd514d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55220/0 (socket says 172.21.15.43:55220) 2024-09-11T23:29:18.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.045+0000 7f15f6ffd640 1 --2- 172.21.15.43:0/2584655544 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f15e00a2cf0 0x7f15e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:18.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.045+0000 7f15fd54c640 1 -- 172.21.15.43:0/1783055408 learned_addr learned my addr 172.21.15.43:0/1783055408 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:18.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.045+0000 7f15fdd4d640 1 --2- 172.21.15.43:0/1783055408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514d5d0 0x55fdd514da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:18.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.045+0000 7f15fd54c640 1 --2- 172.21.15.43:0/1783055408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd5158370 0x55fdd515a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:18.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.045+0000 7f15fe54e640 1 --2- 172.21.15.43:0/2584655544 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f15e00a2cf0 0x7f15e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:18.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.045+0000 7f15fd54c640 1 -- 172.21.15.43:0/1783055408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514d5d0 msgr2=0x55fdd514da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:18.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.045+0000 7f15fd54c640 1 --2- 172.21.15.43:0/1783055408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514d5d0 0x55fdd514da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.045+0000 7f15f6ffd640 1 -- 172.21.15.43:0/2584655544 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8615+0+0 (secure 0 0 0) 0x7f15f00cf9b0 con 0x7f15f810c2f0 2024-09-11T23:29:18.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.045+0000 7f15fd54c640 1 -- 172.21.15.43:0/1783055408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd5158370 msgr2=0x55fdd515a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:18.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.045+0000 7f15fd54c640 1 --2- 172.21.15.43:0/1783055408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd5158370 0x55fdd515a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.045+0000 7f15fd54c640 1 -- 172.21.15.43:0/1783055408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdd503f3a0 con 0x55fdd514eb40 2024-09-11T23:29:18.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.045+0000 7f15fd54c640 1 --2- 172.21.15.43:0/1783055408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd514eb40 0x55fdd514d090 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f15ec005e30 tx=0x7f15ec0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dff22623a6799 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:18.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.046+0000 7f15fe54e640 1 --2- 172.21.15.43:0/2584655544 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f15e00a2cf0 0x7f15e00a51b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f15f009bb10 tx=0x7f15f00912a0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:18.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.046+0000 7f15dffff640 1 -- 172.21.15.43:0/1783055408 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec031450 con 0x55fdd514eb40 2024-09-11T23:29:18.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.046+0000 7f15dffff640 1 -- 172.21.15.43:0/1783055408 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15ec036040 con 0x55fdd514eb40 2024-09-11T23:29:18.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.046+0000 7f15dffff640 1 -- 172.21.15.43:0/1783055408 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec0305c0 con 0x55fdd514eb40 2024-09-11T23:29:18.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.046+0000 7f160238a740 1 -- 172.21.15.43:0/1783055408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd514eb40 msgr2=0x55fdd514d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:18.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.046+0000 7f160238a740 1 --2- 172.21.15.43:0/1783055408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd514eb40 0x55fdd514d090 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f15ec005e30 tx=0x7f15ec0093d0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.046+0000 7f160238a740 1 -- 172.21.15.43:0/1783055408 shutdown_connections 2024-09-11T23:29:18.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.046+0000 7f160238a740 1 --2- 172.21.15.43:0/1783055408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd5158370 0x55fdd515a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.046+0000 7f160238a740 1 --2- 172.21.15.43:0/1783055408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514d5d0 0x55fdd514da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.046+0000 7f160238a740 1 --2- 172.21.15.43:0/1783055408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd514eb40 0x55fdd514d090 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.046+0000 7f160238a740 1 -- 172.21.15.43:0/1783055408 >> 172.21.15.43:0/1783055408 conn(0x55fdd5116310 msgr2=0x55fdd5149c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:18.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.047+0000 7f160238a740 1 -- 172.21.15.43:0/1783055408 shutdown_connections 2024-09-11T23:29:18.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.047+0000 7f160238a740 1 -- 172.21.15.43:0/1783055408 wait complete. 2024-09-11T23:29:18.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.047+0000 7f160238a740 1 -- start start 2024-09-11T23:29:18.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.047+0000 7f160238a740 1 Processor -- start 2024-09-11T23:29:18.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.047+0000 7f160238a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514d5d0 0x55fdd5157d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:18.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.047+0000 7f160238a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd514eb40 0x55fdd5155010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:18.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.047+0000 7f160238a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd5158370 0x55fdd5155600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:18.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.047+0000 7f160238a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fdd515ae90 con 0x55fdd5158370 2024-09-11T23:29:18.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.047+0000 7f160238a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fdd506e060 con 0x55fdd514eb40 2024-09-11T23:29:18.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.047+0000 7f160238a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55fdd515b010 con 0x55fdd514d5d0 2024-09-11T23:29:18.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.047+0000 7f160238a740 10 client.0 _ll_register_callbacks cb 0x55fdd4dc18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:18.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.048+0000 7f15fdd4d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd514eb40 0x55fdd5155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:18.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.048+0000 7f15fd54c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd5158370 0x55fdd5155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:18.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.048+0000 7f15fdd4d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd514eb40 0x55fdd5155010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:49840/0 (socket says 172.21.15.43:49840) 2024-09-11T23:29:18.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.048+0000 7f15fd54c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd5158370 0x55fdd5155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55224/0 (socket says 172.21.15.43:55224) 2024-09-11T23:29:18.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.048+0000 7f15fdd4d640 1 -- 172.21.15.43:0/2174431661 learned_addr learned my addr 172.21.15.43:0/2174431661 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:18.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.048+0000 7f15fd54c640 1 --2- 172.21.15.43:0/2174431661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514d5d0 0x55fdd5157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:18.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.048+0000 7f15fdd4d640 1 -- 172.21.15.43:0/2174431661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514d5d0 msgr2=0x55fdd5157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:18.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.048+0000 7f15fdd4d640 1 --2- 172.21.15.43:0/2174431661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514d5d0 0x55fdd5157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.048+0000 7f15fdd4d640 1 -- 172.21.15.43:0/2174431661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd5158370 msgr2=0x55fdd5155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:18.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.048+0000 7f15fdd4d640 1 --2- 172.21.15.43:0/2174431661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd5158370 0x55fdd5155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.048+0000 7f15fdd4d640 1 -- 172.21.15.43:0/2174431661 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdd4e243c0 con 0x55fdd514eb40 2024-09-11T23:29:18.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.048+0000 7f15fd54c640 1 --2- 172.21.15.43:0/2174431661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514d5d0 0x55fdd5157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:18.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.048+0000 7f15fdd4d640 1 --2- 172.21.15.43:0/2174431661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd514eb40 0x55fdd5155010 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f15e8013760 tx=0x7f15e8013c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87440c12ebdb8cc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:18.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.048+0000 7f15dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:29:18.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.048+0000 7f15dd7fa640 1 -- 172.21.15.43:0/2174431661 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8020060 con 0x55fdd514eb40 2024-09-11T23:29:18.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.048+0000 7f15dd7fa640 1 -- 172.21.15.43:0/2174431661 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8615+0+0 (secure 0 0 0) 0x7f15e8011e30 con 0x55fdd514eb40 2024-09-11T23:29:18.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.048+0000 7f160238a740 1 -- 172.21.15.43:0/2174431661 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15ec036040 con 0x55fdd514eb40 2024-09-11T23:29:18.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.048+0000 7f15dd7fa640 1 -- 172.21.15.43:0/2174431661 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f15e8023410 con 0x55fdd514eb40 2024-09-11T23:29:18.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.048+0000 7f160238a740 10 client.26087 fetch_fsmap learned FSMap version 31 2024-09-11T23:29:18.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.048+0000 7f160238a740 1 -- 172.21.15.43:0/2174431661 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fdd51669a0 con 0x55fdd514eb40 2024-09-11T23:29:18.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.049+0000 7f15dd7fa640 1 -- 172.21.15.43:0/2174431661 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15ec036040 con 0x55fdd514eb40 2024-09-11T23:29:18.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.049+0000 7f15dd7fa640 1 -- 172.21.15.43:0/2174431661 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f15e801a030 con 0x55fdd514eb40 2024-09-11T23:29:18.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.049+0000 7f160238a740 10 client.26087 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:29:18.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.049+0000 7f160238a740 10 client.26087 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T23:29:18.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.049+0000 7f160238a740 20 client.26087 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:18.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.049+0000 7f160238a740 1 --2- 172.21.15.43:0/2174431661 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55fdd5167eb0 0x55fdd516a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:18.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.049+0000 7f160238a740 4 client.26087 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:29:18.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.049+0000 7f160238a740 1 -- 172.21.15.43:0/2174431661 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdd50c0860 con 0x55fdd5167eb0 2024-09-11T23:29:18.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.049+0000 7f15fd54c640 1 --2- 172.21.15.43:0/2174431661 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55fdd5167eb0 0x55fdd516a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:18.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.050+0000 7f15dd7fa640 10 client.26087 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:29:18.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.050+0000 7f15fd54c640 1 --2- 172.21.15.43:0/2174431661 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55fdd5167eb0 0x55fdd516a2a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2388455d80e38d46 server_cookie=ff6d1696a81d706 in_seq=0 out_seq=0 2024-09-11T23:29:18.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.050+0000 7f15dd7fa640 1 -- 172.21.15.43:0/2174431661 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fdd50c0860 con 0x55fdd5167eb0 2024-09-11T23:29:18.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.050+0000 7f15dd7fa640 10 client.26087 handle_command_reply: tid=0 2024-09-11T23:29:18.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.050+0000 7f160238a740 1 client.26087 shutdown 2024-09-11T23:29:18.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.051+0000 7f160238a740 1 -- 172.21.15.43:0/2174431661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd514eb40 msgr2=0x55fdd5155010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:18.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.051+0000 7f160238a740 1 --2- 172.21.15.43:0/2174431661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd514eb40 0x55fdd5155010 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f15e8013760 tx=0x7f15e8013c20 comp rx=0 tx=0).stop 2024-09-11T23:29:18.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.051+0000 7f160238a740 1 -- 172.21.15.43:0/2174431661 shutdown_connections 2024-09-11T23:29:18.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.051+0000 7f160238a740 1 --2- 172.21.15.43:0/2174431661 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55fdd5167eb0 0x55fdd516a2a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.051+0000 7f160238a740 1 --2- 172.21.15.43:0/2174431661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd5158370 0x55fdd5155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.051+0000 7f160238a740 1 --2- 172.21.15.43:0/2174431661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd514eb40 0x55fdd5155010 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.051+0000 7f15dd7fa640 0 client.26087 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:29:18.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.051+0000 7f160238a740 1 --2- 172.21.15.43:0/2174431661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514d5d0 0x55fdd5157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.051+0000 7f160238a740 1 -- 172.21.15.43:0/2174431661 >> 172.21.15.43:0/2174431661 conn(0x55fdd5116310 msgr2=0x55fdd514f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:18.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.051+0000 7f160238a740 1 -- 172.21.15.43:0/2174431661 shutdown_connections 2024-09-11T23:29:18.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.051+0000 7f160238a740 1 -- 172.21.15.43:0/2174431661 wait complete. 2024-09-11T23:29:18.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.051+0000 7f160238a740 20 client.26087 trim_cache size 0 max 16384 2024-09-11T23:29:18.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.068+0000 7f160238a740 1 Processor -- start 2024-09-11T23:29:18.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.068+0000 7f160238a740 1 -- start start 2024-09-11T23:29:18.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.069+0000 7f160238a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514d5d0 0x55fdd5154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:18.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.069+0000 7f160238a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd514eb40 0x55fdd5164160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:18.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.069+0000 7f160238a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd5158370 0x55fdd516fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:18.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.069+0000 7f160238a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81b24f0 con 0x55fdd514eb40 2024-09-11T23:29:18.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.069+0000 7f160238a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81afd90 con 0x55fdd5158370 2024-09-11T23:29:18.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.069+0000 7f160238a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81adc00 con 0x55fdd514d5d0 2024-09-11T23:29:18.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.071+0000 7f15fd54c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514d5d0 0x55fdd5154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:18.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.071+0000 7f15fd54c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd5158370 0x55fdd516fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:18.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.071+0000 7f15fd54c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514d5d0 0x55fdd5154fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:55902/0 (socket says 172.21.15.43:55902) 2024-09-11T23:29:18.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.071+0000 7f15fd54c640 1 -- 172.21.15.43:0/525989712 learned_addr learned my addr 172.21.15.43:0/525989712 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:18.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.072+0000 7f15fdd4d640 1 --2- 172.21.15.43:0/525989712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd514eb40 0x55fdd5164160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:18.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.072+0000 7f15fdd4d640 1 -- 172.21.15.43:0/525989712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514d5d0 msgr2=0x55fdd5154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:18.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.072+0000 7f15fdd4d640 1 --2- 172.21.15.43:0/525989712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514d5d0 0x55fdd5154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.072+0000 7f15fdd4d640 1 -- 172.21.15.43:0/525989712 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd5158370 msgr2=0x55fdd516fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:18.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.072+0000 7f15fdd4d640 1 --2- 172.21.15.43:0/525989712 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd5158370 0x55fdd516fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.072+0000 7f15fdd4d640 1 -- 172.21.15.43:0/525989712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdd5150120 con 0x55fdd514eb40 2024-09-11T23:29:18.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.073+0000 7f15fdd4d640 1 --2- 172.21.15.43:0/525989712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd514eb40 0x55fdd5164160 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x55fdd51565c0 tx=0x7f15e8009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb99bb9565718dee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:18.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.073+0000 7f15df7fe640 1 -- 172.21.15.43:0/525989712 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8020070 con 0x55fdd514eb40 2024-09-11T23:29:18.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.073+0000 7f15df7fe640 1 -- 172.21.15.43:0/525989712 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15e8009680 con 0x55fdd514eb40 2024-09-11T23:29:18.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.074+0000 7f160238a740 1 -- 172.21.15.43:0/525989712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd514eb40 msgr2=0x55fdd5164160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:18.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.074+0000 7f160238a740 1 --2- 172.21.15.43:0/525989712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd514eb40 0x55fdd5164160 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x55fdd51565c0 tx=0x7f15e8009ef0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.077+0000 7f160238a740 1 -- 172.21.15.43:0/525989712 shutdown_connections 2024-09-11T23:29:18.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.077+0000 7f160238a740 1 --2- 172.21.15.43:0/525989712 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd5158370 0x55fdd516fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.077+0000 7f160238a740 1 --2- 172.21.15.43:0/525989712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd514eb40 0x55fdd5164160 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.077+0000 7f160238a740 1 --2- 172.21.15.43:0/525989712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514d5d0 0x55fdd5154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.077+0000 7f160238a740 1 -- 172.21.15.43:0/525989712 >> 172.21.15.43:0/525989712 conn(0x55fdd5116310 msgr2=0x55fdd51548d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:18.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.078+0000 7f160238a740 1 -- 172.21.15.43:0/525989712 shutdown_connections 2024-09-11T23:29:18.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.078+0000 7f160238a740 1 -- 172.21.15.43:0/525989712 wait complete. 2024-09-11T23:29:18.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.078+0000 7f160238a740 1 -- start start 2024-09-11T23:29:18.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.078+0000 7f160238a740 1 Processor -- start 2024-09-11T23:29:18.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.079+0000 7f160238a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd514d5d0 0x55fdd51520e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:18.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.079+0000 7f160238a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514eb40 0x55fdd5152620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:18.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.079+0000 7f160238a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd5158370 0x55fdd5152b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:18.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.079+0000 7f160238a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81ae250 con 0x55fdd514d5d0 2024-09-11T23:29:18.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.079+0000 7f160238a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81af6c0 con 0x55fdd5158370 2024-09-11T23:29:18.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.079+0000 7f160238a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81ae920 con 0x55fdd514eb40 2024-09-11T23:29:18.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.079+0000 7f160238a740 10 client.0 _ll_register_callbacks cb 0x7f15e8009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:18.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.081+0000 7f15fdd4d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514eb40 0x55fdd5152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:18.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.082+0000 7f15fd54c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd5158370 0x55fdd5152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:18.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.082+0000 7f15fd54c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd514d5d0 0x55fdd51520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:18.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.082+0000 7f15fd54c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd514d5d0 0x55fdd51520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55230/0 (socket says 172.21.15.43:55230) 2024-09-11T23:29:18.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.082+0000 7f15fdd4d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514eb40 0x55fdd5152620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:55918/0 (socket says 172.21.15.43:55918) 2024-09-11T23:29:18.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.082+0000 7f15fd54c640 1 -- 172.21.15.43:0/62290187 learned_addr learned my addr 172.21.15.43:0/62290187 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:18.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.082+0000 7f15fdd4d640 1 -- 172.21.15.43:0/62290187 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd5158370 msgr2=0x55fdd5152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:18.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.082+0000 7f15fdd4d640 1 --2- 172.21.15.43:0/62290187 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd5158370 0x55fdd5152b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.082+0000 7f15fdd4d640 1 -- 172.21.15.43:0/62290187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd514d5d0 msgr2=0x55fdd51520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:18.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.082+0000 7f15fdd4d640 1 --2- 172.21.15.43:0/62290187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd514d5d0 0x55fdd51520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.082+0000 7f15fdd4d640 1 -- 172.21.15.43:0/62290187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdd51646a0 con 0x55fdd514eb40 2024-09-11T23:29:18.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.084+0000 7f15fdd4d640 1 --2- 172.21.15.43:0/62290187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514eb40 0x55fdd5152620 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f15e8013730 tx=0x7f15e8010f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a95a961a4f02731 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:18.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.084+0000 7f15dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-11T23:29:18.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.085+0000 7f15dd7fa640 1 -- 172.21.15.43:0/62290187 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8020070 con 0x55fdd514eb40 2024-09-11T23:29:18.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.085+0000 7f15dd7fa640 1 -- 172.21.15.43:0/62290187 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8615+0+0 (secure 0 0 0) 0x7f15e8005030 con 0x55fdd514eb40 2024-09-11T23:29:18.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.085+0000 7f160238a740 1 -- 172.21.15.43:0/62290187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15e8009680 con 0x55fdd514eb40 2024-09-11T23:29:18.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.086+0000 7f15dd7fa640 1 -- 172.21.15.43:0/62290187 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f15e8005230 con 0x55fdd514eb40 2024-09-11T23:29:18.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.086+0000 7f160238a740 10 client.26050 fetch_fsmap learned FSMap version 31 2024-09-11T23:29:18.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.086+0000 7f160238a740 1 -- 172.21.15.43:0/62290187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fdd51dcce0 con 0x55fdd514eb40 2024-09-11T23:29:18.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.087+0000 7f15dd7fa640 1 -- 172.21.15.43:0/62290187 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15e8009680 con 0x55fdd514eb40 2024-09-11T23:29:18.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.087+0000 7f15dd7fa640 1 -- 172.21.15.43:0/62290187 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f15e801a040 con 0x55fdd514eb40 2024-09-11T23:29:18.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.087+0000 7f160238a740 10 client.26050 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:29:18.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.087+0000 7f160238a740 10 client.26050 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T23:29:18.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.087+0000 7f160238a740 20 client.26050 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:18.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.087+0000 7f160238a740 1 --2- 172.21.15.43:0/62290187 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55fdd51dd440 0x55fdd51df860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:18.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.087+0000 7f160238a740 4 client.26050 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:29:18.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.087+0000 7f160238a740 1 -- 172.21.15.43:0/62290187 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fdd50bad70 con 0x55fdd51dd440 2024-09-11T23:29:18.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.088+0000 7f15fd54c640 1 --2- 172.21.15.43:0/62290187 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55fdd51dd440 0x55fdd51df860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:18.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.089+0000 7f15fd54c640 1 --2- 172.21.15.43:0/62290187 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55fdd51dd440 0x55fdd51df860 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3541c4478c740d18 server_cookie=9bc0c052a8247c4d in_seq=0 out_seq=0 2024-09-11T23:29:18.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.090+0000 7f15dd7fa640 10 client.26050 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:29:18.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.090+0000 7f15dd7fa640 1 -- 172.21.15.43:0/62290187 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fdd50bad70 con 0x55fdd51dd440 2024-09-11T23:29:18.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.090+0000 7f15dd7fa640 10 client.26050 handle_command_reply: tid=0 2024-09-11T23:29:18.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.090+0000 7f160238a740 1 client.26050 shutdown 2024-09-11T23:29:18.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.091+0000 7f160238a740 1 -- 172.21.15.43:0/62290187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514eb40 msgr2=0x55fdd5152620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:18.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.091+0000 7f160238a740 1 --2- 172.21.15.43:0/62290187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514eb40 0x55fdd5152620 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f15e8013730 tx=0x7f15e8010f60 comp rx=0 tx=0).stop 2024-09-11T23:29:18.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.091+0000 7f160238a740 1 -- 172.21.15.43:0/62290187 shutdown_connections 2024-09-11T23:29:18.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.091+0000 7f160238a740 1 --2- 172.21.15.43:0/62290187 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55fdd51dd440 0x55fdd51df860 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.091+0000 7f160238a740 1 --2- 172.21.15.43:0/62290187 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fdd5158370 0x55fdd5152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.091+0000 7f160238a740 1 --2- 172.21.15.43:0/62290187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fdd514eb40 0x55fdd5152620 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.091+0000 7f160238a740 1 --2- 172.21.15.43:0/62290187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdd514d5d0 0x55fdd51520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.091+0000 7f160238a740 1 -- 172.21.15.43:0/62290187 >> 172.21.15.43:0/62290187 conn(0x55fdd5116310 msgr2=0x55fdd5157090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:18.235 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:29:18.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.091+0000 7f15dd7fa640 0 client.26050 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:29:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.091+0000 7f160238a740 1 -- 172.21.15.43:0/62290187 shutdown_connections 2024-09-11T23:29:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.092+0000 7f160238a740 1 -- 172.21.15.43:0/62290187 wait complete. 2024-09-11T23:29:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.092+0000 7f160238a740 20 client.26050 trim_cache size 0 max 16384 2024-09-11T23:29:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.093+0000 7f15fffb1640 1 -- 172.21.15.43:0/2584655544 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f15e00a2cf0 msgr2=0x7f15e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.093+0000 7f15fffb1640 1 --2- 172.21.15.43:0/2584655544 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f15e00a2cf0 0x7f15e00a51b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f15f009bb10 tx=0x7f15f00912a0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.093+0000 7f15fffb1640 1 -- 172.21.15.43:0/2584655544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f810c2f0 msgr2=0x7f15f81014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.093+0000 7f15fffb1640 1 --2- 172.21.15.43:0/2584655544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f810c2f0 0x7f15f81014d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f15f000e970 tx=0x7f15f000ed70 comp rx=0 tx=0).stop 2024-09-11T23:29:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.093+0000 7f15fffb1640 1 -- 172.21.15.43:0/2584655544 shutdown_connections 2024-09-11T23:29:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.093+0000 7f15fffb1640 1 --2- 172.21.15.43:0/2584655544 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f15e00a2cf0 0x7f15e00a51b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.093+0000 7f15fffb1640 1 --2- 172.21.15.43:0/2584655544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f810c2f0 0x7f15f81014d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.093+0000 7f15fffb1640 1 --2- 172.21.15.43:0/2584655544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15f81037a0 0x7f15f8100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.093+0000 7f15fffb1640 1 --2- 172.21.15.43:0/2584655544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15f8102df0 0x7f15f8100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:18.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.093+0000 7f15fffb1640 1 -- 172.21.15.43:0/2584655544 >> 172.21.15.43:0/2584655544 conn(0x7f15f80fc7d0 msgr2=0x7f15f80fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:18.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.093+0000 7f15fffb1640 1 -- 172.21.15.43:0/2584655544 shutdown_connections 2024-09-11T23:29:18.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:17.094+0000 7f15fffb1640 1 -- 172.21.15.43:0/2584655544 wait complete. 2024-09-11T23:29:18.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:18 smithi090 ceph-mon[35130]: pgmap v228: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 30 KiB/s rd, 74 KiB/s wr, 115 op/s 2024-09-11T23:29:18.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:18 smithi097 ceph-mon[35562]: pgmap v228: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 30 KiB/s rd, 74 KiB/s wr, 115 op/s 2024-09-11T23:29:18.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:18 smithi043 ceph-mon[30950]: pgmap v228: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 30 KiB/s rd, 74 KiB/s wr, 115 op/s 2024-09-11T23:29:19.801 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:29:20.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:20 smithi090 ceph-mon[35130]: pgmap v229: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 30 KiB/s rd, 74 KiB/s wr, 115 op/s 2024-09-11T23:29:20.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:20 smithi097 ceph-mon[35562]: pgmap v229: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 30 KiB/s rd, 74 KiB/s wr, 115 op/s 2024-09-11T23:29:20.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:20 smithi043 ceph-mon[30950]: pgmap v229: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 30 KiB/s rd, 74 KiB/s wr, 115 op/s 2024-09-11T23:29:22.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:22 smithi090 ceph-mon[35130]: pgmap v230: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 17 KiB/s rd, 41 KiB/s wr, 109 op/s 2024-09-11T23:29:22.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:22 smithi097 ceph-mon[35562]: pgmap v230: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 17 KiB/s rd, 41 KiB/s wr, 109 op/s 2024-09-11T23:29:22.606 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:22 smithi043 ceph-mon[30950]: pgmap v230: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 17 KiB/s rd, 41 KiB/s wr, 109 op/s 2024-09-11T23:29:23.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:23 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:23.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:23 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:23.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:23 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:29:24.470 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.468+0000 7f5b681d6640 1 -- 172.21.15.43:0/3228233108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60102050 msgr2=0x7f5b6010d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.468+0000 7f5b681d6640 1 --2- 172.21.15.43:0/3228233108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60102050 0x7f5b6010d7d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5b58009ca0 tx=0x7f5b5802a950 comp rx=0 tx=0).stop 2024-09-11T23:29:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.468+0000 7f5b681d6640 1 -- 172.21.15.43:0/3228233108 shutdown_connections 2024-09-11T23:29:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.468+0000 7f5b681d6640 1 --2- 172.21.15.43:0/3228233108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60102050 0x7f5b6010d7d0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.468+0000 7f5b681d6640 1 --2- 172.21.15.43:0/3228233108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b601016d0 0x7f5b60101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.468+0000 7f5b681d6640 1 --2- 172.21.15.43:0/3228233108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b60105af0 0x7f5b60105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.468+0000 7f5b681d6640 1 -- 172.21.15.43:0/3228233108 >> 172.21.15.43:0/3228233108 conn(0x7f5b60075f20 msgr2=0x7f5b60076330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.468+0000 7f5b681d6640 1 -- 172.21.15.43:0/3228233108 shutdown_connections 2024-09-11T23:29:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.468+0000 7f5b681d6640 1 -- 172.21.15.43:0/3228233108 wait complete. 2024-09-11T23:29:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.468+0000 7f5b681d6640 1 Processor -- start 2024-09-11T23:29:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.468+0000 7f5b681d6640 1 -- start start 2024-09-11T23:29:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.469+0000 7f5b681d6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b601016d0 0x7f5b60198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.469+0000 7f5b681d6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b60102050 0x7f5b60198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.469+0000 7f5b681d6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60105af0 0x7f5b6019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.469+0000 7f5b681d6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b60110e30 con 0x7f5b60105af0 2024-09-11T23:29:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.469+0000 7f5b681d6640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5b60110cb0 con 0x7f5b601016d0 2024-09-11T23:29:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.469+0000 7f5b681d6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5b60110fb0 con 0x7f5b60102050 2024-09-11T23:29:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.469+0000 7f5b66773640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60105af0 0x7f5b6019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.469+0000 7f5b66773640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60105af0 0x7f5b6019cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56842/0 (socket says 172.21.15.43:56842) 2024-09-11T23:29:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.469+0000 7f5b66773640 1 -- 172.21.15.43:0/2533607341 learned_addr learned my addr 172.21.15.43:0/2533607341 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.469+0000 7f5b65771640 1 --2- 172.21.15.43:0/2533607341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b60102050 0x7f5b60198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.469+0000 7f5b65f72640 1 --2- 172.21.15.43:0/2533607341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b601016d0 0x7f5b60198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.469+0000 7f5b66773640 1 -- 172.21.15.43:0/2533607341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b60102050 msgr2=0x7f5b60198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.469+0000 7f5b66773640 1 --2- 172.21.15.43:0/2533607341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b60102050 0x7f5b60198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.469+0000 7f5b66773640 1 -- 172.21.15.43:0/2533607341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b601016d0 msgr2=0x7f5b60198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.469+0000 7f5b66773640 1 --2- 172.21.15.43:0/2533607341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b601016d0 0x7f5b60198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.469+0000 7f5b66773640 1 -- 172.21.15.43:0/2533607341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6019d300 con 0x7f5b60105af0 2024-09-11T23:29:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.469+0000 7f5b66773640 1 --2- 172.21.15.43:0/2533607341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60105af0 0x7f5b6019cbd0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5b58009dc0 tx=0x7f5b58003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.469+0000 7f5b56ffd640 1 -- 172.21.15.43:0/2533607341 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58034040 con 0x7f5b60105af0 2024-09-11T23:29:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.470+0000 7f5b56ffd640 1 -- 172.21.15.43:0/2533607341 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b5802fe20 con 0x7f5b60105af0 2024-09-11T23:29:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.470+0000 7f5b681d6640 1 -- 172.21.15.43:0/2533607341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6019d590 con 0x7f5b60105af0 2024-09-11T23:29:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.470+0000 7f5b56ffd640 1 -- 172.21.15.43:0/2533607341 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b580047f0 con 0x7f5b60105af0 2024-09-11T23:29:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.470+0000 7f5b681d6640 1 -- 172.21.15.43:0/2533607341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b601a4e80 con 0x7f5b60105af0 2024-09-11T23:29:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.472+0000 7f5b56ffd640 1 -- 172.21.15.43:0/2533607341 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5b58037020 con 0x7f5b60105af0 2024-09-11T23:29:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.472+0000 7f5b6b051740 1 Processor -- start 2024-09-11T23:29:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.472+0000 7f5b6b051740 1 -- start start 2024-09-11T23:29:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.472+0000 7f5b6b051740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257ea6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.472+0000 7f5b6b051740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f257ea72f0 0x55f257eb5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.472+0000 7f5b6b051740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f257eb60c0 0x55f257eb84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.473+0000 7f5b6b051740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f257e0b2f0 con 0x55f257ea69d0 2024-09-11T23:29:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.473+0000 7f5b6b051740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f257db3270 con 0x55f257eb60c0 2024-09-11T23:29:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.473+0000 7f5b6b051740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f257e072b0 con 0x55f257ea72f0 2024-09-11T23:29:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.473+0000 7f5b65771640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257ea6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.473+0000 7f5b56ffd640 1 --2- 172.21.15.43:0/2533607341 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5b4c0a2d70 0x7f5b4c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.473+0000 7f5b65771640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257ea6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56854/0 (socket says 172.21.15.43:56854) 2024-09-11T23:29:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.473+0000 7f5b65771640 1 -- 172.21.15.43:0/4270428673 learned_addr learned my addr 172.21.15.43:0/4270428673 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.473+0000 7f5b65f72640 1 --2- 172.21.15.43:0/4270428673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f257ea72f0 0x55f257eb5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.473+0000 7f5b66773640 1 --2- 172.21.15.43:0/2533607341 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5b4c0a2d70 0x7f5b4c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.473+0000 7f5b56ffd640 1 -- 172.21.15.43:0/2533607341 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f5b580f0170 con 0x7f5b60105af0 2024-09-11T23:29:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.473+0000 7f5b65771640 1 --2- 172.21.15.43:0/4270428673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f257eb60c0 0x55f257eb84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.473+0000 7f5b65771640 1 -- 172.21.15.43:0/4270428673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f257ea72f0 msgr2=0x55f257eb5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.473+0000 7f5b65771640 1 --2- 172.21.15.43:0/4270428673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f257ea72f0 0x55f257eb5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.473+0000 7f5b65771640 1 -- 172.21.15.43:0/4270428673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f257eb60c0 msgr2=0x55f257eb84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.473+0000 7f5b65771640 1 --2- 172.21.15.43:0/4270428673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f257eb60c0 0x55f257eb84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.473+0000 7f5b65771640 1 -- 172.21.15.43:0/4270428673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f257db2340 con 0x55f257ea69d0 2024-09-11T23:29:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.473+0000 7f5b65f72640 1 --2- 172.21.15.43:0/4270428673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f257ea72f0 0x55f257eb5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:29:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.473+0000 7f5b65771640 1 --2- 172.21.15.43:0/4270428673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257ea6db0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5b5000b8f0 tx=0x7f5b50004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a63c50a6eac13ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.473+0000 7f5b4bfff640 1 -- 172.21.15.43:0/4270428673 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50003d70 con 0x55f257ea69d0 2024-09-11T23:29:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.473+0000 7f5b66773640 1 --2- 172.21.15.43:0/2533607341 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5b4c0a2d70 0x7f5b4c0a5230 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5b58006010 tx=0x7f5b5808e730 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.474+0000 7f5b4bfff640 1 -- 172.21.15.43:0/4270428673 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b50005500 con 0x55f257ea69d0 2024-09-11T23:29:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.474+0000 7f5b4bfff640 1 -- 172.21.15.43:0/4270428673 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b500106c0 con 0x55f257ea69d0 2024-09-11T23:29:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.474+0000 7f5b6b051740 1 -- 172.21.15.43:0/4270428673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 msgr2=0x55f257ea6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.474+0000 7f5b6b051740 1 --2- 172.21.15.43:0/4270428673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257ea6db0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5b5000b8f0 tx=0x7f5b50004250 comp rx=0 tx=0).stop 2024-09-11T23:29:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.474+0000 7f5b6b051740 1 -- 172.21.15.43:0/4270428673 shutdown_connections 2024-09-11T23:29:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.474+0000 7f5b6b051740 1 --2- 172.21.15.43:0/4270428673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f257eb60c0 0x55f257eb84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.474+0000 7f5b6b051740 1 --2- 172.21.15.43:0/4270428673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f257ea72f0 0x55f257eb5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.474+0000 7f5b6b051740 1 --2- 172.21.15.43:0/4270428673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257ea6db0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.474+0000 7f5b6b051740 1 -- 172.21.15.43:0/4270428673 >> 172.21.15.43:0/4270428673 conn(0x55f257ea3060 msgr2=0x55f257ea3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.474+0000 7f5b6b051740 1 -- 172.21.15.43:0/4270428673 shutdown_connections 2024-09-11T23:29:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.474+0000 7f5b6b051740 1 -- 172.21.15.43:0/4270428673 wait complete. 2024-09-11T23:29:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.475+0000 7f5b6b051740 1 -- start start 2024-09-11T23:29:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.475+0000 7f5b6b051740 1 Processor -- start 2024-09-11T23:29:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.475+0000 7f5b6b051740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257eb2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.475+0000 7f5b6b051740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f257ea72f0 0x55f257eb3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.475+0000 7f5b6b051740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f257eb60c0 0x55f257eb3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.475+0000 7f5b6b051740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f257eb8cf0 con 0x55f257ea69d0 2024-09-11T23:29:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.475+0000 7f5b6b051740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f257d94290 con 0x55f257ea72f0 2024-09-11T23:29:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.475+0000 7f5b6b051740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f257eb8e70 con 0x55f257eb60c0 2024-09-11T23:29:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.475+0000 7f5b6b051740 10 client.0 _ll_register_callbacks cb 0x55f257dc5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.475+0000 7f5b65771640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257eb2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.475+0000 7f5b65771640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257eb2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56862/0 (socket says 172.21.15.43:56862) 2024-09-11T23:29:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.475+0000 7f5b65771640 1 -- 172.21.15.43:0/753019229 learned_addr learned my addr 172.21.15.43:0/753019229 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.476+0000 7f5b65f72640 1 --2- 172.21.15.43:0/753019229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f257ea72f0 0x55f257eb3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.476+0000 7f5b65771640 1 -- 172.21.15.43:0/753019229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f257eb60c0 msgr2=0x55f257eb3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.476+0000 7f5b65771640 1 --2- 172.21.15.43:0/753019229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f257eb60c0 0x55f257eb3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.476+0000 7f5b65771640 1 -- 172.21.15.43:0/753019229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f257ea72f0 msgr2=0x55f257eb3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.476+0000 7f5b65771640 1 --2- 172.21.15.43:0/753019229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f257ea72f0 0x55f257eb3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.476+0000 7f5b65771640 1 -- 172.21.15.43:0/753019229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f257b1e210 con 0x55f257ea69d0 2024-09-11T23:29:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.476+0000 7f5b65f72640 1 --2- 172.21.15.43:0/753019229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f257ea72f0 0x55f257eb3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:29:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.476+0000 7f5b65771640 1 --2- 172.21.15.43:0/753019229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257eb2fd0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5b50004bb0 tx=0x7f5b50004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80cbaf8cf5b7f3c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.476+0000 7f5b497fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:29:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.476+0000 7f5b497fa640 1 -- 172.21.15.43:0/753019229 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50004e50 con 0x55f257ea69d0 2024-09-11T23:29:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.476+0000 7f5b497fa640 1 -- 172.21.15.43:0/753019229 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f5b50004ff0 con 0x55f257ea69d0 2024-09-11T23:29:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.476+0000 7f5b6b051740 1 -- 172.21.15.43:0/753019229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b50005500 con 0x55f257ea69d0 2024-09-11T23:29:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.476+0000 7f5b497fa640 1 -- 172.21.15.43:0/753019229 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5b50031050 con 0x55f257ea69d0 2024-09-11T23:29:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.476+0000 7f5b6b051740 10 client.16608 fetch_fsmap learned FSMap version 31 2024-09-11T23:29:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.476+0000 7f5b6b051740 1 -- 172.21.15.43:0/753019229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f257bea620 con 0x55f257ea69d0 2024-09-11T23:29:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.476+0000 7f5b497fa640 1 -- 172.21.15.43:0/753019229 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b50005500 con 0x55f257ea69d0 2024-09-11T23:29:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.477+0000 7f5b497fa640 1 -- 172.21.15.43:0/753019229 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f5b5001e440 con 0x55f257ea69d0 2024-09-11T23:29:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.477+0000 7f5b6b051740 10 client.16608 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:29:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.477+0000 7f5b6b051740 10 client.16608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.477+0000 7f5b6b051740 20 client.16608 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.477+0000 7f5b6b051740 1 --2- 172.21.15.43:0/753019229 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f257ec5fe0 0x55f257ec83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.477+0000 7f5b6b051740 4 client.16608 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:29:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.477+0000 7f5b6b051740 1 -- 172.21.15.43:0/753019229 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f257e18a20 con 0x55f257ec5fe0 2024-09-11T23:29:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.477+0000 7f5b65f72640 1 --2- 172.21.15.43:0/753019229 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f257ec5fe0 0x55f257ec83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.477+0000 7f5b65f72640 1 --2- 172.21.15.43:0/753019229 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f257ec5fe0 0x55f257ec83d0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c947f9d303a88ef2 server_cookie=6d9855f92208fda3 in_seq=0 out_seq=0 2024-09-11T23:29:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.477+0000 7f5b497fa640 10 client.16608 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.478+0000 7f5b497fa640 1 -- 172.21.15.43:0/753019229 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f257e18a20 con 0x55f257ec5fe0 2024-09-11T23:29:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.478+0000 7f5b497fa640 10 client.16608 handle_command_reply: tid=0 2024-09-11T23:29:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.478+0000 7f5b6b051740 1 client.16608 shutdown 2024-09-11T23:29:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.478+0000 7f5b6b051740 1 -- 172.21.15.43:0/753019229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 msgr2=0x55f257eb2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.478+0000 7f5b6b051740 1 --2- 172.21.15.43:0/753019229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257eb2fd0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5b50004bb0 tx=0x7f5b50004940 comp rx=0 tx=0).stop 2024-09-11T23:29:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.478+0000 7f5b6b051740 1 -- 172.21.15.43:0/753019229 shutdown_connections 2024-09-11T23:29:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.478+0000 7f5b6b051740 1 --2- 172.21.15.43:0/753019229 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f257ec5fe0 0x55f257ec83d0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.478+0000 7f5b6b051740 1 --2- 172.21.15.43:0/753019229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f257eb60c0 0x55f257eb3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.478+0000 7f5b497fa640 0 client.16608 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.478+0000 7f5b6b051740 1 --2- 172.21.15.43:0/753019229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f257ea72f0 0x55f257eb3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.478+0000 7f5b6b051740 1 --2- 172.21.15.43:0/753019229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257eb2fd0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.478+0000 7f5b6b051740 1 -- 172.21.15.43:0/753019229 >> 172.21.15.43:0/753019229 conn(0x55f257ea3060 msgr2=0x55f257ea8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.478+0000 7f5b6b051740 1 -- 172.21.15.43:0/753019229 shutdown_connections 2024-09-11T23:29:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.478+0000 7f5b6b051740 1 -- 172.21.15.43:0/753019229 wait complete. 2024-09-11T23:29:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.478+0000 7f5b6b051740 20 client.16608 trim_cache size 0 max 16384 2024-09-11T23:29:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.493+0000 7f5b6b051740 1 Processor -- start 2024-09-11T23:29:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.493+0000 7f5b6b051740 1 -- start start 2024-09-11T23:29:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.493+0000 7f5b6b051740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257ecd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.493+0000 7f5b6b051740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f257ea72f0 0x55f257ecdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.493+0000 7f5b6b051740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f257eb60c0 0x55f257eb0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.493+0000 7f5b6b051740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b601adef0 con 0x55f257ea69d0 2024-09-11T23:29:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.493+0000 7f5b6b051740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5b601ab790 con 0x55f257eb60c0 2024-09-11T23:29:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.493+0000 7f5b6b051740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5b601a9600 con 0x55f257ea72f0 2024-09-11T23:29:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.493+0000 7f5b65771640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257ecd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.493+0000 7f5b65771640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257ecd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56876/0 (socket says 172.21.15.43:56876) 2024-09-11T23:29:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.493+0000 7f5b65771640 1 -- 172.21.15.43:0/3514120477 learned_addr learned my addr 172.21.15.43:0/3514120477 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.493+0000 7f5b65771640 1 --2- 172.21.15.43:0/3514120477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f257eb60c0 0x55f257eb0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.493+0000 7f5b65f72640 1 --2- 172.21.15.43:0/3514120477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f257ea72f0 0x55f257ecdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.493+0000 7f5b65771640 1 -- 172.21.15.43:0/3514120477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f257ea72f0 msgr2=0x55f257ecdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.493+0000 7f5b65771640 1 --2- 172.21.15.43:0/3514120477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f257ea72f0 0x55f257ecdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.493+0000 7f5b65771640 1 -- 172.21.15.43:0/3514120477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f257eb60c0 msgr2=0x55f257eb0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.493+0000 7f5b65771640 1 --2- 172.21.15.43:0/3514120477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f257eb60c0 0x55f257eb0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.493+0000 7f5b65771640 1 -- 172.21.15.43:0/3514120477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f257ea2e90 con 0x55f257ea69d0 2024-09-11T23:29:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.493+0000 7f5b65f72640 1 --2- 172.21.15.43:0/3514120477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f257ea72f0 0x55f257ecdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:29:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.493+0000 7f5b65771640 1 --2- 172.21.15.43:0/3514120477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257ecd9d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5b50001eb0 tx=0x7f5b5000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6d135c7270a022 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.494+0000 7f5b4b7fe640 1 -- 172.21.15.43:0/3514120477 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50004120 con 0x55f257ea69d0 2024-09-11T23:29:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.494+0000 7f5b4b7fe640 1 -- 172.21.15.43:0/3514120477 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b50022070 con 0x55f257ea69d0 2024-09-11T23:29:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.494+0000 7f5b4b7fe640 1 -- 172.21.15.43:0/3514120477 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50005500 con 0x55f257ea69d0 2024-09-11T23:29:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.494+0000 7f5b6b051740 1 -- 172.21.15.43:0/3514120477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 msgr2=0x55f257ecd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.494+0000 7f5b6b051740 1 --2- 172.21.15.43:0/3514120477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257ecd9d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5b50001eb0 tx=0x7f5b5000bf90 comp rx=0 tx=0).stop 2024-09-11T23:29:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.494+0000 7f5b6b051740 1 -- 172.21.15.43:0/3514120477 shutdown_connections 2024-09-11T23:29:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.494+0000 7f5b6b051740 1 --2- 172.21.15.43:0/3514120477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f257eb60c0 0x55f257eb0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.494+0000 7f5b6b051740 1 --2- 172.21.15.43:0/3514120477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f257ea72f0 0x55f257ecdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.494+0000 7f5b6b051740 1 --2- 172.21.15.43:0/3514120477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257ecd9d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.494+0000 7f5b6b051740 1 -- 172.21.15.43:0/3514120477 >> 172.21.15.43:0/3514120477 conn(0x55f257ea3060 msgr2=0x55f257ea21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.494+0000 7f5b6b051740 1 -- 172.21.15.43:0/3514120477 shutdown_connections 2024-09-11T23:29:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.494+0000 7f5b6b051740 1 -- 172.21.15.43:0/3514120477 wait complete. 2024-09-11T23:29:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.495+0000 7f5b6b051740 1 -- start start 2024-09-11T23:29:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.495+0000 7f5b6b051740 1 Processor -- start 2024-09-11T23:29:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.495+0000 7f5b6b051740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257eb9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.495+0000 7f5b6b051740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f257ea72f0 0x55f257eba2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.495+0000 7f5b6b051740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f257eb60c0 0x55f257eba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.495+0000 7f5b6b051740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b601a9c50 con 0x55f257ea69d0 2024-09-11T23:29:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.495+0000 7f5b6b051740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5b601ab0c0 con 0x55f257ea72f0 2024-09-11T23:29:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.495+0000 7f5b6b051740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5b601aa320 con 0x55f257eb60c0 2024-09-11T23:29:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.495+0000 7f5b6b051740 10 client.0 _ll_register_callbacks cb 0x7f5b5c00a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.495+0000 7f5b65771640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257eb9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.495+0000 7f5b65771640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257eb9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56882/0 (socket says 172.21.15.43:56882) 2024-09-11T23:29:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.495+0000 7f5b65771640 1 -- 172.21.15.43:0/2648550782 learned_addr learned my addr 172.21.15.43:0/2648550782 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.495+0000 7f5b65f72640 1 --2- 172.21.15.43:0/2648550782 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f257ea72f0 0x55f257eba2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.495+0000 7f5b65771640 1 -- 172.21.15.43:0/2648550782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f257eb60c0 msgr2=0x55f257eba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.495+0000 7f5b65771640 1 --2- 172.21.15.43:0/2648550782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f257eb60c0 0x55f257eba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.495+0000 7f5b65771640 1 -- 172.21.15.43:0/2648550782 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f257ea72f0 msgr2=0x55f257eba2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.495+0000 7f5b65771640 1 --2- 172.21.15.43:0/2648550782 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f257ea72f0 0x55f257eba2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.495+0000 7f5b65771640 1 -- 172.21.15.43:0/2648550782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f257f2f9a0 con 0x55f257ea69d0 2024-09-11T23:29:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.496+0000 7f5b65771640 1 --2- 172.21.15.43:0/2648550782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257eb9d80 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5b5001e840 tx=0x7f5b500041b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d80b0372196767 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.496+0000 7f5b497fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:29:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.496+0000 7f5b497fa640 1 -- 172.21.15.43:0/2648550782 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5001e440 con 0x55f257ea69d0 2024-09-11T23:29:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.496+0000 7f5b497fa640 1 -- 172.21.15.43:0/2648550782 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f5b5001e5e0 con 0x55f257ea69d0 2024-09-11T23:29:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.496+0000 7f5b6b051740 1 -- 172.21.15.43:0/2648550782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b50022070 con 0x55f257ea69d0 2024-09-11T23:29:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.496+0000 7f5b497fa640 1 -- 172.21.15.43:0/2648550782 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5b50036480 con 0x55f257ea69d0 2024-09-11T23:29:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.496+0000 7f5b6b051740 10 client.16614 fetch_fsmap learned FSMap version 31 2024-09-11T23:29:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.496+0000 7f5b6b051740 1 -- 172.21.15.43:0/2648550782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f257f30b40 con 0x55f257ea69d0 2024-09-11T23:29:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.496+0000 7f5b497fa640 1 -- 172.21.15.43:0/2648550782 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b50022070 con 0x55f257ea69d0 2024-09-11T23:29:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.496+0000 7f5b497fa640 1 -- 172.21.15.43:0/2648550782 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f5b5003e060 con 0x55f257ea69d0 2024-09-11T23:29:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.496+0000 7f5b6b051740 10 client.16614 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:29:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.496+0000 7f5b6b051740 10 client.16614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.496+0000 7f5b6b051740 20 client.16614 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.496+0000 7f5b6b051740 1 --2- 172.21.15.43:0/2648550782 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f257ec5fe0 0x55f257f33010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.496+0000 7f5b6b051740 4 client.16614 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:29:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.496+0000 7f5b6b051740 1 -- 172.21.15.43:0/2648550782 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f257e12d70 con 0x55f257ec5fe0 2024-09-11T23:29:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.497+0000 7f5b65f72640 1 --2- 172.21.15.43:0/2648550782 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f257ec5fe0 0x55f257f33010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.497+0000 7f5b65f72640 1 --2- 172.21.15.43:0/2648550782 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f257ec5fe0 0x55f257f33010 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b5eb82f01a1fa5f server_cookie=597de073483f5e48 in_seq=0 out_seq=0 2024-09-11T23:29:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.497+0000 7f5b497fa640 10 client.16614 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:24.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:24 smithi090 ceph-mon[35130]: pgmap v231: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 105 op/s 2024-09-11T23:29:24.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:24 smithi090 ceph-mon[35130]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T23:29:24.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:24 smithi097 ceph-mon[35562]: pgmap v231: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 105 op/s 2024-09-11T23:29:24.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:24 smithi097 ceph-mon[35562]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T23:29:24.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:24 smithi043 ceph-mon[30950]: pgmap v231: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 105 op/s 2024-09-11T23:29:24.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:24 smithi043 ceph-mon[30950]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T23:29:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.759+0000 7f5b497fa640 1 -- 172.21.15.43:0/2648550782 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f257e12d70 con 0x55f257ec5fe0 2024-09-11T23:29:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.759+0000 7f5b497fa640 10 client.16614 handle_command_reply: tid=0 2024-09-11T23:29:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.759+0000 7f5b6b051740 1 client.16614 shutdown 2024-09-11T23:29:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.759+0000 7f5b6b051740 1 -- 172.21.15.43:0/2648550782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 msgr2=0x55f257eb9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.759+0000 7f5b6b051740 1 --2- 172.21.15.43:0/2648550782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257eb9d80 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5b5001e840 tx=0x7f5b500041b0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.760+0000 7f5b6b051740 1 -- 172.21.15.43:0/2648550782 shutdown_connections 2024-09-11T23:29:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.760+0000 7f5b6b051740 1 --2- 172.21.15.43:0/2648550782 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f257ec5fe0 0x55f257f33010 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.760+0000 7f5b6b051740 1 --2- 172.21.15.43:0/2648550782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f257eb60c0 0x55f257eba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.760+0000 7f5b6b051740 1 --2- 172.21.15.43:0/2648550782 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f257ea72f0 0x55f257eba2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.760+0000 7f5b497fa640 0 client.16614 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.760+0000 7f5b6b051740 1 --2- 172.21.15.43:0/2648550782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f257ea69d0 0x55f257eb9d80 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.760+0000 7f5b6b051740 1 -- 172.21.15.43:0/2648550782 >> 172.21.15.43:0/2648550782 conn(0x55f257ea3060 msgr2=0x55f257ea84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.760+0000 7f5b6b051740 1 -- 172.21.15.43:0/2648550782 shutdown_connections 2024-09-11T23:29:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.760+0000 7f5b6b051740 1 -- 172.21.15.43:0/2648550782 wait complete. 2024-09-11T23:29:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.760+0000 7f5b6b051740 20 client.16614 trim_cache size 0 max 16384 2024-09-11T23:29:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:29:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:29:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:29:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:29:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.761+0000 7f5b681d6640 1 -- 172.21.15.43:0/2533607341 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5b4c0a2d70 msgr2=0x7f5b4c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.761+0000 7f5b681d6640 1 --2- 172.21.15.43:0/2533607341 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5b4c0a2d70 0x7f5b4c0a5230 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5b58006010 tx=0x7f5b5808e730 comp rx=0 tx=0).stop 2024-09-11T23:29:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.761+0000 7f5b681d6640 1 -- 172.21.15.43:0/2533607341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60105af0 msgr2=0x7f5b6019cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.761+0000 7f5b681d6640 1 --2- 172.21.15.43:0/2533607341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60105af0 0x7f5b6019cbd0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5b58009dc0 tx=0x7f5b58003f10 comp rx=0 tx=0).stop 2024-09-11T23:29:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.761+0000 7f5b681d6640 1 -- 172.21.15.43:0/2533607341 shutdown_connections 2024-09-11T23:29:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.761+0000 7f5b681d6640 1 --2- 172.21.15.43:0/2533607341 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5b4c0a2d70 0x7f5b4c0a5230 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.761+0000 7f5b681d6640 1 --2- 172.21.15.43:0/2533607341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60105af0 0x7f5b6019cbd0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.761+0000 7f5b681d6640 1 --2- 172.21.15.43:0/2533607341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b60102050 0x7f5b60198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.761+0000 7f5b681d6640 1 --2- 172.21.15.43:0/2533607341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b601016d0 0x7f5b60198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.761+0000 7f5b681d6640 1 -- 172.21.15.43:0/2533607341 >> 172.21.15.43:0/2533607341 conn(0x7f5b60075f20 msgr2=0x7f5b600ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.761+0000 7f5b681d6640 1 -- 172.21.15.43:0/2533607341 shutdown_connections 2024-09-11T23:29:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:24.761+0000 7f5b681d6640 1 -- 172.21.15.43:0/2533607341 wait complete. 2024-09-11T23:29:25.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:25 smithi090 ceph-mon[35130]: pgmap v233: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 51 op/s 2024-09-11T23:29:25.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:25 smithi097 ceph-mon[35562]: pgmap v233: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 51 op/s 2024-09-11T23:29:25.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:25 smithi043 ceph-mon[30950]: pgmap v233: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 51 op/s 2024-09-11T23:29:27.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:27 smithi090 ceph-mon[35130]: pgmap v234: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 58 op/s 2024-09-11T23:29:27.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:27 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:27.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:27 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:27.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:27 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:29:27.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:27 smithi097 ceph-mon[35562]: pgmap v234: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 58 op/s 2024-09-11T23:29:27.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:27 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:27.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:27 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:27.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:27 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:29:27.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:27 smithi043 ceph-mon[30950]: pgmap v234: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 58 op/s 2024-09-11T23:29:27.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:27 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:29:27.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:27 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:29:27.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:27 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:29:28.427 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:29:28.455 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:29:29.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:29 smithi090 ceph-mon[35130]: pgmap v235: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 59 op/s 2024-09-11T23:29:29.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:29 smithi097 ceph-mon[35562]: pgmap v235: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 59 op/s 2024-09-11T23:29:29.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:29 smithi043 ceph-mon[30950]: pgmap v235: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 59 op/s 2024-09-11T23:29:31.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:31 smithi090 ceph-mon[35130]: pgmap v236: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 59 op/s 2024-09-11T23:29:31.800 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:31 smithi097 ceph-mon[35562]: pgmap v236: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 59 op/s 2024-09-11T23:29:31.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:31 smithi043 ceph-mon[30950]: pgmap v236: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 59 op/s 2024-09-11T23:29:33.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.562+0000 7efffd685640 1 -- 172.21.15.43:0/1786328356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efff8102dd0 msgr2=0x7efff81031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:33.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.562+0000 7efffd685640 1 --2- 172.21.15.43:0/1786328356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efff8102dd0 0x7efff81031b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7effe8009930 tx=0x7effe802a420 comp rx=0 tx=0).stop 2024-09-11T23:29:33.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.562+0000 7efffd685640 1 -- 172.21.15.43:0/1786328356 shutdown_connections 2024-09-11T23:29:33.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.562+0000 7efffd685640 1 --2- 172.21.15.43:0/1786328356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efff8108300 0x7efff810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:33.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.562+0000 7efffd685640 1 --2- 172.21.15.43:0/1786328356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efff8103780 0x7efff8107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:33.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.562+0000 7efffd685640 1 --2- 172.21.15.43:0/1786328356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efff8102dd0 0x7efff81031b0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:33.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.562+0000 7efffd685640 1 -- 172.21.15.43:0/1786328356 >> 172.21.15.43:0/1786328356 conn(0x7efff80fc7d0 msgr2=0x7efff80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:33.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.562+0000 7efffd685640 1 -- 172.21.15.43:0/1786328356 shutdown_connections 2024-09-11T23:29:33.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.562+0000 7efffd685640 1 -- 172.21.15.43:0/1786328356 wait complete. 2024-09-11T23:29:33.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.563+0000 7efffd685640 1 Processor -- start 2024-09-11T23:29:33.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.563+0000 7efffd685640 1 -- start start 2024-09-11T23:29:33.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.563+0000 7efffd685640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efff8102dd0 0x7efff8198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:33.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.563+0000 7efffd685640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efff8103780 0x7efff8198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:33.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.563+0000 7efffd685640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efff8108300 0x7efff819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:33.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.563+0000 7efffd685640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efff8112090 con 0x7efff8102dd0 2024-09-11T23:29:33.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.563+0000 7efffd685640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efff8111f10 con 0x7efff8103780 2024-09-11T23:29:33.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.563+0000 7efffd685640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efff8112210 con 0x7efff8108300 2024-09-11T23:29:33.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.563+0000 7efff6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efff8102dd0 0x7efff8198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:29:33.587 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":31,"btime":"2024-09-11T23:29:03:134233+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:29:02.865047+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25571":{"gid":25571,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/568491708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":568491708},{"type":"v1","addr":"172.21.15.90:6835","nonce":568491708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24457,24470,24497]},"id":1}]} 2024-09-11T23:29:33.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.563+0000 7efff6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efff8102dd0 0x7efff8198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52852/0 (socket says 172.21.15.43:52852) 2024-09-11T23:29:33.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.563+0000 7efff6ffd640 1 -- 172.21.15.43:0/3585965855 learned_addr learned my addr 172.21.15.43:0/3585965855 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:33.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.564+0000 7efff67fc640 1 --2- 172.21.15.43:0/3585965855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efff8103780 0x7efff8198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:33.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.564+0000 7efff77fe640 1 --2- 172.21.15.43:0/3585965855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efff8108300 0x7efff819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:33.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.564+0000 7efff6ffd640 1 -- 172.21.15.43:0/3585965855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efff8108300 msgr2=0x7efff819cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:33.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.564+0000 7efff6ffd640 1 --2- 172.21.15.43:0/3585965855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efff8108300 0x7efff819cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:33.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.564+0000 7efff6ffd640 1 -- 172.21.15.43:0/3585965855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efff8103780 msgr2=0x7efff8198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:33.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.564+0000 7efff6ffd640 1 --2- 172.21.15.43:0/3585965855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efff8103780 0x7efff8198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:33.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.564+0000 7efff6ffd640 1 -- 172.21.15.43:0/3585965855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff819d600 con 0x7efff8102dd0 2024-09-11T23:29:33.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.564+0000 7efff6ffd640 1 --2- 172.21.15.43:0/3585965855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efff8102dd0 0x7efff8198650 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7effe8009230 tx=0x7effe8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:33.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.564+0000 7effdffff640 1 -- 172.21.15.43:0/3585965855 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe803f030 con 0x7efff8102dd0 2024-09-11T23:29:33.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.564+0000 7effdffff640 1 -- 172.21.15.43:0/3585965855 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effe8003ad0 con 0x7efff8102dd0 2024-09-11T23:29:33.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.564+0000 7efffd685640 1 -- 172.21.15.43:0/3585965855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff819d890 con 0x7efff8102dd0 2024-09-11T23:29:33.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.564+0000 7effdffff640 1 -- 172.21.15.43:0/3585965855 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8035070 con 0x7efff8102dd0 2024-09-11T23:29:33.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.564+0000 7efffd685640 1 -- 172.21.15.43:0/3585965855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff81a51d0 con 0x7efff8102dd0 2024-09-11T23:29:33.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.565+0000 7efffd685640 1 -- 172.21.15.43:0/3585965855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efff81031b0 con 0x7efff8102dd0 2024-09-11T23:29:33.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.567+0000 7effdffff640 1 -- 172.21.15.43:0/3585965855 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7effe8042030 con 0x7efff8102dd0 2024-09-11T23:29:33.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.567+0000 7effdffff640 1 --2- 172.21.15.43:0/3585965855 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7effe00a2d70 0x7effe00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:33.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.567+0000 7efff67fc640 1 --2- 172.21.15.43:0/3585965855 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7effe00a2d70 0x7effe00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:33.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.567+0000 7effdffff640 1 -- 172.21.15.43:0/3585965855 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7effe80ef7b0 con 0x7efff8102dd0 2024-09-11T23:29:33.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.568+0000 7efff67fc640 1 --2- 172.21.15.43:0/3585965855 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7effe00a2d70 0x7effe00a5230 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efff80fefa0 tx=0x7effec005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:33.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.571+0000 7effdffff640 1 -- 172.21.15.43:0/3585965855 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7effe80f1e20 con 0x7efff8102dd0 2024-09-11T23:29:33.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.816+0000 7efffd685640 1 -- 172.21.15.43:0/3585965855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efff810a290 con 0x7efff8102dd0 2024-09-11T23:29:33.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.817+0000 7effdffff640 1 -- 172.21.15.43:0/3585965855 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11894 (secure 0 0 0) 0x7effe80bbd50 con 0x7efff8102dd0 2024-09-11T23:29:33.589 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 31 2024-09-11T23:29:33.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.820+0000 7efffd685640 1 -- 172.21.15.43:0/3585965855 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7effe00a2d70 msgr2=0x7effe00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:33.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.820+0000 7efffd685640 1 --2- 172.21.15.43:0/3585965855 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7effe00a2d70 0x7effe00a5230 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efff80fefa0 tx=0x7effec005f90 comp rx=0 tx=0).stop 2024-09-11T23:29:33.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.820+0000 7efffd685640 1 -- 172.21.15.43:0/3585965855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efff8102dd0 msgr2=0x7efff8198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:33.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.820+0000 7efffd685640 1 --2- 172.21.15.43:0/3585965855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efff8102dd0 0x7efff8198650 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7effe8009230 tx=0x7effe8004270 comp rx=0 tx=0).stop 2024-09-11T23:29:33.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.820+0000 7efffd685640 1 -- 172.21.15.43:0/3585965855 shutdown_connections 2024-09-11T23:29:33.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.820+0000 7efffd685640 1 --2- 172.21.15.43:0/3585965855 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7effe00a2d70 0x7effe00a5230 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:33.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.820+0000 7efffd685640 1 --2- 172.21.15.43:0/3585965855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efff8108300 0x7efff819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:33.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.820+0000 7efffd685640 1 --2- 172.21.15.43:0/3585965855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efff8103780 0x7efff8198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:33.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.820+0000 7efffd685640 1 --2- 172.21.15.43:0/3585965855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efff8102dd0 0x7efff8198650 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:33.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.820+0000 7efffd685640 1 -- 172.21.15.43:0/3585965855 >> 172.21.15.43:0/3585965855 conn(0x7efff80fc7d0 msgr2=0x7efff8105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:33.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.820+0000 7efffd685640 1 -- 172.21.15.43:0/3585965855 shutdown_connections 2024-09-11T23:29:33.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:32.820+0000 7efffd685640 1 -- 172.21.15.43:0/3585965855 wait complete. 2024-09-11T23:29:33.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:33 smithi090 ceph-mon[35130]: pgmap v237: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 57 op/s 2024-09-11T23:29:33.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:33 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3585965855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:29:33.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:33 smithi097 ceph-mon[35562]: pgmap v237: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 57 op/s 2024-09-11T23:29:33.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:33 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3585965855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:29:33.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:33 smithi043 ceph-mon[30950]: pgmap v237: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 57 op/s 2024-09-11T23:29:33.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:33 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3585965855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:29:35.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:35 smithi090 ceph-mon[35130]: pgmap v238: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 55 op/s 2024-09-11T23:29:35.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:35 smithi097 ceph-mon[35562]: pgmap v238: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 55 op/s 2024-09-11T23:29:35.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:35 smithi043 ceph-mon[30950]: pgmap v238: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 55 op/s 2024-09-11T23:29:36.279 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:29:37.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:37 smithi043 ceph-mon[30950]: pgmap v239: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 61 op/s 2024-09-11T23:29:37.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:37 smithi090 ceph-mon[35130]: pgmap v239: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 61 op/s 2024-09-11T23:29:37.800 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:37 smithi097 ceph-mon[35562]: pgmap v239: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 61 op/s 2024-09-11T23:29:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.858+0000 7ff667a8c640 1 -- 172.21.15.43:0/3366494210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff660101370 msgr2=0x7ff66010d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.858+0000 7ff667a8c640 1 --2- 172.21.15.43:0/3366494210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff660101370 0x7ff66010d8a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff658009cd0 tx=0x7ff65802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.858+0000 7ff667a8c640 1 -- 172.21.15.43:0/3366494210 shutdown_connections 2024-09-11T23:29:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.858+0000 7ff667a8c640 1 --2- 172.21.15.43:0/3366494210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff660101370 0x7ff66010d8a0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.858+0000 7ff667a8c640 1 --2- 172.21.15.43:0/3366494210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6601009d0 0x7ff660100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.858+0000 7ff667a8c640 1 --2- 172.21.15.43:0/3366494210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6601069e0 0x7ff660106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.858+0000 7ff667a8c640 1 -- 172.21.15.43:0/3366494210 >> 172.21.15.43:0/3366494210 conn(0x7ff6600fc6b0 msgr2=0x7ff6600fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.858+0000 7ff667a8c640 1 -- 172.21.15.43:0/3366494210 shutdown_connections 2024-09-11T23:29:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.858+0000 7ff667a8c640 1 -- 172.21.15.43:0/3366494210 wait complete. 2024-09-11T23:29:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.858+0000 7ff667a8c640 1 Processor -- start 2024-09-11T23:29:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.859+0000 7ff667a8c640 1 -- start start 2024-09-11T23:29:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.859+0000 7ff667a8c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6601009d0 0x7ff6601983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.859+0000 7ff667a8c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff660101370 0x7ff660198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.859+0000 7ff667a8c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6601069e0 0x7ff66019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.859+0000 7ff667a8c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff660110e70 con 0x7ff660101370 2024-09-11T23:29:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.859+0000 7ff667a8c640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff660110cf0 con 0x7ff6601069e0 2024-09-11T23:29:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.859+0000 7ff667a8c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff660110ff0 con 0x7ff6601009d0 2024-09-11T23:29:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.859+0000 7ff665027640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff660101370 0x7ff660198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.859+0000 7ff665027640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff660101370 0x7ff660198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52878/0 (socket says 172.21.15.43:52878) 2024-09-11T23:29:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.859+0000 7ff665027640 1 -- 172.21.15.43:0/73560988 learned_addr learned my addr 172.21.15.43:0/73560988 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.859+0000 7ff666029640 1 --2- 172.21.15.43:0/73560988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6601069e0 0x7ff66019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.859+0000 7ff665828640 1 --2- 172.21.15.43:0/73560988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6601009d0 0x7ff6601983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.859+0000 7ff665027640 1 -- 172.21.15.43:0/73560988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6601009d0 msgr2=0x7ff6601983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.859+0000 7ff665027640 1 --2- 172.21.15.43:0/73560988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6601009d0 0x7ff6601983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.859+0000 7ff665027640 1 -- 172.21.15.43:0/73560988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6601069e0 msgr2=0x7ff66019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.859+0000 7ff665027640 1 --2- 172.21.15.43:0/73560988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6601069e0 0x7ff66019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.859+0000 7ff665027640 1 -- 172.21.15.43:0/73560988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66019d3c0 con 0x7ff660101370 2024-09-11T23:29:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.859+0000 7ff665027640 1 --2- 172.21.15.43:0/73560988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff660101370 0x7ff660198910 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff65000e9f0 tx=0x7ff65000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.860+0000 7ff656ffd640 1 -- 172.21.15.43:0/73560988 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65000cd80 con 0x7ff660101370 2024-09-11T23:29:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.860+0000 7ff656ffd640 1 -- 172.21.15.43:0/73560988 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff650005500 con 0x7ff660101370 2024-09-11T23:29:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.860+0000 7ff667a8c640 1 -- 172.21.15.43:0/73560988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff66019d6b0 con 0x7ff660101370 2024-09-11T23:29:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.860+0000 7ff656ffd640 1 -- 172.21.15.43:0/73560988 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650010690 con 0x7ff660101370 2024-09-11T23:29:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.860+0000 7ff667a8c640 1 -- 172.21.15.43:0/73560988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6601a4f40 con 0x7ff660101370 2024-09-11T23:29:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.862+0000 7ff656ffd640 1 -- 172.21.15.43:0/73560988 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff6500046c0 con 0x7ff660101370 2024-09-11T23:29:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.862+0000 7ff66a825740 1 Processor -- start 2024-09-11T23:29:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.863+0000 7ff66a825740 1 -- start start 2024-09-11T23:29:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.863+0000 7ff66a825740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12db9b9d0 0x55d12db9bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.863+0000 7ff66a825740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12db9c2f0 0x55d12dbaab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.863+0000 7ff66a825740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d12dbab0c0 0x55d12dbad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.863+0000 7ff66a825740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d12db002f0 con 0x55d12db9c2f0 2024-09-11T23:29:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.863+0000 7ff66a825740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d12daa8270 con 0x55d12db9b9d0 2024-09-11T23:29:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.863+0000 7ff66a825740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d12dafc2b0 con 0x55d12dbab0c0 2024-09-11T23:29:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.863+0000 7ff656ffd640 1 --2- 172.21.15.43:0/73560988 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff64c0a2cf0 0x7ff64c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.863+0000 7ff665828640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12db9c2f0 0x55d12dbaab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.863+0000 7ff665828640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12db9c2f0 0x55d12dbaab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52890/0 (socket says 172.21.15.43:52890) 2024-09-11T23:29:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.863+0000 7ff665828640 1 -- 172.21.15.43:0/3922639072 learned_addr learned my addr 172.21.15.43:0/3922639072 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.863+0000 7ff666029640 1 --2- 172.21.15.43:0/3922639072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12db9b9d0 0x55d12db9bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.863+0000 7ff665027640 1 --2- 172.21.15.43:0/3922639072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d12dbab0c0 0x55d12dbad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.863+0000 7ff666029640 1 --2- 172.21.15.43:0/73560988 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff64c0a2cf0 0x7ff64c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.863+0000 7ff656ffd640 1 -- 172.21.15.43:0/73560988 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7ff6500cf900 con 0x7ff660101370 2024-09-11T23:29:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.864+0000 7ff665828640 1 -- 172.21.15.43:0/3922639072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d12dbab0c0 msgr2=0x55d12dbad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.864+0000 7ff665828640 1 --2- 172.21.15.43:0/3922639072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d12dbab0c0 0x55d12dbad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.864+0000 7ff665828640 1 -- 172.21.15.43:0/3922639072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12db9b9d0 msgr2=0x55d12db9bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.864+0000 7ff665828640 1 --2- 172.21.15.43:0/3922639072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12db9b9d0 0x55d12db9bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.864+0000 7ff665828640 1 -- 172.21.15.43:0/3922639072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d12daa7340 con 0x55d12db9c2f0 2024-09-11T23:29:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.864+0000 7ff665828640 1 --2- 172.21.15.43:0/3922639072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12db9c2f0 0x55d12dbaab80 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009ca0 tx=0x7ff65c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ffd31724fb253c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.864+0000 7ff666029640 1 --2- 172.21.15.43:0/73560988 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff64c0a2cf0 0x7ff64c0a51b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff658005e30 tx=0x7ff658032fd0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.864+0000 7ff64bfff640 1 -- 172.21.15.43:0/3922639072 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c004250 con 0x55d12db9c2f0 2024-09-11T23:29:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.864+0000 7ff64bfff640 1 -- 172.21.15.43:0/3922639072 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff65c0043f0 con 0x55d12db9c2f0 2024-09-11T23:29:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.864+0000 7ff64bfff640 1 -- 172.21.15.43:0/3922639072 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c019820 con 0x55d12db9c2f0 2024-09-11T23:29:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.865+0000 7ff66a825740 1 -- 172.21.15.43:0/3922639072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12db9c2f0 msgr2=0x55d12dbaab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.865+0000 7ff66a825740 1 --2- 172.21.15.43:0/3922639072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12db9c2f0 0x55d12dbaab80 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009ca0 tx=0x7ff65c011810 comp rx=0 tx=0).stop 2024-09-11T23:29:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.865+0000 7ff66a825740 1 -- 172.21.15.43:0/3922639072 shutdown_connections 2024-09-11T23:29:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.865+0000 7ff66a825740 1 --2- 172.21.15.43:0/3922639072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d12dbab0c0 0x55d12dbad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.865+0000 7ff66a825740 1 --2- 172.21.15.43:0/3922639072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12db9c2f0 0x55d12dbaab80 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.865+0000 7ff66a825740 1 --2- 172.21.15.43:0/3922639072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12db9b9d0 0x55d12db9bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.865+0000 7ff66a825740 1 -- 172.21.15.43:0/3922639072 >> 172.21.15.43:0/3922639072 conn(0x55d12db98060 msgr2=0x55d12db98440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.865+0000 7ff66a825740 1 -- 172.21.15.43:0/3922639072 shutdown_connections 2024-09-11T23:29:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.865+0000 7ff66a825740 1 -- 172.21.15.43:0/3922639072 wait complete. 2024-09-11T23:29:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.866+0000 7ff66a825740 1 -- start start 2024-09-11T23:29:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.866+0000 7ff66a825740 1 Processor -- start 2024-09-11T23:29:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.866+0000 7ff66a825740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d12db9b9d0 0x55d12dba7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.866+0000 7ff66a825740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12db9c2f0 0x55d12dba8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.866+0000 7ff66a825740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12dbab0c0 0x55d12dba8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.866+0000 7ff66a825740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d12dbadcf0 con 0x55d12dbab0c0 2024-09-11T23:29:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.866+0000 7ff66a825740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d12da89290 con 0x55d12db9c2f0 2024-09-11T23:29:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.866+0000 7ff66a825740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d12dbade70 con 0x55d12db9b9d0 2024-09-11T23:29:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.866+0000 7ff66a825740 10 client.0 _ll_register_callbacks cb 0x55d12daba140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.866+0000 7ff666029640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12dbab0c0 0x55d12dba8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.867+0000 7ff666029640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12dbab0c0 0x55d12dba8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52904/0 (socket says 172.21.15.43:52904) 2024-09-11T23:29:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.867+0000 7ff665027640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12db9c2f0 0x55d12dba8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.867+0000 7ff666029640 1 -- 172.21.15.43:0/1919926716 learned_addr learned my addr 172.21.15.43:0/1919926716 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.867+0000 7ff665828640 1 --2- 172.21.15.43:0/1919926716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d12db9b9d0 0x55d12dba7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.867+0000 7ff666029640 1 -- 172.21.15.43:0/1919926716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d12db9b9d0 msgr2=0x55d12dba7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.867+0000 7ff666029640 1 --2- 172.21.15.43:0/1919926716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d12db9b9d0 0x55d12dba7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.867+0000 7ff666029640 1 -- 172.21.15.43:0/1919926716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12db9c2f0 msgr2=0x55d12dba8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.867+0000 7ff666029640 1 --2- 172.21.15.43:0/1919926716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12db9c2f0 0x55d12dba8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.867+0000 7ff666029640 1 -- 172.21.15.43:0/1919926716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d12d813210 con 0x55d12dbab0c0 2024-09-11T23:29:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.867+0000 7ff665027640 1 --2- 172.21.15.43:0/1919926716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12db9c2f0 0x55d12dba8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:29:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.867+0000 7ff666029640 1 --2- 172.21.15.43:0/1919926716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12dbab0c0 0x55d12dba8a50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff658009cd0 tx=0x7ff6580095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e736ada7d2e878 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.867+0000 7ff6497fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:29:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.867+0000 7ff6497fa640 1 -- 172.21.15.43:0/1919926716 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658004820 con 0x55d12dbab0c0 2024-09-11T23:29:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.867+0000 7ff6497fa640 1 -- 172.21.15.43:0/1919926716 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7ff6580305a0 con 0x55d12dbab0c0 2024-09-11T23:29:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.867+0000 7ff66a825740 1 -- 172.21.15.43:0/1919926716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff65c0043f0 con 0x55d12dbab0c0 2024-09-11T23:29:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.867+0000 7ff6497fa640 1 -- 172.21.15.43:0/1919926716 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff6580307a0 con 0x55d12dbab0c0 2024-09-11T23:29:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.867+0000 7ff66a825740 10 client.16632 fetch_fsmap learned FSMap version 31 2024-09-11T23:29:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.867+0000 7ff66a825740 1 -- 172.21.15.43:0/1919926716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d12d8df620 con 0x55d12dbab0c0 2024-09-11T23:29:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.868+0000 7ff6497fa640 1 -- 172.21.15.43:0/1919926716 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff65c0043f0 con 0x55d12dbab0c0 2024-09-11T23:29:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.868+0000 7ff6497fa640 1 -- 172.21.15.43:0/1919926716 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7ff658048040 con 0x55d12dbab0c0 2024-09-11T23:29:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.868+0000 7ff66a825740 10 client.16632 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:29:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.868+0000 7ff66a825740 10 client.16632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.868+0000 7ff66a825740 20 client.16632 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.868+0000 7ff66a825740 1 --2- 172.21.15.43:0/1919926716 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55d12dbbb040 0x55d12dbbd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.868+0000 7ff66a825740 4 client.16632 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:29:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.868+0000 7ff66a825740 1 -- 172.21.15.43:0/1919926716 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d12db0da20 con 0x55d12dbbb040 2024-09-11T23:29:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.868+0000 7ff665828640 1 --2- 172.21.15.43:0/1919926716 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55d12dbbb040 0x55d12dbbd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.868+0000 7ff6497fa640 10 client.16632 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.868+0000 7ff665828640 1 --2- 172.21.15.43:0/1919926716 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55d12dbbb040 0x55d12dbbd4b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7a55671b10b231d server_cookie=361f88c04818c3f6 in_seq=0 out_seq=0 2024-09-11T23:29:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.869+0000 7ff6497fa640 1 -- 172.21.15.43:0/1919926716 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d12db0da20 con 0x55d12dbbb040 2024-09-11T23:29:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.869+0000 7ff6497fa640 10 client.16632 handle_command_reply: tid=0 2024-09-11T23:29:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.869+0000 7ff66a825740 1 client.16632 shutdown 2024-09-11T23:29:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.869+0000 7ff66a825740 1 -- 172.21.15.43:0/1919926716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12dbab0c0 msgr2=0x55d12dba8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.869+0000 7ff66a825740 1 --2- 172.21.15.43:0/1919926716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12dbab0c0 0x55d12dba8a50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff658009cd0 tx=0x7ff6580095c0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.869+0000 7ff66a825740 1 -- 172.21.15.43:0/1919926716 shutdown_connections 2024-09-11T23:29:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.869+0000 7ff66a825740 1 --2- 172.21.15.43:0/1919926716 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55d12dbbb040 0x55d12dbbd4b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.869+0000 7ff66a825740 1 --2- 172.21.15.43:0/1919926716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12dbab0c0 0x55d12dba8a50 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.869+0000 7ff66a825740 1 --2- 172.21.15.43:0/1919926716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12db9c2f0 0x55d12dba8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.869+0000 7ff66a825740 1 --2- 172.21.15.43:0/1919926716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d12db9b9d0 0x55d12dba7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.869+0000 7ff6497fa640 0 client.16632 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.869+0000 7ff66a825740 1 -- 172.21.15.43:0/1919926716 >> 172.21.15.43:0/1919926716 conn(0x55d12db98060 msgr2=0x55d12db9dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.869+0000 7ff66a825740 1 -- 172.21.15.43:0/1919926716 shutdown_connections 2024-09-11T23:29:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.869+0000 7ff66a825740 1 -- 172.21.15.43:0/1919926716 wait complete. 2024-09-11T23:29:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.869+0000 7ff66a825740 20 client.16632 trim_cache size 0 max 16384 2024-09-11T23:29:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.884+0000 7ff66a825740 1 Processor -- start 2024-09-11T23:29:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.884+0000 7ff66a825740 1 -- start start 2024-09-11T23:29:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.884+0000 7ff66a825740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d12db9b9d0 0x55d12dba9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.884+0000 7ff66a825740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12db9c2f0 0x55d12dbc2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.884+0000 7ff66a825740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12dbab0c0 0x55d12dbc3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.884+0000 7ff66a825740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6601ae070 con 0x55d12db9c2f0 2024-09-11T23:29:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.884+0000 7ff66a825740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff6601ab910 con 0x55d12dbab0c0 2024-09-11T23:29:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.884+0000 7ff66a825740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff6601a9780 con 0x55d12db9b9d0 2024-09-11T23:29:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.884+0000 7ff665027640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12db9c2f0 0x55d12dbc2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.884+0000 7ff665027640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12db9c2f0 0x55d12dbc2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52912/0 (socket says 172.21.15.43:52912) 2024-09-11T23:29:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.884+0000 7ff665027640 1 -- 172.21.15.43:0/3269272227 learned_addr learned my addr 172.21.15.43:0/3269272227 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.884+0000 7ff666029640 1 --2- 172.21.15.43:0/3269272227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12dbab0c0 0x55d12dbc3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.884+0000 7ff665828640 1 --2- 172.21.15.43:0/3269272227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d12db9b9d0 0x55d12dba9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.884+0000 7ff665027640 1 -- 172.21.15.43:0/3269272227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d12db9b9d0 msgr2=0x55d12dba9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.884+0000 7ff665027640 1 --2- 172.21.15.43:0/3269272227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d12db9b9d0 0x55d12dba9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.884+0000 7ff665027640 1 -- 172.21.15.43:0/3269272227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12dbab0c0 msgr2=0x55d12dbc3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.884+0000 7ff665027640 1 --2- 172.21.15.43:0/3269272227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12dbab0c0 0x55d12dbc3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.884+0000 7ff665027640 1 -- 172.21.15.43:0/3269272227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d12db97e90 con 0x55d12db9c2f0 2024-09-11T23:29:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.884+0000 7ff665027640 1 --2- 172.21.15.43:0/3269272227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12db9c2f0 0x55d12dbc2e10 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff65009bd20 tx=0x7ff65001bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd1798f8fc29656 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.885+0000 7ff64b7fe640 1 -- 172.21.15.43:0/3269272227 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65009bd90 con 0x55d12db9c2f0 2024-09-11T23:29:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.885+0000 7ff64b7fe640 1 -- 172.21.15.43:0/3269272227 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6500a0140 con 0x55d12db9c2f0 2024-09-11T23:29:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.885+0000 7ff64b7fe640 1 -- 172.21.15.43:0/3269272227 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6500a4570 con 0x55d12db9c2f0 2024-09-11T23:29:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.885+0000 7ff66a825740 1 -- 172.21.15.43:0/3269272227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12db9c2f0 msgr2=0x55d12dbc2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.885+0000 7ff66a825740 1 --2- 172.21.15.43:0/3269272227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12db9c2f0 0x55d12dbc2e10 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff65009bd20 tx=0x7ff65001bb30 comp rx=0 tx=0).stop 2024-09-11T23:29:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.885+0000 7ff66a825740 1 -- 172.21.15.43:0/3269272227 shutdown_connections 2024-09-11T23:29:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.885+0000 7ff66a825740 1 --2- 172.21.15.43:0/3269272227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12dbab0c0 0x55d12dbc3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.885+0000 7ff66a825740 1 --2- 172.21.15.43:0/3269272227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12db9c2f0 0x55d12dbc2e10 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.885+0000 7ff66a825740 1 --2- 172.21.15.43:0/3269272227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d12db9b9d0 0x55d12dba9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.885+0000 7ff66a825740 1 -- 172.21.15.43:0/3269272227 >> 172.21.15.43:0/3269272227 conn(0x55d12db98060 msgr2=0x55d12dba9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.885+0000 7ff66a825740 1 -- 172.21.15.43:0/3269272227 shutdown_connections 2024-09-11T23:29:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.885+0000 7ff66a825740 1 -- 172.21.15.43:0/3269272227 wait complete. 2024-09-11T23:29:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.886+0000 7ff66a825740 1 -- start start 2024-09-11T23:29:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.886+0000 7ff66a825740 1 Processor -- start 2024-09-11T23:29:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.886+0000 7ff66a825740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12db9b9d0 0x55d12dbb3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.886+0000 7ff66a825740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d12db9c2f0 0x55d12dbaeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.886+0000 7ff66a825740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12dbab0c0 0x55d12dbaf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.886+0000 7ff66a825740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6601a9dd0 con 0x55d12dbab0c0 2024-09-11T23:29:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.886+0000 7ff66a825740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff6601ab240 con 0x55d12db9b9d0 2024-09-11T23:29:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.886+0000 7ff66a825740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff6601aa4a0 con 0x55d12db9c2f0 2024-09-11T23:29:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.886+0000 7ff66a825740 10 client.0 _ll_register_callbacks cb 0x7ff658009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.886+0000 7ff666029640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12dbab0c0 0x55d12dbaf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.886+0000 7ff666029640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12dbab0c0 0x55d12dbaf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52920/0 (socket says 172.21.15.43:52920) 2024-09-11T23:29:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.886+0000 7ff665027640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d12db9c2f0 0x55d12dbaeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.886+0000 7ff665828640 1 --2- 172.21.15.43:0/2440339335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12db9b9d0 0x55d12dbb3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.886+0000 7ff666029640 1 -- 172.21.15.43:0/2440339335 learned_addr learned my addr 172.21.15.43:0/2440339335 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.886+0000 7ff666029640 1 -- 172.21.15.43:0/2440339335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d12db9c2f0 msgr2=0x55d12dbaeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.887+0000 7ff666029640 1 --2- 172.21.15.43:0/2440339335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d12db9c2f0 0x55d12dbaeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.887+0000 7ff666029640 1 -- 172.21.15.43:0/2440339335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12db9b9d0 msgr2=0x55d12dbb3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.887+0000 7ff666029640 1 --2- 172.21.15.43:0/2440339335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12db9b9d0 0x55d12dbb3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.887+0000 7ff666029640 1 -- 172.21.15.43:0/2440339335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d12dbaf930 con 0x55d12dbab0c0 2024-09-11T23:29:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.887+0000 7ff665828640 1 --2- 172.21.15.43:0/2440339335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12db9b9d0 0x55d12dbb3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:29:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.887+0000 7ff666029640 1 --2- 172.21.15.43:0/2440339335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12dbab0c0 0x55d12dbaf100 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff658009a60 tx=0x7ff658009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd871135e6dd71cd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.887+0000 7ff6497fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:29:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.887+0000 7ff6497fa640 1 -- 172.21.15.43:0/2440339335 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658060070 con 0x55d12dbab0c0 2024-09-11T23:29:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.887+0000 7ff66a825740 1 -- 172.21.15.43:0/2440339335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6500a0140 con 0x55d12dbab0c0 2024-09-11T23:29:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.887+0000 7ff6497fa640 1 -- 172.21.15.43:0/2440339335 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7ff658004620 con 0x55d12dbab0c0 2024-09-11T23:29:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.887+0000 7ff6497fa640 1 -- 172.21.15.43:0/2440339335 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff658004860 con 0x55d12dbab0c0 2024-09-11T23:29:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.887+0000 7ff66a825740 10 client.16638 fetch_fsmap learned FSMap version 31 2024-09-11T23:29:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.887+0000 7ff66a825740 1 -- 172.21.15.43:0/2440339335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d12dc25e80 con 0x55d12dbab0c0 2024-09-11T23:29:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.887+0000 7ff6497fa640 1 -- 172.21.15.43:0/2440339335 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6500a0140 con 0x55d12dbab0c0 2024-09-11T23:29:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.887+0000 7ff6497fa640 1 -- 172.21.15.43:0/2440339335 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7ff658048040 con 0x55d12dbab0c0 2024-09-11T23:29:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.887+0000 7ff66a825740 10 client.16638 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:29:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.887+0000 7ff66a825740 10 client.16638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.887+0000 7ff66a825740 20 client.16638 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.887+0000 7ff66a825740 1 --2- 172.21.15.43:0/2440339335 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55d12dc26540 0x55d12dc28960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.887+0000 7ff66a825740 4 client.16638 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:29:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.887+0000 7ff66a825740 1 -- 172.21.15.43:0/2440339335 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d12db07d70 con 0x55d12dc26540 2024-09-11T23:29:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.888+0000 7ff665828640 1 --2- 172.21.15.43:0/2440339335 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55d12dc26540 0x55d12dc28960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.888+0000 7ff665828640 1 --2- 172.21.15.43:0/2440339335 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55d12dc26540 0x55d12dc28960 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec19b544e7347943 server_cookie=bd4bec2491cb7dd in_seq=0 out_seq=0 2024-09-11T23:29:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:37.888+0000 7ff6497fa640 10 client.16638 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.680+0000 7ff6497fa640 1 -- 172.21.15.43:0/2440339335 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d12db07d70 con 0x55d12dc26540 2024-09-11T23:29:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.680+0000 7ff6497fa640 10 client.16638 handle_command_reply: tid=0 2024-09-11T23:29:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.680+0000 7ff66a825740 1 client.16638 shutdown 2024-09-11T23:29:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.680+0000 7ff66a825740 1 -- 172.21.15.43:0/2440339335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12dbab0c0 msgr2=0x55d12dbaf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.680+0000 7ff66a825740 1 --2- 172.21.15.43:0/2440339335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12dbab0c0 0x55d12dbaf100 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff658009a60 tx=0x7ff658009bd0 comp rx=0 tx=0).stop 2024-09-11T23:29:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.680+0000 7ff66a825740 1 -- 172.21.15.43:0/2440339335 shutdown_connections 2024-09-11T23:29:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.680+0000 7ff66a825740 1 --2- 172.21.15.43:0/2440339335 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55d12dc26540 0x55d12dc28960 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.680+0000 7ff66a825740 1 --2- 172.21.15.43:0/2440339335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d12dbab0c0 0x55d12dbaf100 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.680+0000 7ff66a825740 1 --2- 172.21.15.43:0/2440339335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d12db9c2f0 0x55d12dbaeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.680+0000 7ff6497fa640 0 client.16638 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.680+0000 7ff66a825740 1 --2- 172.21.15.43:0/2440339335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d12db9b9d0 0x55d12dbb3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.680+0000 7ff66a825740 1 -- 172.21.15.43:0/2440339335 >> 172.21.15.43:0/2440339335 conn(0x55d12db98060 msgr2=0x55d12dba7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.681+0000 7ff66a825740 1 -- 172.21.15.43:0/2440339335 shutdown_connections 2024-09-11T23:29:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.681+0000 7ff66a825740 1 -- 172.21.15.43:0/2440339335 wait complete. 2024-09-11T23:29:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.681+0000 7ff66a825740 20 client.16638 trim_cache size 0 max 16384 2024-09-11T23:29:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:29:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:29:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:29:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:29:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.681+0000 7ff667a8c640 1 -- 172.21.15.43:0/73560988 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff64c0a2cf0 msgr2=0x7ff64c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.681+0000 7ff667a8c640 1 --2- 172.21.15.43:0/73560988 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff64c0a2cf0 0x7ff64c0a51b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff658005e30 tx=0x7ff658032fd0 comp rx=0 tx=0).stop 2024-09-11T23:29:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.681+0000 7ff667a8c640 1 -- 172.21.15.43:0/73560988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff660101370 msgr2=0x7ff660198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.681+0000 7ff667a8c640 1 --2- 172.21.15.43:0/73560988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff660101370 0x7ff660198910 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff65000e9f0 tx=0x7ff65000edf0 comp rx=0 tx=0).stop 2024-09-11T23:29:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.681+0000 7ff667a8c640 1 -- 172.21.15.43:0/73560988 shutdown_connections 2024-09-11T23:29:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.681+0000 7ff667a8c640 1 --2- 172.21.15.43:0/73560988 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7ff64c0a2cf0 0x7ff64c0a51b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.682+0000 7ff667a8c640 1 --2- 172.21.15.43:0/73560988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6601069e0 0x7ff66019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.682+0000 7ff667a8c640 1 --2- 172.21.15.43:0/73560988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff660101370 0x7ff660198910 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.682+0000 7ff667a8c640 1 --2- 172.21.15.43:0/73560988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6601009d0 0x7ff6601983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.682+0000 7ff667a8c640 1 -- 172.21.15.43:0/73560988 >> 172.21.15.43:0/73560988 conn(0x7ff6600fc6b0 msgr2=0x7ff66006a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.682+0000 7ff667a8c640 1 -- 172.21.15.43:0/73560988 shutdown_connections 2024-09-11T23:29:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:38.682+0000 7ff667a8c640 1 -- 172.21.15.43:0/73560988 wait complete. 2024-09-11T23:29:39.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:39 smithi090 ceph-mon[35130]: pgmap v240: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.2 KiB/s wr, 49 op/s 2024-09-11T23:29:39.800 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:39 smithi097 ceph-mon[35562]: pgmap v240: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.2 KiB/s wr, 49 op/s 2024-09-11T23:29:39.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:39 smithi043 ceph-mon[30950]: pgmap v240: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.2 KiB/s wr, 49 op/s 2024-09-11T23:29:41.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:41 smithi043 ceph-mon[30950]: pgmap v241: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.3 KiB/s wr, 59 op/s 2024-09-11T23:29:41.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:41 smithi090 ceph-mon[35130]: pgmap v241: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.3 KiB/s wr, 59 op/s 2024-09-11T23:29:41.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:41 smithi097 ceph-mon[35562]: pgmap v241: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.3 KiB/s wr, 59 op/s 2024-09-11T23:29:42.270 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 59d4cefc --release --await' 2024-09-11T23:29:42.270 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 59d4cefc --release --await 2024-09-11T23:29:43.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.310+0000 7fac6d467640 1 -- 172.21.15.43:0/1369657728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac6810ad90 msgr2=0x7fac68117220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:43.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.310+0000 7fac6d467640 1 --2- 172.21.15.43:0/1369657728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac6810ad90 0x7fac68117220 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fac60009f00 tx=0x7fac6002abf0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.310+0000 7fac6d467640 1 -- 172.21.15.43:0/1369657728 shutdown_connections 2024-09-11T23:29:43.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.310+0000 7fac6d467640 1 --2- 172.21.15.43:0/1369657728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac6810ad90 0x7fac68117220 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.310+0000 7fac6d467640 1 --2- 172.21.15.43:0/1369657728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac6810a410 0x7fac6810a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.310+0000 7fac6d467640 1 --2- 172.21.15.43:0/1369657728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac68110470 0x7fac68110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.310+0000 7fac6d467640 1 -- 172.21.15.43:0/1369657728 >> 172.21.15.43:0/1369657728 conn(0x7fac68071630 msgr2=0x7fac68073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:43.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.311+0000 7fac6d467640 1 -- 172.21.15.43:0/1369657728 shutdown_connections 2024-09-11T23:29:43.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.312+0000 7fac6d467640 1 -- 172.21.15.43:0/1369657728 wait complete. 2024-09-11T23:29:43.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.313+0000 7fac6d467640 1 Processor -- start 2024-09-11T23:29:43.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.313+0000 7fac6d467640 1 -- start start 2024-09-11T23:29:43.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.314+0000 7fac6d467640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac6810a410 0x7fac681a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.314+0000 7fac6d467640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac6810ad90 0x7fac681a6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.314+0000 7fac6d467640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac68110470 0x7fac681a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.314+0000 7fac6d467640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fac6811a870 con 0x7fac6810ad90 2024-09-11T23:29:43.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.314+0000 7fac6d467640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac6811a6f0 con 0x7fac6810a410 2024-09-11T23:29:43.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.314+0000 7fac6d467640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fac6811a9f0 con 0x7fac68110470 2024-09-11T23:29:43.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.317+0000 7fac66ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac6810a410 0x7fac681a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.317+0000 7fac66ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac6810a410 0x7fac681a6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:56662/0 (socket says 172.21.15.43:56662) 2024-09-11T23:29:43.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.317+0000 7fac66ffd640 1 -- 172.21.15.43:0/2805266408 learned_addr learned my addr 172.21.15.43:0/2805266408 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:43.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.317+0000 7fac677fe640 1 --2- 172.21.15.43:0/2805266408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac68110470 0x7fac681a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.318+0000 7fac667fc640 1 --2- 172.21.15.43:0/2805266408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac6810ad90 0x7fac681a6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.319+0000 7fac66ffd640 1 -- 172.21.15.43:0/2805266408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac68110470 msgr2=0x7fac681a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:43.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.319+0000 7fac66ffd640 1 --2- 172.21.15.43:0/2805266408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac68110470 0x7fac681a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.319+0000 7fac66ffd640 1 -- 172.21.15.43:0/2805266408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac6810ad90 msgr2=0x7fac681a6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:43.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.319+0000 7fac66ffd640 1 --2- 172.21.15.43:0/2805266408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac6810ad90 0x7fac681a6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.319+0000 7fac66ffd640 1 -- 172.21.15.43:0/2805266408 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac681a1380 con 0x7fac6810a410 2024-09-11T23:29:43.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.319+0000 7fac66ffd640 1 --2- 172.21.15.43:0/2805266408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac6810a410 0x7fac681a6a40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fac580099a0 tx=0x7fac58009e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:43.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.319+0000 7fac677fe640 1 --2- 172.21.15.43:0/2805266408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac68110470 0x7fac681a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:43.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.320+0000 7fac4ffff640 1 -- 172.21.15.43:0/2805266408 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5801d070 con 0x7fac6810a410 2024-09-11T23:29:43.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.320+0000 7fac4ffff640 1 -- 172.21.15.43:0/2805266408 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac58018440 con 0x7fac6810a410 2024-09-11T23:29:43.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.320+0000 7fac4ffff640 1 -- 172.21.15.43:0/2805266408 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac580165a0 con 0x7fac6810a410 2024-09-11T23:29:43.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.321+0000 7fac6d467640 1 -- 172.21.15.43:0/2805266408 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac681a1610 con 0x7fac6810a410 2024-09-11T23:29:43.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.321+0000 7fac6d467640 1 -- 172.21.15.43:0/2805266408 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac6806b8a0 con 0x7fac6810a410 2024-09-11T23:29:43.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.329+0000 7fac6f840740 1 Processor -- start 2024-09-11T23:29:43.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.329+0000 7fac6f840740 1 -- start start 2024-09-11T23:29:43.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.330+0000 7fac6f840740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab18350 0x56012ab168a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.330+0000 7fac6f840740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab16de0 0x56012ab17220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.330+0000 7fac6f840740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56012ab12690 0x56012ab2cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.330+0000 7fac6f840740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56012aa742f0 con 0x56012ab16de0 2024-09-11T23:29:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.330+0000 7fac6f840740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56012aa1bcf0 con 0x56012ab18350 2024-09-11T23:29:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.330+0000 7fac6f840740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56012aa702b0 con 0x56012ab12690 2024-09-11T23:29:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.330+0000 7fac4ffff640 1 -- 172.21.15.43:0/2805266408 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fac58020070 con 0x7fac6810a410 2024-09-11T23:29:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.331+0000 7fac66ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab16de0 0x56012ab17220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.331+0000 7fac66ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab16de0 0x56012ab17220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42650/0 (socket says 172.21.15.43:42650) 2024-09-11T23:29:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.331+0000 7fac66ffd640 1 -- 172.21.15.43:0/1146126032 learned_addr learned my addr 172.21.15.43:0/1146126032 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.333+0000 7fac677fe640 1 --2- 172.21.15.43:0/1146126032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab18350 0x56012ab168a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.334+0000 7fac677fe640 1 -- 172.21.15.43:0/1146126032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56012ab12690 msgr2=0x56012ab2cb40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:29:43.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.334+0000 7fac677fe640 1 --2- 172.21.15.43:0/1146126032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56012ab12690 0x56012ab2cb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.334+0000 7fac677fe640 1 -- 172.21.15.43:0/1146126032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab16de0 msgr2=0x56012ab17220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:43.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.334+0000 7fac677fe640 1 --2- 172.21.15.43:0/1146126032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab16de0 0x56012ab17220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.334+0000 7fac677fe640 1 -- 172.21.15.43:0/1146126032 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56012aa003a0 con 0x56012ab18350 2024-09-11T23:29:43.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.334+0000 7fac66ffd640 1 --2- 172.21.15.43:0/1146126032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab16de0 0x56012ab17220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:43.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.335+0000 7fac677fe640 1 --2- 172.21.15.43:0/1146126032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab18350 0x56012ab168a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fac60006010 tx=0x7fac60033ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eed061b12a3d9743 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:43.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.335+0000 7fac4d7fa640 1 -- 172.21.15.43:0/1146126032 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60043020 con 0x56012ab18350 2024-09-11T23:29:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.335+0000 7fac4d7fa640 1 -- 172.21.15.43:0/1146126032 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac60039040 con 0x56012ab18350 2024-09-11T23:29:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.335+0000 7fac4d7fa640 1 -- 172.21.15.43:0/1146126032 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60038070 con 0x56012ab18350 2024-09-11T23:29:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.335+0000 7fac4ffff640 1 --2- 172.21.15.43:0/2805266408 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fac500a2ec0 0x7fac500a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.336+0000 7fac4ffff640 1 -- 172.21.15.43:0/2805266408 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7fac580d2390 con 0x7fac6810a410 2024-09-11T23:29:43.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.338+0000 7fac6f840740 1 -- 172.21.15.43:0/1146126032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab18350 msgr2=0x56012ab168a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:43.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.338+0000 7fac6f840740 1 --2- 172.21.15.43:0/1146126032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab18350 0x56012ab168a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fac60006010 tx=0x7fac60033ad0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.338+0000 7fac6f840740 1 -- 172.21.15.43:0/1146126032 shutdown_connections 2024-09-11T23:29:43.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.338+0000 7fac6f840740 1 --2- 172.21.15.43:0/1146126032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56012ab12690 0x56012ab2cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.338+0000 7fac6f840740 1 --2- 172.21.15.43:0/1146126032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab16de0 0x56012ab17220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.338+0000 7fac6f840740 1 --2- 172.21.15.43:0/1146126032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab18350 0x56012ab168a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.338+0000 7fac6f840740 1 -- 172.21.15.43:0/1146126032 >> 172.21.15.43:0/1146126032 conn(0x56012aad7310 msgr2=0x56012ab13470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:43.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.340+0000 7fac66ffd640 1 --2- 172.21.15.43:0/2805266408 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fac500a2ec0 0x7fac500a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.340+0000 7fac6f840740 1 -- 172.21.15.43:0/1146126032 shutdown_connections 2024-09-11T23:29:43.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.340+0000 7fac6f840740 1 -- 172.21.15.43:0/1146126032 wait complete. 2024-09-11T23:29:43.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.341+0000 7fac6f840740 1 -- start start 2024-09-11T23:29:43.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.341+0000 7fac66ffd640 1 --2- 172.21.15.43:0/2805266408 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fac500a2ec0 0x7fac500a5380 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fac5809e970 tx=0x7fac580188f0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:43.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.341+0000 7fac6f840740 1 Processor -- start 2024-09-11T23:29:43.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.342+0000 7fac6f840740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab12690 0x56012ab12000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.342+0000 7fac6f840740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56012ab16de0 0x56012ab0f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.342+0000 7fac6f840740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab18350 0x56012ab0f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.342+0000 7fac667fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56012ab16de0 0x56012ab0f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.343+0000 7fac667fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56012ab16de0 0x56012ab0f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:34014/0 (socket says 172.21.15.43:34014) 2024-09-11T23:29:43.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.343+0000 7fac667fc640 1 -- 172.21.15.43:0/802764657 learned_addr learned my addr 172.21.15.43:0/802764657 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:43.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.343+0000 7fac6f840740 1 -- 172.21.15.43:0/802764657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56012ab2d080 con 0x56012ab18350 2024-09-11T23:29:43.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.343+0000 7fac6f840740 1 -- 172.21.15.43:0/802764657 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56012aa2f060 con 0x56012ab12690 2024-09-11T23:29:43.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.343+0000 7fac6f840740 1 -- 172.21.15.43:0/802764657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56012ab2d200 con 0x56012ab16de0 2024-09-11T23:29:43.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.343+0000 7fac6f840740 10 client.0 _ll_register_callbacks cb 0x56012a7828c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:43.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.344+0000 7fac677fe640 1 --2- 172.21.15.43:0/802764657 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab12690 0x56012ab12000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.345+0000 7fac677fe640 1 --2- 172.21.15.43:0/802764657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab18350 0x56012ab0f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.345+0000 7fac677fe640 1 -- 172.21.15.43:0/802764657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56012ab16de0 msgr2=0x56012ab0f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:43.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.345+0000 7fac677fe640 1 --2- 172.21.15.43:0/802764657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56012ab16de0 0x56012ab0f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.345+0000 7fac677fe640 1 -- 172.21.15.43:0/802764657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab18350 msgr2=0x56012ab0f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:43.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.345+0000 7fac677fe640 1 --2- 172.21.15.43:0/802764657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab18350 0x56012ab0f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.345+0000 7fac677fe640 1 -- 172.21.15.43:0/802764657 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56012a7e53c0 con 0x56012ab12690 2024-09-11T23:29:43.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.345+0000 7fac667fc640 1 --2- 172.21.15.43:0/802764657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56012ab16de0 0x56012ab0f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:29:43.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.345+0000 7fac677fe640 1 --2- 172.21.15.43:0/802764657 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab12690 0x56012ab12000 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fac60005cc0 tx=0x7fac60009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f9032ab699dd0b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:43.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.345+0000 7fac4aff5640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:29:43.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.346+0000 7fac4aff5640 1 -- 172.21.15.43:0/802764657 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60050030 con 0x56012ab12690 2024-09-11T23:29:43.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.346+0000 7fac4aff5640 1 -- 172.21.15.43:0/802764657 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7fac60033650 con 0x56012ab12690 2024-09-11T23:29:43.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.346+0000 7fac6f840740 1 -- 172.21.15.43:0/802764657 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac60039040 con 0x56012ab12690 2024-09-11T23:29:43.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.347+0000 7fac4aff5640 1 -- 172.21.15.43:0/802764657 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fac6005b050 con 0x56012ab12690 2024-09-11T23:29:43.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.347+0000 7fac6f840740 10 client.26129 fetch_fsmap learned FSMap version 31 2024-09-11T23:29:43.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.347+0000 7fac6f840740 1 -- 172.21.15.43:0/802764657 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56012ab38a90 con 0x56012ab12690 2024-09-11T23:29:43.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.351+0000 7fac4aff5640 1 -- 172.21.15.43:0/802764657 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac60033850 con 0x56012ab12690 2024-09-11T23:29:43.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.351+0000 7fac4aff5640 1 -- 172.21.15.43:0/802764657 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fac6000b040 con 0x56012ab12690 2024-09-11T23:29:43.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.352+0000 7fac6f840740 10 client.26129 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:29:43.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.352+0000 7fac6f840740 10 client.26129 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T23:29:43.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.353+0000 7fac6f840740 20 client.26129 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:43.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.353+0000 7fac6f840740 1 --2- 172.21.15.43:0/802764657 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x56012ab39de0 0x56012ab3c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.353+0000 7fac6f840740 4 client.26129 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:29:43.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.353+0000 7fac6f840740 1 -- 172.21.15.43:0/802764657 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fac60039040 con 0x56012ab39de0 2024-09-11T23:29:43.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.355+0000 7fac667fc640 1 --2- 172.21.15.43:0/802764657 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x56012ab39de0 0x56012ab3c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.356+0000 7fac4aff5640 10 client.26129 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:29:43.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.356+0000 7fac667fc640 1 --2- 172.21.15.43:0/802764657 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x56012ab39de0 0x56012ab3c1d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2ffd25026958684 server_cookie=71a195d08860bc92 in_seq=0 out_seq=0 2024-09-11T23:29:43.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.358+0000 7fac4aff5640 1 -- 172.21.15.43:0/802764657 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fac60039040 con 0x56012ab39de0 2024-09-11T23:29:43.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.358+0000 7fac4aff5640 10 client.26129 handle_command_reply: tid=0 2024-09-11T23:29:43.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.358+0000 7fac6f840740 1 client.26129 shutdown 2024-09-11T23:29:43.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.358+0000 7fac6f840740 1 -- 172.21.15.43:0/802764657 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab12690 msgr2=0x56012ab12000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:43.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.358+0000 7fac6f840740 1 --2- 172.21.15.43:0/802764657 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab12690 0x56012ab12000 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fac60005cc0 tx=0x7fac60009360 comp rx=0 tx=0).stop 2024-09-11T23:29:43.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.358+0000 7fac6f840740 1 -- 172.21.15.43:0/802764657 shutdown_connections 2024-09-11T23:29:43.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.358+0000 7fac6f840740 1 --2- 172.21.15.43:0/802764657 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x56012ab39de0 0x56012ab3c1d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.358+0000 7fac6f840740 1 --2- 172.21.15.43:0/802764657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab18350 0x56012ab0f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.358+0000 7fac6f840740 1 --2- 172.21.15.43:0/802764657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56012ab16de0 0x56012ab0f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.358+0000 7fac6f840740 1 --2- 172.21.15.43:0/802764657 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab12690 0x56012ab12000 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.358+0000 7fac6f840740 1 -- 172.21.15.43:0/802764657 >> 172.21.15.43:0/802764657 conn(0x56012aad7310 msgr2=0x56012ab19170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:43.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.358+0000 7fac4aff5640 0 client.26129 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:29:43.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.358+0000 7fac6f840740 1 -- 172.21.15.43:0/802764657 shutdown_connections 2024-09-11T23:29:43.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.359+0000 7fac6f840740 1 -- 172.21.15.43:0/802764657 wait complete. 2024-09-11T23:29:43.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.359+0000 7fac6f840740 20 client.26129 trim_cache size 0 max 16384 2024-09-11T23:29:43.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.386+0000 7fac6f840740 1 Processor -- start 2024-09-11T23:29:43.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.387+0000 7fac6f840740 1 -- start start 2024-09-11T23:29:43.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.387+0000 7fac6f840740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56012ab12690 0x56012ab36320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.387+0000 7fac6f840740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab16de0 0x56012ab0f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.387+0000 7fac6f840740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab18350 0x56012ab0a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.388+0000 7fac6f840740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fac681b68d0 con 0x56012ab16de0 2024-09-11T23:29:43.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.388+0000 7fac677fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56012ab12690 0x56012ab36320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.388+0000 7fac6f840740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac681b4170 con 0x56012ab18350 2024-09-11T23:29:43.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.388+0000 7fac6f840740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fac681b1fe0 con 0x56012ab12690 2024-09-11T23:29:43.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.388+0000 7fac677fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56012ab12690 0x56012ab36320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:34028/0 (socket says 172.21.15.43:34028) 2024-09-11T23:29:43.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.388+0000 7fac677fe640 1 -- 172.21.15.43:0/2120880174 learned_addr learned my addr 172.21.15.43:0/2120880174 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:43.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.388+0000 7fac677fe640 1 --2- 172.21.15.43:0/2120880174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab18350 0x56012ab0a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.388+0000 7fac667fc640 1 --2- 172.21.15.43:0/2120880174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab16de0 0x56012ab0f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.389+0000 7fac667fc640 1 -- 172.21.15.43:0/2120880174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56012ab12690 msgr2=0x56012ab36320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:43.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.389+0000 7fac667fc640 1 --2- 172.21.15.43:0/2120880174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56012ab12690 0x56012ab36320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.389+0000 7fac667fc640 1 -- 172.21.15.43:0/2120880174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab18350 msgr2=0x56012ab0a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:43.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.389+0000 7fac667fc640 1 --2- 172.21.15.43:0/2120880174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab18350 0x56012ab0a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.389+0000 7fac667fc640 1 -- 172.21.15.43:0/2120880174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56012ab0fb80 con 0x56012ab16de0 2024-09-11T23:29:43.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.389+0000 7fac667fc640 1 --2- 172.21.15.43:0/2120880174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab16de0 0x56012ab0f1e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x56012aa1be00 tx=0x7fac5c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca9e44b38f005f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:43.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.391+0000 7fac4cff9640 1 -- 172.21.15.43:0/2120880174 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c00f040 con 0x56012ab16de0 2024-09-11T23:29:43.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.391+0000 7fac4cff9640 1 -- 172.21.15.43:0/2120880174 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac5c023070 con 0x56012ab16de0 2024-09-11T23:29:43.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.391+0000 7fac4cff9640 1 -- 172.21.15.43:0/2120880174 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c0156b0 con 0x56012ab16de0 2024-09-11T23:29:43.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.392+0000 7fac6f840740 1 -- 172.21.15.43:0/2120880174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab16de0 msgr2=0x56012ab0f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:43.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.392+0000 7fac6f840740 1 --2- 172.21.15.43:0/2120880174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab16de0 0x56012ab0f1e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x56012aa1be00 tx=0x7fac5c008040 comp rx=0 tx=0).stop 2024-09-11T23:29:43.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.398+0000 7fac6f840740 1 -- 172.21.15.43:0/2120880174 shutdown_connections 2024-09-11T23:29:43.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.398+0000 7fac6f840740 1 --2- 172.21.15.43:0/2120880174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab18350 0x56012ab0a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.398+0000 7fac6f840740 1 --2- 172.21.15.43:0/2120880174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab16de0 0x56012ab0f1e0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.398+0000 7fac6f840740 1 --2- 172.21.15.43:0/2120880174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56012ab12690 0x56012ab36320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.398+0000 7fac6f840740 1 -- 172.21.15.43:0/2120880174 >> 172.21.15.43:0/2120880174 conn(0x56012aad7310 msgr2=0x56012ab0e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:43.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.402+0000 7fac6f840740 1 -- 172.21.15.43:0/2120880174 shutdown_connections 2024-09-11T23:29:43.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.402+0000 7fac6f840740 1 -- 172.21.15.43:0/2120880174 wait complete. 2024-09-11T23:29:43.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.402+0000 7fac6f840740 1 -- start start 2024-09-11T23:29:43.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.403+0000 7fac6f840740 1 Processor -- start 2024-09-11T23:29:43.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.403+0000 7fac6f840740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab12690 0x56012ab328f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.403+0000 7fac6f840740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab16de0 0x56012ab0bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.404+0000 7fac6f840740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56012ab18350 0x56012ab0c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.404+0000 7fac6f840740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fac681b2630 con 0x56012ab16de0 2024-09-11T23:29:43.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.404+0000 7fac6f840740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac681b3aa0 con 0x56012ab12690 2024-09-11T23:29:43.411 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:29:43.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.404+0000 7fac6f840740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fac681b2d00 con 0x56012ab18350 2024-09-11T23:29:43.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.406+0000 7fac6f840740 10 client.0 _ll_register_callbacks cb 0x7fac60009940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:43.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.405+0000 7fac677fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab12690 0x56012ab328f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.408+0000 7fac677fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab12690 0x56012ab328f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:56700/0 (socket says 172.21.15.43:56700) 2024-09-11T23:29:43.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.408+0000 7fac677fe640 1 -- 172.21.15.43:0/2402360531 learned_addr learned my addr 172.21.15.43:0/2402360531 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:43.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.409+0000 7fac677fe640 1 -- 172.21.15.43:0/2402360531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56012ab18350 msgr2=0x56012ab0c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:43.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.409+0000 7fac677fe640 1 --2- 172.21.15.43:0/2402360531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56012ab18350 0x56012ab0c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.409+0000 7fac677fe640 1 -- 172.21.15.43:0/2402360531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab16de0 msgr2=0x56012ab0bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:43.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.409+0000 7fac677fe640 1 --2- 172.21.15.43:0/2402360531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab16de0 0x56012ab0bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.409+0000 7fac677fe640 1 -- 172.21.15.43:0/2402360531 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56012ab0c7d0 con 0x56012ab12690 2024-09-11T23:29:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.412+0000 7fac677fe640 1 --2- 172.21.15.43:0/2402360531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab12690 0x56012ab328f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fac60006010 tx=0x7fac60004980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f6e9875edfe3f11 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.413+0000 7fac4aff5640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:29:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.413+0000 7fac4aff5640 1 -- 172.21.15.43:0/2402360531 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60050040 con 0x56012ab12690 2024-09-11T23:29:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.413+0000 7fac4aff5640 1 -- 172.21.15.43:0/2402360531 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7fac60033850 con 0x56012ab12690 2024-09-11T23:29:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.413+0000 7fac6f840740 1 -- 172.21.15.43:0/2402360531 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac5c023070 con 0x56012ab12690 2024-09-11T23:29:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.415+0000 7fac4aff5640 1 -- 172.21.15.43:0/2402360531 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fac60038030 con 0x56012ab12690 2024-09-11T23:29:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.415+0000 7fac6f840740 10 client.26135 fetch_fsmap learned FSMap version 31 2024-09-11T23:29:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.415+0000 7fac6f840740 1 -- 172.21.15.43:0/2402360531 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56012aba44b0 con 0x56012ab12690 2024-09-11T23:29:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.417+0000 7fac4aff5640 1 -- 172.21.15.43:0/2402360531 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac5c023070 con 0x56012ab12690 2024-09-11T23:29:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.417+0000 7fac4aff5640 1 -- 172.21.15.43:0/2402360531 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7fac6000b040 con 0x56012ab12690 2024-09-11T23:29:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.417+0000 7fac6f840740 10 client.26135 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:29:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.417+0000 7fac6f840740 10 client.26135 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T23:29:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.417+0000 7fac6f840740 20 client.26135 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.417+0000 7fac6f840740 1 --2- 172.21.15.43:0/2402360531 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x56012aba4be0 0x56012aba7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.417+0000 7fac6f840740 4 client.26135 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:29:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.417+0000 7fac6f840740 1 -- 172.21.15.43:0/2402360531 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56012aa81860 con 0x56012aba4be0 2024-09-11T23:29:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.418+0000 7fac667fc640 1 --2- 172.21.15.43:0/2402360531 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x56012aba4be0 0x56012aba7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:43.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.423+0000 7fac4aff5640 10 client.26135 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:29:43.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.423+0000 7fac667fc640 1 --2- 172.21.15.43:0/2402360531 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x56012aba4be0 0x56012aba7000 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7aad150387caa397 server_cookie=725302ac865ec8e0 in_seq=0 out_seq=0 2024-09-11T23:29:43.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.424+0000 7fac4aff5640 1 -- 172.21.15.43:0/2402360531 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56012aa81860 con 0x56012aba4be0 2024-09-11T23:29:43.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.424+0000 7fac4aff5640 10 client.26135 handle_command_reply: tid=0 2024-09-11T23:29:43.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.424+0000 7fac6f840740 1 client.26135 shutdown 2024-09-11T23:29:43.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.424+0000 7fac6f840740 1 -- 172.21.15.43:0/2402360531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab12690 msgr2=0x56012ab328f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:43.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.424+0000 7fac6f840740 1 --2- 172.21.15.43:0/2402360531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab12690 0x56012ab328f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fac60006010 tx=0x7fac60004980 comp rx=0 tx=0).stop 2024-09-11T23:29:43.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.424+0000 7fac6f840740 1 -- 172.21.15.43:0/2402360531 shutdown_connections 2024-09-11T23:29:43.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.424+0000 7fac6f840740 1 --2- 172.21.15.43:0/2402360531 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x56012aba4be0 0x56012aba7000 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.424+0000 7fac6f840740 1 --2- 172.21.15.43:0/2402360531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56012ab18350 0x56012ab0c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.424+0000 7fac6f840740 1 --2- 172.21.15.43:0/2402360531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56012ab16de0 0x56012ab0bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.424+0000 7fac6f840740 1 --2- 172.21.15.43:0/2402360531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56012ab12690 0x56012ab328f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.424+0000 7fac4aff5640 0 client.26135 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:29:43.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.424+0000 7fac6f840740 1 -- 172.21.15.43:0/2402360531 >> 172.21.15.43:0/2402360531 conn(0x56012aad7310 msgr2=0x56012ab188a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:43.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.425+0000 7fac6f840740 1 -- 172.21.15.43:0/2402360531 shutdown_connections 2024-09-11T23:29:43.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.425+0000 7fac6f840740 1 -- 172.21.15.43:0/2402360531 wait complete. 2024-09-11T23:29:43.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.428+0000 7fac6f840740 20 client.26135 trim_cache size 0 max 16384 2024-09-11T23:29:43.429 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:29:43.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.431+0000 7fac6d467640 1 -- 172.21.15.43:0/2805266408 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fac500a2ec0 msgr2=0x7fac500a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:43.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.431+0000 7fac6d467640 1 --2- 172.21.15.43:0/2805266408 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fac500a2ec0 0x7fac500a5380 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fac5809e970 tx=0x7fac580188f0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.431+0000 7fac6d467640 1 -- 172.21.15.43:0/2805266408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac6810a410 msgr2=0x7fac681a6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:43.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.431+0000 7fac6d467640 1 --2- 172.21.15.43:0/2805266408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac6810a410 0x7fac681a6a40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fac580099a0 tx=0x7fac58009e60 comp rx=0 tx=0).stop 2024-09-11T23:29:43.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.431+0000 7fac6d467640 1 -- 172.21.15.43:0/2805266408 shutdown_connections 2024-09-11T23:29:43.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.431+0000 7fac6d467640 1 --2- 172.21.15.43:0/2805266408 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fac500a2ec0 0x7fac500a5380 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.431+0000 7fac6d467640 1 --2- 172.21.15.43:0/2805266408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac68110470 0x7fac681a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.431+0000 7fac6d467640 1 --2- 172.21.15.43:0/2805266408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac6810ad90 0x7fac681a6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.431+0000 7fac6d467640 1 --2- 172.21.15.43:0/2805266408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac6810a410 0x7fac681a6a40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:43.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.431+0000 7fac6d467640 1 -- 172.21.15.43:0/2805266408 >> 172.21.15.43:0/2805266408 conn(0x7fac68071630 msgr2=0x7fac68072dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:43.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.432+0000 7fac6d467640 1 -- 172.21.15.43:0/2805266408 shutdown_connections 2024-09-11T23:29:43.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:43.432+0000 7fac6d467640 1 -- 172.21.15.43:0/2805266408 wait complete. 2024-09-11T23:29:43.548 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:43 smithi043 ceph-mon[30950]: pgmap v242: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 56 op/s 2024-09-11T23:29:43.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:43 smithi090 ceph-mon[35130]: pgmap v242: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 56 op/s 2024-09-11T23:29:43.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:43 smithi097 ceph-mon[35562]: pgmap v242: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 56 op/s 2024-09-11T23:29:45.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:45 smithi090 ceph-mon[35130]: pgmap v243: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.1 KiB/s wr, 47 op/s 2024-09-11T23:29:45.800 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:45 smithi097 ceph-mon[35562]: pgmap v243: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.1 KiB/s wr, 47 op/s 2024-09-11T23:29:45.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:45 smithi043 ceph-mon[30950]: pgmap v243: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.1 KiB/s wr, 47 op/s 2024-09-11T23:29:45.874 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T23:29:45.874 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T23:29:45.912 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-09-11T23:29:45.912 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-11T23:29:46.276 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:29:47.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.161+0000 7f3472045640 1 -- 172.21.15.43:0/2479881951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f346c102de0 msgr2=0x7f346c1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:47.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.161+0000 7f3472045640 1 --2- 172.21.15.43:0/2479881951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f346c102de0 0x7f346c1031c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f345c009900 tx=0x7f345c02a480 comp rx=0 tx=0).stop 2024-09-11T23:29:47.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.161+0000 7f3472045640 1 -- 172.21.15.43:0/2479881951 shutdown_connections 2024-09-11T23:29:47.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.161+0000 7f3472045640 1 --2- 172.21.15.43:0/2479881951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f346c10c180 0x7f346c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.161+0000 7f3472045640 1 --2- 172.21.15.43:0/2479881951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f346c103700 0x7f346c10bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.161+0000 7f3472045640 1 --2- 172.21.15.43:0/2479881951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f346c102de0 0x7f346c1031c0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.161+0000 7f3472045640 1 -- 172.21.15.43:0/2479881951 >> 172.21.15.43:0/2479881951 conn(0x7f346c0fc7a0 msgr2=0x7f346c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:47.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.161+0000 7f3472045640 1 -- 172.21.15.43:0/2479881951 shutdown_connections 2024-09-11T23:29:47.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.161+0000 7f3472045640 1 -- 172.21.15.43:0/2479881951 wait complete. 2024-09-11T23:29:47.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.162+0000 7f3472045640 1 Processor -- start 2024-09-11T23:29:47.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.162+0000 7f3472045640 1 -- start start 2024-09-11T23:29:47.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.162+0000 7f3472045640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f346c102de0 0x7f346c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:47.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.162+0000 7f3472045640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f346c103700 0x7f346c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:47.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.162+0000 7f3472045640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f346c10c180 0x7f346c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:47.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.162+0000 7f3472045640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f346c111f70 con 0x7f346c10c180 2024-09-11T23:29:47.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.162+0000 7f3472045640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f346c111df0 con 0x7f346c102de0 2024-09-11T23:29:47.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.162+0000 7f3472045640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1120f0 con 0x7f346c103700 2024-09-11T23:29:47.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.162+0000 7f346bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f346c10c180 0x7f346c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:47.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.162+0000 7f346bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f346c10c180 0x7f346c19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42686/0 (socket says 172.21.15.43:42686) 2024-09-11T23:29:47.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.162+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f346c102de0 0x7f346c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:47.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.162+0000 7f346bfff640 1 -- 172.21.15.43:0/599886803 learned_addr learned my addr 172.21.15.43:0/599886803 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:47.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.162+0000 7f346affd640 1 --2- 172.21.15.43:0/599886803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f346c103700 0x7f346c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:47.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.163+0000 7f346bfff640 1 -- 172.21.15.43:0/599886803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f346c103700 msgr2=0x7f346c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:47.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.163+0000 7f346bfff640 1 --2- 172.21.15.43:0/599886803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f346c103700 0x7f346c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.163+0000 7f346bfff640 1 -- 172.21.15.43:0/599886803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f346c102de0 msgr2=0x7f346c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:47.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.163+0000 7f346bfff640 1 --2- 172.21.15.43:0/599886803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f346c102de0 0x7f346c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.163+0000 7f346bfff640 1 -- 172.21.15.43:0/599886803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346c19d4c0 con 0x7f346c10c180 2024-09-11T23:29:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.163+0000 7f346b7fe640 1 --2- 172.21.15.43:0/599886803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f346c102de0 0x7f346c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:29:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.163+0000 7f346bfff640 1 --2- 172.21.15.43:0/599886803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f346c10c180 0x7f346c19cde0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f346400eaa0 tx=0x7f346400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.163+0000 7f3468ff9640 1 -- 172.21.15.43:0/599886803 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346400cdb0 con 0x7f346c10c180 2024-09-11T23:29:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.163+0000 7f3468ff9640 1 -- 172.21.15.43:0/599886803 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3464005020 con 0x7f346c10c180 2024-09-11T23:29:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.163+0000 7f3472045640 1 -- 172.21.15.43:0/599886803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f346c19d7b0 con 0x7f346c10c180 2024-09-11T23:29:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.163+0000 7f3468ff9640 1 -- 172.21.15.43:0/599886803 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34640116b0 con 0x7f346c10c180 2024-09-11T23:29:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.163+0000 7f3472045640 1 -- 172.21.15.43:0/599886803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f346c1a5090 con 0x7f346c10c180 2024-09-11T23:29:47.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.165+0000 7f3468ff9640 1 -- 172.21.15.43:0/599886803 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3464005560 con 0x7f346c10c180 2024-09-11T23:29:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f347441e740 1 Processor -- start 2024-09-11T23:29:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f347441e740 1 -- start start 2024-09-11T23:29:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f347441e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b718b88060 0x55b718b88440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f347441e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b88980 0x55b718b92d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f347441e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b718b93240 0x55b718b95630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f347441e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b718aed620 con 0x55b718b88980 2024-09-11T23:29:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f347441e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b718a94e50 con 0x55b718b93240 2024-09-11T23:29:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f347441e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b718ae95e0 con 0x55b718b88060 2024-09-11T23:29:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f3468ff9640 1 --2- 172.21.15.43:0/599886803 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f34540a2d70 0x7f34540a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b88980 0x55b718b92d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b88980 0x55b718b92d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42700/0 (socket says 172.21.15.43:42700) 2024-09-11T23:29:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f346b7fe640 1 -- 172.21.15.43:0/871347811 learned_addr learned my addr 172.21.15.43:0/871347811 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f346affd640 1 --2- 172.21.15.43:0/871347811 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b718b93240 0x55b718b95630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f346bfff640 1 --2- 172.21.15.43:0/599886803 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f34540a2d70 0x7f34540a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f346affd640 1 --2- 172.21.15.43:0/871347811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b718b88060 0x55b718b88440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f3468ff9640 1 -- 172.21.15.43:0/599886803 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f34640cf550 con 0x7f346c10c180 2024-09-11T23:29:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f346b7fe640 1 -- 172.21.15.43:0/871347811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b718b88060 msgr2=0x55b718b88440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f346b7fe640 1 --2- 172.21.15.43:0/871347811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b718b88060 0x55b718b88440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f346b7fe640 1 -- 172.21.15.43:0/871347811 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b718b93240 msgr2=0x55b718b95630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f346b7fe640 1 --2- 172.21.15.43:0/871347811 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b718b93240 0x55b718b95630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f346b7fe640 1 -- 172.21.15.43:0/871347811 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b718a85d40 con 0x55b718b88980 2024-09-11T23:29:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.166+0000 7f346affd640 1 --2- 172.21.15.43:0/871347811 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b718b93240 0x55b718b95630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:29:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.167+0000 7f346bfff640 1 --2- 172.21.15.43:0/599886803 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f34540a2d70 0x7f34540a5230 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3464020000 tx=0x7f346406eb60 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.167+0000 7f346b7fe640 1 --2- 172.21.15.43:0/871347811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b88980 0x55b718b92d00 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f345c009900 tx=0x7f345c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04238c97637a17 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.167+0000 7f3451ffb640 1 -- 172.21.15.43:0/871347811 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c035040 con 0x55b718b88980 2024-09-11T23:29:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.167+0000 7f3451ffb640 1 -- 172.21.15.43:0/871347811 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f345c0319f0 con 0x55b718b88980 2024-09-11T23:29:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.167+0000 7f3451ffb640 1 -- 172.21.15.43:0/871347811 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c004a00 con 0x55b718b88980 2024-09-11T23:29:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.168+0000 7f347441e740 1 -- 172.21.15.43:0/871347811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b88980 msgr2=0x55b718b92d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.168+0000 7f347441e740 1 --2- 172.21.15.43:0/871347811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b88980 0x55b718b92d00 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f345c009900 tx=0x7f345c005e80 comp rx=0 tx=0).stop 2024-09-11T23:29:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.168+0000 7f347441e740 1 -- 172.21.15.43:0/871347811 shutdown_connections 2024-09-11T23:29:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.168+0000 7f347441e740 1 --2- 172.21.15.43:0/871347811 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b718b93240 0x55b718b95630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.168+0000 7f347441e740 1 --2- 172.21.15.43:0/871347811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b88980 0x55b718b92d00 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.168+0000 7f347441e740 1 --2- 172.21.15.43:0/871347811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b718b88060 0x55b718b88440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.168+0000 7f347441e740 1 -- 172.21.15.43:0/871347811 >> 172.21.15.43:0/871347811 conn(0x55b718b845f0 msgr2=0x55b718b849d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.168+0000 7f347441e740 1 -- 172.21.15.43:0/871347811 shutdown_connections 2024-09-11T23:29:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.168+0000 7f347441e740 1 -- 172.21.15.43:0/871347811 wait complete. 2024-09-11T23:29:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.168+0000 7f347441e740 1 -- start start 2024-09-11T23:29:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.168+0000 7f347441e740 1 Processor -- start 2024-09-11T23:29:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.169+0000 7f347441e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b718b88060 0x55b718b90080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:47.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.169+0000 7f347441e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b88980 0x55b718b905c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:47.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.169+0000 7f347441e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b718b93240 0x55b718b90b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:47.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.169+0000 7f347441e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b718b95e10 con 0x55b718b88980 2024-09-11T23:29:47.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.169+0000 7f347441e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b718a75f10 con 0x55b718b93240 2024-09-11T23:29:47.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.169+0000 7f347441e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b718b95f90 con 0x55b718b88060 2024-09-11T23:29:47.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.169+0000 7f347441e740 10 client.0 _ll_register_callbacks cb 0x55b718a99db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:47.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.169+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b88980 0x55b718b905c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:47.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.169+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b88980 0x55b718b905c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42704/0 (socket says 172.21.15.43:42704) 2024-09-11T23:29:47.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.169+0000 7f346b7fe640 1 -- 172.21.15.43:0/3547047565 learned_addr learned my addr 172.21.15.43:0/3547047565 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:47.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.169+0000 7f346affd640 1 --2- 172.21.15.43:0/3547047565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b718b88060 0x55b718b90080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:47.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.169+0000 7f346affd640 1 --2- 172.21.15.43:0/3547047565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b718b93240 0x55b718b90b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.169+0000 7f346b7fe640 1 -- 172.21.15.43:0/3547047565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b718b88060 msgr2=0x55b718b90080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.169+0000 7f346b7fe640 1 --2- 172.21.15.43:0/3547047565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b718b88060 0x55b718b90080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.169+0000 7f346b7fe640 1 -- 172.21.15.43:0/3547047565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b718b93240 msgr2=0x55b718b90b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.169+0000 7f346b7fe640 1 --2- 172.21.15.43:0/3547047565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b718b93240 0x55b718b90b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.169+0000 7f346b7fe640 1 -- 172.21.15.43:0/3547047565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7187fc580 con 0x55b718b88980 2024-09-11T23:29:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.169+0000 7f346b7fe640 1 --2- 172.21.15.43:0/3547047565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b88980 0x55b718b905c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f345c000c00 tx=0x7f345c0042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7164a7294a45c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.169+0000 7f344affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:29:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.170+0000 7f344affd640 1 -- 172.21.15.43:0/3547047565 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c04e070 con 0x55b718b88980 2024-09-11T23:29:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.170+0000 7f344affd640 1 -- 172.21.15.43:0/3547047565 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f345c0035f0 con 0x55b718b88980 2024-09-11T23:29:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.170+0000 7f347441e740 1 -- 172.21.15.43:0/3547047565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f345c0319f0 con 0x55b718b88980 2024-09-11T23:29:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.170+0000 7f344affd640 1 -- 172.21.15.43:0/3547047565 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f345c05a050 con 0x55b718b88980 2024-09-11T23:29:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.170+0000 7f347441e740 10 client.16665 fetch_fsmap learned FSMap version 31 2024-09-11T23:29:47.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.170+0000 7f347441e740 1 -- 172.21.15.43:0/3547047565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b7188c6960 con 0x55b718b88980 2024-09-11T23:29:47.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.170+0000 7f344affd640 1 -- 172.21.15.43:0/3547047565 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f345c0319f0 con 0x55b718b88980 2024-09-11T23:29:47.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.170+0000 7f344affd640 1 -- 172.21.15.43:0/3547047565 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f345c046970 con 0x55b718b88980 2024-09-11T23:29:47.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.170+0000 7f347441e740 10 client.16665 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:29:47.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.170+0000 7f347441e740 10 client.16665 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:29:47.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.170+0000 7f347441e740 20 client.16665 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:47.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.170+0000 7f347441e740 1 --2- 172.21.15.43:0/3547047565 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b718ba2b90 0x55b718ba4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:47.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.170+0000 7f347441e740 4 client.16665 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:29:47.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.170+0000 7f347441e740 1 -- 172.21.15.43:0/3547047565 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b718afab90 con 0x55b718ba2b90 2024-09-11T23:29:47.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.170+0000 7f346affd640 1 --2- 172.21.15.43:0/3547047565 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b718ba2b90 0x55b718ba4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:47.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.171+0000 7f346affd640 1 --2- 172.21.15.43:0/3547047565 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b718ba2b90 0x55b718ba4f80 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc4d5dfd2579042f server_cookie=89ee05ff2be191bc in_seq=0 out_seq=0 2024-09-11T23:29:47.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.171+0000 7f344affd640 10 client.16665 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:47.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.172+0000 7f344affd640 1 -- 172.21.15.43:0/3547047565 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b718afab90 con 0x55b718ba2b90 2024-09-11T23:29:47.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.172+0000 7f344affd640 10 client.16665 handle_command_reply: tid=0 2024-09-11T23:29:47.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.172+0000 7f347441e740 1 client.16665 shutdown 2024-09-11T23:29:47.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.172+0000 7f347441e740 1 -- 172.21.15.43:0/3547047565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b88980 msgr2=0x55b718b905c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:47.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.172+0000 7f347441e740 1 --2- 172.21.15.43:0/3547047565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b88980 0x55b718b905c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f345c000c00 tx=0x7f345c0042d0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.172+0000 7f347441e740 1 -- 172.21.15.43:0/3547047565 shutdown_connections 2024-09-11T23:29:47.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.172+0000 7f347441e740 1 --2- 172.21.15.43:0/3547047565 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b718ba2b90 0x55b718ba4f80 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.172+0000 7f347441e740 1 --2- 172.21.15.43:0/3547047565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b718b93240 0x55b718b90b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.172+0000 7f344affd640 0 client.16665 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:47.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.172+0000 7f347441e740 1 --2- 172.21.15.43:0/3547047565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b88980 0x55b718b905c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.172+0000 7f347441e740 1 --2- 172.21.15.43:0/3547047565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b718b88060 0x55b718b90080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.172+0000 7f347441e740 1 -- 172.21.15.43:0/3547047565 >> 172.21.15.43:0/3547047565 conn(0x55b718b845f0 msgr2=0x55b718b8a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:47.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.172+0000 7f347441e740 1 -- 172.21.15.43:0/3547047565 shutdown_connections 2024-09-11T23:29:47.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.172+0000 7f347441e740 1 -- 172.21.15.43:0/3547047565 wait complete. 2024-09-11T23:29:47.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.172+0000 7f347441e740 20 client.16665 trim_cache size 0 max 16384 2024-09-11T23:29:47.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.189+0000 7f347441e740 1 Processor -- start 2024-09-11T23:29:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.189+0000 7f347441e740 1 -- start start 2024-09-11T23:29:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.190+0000 7f347441e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b718b88060 0x55b718ba7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.190+0000 7f347441e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b718b88980 0x55b718ba84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.190+0000 7f347441e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b93240 0x55b718b8d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.190+0000 7f347441e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1b0250 con 0x55b718b93240 2024-09-11T23:29:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.190+0000 7f347441e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1adaf0 con 0x55b718b88980 2024-09-11T23:29:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.190+0000 7f347441e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1ab960 con 0x55b718b88060 2024-09-11T23:29:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.190+0000 7f346affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b93240 0x55b718b8d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:47.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.190+0000 7f346affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b93240 0x55b718b8d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42714/0 (socket says 172.21.15.43:42714) 2024-09-11T23:29:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.190+0000 7f346affd640 1 -- 172.21.15.43:0/1844373275 learned_addr learned my addr 172.21.15.43:0/1844373275 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.190+0000 7f346affd640 1 --2- 172.21.15.43:0/1844373275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b718b88060 0x55b718ba7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.190+0000 7f346b7fe640 1 --2- 172.21.15.43:0/1844373275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b718b88980 0x55b718ba84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.190+0000 7f346affd640 1 -- 172.21.15.43:0/1844373275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b718b88060 msgr2=0x55b718ba7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.190+0000 7f346affd640 1 --2- 172.21.15.43:0/1844373275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b718b88060 0x55b718ba7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.190+0000 7f346affd640 1 -- 172.21.15.43:0/1844373275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b718b88980 msgr2=0x55b718ba84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.190+0000 7f346affd640 1 --2- 172.21.15.43:0/1844373275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b718b88980 0x55b718ba84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.190+0000 7f346affd640 1 -- 172.21.15.43:0/1844373275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b718b887b0 con 0x55b718b93240 2024-09-11T23:29:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.190+0000 7f346affd640 1 --2- 172.21.15.43:0/1844373275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b93240 0x55b718b8d2e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3460005d00 tx=0x7f3460005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4246b005d482fcab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.190+0000 7f34517fa640 1 -- 172.21.15.43:0/1844373275 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3460010c90 con 0x55b718b93240 2024-09-11T23:29:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.190+0000 7f34517fa640 1 -- 172.21.15.43:0/1844373275 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3460010e30 con 0x55b718b93240 2024-09-11T23:29:47.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.191+0000 7f34517fa640 1 -- 172.21.15.43:0/1844373275 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3460003db0 con 0x55b718b93240 2024-09-11T23:29:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.191+0000 7f347441e740 1 -- 172.21.15.43:0/1844373275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b93240 msgr2=0x55b718b8d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.191+0000 7f347441e740 1 --2- 172.21.15.43:0/1844373275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b93240 0x55b718b8d2e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3460005d00 tx=0x7f3460005c90 comp rx=0 tx=0).stop 2024-09-11T23:29:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.191+0000 7f347441e740 1 -- 172.21.15.43:0/1844373275 shutdown_connections 2024-09-11T23:29:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.191+0000 7f347441e740 1 --2- 172.21.15.43:0/1844373275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b93240 0x55b718b8d2e0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.191+0000 7f347441e740 1 --2- 172.21.15.43:0/1844373275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b718b88980 0x55b718ba84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.191+0000 7f347441e740 1 --2- 172.21.15.43:0/1844373275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b718b88060 0x55b718ba7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.191+0000 7f347441e740 1 -- 172.21.15.43:0/1844373275 >> 172.21.15.43:0/1844373275 conn(0x55b718b845f0 msgr2=0x55b718b918e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.191+0000 7f347441e740 1 -- 172.21.15.43:0/1844373275 shutdown_connections 2024-09-11T23:29:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.191+0000 7f347441e740 1 -- 172.21.15.43:0/1844373275 wait complete. 2024-09-11T23:29:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.191+0000 7f347441e740 1 -- start start 2024-09-11T23:29:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.192+0000 7f347441e740 1 Processor -- start 2024-09-11T23:29:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.192+0000 7f347441e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b88060 0x55b718ba6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:47.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.192+0000 7f347441e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b718b88980 0x55b718ba6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.192+0000 7f347441e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b718b93240 0x55b718c0c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.192+0000 7f347441e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1abfb0 con 0x55b718b88060 2024-09-11T23:29:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.192+0000 7f347441e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1ad420 con 0x55b718b93240 2024-09-11T23:29:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.192+0000 7f347441e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1ac680 con 0x55b718b88980 2024-09-11T23:29:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.192+0000 7f347441e740 10 client.0 _ll_register_callbacks cb 0x7f346000ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.192+0000 7f346affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b88060 0x55b718ba6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.192+0000 7f346affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b88060 0x55b718ba6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42726/0 (socket says 172.21.15.43:42726) 2024-09-11T23:29:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.192+0000 7f346affd640 1 -- 172.21.15.43:0/3038788940 learned_addr learned my addr 172.21.15.43:0/3038788940 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.192+0000 7f346b7fe640 1 --2- 172.21.15.43:0/3038788940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b718b88980 0x55b718ba6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.192+0000 7f346affd640 1 --2- 172.21.15.43:0/3038788940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b718b93240 0x55b718c0c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.192+0000 7f346affd640 1 -- 172.21.15.43:0/3038788940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b718b88980 msgr2=0x55b718ba6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.192+0000 7f346affd640 1 --2- 172.21.15.43:0/3038788940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b718b88980 0x55b718ba6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.192+0000 7f346affd640 1 -- 172.21.15.43:0/3038788940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b718b93240 msgr2=0x55b718c0c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:47.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.192+0000 7f346affd640 1 --2- 172.21.15.43:0/3038788940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b718b93240 0x55b718c0c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.193+0000 7f346affd640 1 -- 172.21.15.43:0/3038788940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b718c0ca10 con 0x55b718b88060 2024-09-11T23:29:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.193+0000 7f346b7fe640 1 --2- 172.21.15.43:0/3038788940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b718b88980 0x55b718ba6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:29:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.193+0000 7f346affd640 1 --2- 172.21.15.43:0/3038788940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b88060 0x55b718ba6430 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f3460012820 tx=0x7f3460012580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cfebf41bd910c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.193+0000 7f344affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:29:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.193+0000 7f344affd640 1 -- 172.21.15.43:0/3038788940 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3460034020 con 0x55b718b88060 2024-09-11T23:29:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.193+0000 7f344affd640 1 -- 172.21.15.43:0/3038788940 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f3460005240 con 0x55b718b88060 2024-09-11T23:29:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.193+0000 7f347441e740 1 -- 172.21.15.43:0/3038788940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3460010e30 con 0x55b718b88060 2024-09-11T23:29:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.193+0000 7f344affd640 1 -- 172.21.15.43:0/3038788940 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3460005440 con 0x55b718b88060 2024-09-11T23:29:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.193+0000 7f347441e740 10 client.16671 fetch_fsmap learned FSMap version 31 2024-09-11T23:29:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.193+0000 7f347441e740 1 -- 172.21.15.43:0/3038788940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b718c0dcb0 con 0x55b718b88060 2024-09-11T23:29:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.193+0000 7f344affd640 1 -- 172.21.15.43:0/3038788940 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3460010e30 con 0x55b718b88060 2024-09-11T23:29:47.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.193+0000 7f344affd640 1 -- 172.21.15.43:0/3038788940 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f34600223e0 con 0x55b718b88060 2024-09-11T23:29:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.193+0000 7f347441e740 10 client.16671 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:29:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.193+0000 7f347441e740 10 client.16671 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:29:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.193+0000 7f347441e740 20 client.16671 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.193+0000 7f347441e740 1 --2- 172.21.15.43:0/3038788940 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b718c0e270 0x55b718c10660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.193+0000 7f347441e740 4 client.16671 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "59d4cefc", "release": true, "await": true}] 2024-09-11T23:29:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.193+0000 7f347441e740 1 -- 172.21.15.43:0/3038788940 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "quiesce db", "set_id": "59d4cefc", "release": true, "await": true}) -- 0x55b718af50a0 con 0x55b718c0e270 2024-09-11T23:29:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.194+0000 7f346b7fe640 1 --2- 172.21.15.43:0/3038788940 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b718c0e270 0x55b718c10660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.194+0000 7f344affd640 10 client.16671 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.194+0000 7f346b7fe640 1 --2- 172.21.15.43:0/3038788940 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b718c0e270 0x55b718c10660 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce3ea5cbacf6735e server_cookie=3aceb72dc77e3ceb in_seq=0 out_seq=0 2024-09-11T23:29:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.199+0000 7f344affd640 1 -- 172.21.15.43:0/3038788940 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b718af50a0 con 0x55b718c0e270 2024-09-11T23:29:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.199+0000 7f344affd640 10 client.16671 handle_command_reply: tid=0 2024-09-11T23:29:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.199+0000 7f347441e740 1 client.16671 shutdown 2024-09-11T23:29:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.199+0000 7f347441e740 1 -- 172.21.15.43:0/3038788940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b88060 msgr2=0x55b718ba6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:47.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.199+0000 7f347441e740 1 --2- 172.21.15.43:0/3038788940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b88060 0x55b718ba6430 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f3460012820 tx=0x7f3460012580 comp rx=0 tx=0).stop 2024-09-11T23:29:47.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.199+0000 7f347441e740 1 -- 172.21.15.43:0/3038788940 shutdown_connections 2024-09-11T23:29:47.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.199+0000 7f347441e740 1 --2- 172.21.15.43:0/3038788940 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55b718c0e270 0x55b718c10660 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.199+0000 7f347441e740 1 --2- 172.21.15.43:0/3038788940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b718b93240 0x55b718c0c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.199+0000 7f347441e740 1 --2- 172.21.15.43:0/3038788940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b718b88980 0x55b718ba6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.199+0000 7f347441e740 1 --2- 172.21.15.43:0/3038788940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b718b88060 0x55b718ba6430 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.199+0000 7f344affd640 0 client.16671 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:47.399 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:29:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 31, 2024-09-11T23:29:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: "leader": 24497, 2024-09-11T23:29:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: "set_version": 20, 2024-09-11T23:29:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: "sets": { 2024-09-11T23:29:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: "59d4cefc": { 2024-09-11T23:29:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 20, 2024-09-11T23:29:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: "age_ref": 0.0, 2024-09-11T23:29:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:29:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RELEASED", 2024-09-11T23:29:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-09-11T23:29:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:29:47.400 INFO:teuthology.orchestra.run.smithi043.stdout: "timeout": 90.0, 2024-09-11T23:29:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: "expiration": 140.2, 2024-09-11T23:29:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: "members": { 2024-09-11T23:29:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: "file:/": { 2024-09-11T23:29:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: "excluded": false, 2024-09-11T23:29:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: "state": { 2024-09-11T23:29:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RELEASED", 2024-09-11T23:29:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0 2024-09-11T23:29:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:29:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:29:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:29:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:29:47.401 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:29:47.401 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:29:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.199+0000 7f347441e740 1 -- 172.21.15.43:0/3038788940 >> 172.21.15.43:0/3038788940 conn(0x55b718b845f0 msgr2=0x55b718b82f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.199+0000 7f347441e740 1 -- 172.21.15.43:0/3038788940 shutdown_connections 2024-09-11T23:29:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.199+0000 7f347441e740 1 -- 172.21.15.43:0/3038788940 wait complete. 2024-09-11T23:29:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.199+0000 7f347441e740 20 client.16671 trim_cache size 0 max 16384 2024-09-11T23:29:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.200+0000 7f3472045640 1 -- 172.21.15.43:0/599886803 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f34540a2d70 msgr2=0x7f34540a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.200+0000 7f3472045640 1 --2- 172.21.15.43:0/599886803 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f34540a2d70 0x7f34540a5230 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3464020000 tx=0x7f346406eb60 comp rx=0 tx=0).stop 2024-09-11T23:29:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.200+0000 7f3472045640 1 -- 172.21.15.43:0/599886803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f346c10c180 msgr2=0x7f346c19cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.200+0000 7f3472045640 1 --2- 172.21.15.43:0/599886803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f346c10c180 0x7f346c19cde0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f346400eaa0 tx=0x7f346400ef60 comp rx=0 tx=0).stop 2024-09-11T23:29:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.200+0000 7f3472045640 1 -- 172.21.15.43:0/599886803 shutdown_connections 2024-09-11T23:29:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.200+0000 7f3472045640 1 --2- 172.21.15.43:0/599886803 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f34540a2d70 0x7f34540a5230 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.200+0000 7f3472045640 1 --2- 172.21.15.43:0/599886803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f346c10c180 0x7f346c19cde0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.200+0000 7f3472045640 1 --2- 172.21.15.43:0/599886803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f346c103700 0x7f346c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.200+0000 7f3472045640 1 --2- 172.21.15.43:0/599886803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f346c102de0 0x7f346c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:47.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.200+0000 7f3472045640 1 -- 172.21.15.43:0/599886803 >> 172.21.15.43:0/599886803 conn(0x7f346c0fc7a0 msgr2=0x7f346c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:47.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.200+0000 7f3472045640 1 -- 172.21.15.43:0/599886803 shutdown_connections 2024-09-11T23:29:47.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:45.200+0000 7f3472045640 1 -- 172.21.15.43:0/599886803 wait complete. 2024-09-11T23:29:47.615 DEBUG:teuthology.parallel:result is None 2024-09-11T23:29:47.616 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T23:29:47.638 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:47 smithi043 ceph-mon[30950]: pgmap v244: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.2 KiB/s wr, 63 op/s 2024-09-11T23:29:47.651 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T23:29:47.651 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-11T23:29:47.670 INFO:tasks.quiescer:joining Quiescers 2024-09-11T23:29:47.670 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-11T23:29:47.670 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:29:47.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:47 smithi090 ceph-mon[35130]: pgmap v244: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.2 KiB/s wr, 63 op/s 2024-09-11T23:29:47.800 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:47 smithi097 ceph-mon[35562]: pgmap v244: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.2 KiB/s wr, 63 op/s 2024-09-11T23:29:49.346 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '59d4cefc', total seconds elapsed: 49.7 2024-09-11T23:29:49.409 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:29:49.436 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:29:49.577 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:49 smithi043 ceph-mon[30950]: pgmap v245: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 58 op/s 2024-09-11T23:29:49.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:49 smithi090 ceph-mon[35130]: pgmap v245: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 58 op/s 2024-09-11T23:29:49.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:49 smithi097 ceph-mon[35562]: pgmap v245: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 58 op/s 2024-09-11T23:29:49.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.971+0000 7f745ca45640 1 -- 172.21.15.43:0/3681612199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7450006a90 msgr2=0x7f7450006e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:49.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.971+0000 7f745ca45640 1 --2- 172.21.15.43:0/3681612199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7450006a90 0x7f7450006e70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f744c009900 tx=0x7f744c02a480 comp rx=0 tx=0).stop 2024-09-11T23:29:49.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.971+0000 7f745ca45640 1 -- 172.21.15.43:0/3681612199 shutdown_connections 2024-09-11T23:29:49.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.971+0000 7f745ca45640 1 --2- 172.21.15.43:0/3681612199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74500a1e50 0x7f74500a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:49.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.971+0000 7f745ca45640 1 --2- 172.21.15.43:0/3681612199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74500073b0 0x7f74500a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:49.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.971+0000 7f745ca45640 1 --2- 172.21.15.43:0/3681612199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7450006a90 0x7f7450006e70 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:49.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.971+0000 7f745ca45640 1 -- 172.21.15.43:0/3681612199 >> 172.21.15.43:0/3681612199 conn(0x7f7450091250 msgr2=0x7f7450093670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:49.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.971+0000 7f745ca45640 1 -- 172.21.15.43:0/3681612199 shutdown_connections 2024-09-11T23:29:49.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.971+0000 7f745ca45640 1 -- 172.21.15.43:0/3681612199 wait complete. 2024-09-11T23:29:49.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.972+0000 7f745ca45640 1 Processor -- start 2024-09-11T23:29:49.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.972+0000 7f745ca45640 1 -- start start 2024-09-11T23:29:49.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.972+0000 7f745ca45640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7450006a90 0x7f745012e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:49.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.972+0000 7f745ca45640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74500073b0 0x7f745012e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:49.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.972+0000 7f745ca45640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74500a1e50 0x7f74501329b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:49.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.972+0000 7f745ca45640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f74500a7b50 con 0x7f74500a1e50 2024-09-11T23:29:49.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.972+0000 7f745ca45640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f74500a79d0 con 0x7f74500073b0 2024-09-11T23:29:49.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.972+0000 7f745ca45640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f74500a7cd0 con 0x7f7450006a90 2024-09-11T23:29:49.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.972+0000 7f7457fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74500a1e50 0x7f74501329b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:49.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.972+0000 7f7457fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74500a1e50 0x7f74501329b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42734/0 (socket says 172.21.15.43:42734) 2024-09-11T23:29:49.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.972+0000 7f7457fff640 1 -- 172.21.15.43:0/2905765381 learned_addr learned my addr 172.21.15.43:0/2905765381 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:49.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.972+0000 7f7456ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74500073b0 0x7f745012e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:49.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.972+0000 7f74577fe640 1 --2- 172.21.15.43:0/2905765381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7450006a90 0x7f745012e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:49.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.973+0000 7f7457fff640 1 -- 172.21.15.43:0/2905765381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7450006a90 msgr2=0x7f745012e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:49.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.973+0000 7f7457fff640 1 --2- 172.21.15.43:0/2905765381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7450006a90 0x7f745012e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:49.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.973+0000 7f7457fff640 1 -- 172.21.15.43:0/2905765381 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74500073b0 msgr2=0x7f745012e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:49.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.973+0000 7f7457fff640 1 --2- 172.21.15.43:0/2905765381 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74500073b0 0x7f745012e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:49.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.973+0000 7f7457fff640 1 -- 172.21.15.43:0/2905765381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7450133090 con 0x7f74500a1e50 2024-09-11T23:29:49.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.973+0000 7f7456ffd640 1 --2- 172.21.15.43:0/2905765381 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74500073b0 0x7f745012e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:29:49.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.973+0000 7f7457fff640 1 --2- 172.21.15.43:0/2905765381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74500a1e50 0x7f74501329b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f74580518b0 tx=0x7f74580747f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:49.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.973+0000 7f7454ff9640 1 -- 172.21.15.43:0/2905765381 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745807f430 con 0x7f74500a1e50 2024-09-11T23:29:49.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.973+0000 7f745ca45640 1 -- 172.21.15.43:0/2905765381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7450133380 con 0x7f74500a1e50 2024-09-11T23:29:49.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.973+0000 7f7454ff9640 1 -- 172.21.15.43:0/2905765381 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7458084020 con 0x7f74500a1e50 2024-09-11T23:29:49.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.973+0000 7f7454ff9640 1 -- 172.21.15.43:0/2905765381 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745807e960 con 0x7f74500a1e50 2024-09-11T23:29:49.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.974+0000 7f745ca45640 1 -- 172.21.15.43:0/2905765381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f745013ac60 con 0x7f74500a1e50 2024-09-11T23:29:49.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.976+0000 7f7454ff9640 1 -- 172.21.15.43:0/2905765381 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f745807eb00 con 0x7f74500a1e50 2024-09-11T23:29:49.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.977+0000 7f7460080740 1 Processor -- start 2024-09-11T23:29:49.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.977+0000 7f7454ff9640 1 --2- 172.21.15.43:0/2905765381 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f74480a2d00 0x7f74480a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:49.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.977+0000 7f7460080740 1 -- start start 2024-09-11T23:29:49.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.977+0000 7f7454ff9640 1 -- 172.21.15.43:0/2905765381 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f745813c9c0 con 0x7f74500a1e50 2024-09-11T23:29:49.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.977+0000 7f7456ffd640 1 --2- 172.21.15.43:0/2905765381 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f74480a2d00 0x7f74480a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:49.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.977+0000 7f7460080740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f71cf0 0x55a102f70240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:49.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.977+0000 7f7460080740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a102f70780 0x55a102f70bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:49.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.977+0000 7f7460080740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f83d30 0x55a102f86120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:49.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.977+0000 7f7460080740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a102ed62f0 con 0x55a102f71cf0 2024-09-11T23:29:49.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.977+0000 7f7460080740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a102e7dcf0 con 0x55a102f83d30 2024-09-11T23:29:49.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.977+0000 7f7460080740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a102ed22b0 con 0x55a102f70780 2024-09-11T23:29:49.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.977+0000 7f74577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f71cf0 0x55a102f70240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:49.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.977+0000 7f74577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f71cf0 0x55a102f70240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42738/0 (socket says 172.21.15.43:42738) 2024-09-11T23:29:49.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.977+0000 7f74577fe640 1 -- 172.21.15.43:0/666808931 learned_addr learned my addr 172.21.15.43:0/666808931 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:49.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.978+0000 7f7457fff640 1 --2- 172.21.15.43:0/666808931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f83d30 0x55a102f86120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:49.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.978+0000 7f74577fe640 1 -- 172.21.15.43:0/666808931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a102f70780 msgr2=0x55a102f70bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:49.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.978+0000 7f74577fe640 1 --2- 172.21.15.43:0/666808931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a102f70780 0x55a102f70bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:49.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.978+0000 7f74577fe640 1 -- 172.21.15.43:0/666808931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f83d30 msgr2=0x55a102f86120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:49.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.978+0000 7f74577fe640 1 --2- 172.21.15.43:0/666808931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f83d30 0x55a102f86120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:49.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.978+0000 7f7456ffd640 1 --2- 172.21.15.43:0/2905765381 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f74480a2d00 0x7f74480a51c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f74500a3190 tx=0x7f7444005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:49.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.978+0000 7f74577fe640 1 -- 172.21.15.43:0/666808931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a102e623a0 con 0x55a102f71cf0 2024-09-11T23:29:49.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.978+0000 7f74577fe640 1 --2- 172.21.15.43:0/666808931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f71cf0 0x55a102f70240 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f744c009900 tx=0x7f744c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aead8ec7ce3e5e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:49.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.978+0000 7f7441ffb640 1 -- 172.21.15.43:0/666808931 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c040070 con 0x55a102f71cf0 2024-09-11T23:29:49.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.978+0000 7f7441ffb640 1 -- 172.21.15.43:0/666808931 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f744c0318b0 con 0x55a102f71cf0 2024-09-11T23:29:49.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.978+0000 7f7441ffb640 1 -- 172.21.15.43:0/666808931 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c035040 con 0x55a102f71cf0 2024-09-11T23:29:49.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.979+0000 7f7460080740 1 -- 172.21.15.43:0/666808931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f71cf0 msgr2=0x55a102f70240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:49.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.979+0000 7f7460080740 1 --2- 172.21.15.43:0/666808931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f71cf0 0x55a102f70240 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f744c009900 tx=0x7f744c005dd0 comp rx=0 tx=0).stop 2024-09-11T23:29:49.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.979+0000 7f7460080740 1 -- 172.21.15.43:0/666808931 shutdown_connections 2024-09-11T23:29:49.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.979+0000 7f7460080740 1 --2- 172.21.15.43:0/666808931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f83d30 0x55a102f86120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:49.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.979+0000 7f7460080740 1 --2- 172.21.15.43:0/666808931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a102f70780 0x55a102f70bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:49.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.979+0000 7f7460080740 1 --2- 172.21.15.43:0/666808931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f71cf0 0x55a102f70240 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.979+0000 7f7460080740 1 -- 172.21.15.43:0/666808931 >> 172.21.15.43:0/666808931 conn(0x55a102f39310 msgr2=0x55a102f6ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.979+0000 7f7460080740 1 -- 172.21.15.43:0/666808931 shutdown_connections 2024-09-11T23:29:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.979+0000 7f7460080740 1 -- 172.21.15.43:0/666808931 wait complete. 2024-09-11T23:29:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.980+0000 7f7460080740 1 -- start start 2024-09-11T23:29:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.980+0000 7f7460080740 1 Processor -- start 2024-09-11T23:29:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.980+0000 7f7460080740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f70780 0x55a102f83680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.980+0000 7f7460080740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a102f71cf0 0x55a102f80930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.980+0000 7f7460080740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f83d30 0x55a102f80e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.980+0000 7f7460080740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a102f869d0 con 0x55a102f70780 2024-09-11T23:29:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.980+0000 7f7460080740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a102e91060 con 0x55a102f83d30 2024-09-11T23:29:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.980+0000 7f7460080740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a102f86b50 con 0x55a102f71cf0 2024-09-11T23:29:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.980+0000 7f7460080740 10 client.0 _ll_register_callbacks cb 0x55a102be48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:49.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.981+0000 7f7457fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f83d30 0x55a102f80e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.981+0000 7f7457fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f83d30 0x55a102f80e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:56780/0 (socket says 172.21.15.43:56780) 2024-09-11T23:29:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.981+0000 7f7457fff640 1 -- 172.21.15.43:0/1753851715 learned_addr learned my addr 172.21.15.43:0/1753851715 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.981+0000 7f74577fe640 1 --2- 172.21.15.43:0/1753851715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f70780 0x55a102f83680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.981+0000 7f7457fff640 1 -- 172.21.15.43:0/1753851715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a102f71cf0 msgr2=0x55a102f80930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.981+0000 7f7457fff640 1 --2- 172.21.15.43:0/1753851715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a102f71cf0 0x55a102f80930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.981+0000 7f7457fff640 1 -- 172.21.15.43:0/1753851715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f70780 msgr2=0x55a102f83680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.981+0000 7f7457fff640 1 --2- 172.21.15.43:0/1753851715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f70780 0x55a102f83680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.981+0000 7f7457fff640 1 -- 172.21.15.43:0/1753851715 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a102c473c0 con 0x55a102f83d30 2024-09-11T23:29:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.981+0000 7f7457fff640 1 --2- 172.21.15.43:0/1753851715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f83d30 0x55a102f80e90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7458065240 tx=0x7f7458106ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7befe73396aa3868 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:49.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.981+0000 7f7436ffd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:29:49.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.982+0000 7f7436ffd640 1 -- 172.21.15.43:0/1753851715 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458083040 con 0x55a102f83d30 2024-09-11T23:29:49.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.982+0000 7f7436ffd640 1 -- 172.21.15.43:0/1753851715 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f7458108300 con 0x55a102f83d30 2024-09-11T23:29:49.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.982+0000 7f7460080740 1 -- 172.21.15.43:0/1753851715 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f744c0318b0 con 0x55a102f83d30 2024-09-11T23:29:49.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.982+0000 7f7436ffd640 1 -- 172.21.15.43:0/1753851715 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f745810d260 con 0x55a102f83d30 2024-09-11T23:29:49.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.982+0000 7f7460080740 10 client.26165 fetch_fsmap learned FSMap version 31 2024-09-11T23:29:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.982+0000 7f7460080740 1 -- 172.21.15.43:0/1753851715 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a102f923e0 con 0x55a102f83d30 2024-09-11T23:29:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.983+0000 7f7436ffd640 1 -- 172.21.15.43:0/1753851715 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7458111b20 con 0x55a102f83d30 2024-09-11T23:29:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.983+0000 7f7436ffd640 1 -- 172.21.15.43:0/1753851715 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f7458107d70 con 0x55a102f83d30 2024-09-11T23:29:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.983+0000 7f7460080740 10 client.26165 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:29:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.983+0000 7f7460080740 10 client.26165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.983+0000 7f7460080740 20 client.26165 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.983+0000 7f7460080740 1 --2- 172.21.15.43:0/1753851715 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55a102f938a0 0x55a102f95d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.983+0000 7f7460080740 4 client.26165 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:29:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.983+0000 7f7460080740 1 -- 172.21.15.43:0/1753851715 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f744c0318b0 con 0x55a102f938a0 2024-09-11T23:29:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.984+0000 7f74577fe640 1 --2- 172.21.15.43:0/1753851715 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55a102f938a0 0x55a102f95d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.984+0000 7f74577fe640 1 --2- 172.21.15.43:0/1753851715 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55a102f938a0 0x55a102f95d60 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=254a067d28b880ae server_cookie=3b55472c4a131886 in_seq=0 out_seq=0 2024-09-11T23:29:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.984+0000 7f7436ffd640 10 client.26165 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.985+0000 7f7436ffd640 1 -- 172.21.15.43:0/1753851715 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f744c0318b0 con 0x55a102f938a0 2024-09-11T23:29:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.985+0000 7f7436ffd640 10 client.26165 handle_command_reply: tid=0 2024-09-11T23:29:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.985+0000 7f7460080740 1 client.26165 shutdown 2024-09-11T23:29:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.986+0000 7f7460080740 1 -- 172.21.15.43:0/1753851715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f83d30 msgr2=0x55a102f80e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.986+0000 7f7460080740 1 --2- 172.21.15.43:0/1753851715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f83d30 0x55a102f80e90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7458065240 tx=0x7f7458106ff0 comp rx=0 tx=0).stop 2024-09-11T23:29:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.986+0000 7f7460080740 1 -- 172.21.15.43:0/1753851715 shutdown_connections 2024-09-11T23:29:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.986+0000 7f7460080740 1 --2- 172.21.15.43:0/1753851715 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55a102f938a0 0x55a102f95d60 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.986+0000 7f7436ffd640 0 client.26165 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.986+0000 7f7460080740 1 --2- 172.21.15.43:0/1753851715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f83d30 0x55a102f80e90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.986+0000 7f7460080740 1 --2- 172.21.15.43:0/1753851715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a102f71cf0 0x55a102f80930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.986+0000 7f7460080740 1 --2- 172.21.15.43:0/1753851715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f70780 0x55a102f83680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.986+0000 7f7460080740 1 -- 172.21.15.43:0/1753851715 >> 172.21.15.43:0/1753851715 conn(0x55a102f39310 msgr2=0x55a102f72b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:49.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.992+0000 7f7460080740 1 -- 172.21.15.43:0/1753851715 shutdown_connections 2024-09-11T23:29:49.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.992+0000 7f7460080740 1 -- 172.21.15.43:0/1753851715 wait complete. 2024-09-11T23:29:49.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:49.992+0000 7f7460080740 20 client.26165 trim_cache size 0 max 16384 2024-09-11T23:29:50.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.016+0000 7f7460080740 1 Processor -- start 2024-09-11T23:29:50.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.016+0000 7f7460080740 1 -- start start 2024-09-11T23:29:50.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.017+0000 7f7460080740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f70780 0x55a102f8fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:50.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.017+0000 7f7460080740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a102f71cf0 0x55a102f801f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:50.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.017+0000 7f7460080740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f83d30 0x55a102f9b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:50.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.017+0000 7f7460080740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7450141b30 con 0x55a102f83d30 2024-09-11T23:29:50.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.017+0000 7f7460080740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f745013f3d0 con 0x55a102f70780 2024-09-11T23:29:50.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.017+0000 7f7460080740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f745013d260 con 0x55a102f71cf0 2024-09-11T23:29:50.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.017+0000 7f7457fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f83d30 0x55a102f9b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:50.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.017+0000 7f74577fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f70780 0x55a102f8fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:50.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.017+0000 7f74577fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f70780 0x55a102f8fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:56794/0 (socket says 172.21.15.43:56794) 2024-09-11T23:29:50.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.017+0000 7f7457fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f83d30 0x55a102f9b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42768/0 (socket says 172.21.15.43:42768) 2024-09-11T23:29:50.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.017+0000 7f74577fe640 1 -- 172.21.15.43:0/2073633143 learned_addr learned my addr 172.21.15.43:0/2073633143 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:50.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.018+0000 7f74577fe640 1 -- 172.21.15.43:0/2073633143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a102f71cf0 msgr2=0x55a102f801f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:29:50.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.018+0000 7f74577fe640 1 --2- 172.21.15.43:0/2073633143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a102f71cf0 0x55a102f801f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.018+0000 7f74577fe640 1 -- 172.21.15.43:0/2073633143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f83d30 msgr2=0x55a102f9b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:50.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.018+0000 7f74577fe640 1 --2- 172.21.15.43:0/2073633143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f83d30 0x55a102f9b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.018+0000 7f74577fe640 1 -- 172.21.15.43:0/2073633143 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a102f732d0 con 0x55a102f70780 2024-09-11T23:29:50.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.018+0000 7f7457fff640 1 --2- 172.21.15.43:0/2073633143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f83d30 0x55a102f9b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:50.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.018+0000 7f74577fe640 1 --2- 172.21.15.43:0/2073633143 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f70780 0x55a102f8fc70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f744c0057c0 tx=0x7f744c047040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a27a8f0b2ea2ae08 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:50.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.018+0000 7f74417fa640 1 -- 172.21.15.43:0/2073633143 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c040040 con 0x55a102f70780 2024-09-11T23:29:50.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.019+0000 7f74417fa640 1 -- 172.21.15.43:0/2073633143 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f744c050910 con 0x55a102f70780 2024-09-11T23:29:50.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.019+0000 7f74417fa640 1 -- 172.21.15.43:0/2073633143 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c035040 con 0x55a102f70780 2024-09-11T23:29:50.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.020+0000 7f7460080740 1 -- 172.21.15.43:0/2073633143 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f70780 msgr2=0x55a102f8fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:50.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.020+0000 7f7460080740 1 --2- 172.21.15.43:0/2073633143 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f70780 0x55a102f8fc70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f744c0057c0 tx=0x7f744c047040 comp rx=0 tx=0).stop 2024-09-11T23:29:50.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.020+0000 7f7460080740 1 -- 172.21.15.43:0/2073633143 shutdown_connections 2024-09-11T23:29:50.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.020+0000 7f7460080740 1 --2- 172.21.15.43:0/2073633143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f83d30 0x55a102f9b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.020+0000 7f7460080740 1 --2- 172.21.15.43:0/2073633143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a102f71cf0 0x55a102f801f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.021+0000 7f7460080740 1 --2- 172.21.15.43:0/2073633143 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f70780 0x55a102f8fc70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.021+0000 7f7460080740 1 -- 172.21.15.43:0/2073633143 >> 172.21.15.43:0/2073633143 conn(0x55a102f39310 msgr2=0x55a102f7f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:50.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.021+0000 7f7460080740 1 -- 172.21.15.43:0/2073633143 shutdown_connections 2024-09-11T23:29:50.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.021+0000 7f7460080740 1 -- 172.21.15.43:0/2073633143 wait complete. 2024-09-11T23:29:50.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.022+0000 7f7460080740 1 -- start start 2024-09-11T23:29:50.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.022+0000 7f7460080740 1 Processor -- start 2024-09-11T23:29:50.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.022+0000 7f7460080740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f70780 0x55a102f87da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:50.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.022+0000 7f7460080740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a102f71cf0 0x55a102f882e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:50.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.022+0000 7f7460080740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f83d30 0x55a102f88820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.023+0000 7f7460080740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f745013d8b0 con 0x55a102f70780 2024-09-11T23:29:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.023+0000 7f7460080740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f745013ed00 con 0x55a102f83d30 2024-09-11T23:29:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.023+0000 7f7460080740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f745013df60 con 0x55a102f71cf0 2024-09-11T23:29:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.023+0000 7f7460080740 10 client.0 _ll_register_callbacks cb 0x7f74581050c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.023+0000 7f74577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f70780 0x55a102f87da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.023+0000 7f74577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f70780 0x55a102f87da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42774/0 (socket says 172.21.15.43:42774) 2024-09-11T23:29:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.023+0000 7f74577fe640 1 -- 172.21.15.43:0/3381773906 learned_addr learned my addr 172.21.15.43:0/3381773906 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.023+0000 7f7456ffd640 1 --2- 172.21.15.43:0/3381773906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a102f71cf0 0x55a102f882e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:50.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.025+0000 7f7457fff640 1 --2- 172.21.15.43:0/3381773906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f83d30 0x55a102f88820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:50.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.025+0000 7f74577fe640 1 -- 172.21.15.43:0/3381773906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a102f71cf0 msgr2=0x55a102f882e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:50.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.025+0000 7f74577fe640 1 --2- 172.21.15.43:0/3381773906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a102f71cf0 0x55a102f882e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.025+0000 7f74577fe640 1 -- 172.21.15.43:0/3381773906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f83d30 msgr2=0x55a102f88820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:50.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.025+0000 7f74577fe640 1 --2- 172.21.15.43:0/3381773906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f83d30 0x55a102f88820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.025+0000 7f74577fe640 1 -- 172.21.15.43:0/3381773906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a10300e860 con 0x55a102f70780 2024-09-11T23:29:50.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.026+0000 7f74577fe640 1 --2- 172.21.15.43:0/3381773906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f70780 0x55a102f87da0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f744c02a4c0 tx=0x7f744c03c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f800c75c1657514 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:50.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.026+0000 7f7436ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:29:50.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.026+0000 7f7436ffd640 1 -- 172.21.15.43:0/3381773906 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c05f070 con 0x55a102f70780 2024-09-11T23:29:50.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.026+0000 7f7436ffd640 1 -- 172.21.15.43:0/3381773906 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f744c050430 con 0x55a102f70780 2024-09-11T23:29:50.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.026+0000 7f7460080740 1 -- 172.21.15.43:0/3381773906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f744c050910 con 0x55a102f70780 2024-09-11T23:29:50.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.028+0000 7f7436ffd640 1 -- 172.21.15.43:0/3381773906 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f744c069050 con 0x55a102f70780 2024-09-11T23:29:50.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.028+0000 7f7460080740 10 client.16689 fetch_fsmap learned FSMap version 31 2024-09-11T23:29:50.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.028+0000 7f7460080740 1 -- 172.21.15.43:0/3381773906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a10300fb60 con 0x55a102f70780 2024-09-11T23:29:50.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.028+0000 7f7436ffd640 1 -- 172.21.15.43:0/3381773906 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f744c050910 con 0x55a102f70780 2024-09-11T23:29:50.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.029+0000 7f7436ffd640 1 -- 172.21.15.43:0/3381773906 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 31) ==== 8350+0+0 (secure 0 0 0) 0x7f744c0595b0 con 0x55a102f70780 2024-09-11T23:29:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.030+0000 7f7460080740 10 client.16689 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T23:29:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.030+0000 7f7460080740 10 client.16689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.030+0000 7f7460080740 20 client.16689 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.030+0000 7f7460080740 1 --2- 172.21.15.43:0/3381773906 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55a10300fd90 0x55a103012180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.030+0000 7f7460080740 4 client.16689 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:29:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.030+0000 7f7460080740 1 -- 172.21.15.43:0/3381773906 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a102ee3860 con 0x55a10300fd90 2024-09-11T23:29:50.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.031+0000 7f7456ffd640 1 --2- 172.21.15.43:0/3381773906 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55a10300fd90 0x55a103012180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:50.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.032+0000 7f7456ffd640 1 --2- 172.21.15.43:0/3381773906 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55a10300fd90 0x55a103012180 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5571a7b05bab28c3 server_cookie=757780fb9b8dbbef in_seq=0 out_seq=0 2024-09-11T23:29:50.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.032+0000 7f7436ffd640 10 client.16689 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:50.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.033+0000 7f7436ffd640 1 -- 172.21.15.43:0/3381773906 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a102ee3860 con 0x55a10300fd90 2024-09-11T23:29:50.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.033+0000 7f7436ffd640 10 client.16689 handle_command_reply: tid=0 2024-09-11T23:29:50.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.033+0000 7f7460080740 1 client.16689 shutdown 2024-09-11T23:29:50.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.034+0000 7f7460080740 1 -- 172.21.15.43:0/3381773906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f70780 msgr2=0x55a102f87da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:50.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.034+0000 7f7460080740 1 --2- 172.21.15.43:0/3381773906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f70780 0x55a102f87da0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f744c02a4c0 tx=0x7f744c03c570 comp rx=0 tx=0).stop 2024-09-11T23:29:50.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.034+0000 7f7460080740 1 -- 172.21.15.43:0/3381773906 shutdown_connections 2024-09-11T23:29:50.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.034+0000 7f7460080740 1 --2- 172.21.15.43:0/3381773906 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55a10300fd90 0x55a103012180 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.034+0000 7f7460080740 1 --2- 172.21.15.43:0/3381773906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102f83d30 0x55a102f88820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.034+0000 7f7436ffd640 0 client.16689 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:50.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.034+0000 7f7460080740 1 --2- 172.21.15.43:0/3381773906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a102f71cf0 0x55a102f882e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.034+0000 7f7460080740 1 --2- 172.21.15.43:0/3381773906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a102f70780 0x55a102f87da0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.034+0000 7f7460080740 1 -- 172.21.15.43:0/3381773906 >> 172.21.15.43:0/3381773906 conn(0x55a102f39310 msgr2=0x55a102f829b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:50.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.034+0000 7f7460080740 1 -- 172.21.15.43:0/3381773906 shutdown_connections 2024-09-11T23:29:50.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.034+0000 7f7460080740 1 -- 172.21.15.43:0/3381773906 wait complete. 2024-09-11T23:29:50.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.035+0000 7f7460080740 20 client.16689 trim_cache size 0 max 16384 2024-09-11T23:29:50.035 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:29:50.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.036+0000 7f745ca45640 1 -- 172.21.15.43:0/2905765381 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f74480a2d00 msgr2=0x7f74480a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:50.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.036+0000 7f745ca45640 1 --2- 172.21.15.43:0/2905765381 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f74480a2d00 0x7f74480a51c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f74500a3190 tx=0x7f7444005f90 comp rx=0 tx=0).stop 2024-09-11T23:29:50.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.036+0000 7f745ca45640 1 -- 172.21.15.43:0/2905765381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74500a1e50 msgr2=0x7f74501329b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:50.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.036+0000 7f745ca45640 1 --2- 172.21.15.43:0/2905765381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74500a1e50 0x7f74501329b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f74580518b0 tx=0x7f74580747f0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.037+0000 7f745ca45640 1 -- 172.21.15.43:0/2905765381 shutdown_connections 2024-09-11T23:29:50.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.037+0000 7f745ca45640 1 --2- 172.21.15.43:0/2905765381 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f74480a2d00 0x7f74480a51c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.037+0000 7f745ca45640 1 --2- 172.21.15.43:0/2905765381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74500a1e50 0x7f74501329b0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.037+0000 7f745ca45640 1 --2- 172.21.15.43:0/2905765381 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74500073b0 0x7f745012e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.037+0000 7f745ca45640 1 --2- 172.21.15.43:0/2905765381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7450006a90 0x7f745012e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.037+0000 7f745ca45640 1 -- 172.21.15.43:0/2905765381 >> 172.21.15.43:0/2905765381 conn(0x7f7450091250 msgr2=0x7f7450091f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:50.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.037+0000 7f745ca45640 1 -- 172.21.15.43:0/2905765381 shutdown_connections 2024-09-11T23:29:50.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.037+0000 7f745ca45640 1 -- 172.21.15.43:0/2905765381 wait complete. 2024-09-11T23:29:50.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.517+0000 7fc578f89640 1 -- 172.21.15.43:0/610345432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc57410c180 msgr2=0x7fc57410e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:50.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.517+0000 7fc578f89640 1 --2- 172.21.15.43:0/610345432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc57410c180 0x7fc57410e600 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009ca0 tx=0x7fc56c02a950 comp rx=0 tx=0).stop 2024-09-11T23:29:50.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.520+0000 7fc578f89640 1 -- 172.21.15.43:0/610345432 shutdown_connections 2024-09-11T23:29:50.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.520+0000 7fc578f89640 1 --2- 172.21.15.43:0/610345432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc57410c180 0x7fc57410e600 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.520+0000 7fc578f89640 1 --2- 172.21.15.43:0/610345432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5741036c0 0x7fc57410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.520+0000 7fc578f89640 1 --2- 172.21.15.43:0/610345432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc574102da0 0x7fc574103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.520+0000 7fc578f89640 1 -- 172.21.15.43:0/610345432 >> 172.21.15.43:0/610345432 conn(0x7fc5740fc7a0 msgr2=0x7fc5740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:50.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.526+0000 7fc578f89640 1 -- 172.21.15.43:0/610345432 shutdown_connections 2024-09-11T23:29:50.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.527+0000 7fc578f89640 1 -- 172.21.15.43:0/610345432 wait complete. 2024-09-11T23:29:50.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.528+0000 7fc578f89640 1 Processor -- start 2024-09-11T23:29:50.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.529+0000 7fc578f89640 1 -- start start 2024-09-11T23:29:50.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.529+0000 7fc578f89640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc574102da0 0x7fc5741a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:50.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.530+0000 7fc578f89640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5741036c0 0x7fc5741a3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:50.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.530+0000 7fc578f89640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc57410c180 0x7fc57410ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:50.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.530+0000 7fc578f89640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc574111fb0 con 0x7fc574102da0 2024-09-11T23:29:50.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.530+0000 7fc578f89640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc574111e30 con 0x7fc57410c180 2024-09-11T23:29:50.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.530+0000 7fc578f89640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc574112130 con 0x7fc5741036c0 2024-09-11T23:29:50.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.530+0000 7fc569d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5741036c0 0x7fc5741a3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:50.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.530+0000 7fc572d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc57410c180 0x7fc57410ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:50.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.530+0000 7fc569d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5741036c0 0x7fc5741a3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:43478/0 (socket says 172.21.15.43:43478) 2024-09-11T23:29:50.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.530+0000 7fc569d9b640 1 -- 172.21.15.43:0/564777068 learned_addr learned my addr 172.21.15.43:0/564777068 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:50.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.530+0000 7fc57259c640 1 --2- 172.21.15.43:0/564777068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc574102da0 0x7fc5741a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:50.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.531+0000 7fc57259c640 1 -- 172.21.15.43:0/564777068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5741036c0 msgr2=0x7fc5741a3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:50.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.531+0000 7fc57259c640 1 --2- 172.21.15.43:0/564777068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5741036c0 0x7fc5741a3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.531+0000 7fc57259c640 1 -- 172.21.15.43:0/564777068 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc57410c180 msgr2=0x7fc57410ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:50.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.531+0000 7fc57259c640 1 --2- 172.21.15.43:0/564777068 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc57410c180 0x7fc57410ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.531+0000 7fc57259c640 1 -- 172.21.15.43:0/564777068 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57410b580 con 0x7fc574102da0 2024-09-11T23:29:50.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.531+0000 7fc569d9b640 1 --2- 172.21.15.43:0/564777068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5741036c0 0x7fc5741a3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:29:50.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.531+0000 7fc572d9d640 1 --2- 172.21.15.43:0/564777068 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc57410c180 0x7fc57410ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:29:50.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.531+0000 7fc57259c640 1 --2- 172.21.15.43:0/564777068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc574102da0 0x7fc5741a36f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc56400e9f0 tx=0x7fc56400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:50.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.532+0000 7fc56bfff640 1 -- 172.21.15.43:0/564777068 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56400ccc0 con 0x7fc574102da0 2024-09-11T23:29:50.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.532+0000 7fc56bfff640 1 -- 172.21.15.43:0/564777068 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc564005020 con 0x7fc574102da0 2024-09-11T23:29:50.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.532+0000 7fc578f89640 1 -- 172.21.15.43:0/564777068 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc57410b870 con 0x7fc574102da0 2024-09-11T23:29:50.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.532+0000 7fc56bfff640 1 -- 172.21.15.43:0/564777068 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5640049e0 con 0x7fc574102da0 2024-09-11T23:29:50.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.532+0000 7fc578f89640 1 -- 172.21.15.43:0/564777068 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5741ab780 con 0x7fc574102da0 2024-09-11T23:29:50.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.534+0000 7fc578f89640 1 -- 172.21.15.43:0/564777068 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc57406b010 con 0x7fc574102da0 2024-09-11T23:29:50.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.545+0000 7fc56bfff640 1 -- 172.21.15.43:0/564777068 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc564005560 con 0x7fc574102da0 2024-09-11T23:29:50.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.546+0000 7fc56bfff640 1 --2- 172.21.15.43:0/564777068 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc5600a2ec0 0x7fc5600a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:50.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.546+0000 7fc56bfff640 1 -- 172.21.15.43:0/564777068 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7fc5640cf370 con 0x7fc574102da0 2024-09-11T23:29:50.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.547+0000 7fc569d9b640 1 --2- 172.21.15.43:0/564777068 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc5600a2ec0 0x7fc5600a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:50.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.549+0000 7fc569d9b640 1 --2- 172.21.15.43:0/564777068 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc5600a2ec0 0x7fc5600a5380 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc5741a0000 tx=0x7fc55c005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:50.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.549+0000 7fc56bfff640 1 -- 172.21.15.43:0/564777068 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5640051d0 con 0x7fc574102da0 2024-09-11T23:29:50.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.834+0000 7fc578f89640 1 -- 172.21.15.43:0/564777068 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5741ab950 con 0x7fc574102da0 2024-09-11T23:29:50.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.834+0000 7fc56bfff640 1 -- 172.21.15.43:0/564777068 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11894 (secure 0 0 0) 0x7fc56409bf00 con 0x7fc574102da0 2024-09-11T23:29:50.835 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:29:50.835 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":31,"btime":"2024-09-11T23:29:03:134233+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:29:02.865047+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25571":{"gid":25571,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/568491708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":568491708},{"type":"v1","addr":"172.21.15.90:6835","nonce":568491708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24457,24470,24497]},"id":1}]} 2024-09-11T23:29:50.835 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 31 2024-09-11T23:29:50.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.836+0000 7fc578f89640 1 -- 172.21.15.43:0/564777068 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc5600a2ec0 msgr2=0x7fc5600a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:50.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.836+0000 7fc578f89640 1 --2- 172.21.15.43:0/564777068 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc5600a2ec0 0x7fc5600a5380 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc5741a0000 tx=0x7fc55c005f90 comp rx=0 tx=0).stop 2024-09-11T23:29:50.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.836+0000 7fc578f89640 1 -- 172.21.15.43:0/564777068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc574102da0 msgr2=0x7fc5741a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:50.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.836+0000 7fc578f89640 1 --2- 172.21.15.43:0/564777068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc574102da0 0x7fc5741a36f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc56400e9f0 tx=0x7fc56400eeb0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.836+0000 7fc578f89640 1 -- 172.21.15.43:0/564777068 shutdown_connections 2024-09-11T23:29:50.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.837+0000 7fc578f89640 1 --2- 172.21.15.43:0/564777068 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc5600a2ec0 0x7fc5600a5380 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.837+0000 7fc578f89640 1 --2- 172.21.15.43:0/564777068 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc57410c180 0x7fc57410ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.837+0000 7fc578f89640 1 --2- 172.21.15.43:0/564777068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5741036c0 0x7fc5741a3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.837+0000 7fc578f89640 1 --2- 172.21.15.43:0/564777068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc574102da0 0x7fc5741a36f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:50.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.837+0000 7fc578f89640 1 -- 172.21.15.43:0/564777068 >> 172.21.15.43:0/564777068 conn(0x7fc5740fc7a0 msgr2=0x7fc574100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:50.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.837+0000 7fc578f89640 1 -- 172.21.15.43:0/564777068 shutdown_connections 2024-09-11T23:29:50.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:50.837+0000 7fc578f89640 1 -- 172.21.15.43:0/564777068 wait complete. 2024-09-11T23:29:51.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:51 smithi090 ceph-mon[35130]: pgmap v246: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 42 KiB/s rd, 44 KiB/s wr, 66 op/s 2024-09-11T23:29:51.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:51 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/564777068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:29:51.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:51 smithi097 ceph-mon[35562]: pgmap v246: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 42 KiB/s rd, 44 KiB/s wr, 66 op/s 2024-09-11T23:29:51.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:51 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/564777068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:29:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.817+0000 7f7b42d28640 1 -- 172.21.15.43:0/349555221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b3c10c0d0 msgr2=0x7f7b3c10e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.817+0000 7f7b42d28640 1 --2- 172.21.15.43:0/349555221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b3c10c0d0 0x7f7b3c10e540 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009ca0 tx=0x7f7b3402a950 comp rx=0 tx=0).stop 2024-09-11T23:29:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.817+0000 7f7b42d28640 1 -- 172.21.15.43:0/349555221 shutdown_connections 2024-09-11T23:29:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.817+0000 7f7b42d28640 1 --2- 172.21.15.43:0/349555221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b3c10c0d0 0x7f7b3c10e540 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.817+0000 7f7b42d28640 1 --2- 172.21.15.43:0/349555221 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c103620 0x7f7b3c10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.817+0000 7f7b42d28640 1 --2- 172.21.15.43:0/349555221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b3c102d00 0x7f7b3c1030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.817+0000 7f7b42d28640 1 -- 172.21.15.43:0/349555221 >> 172.21.15.43:0/349555221 conn(0x7f7b3c0779a0 msgr2=0x7f7b3c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.817+0000 7f7b42d28640 1 -- 172.21.15.43:0/349555221 shutdown_connections 2024-09-11T23:29:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.817+0000 7f7b42d28640 1 -- 172.21.15.43:0/349555221 wait complete. 2024-09-11T23:29:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.817+0000 7f7b42d28640 1 Processor -- start 2024-09-11T23:29:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.817+0000 7f7b42d28640 1 -- start start 2024-09-11T23:29:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.818+0000 7f7b42d28640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c102d00 0x7f7b3c100950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.818+0000 7f7b42d28640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b3c103620 0x7f7b3c100e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.818+0000 7f7b42d28640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b3c10c0d0 0x7f7b3c1013d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.818+0000 7f7b42d28640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c111ea0 con 0x7f7b3c10c0d0 2024-09-11T23:29:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.818+0000 7f7b42d28640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c111d20 con 0x7f7b3c102d00 2024-09-11T23:29:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.818+0000 7f7b42d28640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c112020 con 0x7f7b3c103620 2024-09-11T23:29:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.818+0000 7f7b40ac4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c102d00 0x7f7b3c100950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.818+0000 7f7b40ac4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c102d00 0x7f7b3c100950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:42292/0 (socket says 172.21.15.43:42292) 2024-09-11T23:29:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.818+0000 7f7b40ac4640 1 -- 172.21.15.43:0/2053915549 learned_addr learned my addr 172.21.15.43:0/2053915549 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.818+0000 7f7b3bfff640 1 --2- 172.21.15.43:0/2053915549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b3c103620 0x7f7b3c100e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.818+0000 7f7b412c5640 1 --2- 172.21.15.43:0/2053915549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b3c10c0d0 0x7f7b3c1013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.818+0000 7f7b40ac4640 1 -- 172.21.15.43:0/2053915549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b3c103620 msgr2=0x7f7b3c100e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.818+0000 7f7b40ac4640 1 --2- 172.21.15.43:0/2053915549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b3c103620 0x7f7b3c100e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.818+0000 7f7b40ac4640 1 -- 172.21.15.43:0/2053915549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b3c10c0d0 msgr2=0x7f7b3c1013d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.818+0000 7f7b40ac4640 1 --2- 172.21.15.43:0/2053915549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b3c10c0d0 0x7f7b3c1013d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.818+0000 7f7b40ac4640 1 -- 172.21.15.43:0/2053915549 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b3c1a6840 con 0x7f7b3c102d00 2024-09-11T23:29:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.818+0000 7f7b40ac4640 1 --2- 172.21.15.43:0/2053915549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c102d00 0x7f7b3c100950 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000e9f0 tx=0x7f7b3000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.818+0000 7f7b39ffb640 1 -- 172.21.15.43:0/2053915549 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3000cd80 con 0x7f7b3c102d00 2024-09-11T23:29:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.818+0000 7f7b42d28640 1 -- 172.21.15.43:0/2053915549 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b3c1a6b30 con 0x7f7b3c102d00 2024-09-11T23:29:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.818+0000 7f7b39ffb640 1 -- 172.21.15.43:0/2053915549 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b30005500 con 0x7f7b3c102d00 2024-09-11T23:29:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.818+0000 7f7b39ffb640 1 -- 172.21.15.43:0/2053915549 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30010690 con 0x7f7b3c102d00 2024-09-11T23:29:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.819+0000 7f7b42d28640 1 -- 172.21.15.43:0/2053915549 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b3c1a7030 con 0x7f7b3c102d00 2024-09-11T23:29:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.821+0000 7f7b39ffb640 1 -- 172.21.15.43:0/2053915549 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7b300046c0 con 0x7f7b3c102d00 2024-09-11T23:29:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.821+0000 7f7b45ab6740 1 Processor -- start 2024-09-11T23:29:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.821+0000 7f7b45ab6740 1 -- start start 2024-09-11T23:29:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.821+0000 7f7b45ab6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564348721af0 0x564348721ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.821+0000 7f7b45ab6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564348722410 0x56434872c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.821+0000 7f7b45ab6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56434872ccb0 0x56434872f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.822+0000 7f7b45ab6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5643486862f0 con 0x564348722410 2024-09-11T23:29:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.822+0000 7f7b45ab6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56434862e270 con 0x56434872ccb0 2024-09-11T23:29:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.822+0000 7f7b40ac4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564348722410 0x56434872c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.822+0000 7f7b45ab6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5643486822b0 con 0x564348721af0 2024-09-11T23:29:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.822+0000 7f7b40ac4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564348722410 0x56434872c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57132/0 (socket says 172.21.15.43:57132) 2024-09-11T23:29:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.822+0000 7f7b40ac4640 1 -- 172.21.15.43:0/2827984575 learned_addr learned my addr 172.21.15.43:0/2827984575 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.822+0000 7f7b39ffb640 1 --2- 172.21.15.43:0/2053915549 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7b2c0a2cf0 0x7f7b2c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.822+0000 7f7b3bfff640 1 --2- 172.21.15.43:0/2827984575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56434872ccb0 0x56434872f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.822+0000 7f7b412c5640 1 --2- 172.21.15.43:0/2827984575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564348721af0 0x564348721ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.822+0000 7f7b412c5640 1 --2- 172.21.15.43:0/2053915549 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7b2c0a2cf0 0x7f7b2c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.822+0000 7f7b3bfff640 1 -- 172.21.15.43:0/2827984575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564348721af0 msgr2=0x564348721ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.822+0000 7f7b3bfff640 1 --2- 172.21.15.43:0/2827984575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564348721af0 0x564348721ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.822+0000 7f7b3bfff640 1 -- 172.21.15.43:0/2827984575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564348722410 msgr2=0x56434872c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.822+0000 7f7b3bfff640 1 --2- 172.21.15.43:0/2827984575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564348722410 0x56434872c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.822+0000 7f7b3bfff640 1 -- 172.21.15.43:0/2827984575 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56434862d340 con 0x56434872ccb0 2024-09-11T23:29:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.822+0000 7f7b39ffb640 1 -- 172.21.15.43:0/2053915549 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f7b300cf900 con 0x7f7b3c102d00 2024-09-11T23:29:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.822+0000 7f7b3bfff640 1 --2- 172.21.15.43:0/2827984575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56434872ccb0 0x56434872f0a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009bf0 tx=0x7f7b28011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=989ac8a3d16c537c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.822+0000 7f7b26ffd640 1 -- 172.21.15.43:0/2827984575 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28004050 con 0x56434872ccb0 2024-09-11T23:29:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.822+0000 7f7b412c5640 1 --2- 172.21.15.43:0/2053915549 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7b2c0a2cf0 0x7f7b2c0a51b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009430 tx=0x7f7b34005c50 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.822+0000 7f7b26ffd640 1 -- 172.21.15.43:0/2827984575 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b28005500 con 0x56434872ccb0 2024-09-11T23:29:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.822+0000 7f7b26ffd640 1 -- 172.21.15.43:0/2827984575 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28004dd0 con 0x56434872ccb0 2024-09-11T23:29:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.823+0000 7f7b45ab6740 1 -- 172.21.15.43:0/2827984575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56434872ccb0 msgr2=0x56434872f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.823+0000 7f7b45ab6740 1 --2- 172.21.15.43:0/2827984575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56434872ccb0 0x56434872f0a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009bf0 tx=0x7f7b28011810 comp rx=0 tx=0).stop 2024-09-11T23:29:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.823+0000 7f7b45ab6740 1 -- 172.21.15.43:0/2827984575 shutdown_connections 2024-09-11T23:29:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.823+0000 7f7b45ab6740 1 --2- 172.21.15.43:0/2827984575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56434872ccb0 0x56434872f0a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.823+0000 7f7b45ab6740 1 --2- 172.21.15.43:0/2827984575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564348722410 0x56434872c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.823+0000 7f7b45ab6740 1 --2- 172.21.15.43:0/2827984575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564348721af0 0x564348721ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.823+0000 7f7b45ab6740 1 -- 172.21.15.43:0/2827984575 >> 172.21.15.43:0/2827984575 conn(0x56434871e060 msgr2=0x56434871e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.823+0000 7f7b45ab6740 1 -- 172.21.15.43:0/2827984575 shutdown_connections 2024-09-11T23:29:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.823+0000 7f7b45ab6740 1 -- 172.21.15.43:0/2827984575 wait complete. 2024-09-11T23:29:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.823+0000 7f7b45ab6740 1 -- start start 2024-09-11T23:29:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.823+0000 7f7b45ab6740 1 Processor -- start 2024-09-11T23:29:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.824+0000 7f7b45ab6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564348721af0 0x564348729ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.824+0000 7f7b45ab6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564348722410 0x56434872a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.824+0000 7f7b45ab6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56434872ccb0 0x56434872a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.824+0000 7f7b45ab6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56434872f880 con 0x564348721af0 2024-09-11T23:29:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.824+0000 7f7b45ab6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56434860f290 con 0x56434872ccb0 2024-09-11T23:29:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.824+0000 7f7b45ab6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56434872fa00 con 0x564348722410 2024-09-11T23:29:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.824+0000 7f7b45ab6740 10 client.0 _ll_register_callbacks cb 0x564348640140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.824+0000 7f7b40ac4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564348721af0 0x564348729ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.824+0000 7f7b40ac4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564348721af0 0x564348729ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57146/0 (socket says 172.21.15.43:57146) 2024-09-11T23:29:51.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.824+0000 7f7b40ac4640 1 -- 172.21.15.43:0/3165113614 learned_addr learned my addr 172.21.15.43:0/3165113614 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:51.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.824+0000 7f7b412c5640 1 --2- 172.21.15.43:0/3165113614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56434872ccb0 0x56434872a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:51.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.824+0000 7f7b3bfff640 1 --2- 172.21.15.43:0/3165113614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564348722410 0x56434872a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:51.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.824+0000 7f7b40ac4640 1 -- 172.21.15.43:0/3165113614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564348722410 msgr2=0x56434872a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:51.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.824+0000 7f7b40ac4640 1 --2- 172.21.15.43:0/3165113614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564348722410 0x56434872a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.824+0000 7f7b40ac4640 1 -- 172.21.15.43:0/3165113614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56434872ccb0 msgr2=0x56434872a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:51.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.824+0000 7f7b40ac4640 1 --2- 172.21.15.43:0/3165113614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56434872ccb0 0x56434872a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.824+0000 7f7b40ac4640 1 -- 172.21.15.43:0/3165113614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564348399210 con 0x564348721af0 2024-09-11T23:29:51.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.824+0000 7f7b412c5640 1 --2- 172.21.15.43:0/3165113614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56434872ccb0 0x56434872a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:29:51.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.825+0000 7f7b40ac4640 1 --2- 172.21.15.43:0/3165113614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564348721af0 0x564348729ad0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000c6a0 tx=0x7f7b300ccfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b1667940b835fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:51.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.825+0000 7f7b1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:29:51.832 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.825+0000 7f7b1bfff640 1 -- 172.21.15.43:0/3165113614 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3009b6f0 con 0x564348721af0 2024-09-11T23:29:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.825+0000 7f7b1bfff640 1 -- 172.21.15.43:0/3165113614 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f7b3009b1e0 con 0x564348721af0 2024-09-11T23:29:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.825+0000 7f7b45ab6740 1 -- 172.21.15.43:0/3165113614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b28005500 con 0x564348721af0 2024-09-11T23:29:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.825+0000 7f7b1bfff640 1 -- 172.21.15.43:0/3165113614 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7b300b9050 con 0x564348721af0 2024-09-11T23:29:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.825+0000 7f7b45ab6740 10 client.16707 fetch_fsmap learned FSMap version 32 2024-09-11T23:29:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.825+0000 7f7b45ab6740 1 -- 172.21.15.43:0/3165113614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564348465620 con 0x564348721af0 2024-09-11T23:29:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.825+0000 7f7b1bfff640 1 -- 172.21.15.43:0/3165113614 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b28005500 con 0x564348721af0 2024-09-11T23:29:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.825+0000 7f7b1bfff640 1 -- 172.21.15.43:0/3165113614 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7b3009ac50 con 0x564348721af0 2024-09-11T23:29:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.825+0000 7f7b45ab6740 10 client.16707 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:29:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.825+0000 7f7b45ab6740 10 client.16707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.825+0000 7f7b45ab6740 20 client.16707 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.825+0000 7f7b45ab6740 1 --2- 172.21.15.43:0/3165113614 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56434873cb80 0x56434873efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.825+0000 7f7b45ab6740 4 client.16707 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:29:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.825+0000 7f7b45ab6740 1 -- 172.21.15.43:0/3165113614 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564348693a20 con 0x56434873cb80 2024-09-11T23:29:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.826+0000 7f7b3bfff640 1 --2- 172.21.15.43:0/3165113614 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56434873cb80 0x56434873efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.826+0000 7f7b1bfff640 10 client.16707 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.826+0000 7f7b3bfff640 1 --2- 172.21.15.43:0/3165113614 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56434873cb80 0x56434873efb0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af226dd33e2978d3 server_cookie=eb4b2f7ffd0dfe7d in_seq=0 out_seq=0 2024-09-11T23:29:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.827+0000 7f7b1bfff640 1 -- 172.21.15.43:0/3165113614 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564348693a20 con 0x56434873cb80 2024-09-11T23:29:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.827+0000 7f7b1bfff640 10 client.16707 handle_command_reply: tid=0 2024-09-11T23:29:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.827+0000 7f7b45ab6740 1 client.16707 shutdown 2024-09-11T23:29:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.827+0000 7f7b45ab6740 1 -- 172.21.15.43:0/3165113614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564348721af0 msgr2=0x564348729ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.827+0000 7f7b45ab6740 1 --2- 172.21.15.43:0/3165113614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564348721af0 0x564348729ad0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000c6a0 tx=0x7f7b300ccfe0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.827+0000 7f7b45ab6740 1 -- 172.21.15.43:0/3165113614 shutdown_connections 2024-09-11T23:29:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.827+0000 7f7b45ab6740 1 --2- 172.21.15.43:0/3165113614 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56434873cb80 0x56434873efb0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.827+0000 7f7b45ab6740 1 --2- 172.21.15.43:0/3165113614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56434872ccb0 0x56434872a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.827+0000 7f7b1bfff640 0 client.16707 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.827+0000 7f7b45ab6740 1 --2- 172.21.15.43:0/3165113614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564348722410 0x56434872a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.827+0000 7f7b45ab6740 1 --2- 172.21.15.43:0/3165113614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564348721af0 0x564348729ad0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.827+0000 7f7b45ab6740 1 -- 172.21.15.43:0/3165113614 >> 172.21.15.43:0/3165113614 conn(0x56434871e060 msgr2=0x564348723dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.827+0000 7f7b45ab6740 1 -- 172.21.15.43:0/3165113614 shutdown_connections 2024-09-11T23:29:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.827+0000 7f7b45ab6740 1 -- 172.21.15.43:0/3165113614 wait complete. 2024-09-11T23:29:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.827+0000 7f7b45ab6740 20 client.16707 trim_cache size 0 max 16384 2024-09-11T23:29:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.841+0000 7f7b45ab6740 1 Processor -- start 2024-09-11T23:29:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.841+0000 7f7b45ab6740 1 -- start start 2024-09-11T23:29:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.841+0000 7f7b45ab6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564348721af0 0x56434872a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.841+0000 7f7b45ab6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564348722410 0x5643487448e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.841+0000 7f7b45ab6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56434872ccb0 0x564348744e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.841+0000 7f7b45ab6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c1b23d0 con 0x564348722410 2024-09-11T23:29:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.841+0000 7f7b45ab6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c1afc70 con 0x56434872ccb0 2024-09-11T23:29:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.841+0000 7f7b45ab6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c1adae0 con 0x564348721af0 2024-09-11T23:29:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.842+0000 7f7b3bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564348722410 0x5643487448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.842+0000 7f7b3bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564348722410 0x5643487448e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57158/0 (socket says 172.21.15.43:57158) 2024-09-11T23:29:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.842+0000 7f7b3bfff640 1 -- 172.21.15.43:0/643505107 learned_addr learned my addr 172.21.15.43:0/643505107 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.842+0000 7f7b412c5640 1 --2- 172.21.15.43:0/643505107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56434872ccb0 0x564348744e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.842+0000 7f7b40ac4640 1 --2- 172.21.15.43:0/643505107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564348721af0 0x56434872a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.842+0000 7f7b3bfff640 1 -- 172.21.15.43:0/643505107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564348721af0 msgr2=0x56434872a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.842+0000 7f7b3bfff640 1 --2- 172.21.15.43:0/643505107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564348721af0 0x56434872a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.842+0000 7f7b3bfff640 1 -- 172.21.15.43:0/643505107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56434872ccb0 msgr2=0x564348744e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.842+0000 7f7b3bfff640 1 --2- 172.21.15.43:0/643505107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56434872ccb0 0x564348744e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.842+0000 7f7b3bfff640 1 -- 172.21.15.43:0/643505107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564348722240 con 0x564348722410 2024-09-11T23:29:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.842+0000 7f7b3bfff640 1 --2- 172.21.15.43:0/643505107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564348722410 0x5643487448e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x564348724690 tx=0x7f7b280096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d1cef3a38d12e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.842+0000 7f7b267fc640 1 -- 172.21.15.43:0/643505107 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28004240 con 0x564348722410 2024-09-11T23:29:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.842+0000 7f7b267fc640 1 -- 172.21.15.43:0/643505107 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b28004810 con 0x564348722410 2024-09-11T23:29:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.842+0000 7f7b267fc640 1 -- 172.21.15.43:0/643505107 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2802e650 con 0x564348722410 2024-09-11T23:29:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.843+0000 7f7b45ab6740 1 -- 172.21.15.43:0/643505107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564348722410 msgr2=0x5643487448e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.843+0000 7f7b45ab6740 1 --2- 172.21.15.43:0/643505107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564348722410 0x5643487448e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x564348724690 tx=0x7f7b280096c0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.843+0000 7f7b45ab6740 1 -- 172.21.15.43:0/643505107 shutdown_connections 2024-09-11T23:29:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.843+0000 7f7b45ab6740 1 --2- 172.21.15.43:0/643505107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56434872ccb0 0x564348744e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.843+0000 7f7b45ab6740 1 --2- 172.21.15.43:0/643505107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564348722410 0x5643487448e0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.843+0000 7f7b45ab6740 1 --2- 172.21.15.43:0/643505107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564348721af0 0x56434872a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.843+0000 7f7b45ab6740 1 -- 172.21.15.43:0/643505107 >> 172.21.15.43:0/643505107 conn(0x56434871e060 msgr2=0x564348728ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.843+0000 7f7b45ab6740 1 -- 172.21.15.43:0/643505107 shutdown_connections 2024-09-11T23:29:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.843+0000 7f7b45ab6740 1 -- 172.21.15.43:0/643505107 wait complete. 2024-09-11T23:29:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.843+0000 7f7b45ab6740 1 -- start start 2024-09-11T23:29:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.843+0000 7f7b45ab6740 1 Processor -- start 2024-09-11T23:29:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b45ab6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564348721af0 0x564348728e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b45ab6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564348722410 0x5643487293a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b45ab6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56434872ccb0 0x5643487298e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b45ab6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c1ae130 con 0x56434872ccb0 2024-09-11T23:29:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b45ab6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c1af5a0 con 0x564348722410 2024-09-11T23:29:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b45ab6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c1ae800 con 0x564348721af0 2024-09-11T23:29:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b45ab6740 10 client.0 _ll_register_callbacks cb 0x7f7b3000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b412c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56434872ccb0 0x5643487298e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b412c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56434872ccb0 0x5643487298e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57172/0 (socket says 172.21.15.43:57172) 2024-09-11T23:29:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b412c5640 1 -- 172.21.15.43:0/3120672074 learned_addr learned my addr 172.21.15.43:0/3120672074 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b40ac4640 1 --2- 172.21.15.43:0/3120672074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564348721af0 0x564348728e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b3bfff640 1 --2- 172.21.15.43:0/3120672074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564348722410 0x5643487293a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b412c5640 1 -- 172.21.15.43:0/3120672074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564348721af0 msgr2=0x564348728e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b412c5640 1 --2- 172.21.15.43:0/3120672074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564348721af0 0x564348728e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b412c5640 1 -- 172.21.15.43:0/3120672074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564348722410 msgr2=0x5643487293a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b412c5640 1 --2- 172.21.15.43:0/3120672074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564348722410 0x5643487293a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b412c5640 1 -- 172.21.15.43:0/3120672074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564348733f60 con 0x56434872ccb0 2024-09-11T23:29:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b40ac4640 1 --2- 172.21.15.43:0/3120672074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564348721af0 0x564348728e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:29:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b3bfff640 1 --2- 172.21.15.43:0/3120672074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564348722410 0x5643487293a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b412c5640 1 --2- 172.21.15.43:0/3120672074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56434872ccb0 0x5643487298e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7b34006010 tx=0x7f7b340095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384830cf35d90783 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:29:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b1bfff640 1 -- 172.21.15.43:0/3120672074 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34034040 con 0x56434872ccb0 2024-09-11T23:29:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b1bfff640 1 -- 172.21.15.43:0/3120672074 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f7b34054470 con 0x56434872ccb0 2024-09-11T23:29:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.844+0000 7f7b45ab6740 1 -- 172.21.15.43:0/3120672074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b28004810 con 0x56434872ccb0 2024-09-11T23:29:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.845+0000 7f7b1bfff640 1 -- 172.21.15.43:0/3120672074 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7b3402f770 con 0x56434872ccb0 2024-09-11T23:29:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.845+0000 7f7b45ab6740 10 client.16713 fetch_fsmap learned FSMap version 32 2024-09-11T23:29:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.845+0000 7f7b45ab6740 1 -- 172.21.15.43:0/3120672074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564348735270 con 0x56434872ccb0 2024-09-11T23:29:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.845+0000 7f7b1bfff640 1 -- 172.21.15.43:0/3120672074 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b28004810 con 0x56434872ccb0 2024-09-11T23:29:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.845+0000 7f7b1bfff640 1 -- 172.21.15.43:0/3120672074 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7b34048040 con 0x56434872ccb0 2024-09-11T23:29:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.845+0000 7f7b45ab6740 10 client.16713 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:29:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.845+0000 7f7b45ab6740 10 client.16713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.845+0000 7f7b45ab6740 20 client.16713 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.845+0000 7f7b45ab6740 1 --2- 172.21.15.43:0/3120672074 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5643487a78b0 0x5643487a9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.845+0000 7f7b45ab6740 4 client.16713 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T23:29:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.845+0000 7f7b45ab6740 1 -- 172.21.15.43:0/3120672074 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56434868dd70 con 0x5643487a78b0 2024-09-11T23:29:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.845+0000 7f7b40ac4640 1 --2- 172.21.15.43:0/3120672074 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5643487a78b0 0x5643487a9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.846+0000 7f7b40ac4640 1 --2- 172.21.15.43:0/3120672074 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5643487a78b0 0x5643487a9ca0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b68e1e097e8d267 server_cookie=870f65a2eb782033 in_seq=0 out_seq=0 2024-09-11T23:29:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.846+0000 7f7b1bfff640 10 client.16713 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:51.859 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:51 smithi043 ceph-mon[30950]: pgmap v246: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 42 KiB/s rd, 44 KiB/s wr, 66 op/s 2024-09-11T23:29:51.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:51 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/564777068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:29:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.877+0000 7f7b1bfff640 1 -- 172.21.15.43:0/3120672074 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56434868dd70 con 0x5643487a78b0 2024-09-11T23:29:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.877+0000 7f7b1bfff640 10 client.16713 handle_command_reply: tid=0 2024-09-11T23:29:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.877+0000 7f7b45ab6740 1 client.16713 shutdown 2024-09-11T23:29:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.878+0000 7f7b45ab6740 1 -- 172.21.15.43:0/3120672074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56434872ccb0 msgr2=0x5643487298e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.878+0000 7f7b45ab6740 1 --2- 172.21.15.43:0/3120672074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56434872ccb0 0x5643487298e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7b34006010 tx=0x7f7b340095c0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.878+0000 7f7b45ab6740 1 -- 172.21.15.43:0/3120672074 shutdown_connections 2024-09-11T23:29:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.878+0000 7f7b45ab6740 1 --2- 172.21.15.43:0/3120672074 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5643487a78b0 0x5643487a9ca0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.878+0000 7f7b45ab6740 1 --2- 172.21.15.43:0/3120672074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56434872ccb0 0x5643487298e0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.878+0000 7f7b45ab6740 1 --2- 172.21.15.43:0/3120672074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564348722410 0x5643487293a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.878+0000 7f7b1bfff640 0 client.16713 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.878+0000 7f7b45ab6740 1 --2- 172.21.15.43:0/3120672074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564348721af0 0x564348728e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.878+0000 7f7b45ab6740 1 -- 172.21.15.43:0/3120672074 >> 172.21.15.43:0/3120672074 conn(0x56434871e060 msgr2=0x5643487234c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.878+0000 7f7b45ab6740 1 -- 172.21.15.43:0/3120672074 shutdown_connections 2024-09-11T23:29:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.878+0000 7f7b45ab6740 1 -- 172.21.15.43:0/3120672074 wait complete. 2024-09-11T23:29:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.878+0000 7f7b45ab6740 20 client.16713 trim_cache size 0 max 16384 2024-09-11T23:29:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:{ 2024-09-11T23:29:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "message": "", 2024-09-11T23:29:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout: "return_code": 0 2024-09-11T23:29:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stdout:} 2024-09-11T23:29:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.878+0000 7f7b42d28640 1 -- 172.21.15.43:0/2053915549 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7b2c0a2cf0 msgr2=0x7f7b2c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.878+0000 7f7b42d28640 1 --2- 172.21.15.43:0/2053915549 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7b2c0a2cf0 0x7f7b2c0a51b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009430 tx=0x7f7b34005c50 comp rx=0 tx=0).stop 2024-09-11T23:29:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.878+0000 7f7b42d28640 1 -- 172.21.15.43:0/2053915549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c102d00 msgr2=0x7f7b3c100950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.878+0000 7f7b42d28640 1 --2- 172.21.15.43:0/2053915549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c102d00 0x7f7b3c100950 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000e9f0 tx=0x7f7b3000edf0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.879+0000 7f7b42d28640 1 -- 172.21.15.43:0/2053915549 shutdown_connections 2024-09-11T23:29:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.879+0000 7f7b42d28640 1 --2- 172.21.15.43:0/2053915549 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f7b2c0a2cf0 0x7f7b2c0a51b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.879+0000 7f7b42d28640 1 --2- 172.21.15.43:0/2053915549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b3c10c0d0 0x7f7b3c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.879+0000 7f7b42d28640 1 --2- 172.21.15.43:0/2053915549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b3c103620 0x7f7b3c100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.879+0000 7f7b42d28640 1 --2- 172.21.15.43:0/2053915549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c102d00 0x7f7b3c100950 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.879+0000 7f7b42d28640 1 -- 172.21.15.43:0/2053915549 >> 172.21.15.43:0/2053915549 conn(0x7f7b3c0779a0 msgr2=0x7f7b3c0fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.879+0000 7f7b42d28640 1 -- 172.21.15.43:0/2053915549 shutdown_connections 2024-09-11T23:29:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi043.front.sepia.ceph.com.smithi043.stderr:2024-09-11T23:29:51.879+0000 7f7b42d28640 1 -- 172.21.15.43:0/2053915549 wait complete. 2024-09-11T23:29:51.932 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --cancel --all' 2024-09-11T23:29:51.932 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --cancel --all 2024-09-11T23:29:52.236 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:29:52.286 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:29:52.287 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:29:52.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:52 smithi043 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:29:52.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:52 smithi090 ceph-mon[35130]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:29:52.800 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:52 smithi097 ceph-mon[35562]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:29:53.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:53 smithi090 ceph-mon[35130]: pgmap v247: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 57 op/s 2024-09-11T23:29:53.800 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:53 smithi097 ceph-mon[35562]: pgmap v247: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 57 op/s 2024-09-11T23:29:53.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:53 smithi043 ceph-mon[30950]: pgmap v247: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 57 op/s 2024-09-11T23:29:54.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.571+0000 7fe852904640 1 -- 172.21.15.43:0/1872718966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe84c102570 msgr2=0x7fe84c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:54.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.571+0000 7fe852904640 1 --2- 172.21.15.43:0/1872718966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe84c102570 0x7fe84c1029b0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe840009900 tx=0x7fe84002a480 comp rx=0 tx=0).stop 2024-09-11T23:29:54.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.571+0000 7fe852904640 1 -- 172.21.15.43:0/1872718966 shutdown_connections 2024-09-11T23:29:54.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.571+0000 7fe852904640 1 --2- 172.21.15.43:0/1872718966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe84c102ef0 0x7fe84c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.571+0000 7fe852904640 1 --2- 172.21.15.43:0/1872718966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe84c102570 0x7fe84c1029b0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.571+0000 7fe852904640 1 --2- 172.21.15.43:0/1872718966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe84c100e20 0x7fe84c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.571+0000 7fe852904640 1 -- 172.21.15.43:0/1872718966 >> 172.21.15.43:0/1872718966 conn(0x7fe84c0fc7d0 msgr2=0x7fe84c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:54.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.571+0000 7fe852904640 1 -- 172.21.15.43:0/1872718966 shutdown_connections 2024-09-11T23:29:54.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.571+0000 7fe852904640 1 -- 172.21.15.43:0/1872718966 wait complete. 2024-09-11T23:29:54.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.571+0000 7fe852904640 1 Processor -- start 2024-09-11T23:29:54.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.571+0000 7fe852904640 1 -- start start 2024-09-11T23:29:54.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.572+0000 7fe852904640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe84c100e20 0x7fe84c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:54.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.572+0000 7fe852904640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe84c102570 0x7fe84c199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:54.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.572+0000 7fe852904640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe84c102ef0 0x7fe84c1a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:54.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.572+0000 7fe852904640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c110f80 con 0x7fe84c102ef0 2024-09-11T23:29:54.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.572+0000 7fe852904640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c110e00 con 0x7fe84c102570 2024-09-11T23:29:54.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.572+0000 7fe852904640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c111100 con 0x7fe84c100e20 2024-09-11T23:29:54.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.572+0000 7fe850ea1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe84c102ef0 0x7fe84c1a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:54.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.572+0000 7fe850ea1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe84c102ef0 0x7fe84c1a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57194/0 (socket says 172.21.15.43:57194) 2024-09-11T23:29:54.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.572+0000 7fe850ea1640 1 -- 172.21.15.43:0/1948095949 learned_addr learned my addr 172.21.15.43:0/1948095949 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:54.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.572+0000 7fe84bfff640 1 --2- 172.21.15.43:0/1948095949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe84c100e20 0x7fe84c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:54.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.572+0000 7fe84b7fe640 1 --2- 172.21.15.43:0/1948095949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe84c102570 0x7fe84c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.572+0000 7fe850ea1640 1 -- 172.21.15.43:0/1948095949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe84c100e20 msgr2=0x7fe84c1992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.572+0000 7fe850ea1640 1 --2- 172.21.15.43:0/1948095949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe84c100e20 0x7fe84c1992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.572+0000 7fe850ea1640 1 -- 172.21.15.43:0/1948095949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe84c102570 msgr2=0x7fe84c199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.572+0000 7fe850ea1640 1 --2- 172.21.15.43:0/1948095949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe84c102570 0x7fe84c199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.572+0000 7fe850ea1640 1 -- 172.21.15.43:0/1948095949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe84c1a4890 con 0x7fe84c102ef0 2024-09-11T23:29:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.572+0000 7fe84b7fe640 1 --2- 172.21.15.43:0/1948095949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe84c102570 0x7fe84c199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.572+0000 7fe850ea1640 1 --2- 172.21.15.43:0/1948095949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe84c102ef0 0x7fe84c1a4350 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe84400cce0 tx=0x7fe84400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.573+0000 7fe8497fa640 1 -- 172.21.15.43:0/1948095949 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84400f450 con 0x7fe84c102ef0 2024-09-11T23:29:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.573+0000 7fe8497fa640 1 -- 172.21.15.43:0/1948095949 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe84400fad0 con 0x7fe84c102ef0 2024-09-11T23:29:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.573+0000 7fe8497fa640 1 -- 172.21.15.43:0/1948095949 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe844018c10 con 0x7fe84c102ef0 2024-09-11T23:29:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.573+0000 7fe852904640 1 -- 172.21.15.43:0/1948095949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe84c1a4a60 con 0x7fe84c102ef0 2024-09-11T23:29:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.573+0000 7fe852904640 1 -- 172.21.15.43:0/1948095949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe84c1a4f20 con 0x7fe84c102ef0 2024-09-11T23:29:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.575+0000 7fe8497fa640 1 -- 172.21.15.43:0/1948095949 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe84400f5f0 con 0x7fe84c102ef0 2024-09-11T23:29:54.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.575+0000 7fe854cdd740 1 Processor -- start 2024-09-11T23:29:54.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.575+0000 7fe854cdd740 1 -- start start 2024-09-11T23:29:54.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.575+0000 7fe854cdd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c84aa00 0x55836c84ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:54.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.575+0000 7fe854cdd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84b320 0x55836c8556a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:54.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.575+0000 7fe854cdd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c855be0 0x55836c857fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:54.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.576+0000 7fe854cdd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55836c7af200 con 0x55836c84aa00 2024-09-11T23:29:54.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.576+0000 7fe854cdd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55836c755dc0 con 0x55836c84b320 2024-09-11T23:29:54.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.576+0000 7fe854cdd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55836c7ab1c0 con 0x55836c855be0 2024-09-11T23:29:54.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.576+0000 7fe84b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c84aa00 0x55836c84ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:54.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.576+0000 7fe84bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84b320 0x55836c8556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:54.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.576+0000 7fe84b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c84aa00 0x55836c84ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57200/0 (socket says 172.21.15.43:57200) 2024-09-11T23:29:54.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.576+0000 7fe84b7fe640 1 -- 172.21.15.43:0/2100106356 learned_addr learned my addr 172.21.15.43:0/2100106356 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:54.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.576+0000 7fe84b7fe640 1 --2- 172.21.15.43:0/2100106356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c855be0 0x55836c857fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:54.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.576+0000 7fe84bfff640 1 -- 172.21.15.43:0/2100106356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c855be0 msgr2=0x55836c857fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:54.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.576+0000 7fe84bfff640 1 --2- 172.21.15.43:0/2100106356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c855be0 0x55836c857fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.576+0000 7fe84bfff640 1 -- 172.21.15.43:0/2100106356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c84aa00 msgr2=0x55836c84ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:54.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.576+0000 7fe84bfff640 1 --2- 172.21.15.43:0/2100106356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c84aa00 0x55836c84ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.576+0000 7fe84bfff640 1 -- 172.21.15.43:0/2100106356 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55836c520620 con 0x55836c84b320 2024-09-11T23:29:54.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.576+0000 7fe8497fa640 1 --2- 172.21.15.43:0/1948095949 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fe8340a2cf0 0x7fe8340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:54.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.576+0000 7fe84b7fe640 1 --2- 172.21.15.43:0/2100106356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c855be0 0x55836c857fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:29:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.576+0000 7fe84b7fe640 1 --2- 172.21.15.43:0/1948095949 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fe8340a2cf0 0x7fe8340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.576+0000 7fe8497fa640 1 -- 172.21.15.43:0/1948095949 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7fe8440cfb10 con 0x7fe84c102ef0 2024-09-11T23:29:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.576+0000 7fe84bfff640 1 --2- 172.21.15.43:0/2100106356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84b320 0x55836c8556a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe83c009c20 tx=0x7fe83c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=818402907d7affaa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.577+0000 7fe8327fc640 1 -- 172.21.15.43:0/2100106356 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83c004050 con 0x55836c84b320 2024-09-11T23:29:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.577+0000 7fe84b7fe640 1 --2- 172.21.15.43:0/1948095949 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fe8340a2cf0 0x7fe8340a51b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55836c848310 tx=0x7fe840005bd0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.577+0000 7fe8327fc640 1 -- 172.21.15.43:0/2100106356 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe83c005500 con 0x55836c84b320 2024-09-11T23:29:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.577+0000 7fe8327fc640 1 -- 172.21.15.43:0/2100106356 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83c0035f0 con 0x55836c84b320 2024-09-11T23:29:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.577+0000 7fe854cdd740 1 -- 172.21.15.43:0/2100106356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84b320 msgr2=0x55836c8556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.577+0000 7fe854cdd740 1 --2- 172.21.15.43:0/2100106356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84b320 0x55836c8556a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe83c009c20 tx=0x7fe83c011810 comp rx=0 tx=0).stop 2024-09-11T23:29:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.577+0000 7fe854cdd740 1 -- 172.21.15.43:0/2100106356 shutdown_connections 2024-09-11T23:29:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.577+0000 7fe854cdd740 1 --2- 172.21.15.43:0/2100106356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c855be0 0x55836c857fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.577+0000 7fe854cdd740 1 --2- 172.21.15.43:0/2100106356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84b320 0x55836c8556a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.577+0000 7fe854cdd740 1 --2- 172.21.15.43:0/2100106356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c84aa00 0x55836c84ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.577+0000 7fe854cdd740 1 -- 172.21.15.43:0/2100106356 >> 172.21.15.43:0/2100106356 conn(0x55836c846f90 msgr2=0x55836c847370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:54.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.577+0000 7fe854cdd740 1 -- 172.21.15.43:0/2100106356 shutdown_connections 2024-09-11T23:29:54.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.577+0000 7fe854cdd740 1 -- 172.21.15.43:0/2100106356 wait complete. 2024-09-11T23:29:54.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.578+0000 7fe854cdd740 1 -- start start 2024-09-11T23:29:54.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.578+0000 7fe854cdd740 1 Processor -- start 2024-09-11T23:29:54.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.578+0000 7fe854cdd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84aa00 0x55836c855360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:54.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.578+0000 7fe854cdd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c84b320 0x55836c8526b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:54.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.578+0000 7fe854cdd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c855be0 0x55836c852bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:54.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.578+0000 7fe854cdd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55836c8587b0 con 0x55836c855be0 2024-09-11T23:29:54.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.578+0000 7fe854cdd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55836c749c50 con 0x55836c84aa00 2024-09-11T23:29:54.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.578+0000 7fe854cdd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55836c858930 con 0x55836c84b320 2024-09-11T23:29:54.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.578+0000 7fe854cdd740 10 client.0 _ll_register_callbacks cb 0x55836c75bba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:54.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.578+0000 7fe850ea1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c855be0 0x55836c852bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:54.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.578+0000 7fe850ea1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c855be0 0x55836c852bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57214/0 (socket says 172.21.15.43:57214) 2024-09-11T23:29:54.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.578+0000 7fe850ea1640 1 -- 172.21.15.43:0/505915411 learned_addr learned my addr 172.21.15.43:0/505915411 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:54.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.578+0000 7fe84bfff640 1 --2- 172.21.15.43:0/505915411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84aa00 0x55836c855360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:54.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.579+0000 7fe84b7fe640 1 --2- 172.21.15.43:0/505915411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c84b320 0x55836c8526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:54.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.579+0000 7fe850ea1640 1 -- 172.21.15.43:0/505915411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c84b320 msgr2=0x55836c8526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:54.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.579+0000 7fe850ea1640 1 --2- 172.21.15.43:0/505915411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c84b320 0x55836c8526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.579+0000 7fe850ea1640 1 -- 172.21.15.43:0/505915411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84aa00 msgr2=0x55836c855360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:54.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.579+0000 7fe850ea1640 1 --2- 172.21.15.43:0/505915411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84aa00 0x55836c855360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.579+0000 7fe850ea1640 1 -- 172.21.15.43:0/505915411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55836c853290 con 0x55836c855be0 2024-09-11T23:29:54.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.579+0000 7fe84b7fe640 1 --2- 172.21.15.43:0/505915411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c84b320 0x55836c8526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:54.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.579+0000 7fe850ea1640 1 --2- 172.21.15.43:0/505915411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c855be0 0x55836c852bf0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe844098970 tx=0x7fe84409bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94d6a814de42b43 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:54.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.579+0000 7fe82b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:29:54.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.579+0000 7fe82b7fe640 1 -- 172.21.15.43:0/505915411 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84409bfa0 con 0x55836c855be0 2024-09-11T23:29:54.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.579+0000 7fe82b7fe640 1 -- 172.21.15.43:0/505915411 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7fe84409c200 con 0x55836c855be0 2024-09-11T23:29:54.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.579+0000 7fe854cdd740 1 -- 172.21.15.43:0/505915411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe83c005500 con 0x55836c855be0 2024-09-11T23:29:54.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.579+0000 7fe82b7fe640 1 -- 172.21.15.43:0/505915411 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe8440a0350 con 0x55836c855be0 2024-09-11T23:29:54.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.579+0000 7fe854cdd740 10 client.16725 fetch_fsmap learned FSMap version 32 2024-09-11T23:29:54.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.579+0000 7fe854cdd740 1 -- 172.21.15.43:0/505915411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55836c864190 con 0x55836c855be0 2024-09-11T23:29:54.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.579+0000 7fe82b7fe640 1 -- 172.21.15.43:0/505915411 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe83c005500 con 0x55836c855be0 2024-09-11T23:29:54.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.580+0000 7fe82b7fe640 1 -- 172.21.15.43:0/505915411 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe84409b3f0 con 0x55836c855be0 2024-09-11T23:29:54.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.580+0000 7fe854cdd740 10 client.16725 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:29:54.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.580+0000 7fe854cdd740 10 client.16725 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:29:54.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.580+0000 7fe854cdd740 20 client.16725 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:54.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.580+0000 7fe854cdd740 1 --2- 172.21.15.43:0/505915411 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55836c865650 0x55836c867a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.580+0000 7fe854cdd740 4 client.16725 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:29:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.580+0000 7fe854cdd740 1 -- 172.21.15.43:0/505915411 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55836c7bc770 con 0x55836c865650 2024-09-11T23:29:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.580+0000 7fe84bfff640 1 --2- 172.21.15.43:0/505915411 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55836c865650 0x55836c867a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.580+0000 7fe82b7fe640 10 client.16725 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.580+0000 7fe84bfff640 1 --2- 172.21.15.43:0/505915411 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55836c865650 0x55836c867a40 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36234dcbb70625c server_cookie=546be2c29c3a70ea in_seq=0 out_seq=0 2024-09-11T23:29:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.581+0000 7fe82b7fe640 1 -- 172.21.15.43:0/505915411 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55836c7bc770 con 0x55836c865650 2024-09-11T23:29:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.581+0000 7fe82b7fe640 10 client.16725 handle_command_reply: tid=0 2024-09-11T23:29:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.581+0000 7fe854cdd740 1 client.16725 shutdown 2024-09-11T23:29:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.581+0000 7fe854cdd740 1 -- 172.21.15.43:0/505915411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c855be0 msgr2=0x55836c852bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.581+0000 7fe854cdd740 1 --2- 172.21.15.43:0/505915411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c855be0 0x55836c852bf0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe844098970 tx=0x7fe84409bbe0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.581+0000 7fe854cdd740 1 -- 172.21.15.43:0/505915411 shutdown_connections 2024-09-11T23:29:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.581+0000 7fe854cdd740 1 --2- 172.21.15.43:0/505915411 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55836c865650 0x55836c867a40 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.581+0000 7fe854cdd740 1 --2- 172.21.15.43:0/505915411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c855be0 0x55836c852bf0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.581+0000 7fe82b7fe640 0 client.16725 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.581+0000 7fe854cdd740 1 --2- 172.21.15.43:0/505915411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c84b320 0x55836c8526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.581+0000 7fe854cdd740 1 --2- 172.21.15.43:0/505915411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84aa00 0x55836c855360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.581+0000 7fe854cdd740 1 -- 172.21.15.43:0/505915411 >> 172.21.15.43:0/505915411 conn(0x55836c846f90 msgr2=0x55836c8461d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.581+0000 7fe854cdd740 1 -- 172.21.15.43:0/505915411 shutdown_connections 2024-09-11T23:29:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.581+0000 7fe854cdd740 1 -- 172.21.15.43:0/505915411 wait complete. 2024-09-11T23:29:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.581+0000 7fe854cdd740 20 client.16725 trim_cache size 0 max 16384 2024-09-11T23:29:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.598+0000 7fe854cdd740 1 Processor -- start 2024-09-11T23:29:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.598+0000 7fe854cdd740 1 -- start start 2024-09-11T23:29:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.598+0000 7fe854cdd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84aa00 0x55836c850f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.598+0000 7fe854cdd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c84b320 0x55836c86d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.598+0000 7fe854cdd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c855be0 0x55836c86d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.598+0000 7fe854cdd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c1b0150 con 0x55836c84b320 2024-09-11T23:29:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.598+0000 7fe854cdd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c1ad9f0 con 0x55836c84aa00 2024-09-11T23:29:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.598+0000 7fe854cdd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c1ab860 con 0x55836c855be0 2024-09-11T23:29:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.599+0000 7fe84b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c84b320 0x55836c86d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.599+0000 7fe84b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c84b320 0x55836c86d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57222/0 (socket says 172.21.15.43:57222) 2024-09-11T23:29:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.599+0000 7fe84b7fe640 1 -- 172.21.15.43:0/3432657539 learned_addr learned my addr 172.21.15.43:0/3432657539 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.599+0000 7fe850ea1640 1 --2- 172.21.15.43:0/3432657539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c855be0 0x55836c86d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.599+0000 7fe84bfff640 1 --2- 172.21.15.43:0/3432657539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84aa00 0x55836c850f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.599+0000 7fe84b7fe640 1 -- 172.21.15.43:0/3432657539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c855be0 msgr2=0x55836c86d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.599+0000 7fe84b7fe640 1 --2- 172.21.15.43:0/3432657539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c855be0 0x55836c86d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.599+0000 7fe84b7fe640 1 -- 172.21.15.43:0/3432657539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84aa00 msgr2=0x55836c850f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.599+0000 7fe84b7fe640 1 --2- 172.21.15.43:0/3432657539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84aa00 0x55836c850f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.599+0000 7fe84b7fe640 1 -- 172.21.15.43:0/3432657539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55836c852d50 con 0x55836c84b320 2024-09-11T23:29:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.599+0000 7fe84b7fe640 1 --2- 172.21.15.43:0/3432657539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c84b320 0x55836c86d2f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe84002a9a0 tx=0x7fe84003f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63812a825c187de3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.599+0000 7fe831ffb640 1 -- 172.21.15.43:0/3432657539 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe840035040 con 0x55836c84b320 2024-09-11T23:29:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.599+0000 7fe831ffb640 1 -- 172.21.15.43:0/3432657539 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe84004f070 con 0x55836c84b320 2024-09-11T23:29:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.599+0000 7fe831ffb640 1 -- 172.21.15.43:0/3432657539 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8400327b0 con 0x55836c84b320 2024-09-11T23:29:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.600+0000 7fe854cdd740 1 -- 172.21.15.43:0/3432657539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c84b320 msgr2=0x55836c86d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.600+0000 7fe854cdd740 1 --2- 172.21.15.43:0/3432657539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c84b320 0x55836c86d2f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe84002a9a0 tx=0x7fe84003f070 comp rx=0 tx=0).stop 2024-09-11T23:29:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.600+0000 7fe854cdd740 1 -- 172.21.15.43:0/3432657539 shutdown_connections 2024-09-11T23:29:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.600+0000 7fe854cdd740 1 --2- 172.21.15.43:0/3432657539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c855be0 0x55836c86d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.600+0000 7fe854cdd740 1 --2- 172.21.15.43:0/3432657539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c84b320 0x55836c86d2f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.600+0000 7fe854cdd740 1 --2- 172.21.15.43:0/3432657539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84aa00 0x55836c850f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.600+0000 7fe854cdd740 1 -- 172.21.15.43:0/3432657539 >> 172.21.15.43:0/3432657539 conn(0x55836c846f90 msgr2=0x55836c8461d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:54.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.600+0000 7fe854cdd740 1 -- 172.21.15.43:0/3432657539 shutdown_connections 2024-09-11T23:29:54.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.600+0000 7fe854cdd740 1 -- 172.21.15.43:0/3432657539 wait complete. 2024-09-11T23:29:54.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.600+0000 7fe854cdd740 1 -- start start 2024-09-11T23:29:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.601+0000 7fe854cdd740 1 Processor -- start 2024-09-11T23:29:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.601+0000 7fe854cdd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84aa00 0x55836c859a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.601+0000 7fe854cdd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c84b320 0x55836c859f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.601+0000 7fe854cdd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c855be0 0x55836c85a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.601+0000 7fe854cdd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c1abeb0 con 0x55836c84b320 2024-09-11T23:29:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.601+0000 7fe854cdd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c1ad320 con 0x55836c84aa00 2024-09-11T23:29:54.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.601+0000 7fe854cdd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c1ac580 con 0x55836c855be0 2024-09-11T23:29:54.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.601+0000 7fe854cdd740 10 client.0 _ll_register_callbacks cb 0x7fe84409bd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:54.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.601+0000 7fe84b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c84b320 0x55836c859f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:54.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.601+0000 7fe84b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c84b320 0x55836c859f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57228/0 (socket says 172.21.15.43:57228) 2024-09-11T23:29:54.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.601+0000 7fe84b7fe640 1 -- 172.21.15.43:0/922658390 learned_addr learned my addr 172.21.15.43:0/922658390 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.601+0000 7fe84bfff640 1 --2- 172.21.15.43:0/922658390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84aa00 0x55836c859a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.601+0000 7fe850ea1640 1 --2- 172.21.15.43:0/922658390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c855be0 0x55836c85a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.601+0000 7fe84bfff640 1 -- 172.21.15.43:0/922658390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c855be0 msgr2=0x55836c85a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.601+0000 7fe84bfff640 1 --2- 172.21.15.43:0/922658390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c855be0 0x55836c85a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.601+0000 7fe84bfff640 1 -- 172.21.15.43:0/922658390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c84b320 msgr2=0x55836c859f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.601+0000 7fe84bfff640 1 --2- 172.21.15.43:0/922658390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c84b320 0x55836c859f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.601+0000 7fe84bfff640 1 -- 172.21.15.43:0/922658390 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55836c851660 con 0x55836c84aa00 2024-09-11T23:29:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.601+0000 7fe84b7fe640 1 --2- 172.21.15.43:0/922658390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c84b320 0x55836c859f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:29:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.602+0000 7fe84bfff640 1 --2- 172.21.15.43:0/922658390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84aa00 0x55836c859a00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe83c01b040 tx=0x7fe83c004870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=194941bfcbd29d17 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.602+0000 7fe82b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:29:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.602+0000 7fe82b7fe640 1 -- 172.21.15.43:0/922658390 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83c004450 con 0x55836c84aa00 2024-09-11T23:29:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.602+0000 7fe854cdd740 1 -- 172.21.15.43:0/922658390 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe84004f070 con 0x55836c84aa00 2024-09-11T23:29:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.602+0000 7fe82b7fe640 1 -- 172.21.15.43:0/922658390 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7fe83c0035f0 con 0x55836c84aa00 2024-09-11T23:29:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.602+0000 7fe82b7fe640 1 -- 172.21.15.43:0/922658390 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe83c003810 con 0x55836c84aa00 2024-09-11T23:29:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.602+0000 7fe854cdd740 10 client.26213 fetch_fsmap learned FSMap version 32 2024-09-11T23:29:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.602+0000 7fe854cdd740 1 -- 172.21.15.43:0/922658390 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55836c842730 con 0x55836c84aa00 2024-09-11T23:29:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.602+0000 7fe82b7fe640 1 -- 172.21.15.43:0/922658390 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe84004f070 con 0x55836c84aa00 2024-09-11T23:29:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.602+0000 7fe82b7fe640 1 -- 172.21.15.43:0/922658390 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe83c036910 con 0x55836c84aa00 2024-09-11T23:29:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.602+0000 7fe854cdd740 10 client.26213 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:29:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.602+0000 7fe854cdd740 10 client.26213 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-11T23:29:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.602+0000 7fe854cdd740 20 client.26213 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.602+0000 7fe854cdd740 1 --2- 172.21.15.43:0/922658390 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55836c8d0860 0x55836c8d2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.602+0000 7fe854cdd740 4 client.26213 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-11T23:29:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.602+0000 7fe854cdd740 1 -- 172.21.15.43:0/922658390 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55836c7b6c80 con 0x55836c8d0860 2024-09-11T23:29:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.603+0000 7fe84b7fe640 1 --2- 172.21.15.43:0/922658390 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55836c8d0860 0x55836c8d2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.603+0000 7fe84b7fe640 1 --2- 172.21.15.43:0/922658390 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55836c8d0860 0x55836c8d2c50 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce02a1ebe5a1f606 server_cookie=5610ccdff9b337a1 in_seq=0 out_seq=0 2024-09-11T23:29:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.603+0000 7fe82b7fe640 10 client.26213 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.603+0000 7fe82b7fe640 1 -- 172.21.15.43:0/922658390 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55836c7b6c80 con 0x55836c8d0860 2024-09-11T23:29:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.603+0000 7fe82b7fe640 10 client.26213 handle_command_reply: tid=0 2024-09-11T23:29:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.603+0000 7fe854cdd740 1 client.26213 shutdown 2024-09-11T23:29:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.603+0000 7fe854cdd740 1 -- 172.21.15.43:0/922658390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84aa00 msgr2=0x55836c859a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.603+0000 7fe854cdd740 1 --2- 172.21.15.43:0/922658390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84aa00 0x55836c859a00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe83c01b040 tx=0x7fe83c004870 comp rx=0 tx=0).stop 2024-09-11T23:29:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.603+0000 7fe854cdd740 1 -- 172.21.15.43:0/922658390 shutdown_connections 2024-09-11T23:29:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.603+0000 7fe854cdd740 1 --2- 172.21.15.43:0/922658390 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55836c8d0860 0x55836c8d2c50 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.604+0000 7fe854cdd740 1 --2- 172.21.15.43:0/922658390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55836c855be0 0x55836c85a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.604+0000 7fe82b7fe640 0 client.26213 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.604+0000 7fe854cdd740 1 --2- 172.21.15.43:0/922658390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55836c84b320 0x55836c859f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.604+0000 7fe854cdd740 1 --2- 172.21.15.43:0/922658390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55836c84aa00 0x55836c859a00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.604+0000 7fe854cdd740 1 -- 172.21.15.43:0/922658390 >> 172.21.15.43:0/922658390 conn(0x55836c846f90 msgr2=0x55836c8461d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.604+0000 7fe854cdd740 1 -- 172.21.15.43:0/922658390 shutdown_connections 2024-09-11T23:29:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.604+0000 7fe854cdd740 1 -- 172.21.15.43:0/922658390 wait complete. 2024-09-11T23:29:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.604+0000 7fe854cdd740 20 client.26213 trim_cache size 0 max 16384 2024-09-11T23:29:54.658 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:29:54.658 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 32, 2024-09-11T23:29:54.658 INFO:teuthology.orchestra.run.smithi043.stdout: "leader": 24497, 2024-09-11T23:29:54.658 INFO:teuthology.orchestra.run.smithi043.stdout: "set_version": 20, 2024-09-11T23:29:54.658 INFO:teuthology.orchestra.run.smithi043.stdout: "sets": {} 2024-09-11T23:29:54.659 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:29:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.604+0000 7fe852904640 1 -- 172.21.15.43:0/1948095949 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fe8340a2cf0 msgr2=0x7fe8340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.604+0000 7fe852904640 1 --2- 172.21.15.43:0/1948095949 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fe8340a2cf0 0x7fe8340a51b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55836c848310 tx=0x7fe840005bd0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.604+0000 7fe852904640 1 -- 172.21.15.43:0/1948095949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe84c102ef0 msgr2=0x7fe84c1a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.604+0000 7fe852904640 1 --2- 172.21.15.43:0/1948095949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe84c102ef0 0x7fe84c1a4350 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe84400cce0 tx=0x7fe84400a620 comp rx=0 tx=0).stop 2024-09-11T23:29:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.604+0000 7fe852904640 1 -- 172.21.15.43:0/1948095949 shutdown_connections 2024-09-11T23:29:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.604+0000 7fe852904640 1 --2- 172.21.15.43:0/1948095949 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fe8340a2cf0 0x7fe8340a51b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.604+0000 7fe852904640 1 --2- 172.21.15.43:0/1948095949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe84c102ef0 0x7fe84c1a4350 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.604+0000 7fe852904640 1 --2- 172.21.15.43:0/1948095949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe84c102570 0x7fe84c199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.604+0000 7fe852904640 1 --2- 172.21.15.43:0/1948095949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe84c100e20 0x7fe84c1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.604+0000 7fe852904640 1 -- 172.21.15.43:0/1948095949 >> 172.21.15.43:0/1948095949 conn(0x7fe84c0fc7d0 msgr2=0x7fe84c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:54.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.605+0000 7fe852904640 1 -- 172.21.15.43:0/1948095949 shutdown_connections 2024-09-11T23:29:54.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:54.605+0000 7fe852904640 1 -- 172.21.15.43:0/1948095949 wait complete. 2024-09-11T23:29:54.943 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:29:55.728 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:55 smithi043 ceph-mon[30950]: pgmap v248: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 51 op/s 2024-09-11T23:29:55.755 INFO:tasks.quiescer:done joining Quiescers 2024-09-11T23:29:55.755 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-11T23:29:55.766 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi043.front.sepia.ceph.com 2024-09-11T23:29:55.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:55 smithi090 ceph-mon[35130]: pgmap v248: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 51 op/s 2024-09-11T23:29:55.774 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-11T23:29:55.816 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-11T23:29:55.816 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T23:29:55.816 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-09-11T23:29:55.816 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-11T23:29:55.817 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:55 smithi097 ceph-mon[35562]: pgmap v248: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 51 op/s 2024-09-11T23:29:55.893 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-11T23:29:56.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.017+0000 7f9e9c906640 1 -- 172.21.15.43:0/3225097844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e9810c180 msgr2=0x7f9e9810e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:56.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.017+0000 7f9e9c906640 1 --2- 172.21.15.43:0/3225097844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e9810c180 0x7f9e9810e620 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009ca0 tx=0x7f9e9002a950 comp rx=0 tx=0).stop 2024-09-11T23:29:56.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.018+0000 7f9e9c906640 1 -- 172.21.15.43:0/3225097844 shutdown_connections 2024-09-11T23:29:56.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.018+0000 7f9e9c906640 1 --2- 172.21.15.43:0/3225097844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e9810c180 0x7f9e9810e620 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.018+0000 7f9e9c906640 1 --2- 172.21.15.43:0/3225097844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e98103670 0x7f9e9810bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.018+0000 7f9e9c906640 1 --2- 172.21.15.43:0/3225097844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e98102d50 0x7f9e98103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.018+0000 7f9e9c906640 1 -- 172.21.15.43:0/3225097844 >> 172.21.15.43:0/3225097844 conn(0x7f9e980fc7d0 msgr2=0x7f9e980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:56.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.018+0000 7f9e9c906640 1 -- 172.21.15.43:0/3225097844 shutdown_connections 2024-09-11T23:29:56.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.018+0000 7f9e9c906640 1 -- 172.21.15.43:0/3225097844 wait complete. 2024-09-11T23:29:56.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.019+0000 7f9e9c906640 1 Processor -- start 2024-09-11T23:29:56.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.019+0000 7f9e9c906640 1 -- start start 2024-09-11T23:29:56.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.019+0000 7f9e9c906640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e98102d50 0x7f9e98196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:56.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.019+0000 7f9e9c906640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e98103670 0x7f9e981967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:56.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.019+0000 7f9e9c906640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e9810c180 0x7f9e9819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:56.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.019+0000 7f9e9c906640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9810fde0 con 0x7f9e9810c180 2024-09-11T23:29:56.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.019+0000 7f9e9c906640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9810fc60 con 0x7f9e98102d50 2024-09-11T23:29:56.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.019+0000 7f9e9c906640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9810ff60 con 0x7f9e98103670 2024-09-11T23:29:56.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.019+0000 7f9e9659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e98102d50 0x7f9e98196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:56.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.020+0000 7f9e9659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e98102d50 0x7f9e98196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:42376/0 (socket says 172.21.15.43:42376) 2024-09-11T23:29:56.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.020+0000 7f9e9659c640 1 -- 172.21.15.43:0/1081699801 learned_addr learned my addr 172.21.15.43:0/1081699801 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:56.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.020+0000 7f9e96d9d640 1 --2- 172.21.15.43:0/1081699801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e9810c180 0x7f9e9819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:56.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.020+0000 7f9e95d9b640 1 --2- 172.21.15.43:0/1081699801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e98103670 0x7f9e981967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:56.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.020+0000 7f9e9659c640 1 -- 172.21.15.43:0/1081699801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e98103670 msgr2=0x7f9e981967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:56.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.020+0000 7f9e9659c640 1 --2- 172.21.15.43:0/1081699801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e98103670 0x7f9e981967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.020+0000 7f9e9659c640 1 -- 172.21.15.43:0/1081699801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e9810c180 msgr2=0x7f9e9819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:56.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.020+0000 7f9e9659c640 1 --2- 172.21.15.43:0/1081699801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e9810c180 0x7f9e9819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.020+0000 7f9e9659c640 1 -- 172.21.15.43:0/1081699801 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e9819b210 con 0x7f9e98102d50 2024-09-11T23:29:56.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.020+0000 7f9e96d9d640 1 --2- 172.21.15.43:0/1081699801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e9810c180 0x7f9e9819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:56.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.020+0000 7f9e9659c640 1 --2- 172.21.15.43:0/1081699801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e98102d50 0x7f9e98196260 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009be0 tx=0x7f9e880071c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:56.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.020+0000 7f9e877fe640 1 -- 172.21.15.43:0/1081699801 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e88007e00 con 0x7f9e98102d50 2024-09-11T23:29:56.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.020+0000 7f9e877fe640 1 -- 172.21.15.43:0/1081699801 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e8801e070 con 0x7f9e98102d50 2024-09-11T23:29:56.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.021+0000 7f9e877fe640 1 -- 172.21.15.43:0/1081699801 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e88018890 con 0x7f9e98102d50 2024-09-11T23:29:56.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.021+0000 7f9e9c906640 1 -- 172.21.15.43:0/1081699801 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e9819b500 con 0x7f9e98102d50 2024-09-11T23:29:56.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.022+0000 7f9e9c906640 1 -- 172.21.15.43:0/1081699801 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e981a2de0 con 0x7f9e98102d50 2024-09-11T23:29:56.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.025+0000 7f9e9ecdf740 1 Processor -- start 2024-09-11T23:29:56.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.025+0000 7f9e9ecdf740 1 -- start start 2024-09-11T23:29:56.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.025+0000 7f9e877fe640 1 -- 172.21.15.43:0/1081699801 <== mon.2 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9e88018a30 con 0x7f9e98102d50 2024-09-11T23:29:56.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.025+0000 7f9e9ecdf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81076b60 0x559a81076f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:56.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.025+0000 7f9e9ecdf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81077480 0x559a81081800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:56.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.025+0000 7f9e9ecdf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81081d40 0x559a81084130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:56.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.025+0000 7f9e9ecdf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559a80fdc280 con 0x559a81077480 2024-09-11T23:29:56.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.025+0000 7f9e9ecdf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559a80f83b10 con 0x559a81081d40 2024-09-11T23:29:56.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.025+0000 7f9e9ecdf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559a80fd8240 con 0x559a81076b60 2024-09-11T23:29:56.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.026+0000 7f9e95d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81081d40 0x559a81084130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:56.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.026+0000 7f9e95d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81081d40 0x559a81084130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:42378/0 (socket says 172.21.15.43:42378) 2024-09-11T23:29:56.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.026+0000 7f9e95d9b640 1 -- 172.21.15.43:0/67153297 learned_addr learned my addr 172.21.15.43:0/67153297 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:56.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.026+0000 7f9e96d9d640 1 --2- 172.21.15.43:0/67153297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81076b60 0x559a81076f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:56.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.026+0000 7f9e96d9d640 1 -- 172.21.15.43:0/67153297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81081d40 msgr2=0x559a81084130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:56.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.026+0000 7f9e96d9d640 1 --2- 172.21.15.43:0/67153297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81081d40 0x559a81084130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.026+0000 7f9e96d9d640 1 -- 172.21.15.43:0/67153297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81077480 msgr2=0x559a81081800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:56.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.026+0000 7f9e9659c640 1 --2- 172.21.15.43:0/67153297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81077480 0x559a81081800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:56.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.026+0000 7f9e96d9d640 1 --2- 172.21.15.43:0/67153297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81077480 0x559a81081800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.026+0000 7f9e96d9d640 1 -- 172.21.15.43:0/67153297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a80f901f0 con 0x559a81076b60 2024-09-11T23:29:56.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.026+0000 7f9e95d9b640 1 --2- 172.21.15.43:0/67153297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81081d40 0x559a81084130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:29:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.027+0000 7f9e96d9d640 1 --2- 172.21.15.43:0/67153297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81076b60 0x559a81076f40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9e90006010 tx=0x7f9e90009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48b2c0dffc5fb1eb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.027+0000 7f9e877fe640 1 --2- 172.21.15.43:0/1081699801 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e800a2c50 0x7f9e800a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.027+0000 7f9e84ff9640 1 -- 172.21.15.43:0/67153297 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90034040 con 0x559a81076b60 2024-09-11T23:29:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.028+0000 7f9e84ff9640 1 -- 172.21.15.43:0/67153297 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e90003e50 con 0x559a81076b60 2024-09-11T23:29:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.028+0000 7f9e84ff9640 1 -- 172.21.15.43:0/67153297 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90004280 con 0x559a81076b60 2024-09-11T23:29:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.028+0000 7f9e877fe640 1 -- 172.21.15.43:0/1081699801 <== mon.2 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f9e880ceda0 con 0x7f9e98102d50 2024-09-11T23:29:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.028+0000 7f9e9659c640 1 --2- 172.21.15.43:0/1081699801 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e800a2c50 0x7f9e800a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.029+0000 7f9e9659c640 1 --2- 172.21.15.43:0/1081699801 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e800a2c50 0x7f9e800a5110 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x559a81073f50 tx=0x7f9e880bb040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.029+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/67153297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81076b60 msgr2=0x559a81076f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.029+0000 7f9e9ecdf740 1 --2- 172.21.15.43:0/67153297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81076b60 0x559a81076f40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9e90006010 tx=0x7f9e90009a20 comp rx=0 tx=0).stop 2024-09-11T23:29:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.029+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/67153297 shutdown_connections 2024-09-11T23:29:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.029+0000 7f9e9ecdf740 1 --2- 172.21.15.43:0/67153297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81081d40 0x559a81084130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.029+0000 7f9e9ecdf740 1 --2- 172.21.15.43:0/67153297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81077480 0x559a81081800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.029+0000 7f9e9ecdf740 1 --2- 172.21.15.43:0/67153297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81076b60 0x559a81076f40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.029+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/67153297 >> 172.21.15.43:0/67153297 conn(0x559a810730f0 msgr2=0x559a810734d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:56.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.029+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/67153297 shutdown_connections 2024-09-11T23:29:56.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.029+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/67153297 wait complete. 2024-09-11T23:29:56.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.029+0000 7f9e9ecdf740 1 -- start start 2024-09-11T23:29:56.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.030+0000 7f9e9ecdf740 1 Processor -- start 2024-09-11T23:29:56.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.030+0000 7f9e9ecdf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81076b60 0x559a8107ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:56.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.030+0000 7f9e9ecdf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81077480 0x559a8107f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:56.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.030+0000 7f9e9ecdf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81081d40 0x559a8107f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:56.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.030+0000 7f9e9ecdf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559a81095920 con 0x559a81076b60 2024-09-11T23:29:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.030+0000 7f9e9ecdf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559a80f96f10 con 0x559a81081d40 2024-09-11T23:29:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.030+0000 7f9e9ecdf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559a81095aa0 con 0x559a81077480 2024-09-11T23:29:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.030+0000 7f9e9ecdf740 10 client.0 _ll_register_callbacks cb 0x559a80f5fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.030+0000 7f9e95d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81077480 0x559a8107f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.030+0000 7f9e95d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81077480 0x559a8107f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:43620/0 (socket says 172.21.15.43:43620) 2024-09-11T23:29:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.030+0000 7f9e95d9b640 1 -- 172.21.15.43:0/225342151 learned_addr learned my addr 172.21.15.43:0/225342151 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.030+0000 7f9e96d9d640 1 --2- 172.21.15.43:0/225342151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81076b60 0x559a8107ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.030+0000 7f9e96d9d640 1 --2- 172.21.15.43:0/225342151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81081d40 0x559a8107f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.031+0000 7f9e96d9d640 1 -- 172.21.15.43:0/225342151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81077480 msgr2=0x559a8107f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.031+0000 7f9e96d9d640 1 --2- 172.21.15.43:0/225342151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81077480 0x559a8107f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.031+0000 7f9e96d9d640 1 -- 172.21.15.43:0/225342151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81081d40 msgr2=0x559a8107f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:56.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.031+0000 7f9e96d9d640 1 --2- 172.21.15.43:0/225342151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81081d40 0x559a8107f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.031+0000 7f9e96d9d640 1 -- 172.21.15.43:0/225342151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a80d4d6a0 con 0x559a81076b60 2024-09-11T23:29:56.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.031+0000 7f9e95d9b640 1 --2- 172.21.15.43:0/225342151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81077480 0x559a8107f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:29:56.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.031+0000 7f9e96d9d640 1 --2- 172.21.15.43:0/225342151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81076b60 0x559a8107ebe0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9e9002af40 tx=0x7f9e9002ff10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111e2363ded2ba1e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:56.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.031+0000 7f9e7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:29:56.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.031+0000 7f9e7dffb640 1 -- 172.21.15.43:0/225342151 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90055060 con 0x559a81076b60 2024-09-11T23:29:56.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.031+0000 7f9e7dffb640 1 -- 172.21.15.43:0/225342151 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f9e90003970 con 0x559a81076b60 2024-09-11T23:29:56.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.031+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/225342151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e90003e50 con 0x559a81076b60 2024-09-11T23:29:56.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.031+0000 7f9e7dffb640 1 -- 172.21.15.43:0/225342151 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9e90034070 con 0x559a81076b60 2024-09-11T23:29:56.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.031+0000 7f9e9ecdf740 10 client.16740 fetch_fsmap learned FSMap version 32 2024-09-11T23:29:56.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.031+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/225342151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559a81095920 con 0x559a81076b60 2024-09-11T23:29:56.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.032+0000 7f9e7dffb640 1 -- 172.21.15.43:0/225342151 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e90003b70 con 0x559a81076b60 2024-09-11T23:29:56.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.032+0000 7f9e7dffb640 1 -- 172.21.15.43:0/225342151 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9e90042070 con 0x559a81076b60 2024-09-11T23:29:56.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.032+0000 7f9e9ecdf740 10 client.16740 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:29:56.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.032+0000 7f9e9ecdf740 10 client.16740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:56.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.032+0000 7f9e9ecdf740 20 client.16740 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:56.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.032+0000 7f9e9ecdf740 1 --2- 172.21.15.43:0/225342151 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x559a810a26a0 0x559a810a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:56.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.032+0000 7f9e9ecdf740 4 client.16740 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:29:56.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.032+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/225342151 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9e90003e50 con 0x559a810a26a0 2024-09-11T23:29:56.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.032+0000 7f9e95d9b640 1 --2- 172.21.15.43:0/225342151 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x559a810a26a0 0x559a810a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:56.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.033+0000 7f9e95d9b640 1 --2- 172.21.15.43:0/225342151 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x559a810a26a0 0x559a810a4a90 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51281648f8ff25f2 server_cookie=347b9d77ed50a5d2 in_seq=0 out_seq=0 2024-09-11T23:29:56.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.033+0000 7f9e7dffb640 10 client.16740 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:56.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.034+0000 7f9e7dffb640 1 -- 172.21.15.43:0/225342151 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9e90003e50 con 0x559a810a26a0 2024-09-11T23:29:56.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.034+0000 7f9e7dffb640 10 client.16740 handle_command_reply: tid=0 2024-09-11T23:29:56.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.034+0000 7f9e9ecdf740 1 client.16740 shutdown 2024-09-11T23:29:56.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.034+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/225342151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81076b60 msgr2=0x559a8107ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:56.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.034+0000 7f9e9ecdf740 1 --2- 172.21.15.43:0/225342151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81076b60 0x559a8107ebe0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9e9002af40 tx=0x7f9e9002ff10 comp rx=0 tx=0).stop 2024-09-11T23:29:56.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.034+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/225342151 shutdown_connections 2024-09-11T23:29:56.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.034+0000 7f9e9ecdf740 1 --2- 172.21.15.43:0/225342151 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x559a810a26a0 0x559a810a4a90 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.034+0000 7f9e7dffb640 0 client.16740 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:56.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.034+0000 7f9e9ecdf740 1 --2- 172.21.15.43:0/225342151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81081d40 0x559a8107f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.034+0000 7f9e9ecdf740 1 --2- 172.21.15.43:0/225342151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81077480 0x559a8107f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.034+0000 7f9e9ecdf740 1 --2- 172.21.15.43:0/225342151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81076b60 0x559a8107ebe0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.034+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/225342151 >> 172.21.15.43:0/225342151 conn(0x559a810730f0 msgr2=0x559a81072270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:56.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.035+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/225342151 shutdown_connections 2024-09-11T23:29:56.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.035+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/225342151 wait complete. 2024-09-11T23:29:56.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.035+0000 7f9e9ecdf740 20 client.16740 trim_cache size 0 max 16384 2024-09-11T23:29:56.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.062+0000 7f9e9ecdf740 1 Processor -- start 2024-09-11T23:29:56.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.062+0000 7f9e9ecdf740 1 -- start start 2024-09-11T23:29:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.063+0000 7f9e9ecdf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81076b60 0x559a81079080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.063+0000 7f9e9ecdf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81077480 0x559a8107f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.063+0000 7f9e9ecdf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81081d40 0x559a810aa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.063+0000 7f9e9ecdf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e981addf0 con 0x559a81081d40 2024-09-11T23:29:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.063+0000 7f9e9ecdf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9e981ab690 con 0x559a81077480 2024-09-11T23:29:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.063+0000 7f9e9ecdf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e981a9500 con 0x559a81076b60 2024-09-11T23:29:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.063+0000 7f9e96d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81081d40 0x559a810aa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.063+0000 7f9e95d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81077480 0x559a8107f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.063+0000 7f9e96d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81081d40 0x559a810aa120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57260/0 (socket says 172.21.15.43:57260) 2024-09-11T23:29:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.063+0000 7f9e95d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81077480 0x559a8107f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:42394/0 (socket says 172.21.15.43:42394) 2024-09-11T23:29:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.063+0000 7f9e96d9d640 1 -- 172.21.15.43:0/1787673233 learned_addr learned my addr 172.21.15.43:0/1787673233 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.063+0000 7f9e96d9d640 1 --2- 172.21.15.43:0/1787673233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81076b60 0x559a81079080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.063+0000 7f9e96d9d640 1 -- 172.21.15.43:0/1787673233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81076b60 msgr2=0x559a81079080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.063+0000 7f9e96d9d640 1 --2- 172.21.15.43:0/1787673233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81076b60 0x559a81079080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.064+0000 7f9e96d9d640 1 -- 172.21.15.43:0/1787673233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81077480 msgr2=0x559a8107f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.064+0000 7f9e96d9d640 1 --2- 172.21.15.43:0/1787673233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81077480 0x559a8107f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.064+0000 7f9e96d9d640 1 -- 172.21.15.43:0/1787673233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a8109ebb0 con 0x559a81081d40 2024-09-11T23:29:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.064+0000 7f9e95d9b640 1 --2- 172.21.15.43:0/1787673233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81077480 0x559a8107f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:29:56.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.064+0000 7f9e96d9d640 1 --2- 172.21.15.43:0/1787673233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81081d40 0x559a810aa120 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9e90005fa0 tx=0x7f9e90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b453a3c4ee21c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.064+0000 7f9e7ffff640 1 -- 172.21.15.43:0/1787673233 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90055070 con 0x559a81081d40 2024-09-11T23:29:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.064+0000 7f9e7ffff640 1 -- 172.21.15.43:0/1787673233 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e90003e50 con 0x559a81081d40 2024-09-11T23:29:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.064+0000 7f9e7ffff640 1 -- 172.21.15.43:0/1787673233 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90004280 con 0x559a81081d40 2024-09-11T23:29:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.065+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/1787673233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81081d40 msgr2=0x559a810aa120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.065+0000 7f9e9ecdf740 1 --2- 172.21.15.43:0/1787673233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81081d40 0x559a810aa120 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9e90005fa0 tx=0x7f9e90004a40 comp rx=0 tx=0).stop 2024-09-11T23:29:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.065+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/1787673233 shutdown_connections 2024-09-11T23:29:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.065+0000 7f9e9ecdf740 1 --2- 172.21.15.43:0/1787673233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81081d40 0x559a810aa120 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.065+0000 7f9e9ecdf740 1 --2- 172.21.15.43:0/1787673233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81077480 0x559a8107f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.065+0000 7f9e9ecdf740 1 --2- 172.21.15.43:0/1787673233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81076b60 0x559a81079080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.065+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/1787673233 >> 172.21.15.43:0/1787673233 conn(0x559a810730f0 msgr2=0x559a81072270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:56.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.065+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/1787673233 shutdown_connections 2024-09-11T23:29:56.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.065+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/1787673233 wait complete. 2024-09-11T23:29:56.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.066+0000 7f9e9ecdf740 1 -- start start 2024-09-11T23:29:56.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.066+0000 7f9e9ecdf740 1 Processor -- start 2024-09-11T23:29:56.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.066+0000 7f9e9ecdf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81076b60 0x559a81097130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:56.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.066+0000 7f9e9ecdf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81077480 0x559a81097670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:56.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.066+0000 7f9e9ecdf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81081d40 0x559a8110c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:56.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.066+0000 7f9e9ecdf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e981a9b50 con 0x559a81081d40 2024-09-11T23:29:56.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.066+0000 7f9e9ecdf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9e981aafc0 con 0x559a81077480 2024-09-11T23:29:56.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.066+0000 7f9e9ecdf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e981aa220 con 0x559a81076b60 2024-09-11T23:29:56.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.066+0000 7f9e9ecdf740 10 client.0 _ll_register_callbacks cb 0x7f9e9806b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:56.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.066+0000 7f9e95d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81077480 0x559a81097670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:56.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.066+0000 7f9e95d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81077480 0x559a81097670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:42408/0 (socket says 172.21.15.43:42408) 2024-09-11T23:29:56.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.067+0000 7f9e95d9b640 1 -- 172.21.15.43:0/4152343762 learned_addr learned my addr 172.21.15.43:0/4152343762 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:56.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.068+0000 7f9e95d9b640 1 -- 172.21.15.43:0/4152343762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81076b60 msgr2=0x559a81097130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:29:56.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.068+0000 7f9e95d9b640 1 --2- 172.21.15.43:0/4152343762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81076b60 0x559a81097130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.068+0000 7f9e95d9b640 1 -- 172.21.15.43:0/4152343762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81081d40 msgr2=0x559a8110c0f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:29:56.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.068+0000 7f9e95d9b640 1 --2- 172.21.15.43:0/4152343762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81081d40 0x559a8110c0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.068+0000 7f9e95d9b640 1 -- 172.21.15.43:0/4152343762 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a8110c630 con 0x559a81077480 2024-09-11T23:29:56.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.072+0000 7f9e95d9b640 1 --2- 172.21.15.43:0/4152343762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81077480 0x559a81097670 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x559a81080220 tx=0x7f9e8c012b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e2cac58151d88e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:56.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.072+0000 7f9e7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-11T23:29:56.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.072+0000 7f9e7dffb640 1 -- 172.21.15.43:0/4152343762 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c004020 con 0x559a81077480 2024-09-11T23:29:56.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.072+0000 7f9e7dffb640 1 -- 172.21.15.43:0/4152343762 <== mon.2 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f9e8c0041c0 con 0x559a81077480 2024-09-11T23:29:56.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.072+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/4152343762 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e90003e50 con 0x559a81077480 2024-09-11T23:29:56.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.072+0000 7f9e7dffb640 1 -- 172.21.15.43:0/4152343762 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9e8c033050 con 0x559a81077480 2024-09-11T23:29:56.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.072+0000 7f9e9ecdf740 10 client.26231 fetch_fsmap learned FSMap version 32 2024-09-11T23:29:56.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.072+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/4152343762 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559a8110d910 con 0x559a81077480 2024-09-11T23:29:56.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.073+0000 7f9e7dffb640 1 -- 172.21.15.43:0/4152343762 <== mon.2 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e8c01fdd0 con 0x559a81077480 2024-09-11T23:29:56.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.073+0000 7f9e7dffb640 1 -- 172.21.15.43:0/4152343762 <== mon.2 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9e8c01d410 con 0x559a81077480 2024-09-11T23:29:56.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.073+0000 7f9e9ecdf740 10 client.26231 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:29:56.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.073+0000 7f9e9ecdf740 10 client.26231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:56.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.073+0000 7f9e9ecdf740 20 client.26231 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:56.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.073+0000 7f9e9ecdf740 1 --2- 172.21.15.43:0/4152343762 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x559a8110e0c0 0x559a811104e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:56.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.073+0000 7f9e9ecdf740 4 client.26231 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:29:56.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.073+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/4152343762 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f9e90003e50 con 0x559a8110e0c0 2024-09-11T23:29:56.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.074+0000 7f9e96d9d640 1 --2- 172.21.15.43:0/4152343762 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x559a8110e0c0 0x559a811104e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:56.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.074+0000 7f9e96d9d640 1 --2- 172.21.15.43:0/4152343762 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x559a8110e0c0 0x559a811104e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2702208e5d46be5b server_cookie=a3bccd40f19e4cef in_seq=0 out_seq=0 2024-09-11T23:29:56.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.074+0000 7f9e7dffb640 10 client.26231 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:56.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.075+0000 7f9e7dffb640 1 -- 172.21.15.43:0/4152343762 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f9e90003e50 con 0x559a8110e0c0 2024-09-11T23:29:56.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.075+0000 7f9e7dffb640 10 client.26231 handle_command_reply: tid=0 2024-09-11T23:29:56.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.076+0000 7f9e9ecdf740 1 client.26231 shutdown 2024-09-11T23:29:56.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.076+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/4152343762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81077480 msgr2=0x559a81097670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.076+0000 7f9e9ecdf740 1 --2- 172.21.15.43:0/4152343762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81077480 0x559a81097670 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x559a81080220 tx=0x7f9e8c012b00 comp rx=0 tx=0).stop 2024-09-11T23:29:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.076+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/4152343762 shutdown_connections 2024-09-11T23:29:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.076+0000 7f9e9ecdf740 1 --2- 172.21.15.43:0/4152343762 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x559a8110e0c0 0x559a811104e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.076+0000 7f9e9ecdf740 1 --2- 172.21.15.43:0/4152343762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559a81081d40 0x559a8110c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.076+0000 7f9e7dffb640 0 client.26231 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.076+0000 7f9e9ecdf740 1 --2- 172.21.15.43:0/4152343762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a81077480 0x559a81097670 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.076+0000 7f9e9ecdf740 1 --2- 172.21.15.43:0/4152343762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a81076b60 0x559a81097130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.076+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/4152343762 >> 172.21.15.43:0/4152343762 conn(0x559a810730f0 msgr2=0x559a8109dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.077+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/4152343762 shutdown_connections 2024-09-11T23:29:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.077+0000 7f9e9ecdf740 1 -- 172.21.15.43:0/4152343762 wait complete. 2024-09-11T23:29:56.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.077+0000 7f9e9ecdf740 20 client.26231 trim_cache size 0 max 16384 2024-09-11T23:29:56.080 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:29:56.080 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-11T23:29:56.080 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "e88910bb-a6ec-4598-bbd7-6d3a8946792b", 2024-09-11T23:29:56.080 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-11T23:29:56.080 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:29:56.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.078+0000 7f9e9c906640 1 -- 172.21.15.43:0/1081699801 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e800a2c50 msgr2=0x7f9e800a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:56.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.078+0000 7f9e9c906640 1 --2- 172.21.15.43:0/1081699801 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e800a2c50 0x7f9e800a5110 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x559a81073f50 tx=0x7f9e880bb040 comp rx=0 tx=0).stop 2024-09-11T23:29:56.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.078+0000 7f9e9c906640 1 -- 172.21.15.43:0/1081699801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e98102d50 msgr2=0x7f9e98196260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.078+0000 7f9e9c906640 1 --2- 172.21.15.43:0/1081699801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e98102d50 0x7f9e98196260 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009be0 tx=0x7f9e880071c0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.078+0000 7f9e9c906640 1 -- 172.21.15.43:0/1081699801 shutdown_connections 2024-09-11T23:29:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.078+0000 7f9e9c906640 1 --2- 172.21.15.43:0/1081699801 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f9e800a2c50 0x7f9e800a5110 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.078+0000 7f9e9c906640 1 --2- 172.21.15.43:0/1081699801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e9810c180 0x7f9e9819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.078+0000 7f9e9c906640 1 --2- 172.21.15.43:0/1081699801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e98103670 0x7f9e981967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.078+0000 7f9e9c906640 1 --2- 172.21.15.43:0/1081699801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e98102d50 0x7f9e98196260 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.078+0000 7f9e9c906640 1 -- 172.21.15.43:0/1081699801 >> 172.21.15.43:0/1081699801 conn(0x7f9e980fc7d0 msgr2=0x7f9e980fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.078+0000 7f9e9c906640 1 -- 172.21.15.43:0/1081699801 shutdown_connections 2024-09-11T23:29:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:56.078+0000 7f9e9c906640 1 -- 172.21.15.43:0/1081699801 wait complete. 2024-09-11T23:29:56.779 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:29:57.115 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:29:57.523 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:57 smithi043 ceph-mon[30950]: scrub queued for path: / 2024-09-11T23:29:57.523 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:57 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:29:57.523 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:57 smithi043 ceph-mon[30950]: scrub summary: active paths [/] 2024-09-11T23:29:57.523 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:57 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:29:57.523 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:57 smithi043 ceph-mon[30950]: pgmap v249: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 47 KiB/s rd, 52 KiB/s wr, 72 op/s 2024-09-11T23:29:57.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.607+0000 7f10b8393640 1 -- 172.21.15.43:0/376179870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10b0102790 msgr2=0x7f10b01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:57.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.607+0000 7f10b8393640 1 --2- 172.21.15.43:0/376179870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10b0102790 0x7f10b01067e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009900 tx=0x7f10a002a410 comp rx=0 tx=0).stop 2024-09-11T23:29:57.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.607+0000 7f10b8393640 1 -- 172.21.15.43:0/376179870 shutdown_connections 2024-09-11T23:29:57.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.607+0000 7f10b8393640 1 --2- 172.21.15.43:0/376179870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10b0106f10 0x7f10b010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.607+0000 7f10b8393640 1 --2- 172.21.15.43:0/376179870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10b0102790 0x7f10b01067e0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.607+0000 7f10b8393640 1 --2- 172.21.15.43:0/376179870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10b0101de0 0x7f10b01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.607+0000 7f10b8393640 1 -- 172.21.15.43:0/376179870 >> 172.21.15.43:0/376179870 conn(0x7f10b00fd8c0 msgr2=0x7f10b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.607+0000 7f10b8393640 1 -- 172.21.15.43:0/376179870 shutdown_connections 2024-09-11T23:29:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.607+0000 7f10b8393640 1 -- 172.21.15.43:0/376179870 wait complete. 2024-09-11T23:29:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.608+0000 7f10b8393640 1 Processor -- start 2024-09-11T23:29:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.608+0000 7f10b8393640 1 -- start start 2024-09-11T23:29:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.608+0000 7f10b8393640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10b0101de0 0x7f10b00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.608+0000 7f10b8393640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10b0102790 0x7f10b0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.608+0000 7f10b8393640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10b0106f10 0x7f10b0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.608+0000 7f10b8393640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f10b0111f70 con 0x7f10b0101de0 2024-09-11T23:29:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.608+0000 7f10b8393640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f10b0111df0 con 0x7f10b0102790 2024-09-11T23:29:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.608+0000 7f10b8393640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01120f0 con 0x7f10b0106f10 2024-09-11T23:29:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.608+0000 7f10b612f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10b0101de0 0x7f10b00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.608+0000 7f10b612f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10b0101de0 0x7f10b00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57280/0 (socket says 172.21.15.43:57280) 2024-09-11T23:29:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.608+0000 7f10b612f640 1 -- 172.21.15.43:0/958062826 learned_addr learned my addr 172.21.15.43:0/958062826 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.608+0000 7f10b592e640 1 --2- 172.21.15.43:0/958062826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10b0102790 0x7f10b0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.608+0000 7f10b6930640 1 --2- 172.21.15.43:0/958062826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10b0106f10 0x7f10b0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.609+0000 7f10b612f640 1 -- 172.21.15.43:0/958062826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10b0106f10 msgr2=0x7f10b0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.609+0000 7f10b612f640 1 --2- 172.21.15.43:0/958062826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10b0106f10 0x7f10b0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.609+0000 7f10b612f640 1 -- 172.21.15.43:0/958062826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10b0102790 msgr2=0x7f10b0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.609+0000 7f10b612f640 1 --2- 172.21.15.43:0/958062826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10b0102790 0x7f10b0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.609+0000 7f10b612f640 1 -- 172.21.15.43:0/958062826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b01a6910 con 0x7f10b0101de0 2024-09-11T23:29:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.609+0000 7f10b612f640 1 --2- 172.21.15.43:0/958062826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10b0101de0 0x7f10b00744f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00d950 tx=0x7f10ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.609+0000 7f10a77fe640 1 -- 172.21.15.43:0/958062826 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac017070 con 0x7f10b0101de0 2024-09-11T23:29:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.609+0000 7f10a77fe640 1 -- 172.21.15.43:0/958062826 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10ac009c30 con 0x7f10b0101de0 2024-09-11T23:29:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.609+0000 7f10b8393640 1 -- 172.21.15.43:0/958062826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10b01a6c00 con 0x7f10b0101de0 2024-09-11T23:29:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.609+0000 7f10a77fe640 1 -- 172.21.15.43:0/958062826 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac00f6b0 con 0x7f10b0101de0 2024-09-11T23:29:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.609+0000 7f10b8393640 1 -- 172.21.15.43:0/958062826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10b01a7160 con 0x7f10b0101de0 2024-09-11T23:29:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.611+0000 7f10a77fe640 1 -- 172.21.15.43:0/958062826 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f10ac0109d0 con 0x7f10b0101de0 2024-09-11T23:29:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.612+0000 7f10ba76c740 1 Processor -- start 2024-09-11T23:29:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.612+0000 7f10ba76c740 1 -- start start 2024-09-11T23:29:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.612+0000 7f10ba76c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec89c4a0 0x5564ec89c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.612+0000 7f10ba76c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ec89cdc0 0x5564ec8a7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.612+0000 7f10ba76c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec8a7680 0x5564ec8a9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.612+0000 7f10ba76c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5564ec8022e0 con 0x5564ec89c4a0 2024-09-11T23:29:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.612+0000 7f10ba76c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5564ec7a9ce0 con 0x5564ec89cdc0 2024-09-11T23:29:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.612+0000 7f10ba76c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5564ec7fe2a0 con 0x5564ec8a7680 2024-09-11T23:29:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.612+0000 7f10b6930640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec89c4a0 0x5564ec89c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:57.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.612+0000 7f10b6930640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec89c4a0 0x5564ec89c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57286/0 (socket says 172.21.15.43:57286) 2024-09-11T23:29:57.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.612+0000 7f10b6930640 1 -- 172.21.15.43:0/2640437613 learned_addr learned my addr 172.21.15.43:0/2640437613 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:57.782 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:57 smithi090 ceph-mon[35130]: scrub queued for path: / 2024-09-11T23:29:57.782 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:57 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:29:57.782 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:57 smithi090 ceph-mon[35130]: scrub summary: active paths [/] 2024-09-11T23:29:57.782 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:57 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:29:57.782 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:57 smithi090 ceph-mon[35130]: pgmap v249: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 47 KiB/s rd, 52 KiB/s wr, 72 op/s 2024-09-11T23:29:57.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.612+0000 7f10b612f640 1 --2- 172.21.15.43:0/2640437613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ec89cdc0 0x5564ec8a7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:57.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.612+0000 7f10a77fe640 1 --2- 172.21.15.43:0/958062826 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f109c0a2d70 0x7f109c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:57.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.612+0000 7f10b592e640 1 --2- 172.21.15.43:0/2640437613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec8a7680 0x5564ec8a9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:57.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.613+0000 7f10b592e640 1 -- 172.21.15.43:0/2640437613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ec89cdc0 msgr2=0x5564ec8a7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:57.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.613+0000 7f10b592e640 1 --2- 172.21.15.43:0/2640437613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ec89cdc0 0x5564ec8a7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.613+0000 7f10b6930640 1 --2- 172.21.15.43:0/958062826 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f109c0a2d70 0x7f109c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.613+0000 7f10b592e640 1 -- 172.21.15.43:0/2640437613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec89c4a0 msgr2=0x5564ec89c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.613+0000 7f10b592e640 1 --2- 172.21.15.43:0/2640437613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec89c4a0 0x5564ec89c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.613+0000 7f10b592e640 1 -- 172.21.15.43:0/2640437613 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564ec78e440 con 0x5564ec8a7680 2024-09-11T23:29:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.613+0000 7f10a77fe640 1 -- 172.21.15.43:0/958062826 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f10ac0cefc0 con 0x7f10b0101de0 2024-09-11T23:29:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.613+0000 7f10b592e640 1 --2- 172.21.15.43:0/2640437613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec8a7680 0x5564ec8a9a70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f10a0005940 tx=0x7f10a0005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccdae0b9b84744bd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.613+0000 7f10b6930640 1 --2- 172.21.15.43:0/958062826 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f109c0a2d70 0x7f109c0a5230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f10a8005f40 tx=0x7f10a8004ab0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.613+0000 7f10a4dea640 1 -- 172.21.15.43:0/2640437613 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0040070 con 0x5564ec8a7680 2024-09-11T23:29:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.613+0000 7f10a4dea640 1 -- 172.21.15.43:0/2640437613 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10a00319f0 con 0x5564ec8a7680 2024-09-11T23:29:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.613+0000 7f10a4dea640 1 -- 172.21.15.43:0/2640437613 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0035040 con 0x5564ec8a7680 2024-09-11T23:29:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.614+0000 7f10ba76c740 1 -- 172.21.15.43:0/2640437613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec8a7680 msgr2=0x5564ec8a9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.614+0000 7f10ba76c740 1 --2- 172.21.15.43:0/2640437613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec8a7680 0x5564ec8a9a70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f10a0005940 tx=0x7f10a0005e80 comp rx=0 tx=0).stop 2024-09-11T23:29:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.614+0000 7f10ba76c740 1 -- 172.21.15.43:0/2640437613 shutdown_connections 2024-09-11T23:29:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.614+0000 7f10ba76c740 1 --2- 172.21.15.43:0/2640437613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec8a7680 0x5564ec8a9a70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.614+0000 7f10ba76c740 1 --2- 172.21.15.43:0/2640437613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ec89cdc0 0x5564ec8a7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.614+0000 7f10ba76c740 1 --2- 172.21.15.43:0/2640437613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec89c4a0 0x5564ec89c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.614+0000 7f10ba76c740 1 -- 172.21.15.43:0/2640437613 >> 172.21.15.43:0/2640437613 conn(0x5564ec898a30 msgr2=0x5564ec898e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.614+0000 7f10ba76c740 1 -- 172.21.15.43:0/2640437613 shutdown_connections 2024-09-11T23:29:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.614+0000 7f10ba76c740 1 -- 172.21.15.43:0/2640437613 wait complete. 2024-09-11T23:29:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.614+0000 7f10ba76c740 1 -- start start 2024-09-11T23:29:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.614+0000 7f10ba76c740 1 Processor -- start 2024-09-11T23:29:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.615+0000 7f10ba76c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec89c4a0 0x5564ec8a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:57.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.615+0000 7f10ba76c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec89cdc0 0x5564ec8a4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.615+0000 7f10ba76c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ec8a7680 0x5564ec8a4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.615+0000 7f10ba76c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5564ec8aa250 con 0x5564ec89cdc0 2024-09-11T23:29:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.615+0000 7f10ba76c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5564ec7bd050 con 0x5564ec8a7680 2024-09-11T23:29:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.615+0000 7f10ba76c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5564ec8aa3d0 con 0x5564ec89c4a0 2024-09-11T23:29:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.615+0000 7f10ba76c740 10 client.0 _ll_register_callbacks cb 0x5564ec7fd160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.615+0000 7f10b612f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec89c4a0 0x5564ec8a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.615+0000 7f10b592e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec89cdc0 0x5564ec8a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.615+0000 7f10b612f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec89c4a0 0x5564ec8a6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:43678/0 (socket says 172.21.15.43:43678) 2024-09-11T23:29:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.615+0000 7f10b612f640 1 -- 172.21.15.43:0/3711442184 learned_addr learned my addr 172.21.15.43:0/3711442184 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.615+0000 7f10b6930640 1 --2- 172.21.15.43:0/3711442184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ec8a7680 0x5564ec8a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.615+0000 7f10b612f640 1 -- 172.21.15.43:0/3711442184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ec8a7680 msgr2=0x5564ec8a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.615+0000 7f10b612f640 1 --2- 172.21.15.43:0/3711442184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ec8a7680 0x5564ec8a4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.615+0000 7f10b612f640 1 -- 172.21.15.43:0/3711442184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec89cdc0 msgr2=0x5564ec8a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.615+0000 7f10b612f640 1 --2- 172.21.15.43:0/3711442184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec89cdc0 0x5564ec8a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.615+0000 7f10b612f640 1 -- 172.21.15.43:0/3711442184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564ec5733c0 con 0x5564ec89c4a0 2024-09-11T23:29:57.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.615+0000 7f10b612f640 1 --2- 172.21.15.43:0/3711442184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec89c4a0 0x5564ec8a6f90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f10ac09b490 tx=0x7f10ac00bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=683c108bddd5ddfc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:57.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.615+0000 7f1099ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-11T23:29:57.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.616+0000 7f1099ffb640 1 -- 172.21.15.43:0/3711442184 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac020070 con 0x5564ec89c4a0 2024-09-11T23:29:57.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.616+0000 7f1099ffb640 1 -- 172.21.15.43:0/3711442184 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f10ac0a3c30 con 0x5564ec89c4a0 2024-09-11T23:29:57.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.616+0000 7f10ba76c740 1 -- 172.21.15.43:0/3711442184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10a00319f0 con 0x5564ec89c4a0 2024-09-11T23:29:57.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.616+0000 7f1099ffb640 1 -- 172.21.15.43:0/3711442184 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f10ac0ccb00 con 0x5564ec89c4a0 2024-09-11T23:29:57.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.616+0000 7f10ba76c740 10 client.26179 fetch_fsmap learned FSMap version 32 2024-09-11T23:29:57.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.616+0000 7f10ba76c740 1 -- 172.21.15.43:0/3711442184 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5564ec8b5c80 con 0x5564ec89c4a0 2024-09-11T23:29:57.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.616+0000 7f1099ffb640 1 -- 172.21.15.43:0/3711442184 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10a00319f0 con 0x5564ec89c4a0 2024-09-11T23:29:57.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.616+0000 7f1099ffb640 1 -- 172.21.15.43:0/3711442184 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f10ac0b4210 con 0x5564ec89c4a0 2024-09-11T23:29:57.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.616+0000 7f10ba76c740 10 client.26179 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:29:57.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.616+0000 7f10ba76c740 10 client.26179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:57.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.616+0000 7f10ba76c740 20 client.26179 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:57.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.616+0000 7f10ba76c740 1 --2- 172.21.15.43:0/3711442184 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5564ec8b7140 0x5564ec8b9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:57.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.616+0000 7f10ba76c740 4 client.26179 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:29:57.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.616+0000 7f10ba76c740 1 -- 172.21.15.43:0/3711442184 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564ec80f850 con 0x5564ec8b7140 2024-09-11T23:29:57.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.616+0000 7f10b592e640 1 --2- 172.21.15.43:0/3711442184 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5564ec8b7140 0x5564ec8b9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:57.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.617+0000 7f10b592e640 1 --2- 172.21.15.43:0/3711442184 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5564ec8b7140 0x5564ec8b9530 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35b3a763f497494a server_cookie=fcffea7296f673e5 in_seq=0 out_seq=0 2024-09-11T23:29:57.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.617+0000 7f1099ffb640 10 client.26179 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:57.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.617+0000 7f1099ffb640 1 -- 172.21.15.43:0/3711442184 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5564ec80f850 con 0x5564ec8b7140 2024-09-11T23:29:57.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.617+0000 7f1099ffb640 10 client.26179 handle_command_reply: tid=0 2024-09-11T23:29:57.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.617+0000 7f10ba76c740 1 client.26179 shutdown 2024-09-11T23:29:57.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.617+0000 7f10ba76c740 1 -- 172.21.15.43:0/3711442184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec89c4a0 msgr2=0x5564ec8a6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:57.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.617+0000 7f10ba76c740 1 --2- 172.21.15.43:0/3711442184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec89c4a0 0x5564ec8a6f90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f10ac09b490 tx=0x7f10ac00bf60 comp rx=0 tx=0).stop 2024-09-11T23:29:57.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.617+0000 7f10ba76c740 1 -- 172.21.15.43:0/3711442184 shutdown_connections 2024-09-11T23:29:57.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.617+0000 7f10ba76c740 1 --2- 172.21.15.43:0/3711442184 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5564ec8b7140 0x5564ec8b9530 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.617+0000 7f10ba76c740 1 --2- 172.21.15.43:0/3711442184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ec8a7680 0x5564ec8a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.617+0000 7f10ba76c740 1 --2- 172.21.15.43:0/3711442184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec89cdc0 0x5564ec8a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.617+0000 7f10ba76c740 1 --2- 172.21.15.43:0/3711442184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec89c4a0 0x5564ec8a6f90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.617+0000 7f1099ffb640 0 client.26179 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:57.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.617+0000 7f10ba76c740 1 -- 172.21.15.43:0/3711442184 >> 172.21.15.43:0/3711442184 conn(0x5564ec898a30 msgr2=0x5564ec89e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:57.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.617+0000 7f10ba76c740 1 -- 172.21.15.43:0/3711442184 shutdown_connections 2024-09-11T23:29:57.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.617+0000 7f10ba76c740 1 -- 172.21.15.43:0/3711442184 wait complete. 2024-09-11T23:29:57.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.618+0000 7f10ba76c740 20 client.26179 trim_cache size 0 max 16384 2024-09-11T23:29:57.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.635+0000 7f10ba76c740 1 Processor -- start 2024-09-11T23:29:57.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.635+0000 7f10ba76c740 1 -- start start 2024-09-11T23:29:57.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.636+0000 7f10ba76c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec898a30 0x5564ec8b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:57.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.636+0000 7f10ba76c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec89c4a0 0x5564ec8bece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:57.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.636+0000 7f10ba76c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ec89cdc0 0x5564ec8bf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:57.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.636+0000 7f10ba76c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01b23f0 con 0x5564ec898a30 2024-09-11T23:29:57.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.636+0000 7f10ba76c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01afc90 con 0x5564ec89cdc0 2024-09-11T23:29:57.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.636+0000 7f10ba76c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01adb00 con 0x5564ec89c4a0 2024-09-11T23:29:57.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.636+0000 7f10b612f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec898a30 0x5564ec8b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:57.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.636+0000 7f10b612f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec898a30 0x5564ec8b34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57312/0 (socket says 172.21.15.43:57312) 2024-09-11T23:29:57.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.636+0000 7f10b612f640 1 -- 172.21.15.43:0/2635255582 learned_addr learned my addr 172.21.15.43:0/2635255582 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:57.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.636+0000 7f10b6930640 1 --2- 172.21.15.43:0/2635255582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ec89cdc0 0x5564ec8bf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:57.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.636+0000 7f10b592e640 1 --2- 172.21.15.43:0/2635255582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec89c4a0 0x5564ec8bece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:57.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.636+0000 7f10b612f640 1 -- 172.21.15.43:0/2635255582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec89c4a0 msgr2=0x5564ec8bece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:57.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.636+0000 7f10b612f640 1 --2- 172.21.15.43:0/2635255582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec89c4a0 0x5564ec8bece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.636+0000 7f10b612f640 1 -- 172.21.15.43:0/2635255582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ec89cdc0 msgr2=0x5564ec8bf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:57.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.636+0000 7f10b612f640 1 --2- 172.21.15.43:0/2635255582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ec89cdc0 0x5564ec8bf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.636+0000 7f10b612f640 1 -- 172.21.15.43:0/2635255582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564ec89cbf0 con 0x5564ec898a30 2024-09-11T23:29:57.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.636+0000 7f10b6930640 1 --2- 172.21.15.43:0/2635255582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ec89cdc0 0x5564ec8bf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:29:57.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.636+0000 7f10b612f640 1 --2- 172.21.15.43:0/2635255582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec898a30 0x5564ec8b34b0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f10ac0a3b90 tx=0x7f10ac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f64995b8dc6f38 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:57.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.636+0000 7f109bfff640 1 -- 172.21.15.43:0/2635255582 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac09b040 con 0x5564ec898a30 2024-09-11T23:29:57.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.636+0000 7f109bfff640 1 -- 172.21.15.43:0/2635255582 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10ac09f800 con 0x5564ec898a30 2024-09-11T23:29:57.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.636+0000 7f109bfff640 1 -- 172.21.15.43:0/2635255582 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac0cc900 con 0x5564ec898a30 2024-09-11T23:29:57.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.637+0000 7f10ba76c740 1 -- 172.21.15.43:0/2635255582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec898a30 msgr2=0x5564ec8b34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:57.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.637+0000 7f10ba76c740 1 --2- 172.21.15.43:0/2635255582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec898a30 0x5564ec8b34b0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f10ac0a3b90 tx=0x7f10ac000bc0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.637+0000 7f10ba76c740 1 -- 172.21.15.43:0/2635255582 shutdown_connections 2024-09-11T23:29:57.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.637+0000 7f10ba76c740 1 --2- 172.21.15.43:0/2635255582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ec89cdc0 0x5564ec8bf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.637+0000 7f10ba76c740 1 --2- 172.21.15.43:0/2635255582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec89c4a0 0x5564ec8bece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.637+0000 7f10ba76c740 1 --2- 172.21.15.43:0/2635255582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec898a30 0x5564ec8b34b0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.637+0000 7f10ba76c740 1 -- 172.21.15.43:0/2635255582 >> 172.21.15.43:0/2635255582 conn(0x5564ec8a48e0 msgr2=0x5564ec8a2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:57.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.637+0000 7f10ba76c740 1 -- 172.21.15.43:0/2635255582 shutdown_connections 2024-09-11T23:29:57.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.637+0000 7f10ba76c740 1 -- 172.21.15.43:0/2635255582 wait complete. 2024-09-11T23:29:57.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.638+0000 7f10ba76c740 1 -- start start 2024-09-11T23:29:57.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.638+0000 7f10ba76c740 1 Processor -- start 2024-09-11T23:29:57.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.638+0000 7f10ba76c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec898a30 0x5564ec8ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:57.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.638+0000 7f10ba76c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec89c4a0 0x5564ec8ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:57.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.638+0000 7f10ba76c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ec89cdc0 0x5564ec8abf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:57.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.638+0000 7f10ba76c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01ae150 con 0x5564ec89c4a0 2024-09-11T23:29:57.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.638+0000 7f10ba76c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01af5c0 con 0x5564ec89cdc0 2024-09-11T23:29:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.638+0000 7f10ba76c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01ae820 con 0x5564ec898a30 2024-09-11T23:29:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.638+0000 7f10ba76c740 10 client.0 _ll_register_callbacks cb 0x7f10ac00df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:29:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.638+0000 7f10b592e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec89c4a0 0x5564ec8ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.638+0000 7f10b612f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec898a30 0x5564ec8ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.638+0000 7f10b592e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec89c4a0 0x5564ec8ab9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57316/0 (socket says 172.21.15.43:57316) 2024-09-11T23:29:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.638+0000 7f10b592e640 1 -- 172.21.15.43:0/1500739883 learned_addr learned my addr 172.21.15.43:0/1500739883 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:29:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.638+0000 7f10b6930640 1 --2- 172.21.15.43:0/1500739883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ec89cdc0 0x5564ec8abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.638+0000 7f10b592e640 1 -- 172.21.15.43:0/1500739883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec898a30 msgr2=0x5564ec8ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.638+0000 7f10b592e640 1 --2- 172.21.15.43:0/1500739883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec898a30 0x5564ec8ab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.638+0000 7f10b592e640 1 -- 172.21.15.43:0/1500739883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ec89cdc0 msgr2=0x5564ec8abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:29:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.638+0000 7f10b592e640 1 --2- 172.21.15.43:0/1500739883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ec89cdc0 0x5564ec8abf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.638+0000 7f10b592e640 1 -- 172.21.15.43:0/1500739883 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564ec920e10 con 0x5564ec89c4a0 2024-09-11T23:29:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.639+0000 7f10b592e640 1 --2- 172.21.15.43:0/1500739883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec89c4a0 0x5564ec8ab9f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009900 tx=0x7f10a0005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f506f5b463c23126 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:29:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.639+0000 7f1099ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:29:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.639+0000 7f1099ffb640 1 -- 172.21.15.43:0/1500739883 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0040040 con 0x5564ec89c4a0 2024-09-11T23:29:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.639+0000 7f1099ffb640 1 -- 172.21.15.43:0/1500739883 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8571+0+0 (secure 0 0 0) 0x7f10a0004250 con 0x5564ec89c4a0 2024-09-11T23:29:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.639+0000 7f10ba76c740 1 -- 172.21.15.43:0/1500739883 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10ac09f800 con 0x5564ec89c4a0 2024-09-11T23:29:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.639+0000 7f1099ffb640 1 -- 172.21.15.43:0/1500739883 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f10a005e050 con 0x5564ec89c4a0 2024-09-11T23:29:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.639+0000 7f10ba76c740 10 client.16761 fetch_fsmap learned FSMap version 32 2024-09-11T23:29:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.639+0000 7f10ba76c740 1 -- 172.21.15.43:0/1500739883 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5564ec922120 con 0x5564ec89c4a0 2024-09-11T23:29:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.639+0000 7f1099ffb640 1 -- 172.21.15.43:0/1500739883 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10ac09f800 con 0x5564ec89c4a0 2024-09-11T23:29:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.639+0000 7f1099ffb640 1 -- 172.21.15.43:0/1500739883 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f10a004e070 con 0x5564ec89c4a0 2024-09-11T23:29:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.639+0000 7f10ba76c740 10 client.16761 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:29:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.639+0000 7f10ba76c740 10 client.16761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:29:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.639+0000 7f10ba76c740 20 client.16761 populate_metadata read hostname 'smithi043' 2024-09-11T23:29:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.639+0000 7f10ba76c740 1 --2- 172.21.15.43:0/1500739883 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5564ec8a7680 0x5564ec924580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:29:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.639+0000 7f10ba76c740 4 client.16761 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:29:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.639+0000 7f10ba76c740 1 -- 172.21.15.43:0/1500739883 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5564ec809d60 con 0x5564ec8a7680 2024-09-11T23:29:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.640+0000 7f10b612f640 1 --2- 172.21.15.43:0/1500739883 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5564ec8a7680 0x5564ec924580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:29:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.640+0000 7f1099ffb640 10 client.16761 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.640+0000 7f10b612f640 1 --2- 172.21.15.43:0/1500739883 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5564ec8a7680 0x5564ec924580 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eb29d98464905e9 server_cookie=71c53182cb5815d0 in_seq=0 out_seq=0 2024-09-11T23:29:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.640+0000 7f1099ffb640 1 -- 172.21.15.43:0/1500739883 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5564ec809d60 con 0x5564ec8a7680 2024-09-11T23:29:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.640+0000 7f1099ffb640 10 client.16761 handle_command_reply: tid=0 2024-09-11T23:29:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.640+0000 7f10ba76c740 1 client.16761 shutdown 2024-09-11T23:29:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.640+0000 7f10ba76c740 1 -- 172.21.15.43:0/1500739883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec89c4a0 msgr2=0x5564ec8ab9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.640+0000 7f10ba76c740 1 --2- 172.21.15.43:0/1500739883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec89c4a0 0x5564ec8ab9f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009900 tx=0x7f10a0005a40 comp rx=0 tx=0).stop 2024-09-11T23:29:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.640+0000 7f10ba76c740 1 -- 172.21.15.43:0/1500739883 shutdown_connections 2024-09-11T23:29:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.640+0000 7f10ba76c740 1 --2- 172.21.15.43:0/1500739883 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5564ec8a7680 0x5564ec924580 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.640+0000 7f10ba76c740 1 --2- 172.21.15.43:0/1500739883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564ec89cdc0 0x5564ec8abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.640+0000 7f10ba76c740 1 --2- 172.21.15.43:0/1500739883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564ec89c4a0 0x5564ec8ab9f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.640+0000 7f10ba76c740 1 --2- 172.21.15.43:0/1500739883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564ec898a30 0x5564ec8ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.640+0000 7f10ba76c740 1 -- 172.21.15.43:0/1500739883 >> 172.21.15.43:0/1500739883 conn(0x5564ec8a48e0 msgr2=0x5564ec897350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.640+0000 7f1099ffb640 0 client.16761 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:29:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.640+0000 7f10ba76c740 1 -- 172.21.15.43:0/1500739883 shutdown_connections 2024-09-11T23:29:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.640+0000 7f10ba76c740 1 -- 172.21.15.43:0/1500739883 wait complete. 2024-09-11T23:29:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.641+0000 7f10ba76c740 20 client.16761 trim_cache size 0 max 16384 2024-09-11T23:29:57.801 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:29:57.801 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:29:57.801 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-09-11T23:29:57.801 INFO:teuthology.orchestra.run.smithi043.stdout: "e88910bb-a6ec-4598-bbd7-6d3a8946792b": { 2024-09-11T23:29:57.801 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-09-11T23:29:57.801 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "e88910bb-a6ec-4598-bbd7-6d3a8946792b", 2024-09-11T23:29:57.802 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-09-11T23:29:57.802 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:29:57.802 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:29:57.802 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:29:57.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.641+0000 7f10b8393640 1 -- 172.21.15.43:0/958062826 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f109c0a2d70 msgr2=0x7f109c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:57.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.641+0000 7f10b8393640 1 --2- 172.21.15.43:0/958062826 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f109c0a2d70 0x7f109c0a5230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f10a8005f40 tx=0x7f10a8004ab0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.641+0000 7f10b8393640 1 -- 172.21.15.43:0/958062826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10b0101de0 msgr2=0x7f10b00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:29:57.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.641+0000 7f10b8393640 1 --2- 172.21.15.43:0/958062826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10b0101de0 0x7f10b00744f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00d950 tx=0x7f10ac00de10 comp rx=0 tx=0).stop 2024-09-11T23:29:57.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.641+0000 7f10b8393640 1 -- 172.21.15.43:0/958062826 shutdown_connections 2024-09-11T23:29:57.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.641+0000 7f10b8393640 1 --2- 172.21.15.43:0/958062826 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f109c0a2d70 0x7f109c0a5230 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.641+0000 7f10b8393640 1 --2- 172.21.15.43:0/958062826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10b0106f10 0x7f10b0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.641+0000 7f10b8393640 1 --2- 172.21.15.43:0/958062826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10b0102790 0x7f10b0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.641+0000 7f10b8393640 1 --2- 172.21.15.43:0/958062826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10b0101de0 0x7f10b00744f0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:29:57.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.641+0000 7f10b8393640 1 -- 172.21.15.43:0/958062826 >> 172.21.15.43:0/958062826 conn(0x7f10b00fd8c0 msgr2=0x7f10b00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:29:57.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.641+0000 7f10b8393640 1 -- 172.21.15.43:0/958062826 shutdown_connections 2024-09-11T23:29:57.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:29:57.641+0000 7f10b8393640 1 -- 172.21.15.43:0/958062826 wait complete. 2024-09-11T23:29:57.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:57 smithi097 ceph-mon[35562]: scrub queued for path: / 2024-09-11T23:29:57.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:57 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:29:57.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:57 smithi097 ceph-mon[35562]: scrub summary: active paths [/] 2024-09-11T23:29:57.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:57 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:29:57.804 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:57 smithi097 ceph-mon[35562]: pgmap v249: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 47 KiB/s rd, 52 KiB/s wr, 72 op/s 2024-09-11T23:29:58.380 INFO:tasks.cephfs.filesystem:scrub status for tag:e88910bb-a6ec-4598-bbd7-6d3a8946792b - {'path': '/', 'tag': 'e88910bb-a6ec-4598-bbd7-6d3a8946792b', 'options': 'recursive,force'} 2024-09-11T23:29:59.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:29:59 smithi090 ceph-mon[35130]: pgmap v250: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 46 KiB/s rd, 52 KiB/s wr, 62 op/s 2024-09-11T23:29:59.800 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:29:59 smithi097 ceph-mon[35562]: pgmap v250: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 46 KiB/s rd, 52 KiB/s wr, 62 op/s 2024-09-11T23:29:59.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:29:59 smithi043 ceph-mon[30950]: pgmap v250: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 46 KiB/s rd, 52 KiB/s wr, 62 op/s 2024-09-11T23:30:00.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:00 smithi090 ceph-mon[35130]: scrub summary: idle 2024-09-11T23:30:00.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:00 smithi090 ceph-mon[35130]: overall HEALTH_OK 2024-09-11T23:30:00.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:00 smithi097 ceph-mon[35562]: scrub summary: idle 2024-09-11T23:30:00.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:00 smithi097 ceph-mon[35562]: overall HEALTH_OK 2024-09-11T23:30:00.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:00 smithi043 ceph-mon[30950]: scrub summary: idle 2024-09-11T23:30:00.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:00 smithi043 ceph-mon[30950]: overall HEALTH_OK 2024-09-11T23:30:01.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:01 smithi090 ceph-mon[35130]: pgmap v251: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 40 KiB/s rd, 44 KiB/s wr, 68 op/s 2024-09-11T23:30:01.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:01 smithi097 ceph-mon[35562]: pgmap v251: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 40 KiB/s rd, 44 KiB/s wr, 68 op/s 2024-09-11T23:30:01.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:01 smithi043 ceph-mon[30950]: pgmap v251: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 40 KiB/s rd, 44 KiB/s wr, 68 op/s 2024-09-11T23:30:03.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:03 smithi090 ceph-mon[35130]: pgmap v252: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 58 op/s 2024-09-11T23:30:03.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:03 smithi097 ceph-mon[35562]: pgmap v252: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 58 op/s 2024-09-11T23:30:03.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:03 smithi043 ceph-mon[30950]: pgmap v252: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 58 op/s 2024-09-11T23:30:05.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:05 smithi090 ceph-mon[35130]: pgmap v253: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 57 op/s 2024-09-11T23:30:05.800 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:05 smithi097 ceph-mon[35562]: pgmap v253: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 57 op/s 2024-09-11T23:30:05.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:05 smithi043 ceph-mon[30950]: pgmap v253: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 57 op/s 2024-09-11T23:30:07.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:07 smithi090 ceph-mon[35130]: pgmap v254: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 27 KiB/s rd, 62 KiB/s wr, 84 op/s 2024-09-11T23:30:07.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:07 smithi097 ceph-mon[35562]: pgmap v254: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 27 KiB/s rd, 62 KiB/s wr, 84 op/s 2024-09-11T23:30:07.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:07 smithi043 ceph-mon[30950]: pgmap v254: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 27 KiB/s rd, 62 KiB/s wr, 84 op/s 2024-09-11T23:30:09.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:09 smithi090 ceph-mon[35130]: pgmap v255: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 17 KiB/s rd, 47 KiB/s wr, 71 op/s 2024-09-11T23:30:09.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:09 smithi097 ceph-mon[35562]: pgmap v255: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 17 KiB/s rd, 47 KiB/s wr, 71 op/s 2024-09-11T23:30:09.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:09 smithi043 ceph-mon[30950]: pgmap v255: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 17 KiB/s rd, 47 KiB/s wr, 71 op/s 2024-09-11T23:30:11.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:11 smithi090 ceph-mon[35130]: pgmap v256: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 18 KiB/s rd, 47 KiB/s wr, 76 op/s 2024-09-11T23:30:11.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:11 smithi090 ceph-mon[35130]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T23:30:11.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:11 smithi097 ceph-mon[35562]: pgmap v256: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 18 KiB/s rd, 47 KiB/s wr, 76 op/s 2024-09-11T23:30:11.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:11 smithi097 ceph-mon[35562]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T23:30:11.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:11 smithi043 ceph-mon[30950]: pgmap v256: 97 pgs: 97 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 18 KiB/s rd, 47 KiB/s wr, 76 op/s 2024-09-11T23:30:11.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:11 smithi043 ceph-mon[30950]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T23:30:13.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:13 smithi090 ceph-mon[35130]: pgmap v258: 97 pgs: 4 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 84 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 70 op/s 2024-09-11T23:30:13.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:13 smithi097 ceph-mon[35562]: pgmap v258: 97 pgs: 4 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 84 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 70 op/s 2024-09-11T23:30:13.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:13 smithi043 ceph-mon[30950]: pgmap v258: 97 pgs: 4 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 84 active+clean; 22 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 70 op/s 2024-09-11T23:30:15.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:15 smithi090 ceph-mon[35130]: pgmap v259: 97 pgs: 6 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 76 active+clean; 21 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 65 op/s 2024-09-11T23:30:15.800 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:15 smithi097 ceph-mon[35562]: pgmap v259: 97 pgs: 6 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 76 active+clean; 21 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 65 op/s 2024-09-11T23:30:15.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:15 smithi043 ceph-mon[30950]: pgmap v259: 97 pgs: 6 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 76 active+clean; 21 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 65 op/s 2024-09-11T23:30:17.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:17 smithi090 ceph-mon[35130]: pgmap v260: 97 pgs: 16 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 50 active+clean; 19 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 409 B/s wr, 31 op/s 2024-09-11T23:30:17.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:17 smithi097 ceph-mon[35562]: pgmap v260: 97 pgs: 16 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 50 active+clean; 19 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 409 B/s wr, 31 op/s 2024-09-11T23:30:17.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:17 smithi043 ceph-mon[30950]: pgmap v260: 97 pgs: 16 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 50 active+clean; 19 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 409 B/s wr, 31 op/s 2024-09-11T23:30:19.769 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:19 smithi090 ceph-mon[35130]: pgmap v261: 97 pgs: 16 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 50 active+clean; 19 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 409 B/s wr, 19 op/s 2024-09-11T23:30:19.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:19 smithi097 ceph-mon[35562]: pgmap v261: 97 pgs: 16 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 50 active+clean; 19 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 409 B/s wr, 19 op/s 2024-09-11T23:30:19.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:19 smithi043 ceph-mon[30950]: pgmap v261: 97 pgs: 16 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 50 active+clean; 19 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 409 B/s wr, 19 op/s 2024-09-11T23:30:21.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:21 smithi097 ceph-mon[35562]: pgmap v262: 97 pgs: 15 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 52 active+clean; 18 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-11T23:30:21.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:21 smithi043 ceph-mon[30950]: pgmap v262: 97 pgs: 15 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 52 active+clean; 18 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-11T23:30:22.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:21 smithi090 ceph-mon[35130]: pgmap v262: 97 pgs: 15 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 52 active+clean; 18 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-11T23:30:23.801 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:23 smithi097 ceph-mon[35562]: pgmap v263: 97 pgs: 12 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 64 active+clean; 18 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 172 B/s wr, 6 op/s 2024-09-11T23:30:23.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:23 smithi043 ceph-mon[30950]: pgmap v263: 97 pgs: 12 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 64 active+clean; 18 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 172 B/s wr, 6 op/s 2024-09-11T23:30:24.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:23 smithi090 ceph-mon[35130]: pgmap v263: 97 pgs: 12 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 64 active+clean; 18 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 172 B/s wr, 6 op/s 2024-09-11T23:30:26.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:25 smithi090 ceph-mon[35130]: pgmap v264: 97 pgs: 12 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 64 active+clean; 18 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-11T23:30:26.300 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:25 smithi097 ceph-mon[35562]: pgmap v264: 97 pgs: 12 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 64 active+clean; 18 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-11T23:30:26.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:25 smithi043 ceph-mon[30950]: pgmap v264: 97 pgs: 12 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 64 active+clean; 18 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-11T23:30:28.163 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:27 smithi043 ceph-mon[30950]: pgmap v265: 97 pgs: 12 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 64 active+clean; 18 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-11T23:30:28.163 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:27 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:30:28.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:27 smithi090 ceph-mon[35130]: pgmap v265: 97 pgs: 12 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 64 active+clean; 18 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-11T23:30:28.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:27 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:30:28.300 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:27 smithi097 ceph-mon[35562]: pgmap v265: 97 pgs: 12 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 64 active+clean; 18 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-11T23:30:28.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:27 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:30:28.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:30:28.660 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:30:29.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:28 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:29.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:28 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:30:29.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:28 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:30:29.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:28 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:29.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:28 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:30:29.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:28 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:30:29.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:28 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:30:29.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:28 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:30:29.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:28 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:30:30.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:29 smithi090 ceph-mon[35130]: pgmap v266: 97 pgs: 12 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 64 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-11T23:30:30.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:29 smithi097 ceph-mon[35562]: pgmap v266: 97 pgs: 12 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 64 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-11T23:30:30.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:29 smithi043 ceph-mon[30950]: pgmap v266: 97 pgs: 12 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 64 active+clean; 17 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-11T23:30:30.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.660+0000 7f2b2674c640 1 -- 172.21.15.43:0/2817037541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b201011e0 msgr2=0x7f2b20105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:30.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.660+0000 7f2b2674c640 1 --2- 172.21.15.43:0/2817037541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b201011e0 0x7f2b20105980 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009900 tx=0x7f2b1002a410 comp rx=0 tx=0).stop 2024-09-11T23:30:30.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.664+0000 7f2b2674c640 1 -- 172.21.15.43:0/2817037541 shutdown_connections 2024-09-11T23:30:30.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.664+0000 7f2b2674c640 1 --2- 172.21.15.43:0/2817037541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b20105ec0 0x7f2b20106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.664+0000 7f2b2674c640 1 --2- 172.21.15.43:0/2817037541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b201011e0 0x7f2b20105980 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.664+0000 7f2b2674c640 1 --2- 172.21.15.43:0/2817037541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b201008c0 0x7f2b20100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.664+0000 7f2b2674c640 1 -- 172.21.15.43:0/2817037541 >> 172.21.15.43:0/2817037541 conn(0x7f2b200fc7d0 msgr2=0x7f2b200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:30.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.664+0000 7f2b2674c640 1 -- 172.21.15.43:0/2817037541 shutdown_connections 2024-09-11T23:30:30.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.664+0000 7f2b2674c640 1 -- 172.21.15.43:0/2817037541 wait complete. 2024-09-11T23:30:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.664+0000 7f2b2674c640 1 Processor -- start 2024-09-11T23:30:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.665+0000 7f2b2674c640 1 -- start start 2024-09-11T23:30:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.665+0000 7f2b2674c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b201008c0 0x7f2b201a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.665+0000 7f2b2674c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b201011e0 0x7f2b201a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.665+0000 7f2b2674c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b20105ec0 0x7f2b2019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.665+0000 7f2b2674c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b20118610 con 0x7f2b20105ec0 2024-09-11T23:30:30.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.665+0000 7f2b2674c640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2b20118490 con 0x7f2b201008c0 2024-09-11T23:30:30.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.665+0000 7f2b2674c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2b20118790 con 0x7f2b201011e0 2024-09-11T23:30:30.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.665+0000 7f2b1f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b201011e0 0x7f2b201a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:30.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.665+0000 7f2b1f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b201011e0 0x7f2b201a4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:44246/0 (socket says 172.21.15.43:44246) 2024-09-11T23:30:30.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.665+0000 7f2b1f7fe640 1 -- 172.21.15.43:0/1054737846 learned_addr learned my addr 172.21.15.43:0/1054737846 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:30.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.665+0000 7f2b1ffff640 1 --2- 172.21.15.43:0/1054737846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b201008c0 0x7f2b201a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:30.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.665+0000 7f2b24ce9640 1 --2- 172.21.15.43:0/1054737846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b20105ec0 0x7f2b2019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:30.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.666+0000 7f2b1f7fe640 1 -- 172.21.15.43:0/1054737846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b201008c0 msgr2=0x7f2b201a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.666+0000 7f2b1f7fe640 1 --2- 172.21.15.43:0/1054737846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b201008c0 0x7f2b201a4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.666+0000 7f2b1f7fe640 1 -- 172.21.15.43:0/1054737846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b20105ec0 msgr2=0x7f2b2019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.666+0000 7f2b1f7fe640 1 --2- 172.21.15.43:0/1054737846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b20105ec0 0x7f2b2019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.666+0000 7f2b1f7fe640 1 -- 172.21.15.43:0/1054737846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2019f070 con 0x7f2b201011e0 2024-09-11T23:30:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.666+0000 7f2b1f7fe640 1 --2- 172.21.15.43:0/1054737846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b201011e0 0x7f2b201a4cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2b10006010 tx=0x7f2b10003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.666+0000 7f2b1d7fa640 1 -- 172.21.15.43:0/1054737846 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1003f070 con 0x7f2b201011e0 2024-09-11T23:30:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.666+0000 7f2b2674c640 1 -- 172.21.15.43:0/1054737846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b2019f300 con 0x7f2b201011e0 2024-09-11T23:30:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.666+0000 7f2b2674c640 1 -- 172.21.15.43:0/1054737846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b20106ee0 con 0x7f2b201011e0 2024-09-11T23:30:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.666+0000 7f2b1d7fa640 1 -- 172.21.15.43:0/1054737846 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b10031c30 con 0x7f2b201011e0 2024-09-11T23:30:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.667+0000 7f2b1d7fa640 1 -- 172.21.15.43:0/1054737846 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10035030 con 0x7f2b201011e0 2024-09-11T23:30:30.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.670+0000 7f2b1d7fa640 1 -- 172.21.15.43:0/1054737846 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2b10042070 con 0x7f2b201011e0 2024-09-11T23:30:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.670+0000 7f2b28b25740 1 Processor -- start 2024-09-11T23:30:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.670+0000 7f2b28b25740 1 -- start start 2024-09-11T23:30:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.670+0000 7f2b28b25740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9840334a0 0x55d984033880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.670+0000 7f2b28b25740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d984033dc0 0x55d98403e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.670+0000 7f2b28b25740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d98403e680 0x55d984040a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.670+0000 7f2b28b25740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d983f992e0 con 0x55d9840334a0 2024-09-11T23:30:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.670+0000 7f2b28b25740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d983f40ce0 con 0x55d984033dc0 2024-09-11T23:30:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.670+0000 7f2b28b25740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d983f952a0 con 0x55d98403e680 2024-09-11T23:30:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.670+0000 7f2b24ce9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9840334a0 0x55d984033880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.671+0000 7f2b24ce9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9840334a0 0x55d984033880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36904/0 (socket says 172.21.15.43:36904) 2024-09-11T23:30:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.671+0000 7f2b24ce9640 1 -- 172.21.15.43:0/771915979 learned_addr learned my addr 172.21.15.43:0/771915979 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.671+0000 7f2b1f7fe640 1 --2- 172.21.15.43:0/771915979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d98403e680 0x55d984040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.671+0000 7f2b24ce9640 1 -- 172.21.15.43:0/771915979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d98403e680 msgr2=0x55d984040a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.671+0000 7f2b24ce9640 1 --2- 172.21.15.43:0/771915979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d98403e680 0x55d984040a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.671+0000 7f2b24ce9640 1 -- 172.21.15.43:0/771915979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d984033dc0 msgr2=0x55d98403e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.671+0000 7f2b24ce9640 1 --2- 172.21.15.43:0/771915979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d984033dc0 0x55d98403e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.671+0000 7f2b24ce9640 1 -- 172.21.15.43:0/771915979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d983f25440 con 0x55d9840334a0 2024-09-11T23:30:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.671+0000 7f2b24ce9640 1 --2- 172.21.15.43:0/771915979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9840334a0 0x55d984033880 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009f60 tx=0x7f2b18009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c3947b9f87ec6d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.672+0000 7f2b067fc640 1 -- 172.21.15.43:0/771915979 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1801e070 con 0x55d9840334a0 2024-09-11T23:30:30.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.672+0000 7f2b067fc640 1 -- 172.21.15.43:0/771915979 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b18004030 con 0x55d9840334a0 2024-09-11T23:30:30.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.672+0000 7f2b067fc640 1 -- 172.21.15.43:0/771915979 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800f720 con 0x55d9840334a0 2024-09-11T23:30:30.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.672+0000 7f2b1d7fa640 1 --2- 172.21.15.43:0/1054737846 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f2b080a2d70 0x7f2b080a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:30.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.672+0000 7f2b1d7fa640 1 -- 172.21.15.43:0/1054737846 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f2b100ef620 con 0x7f2b201011e0 2024-09-11T23:30:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.673+0000 7f2b28b25740 1 -- 172.21.15.43:0/771915979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9840334a0 msgr2=0x55d984033880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.673+0000 7f2b28b25740 1 --2- 172.21.15.43:0/771915979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9840334a0 0x55d984033880 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009f60 tx=0x7f2b18009f90 comp rx=0 tx=0).stop 2024-09-11T23:30:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.673+0000 7f2b28b25740 1 -- 172.21.15.43:0/771915979 shutdown_connections 2024-09-11T23:30:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.673+0000 7f2b28b25740 1 --2- 172.21.15.43:0/771915979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d98403e680 0x55d984040a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.673+0000 7f2b28b25740 1 --2- 172.21.15.43:0/771915979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d984033dc0 0x55d98403e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.673+0000 7f2b28b25740 1 --2- 172.21.15.43:0/771915979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d9840334a0 0x55d984033880 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.673+0000 7f2b28b25740 1 -- 172.21.15.43:0/771915979 >> 172.21.15.43:0/771915979 conn(0x55d98402fa30 msgr2=0x55d98402fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.673+0000 7f2b28b25740 1 -- 172.21.15.43:0/771915979 shutdown_connections 2024-09-11T23:30:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.673+0000 7f2b28b25740 1 -- 172.21.15.43:0/771915979 wait complete. 2024-09-11T23:30:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.674+0000 7f2b1ffff640 1 --2- 172.21.15.43:0/1054737846 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f2b080a2d70 0x7f2b080a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.674+0000 7f2b28b25740 1 -- start start 2024-09-11T23:30:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.674+0000 7f2b28b25740 1 Processor -- start 2024-09-11T23:30:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.674+0000 7f2b28b25740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9840334a0 0x55d98403e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.674+0000 7f2b28b25740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d984033dc0 0x55d98403b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.674+0000 7f2b28b25740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d98403e680 0x55d98403b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.675+0000 7f2b28b25740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d984041250 con 0x55d98403e680 2024-09-11T23:30:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.675+0000 7f2b28b25740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d983f54050 con 0x55d9840334a0 2024-09-11T23:30:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.675+0000 7f2b28b25740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d9840413d0 con 0x55d984033dc0 2024-09-11T23:30:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.675+0000 7f2b28b25740 10 client.0 _ll_register_callbacks cb 0x55d983f94160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:30:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.675+0000 7f2b24ce9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d98403e680 0x55d98403b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.675+0000 7f2b24ce9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d98403e680 0x55d98403b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36920/0 (socket says 172.21.15.43:36920) 2024-09-11T23:30:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.675+0000 7f2b24ce9640 1 -- 172.21.15.43:0/402435665 learned_addr learned my addr 172.21.15.43:0/402435665 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.675+0000 7f2b1ffff640 1 --2- 172.21.15.43:0/402435665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9840334a0 0x55d98403e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.676+0000 7f2b1f7fe640 1 --2- 172.21.15.43:0/402435665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d984033dc0 0x55d98403b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.676+0000 7f2b1ffff640 1 --2- 172.21.15.43:0/1054737846 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f2b080a2d70 0x7f2b080a5230 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55d984030890 tx=0x7f2b140046c0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.676+0000 7f2b24ce9640 1 -- 172.21.15.43:0/402435665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d984033dc0 msgr2=0x55d98403b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.676+0000 7f2b24ce9640 1 --2- 172.21.15.43:0/402435665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d984033dc0 0x55d98403b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.676+0000 7f2b24ce9640 1 -- 172.21.15.43:0/402435665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9840334a0 msgr2=0x55d98403e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.676+0000 7f2b24ce9640 1 --2- 172.21.15.43:0/402435665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9840334a0 0x55d98403e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.676+0000 7f2b24ce9640 1 -- 172.21.15.43:0/402435665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d983d0a3c0 con 0x55d98403e680 2024-09-11T23:30:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.677+0000 7f2aff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:30:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.676+0000 7f2b24ce9640 1 --2- 172.21.15.43:0/402435665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d98403e680 0x55d98403b830 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009f30 tx=0x7f2b18003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a17d07dba94216f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.677+0000 7f2aff7fe640 1 -- 172.21.15.43:0/402435665 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1801e040 con 0x55d98403e680 2024-09-11T23:30:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.677+0000 7f2aff7fe640 1 -- 172.21.15.43:0/402435665 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f2b18004a40 con 0x55d98403e680 2024-09-11T23:30:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.677+0000 7f2b28b25740 1 -- 172.21.15.43:0/402435665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b18004030 con 0x55d98403e680 2024-09-11T23:30:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.678+0000 7f2aff7fe640 1 -- 172.21.15.43:0/402435665 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2b18004c40 con 0x55d98403e680 2024-09-11T23:30:30.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.680+0000 7f2b28b25740 10 client.16773 fetch_fsmap learned FSMap version 32 2024-09-11T23:30:30.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.680+0000 7f2b28b25740 1 -- 172.21.15.43:0/402435665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d98404cca0 con 0x55d98403e680 2024-09-11T23:30:30.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.680+0000 7f2aff7fe640 1 -- 172.21.15.43:0/402435665 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b18004030 con 0x55d98403e680 2024-09-11T23:30:30.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.680+0000 7f2aff7fe640 1 -- 172.21.15.43:0/402435665 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2b18021070 con 0x55d98403e680 2024-09-11T23:30:30.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.680+0000 7f2b28b25740 10 client.16773 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:30:30.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.680+0000 7f2b28b25740 10 client.16773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:30:30.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.680+0000 7f2b28b25740 20 client.16773 populate_metadata read hostname 'smithi043' 2024-09-11T23:30:30.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.680+0000 7f2b28b25740 1 --2- 172.21.15.43:0/402435665 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55d98404e050 0x55d984050440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:30.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.680+0000 7f2b28b25740 4 client.16773 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:30:30.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.680+0000 7f2b28b25740 1 -- 172.21.15.43:0/402435665 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d983fa6850 con 0x55d98404e050 2024-09-11T23:30:30.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.681+0000 7f2b1ffff640 1 --2- 172.21.15.43:0/402435665 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55d98404e050 0x55d984050440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.681+0000 7f2b1ffff640 1 --2- 172.21.15.43:0/402435665 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55d98404e050 0x55d984050440 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1d28afe03ffe132 server_cookie=79d252d957330493 in_seq=0 out_seq=0 2024-09-11T23:30:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.681+0000 7f2aff7fe640 10 client.16773 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:30:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.682+0000 7f2aff7fe640 1 -- 172.21.15.43:0/402435665 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d983fa6850 con 0x55d98404e050 2024-09-11T23:30:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.682+0000 7f2aff7fe640 10 client.16773 handle_command_reply: tid=0 2024-09-11T23:30:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.682+0000 7f2b28b25740 1 client.16773 shutdown 2024-09-11T23:30:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.683+0000 7f2b28b25740 1 -- 172.21.15.43:0/402435665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d98403e680 msgr2=0x55d98403b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.683+0000 7f2b28b25740 1 --2- 172.21.15.43:0/402435665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d98403e680 0x55d98403b830 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009f30 tx=0x7f2b18003c60 comp rx=0 tx=0).stop 2024-09-11T23:30:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.683+0000 7f2b28b25740 1 -- 172.21.15.43:0/402435665 shutdown_connections 2024-09-11T23:30:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.683+0000 7f2b28b25740 1 --2- 172.21.15.43:0/402435665 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55d98404e050 0x55d984050440 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.683+0000 7f2b28b25740 1 --2- 172.21.15.43:0/402435665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d98403e680 0x55d98403b830 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.683+0000 7f2b28b25740 1 --2- 172.21.15.43:0/402435665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d984033dc0 0x55d98403b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.683+0000 7f2b28b25740 1 --2- 172.21.15.43:0/402435665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9840334a0 0x55d98403e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.683+0000 7f2b28b25740 1 -- 172.21.15.43:0/402435665 >> 172.21.15.43:0/402435665 conn(0x55d98402fa30 msgr2=0x55d9840357b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:30.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.683+0000 7f2aff7fe640 0 client.16773 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:30:30.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.683+0000 7f2b28b25740 1 -- 172.21.15.43:0/402435665 shutdown_connections 2024-09-11T23:30:30.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.683+0000 7f2b28b25740 1 -- 172.21.15.43:0/402435665 wait complete. 2024-09-11T23:30:30.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.683+0000 7f2b28b25740 20 client.16773 trim_cache size 0 max 16384 2024-09-11T23:30:30.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.702+0000 7f2b28b25740 1 Processor -- start 2024-09-11T23:30:30.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.702+0000 7f2b28b25740 1 -- start start 2024-09-11T23:30:30.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.702+0000 7f2b28b25740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9840334a0 0x55d98404a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:30.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.702+0000 7f2b28b25740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d984033dc0 0x55d98403d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:30.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.702+0000 7f2b28b25740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d98403e680 0x55d984055b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:30.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.702+0000 7f2b28b25740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c0112f0 con 0x55d98403e680 2024-09-11T23:30:30.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.702+0000 7f2b28b25740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c00eb90 con 0x55d984033dc0 2024-09-11T23:30:30.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.702+0000 7f2b28b25740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c00ca00 con 0x55d9840334a0 2024-09-11T23:30:30.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.702+0000 7f2b24ce9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d98403e680 0x55d984055b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:30.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.702+0000 7f2b1ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9840334a0 0x55d98404a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.702+0000 7f2b1f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d984033dc0 0x55d98403d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.702+0000 7f2b1ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9840334a0 0x55d98404a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:44276/0 (socket says 172.21.15.43:44276) 2024-09-11T23:30:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.702+0000 7f2b1ffff640 1 -- 172.21.15.43:0/3755537742 learned_addr learned my addr 172.21.15.43:0/3755537742 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.703+0000 7f2b24ce9640 1 -- 172.21.15.43:0/3755537742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9840334a0 msgr2=0x55d98404a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.703+0000 7f2b24ce9640 1 --2- 172.21.15.43:0/3755537742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9840334a0 0x55d98404a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.703+0000 7f2b24ce9640 1 -- 172.21.15.43:0/3755537742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d984033dc0 msgr2=0x55d98403d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.703+0000 7f2b24ce9640 1 --2- 172.21.15.43:0/3755537742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d984033dc0 0x55d98403d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.703+0000 7f2b24ce9640 1 -- 172.21.15.43:0/3755537742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d98403e370 con 0x55d98403e680 2024-09-11T23:30:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.703+0000 7f2b24ce9640 1 --2- 172.21.15.43:0/3755537742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d98403e680 0x55d984055b60 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800f570 tx=0x7f2b18009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e1ecb21d7033ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.703+0000 7f2b05ffb640 1 -- 172.21.15.43:0/3755537742 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1801e040 con 0x55d98403e680 2024-09-11T23:30:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.703+0000 7f2b05ffb640 1 -- 172.21.15.43:0/3755537742 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b18027070 con 0x55d98403e680 2024-09-11T23:30:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.703+0000 7f2b05ffb640 1 -- 172.21.15.43:0/3755537742 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800f5f0 con 0x55d98403e680 2024-09-11T23:30:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.704+0000 7f2b28b25740 1 -- 172.21.15.43:0/3755537742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d98403e680 msgr2=0x55d984055b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.704+0000 7f2b28b25740 1 --2- 172.21.15.43:0/3755537742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d98403e680 0x55d984055b60 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800f570 tx=0x7f2b18009f60 comp rx=0 tx=0).stop 2024-09-11T23:30:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.704+0000 7f2b28b25740 1 -- 172.21.15.43:0/3755537742 shutdown_connections 2024-09-11T23:30:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.704+0000 7f2b28b25740 1 --2- 172.21.15.43:0/3755537742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d98403e680 0x55d984055b60 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.704+0000 7f2b28b25740 1 --2- 172.21.15.43:0/3755537742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d984033dc0 0x55d98403d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.704+0000 7f2b28b25740 1 --2- 172.21.15.43:0/3755537742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9840334a0 0x55d98404a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.704+0000 7f2b28b25740 1 -- 172.21.15.43:0/3755537742 >> 172.21.15.43:0/3755537742 conn(0x55d98402fa30 msgr2=0x55d98403cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.704+0000 7f2b28b25740 1 -- 172.21.15.43:0/3755537742 shutdown_connections 2024-09-11T23:30:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.704+0000 7f2b28b25740 1 -- 172.21.15.43:0/3755537742 wait complete. 2024-09-11T23:30:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.704+0000 7f2b28b25740 1 -- start start 2024-09-11T23:30:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.704+0000 7f2b28b25740 1 Processor -- start 2024-09-11T23:30:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.705+0000 7f2b28b25740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9840334a0 0x55d984038270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.705+0000 7f2b28b25740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d984033dc0 0x55d9840387b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.705+0000 7f2b28b25740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d98403e680 0x55d984038cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.705+0000 7f2b28b25740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c00d050 con 0x55d984033dc0 2024-09-11T23:30:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.705+0000 7f2b28b25740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c00e4c0 con 0x55d98403e680 2024-09-11T23:30:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.705+0000 7f2b28b25740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c00d720 con 0x55d9840334a0 2024-09-11T23:30:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.705+0000 7f2b28b25740 10 client.0 _ll_register_callbacks cb 0x7f2b1801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:30:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.705+0000 7f2b1ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9840334a0 0x55d984038270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.705+0000 7f2b1f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d984033dc0 0x55d9840387b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.705+0000 7f2b1ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9840334a0 0x55d984038270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:44284/0 (socket says 172.21.15.43:44284) 2024-09-11T23:30:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.705+0000 7f2b1ffff640 1 -- 172.21.15.43:0/1301101274 learned_addr learned my addr 172.21.15.43:0/1301101274 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.705+0000 7f2b24ce9640 1 --2- 172.21.15.43:0/1301101274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d98403e680 0x55d984038cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.705+0000 7f2b1ffff640 1 -- 172.21.15.43:0/1301101274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d98403e680 msgr2=0x55d984038cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.705+0000 7f2b1ffff640 1 --2- 172.21.15.43:0/1301101274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d98403e680 0x55d984038cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.705+0000 7f2b1ffff640 1 -- 172.21.15.43:0/1301101274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d984033dc0 msgr2=0x55d9840387b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.705+0000 7f2b1ffff640 1 --2- 172.21.15.43:0/1301101274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d984033dc0 0x55d9840387b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.705+0000 7f2b1ffff640 1 -- 172.21.15.43:0/1301101274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9840b7b30 con 0x55d9840334a0 2024-09-11T23:30:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.705+0000 7f2b1f7fe640 1 --2- 172.21.15.43:0/1301101274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d984033dc0 0x55d9840387b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:30:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.706+0000 7f2b1ffff640 1 --2- 172.21.15.43:0/1301101274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9840334a0 0x55d984038270 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2b140114e0 tx=0x7f2b14013070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c5904e3ff3dddb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.706+0000 7f2aff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-11T23:30:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.706+0000 7f2aff7fe640 1 -- 172.21.15.43:0/1301101274 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1402a070 con 0x55d9840334a0 2024-09-11T23:30:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.706+0000 7f2b28b25740 1 -- 172.21.15.43:0/1301101274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b18027070 con 0x55d9840334a0 2024-09-11T23:30:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.706+0000 7f2aff7fe640 1 -- 172.21.15.43:0/1301101274 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f2b140266d0 con 0x55d9840334a0 2024-09-11T23:30:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.706+0000 7f2aff7fe640 1 -- 172.21.15.43:0/1301101274 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2b14033050 con 0x55d9840334a0 2024-09-11T23:30:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.706+0000 7f2b28b25740 10 client.26203 fetch_fsmap learned FSMap version 32 2024-09-11T23:30:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.706+0000 7f2b28b25740 1 -- 172.21.15.43:0/1301101274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d9840b8e20 con 0x55d9840334a0 2024-09-11T23:30:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.706+0000 7f2aff7fe640 1 -- 172.21.15.43:0/1301101274 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b18027070 con 0x55d9840334a0 2024-09-11T23:30:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.707+0000 7f2aff7fe640 1 -- 172.21.15.43:0/1301101274 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2b1402d020 con 0x55d9840334a0 2024-09-11T23:30:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.707+0000 7f2b28b25740 10 client.26203 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:30:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.707+0000 7f2b28b25740 10 client.26203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:30:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.707+0000 7f2b28b25740 20 client.26203 populate_metadata read hostname 'smithi043' 2024-09-11T23:30:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.707+0000 7f2b28b25740 1 --2- 172.21.15.43:0/1301101274 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55d9840b9570 0x55d9840bb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.707+0000 7f2b28b25740 4 client.26203 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:30:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.707+0000 7f2b28b25740 1 -- 172.21.15.43:0/1301101274 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d983fa0d60 con 0x55d9840b9570 2024-09-11T23:30:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.707+0000 7f2b1f7fe640 1 --2- 172.21.15.43:0/1301101274 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55d9840b9570 0x55d9840bb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.707+0000 7f2b1f7fe640 1 --2- 172.21.15.43:0/1301101274 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55d9840b9570 0x55d9840bb990 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faebdda9ba9fe4a9 server_cookie=ea4499ce528ca1fb in_seq=0 out_seq=0 2024-09-11T23:30:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.707+0000 7f2aff7fe640 10 client.26203 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:30:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.707+0000 7f2aff7fe640 1 -- 172.21.15.43:0/1301101274 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d983fa0d60 con 0x55d9840b9570 2024-09-11T23:30:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.707+0000 7f2aff7fe640 10 client.26203 handle_command_reply: tid=0 2024-09-11T23:30:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.707+0000 7f2b28b25740 1 client.26203 shutdown 2024-09-11T23:30:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.708+0000 7f2b28b25740 1 -- 172.21.15.43:0/1301101274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9840334a0 msgr2=0x55d984038270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.708+0000 7f2b28b25740 1 --2- 172.21.15.43:0/1301101274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9840334a0 0x55d984038270 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2b140114e0 tx=0x7f2b14013070 comp rx=0 tx=0).stop 2024-09-11T23:30:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.708+0000 7f2b28b25740 1 -- 172.21.15.43:0/1301101274 shutdown_connections 2024-09-11T23:30:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.708+0000 7f2b28b25740 1 --2- 172.21.15.43:0/1301101274 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55d9840b9570 0x55d9840bb990 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.708+0000 7f2b28b25740 1 --2- 172.21.15.43:0/1301101274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d98403e680 0x55d984038cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.708+0000 7f2b28b25740 1 --2- 172.21.15.43:0/1301101274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d984033dc0 0x55d9840387b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.708+0000 7f2aff7fe640 0 client.26203 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:30:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.708+0000 7f2b28b25740 1 --2- 172.21.15.43:0/1301101274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9840334a0 0x55d984038270 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.708+0000 7f2b28b25740 1 -- 172.21.15.43:0/1301101274 >> 172.21.15.43:0/1301101274 conn(0x55d98402fa30 msgr2=0x55d98402e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:30.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.708+0000 7f2b28b25740 1 -- 172.21.15.43:0/1301101274 shutdown_connections 2024-09-11T23:30:30.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.708+0000 7f2b28b25740 1 -- 172.21.15.43:0/1301101274 wait complete. 2024-09-11T23:30:30.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.708+0000 7f2b28b25740 20 client.26203 trim_cache size 0 max 16384 2024-09-11T23:30:30.714 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:30:30.714 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-11T23:30:30.714 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-11T23:30:30.714 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:30:30.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.708+0000 7f2b2674c640 1 -- 172.21.15.43:0/1054737846 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f2b080a2d70 msgr2=0x7f2b080a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:30.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.708+0000 7f2b2674c640 1 --2- 172.21.15.43:0/1054737846 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f2b080a2d70 0x7f2b080a5230 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55d984030890 tx=0x7f2b140046c0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.708+0000 7f2b2674c640 1 -- 172.21.15.43:0/1054737846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b201011e0 msgr2=0x7f2b201a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:30.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.708+0000 7f2b2674c640 1 --2- 172.21.15.43:0/1054737846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b201011e0 0x7f2b201a4cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2b10006010 tx=0x7f2b10003c10 comp rx=0 tx=0).stop 2024-09-11T23:30:30.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.708+0000 7f2b2674c640 1 -- 172.21.15.43:0/1054737846 shutdown_connections 2024-09-11T23:30:30.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.708+0000 7f2b2674c640 1 --2- 172.21.15.43:0/1054737846 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f2b080a2d70 0x7f2b080a5230 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.708+0000 7f2b2674c640 1 --2- 172.21.15.43:0/1054737846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b20105ec0 0x7f2b2019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.708+0000 7f2b2674c640 1 --2- 172.21.15.43:0/1054737846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b201011e0 0x7f2b201a4cc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.708+0000 7f2b2674c640 1 --2- 172.21.15.43:0/1054737846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b201008c0 0x7f2b201a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:30.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.709+0000 7f2b2674c640 1 -- 172.21.15.43:0/1054737846 >> 172.21.15.43:0/1054737846 conn(0x7f2b200fc7d0 msgr2=0x7f2b20103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:30.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.709+0000 7f2b2674c640 1 -- 172.21.15.43:0/1054737846 shutdown_connections 2024-09-11T23:30:30.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:30.709+0000 7f2b2674c640 1 -- 172.21.15.43:0/1054737846 wait complete. 2024-09-11T23:30:31.949 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:30:31.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:30:32.220 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:31 smithi043 ceph-mon[30950]: pgmap v267: 97 pgs: 9 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 72 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-11T23:30:32.251 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:30:32.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:31 smithi090 ceph-mon[35130]: pgmap v267: 97 pgs: 9 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 72 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-11T23:30:32.300 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:31 smithi097 ceph-mon[35562]: pgmap v267: 97 pgs: 9 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 72 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-11T23:30:32.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.869+0000 7f45002ff640 1 -- 172.21.15.43:0/3661793205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f8100e20 msgr2=0x7f44f8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:32.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.869+0000 7f45002ff640 1 --2- 172.21.15.43:0/3661793205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f8100e20 0x7f44f8101200 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009900 tx=0x7f44e802a480 comp rx=0 tx=0).stop 2024-09-11T23:30:32.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.870+0000 7f45002ff640 1 -- 172.21.15.43:0/3661793205 shutdown_connections 2024-09-11T23:30:32.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.870+0000 7f45002ff640 1 --2- 172.21.15.43:0/3661793205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44f8102ef0 0x7f44f810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:32.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.870+0000 7f45002ff640 1 --2- 172.21.15.43:0/3661793205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44f8102570 0x7f44f81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:32.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.870+0000 7f45002ff640 1 --2- 172.21.15.43:0/3661793205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f8100e20 0x7f44f8101200 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:32.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.870+0000 7f45002ff640 1 -- 172.21.15.43:0/3661793205 >> 172.21.15.43:0/3661793205 conn(0x7f44f80fc7d0 msgr2=0x7f44f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:32.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.870+0000 7f45002ff640 1 -- 172.21.15.43:0/3661793205 shutdown_connections 2024-09-11T23:30:32.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.870+0000 7f45002ff640 1 -- 172.21.15.43:0/3661793205 wait complete. 2024-09-11T23:30:32.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.871+0000 7f45002ff640 1 Processor -- start 2024-09-11T23:30:32.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.871+0000 7f45002ff640 1 -- start start 2024-09-11T23:30:32.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.871+0000 7f45002ff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f8100e20 0x7f44f8073950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:32.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.871+0000 7f45002ff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44f8102570 0x7f44f8073e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.871+0000 7f45002ff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44f8102ef0 0x7f44f806cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.871+0000 7f45002ff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f44f810fdc0 con 0x7f44f8100e20 2024-09-11T23:30:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.871+0000 7f45002ff640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f44f810fc40 con 0x7f44f8102570 2024-09-11T23:30:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.871+0000 7f45002ff640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f44f810ff40 con 0x7f44f8102ef0 2024-09-11T23:30:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.872+0000 7f44fe89c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44f8102ef0 0x7f44f806cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.872+0000 7f44fe89c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44f8102ef0 0x7f44f806cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:44308/0 (socket says 172.21.15.43:44308) 2024-09-11T23:30:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.872+0000 7f44fe89c640 1 -- 172.21.15.43:0/973211823 learned_addr learned my addr 172.21.15.43:0/973211823 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.872+0000 7f44fe89c640 1 -- 172.21.15.43:0/973211823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44f8102570 msgr2=0x7f44f8073e90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:30:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.872+0000 7f44fe89c640 1 --2- 172.21.15.43:0/973211823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44f8102570 0x7f44f8073e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.872+0000 7f44fe89c640 1 -- 172.21.15.43:0/973211823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f8100e20 msgr2=0x7f44f8073950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.872+0000 7f44fe89c640 1 --2- 172.21.15.43:0/973211823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f8100e20 0x7f44f8073950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.872+0000 7f44fe89c640 1 -- 172.21.15.43:0/973211823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f806d430 con 0x7f44f8102ef0 2024-09-11T23:30:32.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.873+0000 7f44fe89c640 1 --2- 172.21.15.43:0/973211823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44f8102ef0 0x7f44f806cb70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f44f0009fd0 tx=0x7f44f000b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:32.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.873+0000 7f44ef7fe640 1 -- 172.21.15.43:0/973211823 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0017070 con 0x7f44f8102ef0 2024-09-11T23:30:32.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.874+0000 7f45002ff640 1 -- 172.21.15.43:0/973211823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f806d720 con 0x7f44f8102ef0 2024-09-11T23:30:32.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.874+0000 7f45002ff640 1 -- 172.21.15.43:0/973211823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f8071700 con 0x7f44f8102ef0 2024-09-11T23:30:32.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.874+0000 7f44ef7fe640 1 -- 172.21.15.43:0/973211823 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f0003e50 con 0x7f44f8102ef0 2024-09-11T23:30:32.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.875+0000 7f44ef7fe640 1 -- 172.21.15.43:0/973211823 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f00106c0 con 0x7f44f8102ef0 2024-09-11T23:30:32.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.875+0000 7f45002ff640 1 -- 172.21.15.43:0/973211823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44dc005290 con 0x7f44f8102ef0 2024-09-11T23:30:32.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.876+0000 7f44ef7fe640 1 -- 172.21.15.43:0/973211823 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f44f0003970 con 0x7f44f8102ef0 2024-09-11T23:30:32.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.877+0000 7f44ef7fe640 1 --2- 172.21.15.43:0/973211823 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f44e40a2dc0 0x7f44e40a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:32.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.877+0000 7f44ef7fe640 1 -- 172.21.15.43:0/973211823 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f44f00cf160 con 0x7f44f8102ef0 2024-09-11T23:30:32.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.877+0000 7f44fe09b640 1 --2- 172.21.15.43:0/973211823 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f44e40a2dc0 0x7f44e40a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:32.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.878+0000 7f44fe09b640 1 --2- 172.21.15.43:0/973211823 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f44e40a2dc0 0x7f44e40a5280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f44e8005c20 tx=0x7f44e8000bc0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:32.880+0000 7f44ef7fe640 1 -- 172.21.15.43:0/973211823 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44f009bd40 con 0x7f44f8102ef0 2024-09-11T23:30:33.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:33.192+0000 7f45002ff640 1 -- 172.21.15.43:0/973211823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44dc005580 con 0x7f44f8102ef0 2024-09-11T23:30:33.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:33.192+0000 7f44ef7fe640 1 -- 172.21.15.43:0/973211823 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11895 (secure 0 0 0) 0x7f44f0097020 con 0x7f44f8102ef0 2024-09-11T23:30:33.193 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:30:33.193 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":32,"btime":"2024-09-11T23:29:51:458202+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:29:50.490489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25571":{"gid":25571,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/568491708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":568491708},{"type":"v1","addr":"172.21.15.90:6835","nonce":568491708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:30:33.193 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 32 2024-09-11T23:30:33.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:33.195+0000 7f45002ff640 1 -- 172.21.15.43:0/973211823 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f44e40a2dc0 msgr2=0x7f44e40a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:33.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:33.195+0000 7f45002ff640 1 --2- 172.21.15.43:0/973211823 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f44e40a2dc0 0x7f44e40a5280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f44e8005c20 tx=0x7f44e8000bc0 comp rx=0 tx=0).stop 2024-09-11T23:30:33.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:33.195+0000 7f45002ff640 1 -- 172.21.15.43:0/973211823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44f8102ef0 msgr2=0x7f44f806cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:33.195+0000 7f45002ff640 1 --2- 172.21.15.43:0/973211823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44f8102ef0 0x7f44f806cb70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f44f0009fd0 tx=0x7f44f000b260 comp rx=0 tx=0).stop 2024-09-11T23:30:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:33.195+0000 7f45002ff640 1 -- 172.21.15.43:0/973211823 shutdown_connections 2024-09-11T23:30:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:33.195+0000 7f45002ff640 1 --2- 172.21.15.43:0/973211823 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f44e40a2dc0 0x7f44e40a5280 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:33.195+0000 7f45002ff640 1 --2- 172.21.15.43:0/973211823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44f8102ef0 0x7f44f806cb70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:33.195+0000 7f45002ff640 1 --2- 172.21.15.43:0/973211823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44f8102570 0x7f44f8073e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:33.195+0000 7f45002ff640 1 --2- 172.21.15.43:0/973211823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f8100e20 0x7f44f8073950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:33.195+0000 7f45002ff640 1 -- 172.21.15.43:0/973211823 >> 172.21.15.43:0/973211823 conn(0x7f44f80fc7d0 msgr2=0x7f44f80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:33.195+0000 7f45002ff640 1 -- 172.21.15.43:0/973211823 shutdown_connections 2024-09-11T23:30:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:33.195+0000 7f45002ff640 1 -- 172.21.15.43:0/973211823 wait complete. 2024-09-11T23:30:34.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:33 smithi043 ceph-mon[30950]: pgmap v268: 97 pgs: 8 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 73 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:30:34.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:33 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/973211823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:30:34.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:33 smithi090 ceph-mon[35130]: pgmap v268: 97 pgs: 8 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 73 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:30:34.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:33 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/973211823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:30:34.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:33 smithi097 ceph-mon[35562]: pgmap v268: 97 pgs: 8 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 73 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:30:34.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:33 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/973211823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:30:35.679 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:30:35.951 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:30:36.073 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:35 smithi043 ceph-mon[30950]: pgmap v269: 97 pgs: 7 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 80 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:30:36.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:35 smithi090 ceph-mon[35130]: pgmap v269: 97 pgs: 7 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 80 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:30:36.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:35 smithi097 ceph-mon[35562]: pgmap v269: 97 pgs: 7 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 80 active+clean; 17 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:30:36.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.442+0000 7fcf0ad2a640 1 -- 172.21.15.43:0/1289206697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf04106f10 msgr2=0x7fcf0410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:36.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.442+0000 7fcf0ad2a640 1 --2- 172.21.15.43:0/1289206697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf04106f10 0x7fcf0410ea90 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009ca0 tx=0x7fcefc02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.442+0000 7fcf0ad2a640 1 -- 172.21.15.43:0/1289206697 shutdown_connections 2024-09-11T23:30:36.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.442+0000 7fcf0ad2a640 1 --2- 172.21.15.43:0/1289206697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf04106f10 0x7fcf0410ea90 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.442+0000 7fcf0ad2a640 1 --2- 172.21.15.43:0/1289206697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf04102790 0x7fcf041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.442+0000 7fcf0ad2a640 1 --2- 172.21.15.43:0/1289206697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf04101de0 0x7fcf041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.442+0000 7fcf0ad2a640 1 -- 172.21.15.43:0/1289206697 >> 172.21.15.43:0/1289206697 conn(0x7fcf040fd8c0 msgr2=0x7fcf040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:36.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.442+0000 7fcf0ad2a640 1 -- 172.21.15.43:0/1289206697 shutdown_connections 2024-09-11T23:30:36.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.442+0000 7fcf0ad2a640 1 -- 172.21.15.43:0/1289206697 wait complete. 2024-09-11T23:30:36.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.443+0000 7fcf0ad2a640 1 Processor -- start 2024-09-11T23:30:36.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.443+0000 7fcf0ad2a640 1 -- start start 2024-09-11T23:30:36.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.443+0000 7fcf0ad2a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf04101de0 0x7fcf040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:36.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.443+0000 7fcf0ad2a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf04102790 0x7fcf04074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:36.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.443+0000 7fcf0ad2a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf04106f10 0x7fcf04074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:36.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.443+0000 7fcf0ad2a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcf04111f70 con 0x7fcf04101de0 2024-09-11T23:30:36.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.443+0000 7fcf0ad2a640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcf04111df0 con 0x7fcf04102790 2024-09-11T23:30:36.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.443+0000 7fcf0ad2a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcf041120f0 con 0x7fcf04106f10 2024-09-11T23:30:36.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.443+0000 7fcf08ac6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf04101de0 0x7fcf040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:36.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.443+0000 7fcf08ac6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf04101de0 0x7fcf040744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36952/0 (socket says 172.21.15.43:36952) 2024-09-11T23:30:36.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.443+0000 7fcf08ac6640 1 -- 172.21.15.43:0/277127717 learned_addr learned my addr 172.21.15.43:0/277127717 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:36.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.443+0000 7fcf092c7640 1 --2- 172.21.15.43:0/277127717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf04106f10 0x7fcf04074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:36.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.443+0000 7fcf03fff640 1 --2- 172.21.15.43:0/277127717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf04102790 0x7fcf04074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:36.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.444+0000 7fcf08ac6640 1 -- 172.21.15.43:0/277127717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf04106f10 msgr2=0x7fcf04074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:36.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.444+0000 7fcf08ac6640 1 --2- 172.21.15.43:0/277127717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf04106f10 0x7fcf04074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.444+0000 7fcf08ac6640 1 -- 172.21.15.43:0/277127717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf04102790 msgr2=0x7fcf04074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:36.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.444+0000 7fcf08ac6640 1 --2- 172.21.15.43:0/277127717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf04102790 0x7fcf04074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.444+0000 7fcf08ac6640 1 -- 172.21.15.43:0/277127717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf041a6890 con 0x7fcf04101de0 2024-09-11T23:30:36.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.444+0000 7fcf03fff640 1 --2- 172.21.15.43:0/277127717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf04102790 0x7fcf04074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:30:36.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.444+0000 7fcf08ac6640 1 --2- 172.21.15.43:0/277127717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf04101de0 0x7fcf040744f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fcef800d770 tx=0x7fcef800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:36.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.444+0000 7fcf01ffb640 1 -- 172.21.15.43:0/277127717 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8017070 con 0x7fcf04101de0 2024-09-11T23:30:36.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.444+0000 7fcf01ffb640 1 -- 172.21.15.43:0/277127717 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef8005500 con 0x7fcf04101de0 2024-09-11T23:30:36.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.444+0000 7fcf01ffb640 1 -- 172.21.15.43:0/277127717 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8004e00 con 0x7fcf04101de0 2024-09-11T23:30:36.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.444+0000 7fcf0ad2a640 1 -- 172.21.15.43:0/277127717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf041a6b80 con 0x7fcf04101de0 2024-09-11T23:30:36.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.444+0000 7fcf0ad2a640 1 -- 172.21.15.43:0/277127717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf041a7070 con 0x7fcf04101de0 2024-09-11T23:30:36.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.446+0000 7fcf01ffb640 1 -- 172.21.15.43:0/277127717 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fcef8004fa0 con 0x7fcf04101de0 2024-09-11T23:30:36.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.447+0000 7fcf0d103740 1 Processor -- start 2024-09-11T23:30:36.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.447+0000 7fcf0d103740 1 -- start start 2024-09-11T23:30:36.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.447+0000 7fcf0d103740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccfc05eb40 0x55ccfc05d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:36.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.447+0000 7fcf01ffb640 1 --2- 172.21.15.43:0/277127717 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fcef40a2cf0 0x7fcef40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:36.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.447+0000 7fcf0d103740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc05d5d0 0x55ccfc05da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:36.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.447+0000 7fcf0d103740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc068370 0x55ccfc06a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:36.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.447+0000 7fcf0d103740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ccfbfc32f0 con 0x55ccfc068370 2024-09-11T23:30:36.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.447+0000 7fcf0d103740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ccfbf6acf0 con 0x55ccfc05d5d0 2024-09-11T23:30:36.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.447+0000 7fcf0d103740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ccfbfbf2b0 con 0x55ccfc05eb40 2024-09-11T23:30:36.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.447+0000 7fcf01ffb640 1 -- 172.21.15.43:0/277127717 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7fcef80cef90 con 0x7fcf04101de0 2024-09-11T23:30:36.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.447+0000 7fcf092c7640 1 --2- 172.21.15.43:0/277127717 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fcef40a2cf0 0x7fcef40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:36.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.447+0000 7fcf03fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc05d5d0 0x55ccfc05da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:36.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.447+0000 7fcf03fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc05d5d0 0x55ccfc05da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:60500/0 (socket says 172.21.15.43:60500) 2024-09-11T23:30:36.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.447+0000 7fcf03fff640 1 -- 172.21.15.43:0/401387966 learned_addr learned my addr 172.21.15.43:0/401387966 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:36.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.447+0000 7fcf092c7640 1 --2- 172.21.15.43:0/401387966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc068370 0x55ccfc06a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:36.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.447+0000 7fcf08ac6640 1 --2- 172.21.15.43:0/401387966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccfc05eb40 0x55ccfc05d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:36.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.448+0000 7fcf03fff640 1 -- 172.21.15.43:0/401387966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccfc05eb40 msgr2=0x55ccfc05d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:36.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.448+0000 7fcf03fff640 1 --2- 172.21.15.43:0/401387966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccfc05eb40 0x55ccfc05d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.448+0000 7fcf03fff640 1 -- 172.21.15.43:0/401387966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc068370 msgr2=0x55ccfc06a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:36.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.448+0000 7fcf03fff640 1 --2- 172.21.15.43:0/401387966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc068370 0x55ccfc06a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.448+0000 7fcf03fff640 1 -- 172.21.15.43:0/401387966 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccfbf4f3a0 con 0x55ccfc05d5d0 2024-09-11T23:30:36.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.448+0000 7fcf092c7640 1 --2- 172.21.15.43:0/277127717 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fcef40a2cf0 0x7fcef40a51b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcefc004af0 tx=0x7fcefc02f4b0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:36.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.448+0000 7fcf03fff640 1 --2- 172.21.15.43:0/401387966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc05d5d0 0x55ccfc05da10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcef000bcb0 tx=0x7fcef000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bda7a5729c227bf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:36.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.448+0000 7fceeeffd640 1 -- 172.21.15.43:0/401387966 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef0015070 con 0x55ccfc05d5d0 2024-09-11T23:30:36.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.448+0000 7fceeeffd640 1 -- 172.21.15.43:0/401387966 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef0011920 con 0x55ccfc05d5d0 2024-09-11T23:30:36.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.448+0000 7fceeeffd640 1 -- 172.21.15.43:0/401387966 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef0010840 con 0x55ccfc05d5d0 2024-09-11T23:30:36.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.449+0000 7fcf0d103740 1 -- 172.21.15.43:0/401387966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc05d5d0 msgr2=0x55ccfc05da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:36.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.449+0000 7fcf0d103740 1 --2- 172.21.15.43:0/401387966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc05d5d0 0x55ccfc05da10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcef000bcb0 tx=0x7fcef000f730 comp rx=0 tx=0).stop 2024-09-11T23:30:36.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.449+0000 7fcf0d103740 1 -- 172.21.15.43:0/401387966 shutdown_connections 2024-09-11T23:30:36.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.449+0000 7fcf0d103740 1 --2- 172.21.15.43:0/401387966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc068370 0x55ccfc06a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.449+0000 7fcf0d103740 1 --2- 172.21.15.43:0/401387966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc05d5d0 0x55ccfc05da10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.449+0000 7fcf0d103740 1 --2- 172.21.15.43:0/401387966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccfc05eb40 0x55ccfc05d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.449+0000 7fcf0d103740 1 -- 172.21.15.43:0/401387966 >> 172.21.15.43:0/401387966 conn(0x55ccfc026310 msgr2=0x55ccfc059c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:36.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.449+0000 7fcf0d103740 1 -- 172.21.15.43:0/401387966 shutdown_connections 2024-09-11T23:30:36.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.449+0000 7fcf0d103740 1 -- 172.21.15.43:0/401387966 wait complete. 2024-09-11T23:30:36.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.449+0000 7fcf0d103740 1 -- start start 2024-09-11T23:30:36.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.449+0000 7fcf0d103740 1 Processor -- start 2024-09-11T23:30:36.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcf0d103740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccfc05d5d0 0x55ccfc067d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:36.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcf0d103740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc05eb40 0x55ccfc064ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:36.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcf0d103740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc068370 0x55ccfc0655e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:36.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcf0d103740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ccfc06b070 con 0x55ccfc05eb40 2024-09-11T23:30:36.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcf0d103740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ccfbf7e060 con 0x55ccfc068370 2024-09-11T23:30:36.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcf0d103740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ccfc06b1f0 con 0x55ccfc05d5d0 2024-09-11T23:30:36.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcf0d103740 10 client.0 _ll_register_callbacks cb 0x55ccfbcd18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:30:36.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcf03fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc05eb40 0x55ccfc064ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:36.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcf03fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc05eb40 0x55ccfc064ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36976/0 (socket says 172.21.15.43:36976) 2024-09-11T23:30:36.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcf03fff640 1 -- 172.21.15.43:0/1709750319 learned_addr learned my addr 172.21.15.43:0/1709750319 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:36.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcf08ac6640 1 --2- 172.21.15.43:0/1709750319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccfc05d5d0 0x55ccfc067d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:36.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcf092c7640 1 --2- 172.21.15.43:0/1709750319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc068370 0x55ccfc0655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:36.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcf03fff640 1 -- 172.21.15.43:0/1709750319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccfc05d5d0 msgr2=0x55ccfc067d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:36.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcf03fff640 1 --2- 172.21.15.43:0/1709750319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccfc05d5d0 0x55ccfc067d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcf03fff640 1 -- 172.21.15.43:0/1709750319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc068370 msgr2=0x55ccfc0655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:36.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcf03fff640 1 --2- 172.21.15.43:0/1709750319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc068370 0x55ccfc0655e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcf03fff640 1 -- 172.21.15.43:0/1709750319 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccfbd343c0 con 0x55ccfc05eb40 2024-09-11T23:30:36.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcf092c7640 1 --2- 172.21.15.43:0/1709750319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc068370 0x55ccfc0655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:30:36.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcf03fff640 1 --2- 172.21.15.43:0/1709750319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc05eb40 0x55ccfc064ff0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fcef000fc30 tx=0x7fcef000b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c316f2c322fb000 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:36.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcee3fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:30:36.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcee3fff640 1 -- 172.21.15.43:0/1709750319 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef0028060 con 0x55ccfc05eb40 2024-09-11T23:30:36.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcee3fff640 1 -- 172.21.15.43:0/1709750319 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7fcef0011440 con 0x55ccfc05eb40 2024-09-11T23:30:36.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.450+0000 7fcf0d103740 1 -- 172.21.15.43:0/1709750319 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcef0011920 con 0x55ccfc05eb40 2024-09-11T23:30:36.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.451+0000 7fcee3fff640 1 -- 172.21.15.43:0/1709750319 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcef0015070 con 0x55ccfc05eb40 2024-09-11T23:30:36.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.451+0000 7fcf0d103740 10 client.16794 fetch_fsmap learned FSMap version 32 2024-09-11T23:30:36.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.451+0000 7fcf0d103740 1 -- 172.21.15.43:0/1709750319 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ccfc076ae0 con 0x55ccfc05eb40 2024-09-11T23:30:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.451+0000 7fcee3fff640 1 -- 172.21.15.43:0/1709750319 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef0011920 con 0x55ccfc05eb40 2024-09-11T23:30:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.451+0000 7fcee3fff640 1 -- 172.21.15.43:0/1709750319 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fcef0005430 con 0x55ccfc05eb40 2024-09-11T23:30:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.451+0000 7fcf0d103740 10 client.16794 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:30:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.451+0000 7fcf0d103740 10 client.16794 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T23:30:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.451+0000 7fcf0d103740 20 client.16794 populate_metadata read hostname 'smithi043' 2024-09-11T23:30:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.451+0000 7fcf0d103740 1 --2- 172.21.15.43:0/1709750319 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55ccfc077e90 0x55ccfc07a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.451+0000 7fcf0d103740 4 client.16794 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:30:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.451+0000 7fcf0d103740 1 -- 172.21.15.43:0/1709750319 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccfbfd0860 con 0x55ccfc077e90 2024-09-11T23:30:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.451+0000 7fcf08ac6640 1 --2- 172.21.15.43:0/1709750319 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55ccfc077e90 0x55ccfc07a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.452+0000 7fcf08ac6640 1 --2- 172.21.15.43:0/1709750319 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55ccfc077e90 0x55ccfc07a280 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=678d33d4305829be server_cookie=f2800ef5a0b24ee in_seq=0 out_seq=0 2024-09-11T23:30:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.452+0000 7fcee3fff640 10 client.16794 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:30:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.452+0000 7fcee3fff640 1 -- 172.21.15.43:0/1709750319 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ccfbfd0860 con 0x55ccfc077e90 2024-09-11T23:30:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.452+0000 7fcee3fff640 10 client.16794 handle_command_reply: tid=0 2024-09-11T23:30:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.452+0000 7fcf0d103740 1 client.16794 shutdown 2024-09-11T23:30:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.453+0000 7fcf0d103740 1 -- 172.21.15.43:0/1709750319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc05eb40 msgr2=0x55ccfc064ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.453+0000 7fcf0d103740 1 --2- 172.21.15.43:0/1709750319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc05eb40 0x55ccfc064ff0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fcef000fc30 tx=0x7fcef000b9f0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.453+0000 7fcf0d103740 1 -- 172.21.15.43:0/1709750319 shutdown_connections 2024-09-11T23:30:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.453+0000 7fcf0d103740 1 --2- 172.21.15.43:0/1709750319 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55ccfc077e90 0x55ccfc07a280 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.453+0000 7fcee3fff640 0 client.16794 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:30:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.453+0000 7fcf0d103740 1 --2- 172.21.15.43:0/1709750319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc068370 0x55ccfc0655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.453+0000 7fcf0d103740 1 --2- 172.21.15.43:0/1709750319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc05eb40 0x55ccfc064ff0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.453+0000 7fcf0d103740 1 --2- 172.21.15.43:0/1709750319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccfc05d5d0 0x55ccfc067d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.453+0000 7fcf0d103740 1 -- 172.21.15.43:0/1709750319 >> 172.21.15.43:0/1709750319 conn(0x55ccfc026310 msgr2=0x55ccfc05f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:36.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.453+0000 7fcf0d103740 1 -- 172.21.15.43:0/1709750319 shutdown_connections 2024-09-11T23:30:36.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.453+0000 7fcf0d103740 1 -- 172.21.15.43:0/1709750319 wait complete. 2024-09-11T23:30:36.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.453+0000 7fcf0d103740 20 client.16794 trim_cache size 0 max 16384 2024-09-11T23:30:36.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.466+0000 7fcf0d103740 1 Processor -- start 2024-09-11T23:30:36.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.466+0000 7fcf0d103740 1 -- start start 2024-09-11T23:30:36.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.466+0000 7fcf0d103740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc05d5d0 0x55ccfc074310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:36.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.466+0000 7fcf0d103740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc05eb40 0x55ccfc066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:36.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.466+0000 7fcf0d103740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccfc068370 0x55ccfc07f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:36.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.466+0000 7fcf0d103740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcf041b2470 con 0x55ccfc05d5d0 2024-09-11T23:30:36.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.466+0000 7fcf0d103740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcf041afd10 con 0x55ccfc05eb40 2024-09-11T23:30:36.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.466+0000 7fcf0d103740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcf041adb80 con 0x55ccfc068370 2024-09-11T23:30:36.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.466+0000 7fcf08ac6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc05d5d0 0x55ccfc074310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:36.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.466+0000 7fcf08ac6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc05d5d0 0x55ccfc074310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36980/0 (socket says 172.21.15.43:36980) 2024-09-11T23:30:36.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.467+0000 7fcf08ac6640 1 -- 172.21.15.43:0/2945957993 learned_addr learned my addr 172.21.15.43:0/2945957993 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:36.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.467+0000 7fcf092c7640 1 --2- 172.21.15.43:0/2945957993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccfc068370 0x55ccfc07f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:36.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.467+0000 7fcf03fff640 1 --2- 172.21.15.43:0/2945957993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc05eb40 0x55ccfc066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:36.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.467+0000 7fcf08ac6640 1 -- 172.21.15.43:0/2945957993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccfc068370 msgr2=0x55ccfc07f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:36.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.467+0000 7fcf08ac6640 1 --2- 172.21.15.43:0/2945957993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccfc068370 0x55ccfc07f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.467+0000 7fcf08ac6640 1 -- 172.21.15.43:0/2945957993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc05eb40 msgr2=0x55ccfc066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:36.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.467+0000 7fcf08ac6640 1 --2- 172.21.15.43:0/2945957993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc05eb40 0x55ccfc066f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.467+0000 7fcf08ac6640 1 -- 172.21.15.43:0/2945957993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccfc060120 con 0x55ccfc05d5d0 2024-09-11T23:30:36.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.467+0000 7fcf08ac6640 1 --2- 172.21.15.43:0/2945957993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc05d5d0 0x55ccfc074310 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009610 tx=0x7fcef8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a828978261464ce7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:36.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.467+0000 7fceee7fc640 1 -- 172.21.15.43:0/2945957993 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef80b3070 con 0x55ccfc05d5d0 2024-09-11T23:30:36.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.467+0000 7fceee7fc640 1 -- 172.21.15.43:0/2945957993 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef809baf0 con 0x55ccfc05d5d0 2024-09-11T23:30:36.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.467+0000 7fceee7fc640 1 -- 172.21.15.43:0/2945957993 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8020040 con 0x55ccfc05d5d0 2024-09-11T23:30:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.468+0000 7fcf0d103740 1 -- 172.21.15.43:0/2945957993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc05d5d0 msgr2=0x55ccfc074310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.468+0000 7fcf0d103740 1 --2- 172.21.15.43:0/2945957993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc05d5d0 0x55ccfc074310 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009610 tx=0x7fcef8009c30 comp rx=0 tx=0).stop 2024-09-11T23:30:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.468+0000 7fcf0d103740 1 -- 172.21.15.43:0/2945957993 shutdown_connections 2024-09-11T23:30:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.468+0000 7fcf0d103740 1 --2- 172.21.15.43:0/2945957993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccfc068370 0x55ccfc07f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.468+0000 7fcf0d103740 1 --2- 172.21.15.43:0/2945957993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc05eb40 0x55ccfc066f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.468+0000 7fcf0d103740 1 --2- 172.21.15.43:0/2945957993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc05d5d0 0x55ccfc074310 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.468+0000 7fcf0d103740 1 -- 172.21.15.43:0/2945957993 >> 172.21.15.43:0/2945957993 conn(0x55ccfc026310 msgr2=0x55ccfc066850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.468+0000 7fcf0d103740 1 -- 172.21.15.43:0/2945957993 shutdown_connections 2024-09-11T23:30:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.468+0000 7fcf0d103740 1 -- 172.21.15.43:0/2945957993 wait complete. 2024-09-11T23:30:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.468+0000 7fcf0d103740 1 -- start start 2024-09-11T23:30:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.468+0000 7fcf0d103740 1 Processor -- start 2024-09-11T23:30:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.469+0000 7fcf0d103740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc05d5d0 0x55ccfc070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.469+0000 7fcf0d103740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccfc05eb40 0x55ccfc061e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.469+0000 7fcf0d103740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc068370 0x55ccfc0623e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.469+0000 7fcf0d103740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcf041ae1d0 con 0x55ccfc068370 2024-09-11T23:30:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.469+0000 7fcf0d103740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcf041af640 con 0x55ccfc05d5d0 2024-09-11T23:30:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.469+0000 7fcf0d103740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcf041ae8a0 con 0x55ccfc05eb40 2024-09-11T23:30:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.469+0000 7fcf092c7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc068370 0x55ccfc0623e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.469+0000 7fcf0d103740 10 client.0 _ll_register_callbacks cb 0x7fcefc031c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:30:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.469+0000 7fcf08ac6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc05d5d0 0x55ccfc070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.469+0000 7fcf092c7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc068370 0x55ccfc0623e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36988/0 (socket says 172.21.15.43:36988) 2024-09-11T23:30:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.469+0000 7fcf092c7640 1 -- 172.21.15.43:0/2447527083 learned_addr learned my addr 172.21.15.43:0/2447527083 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.469+0000 7fcf03fff640 1 --2- 172.21.15.43:0/2447527083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccfc05eb40 0x55ccfc061e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.469+0000 7fcf092c7640 1 -- 172.21.15.43:0/2447527083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccfc05eb40 msgr2=0x55ccfc061e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.469+0000 7fcf092c7640 1 --2- 172.21.15.43:0/2447527083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccfc05eb40 0x55ccfc061e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.469+0000 7fcf092c7640 1 -- 172.21.15.43:0/2447527083 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc05d5d0 msgr2=0x55ccfc070cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.469+0000 7fcf092c7640 1 --2- 172.21.15.43:0/2447527083 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc05d5d0 0x55ccfc070cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.469+0000 7fcf092c7640 1 -- 172.21.15.43:0/2447527083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccfc062c10 con 0x55ccfc068370 2024-09-11T23:30:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.469+0000 7fcf092c7640 1 --2- 172.21.15.43:0/2447527083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc068370 0x55ccfc0623e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009ca0 tx=0x7fcefc004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbe4c99d037b0e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.469+0000 7fcee3fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:30:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.470+0000 7fcee3fff640 1 -- 172.21.15.43:0/2447527083 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc059060 con 0x55ccfc068370 2024-09-11T23:30:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.470+0000 7fcee3fff640 1 -- 172.21.15.43:0/2447527083 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7fcefc045070 con 0x55ccfc068370 2024-09-11T23:30:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.470+0000 7fcf0d103740 1 -- 172.21.15.43:0/2447527083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcef809baf0 con 0x55ccfc068370 2024-09-11T23:30:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.470+0000 7fcee3fff640 1 -- 172.21.15.43:0/2447527083 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcefc0047a0 con 0x55ccfc068370 2024-09-11T23:30:36.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.470+0000 7fcf0d103740 10 client.16800 fetch_fsmap learned FSMap version 32 2024-09-11T23:30:36.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.470+0000 7fcf0d103740 1 -- 172.21.15.43:0/2447527083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ccfc0e2950 con 0x55ccfc068370 2024-09-11T23:30:36.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.470+0000 7fcee3fff640 1 -- 172.21.15.43:0/2447527083 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef809baf0 con 0x55ccfc068370 2024-09-11T23:30:36.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.470+0000 7fcee3fff640 1 -- 172.21.15.43:0/2447527083 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fcefc030750 con 0x55ccfc068370 2024-09-11T23:30:36.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.470+0000 7fcf0d103740 10 client.16800 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:30:36.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.470+0000 7fcf0d103740 10 client.16800 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T23:30:36.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.470+0000 7fcf0d103740 20 client.16800 populate_metadata read hostname 'smithi043' 2024-09-11T23:30:36.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.470+0000 7fcf0d103740 1 --2- 172.21.15.43:0/2447527083 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55ccfc077e90 0x55ccfc0e50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:36.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.470+0000 7fcf0d103740 4 client.16800 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:30:36.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.470+0000 7fcf0d103740 1 -- 172.21.15.43:0/2447527083 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ccfbfcad70 con 0x55ccfc077e90 2024-09-11T23:30:36.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.470+0000 7fcf08ac6640 1 --2- 172.21.15.43:0/2447527083 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55ccfc077e90 0x55ccfc0e50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:36.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.471+0000 7fcf08ac6640 1 --2- 172.21.15.43:0/2447527083 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55ccfc077e90 0x55ccfc0e50c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3212bd8090643f4e server_cookie=b08dcbc79846c075 in_seq=0 out_seq=0 2024-09-11T23:30:36.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.471+0000 7fcee3fff640 10 client.16800 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:30:36.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.471+0000 7fcee3fff640 1 -- 172.21.15.43:0/2447527083 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ccfbfcad70 con 0x55ccfc077e90 2024-09-11T23:30:36.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.471+0000 7fcee3fff640 10 client.16800 handle_command_reply: tid=0 2024-09-11T23:30:36.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.471+0000 7fcf0d103740 1 client.16800 shutdown 2024-09-11T23:30:36.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.471+0000 7fcf0d103740 1 -- 172.21.15.43:0/2447527083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc068370 msgr2=0x55ccfc0623e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:36.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.471+0000 7fcf0d103740 1 --2- 172.21.15.43:0/2447527083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc068370 0x55ccfc0623e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009ca0 tx=0x7fcefc004a80 comp rx=0 tx=0).stop 2024-09-11T23:30:36.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.471+0000 7fcf0d103740 1 -- 172.21.15.43:0/2447527083 shutdown_connections 2024-09-11T23:30:36.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.471+0000 7fcf0d103740 1 --2- 172.21.15.43:0/2447527083 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x55ccfc077e90 0x55ccfc0e50c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.471+0000 7fcee3fff640 0 client.16800 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:30:36.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.471+0000 7fcf0d103740 1 --2- 172.21.15.43:0/2447527083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ccfc068370 0x55ccfc0623e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.471+0000 7fcf0d103740 1 --2- 172.21.15.43:0/2447527083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccfc05eb40 0x55ccfc061e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.471+0000 7fcf0d103740 1 --2- 172.21.15.43:0/2447527083 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ccfc05d5d0 0x55ccfc070cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.471+0000 7fcf0d103740 1 -- 172.21.15.43:0/2447527083 >> 172.21.15.43:0/2447527083 conn(0x55ccfc026310 msgr2=0x55ccfc067070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.471+0000 7fcf0d103740 1 -- 172.21.15.43:0/2447527083 shutdown_connections 2024-09-11T23:30:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.471+0000 7fcf0d103740 1 -- 172.21.15.43:0/2447527083 wait complete. 2024-09-11T23:30:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.471+0000 7fcf0d103740 20 client.16800 trim_cache size 0 max 16384 2024-09-11T23:30:36.489 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:30:36.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.472+0000 7fcf0ad2a640 1 -- 172.21.15.43:0/277127717 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fcef40a2cf0 msgr2=0x7fcef40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:36.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.472+0000 7fcf0ad2a640 1 --2- 172.21.15.43:0/277127717 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fcef40a2cf0 0x7fcef40a51b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcefc004af0 tx=0x7fcefc02f4b0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.472+0000 7fcf0ad2a640 1 -- 172.21.15.43:0/277127717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf04101de0 msgr2=0x7fcf040744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.472+0000 7fcf0ad2a640 1 --2- 172.21.15.43:0/277127717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf04101de0 0x7fcf040744f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fcef800d770 tx=0x7fcef800dc30 comp rx=0 tx=0).stop 2024-09-11T23:30:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.472+0000 7fcf0ad2a640 1 -- 172.21.15.43:0/277127717 shutdown_connections 2024-09-11T23:30:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.472+0000 7fcf0ad2a640 1 --2- 172.21.15.43:0/277127717 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fcef40a2cf0 0x7fcef40a51b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.472+0000 7fcf0ad2a640 1 --2- 172.21.15.43:0/277127717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf04106f10 0x7fcf04074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.472+0000 7fcf0ad2a640 1 --2- 172.21.15.43:0/277127717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf04102790 0x7fcf04074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.472+0000 7fcf0ad2a640 1 --2- 172.21.15.43:0/277127717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf04101de0 0x7fcf040744f0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.472+0000 7fcf0ad2a640 1 -- 172.21.15.43:0/277127717 >> 172.21.15.43:0/277127717 conn(0x7fcf040fd8c0 msgr2=0x7fcf040feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.472+0000 7fcf0ad2a640 1 -- 172.21.15.43:0/277127717 shutdown_connections 2024-09-11T23:30:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:36.472+0000 7fcf0ad2a640 1 -- 172.21.15.43:0/277127717 wait complete. 2024-09-11T23:30:37.296 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:30:37.562 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:30:37.967 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:37 smithi043 ceph-mon[30950]: pgmap v270: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 16 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T23:30:38.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.051+0000 7f551f16f640 1 -- 172.21.15.43:0/2559783464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f551810c170 msgr2=0x7f551810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:38.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.051+0000 7f551f16f640 1 --2- 172.21.15.43:0/2559783464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f551810c170 0x7f551810e610 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5510009ca0 tx=0x7f551002a950 comp rx=0 tx=0).stop 2024-09-11T23:30:38.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.051+0000 7f551f16f640 1 -- 172.21.15.43:0/2559783464 shutdown_connections 2024-09-11T23:30:38.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.052+0000 7f551f16f640 1 --2- 172.21.15.43:0/2559783464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f551810c170 0x7f551810e610 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.052+0000 7f551f16f640 1 --2- 172.21.15.43:0/2559783464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5518103660 0x7f551810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.052+0000 7f551f16f640 1 --2- 172.21.15.43:0/2559783464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5518102d40 0x7f5518103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.052+0000 7f551f16f640 1 -- 172.21.15.43:0/2559783464 >> 172.21.15.43:0/2559783464 conn(0x7f55180fc7a0 msgr2=0x7f55180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:38.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.052+0000 7f551f16f640 1 -- 172.21.15.43:0/2559783464 shutdown_connections 2024-09-11T23:30:38.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.052+0000 7f551f16f640 1 -- 172.21.15.43:0/2559783464 wait complete. 2024-09-11T23:30:38.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.052+0000 7f551f16f640 1 Processor -- start 2024-09-11T23:30:38.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.052+0000 7f551f16f640 1 -- start start 2024-09-11T23:30:38.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.053+0000 7f551f16f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5518102d40 0x7f5518198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.053+0000 7f551f16f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5518103660 0x7f55181989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.053+0000 7f551f16f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f551810c170 0x7f551819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.053+0000 7f551f16f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5518111f50 con 0x7f5518103660 2024-09-11T23:30:38.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.053+0000 7f551f16f640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5518111dd0 con 0x7f551810c170 2024-09-11T23:30:38.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.053+0000 7f551f16f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f55181120d0 con 0x7f5518102d40 2024-09-11T23:30:38.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.053+0000 7f5517fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5518103660 0x7f55181989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.053+0000 7f5517fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5518103660 0x7f55181989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36996/0 (socket says 172.21.15.43:36996) 2024-09-11T23:30:38.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.053+0000 7f5517fff640 1 -- 172.21.15.43:0/3717634380 learned_addr learned my addr 172.21.15.43:0/3717634380 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:38.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.053+0000 7f551cf0b640 1 --2- 172.21.15.43:0/3717634380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5518102d40 0x7f5518198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.053+0000 7f551d70c640 1 --2- 172.21.15.43:0/3717634380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f551810c170 0x7f551819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.053+0000 7f5517fff640 1 -- 172.21.15.43:0/3717634380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5518102d40 msgr2=0x7f5518198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.053+0000 7f5517fff640 1 --2- 172.21.15.43:0/3717634380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5518102d40 0x7f5518198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.053+0000 7f5517fff640 1 -- 172.21.15.43:0/3717634380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f551810c170 msgr2=0x7f551819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.053+0000 7f5517fff640 1 --2- 172.21.15.43:0/3717634380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f551810c170 0x7f551819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.053+0000 7f5517fff640 1 -- 172.21.15.43:0/3717634380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551819d410 con 0x7f5518103660 2024-09-11T23:30:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.053+0000 7f5517fff640 1 --2- 172.21.15.43:0/3717634380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5518103660 0x7f55181989a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f550400e9f0 tx=0x7f550400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.054+0000 7f5515ffb640 1 -- 172.21.15.43:0/3717634380 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550400ccc0 con 0x7f5518103660 2024-09-11T23:30:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.054+0000 7f5515ffb640 1 -- 172.21.15.43:0/3717634380 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f550400ce60 con 0x7f5518103660 2024-09-11T23:30:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.054+0000 7f551f16f640 1 -- 172.21.15.43:0/3717634380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f551819d700 con 0x7f5518103660 2024-09-11T23:30:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.054+0000 7f5515ffb640 1 -- 172.21.15.43:0/3717634380 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504010690 con 0x7f5518103660 2024-09-11T23:30:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.054+0000 7f551f16f640 1 -- 172.21.15.43:0/3717634380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55181a4fe0 con 0x7f5518103660 2024-09-11T23:30:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.056+0000 7f5515ffb640 1 -- 172.21.15.43:0/3717634380 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f55040046c0 con 0x7f5518103660 2024-09-11T23:30:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.056+0000 7f5521548740 1 Processor -- start 2024-09-11T23:30:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.056+0000 7f5521548740 1 -- start start 2024-09-11T23:30:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.057+0000 7f5521548740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a546c2b40 0x560a546c1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.057+0000 7f5521548740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546c15d0 0x560a546c1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.057+0000 7f5521548740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a546cc370 0x560a546ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.057+0000 7f5521548740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560a546272f0 con 0x560a546c15d0 2024-09-11T23:30:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.057+0000 7f5521548740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560a545cecf0 con 0x560a546cc370 2024-09-11T23:30:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.057+0000 7f5521548740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560a546232b0 con 0x560a546c2b40 2024-09-11T23:30:38.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.057+0000 7f5515ffb640 1 --2- 172.21.15.43:0/3717634380 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f55080a2cf0 0x7f55080a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.057+0000 7f551cf0b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546c15d0 0x560a546c1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.057+0000 7f551cf0b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546c15d0 0x560a546c1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37010/0 (socket says 172.21.15.43:37010) 2024-09-11T23:30:38.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.057+0000 7f551cf0b640 1 -- 172.21.15.43:0/451787281 learned_addr learned my addr 172.21.15.43:0/451787281 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:38.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.057+0000 7f551d70c640 1 --2- 172.21.15.43:0/451787281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a546c2b40 0x560a546c1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.057+0000 7f5517fff640 1 --2- 172.21.15.43:0/451787281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a546cc370 0x560a546ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.057+0000 7f551d70c640 1 --2- 172.21.15.43:0/3717634380 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f55080a2cf0 0x7f55080a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.057+0000 7f5515ffb640 1 -- 172.21.15.43:0/3717634380 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f55040cf8d0 con 0x7f5518103660 2024-09-11T23:30:38.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.057+0000 7f551cf0b640 1 -- 172.21.15.43:0/451787281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a546c2b40 msgr2=0x560a546c1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:38.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.057+0000 7f551cf0b640 1 --2- 172.21.15.43:0/451787281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a546c2b40 0x560a546c1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.057+0000 7f551cf0b640 1 -- 172.21.15.43:0/451787281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a546cc370 msgr2=0x560a546ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:38.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.057+0000 7f551cf0b640 1 --2- 172.21.15.43:0/451787281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a546cc370 0x560a546ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.057+0000 7f551cf0b640 1 -- 172.21.15.43:0/451787281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a545b33a0 con 0x560a546c15d0 2024-09-11T23:30:38.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.057+0000 7f551d70c640 1 --2- 172.21.15.43:0/3717634380 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f55080a2cf0 0x7f55080a51b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5510005fa0 tx=0x7f551002a950 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:38.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.057+0000 7f551cf0b640 1 --2- 172.21.15.43:0/451787281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546c15d0 0x560a546c1a10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f550c009cf0 tx=0x7f550c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a11540467030061 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:38.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.058+0000 7f5502ffd640 1 -- 172.21.15.43:0/451787281 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c004230 con 0x560a546c15d0 2024-09-11T23:30:38.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.058+0000 7f5502ffd640 1 -- 172.21.15.43:0/451787281 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f550c0043d0 con 0x560a546c15d0 2024-09-11T23:30:38.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.058+0000 7f5502ffd640 1 -- 172.21.15.43:0/451787281 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c0198c0 con 0x560a546c15d0 2024-09-11T23:30:38.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.058+0000 7f5521548740 1 -- 172.21.15.43:0/451787281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546c15d0 msgr2=0x560a546c1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:38.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.058+0000 7f5521548740 1 --2- 172.21.15.43:0/451787281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546c15d0 0x560a546c1a10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f550c009cf0 tx=0x7f550c011810 comp rx=0 tx=0).stop 2024-09-11T23:30:38.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.058+0000 7f5521548740 1 -- 172.21.15.43:0/451787281 shutdown_connections 2024-09-11T23:30:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.058+0000 7f5521548740 1 --2- 172.21.15.43:0/451787281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a546cc370 0x560a546ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.058+0000 7f5521548740 1 --2- 172.21.15.43:0/451787281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546c15d0 0x560a546c1a10 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.058+0000 7f5521548740 1 --2- 172.21.15.43:0/451787281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a546c2b40 0x560a546c1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.058+0000 7f5521548740 1 -- 172.21.15.43:0/451787281 >> 172.21.15.43:0/451787281 conn(0x560a5468a310 msgr2=0x560a546bdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.058+0000 7f5521548740 1 -- 172.21.15.43:0/451787281 shutdown_connections 2024-09-11T23:30:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.058+0000 7f5521548740 1 -- 172.21.15.43:0/451787281 wait complete. 2024-09-11T23:30:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.059+0000 7f5521548740 1 -- start start 2024-09-11T23:30:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.059+0000 7f5521548740 1 Processor -- start 2024-09-11T23:30:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.059+0000 7f5521548740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a546c15d0 0x560a546cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.059+0000 7f5521548740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a546c2b40 0x560a546c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.059+0000 7f5521548740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546cc370 0x560a546c9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.059+0000 7f5521548740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560a546cf070 con 0x560a546cc370 2024-09-11T23:30:38.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.059+0000 7f5521548740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560a545e2060 con 0x560a546c2b40 2024-09-11T23:30:38.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.059+0000 7f5521548740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560a546cf1f0 con 0x560a546c15d0 2024-09-11T23:30:38.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.059+0000 7f5521548740 10 client.0 _ll_register_callbacks cb 0x560a543358c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:30:38.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.059+0000 7f551d70c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546cc370 0x560a546c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.059+0000 7f551d70c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546cc370 0x560a546c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37026/0 (socket says 172.21.15.43:37026) 2024-09-11T23:30:38.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.059+0000 7f551d70c640 1 -- 172.21.15.43:0/1139864864 learned_addr learned my addr 172.21.15.43:0/1139864864 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:38.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.059+0000 7f551cf0b640 1 --2- 172.21.15.43:0/1139864864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a546c15d0 0x560a546cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.059+0000 7f5517fff640 1 --2- 172.21.15.43:0/1139864864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a546c2b40 0x560a546c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.060+0000 7f551d70c640 1 -- 172.21.15.43:0/1139864864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a546c15d0 msgr2=0x560a546cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:38.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.060+0000 7f551d70c640 1 --2- 172.21.15.43:0/1139864864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a546c15d0 0x560a546cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.060+0000 7f551d70c640 1 -- 172.21.15.43:0/1139864864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a546c2b40 msgr2=0x560a546c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:38.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.060+0000 7f551d70c640 1 --2- 172.21.15.43:0/1139864864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a546c2b40 0x560a546c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.060+0000 7f551d70c640 1 -- 172.21.15.43:0/1139864864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a543983c0 con 0x560a546cc370 2024-09-11T23:30:38.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.060+0000 7f551d70c640 1 --2- 172.21.15.43:0/1139864864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546cc370 0x560a546c9530 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5510009c70 tx=0x7f55100095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4a5e818fdea7f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:38.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.060+0000 7f54f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:30:38.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.060+0000 7f54f7fff640 1 -- 172.21.15.43:0/1139864864 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551004f030 con 0x560a546cc370 2024-09-11T23:30:38.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.060+0000 7f54f7fff640 1 -- 172.21.15.43:0/1139864864 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f5510003b80 con 0x560a546cc370 2024-09-11T23:30:38.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.060+0000 7f5521548740 1 -- 172.21.15.43:0/1139864864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f550c0043d0 con 0x560a546cc370 2024-09-11T23:30:38.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.060+0000 7f54f7fff640 1 -- 172.21.15.43:0/1139864864 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5510034040 con 0x560a546cc370 2024-09-11T23:30:38.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.060+0000 7f5521548740 10 client.16812 fetch_fsmap learned FSMap version 32 2024-09-11T23:30:38.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.060+0000 7f5521548740 1 -- 172.21.15.43:0/1139864864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560a546daaa0 con 0x560a546cc370 2024-09-11T23:30:38.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.060+0000 7f54f7fff640 1 -- 172.21.15.43:0/1139864864 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f550c0043d0 con 0x560a546cc370 2024-09-11T23:30:38.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.060+0000 7f54f7fff640 1 -- 172.21.15.43:0/1139864864 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f55100045b0 con 0x560a546cc370 2024-09-11T23:30:38.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.061+0000 7f5521548740 10 client.16812 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:30:38.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.061+0000 7f5521548740 10 client.16812 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T23:30:38.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.061+0000 7f5521548740 20 client.16812 populate_metadata read hostname 'smithi043' 2024-09-11T23:30:38.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.061+0000 7f5521548740 1 --2- 172.21.15.43:0/1139864864 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x560a546dbf00 0x560a546de370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.061+0000 7f5521548740 4 client.16812 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:30:38.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.061+0000 7f5521548740 1 -- 172.21.15.43:0/1139864864 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a54634860 con 0x560a546dbf00 2024-09-11T23:30:38.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.061+0000 7f551cf0b640 1 --2- 172.21.15.43:0/1139864864 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x560a546dbf00 0x560a546de370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.061+0000 7f54f7fff640 10 client.16812 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:30:38.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.061+0000 7f551cf0b640 1 --2- 172.21.15.43:0/1139864864 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x560a546dbf00 0x560a546de370 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d90480c18933ab87 server_cookie=cfa629638d3946bf in_seq=0 out_seq=0 2024-09-11T23:30:38.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.062+0000 7f54f7fff640 1 -- 172.21.15.43:0/1139864864 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a54634860 con 0x560a546dbf00 2024-09-11T23:30:38.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.062+0000 7f54f7fff640 10 client.16812 handle_command_reply: tid=0 2024-09-11T23:30:38.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.062+0000 7f5521548740 1 client.16812 shutdown 2024-09-11T23:30:38.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.062+0000 7f5521548740 1 -- 172.21.15.43:0/1139864864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546cc370 msgr2=0x560a546c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:38.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.062+0000 7f5521548740 1 --2- 172.21.15.43:0/1139864864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546cc370 0x560a546c9530 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5510009c70 tx=0x7f55100095c0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.062+0000 7f5521548740 1 -- 172.21.15.43:0/1139864864 shutdown_connections 2024-09-11T23:30:38.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.062+0000 7f5521548740 1 --2- 172.21.15.43:0/1139864864 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x560a546dbf00 0x560a546de370 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.062+0000 7f5521548740 1 --2- 172.21.15.43:0/1139864864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546cc370 0x560a546c9530 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.062+0000 7f54f7fff640 0 client.16812 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:30:38.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.062+0000 7f5521548740 1 --2- 172.21.15.43:0/1139864864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a546c2b40 0x560a546c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.062+0000 7f5521548740 1 --2- 172.21.15.43:0/1139864864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a546c15d0 0x560a546cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.062+0000 7f5521548740 1 -- 172.21.15.43:0/1139864864 >> 172.21.15.43:0/1139864864 conn(0x560a5468a310 msgr2=0x560a546c3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:38.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.062+0000 7f5521548740 1 -- 172.21.15.43:0/1139864864 shutdown_connections 2024-09-11T23:30:38.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.062+0000 7f5521548740 1 -- 172.21.15.43:0/1139864864 wait complete. 2024-09-11T23:30:38.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.062+0000 7f5521548740 20 client.16812 trim_cache size 0 max 16384 2024-09-11T23:30:38.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.075+0000 7f5521548740 1 Processor -- start 2024-09-11T23:30:38.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.075+0000 7f5521548740 1 -- start start 2024-09-11T23:30:38.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.076+0000 7f5521548740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546c15d0 0x560a546bde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.076+0000 7f5521548740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a546c2b40 0x560a546d8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.076+0000 7f5521548740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a546cc370 0x560a546c4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.076+0000 7f5521548740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f55181b0350 con 0x560a546c15d0 2024-09-11T23:30:38.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.076+0000 7f5521548740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f55181adbf0 con 0x560a546cc370 2024-09-11T23:30:38.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.076+0000 7f5521548740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f55181aba60 con 0x560a546c2b40 2024-09-11T23:30:38.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.076+0000 7f551cf0b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546c15d0 0x560a546bde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.076+0000 7f551cf0b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546c15d0 0x560a546bde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37040/0 (socket says 172.21.15.43:37040) 2024-09-11T23:30:38.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.076+0000 7f551cf0b640 1 -- 172.21.15.43:0/2669414998 learned_addr learned my addr 172.21.15.43:0/2669414998 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:38.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.076+0000 7f551cf0b640 1 -- 172.21.15.43:0/2669414998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a546c2b40 msgr2=0x560a546d8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:38.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.076+0000 7f5517fff640 1 --2- 172.21.15.43:0/2669414998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a546c2b40 0x560a546d8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.076+0000 7f551d70c640 1 --2- 172.21.15.43:0/2669414998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a546cc370 0x560a546c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.076+0000 7f551cf0b640 1 --2- 172.21.15.43:0/2669414998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a546c2b40 0x560a546d8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.076+0000 7f551cf0b640 1 -- 172.21.15.43:0/2669414998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a546cc370 msgr2=0x560a546c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:38.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.076+0000 7f551cf0b640 1 --2- 172.21.15.43:0/2669414998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a546cc370 0x560a546c4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.076+0000 7f551cf0b640 1 -- 172.21.15.43:0/2669414998 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a546b8a30 con 0x560a546c15d0 2024-09-11T23:30:38.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.076+0000 7f551d70c640 1 --2- 172.21.15.43:0/2669414998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a546cc370 0x560a546c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:30:38.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.076+0000 7f5517fff640 1 --2- 172.21.15.43:0/2669414998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a546c2b40 0x560a546d8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:30:38.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.076+0000 7f551cf0b640 1 --2- 172.21.15.43:0/2669414998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546c15d0 0x560a546bde10 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f550c009c40 tx=0x7f550c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e01a7934564b5d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:38.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.077+0000 7f55027fc640 1 -- 172.21.15.43:0/2669414998 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c031070 con 0x560a546c15d0 2024-09-11T23:30:38.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.077+0000 7f55027fc640 1 -- 172.21.15.43:0/2669414998 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f550c005530 con 0x560a546c15d0 2024-09-11T23:30:38.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.077+0000 7f55027fc640 1 -- 172.21.15.43:0/2669414998 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c019840 con 0x560a546c15d0 2024-09-11T23:30:38.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.077+0000 7f5521548740 1 -- 172.21.15.43:0/2669414998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546c15d0 msgr2=0x560a546bde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:38.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.077+0000 7f5521548740 1 --2- 172.21.15.43:0/2669414998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546c15d0 0x560a546bde10 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f550c009c40 tx=0x7f550c005850 comp rx=0 tx=0).stop 2024-09-11T23:30:38.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.077+0000 7f5521548740 1 -- 172.21.15.43:0/2669414998 shutdown_connections 2024-09-11T23:30:38.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.077+0000 7f5521548740 1 --2- 172.21.15.43:0/2669414998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a546cc370 0x560a546c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.077+0000 7f5521548740 1 --2- 172.21.15.43:0/2669414998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a546c2b40 0x560a546d8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.077+0000 7f5521548740 1 --2- 172.21.15.43:0/2669414998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546c15d0 0x560a546bde10 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.077+0000 7f5521548740 1 -- 172.21.15.43:0/2669414998 >> 172.21.15.43:0/2669414998 conn(0x560a5468a310 msgr2=0x560a546bd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:38.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.077+0000 7f5521548740 1 -- 172.21.15.43:0/2669414998 shutdown_connections 2024-09-11T23:30:38.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.077+0000 7f5521548740 1 -- 172.21.15.43:0/2669414998 wait complete. 2024-09-11T23:30:38.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.078+0000 7f5521548740 1 -- start start 2024-09-11T23:30:38.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.078+0000 7f5521548740 1 Processor -- start 2024-09-11T23:30:38.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.078+0000 7f5521548740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546c15d0 0x560a546d4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.078+0000 7f5521548740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a546c2b40 0x560a546cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.078+0000 7f5521548740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a546cc370 0x560a546d0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.078+0000 7f5521548740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f55181ac0b0 con 0x560a546c15d0 2024-09-11T23:30:38.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.078+0000 7f5521548740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f55181ad520 con 0x560a546cc370 2024-09-11T23:30:38.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.078+0000 7f5521548740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f55181ac780 con 0x560a546c2b40 2024-09-11T23:30:38.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.078+0000 7f5521548740 10 client.0 _ll_register_callbacks cb 0x7f5504005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:30:38.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.078+0000 7f551cf0b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546c15d0 0x560a546d4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.078+0000 7f551cf0b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546c15d0 0x560a546d4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37044/0 (socket says 172.21.15.43:37044) 2024-09-11T23:30:38.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.078+0000 7f551cf0b640 1 -- 172.21.15.43:0/1424638186 learned_addr learned my addr 172.21.15.43:0/1424638186 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:38.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.078+0000 7f5517fff640 1 --2- 172.21.15.43:0/1424638186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a546c2b40 0x560a546cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.078+0000 7f551d70c640 1 --2- 172.21.15.43:0/1424638186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a546cc370 0x560a546d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.078+0000 7f551cf0b640 1 -- 172.21.15.43:0/1424638186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a546c2b40 msgr2=0x560a546cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:38.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.078+0000 7f551cf0b640 1 --2- 172.21.15.43:0/1424638186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a546c2b40 0x560a546cffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.078+0000 7f551cf0b640 1 -- 172.21.15.43:0/1424638186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a546cc370 msgr2=0x560a546d0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:38.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.078+0000 7f551cf0b640 1 --2- 172.21.15.43:0/1424638186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a546cc370 0x560a546d0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.078+0000 7f551cf0b640 1 -- 172.21.15.43:0/1424638186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a546d0d50 con 0x560a546c15d0 2024-09-11T23:30:38.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.079+0000 7f551cf0b640 1 --2- 172.21.15.43:0/1424638186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546c15d0 0x560a546d4c70 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f550c01b040 tx=0x7f550c011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5326e1a6f05a11 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:38.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.079+0000 7f54f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:30:38.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.079+0000 7f54f7fff640 1 -- 172.21.15.43:0/1424638186 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c031070 con 0x560a546c15d0 2024-09-11T23:30:38.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.079+0000 7f54f7fff640 1 -- 172.21.15.43:0/1424638186 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f550c003800 con 0x560a546c15d0 2024-09-11T23:30:38.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.079+0000 7f5521548740 1 -- 172.21.15.43:0/1424638186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f550c005530 con 0x560a546c15d0 2024-09-11T23:30:38.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.079+0000 7f54f7fff640 1 -- 172.21.15.43:0/1424638186 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f550c005a70 con 0x560a546c15d0 2024-09-11T23:30:38.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.079+0000 7f5521548740 10 client.16818 fetch_fsmap learned FSMap version 32 2024-09-11T23:30:38.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.079+0000 7f5521548740 1 -- 172.21.15.43:0/1424638186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560a54746c20 con 0x560a546c15d0 2024-09-11T23:30:38.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.079+0000 7f54f7fff640 1 -- 172.21.15.43:0/1424638186 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f550c005530 con 0x560a546c15d0 2024-09-11T23:30:38.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.079+0000 7f54f7fff640 1 -- 172.21.15.43:0/1424638186 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f550c03f030 con 0x560a546c15d0 2024-09-11T23:30:38.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.079+0000 7f5521548740 10 client.16818 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:30:38.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.079+0000 7f5521548740 10 client.16818 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T23:30:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.079+0000 7f5521548740 20 client.16818 populate_metadata read hostname 'smithi043' 2024-09-11T23:30:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.080+0000 7f5521548740 1 --2- 172.21.15.43:0/1424638186 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x560a546dbf00 0x560a547491b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.080+0000 7f5521548740 4 client.16818 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:30:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.080+0000 7f5521548740 1 -- 172.21.15.43:0/1424638186 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a5462ed70 con 0x560a546dbf00 2024-09-11T23:30:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.080+0000 7f5517fff640 1 --2- 172.21.15.43:0/1424638186 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x560a546dbf00 0x560a547491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:38.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.080+0000 7f5517fff640 1 --2- 172.21.15.43:0/1424638186 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x560a546dbf00 0x560a547491b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c488928eb285193 server_cookie=8e4566962390c6d6 in_seq=0 out_seq=0 2024-09-11T23:30:38.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.080+0000 7f54f7fff640 10 client.16818 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:30:38.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.080+0000 7f54f7fff640 1 -- 172.21.15.43:0/1424638186 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a5462ed70 con 0x560a546dbf00 2024-09-11T23:30:38.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.080+0000 7f54f7fff640 10 client.16818 handle_command_reply: tid=0 2024-09-11T23:30:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.080+0000 7f5521548740 1 client.16818 shutdown 2024-09-11T23:30:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f5521548740 1 -- 172.21.15.43:0/1424638186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546c15d0 msgr2=0x560a546d4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f5521548740 1 --2- 172.21.15.43:0/1424638186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546c15d0 0x560a546d4c70 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f550c01b040 tx=0x7f550c011e50 comp rx=0 tx=0).stop 2024-09-11T23:30:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f5521548740 1 -- 172.21.15.43:0/1424638186 shutdown_connections 2024-09-11T23:30:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f5521548740 1 --2- 172.21.15.43:0/1424638186 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x560a546dbf00 0x560a547491b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f5521548740 1 --2- 172.21.15.43:0/1424638186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a546cc370 0x560a546d0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f5521548740 1 --2- 172.21.15.43:0/1424638186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a546c2b40 0x560a546cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f5521548740 1 --2- 172.21.15.43:0/1424638186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560a546c15d0 0x560a546d4c70 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f5521548740 1 -- 172.21.15.43:0/1424638186 >> 172.21.15.43:0/1424638186 conn(0x560a5468a310 msgr2=0x560a546bc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f54f7fff640 0 client.16818 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:30:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f5521548740 1 -- 172.21.15.43:0/1424638186 shutdown_connections 2024-09-11T23:30:38.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f5521548740 1 -- 172.21.15.43:0/1424638186 wait complete. 2024-09-11T23:30:38.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f5521548740 20 client.16818 trim_cache size 0 max 16384 2024-09-11T23:30:38.100 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:30:38.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f551f16f640 1 -- 172.21.15.43:0/3717634380 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f55080a2cf0 msgr2=0x7f55080a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:38.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f551f16f640 1 --2- 172.21.15.43:0/3717634380 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f55080a2cf0 0x7f55080a51b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5510005fa0 tx=0x7f551002a950 comp rx=0 tx=0).stop 2024-09-11T23:30:38.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f551f16f640 1 -- 172.21.15.43:0/3717634380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5518103660 msgr2=0x7f55181989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:38.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f551f16f640 1 --2- 172.21.15.43:0/3717634380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5518103660 0x7f55181989a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f550400e9f0 tx=0x7f550400eeb0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f551f16f640 1 -- 172.21.15.43:0/3717634380 shutdown_connections 2024-09-11T23:30:38.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f551f16f640 1 --2- 172.21.15.43:0/3717634380 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f55080a2cf0 0x7f55080a51b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f551f16f640 1 --2- 172.21.15.43:0/3717634380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f551810c170 0x7f551819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f551f16f640 1 --2- 172.21.15.43:0/3717634380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5518103660 0x7f55181989a0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f551f16f640 1 --2- 172.21.15.43:0/3717634380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5518102d40 0x7f5518198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:38.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.081+0000 7f551f16f640 1 -- 172.21.15.43:0/3717634380 >> 172.21.15.43:0/3717634380 conn(0x7f55180fc7a0 msgr2=0x7f55180fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:38.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.082+0000 7f551f16f640 1 -- 172.21.15.43:0/3717634380 shutdown_connections 2024-09-11T23:30:38.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:38.082+0000 7f551f16f640 1 -- 172.21.15.43:0/3717634380 wait complete. 2024-09-11T23:30:38.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:37 smithi090 ceph-mon[35130]: pgmap v270: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 16 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T23:30:38.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:37 smithi097 ceph-mon[35562]: pgmap v270: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 16 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T23:30:38.784 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:30:39.040 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:30:39.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.558+0000 7f8e5fad9640 1 -- 172.21.15.43:0/2540704841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e58102790 msgr2=0x7f8e581067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:39.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.558+0000 7f8e5fad9640 1 --2- 172.21.15.43:0/2540704841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e58102790 0x7f8e581067e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009900 tx=0x7f8e4802a410 comp rx=0 tx=0).stop 2024-09-11T23:30:39.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.558+0000 7f8e5fad9640 1 -- 172.21.15.43:0/2540704841 shutdown_connections 2024-09-11T23:30:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.558+0000 7f8e5fad9640 1 --2- 172.21.15.43:0/2540704841 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e58106f10 0x7f8e5810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.558+0000 7f8e5fad9640 1 --2- 172.21.15.43:0/2540704841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e58102790 0x7f8e581067e0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.558+0000 7f8e5fad9640 1 --2- 172.21.15.43:0/2540704841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e58101de0 0x7f8e581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.558+0000 7f8e5fad9640 1 -- 172.21.15.43:0/2540704841 >> 172.21.15.43:0/2540704841 conn(0x7f8e580fd8c0 msgr2=0x7f8e580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.559+0000 7f8e5fad9640 1 -- 172.21.15.43:0/2540704841 shutdown_connections 2024-09-11T23:30:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.559+0000 7f8e5fad9640 1 -- 172.21.15.43:0/2540704841 wait complete. 2024-09-11T23:30:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.559+0000 7f8e5fad9640 1 Processor -- start 2024-09-11T23:30:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.559+0000 7f8e5fad9640 1 -- start start 2024-09-11T23:30:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.559+0000 7f8e5fad9640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e58101de0 0x7f8e580744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.559+0000 7f8e5fad9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e58102790 0x7f8e58074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.559+0000 7f8e5fad9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e58106f10 0x7f8e58074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.559+0000 7f8e5fad9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8e58111f30 con 0x7f8e58102790 2024-09-11T23:30:39.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.559+0000 7f8e5fad9640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8e58111db0 con 0x7f8e58101de0 2024-09-11T23:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.559+0000 7f8e5fad9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8e581120b0 con 0x7f8e58106f10 2024-09-11T23:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.560+0000 7f8e5d074640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e58102790 0x7f8e58074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.560+0000 7f8e5d074640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e58102790 0x7f8e58074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37062/0 (socket says 172.21.15.43:37062) 2024-09-11T23:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.560+0000 7f8e5d074640 1 -- 172.21.15.43:0/2021733648 learned_addr learned my addr 172.21.15.43:0/2021733648 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.560+0000 7f8e5d875640 1 --2- 172.21.15.43:0/2021733648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e58101de0 0x7f8e580744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.560+0000 7f8e5e076640 1 --2- 172.21.15.43:0/2021733648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e58106f10 0x7f8e58074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.560+0000 7f8e5d074640 1 -- 172.21.15.43:0/2021733648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e58106f10 msgr2=0x7f8e58074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.560+0000 7f8e5d074640 1 --2- 172.21.15.43:0/2021733648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e58106f10 0x7f8e58074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.560+0000 7f8e5d074640 1 -- 172.21.15.43:0/2021733648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e58101de0 msgr2=0x7f8e580744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.560+0000 7f8e5d074640 1 --2- 172.21.15.43:0/2021733648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e58101de0 0x7f8e580744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.560+0000 7f8e5d074640 1 -- 172.21.15.43:0/2021733648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e581a6880 con 0x7f8e58102790 2024-09-11T23:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.560+0000 7f8e5d074640 1 --2- 172.21.15.43:0/2021733648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e58102790 0x7f8e58074a30 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8e48006010 tx=0x7f8e480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:39.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.560+0000 7f8e4effd640 1 -- 172.21.15.43:0/2021733648 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4803f030 con 0x7f8e58102790 2024-09-11T23:30:39.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.560+0000 7f8e4effd640 1 -- 172.21.15.43:0/2021733648 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e48003ce0 con 0x7f8e58102790 2024-09-11T23:30:39.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.560+0000 7f8e5fad9640 1 -- 172.21.15.43:0/2021733648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e581a6b10 con 0x7f8e58102790 2024-09-11T23:30:39.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.560+0000 7f8e4effd640 1 -- 172.21.15.43:0/2021733648 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48035070 con 0x7f8e58102790 2024-09-11T23:30:39.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.560+0000 7f8e5fad9640 1 -- 172.21.15.43:0/2021733648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e581a7040 con 0x7f8e58102790 2024-09-11T23:30:39.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.563+0000 7f8e4effd640 1 -- 172.21.15.43:0/2021733648 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8e48042030 con 0x7f8e58102790 2024-09-11T23:30:39.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.563+0000 7f8e61eb2740 1 Processor -- start 2024-09-11T23:30:39.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.563+0000 7f8e61eb2740 1 -- start start 2024-09-11T23:30:39.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.563+0000 7f8e61eb2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5576b8b4bb40 0x5576b8b4a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:39.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.563+0000 7f8e61eb2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b4a5d0 0x5576b8b4aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:39.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.563+0000 7f8e61eb2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5576b8b55370 0x5576b8b57760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:39.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.563+0000 7f8e61eb2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5576b8ab02f0 con 0x5576b8b4a5d0 2024-09-11T23:30:39.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.563+0000 7f8e61eb2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5576b8a57cf0 con 0x5576b8b55370 2024-09-11T23:30:39.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.563+0000 7f8e61eb2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5576b8aac2b0 con 0x5576b8b4bb40 2024-09-11T23:30:39.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.563+0000 7f8e5d875640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b4a5d0 0x5576b8b4aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:39.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.563+0000 7f8e5d875640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b4a5d0 0x5576b8b4aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37064/0 (socket says 172.21.15.43:37064) 2024-09-11T23:30:39.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.563+0000 7f8e5d875640 1 -- 172.21.15.43:0/3603595859 learned_addr learned my addr 172.21.15.43:0/3603595859 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:39.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.563+0000 7f8e5e076640 1 --2- 172.21.15.43:0/3603595859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5576b8b4bb40 0x5576b8b4a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:39.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.563+0000 7f8e5d074640 1 --2- 172.21.15.43:0/3603595859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5576b8b55370 0x5576b8b57760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:39.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.564+0000 7f8e5d875640 1 -- 172.21.15.43:0/3603595859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5576b8b4bb40 msgr2=0x5576b8b4a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:39.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.564+0000 7f8e5d875640 1 --2- 172.21.15.43:0/3603595859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5576b8b4bb40 0x5576b8b4a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.564+0000 7f8e5d875640 1 -- 172.21.15.43:0/3603595859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5576b8b55370 msgr2=0x5576b8b57760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:39.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.564+0000 7f8e5d875640 1 --2- 172.21.15.43:0/3603595859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5576b8b55370 0x5576b8b57760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.564+0000 7f8e4effd640 1 --2- 172.21.15.43:0/2021733648 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f8e440a2d70 0x7f8e440a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:39.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.564+0000 7f8e5d875640 1 -- 172.21.15.43:0/3603595859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576b8a3c3a0 con 0x5576b8b4a5d0 2024-09-11T23:30:39.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.564+0000 7f8e5e076640 1 --2- 172.21.15.43:0/2021733648 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f8e440a2d70 0x7f8e440a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:39.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.564+0000 7f8e4effd640 1 -- 172.21.15.43:0/2021733648 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f8e480ef7b0 con 0x7f8e58102790 2024-09-11T23:30:39.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.564+0000 7f8e5d875640 1 --2- 172.21.15.43:0/3603595859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b4a5d0 0x5576b8b4aa10 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400ad00 tx=0x7f8e5400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb03ac70f0517f47 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:39.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.564+0000 7f8e43fff640 1 -- 172.21.15.43:0/3603595859 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5401d070 con 0x5576b8b4a5d0 2024-09-11T23:30:39.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.564+0000 7f8e43fff640 1 -- 172.21.15.43:0/3603595859 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e54018b20 con 0x5576b8b4a5d0 2024-09-11T23:30:39.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.564+0000 7f8e43fff640 1 -- 172.21.15.43:0/3603595859 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54010c20 con 0x5576b8b4a5d0 2024-09-11T23:30:39.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.565+0000 7f8e61eb2740 1 -- 172.21.15.43:0/3603595859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b4a5d0 msgr2=0x5576b8b4aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:39.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.565+0000 7f8e5e076640 1 --2- 172.21.15.43:0/2021733648 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f8e440a2d70 0x7f8e440a5230 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8e50005fd0 tx=0x7f8e500096a0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:39.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.565+0000 7f8e61eb2740 1 --2- 172.21.15.43:0/3603595859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b4a5d0 0x5576b8b4aa10 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400ad00 tx=0x7f8e5400f790 comp rx=0 tx=0).stop 2024-09-11T23:30:39.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.565+0000 7f8e61eb2740 1 -- 172.21.15.43:0/3603595859 shutdown_connections 2024-09-11T23:30:39.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.565+0000 7f8e61eb2740 1 --2- 172.21.15.43:0/3603595859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5576b8b55370 0x5576b8b57760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.565+0000 7f8e61eb2740 1 --2- 172.21.15.43:0/3603595859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b4a5d0 0x5576b8b4aa10 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.565+0000 7f8e61eb2740 1 --2- 172.21.15.43:0/3603595859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5576b8b4bb40 0x5576b8b4a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.565+0000 7f8e61eb2740 1 -- 172.21.15.43:0/3603595859 >> 172.21.15.43:0/3603595859 conn(0x5576b8b13310 msgr2=0x5576b8b46c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:39.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.565+0000 7f8e61eb2740 1 -- 172.21.15.43:0/3603595859 shutdown_connections 2024-09-11T23:30:39.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.565+0000 7f8e61eb2740 1 -- 172.21.15.43:0/3603595859 wait complete. 2024-09-11T23:30:39.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.565+0000 7f8e61eb2740 1 -- start start 2024-09-11T23:30:39.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.565+0000 7f8e61eb2740 1 Processor -- start 2024-09-11T23:30:39.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e61eb2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5576b8b4a5d0 0x5576b8b54d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:39.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e61eb2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5576b8b4bb40 0x5576b8b52010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:39.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e61eb2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b55370 0x5576b8b52600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:39.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e61eb2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5576b8b58070 con 0x5576b8b55370 2024-09-11T23:30:39.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e61eb2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5576b8a6b060 con 0x5576b8b4a5d0 2024-09-11T23:30:39.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e61eb2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5576b8b581f0 con 0x5576b8b4bb40 2024-09-11T23:30:39.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e5e076640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b55370 0x5576b8b52600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:39.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e5e076640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b55370 0x5576b8b52600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37068/0 (socket says 172.21.15.43:37068) 2024-09-11T23:30:39.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e5e076640 1 -- 172.21.15.43:0/119271137 learned_addr learned my addr 172.21.15.43:0/119271137 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:39.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e5d074640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5576b8b4bb40 0x5576b8b52010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:39.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e61eb2740 10 client.0 _ll_register_callbacks cb 0x5576b87be8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:30:39.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e5d875640 1 --2- 172.21.15.43:0/119271137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5576b8b4a5d0 0x5576b8b54d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:39.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e5e076640 1 -- 172.21.15.43:0/119271137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5576b8b4bb40 msgr2=0x5576b8b52010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:39.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e5e076640 1 --2- 172.21.15.43:0/119271137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5576b8b4bb40 0x5576b8b52010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e5e076640 1 -- 172.21.15.43:0/119271137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5576b8b4a5d0 msgr2=0x5576b8b54d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:39.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e5e076640 1 --2- 172.21.15.43:0/119271137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5576b8b4a5d0 0x5576b8b54d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e5e076640 1 -- 172.21.15.43:0/119271137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576b88213c0 con 0x5576b8b55370 2024-09-11T23:30:39.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e5e076640 1 --2- 172.21.15.43:0/119271137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b55370 0x5576b8b52600 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8e500107c0 tx=0x7f8e50004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178f2afb1261375f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:39.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e417fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:30:39.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e417fa640 1 -- 172.21.15.43:0/119271137 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e50004b20 con 0x5576b8b55370 2024-09-11T23:30:39.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e417fa640 1 -- 172.21.15.43:0/119271137 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f8e50017030 con 0x5576b8b55370 2024-09-11T23:30:39.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.566+0000 7f8e61eb2740 1 -- 172.21.15.43:0/119271137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e54018b20 con 0x5576b8b55370 2024-09-11T23:30:39.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.567+0000 7f8e417fa640 1 -- 172.21.15.43:0/119271137 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8e50004cc0 con 0x5576b8b55370 2024-09-11T23:30:39.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.567+0000 7f8e61eb2740 10 client.16830 fetch_fsmap learned FSMap version 32 2024-09-11T23:30:39.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.567+0000 7f8e61eb2740 1 -- 172.21.15.43:0/119271137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5576b8b63b70 con 0x5576b8b55370 2024-09-11T23:30:39.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.567+0000 7f8e417fa640 1 -- 172.21.15.43:0/119271137 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e54018b20 con 0x5576b8b55370 2024-09-11T23:30:39.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.567+0000 7f8e417fa640 1 -- 172.21.15.43:0/119271137 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8e500279f0 con 0x5576b8b55370 2024-09-11T23:30:39.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.567+0000 7f8e61eb2740 10 client.16830 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:30:39.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.567+0000 7f8e61eb2740 10 client.16830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:30:39.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.567+0000 7f8e61eb2740 20 client.16830 populate_metadata read hostname 'smithi043' 2024-09-11T23:30:39.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.568+0000 7f8e61eb2740 1 --2- 172.21.15.43:0/119271137 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5576b8b65030 0x5576b8b674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:39.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.568+0000 7f8e61eb2740 4 client.16830 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:30:39.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.568+0000 7f8e61eb2740 1 -- 172.21.15.43:0/119271137 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576b8abd860 con 0x5576b8b65030 2024-09-11T23:30:39.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.568+0000 7f8e5d875640 1 --2- 172.21.15.43:0/119271137 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5576b8b65030 0x5576b8b674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:39.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.569+0000 7f8e5d875640 1 --2- 172.21.15.43:0/119271137 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5576b8b65030 0x5576b8b674a0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b378e47f3de210a4 server_cookie=22b49c13f413bf08 in_seq=0 out_seq=0 2024-09-11T23:30:39.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.569+0000 7f8e417fa640 10 client.16830 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:30:39.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.569+0000 7f8e417fa640 1 -- 172.21.15.43:0/119271137 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5576b8abd860 con 0x5576b8b65030 2024-09-11T23:30:39.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.569+0000 7f8e417fa640 10 client.16830 handle_command_reply: tid=0 2024-09-11T23:30:39.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.569+0000 7f8e61eb2740 1 client.16830 shutdown 2024-09-11T23:30:39.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.570+0000 7f8e61eb2740 1 -- 172.21.15.43:0/119271137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b55370 msgr2=0x5576b8b52600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:39.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.570+0000 7f8e61eb2740 1 --2- 172.21.15.43:0/119271137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b55370 0x5576b8b52600 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8e500107c0 tx=0x7f8e50004ab0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.570+0000 7f8e61eb2740 1 -- 172.21.15.43:0/119271137 shutdown_connections 2024-09-11T23:30:39.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.570+0000 7f8e61eb2740 1 --2- 172.21.15.43:0/119271137 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5576b8b65030 0x5576b8b674a0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.570+0000 7f8e61eb2740 1 --2- 172.21.15.43:0/119271137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b55370 0x5576b8b52600 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.570+0000 7f8e61eb2740 1 --2- 172.21.15.43:0/119271137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5576b8b4bb40 0x5576b8b52010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.570+0000 7f8e61eb2740 1 --2- 172.21.15.43:0/119271137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5576b8b4a5d0 0x5576b8b54d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.570+0000 7f8e61eb2740 1 -- 172.21.15.43:0/119271137 >> 172.21.15.43:0/119271137 conn(0x5576b8b13310 msgr2=0x5576b8b4c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:39.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.570+0000 7f8e417fa640 0 client.16830 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:30:39.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.570+0000 7f8e61eb2740 1 -- 172.21.15.43:0/119271137 shutdown_connections 2024-09-11T23:30:39.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.570+0000 7f8e61eb2740 1 -- 172.21.15.43:0/119271137 wait complete. 2024-09-11T23:30:39.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.570+0000 7f8e61eb2740 20 client.16830 trim_cache size 0 max 16384 2024-09-11T23:30:39.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.583+0000 7f8e61eb2740 1 Processor -- start 2024-09-11T23:30:39.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.583+0000 7f8e61eb2740 1 -- start start 2024-09-11T23:30:39.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.583+0000 7f8e61eb2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5576b8b4a5d0 0x5576b8b46e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:39.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.583+0000 7f8e61eb2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b4bb40 0x5576b8b61310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:39.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.583+0000 7f8e61eb2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5576b8b55370 0x5576b8b6cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:39.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.583+0000 7f8e61eb2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8e581b2490 con 0x5576b8b4bb40 2024-09-11T23:30:39.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.583+0000 7f8e61eb2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8e581afd30 con 0x5576b8b4a5d0 2024-09-11T23:30:39.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.583+0000 7f8e61eb2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8e581adba0 con 0x5576b8b55370 2024-09-11T23:30:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.583+0000 7f8e5d074640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b4bb40 0x5576b8b61310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.583+0000 7f8e5d074640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b4bb40 0x5576b8b61310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37084/0 (socket says 172.21.15.43:37084) 2024-09-11T23:30:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.583+0000 7f8e5d074640 1 -- 172.21.15.43:0/208228369 learned_addr learned my addr 172.21.15.43:0/208228369 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.584+0000 7f8e5d875640 1 --2- 172.21.15.43:0/208228369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5576b8b4a5d0 0x5576b8b46e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.584+0000 7f8e5e076640 1 --2- 172.21.15.43:0/208228369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5576b8b55370 0x5576b8b6cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.584+0000 7f8e5d074640 1 -- 172.21.15.43:0/208228369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5576b8b55370 msgr2=0x5576b8b6cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.584+0000 7f8e5d074640 1 --2- 172.21.15.43:0/208228369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5576b8b55370 0x5576b8b6cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.584+0000 7f8e5d074640 1 -- 172.21.15.43:0/208228369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5576b8b4a5d0 msgr2=0x5576b8b46e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.584+0000 7f8e5d074640 1 --2- 172.21.15.43:0/208228369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5576b8b4a5d0 0x5576b8b46e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.584+0000 7f8e5d074640 1 -- 172.21.15.43:0/208228369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576b8b67b20 con 0x5576b8b4bb40 2024-09-11T23:30:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.584+0000 7f8e5e076640 1 --2- 172.21.15.43:0/208228369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5576b8b55370 0x5576b8b6cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:30:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.584+0000 7f8e5d875640 1 --2- 172.21.15.43:0/208228369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5576b8b4a5d0 0x5576b8b46e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:30:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.584+0000 7f8e5d074640 1 --2- 172.21.15.43:0/208228369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b4bb40 0x5576b8b61310 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8e480bb970 tx=0x7f8e4803b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893648b8f0ea08c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.584+0000 7f8e437fe640 1 -- 172.21.15.43:0/208228369 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48049050 con 0x5576b8b4bb40 2024-09-11T23:30:39.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.584+0000 7f8e437fe640 1 -- 172.21.15.43:0/208228369 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e4803b510 con 0x5576b8b4bb40 2024-09-11T23:30:39.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.584+0000 7f8e437fe640 1 -- 172.21.15.43:0/208228369 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e480c48e0 con 0x5576b8b4bb40 2024-09-11T23:30:39.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.585+0000 7f8e61eb2740 1 -- 172.21.15.43:0/208228369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b4bb40 msgr2=0x5576b8b61310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:39.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.585+0000 7f8e61eb2740 1 --2- 172.21.15.43:0/208228369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b4bb40 0x5576b8b61310 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8e480bb970 tx=0x7f8e4803b460 comp rx=0 tx=0).stop 2024-09-11T23:30:39.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.585+0000 7f8e61eb2740 1 -- 172.21.15.43:0/208228369 shutdown_connections 2024-09-11T23:30:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.585+0000 7f8e61eb2740 1 --2- 172.21.15.43:0/208228369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5576b8b55370 0x5576b8b6cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.585+0000 7f8e61eb2740 1 --2- 172.21.15.43:0/208228369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b4bb40 0x5576b8b61310 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.585+0000 7f8e61eb2740 1 --2- 172.21.15.43:0/208228369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5576b8b4a5d0 0x5576b8b46e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.585+0000 7f8e61eb2740 1 -- 172.21.15.43:0/208228369 >> 172.21.15.43:0/208228369 conn(0x5576b8b13310 msgr2=0x5576b8b46700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.585+0000 7f8e61eb2740 1 -- 172.21.15.43:0/208228369 shutdown_connections 2024-09-11T23:30:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.585+0000 7f8e61eb2740 1 -- 172.21.15.43:0/208228369 wait complete. 2024-09-11T23:30:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.585+0000 7f8e61eb2740 1 -- start start 2024-09-11T23:30:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.585+0000 7f8e61eb2740 1 Processor -- start 2024-09-11T23:30:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.586+0000 7f8e61eb2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5576b8b4a5d0 0x5576b8b59370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.586+0000 7f8e61eb2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5576b8b4bb40 0x5576b8b598b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.586+0000 7f8e61eb2740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b55370 0x5576b8b59df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.586+0000 7f8e61eb2740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8e581ae1f0 con 0x5576b8b55370 2024-09-11T23:30:39.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.586+0000 7f8e61eb2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8e581af660 con 0x5576b8b4bb40 2024-09-11T23:30:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.586+0000 7f8e61eb2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8e581ae8c0 con 0x5576b8b4a5d0 2024-09-11T23:30:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.586+0000 7f8e61eb2740 10 client.0 _ll_register_callbacks cb 0x7f8e50010650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:30:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.586+0000 7f8e5e076640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b55370 0x5576b8b59df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.586+0000 7f8e5e076640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b55370 0x5576b8b59df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37100/0 (socket says 172.21.15.43:37100) 2024-09-11T23:30:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.586+0000 7f8e5e076640 1 -- 172.21.15.43:0/627791186 learned_addr learned my addr 172.21.15.43:0/627791186 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.586+0000 7f8e5d875640 1 --2- 172.21.15.43:0/627791186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5576b8b4a5d0 0x5576b8b59370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.586+0000 7f8e5d074640 1 --2- 172.21.15.43:0/627791186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5576b8b4bb40 0x5576b8b598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.586+0000 7f8e5e076640 1 -- 172.21.15.43:0/627791186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5576b8b4a5d0 msgr2=0x5576b8b59370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.586+0000 7f8e5e076640 1 --2- 172.21.15.43:0/627791186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5576b8b4a5d0 0x5576b8b59370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.586+0000 7f8e5e076640 1 -- 172.21.15.43:0/627791186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5576b8b4bb40 msgr2=0x5576b8b598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.586+0000 7f8e5e076640 1 --2- 172.21.15.43:0/627791186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5576b8b4bb40 0x5576b8b598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.586+0000 7f8e5e076640 1 -- 172.21.15.43:0/627791186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576b8bcef30 con 0x5576b8b55370 2024-09-11T23:30:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.586+0000 7f8e5e076640 1 --2- 172.21.15.43:0/627791186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b55370 0x5576b8b59df0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f8e50011db0 tx=0x7f8e5001f2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8153d8a14cb2c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.586+0000 7f8e417fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:30:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.587+0000 7f8e417fa640 1 -- 172.21.15.43:0/627791186 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e50004b20 con 0x5576b8b55370 2024-09-11T23:30:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.587+0000 7f8e417fa640 1 -- 172.21.15.43:0/627791186 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f8e5001cc20 con 0x5576b8b55370 2024-09-11T23:30:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.587+0000 7f8e61eb2740 1 -- 172.21.15.43:0/627791186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5576b8ab7d70 con 0x5576b8b55370 2024-09-11T23:30:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.587+0000 7f8e417fa640 1 -- 172.21.15.43:0/627791186 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8e5001ce20 con 0x5576b8b55370 2024-09-11T23:30:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.587+0000 7f8e61eb2740 10 client.16836 fetch_fsmap learned FSMap version 32 2024-09-11T23:30:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.587+0000 7f8e61eb2740 1 -- 172.21.15.43:0/627791186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5576b8bd0210 con 0x5576b8b55370 2024-09-11T23:30:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.587+0000 7f8e417fa640 1 -- 172.21.15.43:0/627791186 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5576b8ab7d70 con 0x5576b8b55370 2024-09-11T23:30:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.587+0000 7f8e417fa640 1 -- 172.21.15.43:0/627791186 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8e5002f420 con 0x5576b8b55370 2024-09-11T23:30:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.587+0000 7f8e61eb2740 10 client.16836 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:30:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.587+0000 7f8e61eb2740 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:30:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.587+0000 7f8e61eb2740 20 client.16836 populate_metadata read hostname 'smithi043' 2024-09-11T23:30:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.587+0000 7f8e61eb2740 1 --2- 172.21.15.43:0/627791186 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5576b8bd0920 0x5576b8bd2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:39.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.587+0000 7f8e61eb2740 4 client.16836 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:30:39.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.587+0000 7f8e61eb2740 1 -- 172.21.15.43:0/627791186 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576b8a34170 con 0x5576b8bd0920 2024-09-11T23:30:39.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.588+0000 7f8e5d875640 1 --2- 172.21.15.43:0/627791186 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5576b8bd0920 0x5576b8bd2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:39.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.588+0000 7f8e5d875640 1 --2- 172.21.15.43:0/627791186 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5576b8bd0920 0x5576b8bd2d40 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d6b90236bdb2ac6 server_cookie=a0d6d57fed5ca786 in_seq=0 out_seq=0 2024-09-11T23:30:39.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.588+0000 7f8e417fa640 10 client.16836 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:30:39.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.588+0000 7f8e417fa640 1 -- 172.21.15.43:0/627791186 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576b8a34170 con 0x5576b8bd0920 2024-09-11T23:30:39.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.588+0000 7f8e417fa640 10 client.16836 handle_command_reply: tid=0 2024-09-11T23:30:39.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.588+0000 7f8e61eb2740 1 client.16836 shutdown 2024-09-11T23:30:39.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.588+0000 7f8e61eb2740 1 -- 172.21.15.43:0/627791186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b55370 msgr2=0x5576b8b59df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:39.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.588+0000 7f8e61eb2740 1 --2- 172.21.15.43:0/627791186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b55370 0x5576b8b59df0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f8e50011db0 tx=0x7f8e5001f2f0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.588+0000 7f8e61eb2740 1 -- 172.21.15.43:0/627791186 shutdown_connections 2024-09-11T23:30:39.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.588+0000 7f8e61eb2740 1 --2- 172.21.15.43:0/627791186 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5576b8bd0920 0x5576b8bd2d40 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.588+0000 7f8e61eb2740 1 --2- 172.21.15.43:0/627791186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5576b8b55370 0x5576b8b59df0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.588+0000 7f8e61eb2740 1 --2- 172.21.15.43:0/627791186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5576b8b4bb40 0x5576b8b598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.588+0000 7f8e61eb2740 1 --2- 172.21.15.43:0/627791186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5576b8b4a5d0 0x5576b8b59370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.588+0000 7f8e61eb2740 1 -- 172.21.15.43:0/627791186 >> 172.21.15.43:0/627791186 conn(0x5576b8b13310 msgr2=0x5576b8b53870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.588+0000 7f8e417fa640 0 client.16836 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:30:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.588+0000 7f8e61eb2740 1 -- 172.21.15.43:0/627791186 shutdown_connections 2024-09-11T23:30:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.588+0000 7f8e61eb2740 1 -- 172.21.15.43:0/627791186 wait complete. 2024-09-11T23:30:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.588+0000 7f8e61eb2740 20 client.16836 trim_cache size 0 max 16384 2024-09-11T23:30:39.595 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:30:39.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.589+0000 7f8e5fad9640 1 -- 172.21.15.43:0/2021733648 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f8e440a2d70 msgr2=0x7f8e440a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:39.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.589+0000 7f8e5fad9640 1 --2- 172.21.15.43:0/2021733648 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f8e440a2d70 0x7f8e440a5230 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8e50005fd0 tx=0x7f8e500096a0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.589+0000 7f8e5fad9640 1 -- 172.21.15.43:0/2021733648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e58102790 msgr2=0x7f8e58074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:39.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.589+0000 7f8e5fad9640 1 --2- 172.21.15.43:0/2021733648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e58102790 0x7f8e58074a30 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8e48006010 tx=0x7f8e480042f0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.589+0000 7f8e5fad9640 1 -- 172.21.15.43:0/2021733648 shutdown_connections 2024-09-11T23:30:39.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.589+0000 7f8e5fad9640 1 --2- 172.21.15.43:0/2021733648 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f8e440a2d70 0x7f8e440a5230 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.589+0000 7f8e5fad9640 1 --2- 172.21.15.43:0/2021733648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e58106f10 0x7f8e58074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.589+0000 7f8e5fad9640 1 --2- 172.21.15.43:0/2021733648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e58102790 0x7f8e58074a30 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.589+0000 7f8e5fad9640 1 --2- 172.21.15.43:0/2021733648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e58101de0 0x7f8e580744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:39.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.589+0000 7f8e5fad9640 1 -- 172.21.15.43:0/2021733648 >> 172.21.15.43:0/2021733648 conn(0x7f8e580fd8c0 msgr2=0x7f8e580fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:39.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.589+0000 7f8e5fad9640 1 -- 172.21.15.43:0/2021733648 shutdown_connections 2024-09-11T23:30:39.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:39.589+0000 7f8e5fad9640 1 -- 172.21.15.43:0/2021733648 wait complete. 2024-09-11T23:30:40.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:39 smithi090 ceph-mon[35130]: pgmap v271: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 16 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:30:40.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:39 smithi097 ceph-mon[35562]: pgmap v271: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 16 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:30:40.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:39 smithi043 ceph-mon[30950]: pgmap v271: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 16 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:30:41.400 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:30:41.401 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:30:41.674 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:30:42.049 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:41 smithi043 ceph-mon[30950]: pgmap v272: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T23:30:42.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.130+0000 7f5933912640 1 -- 172.21.15.43:0/3883292653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c106f10 msgr2=0x7f592c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:42.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.130+0000 7f5933912640 1 --2- 172.21.15.43:0/3883292653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c106f10 0x7f592c10ea90 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f591c009ca0 tx=0x7f591c02a8c0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.130+0000 7f5933912640 1 -- 172.21.15.43:0/3883292653 shutdown_connections 2024-09-11T23:30:42.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.130+0000 7f5933912640 1 --2- 172.21.15.43:0/3883292653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c106f10 0x7f592c10ea90 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.130+0000 7f5933912640 1 --2- 172.21.15.43:0/3883292653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f592c102790 0x7f592c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.130+0000 7f5933912640 1 --2- 172.21.15.43:0/3883292653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f592c101de0 0x7f592c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.130+0000 7f5933912640 1 -- 172.21.15.43:0/3883292653 >> 172.21.15.43:0/3883292653 conn(0x7f592c0fd8c0 msgr2=0x7f592c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:42.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.130+0000 7f5933912640 1 -- 172.21.15.43:0/3883292653 shutdown_connections 2024-09-11T23:30:42.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.130+0000 7f5933912640 1 -- 172.21.15.43:0/3883292653 wait complete. 2024-09-11T23:30:42.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.131+0000 7f5933912640 1 Processor -- start 2024-09-11T23:30:42.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.131+0000 7f5933912640 1 -- start start 2024-09-11T23:30:42.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.131+0000 7f5933912640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f592c101de0 0x7f592c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:42.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.131+0000 7f5933912640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f592c102790 0x7f592c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:42.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.131+0000 7f5933912640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c106f10 0x7f592c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:42.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.131+0000 7f5933912640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f592c111f70 con 0x7f592c106f10 2024-09-11T23:30:42.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.131+0000 7f5933912640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f592c111df0 con 0x7f592c101de0 2024-09-11T23:30:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.131+0000 7f5933912640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f592c1120f0 con 0x7f592c102790 2024-09-11T23:30:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.132+0000 7f5931eaf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c106f10 0x7f592c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.132+0000 7f5931eaf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c106f10 0x7f592c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46894/0 (socket says 172.21.15.43:46894) 2024-09-11T23:30:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.132+0000 7f5931eaf640 1 -- 172.21.15.43:0/435569364 learned_addr learned my addr 172.21.15.43:0/435569364 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.132+0000 7f5930ead640 1 --2- 172.21.15.43:0/435569364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f592c102790 0x7f592c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.132+0000 7f59316ae640 1 --2- 172.21.15.43:0/435569364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f592c101de0 0x7f592c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.132+0000 7f5931eaf640 1 -- 172.21.15.43:0/435569364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f592c102790 msgr2=0x7f592c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.132+0000 7f5931eaf640 1 --2- 172.21.15.43:0/435569364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f592c102790 0x7f592c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.132+0000 7f5931eaf640 1 -- 172.21.15.43:0/435569364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f592c101de0 msgr2=0x7f592c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.132+0000 7f5931eaf640 1 --2- 172.21.15.43:0/435569364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f592c101de0 0x7f592c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.132+0000 7f5931eaf640 1 -- 172.21.15.43:0/435569364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592c1a68b0 con 0x7f592c106f10 2024-09-11T23:30:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.132+0000 7f5931eaf640 1 --2- 172.21.15.43:0/435569364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c106f10 0x7f592c074f70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f591c031a20 tx=0x7f591c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:42.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.132+0000 7f592a7fc640 1 -- 172.21.15.43:0/435569364 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c003c30 con 0x7f592c106f10 2024-09-11T23:30:42.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.132+0000 7f592a7fc640 1 -- 172.21.15.43:0/435569364 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f591c003dd0 con 0x7f592c106f10 2024-09-11T23:30:42.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.132+0000 7f5933912640 1 -- 172.21.15.43:0/435569364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f592c1a6b40 con 0x7f592c106f10 2024-09-11T23:30:42.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.132+0000 7f592a7fc640 1 -- 172.21.15.43:0/435569364 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c02fc40 con 0x7f592c106f10 2024-09-11T23:30:42.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.132+0000 7f5933912640 1 -- 172.21.15.43:0/435569364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f592c1a7080 con 0x7f592c106f10 2024-09-11T23:30:42.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.135+0000 7f592a7fc640 1 -- 172.21.15.43:0/435569364 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f591c004050 con 0x7f592c106f10 2024-09-11T23:30:42.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.135+0000 7f5935ceb740 1 Processor -- start 2024-09-11T23:30:42.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.135+0000 7f5935ceb740 1 -- start start 2024-09-11T23:30:42.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.135+0000 7f5935ceb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9798c3b60 0x55f9798c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:42.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.135+0000 7f5935ceb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9798c4480 0x55f9798ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:42.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.135+0000 7f5935ceb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798ced40 0x55f9798d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:42.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.135+0000 7f5935ceb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f979829280 con 0x55f9798ced40 2024-09-11T23:30:42.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.135+0000 7f5935ceb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f9797d0b10 con 0x55f9798c4480 2024-09-11T23:30:42.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.135+0000 7f5935ceb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f979825240 con 0x55f9798c3b60 2024-09-11T23:30:42.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.135+0000 7f592a7fc640 1 --2- 172.21.15.43:0/435569364 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f59140a2cf0 0x7f59140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:42.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.135+0000 7f5930ead640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798ced40 0x55f9798d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:42.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.136+0000 7f59316ae640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9798c4480 0x55f9798ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:42.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.136+0000 7f5930ead640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798ced40 0x55f9798d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46902/0 (socket says 172.21.15.43:46902) 2024-09-11T23:30:42.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.136+0000 7f5930ead640 1 -- 172.21.15.43:0/2480910463 learned_addr learned my addr 172.21.15.43:0/2480910463 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:42.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.136+0000 7f5931eaf640 1 --2- 172.21.15.43:0/435569364 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f59140a2cf0 0x7f59140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:42.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.136+0000 7f592a7fc640 1 -- 172.21.15.43:0/435569364 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f591c0f0750 con 0x7f592c106f10 2024-09-11T23:30:42.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.136+0000 7f5930ead640 1 --2- 172.21.15.43:0/2480910463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9798c3b60 0x55f9798c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.136+0000 7f59316ae640 1 -- 172.21.15.43:0/2480910463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9798c3b60 msgr2=0x55f9798c3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.136+0000 7f59316ae640 1 --2- 172.21.15.43:0/2480910463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9798c3b60 0x55f9798c3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.136+0000 7f59316ae640 1 -- 172.21.15.43:0/2480910463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798ced40 msgr2=0x55f9798d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.136+0000 7f59316ae640 1 --2- 172.21.15.43:0/2480910463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798ced40 0x55f9798d1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.136+0000 7f59316ae640 1 -- 172.21.15.43:0/2480910463 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9797dd1f0 con 0x55f9798c4480 2024-09-11T23:30:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.136+0000 7f5930ead640 1 --2- 172.21.15.43:0/2480910463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798ced40 0x55f9798d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:30:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.136+0000 7f5931eaf640 1 --2- 172.21.15.43:0/435569364 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f59140a2cf0 0x7f59140a51b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f591c0bcce0 tx=0x7f591c08ed10 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.136+0000 7f59316ae640 1 --2- 172.21.15.43:0/2480910463 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9798c4480 0x55f9798ce800 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f592400bbf0 tx=0x7f59240107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de91a1fddc014236 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.136+0000 7f59137fe640 1 -- 172.21.15.43:0/2480910463 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924004030 con 0x55f9798c4480 2024-09-11T23:30:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.136+0000 7f59137fe640 1 -- 172.21.15.43:0/2480910463 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59240041d0 con 0x55f9798c4480 2024-09-11T23:30:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.136+0000 7f59137fe640 1 -- 172.21.15.43:0/2480910463 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59240117b0 con 0x55f9798c4480 2024-09-11T23:30:42.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.137+0000 7f5935ceb740 1 -- 172.21.15.43:0/2480910463 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9798c4480 msgr2=0x55f9798ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.137+0000 7f5935ceb740 1 --2- 172.21.15.43:0/2480910463 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9798c4480 0x55f9798ce800 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f592400bbf0 tx=0x7f59240107a0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.137+0000 7f5935ceb740 1 -- 172.21.15.43:0/2480910463 shutdown_connections 2024-09-11T23:30:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.137+0000 7f5935ceb740 1 --2- 172.21.15.43:0/2480910463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798ced40 0x55f9798d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.137+0000 7f5935ceb740 1 --2- 172.21.15.43:0/2480910463 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9798c4480 0x55f9798ce800 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.137+0000 7f5935ceb740 1 --2- 172.21.15.43:0/2480910463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9798c3b60 0x55f9798c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.137+0000 7f5935ceb740 1 -- 172.21.15.43:0/2480910463 >> 172.21.15.43:0/2480910463 conn(0x55f9798c00f0 msgr2=0x55f9798c04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.137+0000 7f5935ceb740 1 -- 172.21.15.43:0/2480910463 shutdown_connections 2024-09-11T23:30:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.137+0000 7f5935ceb740 1 -- 172.21.15.43:0/2480910463 wait complete. 2024-09-11T23:30:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.138+0000 7f5935ceb740 1 -- start start 2024-09-11T23:30:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.138+0000 7f5935ceb740 1 Processor -- start 2024-09-11T23:30:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.138+0000 7f5935ceb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9798c3b60 0x55f9798ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.138+0000 7f5935ceb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798c4480 0x55f9798cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.138+0000 7f5935ceb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9798ced40 0x55f9798cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:42.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.138+0000 7f5935ceb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f9798d1910 con 0x55f9798c4480 2024-09-11T23:30:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.138+0000 7f5935ceb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f9797e3f10 con 0x55f9798c3b60 2024-09-11T23:30:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.138+0000 7f5935ceb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f9798d1a90 con 0x55f9798ced40 2024-09-11T23:30:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.138+0000 7f5935ceb740 10 client.0 _ll_register_callbacks cb 0x55f9797aca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:30:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.138+0000 7f59316ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798c4480 0x55f9798cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.138+0000 7f59316ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798c4480 0x55f9798cbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46904/0 (socket says 172.21.15.43:46904) 2024-09-11T23:30:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.138+0000 7f59316ae640 1 -- 172.21.15.43:0/2525635853 learned_addr learned my addr 172.21.15.43:0/2525635853 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.138+0000 7f5930ead640 1 --2- 172.21.15.43:0/2525635853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9798c3b60 0x55f9798ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.138+0000 7f5930ead640 1 --2- 172.21.15.43:0/2525635853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9798ced40 0x55f9798cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.139+0000 7f59316ae640 1 -- 172.21.15.43:0/2525635853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9798ced40 msgr2=0x55f9798cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.139+0000 7f59316ae640 1 --2- 172.21.15.43:0/2525635853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9798ced40 0x55f9798cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.139+0000 7f59316ae640 1 -- 172.21.15.43:0/2525635853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9798c3b60 msgr2=0x55f9798ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.139+0000 7f59316ae640 1 --2- 172.21.15.43:0/2525635853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9798c3b60 0x55f9798ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.139+0000 7f59316ae640 1 -- 172.21.15.43:0/2525635853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f97959a6a0 con 0x55f9798c4480 2024-09-11T23:30:42.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.139+0000 7f59316ae640 1 --2- 172.21.15.43:0/2525635853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798c4480 0x55f9798cbb50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5924000c00 tx=0x7f592400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68ae702d860184d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:42.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.139+0000 7f5910ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:30:42.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.139+0000 7f5910ff9640 1 -- 172.21.15.43:0/2525635853 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592402a060 con 0x55f9798c4480 2024-09-11T23:30:42.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.139+0000 7f5910ff9640 1 -- 172.21.15.43:0/2525635853 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f5924004380 con 0x55f9798c4480 2024-09-11T23:30:42.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.139+0000 7f5935ceb740 1 -- 172.21.15.43:0/2525635853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59240041d0 con 0x55f9798c4480 2024-09-11T23:30:42.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.139+0000 7f5910ff9640 1 -- 172.21.15.43:0/2525635853 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f59240035f0 con 0x55f9798c4480 2024-09-11T23:30:42.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.139+0000 7f5935ceb740 10 client.16848 fetch_fsmap learned FSMap version 32 2024-09-11T23:30:42.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.139+0000 7f5935ceb740 1 -- 172.21.15.43:0/2525635853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f9798dd4a0 con 0x55f9798c4480 2024-09-11T23:30:42.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.139+0000 7f5910ff9640 1 -- 172.21.15.43:0/2525635853 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59240041d0 con 0x55f9798c4480 2024-09-11T23:30:42.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.139+0000 7f5910ff9640 1 -- 172.21.15.43:0/2525635853 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f59240047e0 con 0x55f9798c4480 2024-09-11T23:30:42.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.140+0000 7f5935ceb740 10 client.16848 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:30:42.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.140+0000 7f5935ceb740 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:30:42.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.140+0000 7f5935ceb740 20 client.16848 populate_metadata read hostname 'smithi043' 2024-09-11T23:30:42.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.140+0000 7f5935ceb740 1 --2- 172.21.15.43:0/2525635853 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f9798de8b0 0x55f9798e0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:42.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.140+0000 7f5935ceb740 4 client.16848 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:30:42.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.140+0000 7f5935ceb740 1 -- 172.21.15.43:0/2525635853 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9798367f0 con 0x55f9798de8b0 2024-09-11T23:30:42.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.140+0000 7f5930ead640 1 --2- 172.21.15.43:0/2525635853 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f9798de8b0 0x55f9798e0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:42.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.140+0000 7f5930ead640 1 --2- 172.21.15.43:0/2525635853 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f9798de8b0 0x55f9798e0ca0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=476cc1984c820dcc server_cookie=9817c2ed1fa0070f in_seq=0 out_seq=0 2024-09-11T23:30:42.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.140+0000 7f5910ff9640 10 client.16848 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:30:42.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.141+0000 7f5910ff9640 1 -- 172.21.15.43:0/2525635853 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9798367f0 con 0x55f9798de8b0 2024-09-11T23:30:42.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.141+0000 7f5910ff9640 10 client.16848 handle_command_reply: tid=0 2024-09-11T23:30:42.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.141+0000 7f5935ceb740 1 client.16848 shutdown 2024-09-11T23:30:42.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.141+0000 7f5935ceb740 1 -- 172.21.15.43:0/2525635853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798c4480 msgr2=0x55f9798cbb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:42.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.141+0000 7f5935ceb740 1 --2- 172.21.15.43:0/2525635853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798c4480 0x55f9798cbb50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5924000c00 tx=0x7f592400bfd0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.141+0000 7f5935ceb740 1 -- 172.21.15.43:0/2525635853 shutdown_connections 2024-09-11T23:30:42.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.141+0000 7f5935ceb740 1 --2- 172.21.15.43:0/2525635853 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f9798de8b0 0x55f9798e0ca0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.141+0000 7f5935ceb740 1 --2- 172.21.15.43:0/2525635853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9798ced40 0x55f9798cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.141+0000 7f5935ceb740 1 --2- 172.21.15.43:0/2525635853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798c4480 0x55f9798cbb50 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.141+0000 7f5910ff9640 0 client.16848 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:30:42.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.141+0000 7f5935ceb740 1 --2- 172.21.15.43:0/2525635853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9798c3b60 0x55f9798ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.141+0000 7f5935ceb740 1 -- 172.21.15.43:0/2525635853 >> 172.21.15.43:0/2525635853 conn(0x55f9798c00f0 msgr2=0x55f9798bf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:42.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.141+0000 7f5935ceb740 1 -- 172.21.15.43:0/2525635853 shutdown_connections 2024-09-11T23:30:42.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.141+0000 7f5935ceb740 1 -- 172.21.15.43:0/2525635853 wait complete. 2024-09-11T23:30:42.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.141+0000 7f5935ceb740 20 client.16848 trim_cache size 0 max 16384 2024-09-11T23:30:42.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.159+0000 7f5935ceb740 1 Processor -- start 2024-09-11T23:30:42.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.159+0000 7f5935ceb740 1 -- start start 2024-09-11T23:30:42.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.159+0000 7f5935ceb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9798c3b60 0x55f9798cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:42.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.159+0000 7f5935ceb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9798c4480 0x55f9798e6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:42.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.159+0000 7f5935ceb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798ced40 0x55f9798e6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:42.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.159+0000 7f5935ceb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f592c1b2500 con 0x55f9798ced40 2024-09-11T23:30:42.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.159+0000 7f5935ceb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f592c1afda0 con 0x55f9798c4480 2024-09-11T23:30:42.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.159+0000 7f5935ceb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f592c1adc10 con 0x55f9798c3b60 2024-09-11T23:30:42.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.159+0000 7f5930ead640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798ced40 0x55f9798e6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:42.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.159+0000 7f5930ead640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798ced40 0x55f9798e6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46906/0 (socket says 172.21.15.43:46906) 2024-09-11T23:30:42.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.159+0000 7f5930ead640 1 -- 172.21.15.43:0/2451855180 learned_addr learned my addr 172.21.15.43:0/2451855180 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:42.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.159+0000 7f59316ae640 1 --2- 172.21.15.43:0/2451855180 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9798c4480 0x55f9798e6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:42.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.159+0000 7f5930ead640 1 --2- 172.21.15.43:0/2451855180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9798c3b60 0x55f9798cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:42.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.159+0000 7f5930ead640 1 -- 172.21.15.43:0/2451855180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9798c3b60 msgr2=0x55f9798cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:42.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.159+0000 7f5930ead640 1 --2- 172.21.15.43:0/2451855180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9798c3b60 0x55f9798cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.159+0000 7f5930ead640 1 -- 172.21.15.43:0/2451855180 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9798c4480 msgr2=0x55f9798e6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:42.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.159+0000 7f5930ead640 1 --2- 172.21.15.43:0/2451855180 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9798c4480 0x55f9798e6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.159+0000 7f5930ead640 1 -- 172.21.15.43:0/2451855180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9798bff20 con 0x55f9798ced40 2024-09-11T23:30:42.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.160+0000 7f5930ead640 1 --2- 172.21.15.43:0/2451855180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798ced40 0x55f9798e6780 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f59200058c0 tx=0x7f5920005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1401d310b9ea9b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:42.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.160+0000 7f5912ffd640 1 -- 172.21.15.43:0/2451855180 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920026040 con 0x55f9798ced40 2024-09-11T23:30:42.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.160+0000 7f5912ffd640 1 -- 172.21.15.43:0/2451855180 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f592000f8c0 con 0x55f9798ced40 2024-09-11T23:30:42.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.160+0000 7f5912ffd640 1 -- 172.21.15.43:0/2451855180 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920003b20 con 0x55f9798ced40 2024-09-11T23:30:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.160+0000 7f5935ceb740 1 -- 172.21.15.43:0/2451855180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798ced40 msgr2=0x55f9798e6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.160+0000 7f5935ceb740 1 --2- 172.21.15.43:0/2451855180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798ced40 0x55f9798e6780 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f59200058c0 tx=0x7f5920005f90 comp rx=0 tx=0).stop 2024-09-11T23:30:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.160+0000 7f5935ceb740 1 -- 172.21.15.43:0/2451855180 shutdown_connections 2024-09-11T23:30:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.160+0000 7f5935ceb740 1 --2- 172.21.15.43:0/2451855180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798ced40 0x55f9798e6780 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.160+0000 7f5935ceb740 1 --2- 172.21.15.43:0/2451855180 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9798c4480 0x55f9798e6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.160+0000 7f5935ceb740 1 --2- 172.21.15.43:0/2451855180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9798c3b60 0x55f9798cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.160+0000 7f5935ceb740 1 -- 172.21.15.43:0/2451855180 >> 172.21.15.43:0/2451855180 conn(0x55f9798c00f0 msgr2=0x55f9798cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.160+0000 7f5935ceb740 1 -- 172.21.15.43:0/2451855180 shutdown_connections 2024-09-11T23:30:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.160+0000 7f5935ceb740 1 -- 172.21.15.43:0/2451855180 wait complete. 2024-09-11T23:30:42.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.161+0000 7f5935ceb740 1 -- start start 2024-09-11T23:30:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.161+0000 7f5935ceb740 1 Processor -- start 2024-09-11T23:30:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.161+0000 7f5935ceb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798c3b60 0x55f9798d74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.161+0000 7f5935ceb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9798c4480 0x55f9798d37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.161+0000 7f5935ceb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9798ced40 0x55f9798d3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.161+0000 7f5935ceb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f592c1ae260 con 0x55f9798c3b60 2024-09-11T23:30:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.161+0000 7f5935ceb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f592c1af6d0 con 0x55f9798ced40 2024-09-11T23:30:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.161+0000 7f5935ceb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f592c1ae930 con 0x55f9798c4480 2024-09-11T23:30:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.161+0000 7f5935ceb740 10 client.0 _ll_register_callbacks cb 0x7f59200097d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:30:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.161+0000 7f5930ead640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798c3b60 0x55f9798d74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.161+0000 7f5930ead640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798c3b60 0x55f9798d74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46912/0 (socket says 172.21.15.43:46912) 2024-09-11T23:30:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.161+0000 7f5930ead640 1 -- 172.21.15.43:0/2612271510 learned_addr learned my addr 172.21.15.43:0/2612271510 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.161+0000 7f5930ead640 1 --2- 172.21.15.43:0/2612271510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9798ced40 0x55f9798d3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:42.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.161+0000 7f59316ae640 1 --2- 172.21.15.43:0/2612271510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9798c4480 0x55f9798d37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.162+0000 7f5930ead640 1 -- 172.21.15.43:0/2612271510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9798c4480 msgr2=0x55f9798d37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.162+0000 7f5930ead640 1 --2- 172.21.15.43:0/2612271510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9798c4480 0x55f9798d37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.162+0000 7f5930ead640 1 -- 172.21.15.43:0/2612271510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9798ced40 msgr2=0x55f9798d3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.162+0000 7f5930ead640 1 --2- 172.21.15.43:0/2612271510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9798ced40 0x55f9798d3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.162+0000 7f5930ead640 1 -- 172.21.15.43:0/2612271510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9798c11d0 con 0x55f9798c3b60 2024-09-11T23:30:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.162+0000 7f5930ead640 1 --2- 172.21.15.43:0/2612271510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798c3b60 0x55f9798d74a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f59200057a0 tx=0x7f5920001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7607c59c1897dd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.162+0000 7f5910ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:30:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.162+0000 7f5910ff9640 1 -- 172.21.15.43:0/2612271510 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920026070 con 0x55f9798c3b60 2024-09-11T23:30:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.162+0000 7f5910ff9640 1 -- 172.21.15.43:0/2612271510 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f59200042b0 con 0x55f9798c3b60 2024-09-11T23:30:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.162+0000 7f5935ceb740 1 -- 172.21.15.43:0/2612271510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f592000f8c0 con 0x55f9798c3b60 2024-09-11T23:30:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.162+0000 7f5910ff9640 1 -- 172.21.15.43:0/2612271510 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f59200044b0 con 0x55f9798c3b60 2024-09-11T23:30:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.162+0000 7f5935ceb740 10 client.16854 fetch_fsmap learned FSMap version 32 2024-09-11T23:30:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.162+0000 7f5935ceb740 1 -- 172.21.15.43:0/2612271510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f97994a0b0 con 0x55f9798c3b60 2024-09-11T23:30:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.162+0000 7f5910ff9640 1 -- 172.21.15.43:0/2612271510 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f592000f8c0 con 0x55f9798c3b60 2024-09-11T23:30:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.162+0000 7f5910ff9640 1 -- 172.21.15.43:0/2612271510 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f592001f070 con 0x55f9798c3b60 2024-09-11T23:30:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.163+0000 7f5935ceb740 10 client.16854 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:30:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.163+0000 7f5935ceb740 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:30:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.163+0000 7f5935ceb740 20 client.16854 populate_metadata read hostname 'smithi043' 2024-09-11T23:30:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.163+0000 7f5935ceb740 1 --2- 172.21.15.43:0/2612271510 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f97994a670 0x55f97994ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.163+0000 7f5935ceb740 4 client.16854 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:30:42.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.163+0000 7f5935ceb740 1 -- 172.21.15.43:0/2612271510 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f979830d00 con 0x55f97994a670 2024-09-11T23:30:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.163+0000 7f59316ae640 1 --2- 172.21.15.43:0/2612271510 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f97994a670 0x55f97994ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.163+0000 7f59316ae640 1 --2- 172.21.15.43:0/2612271510 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f97994a670 0x55f97994ca60 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb8a66606868bd23 server_cookie=9ff565674fd5b60a in_seq=0 out_seq=0 2024-09-11T23:30:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.163+0000 7f5910ff9640 10 client.16854 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:30:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.164+0000 7f5910ff9640 1 -- 172.21.15.43:0/2612271510 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f979830d00 con 0x55f97994a670 2024-09-11T23:30:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.164+0000 7f5910ff9640 10 client.16854 handle_command_reply: tid=0 2024-09-11T23:30:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.164+0000 7f5935ceb740 1 client.16854 shutdown 2024-09-11T23:30:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.164+0000 7f5935ceb740 1 -- 172.21.15.43:0/2612271510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798c3b60 msgr2=0x55f9798d74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.164+0000 7f5935ceb740 1 --2- 172.21.15.43:0/2612271510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798c3b60 0x55f9798d74a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f59200057a0 tx=0x7f5920001e50 comp rx=0 tx=0).stop 2024-09-11T23:30:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.164+0000 7f5935ceb740 1 -- 172.21.15.43:0/2612271510 shutdown_connections 2024-09-11T23:30:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.164+0000 7f5935ceb740 1 --2- 172.21.15.43:0/2612271510 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55f97994a670 0x55f97994ca60 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.164+0000 7f5935ceb740 1 --2- 172.21.15.43:0/2612271510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9798ced40 0x55f9798d3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.164+0000 7f5935ceb740 1 --2- 172.21.15.43:0/2612271510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9798c4480 0x55f9798d37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.164+0000 7f5935ceb740 1 --2- 172.21.15.43:0/2612271510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9798c3b60 0x55f9798d74a0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.164+0000 7f5935ceb740 1 -- 172.21.15.43:0/2612271510 >> 172.21.15.43:0/2612271510 conn(0x55f9798c00f0 msgr2=0x55f9798bf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:42.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.164+0000 7f5910ff9640 0 client.16854 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:30:42.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.164+0000 7f5935ceb740 1 -- 172.21.15.43:0/2612271510 shutdown_connections 2024-09-11T23:30:42.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.164+0000 7f5935ceb740 1 -- 172.21.15.43:0/2612271510 wait complete. 2024-09-11T23:30:42.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.165+0000 7f5935ceb740 20 client.16854 trim_cache size 0 max 16384 2024-09-11T23:30:42.171 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:30:42.171 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-11T23:30:42.171 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "2e0b6fdd-06b4-4c96-afc1-aee3625d465b", 2024-09-11T23:30:42.172 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-11T23:30:42.172 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:30:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.165+0000 7f5933912640 1 -- 172.21.15.43:0/435569364 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f59140a2cf0 msgr2=0x7f59140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.165+0000 7f5933912640 1 --2- 172.21.15.43:0/435569364 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f59140a2cf0 0x7f59140a51b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f591c0bcce0 tx=0x7f591c08ed10 comp rx=0 tx=0).stop 2024-09-11T23:30:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.165+0000 7f5933912640 1 -- 172.21.15.43:0/435569364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c106f10 msgr2=0x7f592c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.165+0000 7f5933912640 1 --2- 172.21.15.43:0/435569364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c106f10 0x7f592c074f70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f591c031a20 tx=0x7f591c031a50 comp rx=0 tx=0).stop 2024-09-11T23:30:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.165+0000 7f5933912640 1 -- 172.21.15.43:0/435569364 shutdown_connections 2024-09-11T23:30:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.165+0000 7f5933912640 1 --2- 172.21.15.43:0/435569364 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f59140a2cf0 0x7f59140a51b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.165+0000 7f5933912640 1 --2- 172.21.15.43:0/435569364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c106f10 0x7f592c074f70 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.165+0000 7f5933912640 1 --2- 172.21.15.43:0/435569364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f592c102790 0x7f592c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.165+0000 7f5933912640 1 --2- 172.21.15.43:0/435569364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f592c101de0 0x7f592c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.165+0000 7f5933912640 1 -- 172.21.15.43:0/435569364 >> 172.21.15.43:0/435569364 conn(0x7f592c0fd8c0 msgr2=0x7f592c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.165+0000 7f5933912640 1 -- 172.21.15.43:0/435569364 shutdown_connections 2024-09-11T23:30:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:42.165+0000 7f5933912640 1 -- 172.21.15.43:0/435569364 wait complete. 2024-09-11T23:30:42.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:41 smithi090 ceph-mon[35130]: pgmap v272: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T23:30:42.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:41 smithi097 ceph-mon[35562]: pgmap v272: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T23:30:42.865 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:30:43.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:42 smithi043 ceph-mon[30950]: scrub queued for path: / 2024-09-11T23:30:43.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:42 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:30:43.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:42 smithi043 ceph-mon[30950]: scrub summary: active paths [/] 2024-09-11T23:30:43.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:42 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:30:43.127 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:30:43.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:42 smithi090 ceph-mon[35130]: scrub queued for path: / 2024-09-11T23:30:43.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:42 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:30:43.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:42 smithi090 ceph-mon[35130]: scrub summary: active paths [/] 2024-09-11T23:30:43.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:42 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:30:43.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:42 smithi097 ceph-mon[35562]: scrub queued for path: / 2024-09-11T23:30:43.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:42 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:30:43.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:42 smithi097 ceph-mon[35562]: scrub summary: active paths [/] 2024-09-11T23:30:43.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:42 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:30:43.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.596+0000 7f9787900640 1 -- 172.21.15.43:0/649690430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9780100d00 msgr2=0x7f9780106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:43.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.596+0000 7f9787900640 1 --2- 172.21.15.43:0/649690430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9780100d00 0x7f9780106840 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9770009930 tx=0x7f977002a420 comp rx=0 tx=0).stop 2024-09-11T23:30:43.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.596+0000 7f9787900640 1 -- 172.21.15.43:0/649690430 shutdown_connections 2024-09-11T23:30:43.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.596+0000 7f9787900640 1 --2- 172.21.15.43:0/649690430 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9780106f50 0x7f978010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.596+0000 7f9787900640 1 --2- 172.21.15.43:0/649690430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9780100d00 0x7f9780106840 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.596+0000 7f9787900640 1 --2- 172.21.15.43:0/649690430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9780100350 0x7f9780100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.596+0000 7f9787900640 1 -- 172.21.15.43:0/649690430 >> 172.21.15.43:0/649690430 conn(0x7f9780077210 msgr2=0x7f97800fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:43.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.596+0000 7f9787900640 1 -- 172.21.15.43:0/649690430 shutdown_connections 2024-09-11T23:30:43.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.596+0000 7f9787900640 1 -- 172.21.15.43:0/649690430 wait complete. 2024-09-11T23:30:43.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.597+0000 7f9787900640 1 Processor -- start 2024-09-11T23:30:43.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.597+0000 7f9787900640 1 -- start start 2024-09-11T23:30:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.597+0000 7f9787900640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9780100350 0x7f978019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.597+0000 7f9787900640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9780100d00 0x7f978019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.597+0000 7f9787900640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9780106f50 0x7f978019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.597+0000 7f9787900640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9780111fa0 con 0x7f9780100d00 2024-09-11T23:30:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.597+0000 7f9787900640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9780111e20 con 0x7f9780106f50 2024-09-11T23:30:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.597+0000 7f9787900640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9780112120 con 0x7f9780100350 2024-09-11T23:30:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.597+0000 7f9784e9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9780100d00 0x7f978019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.597+0000 7f9784e9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9780100d00 0x7f978019ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46932/0 (socket says 172.21.15.43:46932) 2024-09-11T23:30:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.597+0000 7f9784e9b640 1 -- 172.21.15.43:0/2000540713 learned_addr learned my addr 172.21.15.43:0/2000540713 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.597+0000 7f9785e9d640 1 --2- 172.21.15.43:0/2000540713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9780106f50 0x7f978019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.597+0000 7f978569c640 1 --2- 172.21.15.43:0/2000540713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9780100350 0x7f978019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.597+0000 7f9784e9b640 1 -- 172.21.15.43:0/2000540713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9780100350 msgr2=0x7f978019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.597+0000 7f9784e9b640 1 --2- 172.21.15.43:0/2000540713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9780100350 0x7f978019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.597+0000 7f9784e9b640 1 -- 172.21.15.43:0/2000540713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9780106f50 msgr2=0x7f978019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.597+0000 7f9784e9b640 1 --2- 172.21.15.43:0/2000540713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9780106f50 0x7f978019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.597+0000 7f9784e9b640 1 -- 172.21.15.43:0/2000540713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978019f6c0 con 0x7f9780100d00 2024-09-11T23:30:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.597+0000 7f978569c640 1 --2- 172.21.15.43:0/2000540713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9780100350 0x7f978019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:30:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.598+0000 7f9784e9b640 1 --2- 172.21.15.43:0/2000540713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9780100d00 0x7f978019ac50 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9770005940 tx=0x7f97700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.598+0000 7f97767fc640 1 -- 172.21.15.43:0/2000540713 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977003f030 con 0x7f9780100d00 2024-09-11T23:30:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.598+0000 7f97767fc640 1 -- 172.21.15.43:0/2000540713 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9770003ce0 con 0x7f9780100d00 2024-09-11T23:30:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.598+0000 7f97767fc640 1 -- 172.21.15.43:0/2000540713 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770035070 con 0x7f9780100d00 2024-09-11T23:30:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.598+0000 7f9787900640 1 -- 172.21.15.43:0/2000540713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978019f950 con 0x7f9780100d00 2024-09-11T23:30:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.598+0000 7f9787900640 1 -- 172.21.15.43:0/2000540713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97801a7290 con 0x7f9780100d00 2024-09-11T23:30:43.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.601+0000 7f97767fc640 1 -- 172.21.15.43:0/2000540713 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9770042030 con 0x7f9780100d00 2024-09-11T23:30:43.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.601+0000 7f9789cd9740 1 Processor -- start 2024-09-11T23:30:43.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.601+0000 7f9789cd9740 1 -- start start 2024-09-11T23:30:43.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.601+0000 7f9789cd9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635af17d4a0 0x5635af17d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.601+0000 7f9789cd9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af17ddc0 0x5635af188140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.601+0000 7f9789cd9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635af188680 0x5635af18aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.601+0000 7f9789cd9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5635af0e32e0 con 0x5635af17ddc0 2024-09-11T23:30:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.601+0000 7f9789cd9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5635af08ace0 con 0x5635af188680 2024-09-11T23:30:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.601+0000 7f9789cd9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5635af0df2a0 con 0x5635af17d4a0 2024-09-11T23:30:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.601+0000 7f978569c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af17ddc0 0x5635af188140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.602+0000 7f978569c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af17ddc0 0x5635af188140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46948/0 (socket says 172.21.15.43:46948) 2024-09-11T23:30:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.602+0000 7f978569c640 1 -- 172.21.15.43:0/3962187275 learned_addr learned my addr 172.21.15.43:0/3962187275 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.602+0000 7f9784e9b640 1 --2- 172.21.15.43:0/3962187275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635af188680 0x5635af18aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.602+0000 7f9785e9d640 1 --2- 172.21.15.43:0/3962187275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635af17d4a0 0x5635af17d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.602+0000 7f978569c640 1 -- 172.21.15.43:0/3962187275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635af17d4a0 msgr2=0x5635af17d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.602+0000 7f978569c640 1 --2- 172.21.15.43:0/3962187275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635af17d4a0 0x5635af17d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.602+0000 7f978569c640 1 -- 172.21.15.43:0/3962187275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635af188680 msgr2=0x5635af18aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.602+0000 7f978569c640 1 --2- 172.21.15.43:0/3962187275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635af188680 0x5635af18aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.602+0000 7f978569c640 1 -- 172.21.15.43:0/3962187275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635af06f440 con 0x5635af17ddc0 2024-09-11T23:30:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.602+0000 7f97767fc640 1 --2- 172.21.15.43:0/2000540713 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f976c0a2cf0 0x7f976c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.602+0000 7f9785e9d640 1 --2- 172.21.15.43:0/2000540713 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f976c0a2cf0 0x7f976c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.602+0000 7f97767fc640 1 -- 172.21.15.43:0/2000540713 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f97700ef680 con 0x7f9780100d00 2024-09-11T23:30:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.602+0000 7f978569c640 1 --2- 172.21.15.43:0/3962187275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af17ddc0 0x5635af188140 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f977c00bd40 tx=0x7f977c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff49b514832ffdc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.602+0000 7f976b7fe640 1 -- 172.21.15.43:0/3962187275 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c015070 con 0x5635af17ddc0 2024-09-11T23:30:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.602+0000 7f976b7fe640 1 -- 172.21.15.43:0/3962187275 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f977c00be30 con 0x5635af17ddc0 2024-09-11T23:30:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.602+0000 7f976b7fe640 1 -- 172.21.15.43:0/3962187275 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c011a70 con 0x5635af17ddc0 2024-09-11T23:30:43.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.603+0000 7f9789cd9740 1 -- 172.21.15.43:0/3962187275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af17ddc0 msgr2=0x5635af188140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:43.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.603+0000 7f9785e9d640 1 --2- 172.21.15.43:0/2000540713 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f976c0a2cf0 0x7f976c0a51b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f977800ae90 tx=0x7f977800a4d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:43.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.603+0000 7f9789cd9740 1 --2- 172.21.15.43:0/3962187275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af17ddc0 0x5635af188140 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f977c00bd40 tx=0x7f977c00f730 comp rx=0 tx=0).stop 2024-09-11T23:30:43.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.603+0000 7f9789cd9740 1 -- 172.21.15.43:0/3962187275 shutdown_connections 2024-09-11T23:30:43.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.603+0000 7f9789cd9740 1 --2- 172.21.15.43:0/3962187275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635af188680 0x5635af18aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.603+0000 7f9789cd9740 1 --2- 172.21.15.43:0/3962187275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af17ddc0 0x5635af188140 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.603+0000 7f9789cd9740 1 --2- 172.21.15.43:0/3962187275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635af17d4a0 0x5635af17d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.603+0000 7f9789cd9740 1 -- 172.21.15.43:0/3962187275 >> 172.21.15.43:0/3962187275 conn(0x5635af179a30 msgr2=0x5635af179e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:43.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.603+0000 7f9789cd9740 1 -- 172.21.15.43:0/3962187275 shutdown_connections 2024-09-11T23:30:43.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.603+0000 7f9789cd9740 1 -- 172.21.15.43:0/3962187275 wait complete. 2024-09-11T23:30:43.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.604+0000 7f9789cd9740 1 -- start start 2024-09-11T23:30:43.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.604+0000 7f9789cd9740 1 Processor -- start 2024-09-11T23:30:43.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.604+0000 7f9789cd9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635af17d4a0 0x5635af187fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:43.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.604+0000 7f9789cd9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635af17ddc0 0x5635af185250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:43.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.604+0000 7f9789cd9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af188680 0x5635af185790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:43.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.604+0000 7f9789cd9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5635af18b250 con 0x5635af188680 2024-09-11T23:30:43.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.604+0000 7f9789cd9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5635af09e050 con 0x5635af17ddc0 2024-09-11T23:30:43.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.604+0000 7f9789cd9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5635af18b3d0 con 0x5635af17d4a0 2024-09-11T23:30:43.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.604+0000 7f9789cd9740 10 client.0 _ll_register_callbacks cb 0x5635af0de160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:30:43.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.604+0000 7f9785e9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af188680 0x5635af185790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:43.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.604+0000 7f9785e9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af188680 0x5635af185790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46956/0 (socket says 172.21.15.43:46956) 2024-09-11T23:30:43.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.604+0000 7f9785e9d640 1 -- 172.21.15.43:0/1331208486 learned_addr learned my addr 172.21.15.43:0/1331208486 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:43.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.604+0000 7f9784e9b640 1 --2- 172.21.15.43:0/1331208486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635af17ddc0 0x5635af185250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:43.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.604+0000 7f978569c640 1 --2- 172.21.15.43:0/1331208486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635af17d4a0 0x5635af187fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:43.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.604+0000 7f9785e9d640 1 -- 172.21.15.43:0/1331208486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635af17d4a0 msgr2=0x5635af187fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:43.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.604+0000 7f9785e9d640 1 --2- 172.21.15.43:0/1331208486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635af17d4a0 0x5635af187fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.604+0000 7f9785e9d640 1 -- 172.21.15.43:0/1331208486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635af17ddc0 msgr2=0x5635af185250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:43.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.604+0000 7f9785e9d640 1 --2- 172.21.15.43:0/1331208486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635af17ddc0 0x5635af185250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.604+0000 7f9785e9d640 1 -- 172.21.15.43:0/1331208486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635aee543c0 con 0x5635af188680 2024-09-11T23:30:43.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.604+0000 7f978569c640 1 --2- 172.21.15.43:0/1331208486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635af17d4a0 0x5635af187fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:30:43.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.605+0000 7f9785e9d640 1 --2- 172.21.15.43:0/1331208486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af188680 0x5635af185790 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f977801ec20 tx=0x7f9778004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbfe86e8d6abfba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:43.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.605+0000 7f9768ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:30:43.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.605+0000 7f9768ff9640 1 -- 172.21.15.43:0/1331208486 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977802b060 con 0x5635af188680 2024-09-11T23:30:43.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.605+0000 7f9768ff9640 1 -- 172.21.15.43:0/1331208486 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f9778016030 con 0x5635af188680 2024-09-11T23:30:43.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.605+0000 7f9789cd9740 1 -- 172.21.15.43:0/1331208486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f977c00be30 con 0x5635af188680 2024-09-11T23:30:43.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.605+0000 7f9768ff9640 1 -- 172.21.15.43:0/1331208486 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f977801ec50 con 0x5635af188680 2024-09-11T23:30:43.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.605+0000 7f9789cd9740 10 client.16866 fetch_fsmap learned FSMap version 32 2024-09-11T23:30:43.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.605+0000 7f9789cd9740 1 -- 172.21.15.43:0/1331208486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5635af196c30 con 0x5635af188680 2024-09-11T23:30:43.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.605+0000 7f9768ff9640 1 -- 172.21.15.43:0/1331208486 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f977c00be30 con 0x5635af188680 2024-09-11T23:30:43.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.605+0000 7f9768ff9640 1 -- 172.21.15.43:0/1331208486 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9778026530 con 0x5635af188680 2024-09-11T23:30:43.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.605+0000 7f9789cd9740 10 client.16866 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:30:43.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.605+0000 7f9789cd9740 10 client.16866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:30:43.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.606+0000 7f9789cd9740 20 client.16866 populate_metadata read hostname 'smithi043' 2024-09-11T23:30:43.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.606+0000 7f9789cd9740 1 --2- 172.21.15.43:0/1331208486 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5635af1980f0 0x5635af19a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:43.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.606+0000 7f9789cd9740 4 client.16866 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:30:43.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.606+0000 7f9789cd9740 1 -- 172.21.15.43:0/1331208486 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635af0f0850 con 0x5635af1980f0 2024-09-11T23:30:43.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.606+0000 7f978569c640 1 --2- 172.21.15.43:0/1331208486 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5635af1980f0 0x5635af19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:43.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.606+0000 7f978569c640 1 --2- 172.21.15.43:0/1331208486 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5635af1980f0 0x5635af19a540 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76ad275869316b0f server_cookie=d9ec5f1a73017d2f in_seq=0 out_seq=0 2024-09-11T23:30:43.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.606+0000 7f9768ff9640 10 client.16866 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:30:43.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.607+0000 7f9768ff9640 1 -- 172.21.15.43:0/1331208486 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5635af0f0850 con 0x5635af1980f0 2024-09-11T23:30:43.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.607+0000 7f9768ff9640 10 client.16866 handle_command_reply: tid=0 2024-09-11T23:30:43.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.607+0000 7f9789cd9740 1 client.16866 shutdown 2024-09-11T23:30:43.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.607+0000 7f9789cd9740 1 -- 172.21.15.43:0/1331208486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af188680 msgr2=0x5635af185790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:43.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.607+0000 7f9789cd9740 1 --2- 172.21.15.43:0/1331208486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af188680 0x5635af185790 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f977801ec20 tx=0x7f9778004ab0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.607+0000 7f9789cd9740 1 -- 172.21.15.43:0/1331208486 shutdown_connections 2024-09-11T23:30:43.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.607+0000 7f9789cd9740 1 --2- 172.21.15.43:0/1331208486 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5635af1980f0 0x5635af19a540 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.607+0000 7f9789cd9740 1 --2- 172.21.15.43:0/1331208486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af188680 0x5635af185790 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.607+0000 7f9789cd9740 1 --2- 172.21.15.43:0/1331208486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635af17ddc0 0x5635af185250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.607+0000 7f9789cd9740 1 --2- 172.21.15.43:0/1331208486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635af17d4a0 0x5635af187fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.607+0000 7f9768ff9640 0 client.16866 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:30:43.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.607+0000 7f9789cd9740 1 -- 172.21.15.43:0/1331208486 >> 172.21.15.43:0/1331208486 conn(0x5635af179a30 msgr2=0x5635af17f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:43.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.607+0000 7f9789cd9740 1 -- 172.21.15.43:0/1331208486 shutdown_connections 2024-09-11T23:30:43.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.607+0000 7f9789cd9740 1 -- 172.21.15.43:0/1331208486 wait complete. 2024-09-11T23:30:43.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.607+0000 7f9789cd9740 20 client.16866 trim_cache size 0 max 16384 2024-09-11T23:30:43.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.624+0000 7f9789cd9740 1 Processor -- start 2024-09-11T23:30:43.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.624+0000 7f9789cd9740 1 -- start start 2024-09-11T23:30:43.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.625+0000 7f9789cd9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635af179a30 0x5635af1944b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:43.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.625+0000 7f9789cd9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af17d4a0 0x5635af185210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:43.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.625+0000 7f9789cd9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635af17ddc0 0x5635af180740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:43.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.625+0000 7f9789cd9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97801b2600 con 0x5635af17d4a0 2024-09-11T23:30:43.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.625+0000 7f9789cd9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f97801afea0 con 0x5635af179a30 2024-09-11T23:30:43.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.625+0000 7f9789cd9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f97801add10 con 0x5635af17ddc0 2024-09-11T23:30:43.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.625+0000 7f9784e9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af17d4a0 0x5635af185210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:43.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.625+0000 7f9784e9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af17d4a0 0x5635af185210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46966/0 (socket says 172.21.15.43:46966) 2024-09-11T23:30:43.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.625+0000 7f9784e9b640 1 -- 172.21.15.43:0/3090573533 learned_addr learned my addr 172.21.15.43:0/3090573533 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:43.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.625+0000 7f9785e9d640 1 --2- 172.21.15.43:0/3090573533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635af17ddc0 0x5635af180740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:43.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.625+0000 7f978569c640 1 --2- 172.21.15.43:0/3090573533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635af179a30 0x5635af1944b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:43.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.625+0000 7f9784e9b640 1 -- 172.21.15.43:0/3090573533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635af17ddc0 msgr2=0x5635af180740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:43.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.625+0000 7f9784e9b640 1 --2- 172.21.15.43:0/3090573533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635af17ddc0 0x5635af180740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.625+0000 7f9784e9b640 1 -- 172.21.15.43:0/3090573533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635af179a30 msgr2=0x5635af1944b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:43.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.625+0000 7f9784e9b640 1 --2- 172.21.15.43:0/3090573533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635af179a30 0x5635af1944b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.625+0000 7f9784e9b640 1 -- 172.21.15.43:0/3090573533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635af188310 con 0x5635af17d4a0 2024-09-11T23:30:43.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.625+0000 7f9784e9b640 1 --2- 172.21.15.43:0/3090573533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af17d4a0 0x5635af185210 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9770004650 tx=0x7f97700045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef66fab6e613e59a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:43.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.626+0000 7f976affd640 1 -- 172.21.15.43:0/3090573533 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770049040 con 0x5635af17d4a0 2024-09-11T23:30:43.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.626+0000 7f976affd640 1 -- 172.21.15.43:0/3090573533 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97700bc240 con 0x5635af17d4a0 2024-09-11T23:30:43.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.626+0000 7f976affd640 1 -- 172.21.15.43:0/3090573533 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97700c47b0 con 0x5635af17d4a0 2024-09-11T23:30:43.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.626+0000 7f9789cd9740 1 -- 172.21.15.43:0/3090573533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af17d4a0 msgr2=0x5635af185210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.626+0000 7f9789cd9740 1 --2- 172.21.15.43:0/3090573533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af17d4a0 0x5635af185210 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9770004650 tx=0x7f97700045a0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.626+0000 7f9789cd9740 1 -- 172.21.15.43:0/3090573533 shutdown_connections 2024-09-11T23:30:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.626+0000 7f9789cd9740 1 --2- 172.21.15.43:0/3090573533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635af17ddc0 0x5635af180740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.626+0000 7f9789cd9740 1 --2- 172.21.15.43:0/3090573533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af17d4a0 0x5635af185210 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.626+0000 7f9789cd9740 1 --2- 172.21.15.43:0/3090573533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635af179a30 0x5635af1944b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.626+0000 7f9789cd9740 1 -- 172.21.15.43:0/3090573533 >> 172.21.15.43:0/3090573533 conn(0x5635af1858f0 msgr2=0x5635af180030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.626+0000 7f9789cd9740 1 -- 172.21.15.43:0/3090573533 shutdown_connections 2024-09-11T23:30:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.626+0000 7f9789cd9740 1 -- 172.21.15.43:0/3090573533 wait complete. 2024-09-11T23:30:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.627+0000 7f9789cd9740 1 -- start start 2024-09-11T23:30:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.627+0000 7f9789cd9740 1 Processor -- start 2024-09-11T23:30:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.627+0000 7f9789cd9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635af179a30 0x5635af190e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:43.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.627+0000 7f9789cd9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635af17d4a0 0x5635af18c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.627+0000 7f9789cd9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af17ddc0 0x5635af18c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.627+0000 7f9789cd9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97801ae360 con 0x5635af17ddc0 2024-09-11T23:30:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.627+0000 7f9789cd9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f97801af7d0 con 0x5635af17d4a0 2024-09-11T23:30:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.627+0000 7f9789cd9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f97801aea30 con 0x5635af179a30 2024-09-11T23:30:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.627+0000 7f9789cd9740 10 client.0 _ll_register_callbacks cb 0x7f977800a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:30:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.627+0000 7f9785e9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af17ddc0 0x5635af18c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.627+0000 7f9784e9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635af17d4a0 0x5635af18c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.627+0000 7f9785e9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af17ddc0 0x5635af18c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46980/0 (socket says 172.21.15.43:46980) 2024-09-11T23:30:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.627+0000 7f9785e9d640 1 -- 172.21.15.43:0/1318241287 learned_addr learned my addr 172.21.15.43:0/1318241287 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:30:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.627+0000 7f978569c640 1 --2- 172.21.15.43:0/1318241287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635af179a30 0x5635af190e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.627+0000 7f9785e9d640 1 -- 172.21.15.43:0/1318241287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635af179a30 msgr2=0x5635af190e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.627+0000 7f9785e9d640 1 --2- 172.21.15.43:0/1318241287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635af179a30 0x5635af190e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.627+0000 7f9785e9d640 1 -- 172.21.15.43:0/1318241287 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635af17d4a0 msgr2=0x5635af18c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:30:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.627+0000 7f9785e9d640 1 --2- 172.21.15.43:0/1318241287 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635af17d4a0 0x5635af18c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.627+0000 7f9785e9d640 1 -- 172.21.15.43:0/1318241287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635af18cef0 con 0x5635af17ddc0 2024-09-11T23:30:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.628+0000 7f9785e9d640 1 --2- 172.21.15.43:0/1318241287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af17ddc0 0x5635af18c6c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9778011010 tx=0x7f977800fbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d42e0132dff1fce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:30:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.628+0000 7f9768ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:30:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.628+0000 7f9768ff9640 1 -- 172.21.15.43:0/1318241287 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977803b020 con 0x5635af17ddc0 2024-09-11T23:30:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.628+0000 7f9768ff9640 1 -- 172.21.15.43:0/1318241287 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f9778037460 con 0x5635af17ddc0 2024-09-11T23:30:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.628+0000 7f9789cd9740 1 -- 172.21.15.43:0/1318241287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97700bc240 con 0x5635af17ddc0 2024-09-11T23:30:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.628+0000 7f9768ff9640 1 -- 172.21.15.43:0/1318241287 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9778046050 con 0x5635af17ddc0 2024-09-11T23:30:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.628+0000 7f9789cd9740 10 client.16872 fetch_fsmap learned FSMap version 32 2024-09-11T23:30:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.628+0000 7f9789cd9740 1 -- 172.21.15.43:0/1318241287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5635af202e90 con 0x5635af17ddc0 2024-09-11T23:30:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.628+0000 7f9768ff9640 1 -- 172.21.15.43:0/1318241287 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97700bc240 con 0x5635af17ddc0 2024-09-11T23:30:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.628+0000 7f9768ff9640 1 -- 172.21.15.43:0/1318241287 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f977802e6e0 con 0x5635af17ddc0 2024-09-11T23:30:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.628+0000 7f9789cd9740 10 client.16872 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:30:43.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.628+0000 7f9789cd9740 10 client.16872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:30:43.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.628+0000 7f9789cd9740 20 client.16872 populate_metadata read hostname 'smithi043' 2024-09-11T23:30:43.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.628+0000 7f9789cd9740 1 --2- 172.21.15.43:0/1318241287 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5635af188680 0x5635af2055e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:30:43.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.628+0000 7f9789cd9740 4 client.16872 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:30:43.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.628+0000 7f9789cd9740 1 -- 172.21.15.43:0/1318241287 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5635af0ead60 con 0x5635af188680 2024-09-11T23:30:43.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.629+0000 7f978569c640 1 --2- 172.21.15.43:0/1318241287 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5635af188680 0x5635af2055e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:30:43.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.629+0000 7f978569c640 1 --2- 172.21.15.43:0/1318241287 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5635af188680 0x5635af2055e0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad26f0c27a77de86 server_cookie=7d74493fc51ab56e in_seq=0 out_seq=0 2024-09-11T23:30:43.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.629+0000 7f9768ff9640 10 client.16872 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:30:43.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.629+0000 7f9768ff9640 1 -- 172.21.15.43:0/1318241287 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5635af0ead60 con 0x5635af188680 2024-09-11T23:30:43.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.629+0000 7f9768ff9640 10 client.16872 handle_command_reply: tid=0 2024-09-11T23:30:43.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.629+0000 7f9789cd9740 1 client.16872 shutdown 2024-09-11T23:30:43.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.629+0000 7f9789cd9740 1 -- 172.21.15.43:0/1318241287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af17ddc0 msgr2=0x5635af18c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.629+0000 7f9789cd9740 1 --2- 172.21.15.43:0/1318241287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af17ddc0 0x5635af18c6c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9778011010 tx=0x7f977800fbc0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.629+0000 7f9789cd9740 1 -- 172.21.15.43:0/1318241287 shutdown_connections 2024-09-11T23:30:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.629+0000 7f9789cd9740 1 --2- 172.21.15.43:0/1318241287 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5635af188680 0x5635af2055e0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.629+0000 7f9789cd9740 1 --2- 172.21.15.43:0/1318241287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5635af17ddc0 0x5635af18c6c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.629+0000 7f9789cd9740 1 --2- 172.21.15.43:0/1318241287 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635af17d4a0 0x5635af18c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.629+0000 7f9789cd9740 1 --2- 172.21.15.43:0/1318241287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635af179a30 0x5635af190e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.629+0000 7f9789cd9740 1 -- 172.21.15.43:0/1318241287 >> 172.21.15.43:0/1318241287 conn(0x5635af1858f0 msgr2=0x5635af1872d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.629+0000 7f9768ff9640 0 client.16872 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:30:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.630+0000 7f9789cd9740 1 -- 172.21.15.43:0/1318241287 shutdown_connections 2024-09-11T23:30:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.630+0000 7f9789cd9740 1 -- 172.21.15.43:0/1318241287 wait complete. 2024-09-11T23:30:43.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.630+0000 7f9789cd9740 20 client.16872 trim_cache size 0 max 16384 2024-09-11T23:30:43.638 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:30:43.638 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:30:43.638 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-09-11T23:30:43.638 INFO:teuthology.orchestra.run.smithi043.stdout: "2e0b6fdd-06b4-4c96-afc1-aee3625d465b": { 2024-09-11T23:30:43.638 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-09-11T23:30:43.638 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "2e0b6fdd-06b4-4c96-afc1-aee3625d465b", 2024-09-11T23:30:43.638 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-09-11T23:30:43.638 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:30:43.638 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:30:43.638 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:30:43.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.630+0000 7f9787900640 1 -- 172.21.15.43:0/2000540713 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f976c0a2cf0 msgr2=0x7f976c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:43.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.630+0000 7f9787900640 1 --2- 172.21.15.43:0/2000540713 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f976c0a2cf0 0x7f976c0a51b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f977800ae90 tx=0x7f977800a4d0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.630+0000 7f9787900640 1 -- 172.21.15.43:0/2000540713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9780100d00 msgr2=0x7f978019ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:30:43.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.630+0000 7f9787900640 1 --2- 172.21.15.43:0/2000540713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9780100d00 0x7f978019ac50 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9770005940 tx=0x7f97700042f0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.630+0000 7f9787900640 1 -- 172.21.15.43:0/2000540713 shutdown_connections 2024-09-11T23:30:43.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.630+0000 7f9787900640 1 --2- 172.21.15.43:0/2000540713 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f976c0a2cf0 0x7f976c0a51b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.630+0000 7f9787900640 1 --2- 172.21.15.43:0/2000540713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9780106f50 0x7f978019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.630+0000 7f9787900640 1 --2- 172.21.15.43:0/2000540713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9780100d00 0x7f978019ac50 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.630+0000 7f9787900640 1 --2- 172.21.15.43:0/2000540713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9780100350 0x7f978019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:30:43.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.630+0000 7f9787900640 1 -- 172.21.15.43:0/2000540713 >> 172.21.15.43:0/2000540713 conn(0x7f9780077210 msgr2=0x7f97801057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:30:43.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.630+0000 7f9787900640 1 -- 172.21.15.43:0/2000540713 shutdown_connections 2024-09-11T23:30:43.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:30:43.630+0000 7f9787900640 1 -- 172.21.15.43:0/2000540713 wait complete. 2024-09-11T23:30:44.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:43 smithi090 ceph-mon[35130]: pgmap v273: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:30:44.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:43 smithi090 ceph-mon[35130]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T23:30:44.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:43 smithi097 ceph-mon[35562]: pgmap v273: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:30:44.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:43 smithi097 ceph-mon[35562]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T23:30:44.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:43 smithi043 ceph-mon[30950]: pgmap v273: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:30:44.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:43 smithi043 ceph-mon[30950]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T23:30:44.398 INFO:tasks.cephfs.filesystem:scrub status for tag:2e0b6fdd-06b4-4c96-afc1-aee3625d465b - {'path': '/', 'tag': '2e0b6fdd-06b4-4c96-afc1-aee3625d465b', 'options': 'recursive,force'} 2024-09-11T23:30:45.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:44 smithi090 ceph-mon[35130]: scrub summary: idle 2024-09-11T23:30:45.300 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:44 smithi097 ceph-mon[35562]: scrub summary: idle 2024-09-11T23:30:45.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:44 smithi043 ceph-mon[30950]: scrub summary: idle 2024-09-11T23:30:46.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:45 smithi090 ceph-mon[35130]: pgmap v275: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T23:30:46.300 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:45 smithi097 ceph-mon[35562]: pgmap v275: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T23:30:46.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:45 smithi043 ceph-mon[30950]: pgmap v275: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T23:30:48.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:47 smithi090 ceph-mon[35130]: pgmap v276: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-11T23:30:48.300 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:47 smithi097 ceph-mon[35562]: pgmap v276: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-11T23:30:48.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:47 smithi043 ceph-mon[30950]: pgmap v276: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-11T23:30:50.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:49 smithi090 ceph-mon[35130]: pgmap v277: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-11T23:30:50.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:49 smithi097 ceph-mon[35562]: pgmap v277: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-11T23:30:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:49 smithi043 ceph-mon[30950]: pgmap v277: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-11T23:30:52.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:51 smithi090 ceph-mon[35130]: pgmap v278: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-11T23:30:52.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:51 smithi097 ceph-mon[35562]: pgmap v278: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-11T23:30:52.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:51 smithi043 ceph-mon[30950]: pgmap v278: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-11T23:30:54.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:53 smithi090 ceph-mon[35130]: pgmap v279: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-11T23:30:54.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:53 smithi097 ceph-mon[35562]: pgmap v279: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-11T23:30:54.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:53 smithi043 ceph-mon[30950]: pgmap v279: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-11T23:30:56.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:55 smithi090 ceph-mon[35130]: pgmap v280: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-09-11T23:30:56.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:55 smithi097 ceph-mon[35562]: pgmap v280: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-09-11T23:30:56.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:55 smithi043 ceph-mon[30950]: pgmap v280: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-09-11T23:30:58.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:57 smithi090 ceph-mon[35130]: pgmap v281: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7 op/s 2024-09-11T23:30:58.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:57 smithi097 ceph-mon[35562]: pgmap v281: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7 op/s 2024-09-11T23:30:58.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:57 smithi043 ceph-mon[30950]: pgmap v281: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7 op/s 2024-09-11T23:31:00.273 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:30:59 smithi090 ceph-mon[35130]: pgmap v282: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:31:00.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:30:59 smithi097 ceph-mon[35562]: pgmap v282: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:31:00.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:30:59 smithi043 ceph-mon[30950]: pgmap v282: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:31:02.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:01 smithi090 ceph-mon[35130]: pgmap v283: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T23:31:02.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:01 smithi097 ceph-mon[35562]: pgmap v283: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T23:31:02.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:01 smithi043 ceph-mon[30950]: pgmap v283: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T23:31:04.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:03 smithi090 ceph-mon[35130]: pgmap v284: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-11T23:31:04.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:03 smithi097 ceph-mon[35562]: pgmap v284: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-11T23:31:04.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:03 smithi043 ceph-mon[30950]: pgmap v284: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-11T23:31:06.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:05 smithi090 ceph-mon[35130]: pgmap v285: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-11T23:31:06.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:05 smithi097 ceph-mon[35562]: pgmap v285: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-11T23:31:06.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:05 smithi043 ceph-mon[30950]: pgmap v285: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-11T23:31:08.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:07 smithi090 ceph-mon[35130]: pgmap v286: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-11T23:31:08.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:07 smithi097 ceph-mon[35562]: pgmap v286: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-11T23:31:08.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:07 smithi043 ceph-mon[30950]: pgmap v286: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-11T23:31:10.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:09 smithi090 ceph-mon[35130]: pgmap v287: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-11T23:31:10.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:09 smithi097 ceph-mon[35562]: pgmap v287: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-11T23:31:10.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:09 smithi043 ceph-mon[30950]: pgmap v287: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-11T23:31:11.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:10 smithi090 ceph-mon[35130]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T23:31:11.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:10 smithi097 ceph-mon[35562]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T23:31:11.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:10 smithi043 ceph-mon[30950]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T23:31:12.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:11 smithi090 ceph-mon[35130]: pgmap v289: 97 pgs: 10 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 81 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-11T23:31:12.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:11 smithi097 ceph-mon[35562]: pgmap v289: 97 pgs: 10 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 81 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-11T23:31:12.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:11 smithi043 ceph-mon[30950]: pgmap v289: 97 pgs: 10 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 81 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-11T23:31:14.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:13 smithi090 ceph-mon[35130]: pgmap v290: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-11T23:31:14.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:13 smithi097 ceph-mon[35562]: pgmap v290: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-11T23:31:14.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:13 smithi043 ceph-mon[30950]: pgmap v290: 97 pgs: 19 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 68 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-11T23:31:14.400 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:31:14.870 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:31:16.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:15 smithi090 ceph-mon[35130]: pgmap v291: 97 pgs: 25 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 60 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-11T23:31:16.300 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:15 smithi097 ceph-mon[35562]: pgmap v291: 97 pgs: 25 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 60 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-11T23:31:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:15 smithi043 ceph-mon[30950]: pgmap v291: 97 pgs: 25 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 60 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-11T23:31:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.361+0000 7f356b249640 1 -- 172.21.15.43:0/2781934535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3564102d40 msgr2=0x7f3564103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.362+0000 7f356b249640 1 --2- 172.21.15.43:0/2781934535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3564102d40 0x7f3564103120 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f354c009900 tx=0x7f354c02a480 comp rx=0 tx=0).stop 2024-09-11T23:31:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.362+0000 7f356b249640 1 -- 172.21.15.43:0/2781934535 shutdown_connections 2024-09-11T23:31:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.362+0000 7f356b249640 1 --2- 172.21.15.43:0/2781934535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f356410c170 0x7f356410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.362+0000 7f356b249640 1 --2- 172.21.15.43:0/2781934535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3564103660 0x7f356410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.362+0000 7f356b249640 1 --2- 172.21.15.43:0/2781934535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3564102d40 0x7f3564103120 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.362+0000 7f356b249640 1 -- 172.21.15.43:0/2781934535 >> 172.21.15.43:0/2781934535 conn(0x7f35640fc7a0 msgr2=0x7f35640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.362+0000 7f356b249640 1 -- 172.21.15.43:0/2781934535 shutdown_connections 2024-09-11T23:31:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.362+0000 7f356b249640 1 -- 172.21.15.43:0/2781934535 wait complete. 2024-09-11T23:31:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.362+0000 7f356b249640 1 Processor -- start 2024-09-11T23:31:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.362+0000 7f356b249640 1 -- start start 2024-09-11T23:31:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.363+0000 7f356b249640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3564102d40 0x7f356419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.363+0000 7f356b249640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3564103660 0x7f356419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.363+0000 7f356b249640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f356410c170 0x7f35641a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.363+0000 7f356b249640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3564102930 con 0x7f3564102d40 2024-09-11T23:31:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.363+0000 7f356b249640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f35641027b0 con 0x7f3564103660 2024-09-11T23:31:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.363+0000 7f356b249640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3564118720 con 0x7f356410c170 2024-09-11T23:31:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.363+0000 7f3568fe5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3564102d40 0x7f356419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.363+0000 7f3568fe5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3564102d40 0x7f356419ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41580/0 (socket says 172.21.15.43:41580) 2024-09-11T23:31:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.363+0000 7f3568fe5640 1 -- 172.21.15.43:0/3482667986 learned_addr learned my addr 172.21.15.43:0/3482667986 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.363+0000 7f35697e6640 1 --2- 172.21.15.43:0/3482667986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f356410c170 0x7f35641a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.363+0000 7f3563fff640 1 --2- 172.21.15.43:0/3482667986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3564103660 0x7f356419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.363+0000 7f3568fe5640 1 -- 172.21.15.43:0/3482667986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f356410c170 msgr2=0x7f35641a3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.363+0000 7f3568fe5640 1 --2- 172.21.15.43:0/3482667986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f356410c170 0x7f35641a3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.363+0000 7f3568fe5640 1 -- 172.21.15.43:0/3482667986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3564103660 msgr2=0x7f356419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.363+0000 7f3568fe5640 1 --2- 172.21.15.43:0/3482667986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3564103660 0x7f356419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.363+0000 7f3568fe5640 1 -- 172.21.15.43:0/3482667986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35641a3a80 con 0x7f3564102d40 2024-09-11T23:31:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.363+0000 7f3568fe5640 1 --2- 172.21.15.43:0/3482667986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3564102d40 0x7f356419ea50 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f354c005ab0 tx=0x7f354c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.364+0000 7f3561ffb640 1 -- 172.21.15.43:0/3482667986 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c03f030 con 0x7f3564102d40 2024-09-11T23:31:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.364+0000 7f356b249640 1 -- 172.21.15.43:0/3482667986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35641a3d10 con 0x7f3564102d40 2024-09-11T23:31:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.364+0000 7f3561ffb640 1 -- 172.21.15.43:0/3482667986 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f354c003ce0 con 0x7f3564102d40 2024-09-11T23:31:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.364+0000 7f3561ffb640 1 -- 172.21.15.43:0/3482667986 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c035070 con 0x7f3564102d40 2024-09-11T23:31:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.364+0000 7f356b249640 1 -- 172.21.15.43:0/3482667986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35641ab600 con 0x7f3564102d40 2024-09-11T23:31:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.366+0000 7f3561ffb640 1 -- 172.21.15.43:0/3482667986 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f354c042030 con 0x7f3564102d40 2024-09-11T23:31:17.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.366+0000 7f356d622740 1 Processor -- start 2024-09-11T23:31:17.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.366+0000 7f356d622740 1 -- start start 2024-09-11T23:31:17.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.366+0000 7f356d622740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfad4a0 0x56461dfad880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:17.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.366+0000 7f356d622740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfaddc0 0x56461dfb8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:17.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.366+0000 7f356d622740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461dfbaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:17.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.366+0000 7f356d622740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56461df132e0 con 0x56461dfb8680 2024-09-11T23:31:17.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.366+0000 7f356d622740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56461debace0 con 0x56461dfaddc0 2024-09-11T23:31:17.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.366+0000 7f356d622740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56461df0f2a0 con 0x56461dfad4a0 2024-09-11T23:31:17.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.366+0000 7f3563fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461dfbaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:17.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.366+0000 7f3563fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461dfbaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41596/0 (socket says 172.21.15.43:41596) 2024-09-11T23:31:17.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.366+0000 7f3563fff640 1 -- 172.21.15.43:0/1081056623 learned_addr learned my addr 172.21.15.43:0/1081056623 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:17.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.367+0000 7f3568fe5640 1 --2- 172.21.15.43:0/1081056623 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfaddc0 0x56461dfb8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:17.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.367+0000 7f35697e6640 1 --2- 172.21.15.43:0/1081056623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfad4a0 0x56461dfad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:17.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.367+0000 7f3563fff640 1 -- 172.21.15.43:0/1081056623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfad4a0 msgr2=0x56461dfad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:17.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.367+0000 7f3563fff640 1 --2- 172.21.15.43:0/1081056623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfad4a0 0x56461dfad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.367+0000 7f3563fff640 1 -- 172.21.15.43:0/1081056623 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfaddc0 msgr2=0x56461dfb8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:17.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.367+0000 7f3563fff640 1 --2- 172.21.15.43:0/1081056623 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfaddc0 0x56461dfb8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.367+0000 7f3563fff640 1 -- 172.21.15.43:0/1081056623 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56461de9f440 con 0x56461dfb8680 2024-09-11T23:31:17.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.367+0000 7f3561ffb640 1 --2- 172.21.15.43:0/3482667986 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f35500a2cf0 0x7f35500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:17.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.367+0000 7f35697e6640 1 --2- 172.21.15.43:0/1081056623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfad4a0 0x56461dfad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:31:17.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.367+0000 7f35697e6640 1 --2- 172.21.15.43:0/3482667986 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f35500a2cf0 0x7f35500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:17.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.367+0000 7f3561ffb640 1 -- 172.21.15.43:0/3482667986 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8615+0+0 (secure 0 0 0) 0x7f354c0ef8b0 con 0x7f3564102d40 2024-09-11T23:31:17.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.367+0000 7f3563fff640 1 --2- 172.21.15.43:0/1081056623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461dfbaa70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f355400ae80 tx=0x7f355400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2885f541c33a164c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.367+0000 7f355affd640 1 -- 172.21.15.43:0/1081056623 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3554003b70 con 0x56461dfb8680 2024-09-11T23:31:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.367+0000 7f355affd640 1 -- 172.21.15.43:0/1081056623 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3554003d10 con 0x56461dfb8680 2024-09-11T23:31:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.367+0000 7f355affd640 1 -- 172.21.15.43:0/1081056623 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3554010ba0 con 0x56461dfb8680 2024-09-11T23:31:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.368+0000 7f356d622740 1 -- 172.21.15.43:0/1081056623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 msgr2=0x56461dfbaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.368+0000 7f35697e6640 1 --2- 172.21.15.43:0/3482667986 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f35500a2cf0 0x7f35500a51b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f355c00ac00 tx=0x7f355c00a250 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.368+0000 7f356d622740 1 --2- 172.21.15.43:0/1081056623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461dfbaa70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f355400ae80 tx=0x7f355400f790 comp rx=0 tx=0).stop 2024-09-11T23:31:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.368+0000 7f356d622740 1 -- 172.21.15.43:0/1081056623 shutdown_connections 2024-09-11T23:31:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.368+0000 7f356d622740 1 --2- 172.21.15.43:0/1081056623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461dfbaa70 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.368+0000 7f356d622740 1 --2- 172.21.15.43:0/1081056623 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfaddc0 0x56461dfb8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.368+0000 7f356d622740 1 --2- 172.21.15.43:0/1081056623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfad4a0 0x56461dfad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.368+0000 7f356d622740 1 -- 172.21.15.43:0/1081056623 >> 172.21.15.43:0/1081056623 conn(0x56461dfa9a30 msgr2=0x56461dfa9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.368+0000 7f356d622740 1 -- 172.21.15.43:0/1081056623 shutdown_connections 2024-09-11T23:31:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.368+0000 7f356d622740 1 -- 172.21.15.43:0/1081056623 wait complete. 2024-09-11T23:31:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.368+0000 7f356d622740 1 -- start start 2024-09-11T23:31:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.369+0000 7f356d622740 1 Processor -- start 2024-09-11T23:31:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.369+0000 7f356d622740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfad4a0 0x56461dfb7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.369+0000 7f356d622740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfaddc0 0x56461dfb51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.369+0000 7f356d622740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461dfb56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.369+0000 7f356d622740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56461dfbb250 con 0x56461dfb8680 2024-09-11T23:31:17.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.369+0000 7f356d622740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56461dece050 con 0x56461dfaddc0 2024-09-11T23:31:17.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.369+0000 7f356d622740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56461dfbb3d0 con 0x56461dfad4a0 2024-09-11T23:31:17.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.369+0000 7f356d622740 10 client.0 _ll_register_callbacks cb 0x56461df0e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:17.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.369+0000 7f35697e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461dfb56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:17.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.369+0000 7f35697e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461dfb56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41612/0 (socket says 172.21.15.43:41612) 2024-09-11T23:31:17.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.369+0000 7f35697e6640 1 -- 172.21.15.43:0/809464641 learned_addr learned my addr 172.21.15.43:0/809464641 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:17.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.369+0000 7f3568fe5640 1 --2- 172.21.15.43:0/809464641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfad4a0 0x56461dfb7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:17.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.369+0000 7f3563fff640 1 --2- 172.21.15.43:0/809464641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfaddc0 0x56461dfb51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:17.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.369+0000 7f35697e6640 1 -- 172.21.15.43:0/809464641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfad4a0 msgr2=0x56461dfb7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:17.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.369+0000 7f35697e6640 1 --2- 172.21.15.43:0/809464641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfad4a0 0x56461dfb7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.369+0000 7f35697e6640 1 -- 172.21.15.43:0/809464641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfaddc0 msgr2=0x56461dfb51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:17.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.369+0000 7f35697e6640 1 --2- 172.21.15.43:0/809464641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfaddc0 0x56461dfb51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.369+0000 7f35697e6640 1 -- 172.21.15.43:0/809464641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56461dc843c0 con 0x56461dfb8680 2024-09-11T23:31:17.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.369+0000 7f3568fe5640 1 --2- 172.21.15.43:0/809464641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfad4a0 0x56461dfb7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:31:17.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.369+0000 7f35697e6640 1 --2- 172.21.15.43:0/809464641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461dfb56e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f355c00a4e0 tx=0x7f355c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c53432d6ba986e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:17.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.369+0000 7f3543fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:31:17.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.370+0000 7f3543fff640 1 -- 172.21.15.43:0/809464641 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c02a060 con 0x56461dfb8680 2024-09-11T23:31:17.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.370+0000 7f3543fff640 1 -- 172.21.15.43:0/809464641 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8615+0+0 (secure 0 0 0) 0x7f355c01b670 con 0x56461dfb8680 2024-09-11T23:31:17.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.370+0000 7f356d622740 1 -- 172.21.15.43:0/809464641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3554003d10 con 0x56461dfb8680 2024-09-11T23:31:17.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.370+0000 7f3543fff640 1 -- 172.21.15.43:0/809464641 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f355c01de10 con 0x56461dfb8680 2024-09-11T23:31:17.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.370+0000 7f356d622740 10 client.16884 fetch_fsmap learned FSMap version 32 2024-09-11T23:31:17.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.370+0000 7f356d622740 1 -- 172.21.15.43:0/809464641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56461dfc6ba0 con 0x56461dfb8680 2024-09-11T23:31:17.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.370+0000 7f3543fff640 1 -- 172.21.15.43:0/809464641 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3554003d10 con 0x56461dfb8680 2024-09-11T23:31:17.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.370+0000 7f3543fff640 1 -- 172.21.15.43:0/809464641 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f355c019040 con 0x56461dfb8680 2024-09-11T23:31:17.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.370+0000 7f356d622740 10 client.16884 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:31:17.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.370+0000 7f356d622740 10 client.16884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:31:17.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.370+0000 7f356d622740 20 client.16884 populate_metadata read hostname 'smithi043' 2024-09-11T23:31:17.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.370+0000 7f356d622740 1 --2- 172.21.15.43:0/809464641 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56461dfc8060 0x56461dfca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:17.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.370+0000 7f356d622740 4 client.16884 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:31:17.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.370+0000 7f356d622740 1 -- 172.21.15.43:0/809464641 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56461df20850 con 0x56461dfc8060 2024-09-11T23:31:17.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.371+0000 7f3568fe5640 1 --2- 172.21.15.43:0/809464641 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56461dfc8060 0x56461dfca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:17.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.371+0000 7f3543fff640 10 client.16884 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:31:17.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.371+0000 7f3568fe5640 1 --2- 172.21.15.43:0/809464641 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56461dfc8060 0x56461dfca4d0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c75bbb29914db37a server_cookie=e61f83f3a5a57d12 in_seq=0 out_seq=0 2024-09-11T23:31:17.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.372+0000 7f3543fff640 1 -- 172.21.15.43:0/809464641 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56461df20850 con 0x56461dfc8060 2024-09-11T23:31:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.372+0000 7f3543fff640 10 client.16884 handle_command_reply: tid=0 2024-09-11T23:31:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.372+0000 7f356d622740 1 client.16884 shutdown 2024-09-11T23:31:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.372+0000 7f356d622740 1 -- 172.21.15.43:0/809464641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 msgr2=0x56461dfb56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.372+0000 7f356d622740 1 --2- 172.21.15.43:0/809464641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461dfb56e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f355c00a4e0 tx=0x7f355c0046c0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.372+0000 7f356d622740 1 -- 172.21.15.43:0/809464641 shutdown_connections 2024-09-11T23:31:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.372+0000 7f356d622740 1 --2- 172.21.15.43:0/809464641 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56461dfc8060 0x56461dfca4d0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.372+0000 7f3543fff640 0 client.16884 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:31:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.372+0000 7f356d622740 1 --2- 172.21.15.43:0/809464641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461dfb56e0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.372+0000 7f356d622740 1 --2- 172.21.15.43:0/809464641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfaddc0 0x56461dfb51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.372+0000 7f356d622740 1 --2- 172.21.15.43:0/809464641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfad4a0 0x56461dfb7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.372+0000 7f356d622740 1 -- 172.21.15.43:0/809464641 >> 172.21.15.43:0/809464641 conn(0x56461dfa9a30 msgr2=0x56461dfaf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.372+0000 7f356d622740 1 -- 172.21.15.43:0/809464641 shutdown_connections 2024-09-11T23:31:17.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.372+0000 7f356d622740 1 -- 172.21.15.43:0/809464641 wait complete. 2024-09-11T23:31:17.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.372+0000 7f356d622740 20 client.16884 trim_cache size 0 max 16384 2024-09-11T23:31:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.389+0000 7f356d622740 1 Processor -- start 2024-09-11T23:31:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.389+0000 7f356d622740 1 -- start start 2024-09-11T23:31:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.390+0000 7f356d622740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfad4a0 0x56461dfb3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.390+0000 7f356d622740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfaddc0 0x56461dfcfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.390+0000 7f356d622740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461dfd02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.390+0000 7f356d622740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f35641b68c0 con 0x56461dfb8680 2024-09-11T23:31:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.390+0000 7f356d622740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f35641b4160 con 0x56461dfaddc0 2024-09-11T23:31:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.390+0000 7f356d622740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f35641b1fd0 con 0x56461dfad4a0 2024-09-11T23:31:17.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.390+0000 7f35697e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461dfd02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.390+0000 7f35697e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461dfd02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41618/0 (socket says 172.21.15.43:41618) 2024-09-11T23:31:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.390+0000 7f35697e6640 1 -- 172.21.15.43:0/2914309814 learned_addr learned my addr 172.21.15.43:0/2914309814 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.390+0000 7f3568fe5640 1 --2- 172.21.15.43:0/2914309814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfad4a0 0x56461dfb3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.390+0000 7f3563fff640 1 --2- 172.21.15.43:0/2914309814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfaddc0 0x56461dfcfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.390+0000 7f35697e6640 1 -- 172.21.15.43:0/2914309814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfad4a0 msgr2=0x56461dfb3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.390+0000 7f35697e6640 1 --2- 172.21.15.43:0/2914309814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfad4a0 0x56461dfb3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.390+0000 7f35697e6640 1 -- 172.21.15.43:0/2914309814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfaddc0 msgr2=0x56461dfcfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.390+0000 7f35697e6640 1 --2- 172.21.15.43:0/2914309814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfaddc0 0x56461dfcfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.390+0000 7f35697e6640 1 -- 172.21.15.43:0/2914309814 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56461dfb4170 con 0x56461dfb8680 2024-09-11T23:31:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.390+0000 7f3563fff640 1 --2- 172.21.15.43:0/2914309814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfaddc0 0x56461dfcfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:31:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.390+0000 7f35697e6640 1 --2- 172.21.15.43:0/2914309814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461dfd02e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f355c00aa60 tx=0x7f355c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffee8a930bf6690 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.390+0000 7f355a7fc640 1 -- 172.21.15.43:0/2914309814 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c03b070 con 0x56461dfb8680 2024-09-11T23:31:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.391+0000 7f355a7fc640 1 -- 172.21.15.43:0/2914309814 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f355c02d8f0 con 0x56461dfb8680 2024-09-11T23:31:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.391+0000 7f355a7fc640 1 -- 172.21.15.43:0/2914309814 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c0052a0 con 0x56461dfb8680 2024-09-11T23:31:17.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.391+0000 7f356d622740 1 -- 172.21.15.43:0/2914309814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 msgr2=0x56461dfd02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:17.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.391+0000 7f356d622740 1 --2- 172.21.15.43:0/2914309814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461dfd02e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f355c00aa60 tx=0x7f355c005850 comp rx=0 tx=0).stop 2024-09-11T23:31:17.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.391+0000 7f356d622740 1 -- 172.21.15.43:0/2914309814 shutdown_connections 2024-09-11T23:31:17.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.391+0000 7f356d622740 1 --2- 172.21.15.43:0/2914309814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461dfd02e0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.391+0000 7f356d622740 1 --2- 172.21.15.43:0/2914309814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfaddc0 0x56461dfcfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.391+0000 7f356d622740 1 --2- 172.21.15.43:0/2914309814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfad4a0 0x56461dfb3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.391+0000 7f356d622740 1 -- 172.21.15.43:0/2914309814 >> 172.21.15.43:0/2914309814 conn(0x56461dfa9a30 msgr2=0x56461dfa8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:17.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.391+0000 7f356d622740 1 -- 172.21.15.43:0/2914309814 shutdown_connections 2024-09-11T23:31:17.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.391+0000 7f356d622740 1 -- 172.21.15.43:0/2914309814 wait complete. 2024-09-11T23:31:17.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.392+0000 7f356d622740 1 -- start start 2024-09-11T23:31:17.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.392+0000 7f356d622740 1 Processor -- start 2024-09-11T23:31:17.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.392+0000 7f356d622740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfad4a0 0x56461dfbe7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:17.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.392+0000 7f356d622740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfaddc0 0x56461dfbed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:17.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.392+0000 7f356d622740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461e031d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:17.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.392+0000 7f356d622740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f35641b2620 con 0x56461dfb8680 2024-09-11T23:31:17.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.392+0000 7f356d622740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f35641b3a90 con 0x56461dfad4a0 2024-09-11T23:31:17.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.392+0000 7f356d622740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f35641b2cf0 con 0x56461dfaddc0 2024-09-11T23:31:17.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.392+0000 7f35697e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461e031d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:17.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.392+0000 7f356d622740 10 client.0 _ll_register_callbacks cb 0x7f355c00a870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:17.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.392+0000 7f35697e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461e031d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41624/0 (socket says 172.21.15.43:41624) 2024-09-11T23:31:17.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.392+0000 7f35697e6640 1 -- 172.21.15.43:0/3686802921 learned_addr learned my addr 172.21.15.43:0/3686802921 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:17.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.392+0000 7f3563fff640 1 --2- 172.21.15.43:0/3686802921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfaddc0 0x56461dfbed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:17.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.392+0000 7f3568fe5640 1 --2- 172.21.15.43:0/3686802921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfad4a0 0x56461dfbe7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:17.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.393+0000 7f35697e6640 1 -- 172.21.15.43:0/3686802921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfaddc0 msgr2=0x56461dfbed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:17.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.393+0000 7f35697e6640 1 --2- 172.21.15.43:0/3686802921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfaddc0 0x56461dfbed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.393+0000 7f35697e6640 1 -- 172.21.15.43:0/3686802921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfad4a0 msgr2=0x56461dfbe7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:17.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.393+0000 7f35697e6640 1 --2- 172.21.15.43:0/3686802921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfad4a0 0x56461dfbe7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.393+0000 7f35697e6640 1 -- 172.21.15.43:0/3686802921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56461dfb8260 con 0x56461dfb8680 2024-09-11T23:31:17.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.393+0000 7f3563fff640 1 --2- 172.21.15.43:0/3686802921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfaddc0 0x56461dfbed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:31:17.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.393+0000 7f3568fe5640 1 --2- 172.21.15.43:0/3686802921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfad4a0 0x56461dfbe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:31:17.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.393+0000 7f35697e6640 1 --2- 172.21.15.43:0/3686802921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461e031d80 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f355c00a500 tx=0x7f355c02d640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c51b7e800bb3fc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:17.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.393+0000 7f3543fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:31:17.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.393+0000 7f3543fff640 1 -- 172.21.15.43:0/3686802921 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c044060 con 0x56461dfb8680 2024-09-11T23:31:17.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.393+0000 7f3543fff640 1 -- 172.21.15.43:0/3686802921 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8615+0+0 (secure 0 0 0) 0x7f355c005250 con 0x56461dfb8680 2024-09-11T23:31:17.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.393+0000 7f356d622740 1 -- 172.21.15.43:0/3686802921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f355c02d8f0 con 0x56461dfb8680 2024-09-11T23:31:17.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.393+0000 7f3543fff640 1 -- 172.21.15.43:0/3686802921 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f355c03b050 con 0x56461dfb8680 2024-09-11T23:31:17.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.393+0000 7f356d622740 10 client.16890 fetch_fsmap learned FSMap version 32 2024-09-11T23:31:17.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.393+0000 7f356d622740 1 -- 172.21.15.43:0/3686802921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56461e033230 con 0x56461dfb8680 2024-09-11T23:31:17.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.394+0000 7f3543fff640 1 -- 172.21.15.43:0/3686802921 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f355c02d8f0 con 0x56461dfb8680 2024-09-11T23:31:17.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.394+0000 7f3543fff640 1 -- 172.21.15.43:0/3686802921 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f355c019040 con 0x56461dfb8680 2024-09-11T23:31:17.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.394+0000 7f356d622740 10 client.16890 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:31:17.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.394+0000 7f356d622740 10 client.16890 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:31:17.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.394+0000 7f356d622740 20 client.16890 populate_metadata read hostname 'smithi043' 2024-09-11T23:31:17.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.394+0000 7f356d622740 1 --2- 172.21.15.43:0/3686802921 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56461dfc8060 0x56461e035880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:17.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.394+0000 7f356d622740 4 client.16890 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:31:17.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.394+0000 7f356d622740 1 -- 172.21.15.43:0/3686802921 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56461df1ad60 con 0x56461dfc8060 2024-09-11T23:31:17.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.394+0000 7f3568fe5640 1 --2- 172.21.15.43:0/3686802921 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56461dfc8060 0x56461e035880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:17.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.394+0000 7f3568fe5640 1 --2- 172.21.15.43:0/3686802921 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56461dfc8060 0x56461e035880 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1cd934cb7d010e5 server_cookie=224304db32dc1ebf in_seq=0 out_seq=0 2024-09-11T23:31:17.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.394+0000 7f3543fff640 10 client.16890 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:31:17.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.395+0000 7f3543fff640 1 -- 172.21.15.43:0/3686802921 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56461df1ad60 con 0x56461dfc8060 2024-09-11T23:31:17.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.395+0000 7f3543fff640 10 client.16890 handle_command_reply: tid=0 2024-09-11T23:31:17.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.395+0000 7f356d622740 1 client.16890 shutdown 2024-09-11T23:31:17.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.395+0000 7f356d622740 1 -- 172.21.15.43:0/3686802921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 msgr2=0x56461e031d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:17.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.395+0000 7f356d622740 1 --2- 172.21.15.43:0/3686802921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461e031d80 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f355c00a500 tx=0x7f355c02d640 comp rx=0 tx=0).stop 2024-09-11T23:31:17.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.395+0000 7f356d622740 1 -- 172.21.15.43:0/3686802921 shutdown_connections 2024-09-11T23:31:17.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.395+0000 7f356d622740 1 --2- 172.21.15.43:0/3686802921 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x56461dfc8060 0x56461e035880 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.395+0000 7f356d622740 1 --2- 172.21.15.43:0/3686802921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56461dfb8680 0x56461e031d80 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.395+0000 7f356d622740 1 --2- 172.21.15.43:0/3686802921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56461dfaddc0 0x56461dfbed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.395+0000 7f3543fff640 0 client.16890 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:31:17.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.395+0000 7f356d622740 1 --2- 172.21.15.43:0/3686802921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56461dfad4a0 0x56461dfbe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.395+0000 7f356d622740 1 -- 172.21.15.43:0/3686802921 >> 172.21.15.43:0/3686802921 conn(0x56461dfa9a30 msgr2=0x56461dfb7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:17.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.395+0000 7f356d622740 1 -- 172.21.15.43:0/3686802921 shutdown_connections 2024-09-11T23:31:17.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.395+0000 7f356d622740 1 -- 172.21.15.43:0/3686802921 wait complete. 2024-09-11T23:31:17.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.395+0000 7f356d622740 20 client.16890 trim_cache size 0 max 16384 2024-09-11T23:31:17.404 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:31:17.404 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-11T23:31:17.404 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-11T23:31:17.404 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:31:17.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.396+0000 7f356b249640 1 -- 172.21.15.43:0/3482667986 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f35500a2cf0 msgr2=0x7f35500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:17.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.396+0000 7f356b249640 1 --2- 172.21.15.43:0/3482667986 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f35500a2cf0 0x7f35500a51b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f355c00ac00 tx=0x7f355c00a250 comp rx=0 tx=0).stop 2024-09-11T23:31:17.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.396+0000 7f356b249640 1 -- 172.21.15.43:0/3482667986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3564102d40 msgr2=0x7f356419ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:17.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.396+0000 7f356b249640 1 --2- 172.21.15.43:0/3482667986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3564102d40 0x7f356419ea50 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f354c005ab0 tx=0x7f354c0042f0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.396+0000 7f356b249640 1 -- 172.21.15.43:0/3482667986 shutdown_connections 2024-09-11T23:31:17.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.396+0000 7f356b249640 1 --2- 172.21.15.43:0/3482667986 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f35500a2cf0 0x7f35500a51b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.396+0000 7f356b249640 1 --2- 172.21.15.43:0/3482667986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f356410c170 0x7f35641a3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.396+0000 7f356b249640 1 --2- 172.21.15.43:0/3482667986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3564103660 0x7f356419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.396+0000 7f356b249640 1 --2- 172.21.15.43:0/3482667986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3564102d40 0x7f356419ea50 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:17.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.396+0000 7f356b249640 1 -- 172.21.15.43:0/3482667986 >> 172.21.15.43:0/3482667986 conn(0x7f35640fc7a0 msgr2=0x7f35640fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:17.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.396+0000 7f356b249640 1 -- 172.21.15.43:0/3482667986 shutdown_connections 2024-09-11T23:31:17.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:17.396+0000 7f356b249640 1 -- 172.21.15.43:0/3482667986 wait complete. 2024-09-11T23:31:18.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:17 smithi043 ceph-mon[30950]: pgmap v292: 97 pgs: 11 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 82 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-11T23:31:18.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:17 smithi090 ceph-mon[35130]: pgmap v292: 97 pgs: 11 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 82 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-11T23:31:18.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:17 smithi097 ceph-mon[35562]: pgmap v292: 97 pgs: 11 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 82 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-11T23:31:18.305 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:31:18.305 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:31:18.569 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:31:19.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.049+0000 7f5013738640 1 -- 172.21.15.43:0/144939627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f500c102570 msgr2=0x7f500c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:19.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.049+0000 7f5013738640 1 --2- 172.21.15.43:0/144939627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f500c102570 0x7f500c1029b0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc009900 tx=0x7f4ffc02a480 comp rx=0 tx=0).stop 2024-09-11T23:31:19.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.049+0000 7f5013738640 1 -- 172.21.15.43:0/144939627 shutdown_connections 2024-09-11T23:31:19.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.049+0000 7f5013738640 1 --2- 172.21.15.43:0/144939627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f500c102ef0 0x7f500c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:19.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.049+0000 7f5013738640 1 --2- 172.21.15.43:0/144939627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f500c102570 0x7f500c1029b0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:19.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.049+0000 7f5013738640 1 --2- 172.21.15.43:0/144939627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f500c100e20 0x7f500c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:19.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.049+0000 7f5013738640 1 -- 172.21.15.43:0/144939627 >> 172.21.15.43:0/144939627 conn(0x7f500c0fc7d0 msgr2=0x7f500c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:19.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.049+0000 7f5013738640 1 -- 172.21.15.43:0/144939627 shutdown_connections 2024-09-11T23:31:19.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.049+0000 7f5013738640 1 -- 172.21.15.43:0/144939627 wait complete. 2024-09-11T23:31:19.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.049+0000 7f5013738640 1 Processor -- start 2024-09-11T23:31:19.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.049+0000 7f5013738640 1 -- start start 2024-09-11T23:31:19.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f5013738640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f500c100e20 0x7f500c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:19.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f5013738640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f500c102570 0x7f500c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:19.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f5013738640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f500c102ef0 0x7f500c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:19.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f5013738640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f500c110fa0 con 0x7f500c102ef0 2024-09-11T23:31:19.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f5013738640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f500c110e20 con 0x7f500c102570 2024-09-11T23:31:19.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f5013738640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f500c111120 con 0x7f500c100e20 2024-09-11T23:31:19.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f5011cd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f500c102ef0 0x7f500c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:19.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f5011cd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f500c102ef0 0x7f500c19ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41646/0 (socket says 172.21.15.43:41646) 2024-09-11T23:31:19.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f5011cd5640 1 -- 172.21.15.43:0/3726514956 learned_addr learned my addr 172.21.15.43:0/3726514956 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:19.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f50114d4640 1 --2- 172.21.15.43:0/3726514956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f500c100e20 0x7f500c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:19.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f5010cd3640 1 --2- 172.21.15.43:0/3726514956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f500c102570 0x7f500c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:19.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f5011cd5640 1 -- 172.21.15.43:0/3726514956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f500c100e20 msgr2=0x7f500c1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:19.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f5011cd5640 1 --2- 172.21.15.43:0/3726514956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f500c100e20 0x7f500c1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:19.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f5011cd5640 1 -- 172.21.15.43:0/3726514956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f500c102570 msgr2=0x7f500c198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:19.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f5011cd5640 1 --2- 172.21.15.43:0/3726514956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f500c102570 0x7f500c198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:19.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f5011cd5640 1 -- 172.21.15.43:0/3726514956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500c19d570 con 0x7f500c102ef0 2024-09-11T23:31:19.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f5011cd5640 1 --2- 172.21.15.43:0/3726514956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f500c102ef0 0x7f500c19ce90 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f500400b970 tx=0x7f500400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:19.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f50027fc640 1 -- 172.21.15.43:0/3726514956 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5004019070 con 0x7f500c102ef0 2024-09-11T23:31:19.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f50027fc640 1 -- 172.21.15.43:0/3726514956 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f500400cb40 con 0x7f500c102ef0 2024-09-11T23:31:19.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f5013738640 1 -- 172.21.15.43:0/3726514956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f500c19d860 con 0x7f500c102ef0 2024-09-11T23:31:19.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f50027fc640 1 -- 172.21.15.43:0/3726514956 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5004007650 con 0x7f500c102ef0 2024-09-11T23:31:19.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.050+0000 7f5013738640 1 -- 172.21.15.43:0/3726514956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f500c1a5140 con 0x7f500c102ef0 2024-09-11T23:31:19.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.052+0000 7f5013738640 1 -- 172.21.15.43:0/3726514956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f500c101200 con 0x7f500c102ef0 2024-09-11T23:31:19.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.053+0000 7f50027fc640 1 -- 172.21.15.43:0/3726514956 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f500401c070 con 0x7f500c102ef0 2024-09-11T23:31:19.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.053+0000 7f50027fc640 1 --2- 172.21.15.43:0/3726514956 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f4ff80a2d70 0x7f4ff80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:19.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.054+0000 7f50114d4640 1 --2- 172.21.15.43:0/3726514956 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f4ff80a2d70 0x7f4ff80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:19.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.054+0000 7f50027fc640 1 -- 172.21.15.43:0/3726514956 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8615+0+0 (secure 0 0 0) 0x7f50040ce980 con 0x7f500c102ef0 2024-09-11T23:31:19.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.054+0000 7f50114d4640 1 --2- 172.21.15.43:0/3726514956 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f4ff80a2d70 0x7f4ff80a5230 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f500800bbe0 tx=0x7f500800b390 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:19.404 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:31:19.404 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":32,"btime":"2024-09-11T23:29:51:458202+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:29:50.490489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25571":{"gid":25571,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/568491708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":568491708},{"type":"v1","addr":"172.21.15.90:6835","nonce":568491708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:31:19.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.058+0000 7f50027fc640 1 -- 172.21.15.43:0/3726514956 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f500409afa0 con 0x7f500c102ef0 2024-09-11T23:31:19.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.272+0000 7f5013738640 1 -- 172.21.15.43:0/3726514956 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f500c10a0a0 con 0x7f500c102ef0 2024-09-11T23:31:19.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.273+0000 7f50027fc640 1 -- 172.21.15.43:0/3726514956 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11895 (secure 0 0 0) 0x7f500409f2a0 con 0x7f500c102ef0 2024-09-11T23:31:19.404 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 32 2024-09-11T23:31:19.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.275+0000 7f5013738640 1 -- 172.21.15.43:0/3726514956 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f4ff80a2d70 msgr2=0x7f4ff80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:19.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.275+0000 7f5013738640 1 --2- 172.21.15.43:0/3726514956 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f4ff80a2d70 0x7f4ff80a5230 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f500800bbe0 tx=0x7f500800b390 comp rx=0 tx=0).stop 2024-09-11T23:31:19.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.275+0000 7f5013738640 1 -- 172.21.15.43:0/3726514956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f500c102ef0 msgr2=0x7f500c19ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:19.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.275+0000 7f5013738640 1 --2- 172.21.15.43:0/3726514956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f500c102ef0 0x7f500c19ce90 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f500400b970 tx=0x7f500400be30 comp rx=0 tx=0).stop 2024-09-11T23:31:19.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.275+0000 7f5013738640 1 -- 172.21.15.43:0/3726514956 shutdown_connections 2024-09-11T23:31:19.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.275+0000 7f5013738640 1 --2- 172.21.15.43:0/3726514956 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f4ff80a2d70 0x7f4ff80a5230 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:19.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.275+0000 7f5013738640 1 --2- 172.21.15.43:0/3726514956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f500c102ef0 0x7f500c19ce90 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:19.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.275+0000 7f5013738640 1 --2- 172.21.15.43:0/3726514956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f500c102570 0x7f500c198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:19.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.275+0000 7f5013738640 1 --2- 172.21.15.43:0/3726514956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f500c100e20 0x7f500c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:19.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.275+0000 7f5013738640 1 -- 172.21.15.43:0/3726514956 >> 172.21.15.43:0/3726514956 conn(0x7f500c0fc7d0 msgr2=0x7f500c06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:19.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.275+0000 7f5013738640 1 -- 172.21.15.43:0/3726514956 shutdown_connections 2024-09-11T23:31:19.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:19.275+0000 7f5013738640 1 -- 172.21.15.43:0/3726514956 wait complete. 2024-09-11T23:31:20.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:19 smithi090 ceph-mon[35130]: pgmap v293: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-11T23:31:20.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:20 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3726514956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:31:20.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:19 smithi097 ceph-mon[35562]: pgmap v293: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-11T23:31:20.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:19 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3726514956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:31:20.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:19 smithi043 ceph-mon[30950]: pgmap v293: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-11T23:31:20.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:19 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3726514956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:31:21.067 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:31:21.333 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:31:21.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.862+0000 7f5471ba5640 1 -- 172.21.15.43:0/2940580692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f546c103720 msgr2=0x7f546c107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:21.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.862+0000 7f5471ba5640 1 --2- 172.21.15.43:0/2940580692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f546c103720 0x7f546c107b60 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5460009930 tx=0x7f546002a420 comp rx=0 tx=0).stop 2024-09-11T23:31:21.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.862+0000 7f5471ba5640 1 -- 172.21.15.43:0/2940580692 shutdown_connections 2024-09-11T23:31:21.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.862+0000 7f5471ba5640 1 --2- 172.21.15.43:0/2940580692 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f546c1082a0 0x7f546c10ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.862+0000 7f5471ba5640 1 --2- 172.21.15.43:0/2940580692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f546c103720 0x7f546c107b60 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.862+0000 7f5471ba5640 1 --2- 172.21.15.43:0/2940580692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f546c102d70 0x7f546c103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.862+0000 7f5471ba5640 1 -- 172.21.15.43:0/2940580692 >> 172.21.15.43:0/2940580692 conn(0x7f546c0fc7d0 msgr2=0x7f546c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:21.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.862+0000 7f5471ba5640 1 -- 172.21.15.43:0/2940580692 shutdown_connections 2024-09-11T23:31:21.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.862+0000 7f5471ba5640 1 -- 172.21.15.43:0/2940580692 wait complete. 2024-09-11T23:31:21.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.862+0000 7f5471ba5640 1 Processor -- start 2024-09-11T23:31:21.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.862+0000 7f5471ba5640 1 -- start start 2024-09-11T23:31:21.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.862+0000 7f5471ba5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f546c102d70 0x7f546c1009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:21.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.862+0000 7f5471ba5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f546c103720 0x7f546c100f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:21.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.862+0000 7f5471ba5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f546c1082a0 0x7f546c101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:21.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.862+0000 7f5471ba5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f546c112060 con 0x7f546c102d70 2024-09-11T23:31:21.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.862+0000 7f5471ba5640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f546c111ee0 con 0x7f546c103720 2024-09-11T23:31:21.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.862+0000 7f5471ba5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1121e0 con 0x7f546c1082a0 2024-09-11T23:31:21.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.863+0000 7f546b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f546c102d70 0x7f546c1009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:21.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.863+0000 7f546b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f546c102d70 0x7f546c1009d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52544/0 (socket says 172.21.15.43:52544) 2024-09-11T23:31:21.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.863+0000 7f546b7fe640 1 -- 172.21.15.43:0/1948831155 learned_addr learned my addr 172.21.15.43:0/1948831155 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:21.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.863+0000 7f546affd640 1 --2- 172.21.15.43:0/1948831155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f546c103720 0x7f546c100f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:21.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.863+0000 7f546bfff640 1 --2- 172.21.15.43:0/1948831155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f546c1082a0 0x7f546c101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:21.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.863+0000 7f546b7fe640 1 -- 172.21.15.43:0/1948831155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f546c1082a0 msgr2=0x7f546c101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:21.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.863+0000 7f546b7fe640 1 --2- 172.21.15.43:0/1948831155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f546c1082a0 0x7f546c101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.863+0000 7f546b7fe640 1 -- 172.21.15.43:0/1948831155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f546c103720 msgr2=0x7f546c100f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:21.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.863+0000 7f546b7fe640 1 --2- 172.21.15.43:0/1948831155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f546c103720 0x7f546c100f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.863+0000 7f546b7fe640 1 -- 172.21.15.43:0/1948831155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546c1a6a10 con 0x7f546c102d70 2024-09-11T23:31:21.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.863+0000 7f546affd640 1 --2- 172.21.15.43:0/1948831155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f546c103720 0x7f546c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:31:21.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.863+0000 7f546b7fe640 1 --2- 172.21.15.43:0/1948831155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f546c102d70 0x7f546c1009d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f545c00eb90 tx=0x7f545c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:21.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.863+0000 7f5468ff9640 1 -- 172.21.15.43:0/1948831155 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c004d20 con 0x7f546c102d70 2024-09-11T23:31:21.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.863+0000 7f5468ff9640 1 -- 172.21.15.43:0/1948831155 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f545c00ee90 con 0x7f546c102d70 2024-09-11T23:31:21.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.863+0000 7f5471ba5640 1 -- 172.21.15.43:0/1948831155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546c1a6d00 con 0x7f546c102d70 2024-09-11T23:31:21.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.863+0000 7f5468ff9640 1 -- 172.21.15.43:0/1948831155 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c010830 con 0x7f546c102d70 2024-09-11T23:31:21.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.863+0000 7f5471ba5640 1 -- 172.21.15.43:0/1948831155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f546c1a71c0 con 0x7f546c102d70 2024-09-11T23:31:21.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.866+0000 7f5468ff9640 1 -- 172.21.15.43:0/1948831155 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f545c005020 con 0x7f546c102d70 2024-09-11T23:31:21.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.866+0000 7f5473f7e740 1 Processor -- start 2024-09-11T23:31:21.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.866+0000 7f5473f7e740 1 -- start start 2024-09-11T23:31:21.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.866+0000 7f5473f7e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562966894b40 0x562966893090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:21.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.866+0000 7f5473f7e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629668935d0 0x562966893a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:21.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.866+0000 7f5473f7e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56296689e370 0x5629668a0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:21.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.866+0000 7f5473f7e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5629667f92f0 con 0x56296689e370 2024-09-11T23:31:21.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.866+0000 7f5473f7e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5629667a0cf0 con 0x562966894b40 2024-09-11T23:31:21.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.866+0000 7f5473f7e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5629667f52b0 con 0x5629668935d0 2024-09-11T23:31:21.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.866+0000 7f546bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562966894b40 0x562966893090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:21.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.866+0000 7f546b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629668935d0 0x562966893a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:21.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.866+0000 7f546affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56296689e370 0x5629668a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:21.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.866+0000 7f546bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562966894b40 0x562966893090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:33546/0 (socket says 172.21.15.43:33546) 2024-09-11T23:31:21.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.866+0000 7f546bfff640 1 -- 172.21.15.43:0/1619049465 learned_addr learned my addr 172.21.15.43:0/1619049465 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:21.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.867+0000 7f5468ff9640 1 --2- 172.21.15.43:0/1948831155 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f54540a2d70 0x7f54540a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:21.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.867+0000 7f546affd640 1 -- 172.21.15.43:0/1619049465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629668935d0 msgr2=0x562966893a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:21.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.867+0000 7f546affd640 1 --2- 172.21.15.43:0/1619049465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629668935d0 0x562966893a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.867+0000 7f546affd640 1 -- 172.21.15.43:0/1619049465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562966894b40 msgr2=0x562966893090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.867+0000 7f546affd640 1 --2- 172.21.15.43:0/1619049465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562966894b40 0x562966893090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.867+0000 7f546affd640 1 -- 172.21.15.43:0/1619049465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629667853a0 con 0x56296689e370 2024-09-11T23:31:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.867+0000 7f546bfff640 1 --2- 172.21.15.43:0/1948831155 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f54540a2d70 0x7f54540a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.867+0000 7f546affd640 1 --2- 172.21.15.43:0/1619049465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56296689e370 0x5629668a0760 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f546002a940 tx=0x7f5460032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3f2f1f93615fee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.867+0000 7f5468ff9640 1 -- 172.21.15.43:0/1948831155 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8615+0+0 (secure 0 0 0) 0x7f545c0cfbd0 con 0x7f546c102d70 2024-09-11T23:31:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.867+0000 7f5451ffb640 1 -- 172.21.15.43:0/1619049465 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546003f070 con 0x56296689e370 2024-09-11T23:31:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.867+0000 7f5451ffb640 1 -- 172.21.15.43:0/1619049465 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54600317b0 con 0x56296689e370 2024-09-11T23:31:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.867+0000 7f5451ffb640 1 -- 172.21.15.43:0/1619049465 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460035070 con 0x56296689e370 2024-09-11T23:31:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.868+0000 7f5473f7e740 1 -- 172.21.15.43:0/1619049465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56296689e370 msgr2=0x5629668a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.868+0000 7f546bfff640 1 --2- 172.21.15.43:0/1948831155 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f54540a2d70 0x7f54540a5230 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5464005fd0 tx=0x7f54640046c0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.868+0000 7f5473f7e740 1 --2- 172.21.15.43:0/1619049465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56296689e370 0x5629668a0760 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f546002a940 tx=0x7f5460032cb0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.868+0000 7f5473f7e740 1 -- 172.21.15.43:0/1619049465 shutdown_connections 2024-09-11T23:31:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.868+0000 7f5473f7e740 1 --2- 172.21.15.43:0/1619049465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56296689e370 0x5629668a0760 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.868+0000 7f5473f7e740 1 --2- 172.21.15.43:0/1619049465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5629668935d0 0x562966893a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.868+0000 7f5473f7e740 1 --2- 172.21.15.43:0/1619049465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562966894b40 0x562966893090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.868+0000 7f5473f7e740 1 -- 172.21.15.43:0/1619049465 >> 172.21.15.43:0/1619049465 conn(0x56296685c310 msgr2=0x56296688fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.868+0000 7f5473f7e740 1 -- 172.21.15.43:0/1619049465 shutdown_connections 2024-09-11T23:31:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.868+0000 7f5473f7e740 1 -- 172.21.15.43:0/1619049465 wait complete. 2024-09-11T23:31:21.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.868+0000 7f5473f7e740 1 -- start start 2024-09-11T23:31:21.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.868+0000 7f5473f7e740 1 Processor -- start 2024-09-11T23:31:21.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f5473f7e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629668935d0 0x56296689dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:21.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f5473f7e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562966894b40 0x56296689b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:21.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f5473f7e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56296689e370 0x56296689b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:21.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f5473f7e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5629668a1010 con 0x5629668935d0 2024-09-11T23:31:21.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f5473f7e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5629667b4060 con 0x56296689e370 2024-09-11T23:31:21.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f5473f7e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5629668a1190 con 0x562966894b40 2024-09-11T23:31:21.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f5473f7e740 10 client.0 _ll_register_callbacks cb 0x5629665078c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:21.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f546b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629668935d0 0x56296689dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:21.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f546b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629668935d0 0x56296689dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52562/0 (socket says 172.21.15.43:52562) 2024-09-11T23:31:21.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f546affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562966894b40 0x56296689b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:21.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f546b7fe640 1 -- 172.21.15.43:0/3974075848 learned_addr learned my addr 172.21.15.43:0/3974075848 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:21.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f546bfff640 1 --2- 172.21.15.43:0/3974075848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56296689e370 0x56296689b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:21.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f546b7fe640 1 -- 172.21.15.43:0/3974075848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562966894b40 msgr2=0x56296689b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:21.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f546b7fe640 1 --2- 172.21.15.43:0/3974075848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562966894b40 0x56296689b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f546b7fe640 1 -- 172.21.15.43:0/3974075848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56296689e370 msgr2=0x56296689b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:21.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f546b7fe640 1 --2- 172.21.15.43:0/3974075848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56296689e370 0x56296689b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f546b7fe640 1 -- 172.21.15.43:0/3974075848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56296656a3c0 con 0x5629668935d0 2024-09-11T23:31:21.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f546b7fe640 1 --2- 172.21.15.43:0/3974075848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629668935d0 0x56296689dd80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f545c00cba0 tx=0x7f545c004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e890b95d48b3ec92 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:21.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f544affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:31:21.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f544affd640 1 -- 172.21.15.43:0/3974075848 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c09bbd0 con 0x5629668935d0 2024-09-11T23:31:21.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f544affd640 1 -- 172.21.15.43:0/3974075848 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8615+0+0 (secure 0 0 0) 0x7f545c005590 con 0x5629668935d0 2024-09-11T23:31:21.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.869+0000 7f5473f7e740 1 -- 172.21.15.43:0/3974075848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54600317b0 con 0x5629668935d0 2024-09-11T23:31:21.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.870+0000 7f544affd640 1 -- 172.21.15.43:0/3974075848 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f545c09b9f0 con 0x5629668935d0 2024-09-11T23:31:21.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.870+0000 7f5473f7e740 10 client.16908 fetch_fsmap learned FSMap version 32 2024-09-11T23:31:21.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.870+0000 7f5473f7e740 1 -- 172.21.15.43:0/3974075848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5629668acb40 con 0x5629668935d0 2024-09-11T23:31:21.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.870+0000 7f544affd640 1 -- 172.21.15.43:0/3974075848 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54600317b0 con 0x5629668935d0 2024-09-11T23:31:21.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.870+0000 7f544affd640 1 -- 172.21.15.43:0/3974075848 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f545c0a00c0 con 0x5629668935d0 2024-09-11T23:31:21.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.870+0000 7f5473f7e740 10 client.16908 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:31:21.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.870+0000 7f5473f7e740 10 client.16908 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T23:31:21.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.870+0000 7f5473f7e740 20 client.16908 populate_metadata read hostname 'smithi043' 2024-09-11T23:31:21.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.870+0000 7f5473f7e740 1 --2- 172.21.15.43:0/3974075848 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5629668ae000 0x5629668b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:21.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.870+0000 7f5473f7e740 4 client.16908 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:31:21.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.870+0000 7f5473f7e740 1 -- 172.21.15.43:0/3974075848 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562966806860 con 0x5629668ae000 2024-09-11T23:31:21.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.870+0000 7f546affd640 1 --2- 172.21.15.43:0/3974075848 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5629668ae000 0x5629668b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:21.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.871+0000 7f544affd640 10 client.16908 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:31:21.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.871+0000 7f546affd640 1 --2- 172.21.15.43:0/3974075848 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5629668ae000 0x5629668b0470 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=22f3f9cef35b8857 server_cookie=30feab7afb5e381c in_seq=0 out_seq=0 2024-09-11T23:31:21.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.871+0000 7f544affd640 1 -- 172.21.15.43:0/3974075848 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562966806860 con 0x5629668ae000 2024-09-11T23:31:21.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.871+0000 7f544affd640 10 client.16908 handle_command_reply: tid=0 2024-09-11T23:31:21.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.871+0000 7f5473f7e740 1 client.16908 shutdown 2024-09-11T23:31:21.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.871+0000 7f5473f7e740 1 -- 172.21.15.43:0/3974075848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629668935d0 msgr2=0x56296689dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:21.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.871+0000 7f5473f7e740 1 --2- 172.21.15.43:0/3974075848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629668935d0 0x56296689dd80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f545c00cba0 tx=0x7f545c004ec0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.872+0000 7f5473f7e740 1 -- 172.21.15.43:0/3974075848 shutdown_connections 2024-09-11T23:31:21.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.872+0000 7f5473f7e740 1 --2- 172.21.15.43:0/3974075848 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x5629668ae000 0x5629668b0470 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.872+0000 7f5473f7e740 1 --2- 172.21.15.43:0/3974075848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56296689e370 0x56296689b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.872+0000 7f5473f7e740 1 --2- 172.21.15.43:0/3974075848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562966894b40 0x56296689b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.872+0000 7f5473f7e740 1 --2- 172.21.15.43:0/3974075848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5629668935d0 0x56296689dd80 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.872+0000 7f544affd640 0 client.16908 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:31:21.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.872+0000 7f5473f7e740 1 -- 172.21.15.43:0/3974075848 >> 172.21.15.43:0/3974075848 conn(0x56296685c310 msgr2=0x562966895990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:21.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.872+0000 7f5473f7e740 1 -- 172.21.15.43:0/3974075848 shutdown_connections 2024-09-11T23:31:21.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.872+0000 7f5473f7e740 1 -- 172.21.15.43:0/3974075848 wait complete. 2024-09-11T23:31:21.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.872+0000 7f5473f7e740 20 client.16908 trim_cache size 0 max 16384 2024-09-11T23:31:21.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.885+0000 7f5473f7e740 1 Processor -- start 2024-09-11T23:31:21.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.885+0000 7f5473f7e740 1 -- start start 2024-09-11T23:31:21.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.885+0000 7f5473f7e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629668935d0 0x56296688ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:21.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.885+0000 7f5473f7e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562966894b40 0x562966896180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:21.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.885+0000 7f5473f7e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56296689e370 0x5629668966c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:21.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.885+0000 7f5473f7e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1b23d0 con 0x56296689e370 2024-09-11T23:31:21.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.885+0000 7f5473f7e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1afc70 con 0x5629668935d0 2024-09-11T23:31:21.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.885+0000 7f5473f7e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1adae0 con 0x562966894b40 2024-09-11T23:31:21.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.885+0000 7f546bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56296689e370 0x5629668966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:21.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.885+0000 7f546bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56296689e370 0x5629668966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52564/0 (socket says 172.21.15.43:52564) 2024-09-11T23:31:21.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.885+0000 7f546bfff640 1 -- 172.21.15.43:0/1096714145 learned_addr learned my addr 172.21.15.43:0/1096714145 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:21.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.885+0000 7f546b7fe640 1 --2- 172.21.15.43:0/1096714145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629668935d0 0x56296688ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:21.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.885+0000 7f546affd640 1 --2- 172.21.15.43:0/1096714145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562966894b40 0x562966896180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:21.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.885+0000 7f546bfff640 1 -- 172.21.15.43:0/1096714145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562966894b40 msgr2=0x562966896180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.885+0000 7f546bfff640 1 --2- 172.21.15.43:0/1096714145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562966894b40 0x562966896180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.885+0000 7f546bfff640 1 -- 172.21.15.43:0/1096714145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629668935d0 msgr2=0x56296688ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.885+0000 7f546bfff640 1 --2- 172.21.15.43:0/1096714145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629668935d0 0x56296688ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.885+0000 7f546bfff640 1 -- 172.21.15.43:0/1096714145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56296689bfd0 con 0x56296689e370 2024-09-11T23:31:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.885+0000 7f546affd640 1 --2- 172.21.15.43:0/1096714145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562966894b40 0x562966896180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:31:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.886+0000 7f546bfff640 1 --2- 172.21.15.43:0/1096714145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56296689e370 0x5629668966c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f546401f590 tx=0x7f5464014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f301fc8e42dcf37 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.886+0000 7f54517fa640 1 -- 172.21.15.43:0/1096714145 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464004230 con 0x56296689e370 2024-09-11T23:31:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.886+0000 7f54517fa640 1 -- 172.21.15.43:0/1096714145 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5464016070 con 0x56296689e370 2024-09-11T23:31:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.886+0000 7f54517fa640 1 -- 172.21.15.43:0/1096714145 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464003c60 con 0x56296689e370 2024-09-11T23:31:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.886+0000 7f5473f7e740 1 -- 172.21.15.43:0/1096714145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56296689e370 msgr2=0x5629668966c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.886+0000 7f5473f7e740 1 --2- 172.21.15.43:0/1096714145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56296689e370 0x5629668966c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f546401f590 tx=0x7f5464014000 comp rx=0 tx=0).stop 2024-09-11T23:31:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.886+0000 7f5473f7e740 1 -- 172.21.15.43:0/1096714145 shutdown_connections 2024-09-11T23:31:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.886+0000 7f5473f7e740 1 --2- 172.21.15.43:0/1096714145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56296689e370 0x5629668966c0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.886+0000 7f5473f7e740 1 --2- 172.21.15.43:0/1096714145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562966894b40 0x562966896180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.886+0000 7f5473f7e740 1 --2- 172.21.15.43:0/1096714145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629668935d0 0x56296688ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.886+0000 7f5473f7e740 1 -- 172.21.15.43:0/1096714145 >> 172.21.15.43:0/1096714145 conn(0x56296685c310 msgr2=0x56296688e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.887+0000 7f5473f7e740 1 -- 172.21.15.43:0/1096714145 shutdown_connections 2024-09-11T23:31:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.887+0000 7f5473f7e740 1 -- 172.21.15.43:0/1096714145 wait complete. 2024-09-11T23:31:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.887+0000 7f5473f7e740 1 -- start start 2024-09-11T23:31:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.887+0000 7f5473f7e740 1 Processor -- start 2024-09-11T23:31:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.887+0000 7f5473f7e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629668935d0 0x562966897f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.887+0000 7f5473f7e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562966894b40 0x562966898470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.887+0000 7f5473f7e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56296689e370 0x5629668989b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.887+0000 7f5473f7e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1ae130 con 0x56296689e370 2024-09-11T23:31:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.887+0000 7f5473f7e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1af5a0 con 0x5629668935d0 2024-09-11T23:31:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.887+0000 7f5473f7e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1ae800 con 0x562966894b40 2024-09-11T23:31:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.887+0000 7f5473f7e740 10 client.0 _ll_register_callbacks cb 0x7f545c004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.887+0000 7f546bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56296689e370 0x5629668989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.887+0000 7f546bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56296689e370 0x5629668989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52578/0 (socket says 172.21.15.43:52578) 2024-09-11T23:31:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.887+0000 7f546bfff640 1 -- 172.21.15.43:0/2861569567 learned_addr learned my addr 172.21.15.43:0/2861569567 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.887+0000 7f546b7fe640 1 --2- 172.21.15.43:0/2861569567 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629668935d0 0x562966897f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.887+0000 7f546affd640 1 --2- 172.21.15.43:0/2861569567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562966894b40 0x562966898470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.888+0000 7f546bfff640 1 -- 172.21.15.43:0/2861569567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562966894b40 msgr2=0x562966898470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.888+0000 7f546bfff640 1 --2- 172.21.15.43:0/2861569567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562966894b40 0x562966898470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.888+0000 7f546bfff640 1 -- 172.21.15.43:0/2861569567 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629668935d0 msgr2=0x562966897f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.888+0000 7f546bfff640 1 --2- 172.21.15.43:0/2861569567 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629668935d0 0x562966897f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.888+0000 7f546bfff640 1 -- 172.21.15.43:0/2861569567 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562966917d60 con 0x56296689e370 2024-09-11T23:31:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.888+0000 7f546affd640 1 --2- 172.21.15.43:0/2861569567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562966894b40 0x562966898470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:31:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.888+0000 7f546b7fe640 1 --2- 172.21.15.43:0/2861569567 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629668935d0 0x562966897f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:31:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.888+0000 7f546bfff640 1 --2- 172.21.15.43:0/2861569567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56296689e370 0x5629668989b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f546401f560 tx=0x7f54640129d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3cc300cb5cb097 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.888+0000 7f544affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:31:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.888+0000 7f544affd640 1 -- 172.21.15.43:0/2861569567 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464038060 con 0x56296689e370 2024-09-11T23:31:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.888+0000 7f544affd640 1 -- 172.21.15.43:0/2861569567 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8615+0+0 (secure 0 0 0) 0x7f5464028430 con 0x56296689e370 2024-09-11T23:31:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.888+0000 7f5473f7e740 1 -- 172.21.15.43:0/2861569567 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5464016070 con 0x56296689e370 2024-09-11T23:31:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.888+0000 7f544affd640 1 -- 172.21.15.43:0/2861569567 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5464028630 con 0x56296689e370 2024-09-11T23:31:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.888+0000 7f5473f7e740 10 client.16914 fetch_fsmap learned FSMap version 32 2024-09-11T23:31:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.888+0000 7f5473f7e740 1 -- 172.21.15.43:0/2861569567 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562966918eb0 con 0x56296689e370 2024-09-11T23:31:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.888+0000 7f544affd640 1 -- 172.21.15.43:0/2861569567 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5464016070 con 0x56296689e370 2024-09-11T23:31:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.889+0000 7f544affd640 1 -- 172.21.15.43:0/2861569567 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5464033670 con 0x56296689e370 2024-09-11T23:31:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.889+0000 7f5473f7e740 10 client.16914 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:31:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.889+0000 7f5473f7e740 10 client.16914 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T23:31:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.889+0000 7f5473f7e740 20 client.16914 populate_metadata read hostname 'smithi043' 2024-09-11T23:31:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.889+0000 7f5473f7e740 1 --2- 172.21.15.43:0/2861569567 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x562966919580 0x56296691b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.889+0000 7f5473f7e740 4 client.16914 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:31:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.889+0000 7f5473f7e740 1 -- 172.21.15.43:0/2861569567 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562966800d70 con 0x562966919580 2024-09-11T23:31:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.889+0000 7f546b7fe640 1 --2- 172.21.15.43:0/2861569567 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x562966919580 0x56296691b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.889+0000 7f544affd640 10 client.16914 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:31:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.889+0000 7f546b7fe640 1 --2- 172.21.15.43:0/2861569567 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x562966919580 0x56296691b9a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c67da34dab09e4c server_cookie=ddbffaf4d6ed5dc5 in_seq=0 out_seq=0 2024-09-11T23:31:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.889+0000 7f544affd640 1 -- 172.21.15.43:0/2861569567 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562966800d70 con 0x562966919580 2024-09-11T23:31:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.889+0000 7f544affd640 10 client.16914 handle_command_reply: tid=0 2024-09-11T23:31:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.889+0000 7f5473f7e740 1 client.16914 shutdown 2024-09-11T23:31:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f5473f7e740 1 -- 172.21.15.43:0/2861569567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56296689e370 msgr2=0x5629668989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f5473f7e740 1 --2- 172.21.15.43:0/2861569567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56296689e370 0x5629668989b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f546401f560 tx=0x7f54640129d0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f5473f7e740 1 -- 172.21.15.43:0/2861569567 shutdown_connections 2024-09-11T23:31:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f5473f7e740 1 --2- 172.21.15.43:0/2861569567 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x562966919580 0x56296691b9a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f5473f7e740 1 --2- 172.21.15.43:0/2861569567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56296689e370 0x5629668989b0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f5473f7e740 1 --2- 172.21.15.43:0/2861569567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562966894b40 0x562966898470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f5473f7e740 1 --2- 172.21.15.43:0/2861569567 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629668935d0 0x562966897f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f5473f7e740 1 -- 172.21.15.43:0/2861569567 >> 172.21.15.43:0/2861569567 conn(0x56296685c310 msgr2=0x56296688ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f544affd640 0 client.16914 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:31:21.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f5473f7e740 1 -- 172.21.15.43:0/2861569567 shutdown_connections 2024-09-11T23:31:21.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f5473f7e740 1 -- 172.21.15.43:0/2861569567 wait complete. 2024-09-11T23:31:21.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f5473f7e740 20 client.16914 trim_cache size 0 max 16384 2024-09-11T23:31:21.903 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:31:21.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f5471ba5640 1 -- 172.21.15.43:0/1948831155 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f54540a2d70 msgr2=0x7f54540a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:21.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f5471ba5640 1 --2- 172.21.15.43:0/1948831155 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f54540a2d70 0x7f54540a5230 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5464005fd0 tx=0x7f54640046c0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f5471ba5640 1 -- 172.21.15.43:0/1948831155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f546c102d70 msgr2=0x7f546c1009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:21.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f5471ba5640 1 --2- 172.21.15.43:0/1948831155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f546c102d70 0x7f546c1009d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f545c00eb90 tx=0x7f545c00c6a0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f5471ba5640 1 -- 172.21.15.43:0/1948831155 shutdown_connections 2024-09-11T23:31:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f5471ba5640 1 --2- 172.21.15.43:0/1948831155 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f54540a2d70 0x7f54540a5230 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f5471ba5640 1 --2- 172.21.15.43:0/1948831155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f546c1082a0 0x7f546c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f5471ba5640 1 --2- 172.21.15.43:0/1948831155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f546c103720 0x7f546c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f5471ba5640 1 --2- 172.21.15.43:0/1948831155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f546c102d70 0x7f546c1009d0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.890+0000 7f5471ba5640 1 -- 172.21.15.43:0/1948831155 >> 172.21.15.43:0/1948831155 conn(0x7f546c0fc7d0 msgr2=0x7f546c105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.891+0000 7f5471ba5640 1 -- 172.21.15.43:0/1948831155 shutdown_connections 2024-09-11T23:31:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:21.891+0000 7f5471ba5640 1 -- 172.21.15.43:0/1948831155 wait complete. 2024-09-11T23:31:22.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:22 smithi043 ceph-mon[30950]: pgmap v294: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 196 B/s wr, 7 op/s 2024-09-11T23:31:22.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:22 smithi090 ceph-mon[35130]: pgmap v294: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 196 B/s wr, 7 op/s 2024-09-11T23:31:22.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:22 smithi097 ceph-mon[35562]: pgmap v294: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 196 B/s wr, 7 op/s 2024-09-11T23:31:22.595 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:31:22.852 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:31:23.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.362+0000 7fca798d0640 1 -- 172.21.15.43:0/1942110190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca74102d40 msgr2=0x7fca74103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:23.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.362+0000 7fca798d0640 1 --2- 172.21.15.43:0/1942110190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca74102d40 0x7fca74103120 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fca64009900 tx=0x7fca6402a480 comp rx=0 tx=0).stop 2024-09-11T23:31:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.362+0000 7fca798d0640 1 -- 172.21.15.43:0/1942110190 shutdown_connections 2024-09-11T23:31:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.362+0000 7fca798d0640 1 --2- 172.21.15.43:0/1942110190 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca7410c170 0x7fca7410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.362+0000 7fca798d0640 1 --2- 172.21.15.43:0/1942110190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca74103660 0x7fca7410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.362+0000 7fca798d0640 1 --2- 172.21.15.43:0/1942110190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca74102d40 0x7fca74103120 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.362+0000 7fca798d0640 1 -- 172.21.15.43:0/1942110190 >> 172.21.15.43:0/1942110190 conn(0x7fca740fc7a0 msgr2=0x7fca740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.362+0000 7fca798d0640 1 -- 172.21.15.43:0/1942110190 shutdown_connections 2024-09-11T23:31:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.362+0000 7fca798d0640 1 -- 172.21.15.43:0/1942110190 wait complete. 2024-09-11T23:31:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.363+0000 7fca798d0640 1 Processor -- start 2024-09-11T23:31:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.363+0000 7fca798d0640 1 -- start start 2024-09-11T23:31:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.363+0000 7fca798d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca74102d40 0x7fca741984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.363+0000 7fca798d0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca74103660 0x7fca74198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.363+0000 7fca798d0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca7410c170 0x7fca7419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.363+0000 7fca798d0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fca74111f70 con 0x7fca74102d40 2024-09-11T23:31:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.363+0000 7fca798d0640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fca74111df0 con 0x7fca7410c170 2024-09-11T23:31:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.363+0000 7fca798d0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fca741120f0 con 0x7fca74103660 2024-09-11T23:31:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.363+0000 7fca72ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca74102d40 0x7fca741984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.363+0000 7fca72ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca74102d40 0x7fca741984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52600/0 (socket says 172.21.15.43:52600) 2024-09-11T23:31:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.363+0000 7fca72ffd640 1 -- 172.21.15.43:0/2737467607 learned_addr learned my addr 172.21.15.43:0/2737467607 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.364+0000 7fca737fe640 1 --2- 172.21.15.43:0/2737467607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca7410c170 0x7fca7419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.364+0000 7fca6bfff640 1 --2- 172.21.15.43:0/2737467607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca74103660 0x7fca74198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.364+0000 7fca72ffd640 1 -- 172.21.15.43:0/2737467607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca74103660 msgr2=0x7fca74198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.364+0000 7fca72ffd640 1 --2- 172.21.15.43:0/2737467607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca74103660 0x7fca74198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.364+0000 7fca72ffd640 1 -- 172.21.15.43:0/2737467607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca7410c170 msgr2=0x7fca7419cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.364+0000 7fca72ffd640 1 --2- 172.21.15.43:0/2737467607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca7410c170 0x7fca7419cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.364+0000 7fca72ffd640 1 -- 172.21.15.43:0/2737467607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7419d4a0 con 0x7fca74102d40 2024-09-11T23:31:23.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.364+0000 7fca737fe640 1 --2- 172.21.15.43:0/2737467607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca7410c170 0x7fca7419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:31:23.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.364+0000 7fca72ffd640 1 --2- 172.21.15.43:0/2737467607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca74102d40 0x7fca741984f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fca64006010 tx=0x7fca640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:23.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.364+0000 7fca788ce640 1 -- 172.21.15.43:0/2737467607 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6403f030 con 0x7fca74102d40 2024-09-11T23:31:23.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.364+0000 7fca788ce640 1 -- 172.21.15.43:0/2737467607 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca64003ce0 con 0x7fca74102d40 2024-09-11T23:31:23.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.364+0000 7fca788ce640 1 -- 172.21.15.43:0/2737467607 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64035070 con 0x7fca74102d40 2024-09-11T23:31:23.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.364+0000 7fca798d0640 1 -- 172.21.15.43:0/2737467607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca7419d730 con 0x7fca74102d40 2024-09-11T23:31:23.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.364+0000 7fca798d0640 1 -- 172.21.15.43:0/2737467607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca7419dbc0 con 0x7fca74102d40 2024-09-11T23:31:23.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.366+0000 7fca788ce640 1 -- 172.21.15.43:0/2737467607 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fca64042030 con 0x7fca74102d40 2024-09-11T23:31:23.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.367+0000 7fca7bca9740 1 Processor -- start 2024-09-11T23:31:23.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.367+0000 7fca7bca9740 1 -- start start 2024-09-11T23:31:23.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.367+0000 7fca7bca9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556439673b40 0x556439672090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:23.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.367+0000 7fca7bca9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564396725d0 0x556439672a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:23.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.367+0000 7fca7bca9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556439681780 0x556439683b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:23.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.367+0000 7fca7bca9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5564395d82f0 con 0x556439681780 2024-09-11T23:31:23.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.367+0000 7fca7bca9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55643957fcf0 con 0x556439673b40 2024-09-11T23:31:23.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.367+0000 7fca7bca9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5564395d42b0 con 0x5564396725d0 2024-09-11T23:31:23.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.367+0000 7fca6bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556439681780 0x556439683b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:23.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.367+0000 7fca6bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556439681780 0x556439683b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52608/0 (socket says 172.21.15.43:52608) 2024-09-11T23:31:23.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.367+0000 7fca6bfff640 1 -- 172.21.15.43:0/641374846 learned_addr learned my addr 172.21.15.43:0/641374846 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:23.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.367+0000 7fca72ffd640 1 --2- 172.21.15.43:0/641374846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564396725d0 0x556439672a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:23.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.367+0000 7fca737fe640 1 --2- 172.21.15.43:0/641374846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556439673b40 0x556439672090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:23.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.367+0000 7fca788ce640 1 --2- 172.21.15.43:0/2737467607 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fca600a2d70 0x7fca600a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:23.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.367+0000 7fca6bfff640 1 -- 172.21.15.43:0/641374846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564396725d0 msgr2=0x556439672a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:23.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.367+0000 7fca6bfff640 1 --2- 172.21.15.43:0/641374846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564396725d0 0x556439672a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.367+0000 7fca6bfff640 1 -- 172.21.15.43:0/641374846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556439673b40 msgr2=0x556439672090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:23.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.367+0000 7fca6bfff640 1 --2- 172.21.15.43:0/641374846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556439673b40 0x556439672090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.367+0000 7fca6bfff640 1 -- 172.21.15.43:0/641374846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564395643a0 con 0x556439681780 2024-09-11T23:31:23.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.368+0000 7fca737fe640 1 --2- 172.21.15.43:0/2737467607 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fca600a2d70 0x7fca600a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:23.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.368+0000 7fca788ce640 1 -- 172.21.15.43:0/2737467607 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8615+0+0 (secure 0 0 0) 0x7fca640efa80 con 0x7fca74102d40 2024-09-11T23:31:23.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.368+0000 7fca6bfff640 1 --2- 172.21.15.43:0/641374846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556439681780 0x556439683b70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00bc90 tx=0x7fca5c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d628818521ffeb9b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:23.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.368+0000 7fca69ffb640 1 -- 172.21.15.43:0/641374846 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c015070 con 0x556439681780 2024-09-11T23:31:23.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.368+0000 7fca69ffb640 1 -- 172.21.15.43:0/641374846 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca5c011920 con 0x556439681780 2024-09-11T23:31:23.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.368+0000 7fca69ffb640 1 -- 172.21.15.43:0/641374846 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c010950 con 0x556439681780 2024-09-11T23:31:23.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.369+0000 7fca7bca9740 1 -- 172.21.15.43:0/641374846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556439681780 msgr2=0x556439683b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:23.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.369+0000 7fca7bca9740 1 --2- 172.21.15.43:0/641374846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556439681780 0x556439683b70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00bc90 tx=0x7fca5c00f730 comp rx=0 tx=0).stop 2024-09-11T23:31:23.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.369+0000 7fca737fe640 1 --2- 172.21.15.43:0/2737467607 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fca600a2d70 0x7fca600a5230 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fca6c00ae80 tx=0x7fca6c004a40 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:23.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.369+0000 7fca7bca9740 1 -- 172.21.15.43:0/641374846 shutdown_connections 2024-09-11T23:31:23.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.369+0000 7fca7bca9740 1 --2- 172.21.15.43:0/641374846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556439681780 0x556439683b70 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.369+0000 7fca7bca9740 1 --2- 172.21.15.43:0/641374846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5564396725d0 0x556439672a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.369+0000 7fca7bca9740 1 --2- 172.21.15.43:0/641374846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556439673b40 0x556439672090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.369+0000 7fca7bca9740 1 -- 172.21.15.43:0/641374846 >> 172.21.15.43:0/641374846 conn(0x55643963b310 msgr2=0x55643966ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:23.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.369+0000 7fca7bca9740 1 -- 172.21.15.43:0/641374846 shutdown_connections 2024-09-11T23:31:23.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.369+0000 7fca7bca9740 1 -- 172.21.15.43:0/641374846 wait complete. 2024-09-11T23:31:23.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.369+0000 7fca7bca9740 1 -- start start 2024-09-11T23:31:23.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.369+0000 7fca7bca9740 1 Processor -- start 2024-09-11T23:31:23.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.370+0000 7fca7bca9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564396725d0 0x556439681140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:23.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.370+0000 7fca7bca9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556439673b40 0x55643967e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:23.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.370+0000 7fca7bca9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556439681780 0x55643967e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:23.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.370+0000 7fca7bca9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556439684480 con 0x556439673b40 2024-09-11T23:31:23.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.370+0000 7fca7bca9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556439593060 con 0x5564396725d0 2024-09-11T23:31:23.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.370+0000 7fca7bca9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556439684600 con 0x556439681780 2024-09-11T23:31:23.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.370+0000 7fca7bca9740 10 client.0 _ll_register_callbacks cb 0x5564392e68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:23.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.371+0000 7fca72ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564396725d0 0x556439681140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:23.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.371+0000 7fca72ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564396725d0 0x556439681140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:33624/0 (socket says 172.21.15.43:33624) 2024-09-11T23:31:23.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.371+0000 7fca72ffd640 1 -- 172.21.15.43:0/3443633045 learned_addr learned my addr 172.21.15.43:0/3443633045 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:23.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.371+0000 7fca6bfff640 1 --2- 172.21.15.43:0/3443633045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556439673b40 0x55643967e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:23.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.371+0000 7fca737fe640 1 --2- 172.21.15.43:0/3443633045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556439681780 0x55643967e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:23.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.371+0000 7fca6bfff640 1 -- 172.21.15.43:0/3443633045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556439681780 msgr2=0x55643967e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:23.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.371+0000 7fca6bfff640 1 --2- 172.21.15.43:0/3443633045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556439681780 0x55643967e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.371+0000 7fca6bfff640 1 -- 172.21.15.43:0/3443633045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564396725d0 msgr2=0x556439681140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:23.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.371+0000 7fca6bfff640 1 --2- 172.21.15.43:0/3443633045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564396725d0 0x556439681140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.371+0000 7fca6bfff640 1 -- 172.21.15.43:0/3443633045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564393493c0 con 0x556439673b40 2024-09-11T23:31:23.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.371+0000 7fca737fe640 1 --2- 172.21.15.43:0/3443633045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556439681780 0x55643967e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:31:23.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.371+0000 7fca6bfff640 1 --2- 172.21.15.43:0/3443633045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556439673b40 0x55643967e3f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fca5c006010 tx=0x7fca5c00ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e150c137745e5d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:23.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.371+0000 7fca52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:31:23.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.371+0000 7fca52ffd640 1 -- 172.21.15.43:0/3443633045 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c028060 con 0x556439673b40 2024-09-11T23:31:23.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.371+0000 7fca52ffd640 1 -- 172.21.15.43:0/3443633045 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8615+0+0 (secure 0 0 0) 0x7fca5c0116d0 con 0x556439673b40 2024-09-11T23:31:23.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.371+0000 7fca7bca9740 1 -- 172.21.15.43:0/3443633045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca5c011920 con 0x556439673b40 2024-09-11T23:31:23.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.372+0000 7fca52ffd640 1 -- 172.21.15.43:0/3443633045 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fca5c015070 con 0x556439673b40 2024-09-11T23:31:23.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.372+0000 7fca7bca9740 10 client.16926 fetch_fsmap learned FSMap version 32 2024-09-11T23:31:23.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.372+0000 7fca7bca9740 1 -- 172.21.15.43:0/3443633045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556439675310 con 0x556439673b40 2024-09-11T23:31:23.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.372+0000 7fca52ffd640 1 -- 172.21.15.43:0/3443633045 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca5c011920 con 0x556439673b40 2024-09-11T23:31:23.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.372+0000 7fca52ffd640 1 -- 172.21.15.43:0/3443633045 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fca5c0047c0 con 0x556439673b40 2024-09-11T23:31:23.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.372+0000 7fca7bca9740 10 client.16926 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:31:23.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.372+0000 7fca7bca9740 10 client.16926 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T23:31:23.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.372+0000 7fca7bca9740 20 client.16926 populate_metadata read hostname 'smithi043' 2024-09-11T23:31:23.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.372+0000 7fca7bca9740 1 --2- 172.21.15.43:0/3443633045 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5564396766c0 0x556439678b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:23.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.372+0000 7fca7bca9740 4 client.16926 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:31:23.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.372+0000 7fca7bca9740 1 -- 172.21.15.43:0/3443633045 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564395e5860 con 0x5564396766c0 2024-09-11T23:31:23.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.372+0000 7fca72ffd640 1 --2- 172.21.15.43:0/3443633045 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5564396766c0 0x556439678b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:23.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.373+0000 7fca72ffd640 1 --2- 172.21.15.43:0/3443633045 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5564396766c0 0x556439678b20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e948288fdba12b0d server_cookie=4cfdce6621fd310e in_seq=0 out_seq=0 2024-09-11T23:31:23.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.373+0000 7fca52ffd640 10 client.16926 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:31:23.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.374+0000 7fca52ffd640 1 -- 172.21.15.43:0/3443633045 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5564395e5860 con 0x5564396766c0 2024-09-11T23:31:23.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.374+0000 7fca52ffd640 10 client.16926 handle_command_reply: tid=0 2024-09-11T23:31:23.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.374+0000 7fca7bca9740 1 client.16926 shutdown 2024-09-11T23:31:23.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.374+0000 7fca7bca9740 1 -- 172.21.15.43:0/3443633045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556439673b40 msgr2=0x55643967e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:23.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.374+0000 7fca7bca9740 1 --2- 172.21.15.43:0/3443633045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556439673b40 0x55643967e3f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fca5c006010 tx=0x7fca5c00ba00 comp rx=0 tx=0).stop 2024-09-11T23:31:23.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.374+0000 7fca7bca9740 1 -- 172.21.15.43:0/3443633045 shutdown_connections 2024-09-11T23:31:23.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.374+0000 7fca7bca9740 1 --2- 172.21.15.43:0/3443633045 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x5564396766c0 0x556439678b20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.374+0000 7fca7bca9740 1 --2- 172.21.15.43:0/3443633045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556439681780 0x55643967e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.374+0000 7fca7bca9740 1 --2- 172.21.15.43:0/3443633045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556439673b40 0x55643967e3f0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.374+0000 7fca7bca9740 1 --2- 172.21.15.43:0/3443633045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564396725d0 0x556439681140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.374+0000 7fca52ffd640 0 client.16926 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:31:23.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.374+0000 7fca7bca9740 1 -- 172.21.15.43:0/3443633045 >> 172.21.15.43:0/3443633045 conn(0x55643963b310 msgr2=0x556439674960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:23.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.374+0000 7fca7bca9740 1 -- 172.21.15.43:0/3443633045 shutdown_connections 2024-09-11T23:31:23.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.374+0000 7fca7bca9740 1 -- 172.21.15.43:0/3443633045 wait complete. 2024-09-11T23:31:23.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.374+0000 7fca7bca9740 20 client.16926 trim_cache size 0 max 16384 2024-09-11T23:31:23.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.387+0000 7fca7bca9740 1 Processor -- start 2024-09-11T23:31:23.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.387+0000 7fca7bca9740 1 -- start start 2024-09-11T23:31:23.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.387+0000 7fca7bca9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564396725d0 0x556439679ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:23.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.387+0000 7fca7bca9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556439673b40 0x55643968d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:23.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.387+0000 7fca7bca9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556439681780 0x556439680360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:23.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.387+0000 7fca7bca9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fca741ae120 con 0x556439673b40 2024-09-11T23:31:23.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.387+0000 7fca7bca9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fca741ab9c0 con 0x5564396725d0 2024-09-11T23:31:23.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.387+0000 7fca7bca9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fca741a9830 con 0x556439681780 2024-09-11T23:31:23.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.388+0000 7fca6bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556439673b40 0x55643968d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:23.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.388+0000 7fca6bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556439673b40 0x55643968d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52624/0 (socket says 172.21.15.43:52624) 2024-09-11T23:31:23.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.388+0000 7fca6bfff640 1 -- 172.21.15.43:0/2623950978 learned_addr learned my addr 172.21.15.43:0/2623950978 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:23.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.388+0000 7fca72ffd640 1 --2- 172.21.15.43:0/2623950978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564396725d0 0x556439679ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:23.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.388+0000 7fca737fe640 1 --2- 172.21.15.43:0/2623950978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556439681780 0x556439680360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:23.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.388+0000 7fca6bfff640 1 -- 172.21.15.43:0/2623950978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556439681780 msgr2=0x556439680360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:23.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.388+0000 7fca6bfff640 1 --2- 172.21.15.43:0/2623950978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556439681780 0x556439680360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.388+0000 7fca6bfff640 1 -- 172.21.15.43:0/2623950978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564396725d0 msgr2=0x556439679ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:23.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.388+0000 7fca6bfff640 1 --2- 172.21.15.43:0/2623950978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564396725d0 0x556439679ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.388+0000 7fca6bfff640 1 -- 172.21.15.43:0/2623950978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55643966f960 con 0x556439673b40 2024-09-11T23:31:23.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.388+0000 7fca6bfff640 1 --2- 172.21.15.43:0/2623950978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556439673b40 0x55643968d720 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fca5c02d040 tx=0x7fca5c005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a403206b82811700 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:23.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.388+0000 7fca697fa640 1 -- 172.21.15.43:0/2623950978 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c028070 con 0x556439673b40 2024-09-11T23:31:23.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.388+0000 7fca697fa640 1 -- 172.21.15.43:0/2623950978 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca5c00bd80 con 0x556439673b40 2024-09-11T23:31:23.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.388+0000 7fca697fa640 1 -- 172.21.15.43:0/2623950978 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c010ad0 con 0x556439673b40 2024-09-11T23:31:23.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.389+0000 7fca7bca9740 1 -- 172.21.15.43:0/2623950978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556439673b40 msgr2=0x55643968d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:23.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.389+0000 7fca7bca9740 1 --2- 172.21.15.43:0/2623950978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556439673b40 0x55643968d720 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fca5c02d040 tx=0x7fca5c005cc0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.389+0000 7fca7bca9740 1 -- 172.21.15.43:0/2623950978 shutdown_connections 2024-09-11T23:31:23.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.389+0000 7fca7bca9740 1 --2- 172.21.15.43:0/2623950978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556439681780 0x556439680360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.389+0000 7fca7bca9740 1 --2- 172.21.15.43:0/2623950978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556439673b40 0x55643968d720 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.389+0000 7fca7bca9740 1 --2- 172.21.15.43:0/2623950978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564396725d0 0x556439679ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.389+0000 7fca7bca9740 1 -- 172.21.15.43:0/2623950978 >> 172.21.15.43:0/2623950978 conn(0x55643963b310 msgr2=0x55643967fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:23.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.389+0000 7fca7bca9740 1 -- 172.21.15.43:0/2623950978 shutdown_connections 2024-09-11T23:31:23.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.389+0000 7fca7bca9740 1 -- 172.21.15.43:0/2623950978 wait complete. 2024-09-11T23:31:23.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.389+0000 7fca7bca9740 1 -- start start 2024-09-11T23:31:23.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.389+0000 7fca7bca9740 1 Processor -- start 2024-09-11T23:31:23.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.390+0000 7fca7bca9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564396725d0 0x556439689b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:23.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.390+0000 7fca7bca9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556439673b40 0x55643968a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:23.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.390+0000 7fca7bca9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556439681780 0x55643969cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:23.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.390+0000 7fca7bca9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fca741a9e80 con 0x5564396725d0 2024-09-11T23:31:23.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.390+0000 7fca7bca9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fca741ab2f0 con 0x556439673b40 2024-09-11T23:31:23.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.390+0000 7fca7bca9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fca741aa550 con 0x556439681780 2024-09-11T23:31:23.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.390+0000 7fca7bca9740 10 client.0 _ll_register_callbacks cb 0x7fca640b8240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:23.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.390+0000 7fca72ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564396725d0 0x556439689b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:23.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.390+0000 7fca72ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564396725d0 0x556439689b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52640/0 (socket says 172.21.15.43:52640) 2024-09-11T23:31:23.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.390+0000 7fca72ffd640 1 -- 172.21.15.43:0/2267136944 learned_addr learned my addr 172.21.15.43:0/2267136944 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:23.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.390+0000 7fca6bfff640 1 --2- 172.21.15.43:0/2267136944 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556439673b40 0x55643968a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:23.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.390+0000 7fca737fe640 1 --2- 172.21.15.43:0/2267136944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556439681780 0x55643969cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:23.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.390+0000 7fca72ffd640 1 -- 172.21.15.43:0/2267136944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556439681780 msgr2=0x55643969cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:23.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.390+0000 7fca72ffd640 1 --2- 172.21.15.43:0/2267136944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556439681780 0x55643969cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.390+0000 7fca72ffd640 1 -- 172.21.15.43:0/2267136944 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556439673b40 msgr2=0x55643968a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:23.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.390+0000 7fca72ffd640 1 --2- 172.21.15.43:0/2267136944 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556439673b40 0x55643968a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.390+0000 7fca72ffd640 1 -- 172.21.15.43:0/2267136944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55643969d5d0 con 0x5564396725d0 2024-09-11T23:31:23.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.390+0000 7fca72ffd640 1 --2- 172.21.15.43:0/2267136944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564396725d0 0x556439689b10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fca640bbca0 tx=0x7fca640b82f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab642009c7811f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:23.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.390+0000 7fca52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:31:23.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.390+0000 7fca52ffd640 1 -- 172.21.15.43:0/2267136944 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64049040 con 0x5564396725d0 2024-09-11T23:31:23.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.390+0000 7fca52ffd640 1 -- 172.21.15.43:0/2267136944 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8615+0+0 (secure 0 0 0) 0x7fca640b8610 con 0x5564396725d0 2024-09-11T23:31:23.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.391+0000 7fca7bca9740 1 -- 172.21.15.43:0/2267136944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca5c00bd80 con 0x5564396725d0 2024-09-11T23:31:23.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.391+0000 7fca52ffd640 1 -- 172.21.15.43:0/2267136944 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fca640bb900 con 0x5564396725d0 2024-09-11T23:31:23.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.391+0000 7fca7bca9740 10 client.16932 fetch_fsmap learned FSMap version 32 2024-09-11T23:31:23.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.391+0000 7fca7bca9740 1 -- 172.21.15.43:0/2267136944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5564396894b0 con 0x5564396725d0 2024-09-11T23:31:23.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.391+0000 7fca52ffd640 1 -- 172.21.15.43:0/2267136944 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca5c00bd80 con 0x5564396725d0 2024-09-11T23:31:23.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.391+0000 7fca52ffd640 1 -- 172.21.15.43:0/2267136944 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fca640ed390 con 0x5564396725d0 2024-09-11T23:31:23.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.391+0000 7fca7bca9740 10 client.16932 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:31:23.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.391+0000 7fca7bca9740 10 client.16932 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T23:31:23.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.391+0000 7fca7bca9740 20 client.16932 populate_metadata read hostname 'smithi043' 2024-09-11T23:31:23.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.391+0000 7fca7bca9740 1 --2- 172.21.15.43:0/2267136944 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x556439700a10 0x556439702e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:23.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.391+0000 7fca7bca9740 4 client.16932 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:31:23.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.391+0000 7fca7bca9740 1 -- 172.21.15.43:0/2267136944 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5564395dfd70 con 0x556439700a10 2024-09-11T23:31:23.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.391+0000 7fca6bfff640 1 --2- 172.21.15.43:0/2267136944 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x556439700a10 0x556439702e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:23.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.392+0000 7fca6bfff640 1 --2- 172.21.15.43:0/2267136944 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x556439700a10 0x556439702e30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e496c1a8b8b0f0d1 server_cookie=5056abc892f69a76 in_seq=0 out_seq=0 2024-09-11T23:31:23.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.392+0000 7fca52ffd640 10 client.16932 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:31:23.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.392+0000 7fca52ffd640 1 -- 172.21.15.43:0/2267136944 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5564395dfd70 con 0x556439700a10 2024-09-11T23:31:23.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.392+0000 7fca52ffd640 10 client.16932 handle_command_reply: tid=0 2024-09-11T23:31:23.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.392+0000 7fca7bca9740 1 client.16932 shutdown 2024-09-11T23:31:23.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.392+0000 7fca7bca9740 1 -- 172.21.15.43:0/2267136944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564396725d0 msgr2=0x556439689b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:23.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.392+0000 7fca7bca9740 1 --2- 172.21.15.43:0/2267136944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564396725d0 0x556439689b10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fca640bbca0 tx=0x7fca640b82f0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.392+0000 7fca7bca9740 1 -- 172.21.15.43:0/2267136944 shutdown_connections 2024-09-11T23:31:23.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.392+0000 7fca7bca9740 1 --2- 172.21.15.43:0/2267136944 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x556439700a10 0x556439702e30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.392+0000 7fca52ffd640 0 client.16932 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:31:23.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.392+0000 7fca7bca9740 1 --2- 172.21.15.43:0/2267136944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556439681780 0x55643969cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.392+0000 7fca7bca9740 1 --2- 172.21.15.43:0/2267136944 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556439673b40 0x55643968a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.392+0000 7fca7bca9740 1 --2- 172.21.15.43:0/2267136944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5564396725d0 0x556439689b10 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.392+0000 7fca7bca9740 1 -- 172.21.15.43:0/2267136944 >> 172.21.15.43:0/2267136944 conn(0x55643963b310 msgr2=0x556439680470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:23.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.392+0000 7fca7bca9740 1 -- 172.21.15.43:0/2267136944 shutdown_connections 2024-09-11T23:31:23.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.392+0000 7fca7bca9740 1 -- 172.21.15.43:0/2267136944 wait complete. 2024-09-11T23:31:23.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.392+0000 7fca7bca9740 20 client.16932 trim_cache size 0 max 16384 2024-09-11T23:31:23.403 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:31:23.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.393+0000 7fca798d0640 1 -- 172.21.15.43:0/2737467607 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fca600a2d70 msgr2=0x7fca600a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:23.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.393+0000 7fca798d0640 1 --2- 172.21.15.43:0/2737467607 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fca600a2d70 0x7fca600a5230 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fca6c00ae80 tx=0x7fca6c004a40 comp rx=0 tx=0).stop 2024-09-11T23:31:23.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.393+0000 7fca798d0640 1 -- 172.21.15.43:0/2737467607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca74102d40 msgr2=0x7fca741984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:23.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.393+0000 7fca798d0640 1 --2- 172.21.15.43:0/2737467607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca74102d40 0x7fca741984f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fca64006010 tx=0x7fca640042f0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.393+0000 7fca798d0640 1 -- 172.21.15.43:0/2737467607 shutdown_connections 2024-09-11T23:31:23.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.393+0000 7fca798d0640 1 --2- 172.21.15.43:0/2737467607 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fca600a2d70 0x7fca600a5230 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.393+0000 7fca798d0640 1 --2- 172.21.15.43:0/2737467607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca7410c170 0x7fca7419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.393+0000 7fca798d0640 1 --2- 172.21.15.43:0/2737467607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca74103660 0x7fca74198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.393+0000 7fca798d0640 1 --2- 172.21.15.43:0/2737467607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca74102d40 0x7fca741984f0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:23.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.393+0000 7fca798d0640 1 -- 172.21.15.43:0/2737467607 >> 172.21.15.43:0/2737467607 conn(0x7fca740fc7a0 msgr2=0x7fca740fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:23.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.393+0000 7fca798d0640 1 -- 172.21.15.43:0/2737467607 shutdown_connections 2024-09-11T23:31:23.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:23.393+0000 7fca798d0640 1 -- 172.21.15.43:0/2737467607 wait complete. 2024-09-11T23:31:24.070 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:31:24.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:24 smithi090 ceph-mon[35130]: pgmap v295: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-11T23:31:24.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:24 smithi090 ceph-mon[35130]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T23:31:24.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:24 smithi097 ceph-mon[35562]: pgmap v295: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-11T23:31:24.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:24 smithi097 ceph-mon[35562]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T23:31:24.332 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:31:24.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:24 smithi043 ceph-mon[30950]: pgmap v295: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-11T23:31:24.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:24 smithi043 ceph-mon[30950]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T23:31:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.839+0000 7fed00f80640 1 -- 172.21.15.43:0/2572705306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecfc102790 msgr2=0x7fecfc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.839+0000 7fed00f80640 1 --2- 172.21.15.43:0/2572705306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecfc102790 0x7fecfc1067e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fece4009900 tx=0x7fece402a3f0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.839+0000 7fed00f80640 1 -- 172.21.15.43:0/2572705306 shutdown_connections 2024-09-11T23:31:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.839+0000 7fed00f80640 1 --2- 172.21.15.43:0/2572705306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fecfc106f10 0x7fecfc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.839+0000 7fed00f80640 1 --2- 172.21.15.43:0/2572705306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecfc102790 0x7fecfc1067e0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.839+0000 7fed00f80640 1 --2- 172.21.15.43:0/2572705306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecfc101de0 0x7fecfc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.839+0000 7fed00f80640 1 -- 172.21.15.43:0/2572705306 >> 172.21.15.43:0/2572705306 conn(0x7fecfc0fd8c0 msgr2=0x7fecfc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.839+0000 7fed00f80640 1 -- 172.21.15.43:0/2572705306 shutdown_connections 2024-09-11T23:31:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.839+0000 7fed00f80640 1 -- 172.21.15.43:0/2572705306 wait complete. 2024-09-11T23:31:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.840+0000 7fed00f80640 1 Processor -- start 2024-09-11T23:31:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.840+0000 7fed00f80640 1 -- start start 2024-09-11T23:31:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.840+0000 7fed00f80640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecfc101de0 0x7fecfc198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.840+0000 7fed00f80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecfc102790 0x7fecfc1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.840+0000 7fed00f80640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fecfc106f10 0x7fecfc19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.840+0000 7fed00f80640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc111f30 con 0x7fecfc102790 2024-09-11T23:31:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.840+0000 7fed00f80640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc111db0 con 0x7fecfc106f10 2024-09-11T23:31:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.840+0000 7fed00f80640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc1120b0 con 0x7fecfc101de0 2024-09-11T23:31:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.840+0000 7fecf9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecfc102790 0x7fecfc1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.840+0000 7fecf9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecfc102790 0x7fecfc1988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52662/0 (socket says 172.21.15.43:52662) 2024-09-11T23:31:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.840+0000 7fecf9d9b640 1 -- 172.21.15.43:0/2832362831 learned_addr learned my addr 172.21.15.43:0/2832362831 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.840+0000 7fecfa59c640 1 --2- 172.21.15.43:0/2832362831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecfc101de0 0x7fecfc198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.840+0000 7fecfad9d640 1 --2- 172.21.15.43:0/2832362831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fecfc106f10 0x7fecfc19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.841+0000 7fecf9d9b640 1 -- 172.21.15.43:0/2832362831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecfc101de0 msgr2=0x7fecfc198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.841+0000 7fecf9d9b640 1 --2- 172.21.15.43:0/2832362831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecfc101de0 0x7fecfc198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.841+0000 7fecf9d9b640 1 -- 172.21.15.43:0/2832362831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fecfc106f10 msgr2=0x7fecfc19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.841+0000 7fecf9d9b640 1 --2- 172.21.15.43:0/2832362831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fecfc106f10 0x7fecfc19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.841+0000 7fecf9d9b640 1 -- 172.21.15.43:0/2832362831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecfc19d330 con 0x7fecfc102790 2024-09-11T23:31:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.841+0000 7fecf9d9b640 1 --2- 172.21.15.43:0/2832362831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecfc102790 0x7fecfc1988c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fece4006010 tx=0x7fece40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.841+0000 7fecf37fe640 1 -- 172.21.15.43:0/2832362831 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece403f030 con 0x7fecfc102790 2024-09-11T23:31:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.841+0000 7fecf37fe640 1 -- 172.21.15.43:0/2832362831 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fece4003ce0 con 0x7fecfc102790 2024-09-11T23:31:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.841+0000 7fed00f80640 1 -- 172.21.15.43:0/2832362831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecfc19d5c0 con 0x7fecfc102790 2024-09-11T23:31:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.841+0000 7fecf37fe640 1 -- 172.21.15.43:0/2832362831 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4035070 con 0x7fecfc102790 2024-09-11T23:31:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.841+0000 7fed00f80640 1 -- 172.21.15.43:0/2832362831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecfc1a4f00 con 0x7fecfc102790 2024-09-11T23:31:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.843+0000 7fecf37fe640 1 -- 172.21.15.43:0/2832362831 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fece4042030 con 0x7fecfc102790 2024-09-11T23:31:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.843+0000 7fed03359740 1 Processor -- start 2024-09-11T23:31:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.844+0000 7fed03359740 1 -- start start 2024-09-11T23:31:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.844+0000 7fed03359740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36347f50 0x562c363464a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.844+0000 7fed03359740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c363469e0 0x562c36346e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.844+0000 7fed03359740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c36351780 0x562c36353b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.844+0000 7fed03359740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562c362a82f0 con 0x562c36351780 2024-09-11T23:31:24.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.844+0000 7fed03359740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562c3624fcf0 con 0x562c363469e0 2024-09-11T23:31:24.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.844+0000 7fed03359740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562c362a42b0 con 0x562c36347f50 2024-09-11T23:31:24.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.844+0000 7fecf37fe640 1 --2- 172.21.15.43:0/2832362831 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fece80a2cf0 0x7fece80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:24.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.844+0000 7fecf9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c36351780 0x562c36353b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:24.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.844+0000 7fecf9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c36351780 0x562c36353b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52668/0 (socket says 172.21.15.43:52668) 2024-09-11T23:31:24.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.844+0000 7fecf9d9b640 1 -- 172.21.15.43:0/219188769 learned_addr learned my addr 172.21.15.43:0/219188769 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:24.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.844+0000 7fecfa59c640 1 --2- 172.21.15.43:0/219188769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c363469e0 0x562c36346e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:24.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.844+0000 7fecfad9d640 1 --2- 172.21.15.43:0/219188769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36347f50 0x562c363464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:24.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.844+0000 7fecfad9d640 1 --2- 172.21.15.43:0/2832362831 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fece80a2cf0 0x7fece80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:24.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.844+0000 7fecf37fe640 1 -- 172.21.15.43:0/2832362831 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fece40ef850 con 0x7fecfc102790 2024-09-11T23:31:24.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.844+0000 7fecfad9d640 1 -- 172.21.15.43:0/219188769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c363469e0 msgr2=0x562c36346e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:24.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.844+0000 7fecfad9d640 1 --2- 172.21.15.43:0/219188769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c363469e0 0x562c36346e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.844+0000 7fecfad9d640 1 -- 172.21.15.43:0/219188769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c36351780 msgr2=0x562c36353b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:24.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.844+0000 7fecfad9d640 1 --2- 172.21.15.43:0/219188769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c36351780 0x562c36353b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.844+0000 7fecfad9d640 1 -- 172.21.15.43:0/219188769 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c362343a0 con 0x562c36347f50 2024-09-11T23:31:24.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.845+0000 7fecfad9d640 1 --2- 172.21.15.43:0/219188769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36347f50 0x562c363464a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fecf4005fd0 tx=0x7fecf4010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b57170d6ff8b6a74 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:24.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.845+0000 7fecfad9d640 1 --2- 172.21.15.43:0/2832362831 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fece80a2cf0 0x7fece80a51b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fecf4010c40 tx=0x7fecf400c000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:24.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.845+0000 7fecf0ff9640 1 -- 172.21.15.43:0/219188769 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4024070 con 0x562c36347f50 2024-09-11T23:31:24.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.845+0000 7fecf0ff9640 1 -- 172.21.15.43:0/219188769 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf4009c40 con 0x562c36347f50 2024-09-11T23:31:24.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.845+0000 7fecf0ff9640 1 -- 172.21.15.43:0/219188769 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4005280 con 0x562c36347f50 2024-09-11T23:31:24.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.846+0000 7fed03359740 1 -- 172.21.15.43:0/219188769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36347f50 msgr2=0x562c363464a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:24.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.846+0000 7fed03359740 1 --2- 172.21.15.43:0/219188769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36347f50 0x562c363464a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fecf4005fd0 tx=0x7fecf4010740 comp rx=0 tx=0).stop 2024-09-11T23:31:24.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.846+0000 7fed03359740 1 -- 172.21.15.43:0/219188769 shutdown_connections 2024-09-11T23:31:24.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.846+0000 7fed03359740 1 --2- 172.21.15.43:0/219188769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c36351780 0x562c36353b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.846+0000 7fed03359740 1 --2- 172.21.15.43:0/219188769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c363469e0 0x562c36346e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.846+0000 7fed03359740 1 --2- 172.21.15.43:0/219188769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36347f50 0x562c363464a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.846+0000 7fed03359740 1 -- 172.21.15.43:0/219188769 >> 172.21.15.43:0/219188769 conn(0x562c3630b310 msgr2=0x562c36343070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:24.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.846+0000 7fed03359740 1 -- 172.21.15.43:0/219188769 shutdown_connections 2024-09-11T23:31:24.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.846+0000 7fed03359740 1 -- 172.21.15.43:0/219188769 wait complete. 2024-09-11T23:31:24.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.846+0000 7fed03359740 1 -- start start 2024-09-11T23:31:24.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.846+0000 7fed03359740 1 Processor -- start 2024-09-11T23:31:24.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.846+0000 7fed03359740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c363469e0 0x562c36351140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:24.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.846+0000 7fed03359740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c36347f50 0x562c3634e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:24.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.846+0000 7fed03359740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36351780 0x562c3634eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:24.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.846+0000 7fed03359740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562c36354420 con 0x562c363469e0 2024-09-11T23:31:24.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.846+0000 7fed03359740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562c36263060 con 0x562c36347f50 2024-09-11T23:31:24.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.846+0000 7fed03359740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562c363545a0 con 0x562c36351780 2024-09-11T23:31:24.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.847+0000 7fed03359740 10 client.0 _ll_register_callbacks cb 0x562c35fb68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:24.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.847+0000 7fecfa59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c363469e0 0x562c36351140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:24.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.847+0000 7fecfa59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c363469e0 0x562c36351140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52682/0 (socket says 172.21.15.43:52682) 2024-09-11T23:31:24.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.847+0000 7fecfa59c640 1 -- 172.21.15.43:0/294203080 learned_addr learned my addr 172.21.15.43:0/294203080 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:24.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.847+0000 7fecfad9d640 1 --2- 172.21.15.43:0/294203080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36351780 0x562c3634eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:24.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.847+0000 7fecf9d9b640 1 --2- 172.21.15.43:0/294203080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c36347f50 0x562c3634e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:24.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.847+0000 7fecfa59c640 1 -- 172.21.15.43:0/294203080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36351780 msgr2=0x562c3634eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:24.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.847+0000 7fecfa59c640 1 --2- 172.21.15.43:0/294203080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36351780 0x562c3634eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.847+0000 7fecfa59c640 1 -- 172.21.15.43:0/294203080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c36347f50 msgr2=0x562c3634e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:24.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.847+0000 7fecfa59c640 1 --2- 172.21.15.43:0/294203080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c36347f50 0x562c3634e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.847+0000 7fecfa59c640 1 -- 172.21.15.43:0/294203080 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c360193c0 con 0x562c363469e0 2024-09-11T23:31:24.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.847+0000 7fecfa59c640 1 --2- 172.21.15.43:0/294203080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c363469e0 0x562c36351140 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fecec010a50 tx=0x7fecec010e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5f7b734fc8da0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:24.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.847+0000 7fece1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:31:24.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.847+0000 7fece1ffb640 1 -- 172.21.15.43:0/294203080 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecec01e060 con 0x562c363469e0 2024-09-11T23:31:24.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.847+0000 7fece1ffb640 1 -- 172.21.15.43:0/294203080 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fecec00fb10 con 0x562c363469e0 2024-09-11T23:31:24.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.847+0000 7fed03359740 1 -- 172.21.15.43:0/294203080 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecf4009c40 con 0x562c363469e0 2024-09-11T23:31:24.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.847+0000 7fece1ffb640 1 -- 172.21.15.43:0/294203080 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fecec00fd10 con 0x562c363469e0 2024-09-11T23:31:24.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.847+0000 7fed03359740 10 client.16944 fetch_fsmap learned FSMap version 32 2024-09-11T23:31:24.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.847+0000 7fed03359740 1 -- 172.21.15.43:0/294203080 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562c3635ff20 con 0x562c363469e0 2024-09-11T23:31:24.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.848+0000 7fece1ffb640 1 -- 172.21.15.43:0/294203080 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf4009c40 con 0x562c363469e0 2024-09-11T23:31:24.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.848+0000 7fece1ffb640 1 -- 172.21.15.43:0/294203080 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fecec00e440 con 0x562c363469e0 2024-09-11T23:31:24.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.848+0000 7fed03359740 10 client.16944 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:31:24.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.848+0000 7fed03359740 10 client.16944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:31:24.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.848+0000 7fed03359740 20 client.16944 populate_metadata read hostname 'smithi043' 2024-09-11T23:31:24.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.848+0000 7fed03359740 1 --2- 172.21.15.43:0/294203080 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562c363613e0 0x562c36363830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:24.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.848+0000 7fed03359740 4 client.16944 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:31:24.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.848+0000 7fed03359740 1 -- 172.21.15.43:0/294203080 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c362b5860 con 0x562c363613e0 2024-09-11T23:31:24.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.848+0000 7fecf9d9b640 1 --2- 172.21.15.43:0/294203080 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562c363613e0 0x562c36363830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:24.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.849+0000 7fecf9d9b640 1 --2- 172.21.15.43:0/294203080 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562c363613e0 0x562c36363830 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c702de5dfebbe5f server_cookie=253a5c35200e6fce in_seq=0 out_seq=0 2024-09-11T23:31:24.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.849+0000 7fece1ffb640 10 client.16944 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:31:24.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.849+0000 7fece1ffb640 1 -- 172.21.15.43:0/294203080 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c362b5860 con 0x562c363613e0 2024-09-11T23:31:24.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.849+0000 7fece1ffb640 10 client.16944 handle_command_reply: tid=0 2024-09-11T23:31:24.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.849+0000 7fed03359740 1 client.16944 shutdown 2024-09-11T23:31:24.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.850+0000 7fed03359740 1 -- 172.21.15.43:0/294203080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c363469e0 msgr2=0x562c36351140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:24.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.850+0000 7fed03359740 1 --2- 172.21.15.43:0/294203080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c363469e0 0x562c36351140 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fecec010a50 tx=0x7fecec010e50 comp rx=0 tx=0).stop 2024-09-11T23:31:24.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.850+0000 7fed03359740 1 -- 172.21.15.43:0/294203080 shutdown_connections 2024-09-11T23:31:24.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.850+0000 7fed03359740 1 --2- 172.21.15.43:0/294203080 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562c363613e0 0x562c36363830 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.850+0000 7fed03359740 1 --2- 172.21.15.43:0/294203080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36351780 0x562c3634eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.850+0000 7fed03359740 1 --2- 172.21.15.43:0/294203080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c36347f50 0x562c3634e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.850+0000 7fed03359740 1 --2- 172.21.15.43:0/294203080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c363469e0 0x562c36351140 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.850+0000 7fece1ffb640 0 client.16944 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:31:24.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.850+0000 7fed03359740 1 -- 172.21.15.43:0/294203080 >> 172.21.15.43:0/294203080 conn(0x562c3630b310 msgr2=0x562c36348d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:24.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.850+0000 7fed03359740 1 -- 172.21.15.43:0/294203080 shutdown_connections 2024-09-11T23:31:24.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.850+0000 7fed03359740 1 -- 172.21.15.43:0/294203080 wait complete. 2024-09-11T23:31:24.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.850+0000 7fed03359740 20 client.16944 trim_cache size 0 max 16384 2024-09-11T23:31:24.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.863+0000 7fed03359740 1 Processor -- start 2024-09-11T23:31:24.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.863+0000 7fed03359740 1 -- start start 2024-09-11T23:31:24.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.863+0000 7fed03359740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c363469e0 0x562c3635d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:24.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.863+0000 7fed03359740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c36347f50 0x562c3634dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:24.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.863+0000 7fed03359740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36351780 0x562c36369100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:24.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.863+0000 7fed03359740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc1adf90 con 0x562c363469e0 2024-09-11T23:31:24.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.863+0000 7fed03359740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc1ab830 con 0x562c36347f50 2024-09-11T23:31:24.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.863+0000 7fed03359740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc1a96a0 con 0x562c36351780 2024-09-11T23:31:24.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.863+0000 7fecfa59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c363469e0 0x562c3635d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:24.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.863+0000 7fecfa59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c363469e0 0x562c3635d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52694/0 (socket says 172.21.15.43:52694) 2024-09-11T23:31:24.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.863+0000 7fecfa59c640 1 -- 172.21.15.43:0/3217183729 learned_addr learned my addr 172.21.15.43:0/3217183729 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:24.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.864+0000 7fecfad9d640 1 --2- 172.21.15.43:0/3217183729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36351780 0x562c36369100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:24.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.864+0000 7fecf9d9b640 1 --2- 172.21.15.43:0/3217183729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c36347f50 0x562c3634dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:24.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.864+0000 7fecfa59c640 1 -- 172.21.15.43:0/3217183729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36351780 msgr2=0x562c36369100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:24.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.864+0000 7fecfa59c640 1 --2- 172.21.15.43:0/3217183729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36351780 0x562c36369100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.864+0000 7fecfa59c640 1 -- 172.21.15.43:0/3217183729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c36347f50 msgr2=0x562c3634dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:24.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.864+0000 7fecfa59c640 1 --2- 172.21.15.43:0/3217183729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c36347f50 0x562c3634dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.864+0000 7fecfa59c640 1 -- 172.21.15.43:0/3217183729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c363431f0 con 0x562c363469e0 2024-09-11T23:31:24.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.864+0000 7fecfad9d640 1 --2- 172.21.15.43:0/3217183729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36351780 0x562c36369100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:31:24.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.864+0000 7fecfa59c640 1 --2- 172.21.15.43:0/3217183729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c363469e0 0x562c3635d6c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fecec010a20 tx=0x7fecec0106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f80dd7a0c982411 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:24.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.864+0000 7fece3fff640 1 -- 172.21.15.43:0/3217183729 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecec030070 con 0x562c363469e0 2024-09-11T23:31:24.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.864+0000 7fece3fff640 1 -- 172.21.15.43:0/3217183729 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecec00a340 con 0x562c363469e0 2024-09-11T23:31:24.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.864+0000 7fece3fff640 1 -- 172.21.15.43:0/3217183729 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecec02b860 con 0x562c363469e0 2024-09-11T23:31:24.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.865+0000 7fed03359740 1 -- 172.21.15.43:0/3217183729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c363469e0 msgr2=0x562c3635d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:24.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.865+0000 7fed03359740 1 --2- 172.21.15.43:0/3217183729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c363469e0 0x562c3635d6c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fecec010a20 tx=0x7fecec0106e0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.865+0000 7fed03359740 1 -- 172.21.15.43:0/3217183729 shutdown_connections 2024-09-11T23:31:24.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.865+0000 7fed03359740 1 --2- 172.21.15.43:0/3217183729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36351780 0x562c36369100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.865+0000 7fed03359740 1 --2- 172.21.15.43:0/3217183729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c36347f50 0x562c3634dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.865+0000 7fed03359740 1 --2- 172.21.15.43:0/3217183729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c363469e0 0x562c3635d6c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.865+0000 7fed03359740 1 -- 172.21.15.43:0/3217183729 >> 172.21.15.43:0/3217183729 conn(0x562c3630b310 msgr2=0x562c3634d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:24.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.865+0000 7fed03359740 1 -- 172.21.15.43:0/3217183729 shutdown_connections 2024-09-11T23:31:24.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.865+0000 7fed03359740 1 -- 172.21.15.43:0/3217183729 wait complete. 2024-09-11T23:31:24.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.865+0000 7fed03359740 1 -- start start 2024-09-11T23:31:24.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.865+0000 7fed03359740 1 Processor -- start 2024-09-11T23:31:24.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fed03359740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c363469e0 0x562c3634b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:24.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fed03359740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c36347f50 0x562c3634b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:24.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fed03359740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36351780 0x562c3634bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:24.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fed03359740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc1a9cf0 con 0x562c363469e0 2024-09-11T23:31:24.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fed03359740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc1ab160 con 0x562c36347f50 2024-09-11T23:31:24.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fed03359740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc1aa3c0 con 0x562c36351780 2024-09-11T23:31:24.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fed03359740 10 client.0 _ll_register_callbacks cb 0x7fecec00a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:24.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fecfa59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c363469e0 0x562c3634b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:24.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fecfa59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c363469e0 0x562c3634b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52706/0 (socket says 172.21.15.43:52706) 2024-09-11T23:31:24.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fecfa59c640 1 -- 172.21.15.43:0/3913630486 learned_addr learned my addr 172.21.15.43:0/3913630486 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:24.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fecfad9d640 1 --2- 172.21.15.43:0/3913630486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36351780 0x562c3634bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:24.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fecf9d9b640 1 --2- 172.21.15.43:0/3913630486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c36347f50 0x562c3634b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:24.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fecfa59c640 1 -- 172.21.15.43:0/3913630486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36351780 msgr2=0x562c3634bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:24.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fecfa59c640 1 --2- 172.21.15.43:0/3913630486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36351780 0x562c3634bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fecfa59c640 1 -- 172.21.15.43:0/3913630486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c36347f50 msgr2=0x562c3634b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:24.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fecfa59c640 1 --2- 172.21.15.43:0/3913630486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c36347f50 0x562c3634b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fecfa59c640 1 -- 172.21.15.43:0/3913630486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c363514b0 con 0x562c363469e0 2024-09-11T23:31:24.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fecfa59c640 1 --2- 172.21.15.43:0/3913630486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c363469e0 0x562c3634b340 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fecec000c00 tx=0x7fecec010e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba32ed01e22b55f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:24.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fece1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:31:24.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fece1ffb640 1 -- 172.21.15.43:0/3913630486 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecec039060 con 0x562c363469e0 2024-09-11T23:31:24.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fece1ffb640 1 -- 172.21.15.43:0/3913630486 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fecec00fb10 con 0x562c363469e0 2024-09-11T23:31:24.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.866+0000 7fed03359740 1 -- 172.21.15.43:0/3913630486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecec00a340 con 0x562c363469e0 2024-09-11T23:31:24.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.867+0000 7fece1ffb640 1 -- 172.21.15.43:0/3913630486 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fecec01e070 con 0x562c363469e0 2024-09-11T23:31:24.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.867+0000 7fed03359740 10 client.16950 fetch_fsmap learned FSMap version 32 2024-09-11T23:31:24.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.867+0000 7fed03359740 1 -- 172.21.15.43:0/3913630486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562c363cc050 con 0x562c363469e0 2024-09-11T23:31:24.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.867+0000 7fece1ffb640 1 -- 172.21.15.43:0/3913630486 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecec00a340 con 0x562c363469e0 2024-09-11T23:31:24.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.867+0000 7fece1ffb640 1 -- 172.21.15.43:0/3913630486 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fecec02ba50 con 0x562c363469e0 2024-09-11T23:31:24.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.867+0000 7fed03359740 10 client.16950 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:31:24.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.867+0000 7fed03359740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:31:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.867+0000 7fed03359740 20 client.16950 populate_metadata read hostname 'smithi043' 2024-09-11T23:31:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.867+0000 7fed03359740 1 --2- 172.21.15.43:0/3913630486 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562c363cc6b0 0x562c363ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.867+0000 7fed03359740 4 client.16950 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:31:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.867+0000 7fed03359740 1 -- 172.21.15.43:0/3913630486 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c362afd70 con 0x562c363cc6b0 2024-09-11T23:31:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.867+0000 7fecf9d9b640 1 --2- 172.21.15.43:0/3913630486 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562c363cc6b0 0x562c363ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.868+0000 7fecf9d9b640 1 --2- 172.21.15.43:0/3913630486 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562c363cc6b0 0x562c363ceaa0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db6e0e9bbf4d7eaa server_cookie=71b8e41bfc2e00fd in_seq=0 out_seq=0 2024-09-11T23:31:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.868+0000 7fece1ffb640 10 client.16950 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:31:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.868+0000 7fece1ffb640 1 -- 172.21.15.43:0/3913630486 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c362afd70 con 0x562c363cc6b0 2024-09-11T23:31:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.868+0000 7fece1ffb640 10 client.16950 handle_command_reply: tid=0 2024-09-11T23:31:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.868+0000 7fed03359740 1 client.16950 shutdown 2024-09-11T23:31:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.868+0000 7fed03359740 1 -- 172.21.15.43:0/3913630486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c363469e0 msgr2=0x562c3634b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.868+0000 7fed03359740 1 --2- 172.21.15.43:0/3913630486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c363469e0 0x562c3634b340 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fecec000c00 tx=0x7fecec010e50 comp rx=0 tx=0).stop 2024-09-11T23:31:24.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.868+0000 7fed03359740 1 -- 172.21.15.43:0/3913630486 shutdown_connections 2024-09-11T23:31:24.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.868+0000 7fed03359740 1 --2- 172.21.15.43:0/3913630486 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x562c363cc6b0 0x562c363ceaa0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.868+0000 7fed03359740 1 --2- 172.21.15.43:0/3913630486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c36351780 0x562c3634bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.868+0000 7fed03359740 1 --2- 172.21.15.43:0/3913630486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c36347f50 0x562c3634b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.868+0000 7fed03359740 1 --2- 172.21.15.43:0/3913630486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562c363469e0 0x562c3634b340 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.868+0000 7fed03359740 1 -- 172.21.15.43:0/3913630486 >> 172.21.15.43:0/3913630486 conn(0x562c3630b310 msgr2=0x562c36348520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:24.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.868+0000 7fece1ffb640 0 client.16950 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:31:24.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.868+0000 7fed03359740 1 -- 172.21.15.43:0/3913630486 shutdown_connections 2024-09-11T23:31:24.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.868+0000 7fed03359740 1 -- 172.21.15.43:0/3913630486 wait complete. 2024-09-11T23:31:24.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.868+0000 7fed03359740 20 client.16950 trim_cache size 0 max 16384 2024-09-11T23:31:24.884 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:31:24.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.869+0000 7fed00f80640 1 -- 172.21.15.43:0/2832362831 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fece80a2cf0 msgr2=0x7fece80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:24.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.869+0000 7fed00f80640 1 --2- 172.21.15.43:0/2832362831 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fece80a2cf0 0x7fece80a51b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fecf4010c40 tx=0x7fecf400c000 comp rx=0 tx=0).stop 2024-09-11T23:31:24.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.869+0000 7fed00f80640 1 -- 172.21.15.43:0/2832362831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecfc102790 msgr2=0x7fecfc1988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:24.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.869+0000 7fed00f80640 1 --2- 172.21.15.43:0/2832362831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecfc102790 0x7fecfc1988c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fece4006010 tx=0x7fece40042f0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.869+0000 7fed00f80640 1 -- 172.21.15.43:0/2832362831 shutdown_connections 2024-09-11T23:31:24.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.869+0000 7fed00f80640 1 --2- 172.21.15.43:0/2832362831 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fece80a2cf0 0x7fece80a51b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.869+0000 7fed00f80640 1 --2- 172.21.15.43:0/2832362831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fecfc106f10 0x7fecfc19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.869+0000 7fed00f80640 1 --2- 172.21.15.43:0/2832362831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecfc102790 0x7fecfc1988c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.869+0000 7fed00f80640 1 --2- 172.21.15.43:0/2832362831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecfc101de0 0x7fecfc198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:24.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.869+0000 7fed00f80640 1 -- 172.21.15.43:0/2832362831 >> 172.21.15.43:0/2832362831 conn(0x7fecfc0fd8c0 msgr2=0x7fecfc0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:24.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.869+0000 7fed00f80640 1 -- 172.21.15.43:0/2832362831 shutdown_connections 2024-09-11T23:31:24.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:24.869+0000 7fed00f80640 1 -- 172.21.15.43:0/2832362831 wait complete. 2024-09-11T23:31:26.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:26 smithi090 ceph-mon[35130]: pgmap v297: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-11T23:31:26.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:26 smithi097 ceph-mon[35562]: pgmap v297: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-11T23:31:26.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:26 smithi043 ceph-mon[30950]: pgmap v297: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-11T23:31:26.550 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:31:26.550 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:31:26.820 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:31:27.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.307+0000 7f46ef0c6640 1 -- 172.21.15.43:0/698384205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e8106a80 msgr2=0x7f46e8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:27.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.307+0000 7f46ef0c6640 1 --2- 172.21.15.43:0/698384205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e8106a80 0x7f46e8106e60 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009930 tx=0x7f46d802a420 comp rx=0 tx=0).stop 2024-09-11T23:31:27.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.308+0000 7f46ef0c6640 1 -- 172.21.15.43:0/698384205 shutdown_connections 2024-09-11T23:31:27.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.308+0000 7f46ef0c6640 1 --2- 172.21.15.43:0/698384205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46e8101420 0x7f46e810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.308+0000 7f46ef0c6640 1 --2- 172.21.15.43:0/698384205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46e8100a80 0x7f46e8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.308+0000 7f46ef0c6640 1 --2- 172.21.15.43:0/698384205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e8106a80 0x7f46e8106e60 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.308+0000 7f46ef0c6640 1 -- 172.21.15.43:0/698384205 >> 172.21.15.43:0/698384205 conn(0x7f46e80fc7a0 msgr2=0x7f46e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:27.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.308+0000 7f46ef0c6640 1 -- 172.21.15.43:0/698384205 shutdown_connections 2024-09-11T23:31:27.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.308+0000 7f46ef0c6640 1 -- 172.21.15.43:0/698384205 wait complete. 2024-09-11T23:31:27.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.308+0000 7f46ef0c6640 1 Processor -- start 2024-09-11T23:31:27.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.308+0000 7f46ef0c6640 1 -- start start 2024-09-11T23:31:27.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.309+0000 7f46ef0c6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46e8100a80 0x7f46e81992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:27.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.309+0000 7f46ef0c6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46e8101420 0x7f46e8199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:27.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.309+0000 7f46ef0c6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e8106a80 0x7f46e81a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:27.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.309+0000 7f46ef0c6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f46e8110e80 con 0x7f46e8106a80 2024-09-11T23:31:27.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.309+0000 7f46ef0c6640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f46e8110d00 con 0x7f46e8101420 2024-09-11T23:31:27.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.309+0000 7f46ef0c6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f46e8111000 con 0x7f46e8100a80 2024-09-11T23:31:27.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.309+0000 7f46ed663640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e8106a80 0x7f46e81a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:27.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.309+0000 7f46ed663640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e8106a80 0x7f46e81a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52722/0 (socket says 172.21.15.43:52722) 2024-09-11T23:31:27.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.309+0000 7f46ed663640 1 -- 172.21.15.43:0/884930665 learned_addr learned my addr 172.21.15.43:0/884930665 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:27.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.309+0000 7f46ece62640 1 --2- 172.21.15.43:0/884930665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46e8100a80 0x7f46e81992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:27.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.309+0000 7f46e7fff640 1 --2- 172.21.15.43:0/884930665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46e8101420 0x7f46e8199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:27.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.309+0000 7f46ed663640 1 -- 172.21.15.43:0/884930665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46e8100a80 msgr2=0x7f46e81992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:27.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.309+0000 7f46ed663640 1 --2- 172.21.15.43:0/884930665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46e8100a80 0x7f46e81992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.309+0000 7f46ed663640 1 -- 172.21.15.43:0/884930665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46e8101420 msgr2=0x7f46e8199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:27.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.309+0000 7f46ed663640 1 --2- 172.21.15.43:0/884930665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46e8101420 0x7f46e8199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.309+0000 7f46ed663640 1 -- 172.21.15.43:0/884930665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e81a4890 con 0x7f46e8106a80 2024-09-11T23:31:27.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.309+0000 7f46ed663640 1 --2- 172.21.15.43:0/884930665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e8106a80 0x7f46e81a4350 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f46e000eaa0 tx=0x7f46e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:27.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.310+0000 7f46e5ffb640 1 -- 172.21.15.43:0/884930665 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e000cdb0 con 0x7f46e8106a80 2024-09-11T23:31:27.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.310+0000 7f46e5ffb640 1 -- 172.21.15.43:0/884930665 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46e0005020 con 0x7f46e8106a80 2024-09-11T23:31:27.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.310+0000 7f46ef0c6640 1 -- 172.21.15.43:0/884930665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e81a4a60 con 0x7f46e8106a80 2024-09-11T23:31:27.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.310+0000 7f46e5ffb640 1 -- 172.21.15.43:0/884930665 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e00116b0 con 0x7f46e8106a80 2024-09-11T23:31:27.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.310+0000 7f46ef0c6640 1 -- 172.21.15.43:0/884930665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e81a4f70 con 0x7f46e8106a80 2024-09-11T23:31:27.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.313+0000 7f46e5ffb640 1 -- 172.21.15.43:0/884930665 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f46e0005560 con 0x7f46e8106a80 2024-09-11T23:31:27.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.313+0000 7f46f149f740 1 Processor -- start 2024-09-11T23:31:27.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.313+0000 7f46f149f740 1 -- start start 2024-09-11T23:31:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.313+0000 7f46f149f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff6cccb60 0x560ff6cccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.313+0000 7f46f149f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6ccd480 0x560ff6cd7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.313+0000 7f46f149f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ff6cd7d40 0x560ff6cda130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.313+0000 7f46f149f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560ff6c32280 con 0x560ff6ccd480 2024-09-11T23:31:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.313+0000 7f46f149f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560ff6bd9b10 con 0x560ff6cd7d40 2024-09-11T23:31:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.313+0000 7f46f149f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560ff6c2e240 con 0x560ff6cccb60 2024-09-11T23:31:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.313+0000 7f46e5ffb640 1 --2- 172.21.15.43:0/884930665 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f46d00a2cf0 0x7f46d00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.314+0000 7f46ece62640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6ccd480 0x560ff6cd7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.314+0000 7f46ece62640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6ccd480 0x560ff6cd7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52738/0 (socket says 172.21.15.43:52738) 2024-09-11T23:31:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.314+0000 7f46ece62640 1 -- 172.21.15.43:0/2060163470 learned_addr learned my addr 172.21.15.43:0/2060163470 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.314+0000 7f46e7fff640 1 --2- 172.21.15.43:0/2060163470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ff6cd7d40 0x560ff6cda130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:27.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.314+0000 7f46ed663640 1 --2- 172.21.15.43:0/884930665 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f46d00a2cf0 0x7f46d00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:27.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.314+0000 7f46e7fff640 1 --2- 172.21.15.43:0/2060163470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff6cccb60 0x560ff6cccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:27.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.314+0000 7f46e5ffb640 1 -- 172.21.15.43:0/884930665 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f46e00cf630 con 0x7f46e8106a80 2024-09-11T23:31:27.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.314+0000 7f46ece62640 1 -- 172.21.15.43:0/2060163470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff6cccb60 msgr2=0x560ff6cccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:27.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.314+0000 7f46ece62640 1 --2- 172.21.15.43:0/2060163470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff6cccb60 0x560ff6cccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.314+0000 7f46ece62640 1 -- 172.21.15.43:0/2060163470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ff6cd7d40 msgr2=0x560ff6cda130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:27.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.314+0000 7f46ece62640 1 --2- 172.21.15.43:0/2060163470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ff6cd7d40 0x560ff6cda130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.314+0000 7f46ece62640 1 -- 172.21.15.43:0/2060163470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ff6be61f0 con 0x560ff6ccd480 2024-09-11T23:31:27.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.314+0000 7f46e7fff640 1 --2- 172.21.15.43:0/2060163470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ff6cd7d40 0x560ff6cda130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:31:27.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.314+0000 7f46ece62640 1 --2- 172.21.15.43:0/2060163470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6ccd480 0x560ff6cd7800 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009930 tx=0x7f46d8031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88c19cea9b77f64 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:27.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.314+0000 7f46ed663640 1 --2- 172.21.15.43:0/884930665 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f46d00a2cf0 0x7f46d00a51b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f46e00cce00 tx=0x7f46e00054d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.314+0000 7f46ceffd640 1 -- 172.21.15.43:0/2060163470 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8040070 con 0x560ff6ccd480 2024-09-11T23:31:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.314+0000 7f46ceffd640 1 -- 172.21.15.43:0/2060163470 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46d8005020 con 0x560ff6ccd480 2024-09-11T23:31:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.314+0000 7f46ceffd640 1 -- 172.21.15.43:0/2060163470 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8035040 con 0x560ff6ccd480 2024-09-11T23:31:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.315+0000 7f46f149f740 1 -- 172.21.15.43:0/2060163470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6ccd480 msgr2=0x560ff6cd7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.315+0000 7f46f149f740 1 --2- 172.21.15.43:0/2060163470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6ccd480 0x560ff6cd7800 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009930 tx=0x7f46d8031630 comp rx=0 tx=0).stop 2024-09-11T23:31:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.315+0000 7f46f149f740 1 -- 172.21.15.43:0/2060163470 shutdown_connections 2024-09-11T23:31:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.315+0000 7f46f149f740 1 --2- 172.21.15.43:0/2060163470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ff6cd7d40 0x560ff6cda130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.315+0000 7f46f149f740 1 --2- 172.21.15.43:0/2060163470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6ccd480 0x560ff6cd7800 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.315+0000 7f46f149f740 1 --2- 172.21.15.43:0/2060163470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff6cccb60 0x560ff6cccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.315+0000 7f46f149f740 1 -- 172.21.15.43:0/2060163470 >> 172.21.15.43:0/2060163470 conn(0x560ff6cc90f0 msgr2=0x560ff6cc94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:27.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.315+0000 7f46f149f740 1 -- 172.21.15.43:0/2060163470 shutdown_connections 2024-09-11T23:31:27.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.315+0000 7f46f149f740 1 -- 172.21.15.43:0/2060163470 wait complete. 2024-09-11T23:31:27.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.316+0000 7f46f149f740 1 -- start start 2024-09-11T23:31:27.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.316+0000 7f46f149f740 1 Processor -- start 2024-09-11T23:31:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.316+0000 7f46f149f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff6cccb60 0x560ff6cd7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.316+0000 7f46f149f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ff6ccd480 0x560ff6cd4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.316+0000 7f46f149f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6cd7d40 0x560ff6cd5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.316+0000 7f46f149f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560ff6cda910 con 0x560ff6cd7d40 2024-09-11T23:31:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.316+0000 7f46f149f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560ff6becf10 con 0x560ff6ccd480 2024-09-11T23:31:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.316+0000 7f46f149f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560ff6cdaa90 con 0x560ff6cccb60 2024-09-11T23:31:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.316+0000 7f46f149f740 10 client.0 _ll_register_callbacks cb 0x560ff6bb5a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.316+0000 7f46e7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6cd7d40 0x560ff6cd5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.316+0000 7f46e7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6cd7d40 0x560ff6cd5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52750/0 (socket says 172.21.15.43:52750) 2024-09-11T23:31:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.316+0000 7f46e7fff640 1 -- 172.21.15.43:0/4177528209 learned_addr learned my addr 172.21.15.43:0/4177528209 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.316+0000 7f46e7fff640 1 --2- 172.21.15.43:0/4177528209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff6cccb60 0x560ff6cd7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.316+0000 7f46ece62640 1 --2- 172.21.15.43:0/4177528209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ff6ccd480 0x560ff6cd4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.317+0000 7f46e7fff640 1 -- 172.21.15.43:0/4177528209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff6cccb60 msgr2=0x560ff6cd7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.317+0000 7f46e7fff640 1 --2- 172.21.15.43:0/4177528209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff6cccb60 0x560ff6cd7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.317+0000 7f46e7fff640 1 -- 172.21.15.43:0/4177528209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ff6ccd480 msgr2=0x560ff6cd4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.317+0000 7f46e7fff640 1 --2- 172.21.15.43:0/4177528209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ff6ccd480 0x560ff6cd4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.317+0000 7f46e7fff640 1 -- 172.21.15.43:0/4177528209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ff69a36a0 con 0x560ff6cd7d40 2024-09-11T23:31:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.317+0000 7f46e7fff640 1 --2- 172.21.15.43:0/4177528209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6cd7d40 0x560ff6cd5250 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f46dc013f30 tx=0x7f46dc005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e8b5c4944b74d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:27.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.317+0000 7f46c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:31:27.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.317+0000 7f46c7fff640 1 -- 172.21.15.43:0/4177528209 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc004020 con 0x560ff6cd7d40 2024-09-11T23:31:27.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.317+0000 7f46c7fff640 1 -- 172.21.15.43:0/4177528209 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f46dc0041c0 con 0x560ff6cd7d40 2024-09-11T23:31:27.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.317+0000 7f46f149f740 1 -- 172.21.15.43:0/4177528209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46d8005020 con 0x560ff6cd7d40 2024-09-11T23:31:27.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.317+0000 7f46c7fff640 1 -- 172.21.15.43:0/4177528209 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f46dc027050 con 0x560ff6cd7d40 2024-09-11T23:31:27.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.317+0000 7f46f149f740 10 client.16962 fetch_fsmap learned FSMap version 32 2024-09-11T23:31:27.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.317+0000 7f46f149f740 1 -- 172.21.15.43:0/4177528209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560ff6ce6440 con 0x560ff6cd7d40 2024-09-11T23:31:27.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.317+0000 7f46c7fff640 1 -- 172.21.15.43:0/4177528209 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46d8005020 con 0x560ff6cd7d40 2024-09-11T23:31:27.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.318+0000 7f46c7fff640 1 -- 172.21.15.43:0/4177528209 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f46dc00f040 con 0x560ff6cd7d40 2024-09-11T23:31:27.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.318+0000 7f46f149f740 10 client.16962 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:31:27.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.318+0000 7f46f149f740 10 client.16962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:31:27.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.318+0000 7f46f149f740 20 client.16962 populate_metadata read hostname 'smithi043' 2024-09-11T23:31:27.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.318+0000 7f46f149f740 1 --2- 172.21.15.43:0/4177528209 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560ff6ce7900 0x560ff6ce9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:27.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.318+0000 7f46f149f740 4 client.16962 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:31:27.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.318+0000 7f46f149f740 1 -- 172.21.15.43:0/4177528209 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ff6c3f7f0 con 0x560ff6ce7900 2024-09-11T23:31:27.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.318+0000 7f46ece62640 1 --2- 172.21.15.43:0/4177528209 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560ff6ce7900 0x560ff6ce9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:27.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.318+0000 7f46c7fff640 10 client.16962 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:31:27.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.318+0000 7f46ece62640 1 --2- 172.21.15.43:0/4177528209 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560ff6ce7900 0x560ff6ce9d10 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdd558bc8c183424 server_cookie=b6b936d4189b1de6 in_seq=0 out_seq=0 2024-09-11T23:31:27.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.319+0000 7f46c7fff640 1 -- 172.21.15.43:0/4177528209 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ff6c3f7f0 con 0x560ff6ce7900 2024-09-11T23:31:27.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.319+0000 7f46c7fff640 10 client.16962 handle_command_reply: tid=0 2024-09-11T23:31:27.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.319+0000 7f46f149f740 1 client.16962 shutdown 2024-09-11T23:31:27.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.319+0000 7f46f149f740 1 -- 172.21.15.43:0/4177528209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6cd7d40 msgr2=0x560ff6cd5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:27.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.319+0000 7f46f149f740 1 --2- 172.21.15.43:0/4177528209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6cd7d40 0x560ff6cd5250 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f46dc013f30 tx=0x7f46dc005d20 comp rx=0 tx=0).stop 2024-09-11T23:31:27.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.319+0000 7f46f149f740 1 -- 172.21.15.43:0/4177528209 shutdown_connections 2024-09-11T23:31:27.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.319+0000 7f46f149f740 1 --2- 172.21.15.43:0/4177528209 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560ff6ce7900 0x560ff6ce9d10 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.319+0000 7f46c7fff640 0 client.16962 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:31:27.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.319+0000 7f46f149f740 1 --2- 172.21.15.43:0/4177528209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6cd7d40 0x560ff6cd5250 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.319+0000 7f46f149f740 1 --2- 172.21.15.43:0/4177528209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ff6ccd480 0x560ff6cd4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.319+0000 7f46f149f740 1 --2- 172.21.15.43:0/4177528209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff6cccb60 0x560ff6cd7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.319+0000 7f46f149f740 1 -- 172.21.15.43:0/4177528209 >> 172.21.15.43:0/4177528209 conn(0x560ff6cc90f0 msgr2=0x560ff6cc8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:27.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.319+0000 7f46f149f740 1 -- 172.21.15.43:0/4177528209 shutdown_connections 2024-09-11T23:31:27.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.319+0000 7f46f149f740 1 -- 172.21.15.43:0/4177528209 wait complete. 2024-09-11T23:31:27.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.320+0000 7f46f149f740 20 client.16962 trim_cache size 0 max 16384 2024-09-11T23:31:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.337+0000 7f46f149f740 1 Processor -- start 2024-09-11T23:31:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.337+0000 7f46f149f740 1 -- start start 2024-09-11T23:31:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.337+0000 7f46f149f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff6cccb60 0x560ff6cd6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.337+0000 7f46f149f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ff6ccd480 0x560ff6cef5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.337+0000 7f46f149f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6cd7d40 0x560ff6cefb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.337+0000 7f46f149f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f46e81b02e0 con 0x560ff6cd7d40 2024-09-11T23:31:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.337+0000 7f46f149f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f46e81adb80 con 0x560ff6ccd480 2024-09-11T23:31:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.337+0000 7f46f149f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f46e81ab9f0 con 0x560ff6cccb60 2024-09-11T23:31:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.337+0000 7f46e7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6cd7d40 0x560ff6cefb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.337+0000 7f46e7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6cd7d40 0x560ff6cefb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52764/0 (socket says 172.21.15.43:52764) 2024-09-11T23:31:27.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.337+0000 7f46e7fff640 1 -- 172.21.15.43:0/61475075 learned_addr learned my addr 172.21.15.43:0/61475075 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:27.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.337+0000 7f46e7fff640 1 --2- 172.21.15.43:0/61475075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff6cccb60 0x560ff6cd6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:27.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.337+0000 7f46ece62640 1 --2- 172.21.15.43:0/61475075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ff6ccd480 0x560ff6cef5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:27.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.337+0000 7f46e7fff640 1 -- 172.21.15.43:0/61475075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff6cccb60 msgr2=0x560ff6cd6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:27.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.337+0000 7f46e7fff640 1 --2- 172.21.15.43:0/61475075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff6cccb60 0x560ff6cd6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.337+0000 7f46e7fff640 1 -- 172.21.15.43:0/61475075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ff6ccd480 msgr2=0x560ff6cef5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:27.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.337+0000 7f46e7fff640 1 --2- 172.21.15.43:0/61475075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ff6ccd480 0x560ff6cef5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.337+0000 7f46e7fff640 1 -- 172.21.15.43:0/61475075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ff6cc8f20 con 0x560ff6cd7d40 2024-09-11T23:31:27.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.337+0000 7f46e7fff640 1 --2- 172.21.15.43:0/61475075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6cd7d40 0x560ff6cefb20 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f46dc013ab0 tx=0x7f46dc005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e22b11cd6bb8ea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:27.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.338+0000 7f46ce7fc640 1 -- 172.21.15.43:0/61475075 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc00ec40 con 0x560ff6cd7d40 2024-09-11T23:31:27.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.338+0000 7f46ce7fc640 1 -- 172.21.15.43:0/61475075 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46dc010710 con 0x560ff6cd7d40 2024-09-11T23:31:27.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.338+0000 7f46ce7fc640 1 -- 172.21.15.43:0/61475075 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc004020 con 0x560ff6cd7d40 2024-09-11T23:31:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.338+0000 7f46f149f740 1 -- 172.21.15.43:0/61475075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6cd7d40 msgr2=0x560ff6cefb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.338+0000 7f46f149f740 1 --2- 172.21.15.43:0/61475075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6cd7d40 0x560ff6cefb20 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f46dc013ab0 tx=0x7f46dc005f70 comp rx=0 tx=0).stop 2024-09-11T23:31:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.338+0000 7f46f149f740 1 -- 172.21.15.43:0/61475075 shutdown_connections 2024-09-11T23:31:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.338+0000 7f46f149f740 1 --2- 172.21.15.43:0/61475075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6cd7d40 0x560ff6cefb20 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.338+0000 7f46f149f740 1 --2- 172.21.15.43:0/61475075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ff6ccd480 0x560ff6cef5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.338+0000 7f46f149f740 1 --2- 172.21.15.43:0/61475075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff6cccb60 0x560ff6cd6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.338+0000 7f46f149f740 1 -- 172.21.15.43:0/61475075 >> 172.21.15.43:0/61475075 conn(0x560ff6cc90f0 msgr2=0x560ff6cd6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.338+0000 7f46f149f740 1 -- 172.21.15.43:0/61475075 shutdown_connections 2024-09-11T23:31:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.338+0000 7f46f149f740 1 -- 172.21.15.43:0/61475075 wait complete. 2024-09-11T23:31:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.339+0000 7f46f149f740 1 -- start start 2024-09-11T23:31:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.339+0000 7f46f149f740 1 Processor -- start 2024-09-11T23:31:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.339+0000 7f46f149f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ff6cccb60 0x560ff6cdfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.339+0000 7f46f149f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff6ccd480 0x560ff6ce0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.339+0000 7f46f149f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6cd7d40 0x560ff6cd0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.339+0000 7f46f149f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f46e81ac040 con 0x560ff6cd7d40 2024-09-11T23:31:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.339+0000 7f46f149f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f46e81ad4b0 con 0x560ff6cccb60 2024-09-11T23:31:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.339+0000 7f46f149f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f46e81ac710 con 0x560ff6ccd480 2024-09-11T23:31:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.339+0000 7f46f149f740 10 client.0 _ll_register_callbacks cb 0x7f46dc013b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.339+0000 7f46e7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6cd7d40 0x560ff6cd0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.339+0000 7f46e7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6cd7d40 0x560ff6cd0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52766/0 (socket says 172.21.15.43:52766) 2024-09-11T23:31:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.339+0000 7f46e7fff640 1 -- 172.21.15.43:0/2355217288 learned_addr learned my addr 172.21.15.43:0/2355217288 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.339+0000 7f46e7fff640 1 --2- 172.21.15.43:0/2355217288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ff6cccb60 0x560ff6cdfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.339+0000 7f46ece62640 1 --2- 172.21.15.43:0/2355217288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff6ccd480 0x560ff6ce0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.340+0000 7f46e7fff640 1 -- 172.21.15.43:0/2355217288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff6ccd480 msgr2=0x560ff6ce0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.340+0000 7f46e7fff640 1 --2- 172.21.15.43:0/2355217288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff6ccd480 0x560ff6ce0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.340+0000 7f46e7fff640 1 -- 172.21.15.43:0/2355217288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ff6cccb60 msgr2=0x560ff6cdfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.340+0000 7f46e7fff640 1 --2- 172.21.15.43:0/2355217288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ff6cccb60 0x560ff6cdfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.340+0000 7f46e7fff640 1 -- 172.21.15.43:0/2355217288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ff6cd1560 con 0x560ff6cd7d40 2024-09-11T23:31:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.340+0000 7f46e7fff640 1 --2- 172.21.15.43:0/2355217288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6cd7d40 0x560ff6cd0d30 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f46dc013c80 tx=0x7f46dc005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1ec7d15726b700 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.340+0000 7f46c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:31:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.340+0000 7f46c7fff640 1 -- 172.21.15.43:0/2355217288 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc004450 con 0x560ff6cd7d40 2024-09-11T23:31:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.340+0000 7f46c7fff640 1 -- 172.21.15.43:0/2355217288 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f46dc010a10 con 0x560ff6cd7d40 2024-09-11T23:31:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.340+0000 7f46f149f740 1 -- 172.21.15.43:0/2355217288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46dc010710 con 0x560ff6cd7d40 2024-09-11T23:31:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.340+0000 7f46c7fff640 1 -- 172.21.15.43:0/2355217288 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f46dc005c20 con 0x560ff6cd7d40 2024-09-11T23:31:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.340+0000 7f46f149f740 10 client.16968 fetch_fsmap learned FSMap version 32 2024-09-11T23:31:27.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.340+0000 7f46f149f740 1 -- 172.21.15.43:0/2355217288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560ff6cd2840 con 0x560ff6cd7d40 2024-09-11T23:31:27.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.340+0000 7f46c7fff640 1 -- 172.21.15.43:0/2355217288 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46dc010710 con 0x560ff6cd7d40 2024-09-11T23:31:27.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.340+0000 7f46c7fff640 1 -- 172.21.15.43:0/2355217288 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f46dc02a070 con 0x560ff6cd7d40 2024-09-11T23:31:27.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.340+0000 7f46f149f740 10 client.16968 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T23:31:27.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.340+0000 7f46f149f740 10 client.16968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:31:27.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.340+0000 7f46f149f740 20 client.16968 populate_metadata read hostname 'smithi043' 2024-09-11T23:31:27.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.340+0000 7f46f149f740 1 --2- 172.21.15.43:0/2355217288 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560ff6d53770 0x560ff6d55b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:27.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.340+0000 7f46f149f740 4 client.16968 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:31:27.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.340+0000 7f46f149f740 1 -- 172.21.15.43:0/2355217288 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560ff6c39d00 con 0x560ff6d53770 2024-09-11T23:31:27.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.341+0000 7f46ece62640 1 --2- 172.21.15.43:0/2355217288 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560ff6d53770 0x560ff6d55b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:27.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.341+0000 7f46ece62640 1 --2- 172.21.15.43:0/2355217288 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560ff6d53770 0x560ff6d55b90 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2450ab3ac6f116f5 server_cookie=d251b9bf9db01814 in_seq=0 out_seq=0 2024-09-11T23:31:27.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.341+0000 7f46c7fff640 10 client.16968 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:31:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.342+0000 7f46c7fff640 1 -- 172.21.15.43:0/2355217288 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560ff6c39d00 con 0x560ff6d53770 2024-09-11T23:31:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.342+0000 7f46c7fff640 10 client.16968 handle_command_reply: tid=0 2024-09-11T23:31:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.342+0000 7f46f149f740 1 client.16968 shutdown 2024-09-11T23:31:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.342+0000 7f46f149f740 1 -- 172.21.15.43:0/2355217288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6cd7d40 msgr2=0x560ff6cd0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.342+0000 7f46f149f740 1 --2- 172.21.15.43:0/2355217288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6cd7d40 0x560ff6cd0d30 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f46dc013c80 tx=0x7f46dc005a10 comp rx=0 tx=0).stop 2024-09-11T23:31:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.343+0000 7f46f149f740 1 -- 172.21.15.43:0/2355217288 shutdown_connections 2024-09-11T23:31:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.343+0000 7f46f149f740 1 --2- 172.21.15.43:0/2355217288 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x560ff6d53770 0x560ff6d55b90 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.343+0000 7f46f149f740 1 --2- 172.21.15.43:0/2355217288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ff6cd7d40 0x560ff6cd0d30 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.343+0000 7f46f149f740 1 --2- 172.21.15.43:0/2355217288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ff6ccd480 0x560ff6ce0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.343+0000 7f46f149f740 1 --2- 172.21.15.43:0/2355217288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ff6cccb60 0x560ff6cdfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.343+0000 7f46f149f740 1 -- 172.21.15.43:0/2355217288 >> 172.21.15.43:0/2355217288 conn(0x560ff6cc90f0 msgr2=0x560ff6ccedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.343+0000 7f46c7fff640 0 client.16968 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:31:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.343+0000 7f46f149f740 1 -- 172.21.15.43:0/2355217288 shutdown_connections 2024-09-11T23:31:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.343+0000 7f46f149f740 1 -- 172.21.15.43:0/2355217288 wait complete. 2024-09-11T23:31:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.343+0000 7f46f149f740 20 client.16968 trim_cache size 0 max 16384 2024-09-11T23:31:27.349 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:31:27.349 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-09-11T23:31:27.350 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "cbba29dc-7900-47db-86ba-e8bc90e2bd13", 2024-09-11T23:31:27.350 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-09-11T23:31:27.350 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:31:27.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.343+0000 7f46ef0c6640 1 -- 172.21.15.43:0/884930665 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f46d00a2cf0 msgr2=0x7f46d00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:27.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.343+0000 7f46ef0c6640 1 --2- 172.21.15.43:0/884930665 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f46d00a2cf0 0x7f46d00a51b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f46e00cce00 tx=0x7f46e00054d0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.343+0000 7f46ef0c6640 1 -- 172.21.15.43:0/884930665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e8106a80 msgr2=0x7f46e81a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:27.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.343+0000 7f46ef0c6640 1 --2- 172.21.15.43:0/884930665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e8106a80 0x7f46e81a4350 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f46e000eaa0 tx=0x7f46e000ef60 comp rx=0 tx=0).stop 2024-09-11T23:31:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.343+0000 7f46ef0c6640 1 -- 172.21.15.43:0/884930665 shutdown_connections 2024-09-11T23:31:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.343+0000 7f46ef0c6640 1 --2- 172.21.15.43:0/884930665 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f46d00a2cf0 0x7f46d00a51b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.343+0000 7f46ef0c6640 1 --2- 172.21.15.43:0/884930665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e8106a80 0x7f46e81a4350 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.343+0000 7f46ef0c6640 1 --2- 172.21.15.43:0/884930665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46e8101420 0x7f46e8199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.343+0000 7f46ef0c6640 1 --2- 172.21.15.43:0/884930665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46e8100a80 0x7f46e81992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.343+0000 7f46ef0c6640 1 -- 172.21.15.43:0/884930665 >> 172.21.15.43:0/884930665 conn(0x7f46e80fc7a0 msgr2=0x7f46e80fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.344+0000 7f46ef0c6640 1 -- 172.21.15.43:0/884930665 shutdown_connections 2024-09-11T23:31:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:27.344+0000 7f46ef0c6640 1 -- 172.21.15.43:0/884930665 wait complete. 2024-09-11T23:31:28.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:28 smithi043 ceph-mon[30950]: pgmap v298: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-11T23:31:28.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:28 smithi043 ceph-mon[30950]: scrub queued for path: / 2024-09-11T23:31:28.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:28 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:31:28.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:28 smithi043 ceph-mon[30950]: scrub summary: active paths [/] 2024-09-11T23:31:28.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:28 smithi043 ceph-mon[30950]: scrub summary: idle+waiting paths [/] 2024-09-11T23:31:28.092 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:31:28.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:28 smithi090 ceph-mon[35130]: pgmap v298: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-11T23:31:28.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:28 smithi090 ceph-mon[35130]: scrub queued for path: / 2024-09-11T23:31:28.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:28 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:31:28.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:28 smithi090 ceph-mon[35130]: scrub summary: active paths [/] 2024-09-11T23:31:28.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:28 smithi090 ceph-mon[35130]: scrub summary: idle+waiting paths [/] 2024-09-11T23:31:28.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:28 smithi097 ceph-mon[35562]: pgmap v298: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-11T23:31:28.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:28 smithi097 ceph-mon[35562]: scrub queued for path: / 2024-09-11T23:31:28.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:28 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:31:28.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:28 smithi097 ceph-mon[35562]: scrub summary: active paths [/] 2024-09-11T23:31:28.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:28 smithi097 ceph-mon[35562]: scrub summary: idle+waiting paths [/] 2024-09-11T23:31:28.350 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:31:28.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.859+0000 7fbafc5f0640 1 -- 172.21.15.43:0/391657376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbaf4102d30 msgr2=0x7fbaf4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.860+0000 7fbafc5f0640 1 --2- 172.21.15.43:0/391657376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbaf4102d30 0x7fbaf4103110 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0009930 tx=0x7fbaf002a420 comp rx=0 tx=0).stop 2024-09-11T23:31:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.860+0000 7fbafc5f0640 1 -- 172.21.15.43:0/391657376 shutdown_connections 2024-09-11T23:31:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.860+0000 7fbafc5f0640 1 --2- 172.21.15.43:0/391657376 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbaf41082a0 0x7fbaf410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.860+0000 7fbafc5f0640 1 --2- 172.21.15.43:0/391657376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf41036e0 0x7fbaf4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.860+0000 7fbafc5f0640 1 --2- 172.21.15.43:0/391657376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbaf4102d30 0x7fbaf4103110 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.860+0000 7fbafc5f0640 1 -- 172.21.15.43:0/391657376 >> 172.21.15.43:0/391657376 conn(0x7fbaf40fc7d0 msgr2=0x7fbaf40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.860+0000 7fbafc5f0640 1 -- 172.21.15.43:0/391657376 shutdown_connections 2024-09-11T23:31:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.860+0000 7fbafc5f0640 1 -- 172.21.15.43:0/391657376 wait complete. 2024-09-11T23:31:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.861+0000 7fbafc5f0640 1 Processor -- start 2024-09-11T23:31:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.861+0000 7fbafc5f0640 1 -- start start 2024-09-11T23:31:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.861+0000 7fbafc5f0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf4102d30 0x7fbaf4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.861+0000 7fbafc5f0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbaf41036e0 0x7fbaf4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.861+0000 7fbafc5f0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbaf41082a0 0x7fbaf4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.861+0000 7fbafc5f0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf41120e0 con 0x7fbaf41036e0 2024-09-11T23:31:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.861+0000 7fbafc5f0640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf4111f60 con 0x7fbaf41082a0 2024-09-11T23:31:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.861+0000 7fbafc5f0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf4112260 con 0x7fbaf4102d30 2024-09-11T23:31:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.861+0000 7fbafa38c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf4102d30 0x7fbaf4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.861+0000 7fbafa38c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf4102d30 0x7fbaf4100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:44814/0 (socket says 172.21.15.43:44814) 2024-09-11T23:31:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.861+0000 7fbaf9b8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbaf41036e0 0x7fbaf4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.861+0000 7fbafa38c640 1 -- 172.21.15.43:0/2682012394 learned_addr learned my addr 172.21.15.43:0/2682012394 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.862+0000 7fbafab8d640 1 --2- 172.21.15.43:0/2682012394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbaf41082a0 0x7fbaf4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.862+0000 7fbafa38c640 1 -- 172.21.15.43:0/2682012394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbaf41082a0 msgr2=0x7fbaf4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.862+0000 7fbafa38c640 1 --2- 172.21.15.43:0/2682012394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbaf41082a0 0x7fbaf4101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.862+0000 7fbafa38c640 1 -- 172.21.15.43:0/2682012394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbaf41036e0 msgr2=0x7fbaf4100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.862+0000 7fbafa38c640 1 --2- 172.21.15.43:0/2682012394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbaf41036e0 0x7fbaf4100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.862+0000 7fbafa38c640 1 -- 172.21.15.43:0/2682012394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf41a6a60 con 0x7fbaf4102d30 2024-09-11T23:31:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.862+0000 7fbafa38c640 1 --2- 172.21.15.43:0/2682012394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf4102d30 0x7fbaf4100990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbaf002a940 tx=0x7fbaf0003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.862+0000 7fbaeb7fe640 1 -- 172.21.15.43:0/2682012394 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf003f070 con 0x7fbaf4102d30 2024-09-11T23:31:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.862+0000 7fbafc5f0640 1 -- 172.21.15.43:0/2682012394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf41a6cf0 con 0x7fbaf4102d30 2024-09-11T23:31:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.862+0000 7fbaeb7fe640 1 -- 172.21.15.43:0/2682012394 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaf0031c90 con 0x7fbaf4102d30 2024-09-11T23:31:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.862+0000 7fbaeb7fe640 1 -- 172.21.15.43:0/2682012394 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0035030 con 0x7fbaf4102d30 2024-09-11T23:31:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.862+0000 7fbafc5f0640 1 -- 172.21.15.43:0/2682012394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf41a7180 con 0x7fbaf4102d30 2024-09-11T23:31:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.865+0000 7fbaeb7fe640 1 -- 172.21.15.43:0/2682012394 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbaf0042070 con 0x7fbaf4102d30 2024-09-11T23:31:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.865+0000 7fbafe9c9740 1 Processor -- start 2024-09-11T23:31:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.865+0000 7fbafe9c9740 1 -- start start 2024-09-11T23:31:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.865+0000 7fbafe9c9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220de490 0x5643220de870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.865+0000 7fbafe9c9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220dedb0 0x5643220e9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.865+0000 7fbafe9c9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643220e9670 0x5643220eba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.865+0000 7fbafe9c9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5643220442e0 con 0x5643220dedb0 2024-09-11T23:31:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.865+0000 7fbafe9c9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564321febce0 con 0x5643220de490 2024-09-11T23:31:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.865+0000 7fbafe9c9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5643220402a0 con 0x5643220e9670 2024-09-11T23:31:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.865+0000 7fbafab8d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220de490 0x5643220de870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.865+0000 7fbafab8d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220de490 0x5643220de870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.43:33750/0 (socket says 172.21.15.43:33750) 2024-09-11T23:31:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.865+0000 7fbafab8d640 1 -- 172.21.15.43:0/4208217733 learned_addr learned my addr 172.21.15.43:0/4208217733 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.865+0000 7fbaf9b8b640 1 --2- 172.21.15.43:0/4208217733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643220e9670 0x5643220eba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.865+0000 7fbafa38c640 1 --2- 172.21.15.43:0/4208217733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220dedb0 0x5643220e9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.866+0000 7fbafab8d640 1 -- 172.21.15.43:0/4208217733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643220e9670 msgr2=0x5643220eba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.866+0000 7fbafab8d640 1 --2- 172.21.15.43:0/4208217733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643220e9670 0x5643220eba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.866+0000 7fbafab8d640 1 -- 172.21.15.43:0/4208217733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220dedb0 msgr2=0x5643220e9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.866+0000 7fbaeb7fe640 1 --2- 172.21.15.43:0/2682012394 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fbae00a2cf0 0x7fbae00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.866+0000 7fbafab8d640 1 --2- 172.21.15.43:0/4208217733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220dedb0 0x5643220e9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.866+0000 7fbafab8d640 1 -- 172.21.15.43:0/4208217733 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564321fd0440 con 0x5643220de490 2024-09-11T23:31:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.866+0000 7fbaeb7fe640 1 -- 172.21.15.43:0/2682012394 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fbaf00efb90 con 0x7fbaf4102d30 2024-09-11T23:31:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.866+0000 7fbafab8d640 1 --2- 172.21.15.43:0/2682012394 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fbae00a2cf0 0x7fbae00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.866+0000 7fbafab8d640 1 --2- 172.21.15.43:0/4208217733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220de490 0x5643220de870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbaec009f40 tx=0x7fbaec010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80826d6e63b53b28 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:29.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.866+0000 7fbae8ff9640 1 -- 172.21.15.43:0/4208217733 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec011500 con 0x5643220de490 2024-09-11T23:31:29.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.866+0000 7fbae8ff9640 1 -- 172.21.15.43:0/4208217733 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaec011b80 con 0x5643220de490 2024-09-11T23:31:29.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.866+0000 7fbae8ff9640 1 -- 172.21.15.43:0/4208217733 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec01ea60 con 0x5643220de490 2024-09-11T23:31:29.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.866+0000 7fbafab8d640 1 --2- 172.21.15.43:0/2682012394 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fbae00a2cf0 0x7fbae00a51b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbaec01f000 tx=0x7fbaec020000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:29.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.867+0000 7fbafe9c9740 1 -- 172.21.15.43:0/4208217733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220de490 msgr2=0x5643220de870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:29.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.867+0000 7fbafe9c9740 1 --2- 172.21.15.43:0/4208217733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220de490 0x5643220de870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbaec009f40 tx=0x7fbaec010860 comp rx=0 tx=0).stop 2024-09-11T23:31:29.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.867+0000 7fbafe9c9740 1 -- 172.21.15.43:0/4208217733 shutdown_connections 2024-09-11T23:31:29.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.867+0000 7fbafe9c9740 1 --2- 172.21.15.43:0/4208217733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643220e9670 0x5643220eba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.867+0000 7fbafe9c9740 1 --2- 172.21.15.43:0/4208217733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220dedb0 0x5643220e9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.867+0000 7fbafe9c9740 1 --2- 172.21.15.43:0/4208217733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220de490 0x5643220de870 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.867+0000 7fbafe9c9740 1 -- 172.21.15.43:0/4208217733 >> 172.21.15.43:0/4208217733 conn(0x5643220daa20 msgr2=0x5643220dae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:29.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.867+0000 7fbafe9c9740 1 -- 172.21.15.43:0/4208217733 shutdown_connections 2024-09-11T23:31:29.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.867+0000 7fbafe9c9740 1 -- 172.21.15.43:0/4208217733 wait complete. 2024-09-11T23:31:29.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.867+0000 7fbafe9c9740 1 -- start start 2024-09-11T23:31:29.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.868+0000 7fbafe9c9740 1 Processor -- start 2024-09-11T23:31:29.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.868+0000 7fbafe9c9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220de490 0x5643220e8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:29.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.868+0000 7fbafe9c9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220dedb0 0x5643220e6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:29.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.868+0000 7fbafe9c9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643220e9670 0x5643220e67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:29.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.868+0000 7fbafe9c9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5643220ec240 con 0x5643220de490 2024-09-11T23:31:29.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.868+0000 7fbafe9c9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564321fff050 con 0x5643220dedb0 2024-09-11T23:31:29.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.868+0000 7fbafe9c9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5643220ec3c0 con 0x5643220e9670 2024-09-11T23:31:29.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.868+0000 7fbafe9c9740 10 client.0 _ll_register_callbacks cb 0x56432203f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:29.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.868+0000 7fbafa38c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220de490 0x5643220e8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:29.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.868+0000 7fbafa38c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220de490 0x5643220e8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52814/0 (socket says 172.21.15.43:52814) 2024-09-11T23:31:29.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.868+0000 7fbafa38c640 1 -- 172.21.15.43:0/2073341213 learned_addr learned my addr 172.21.15.43:0/2073341213 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:29.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.868+0000 7fbaf9b8b640 1 --2- 172.21.15.43:0/2073341213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220dedb0 0x5643220e6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:29.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.868+0000 7fbafab8d640 1 --2- 172.21.15.43:0/2073341213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643220e9670 0x5643220e67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:29.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.868+0000 7fbafa38c640 1 -- 172.21.15.43:0/2073341213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643220e9670 msgr2=0x5643220e67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:29.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.868+0000 7fbafa38c640 1 --2- 172.21.15.43:0/2073341213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643220e9670 0x5643220e67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.868+0000 7fbafa38c640 1 -- 172.21.15.43:0/2073341213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220dedb0 msgr2=0x5643220e6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:29.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.868+0000 7fbafa38c640 1 --2- 172.21.15.43:0/2073341213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220dedb0 0x5643220e6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.868+0000 7fbafa38c640 1 -- 172.21.15.43:0/2073341213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564321db53c0 con 0x5643220de490 2024-09-11T23:31:29.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.868+0000 7fbafa38c640 1 --2- 172.21.15.43:0/2073341213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220de490 0x5643220e8fc0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0031fd0 tx=0x7fbaf002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a76070acbba700a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:29.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.868+0000 7fbaddffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:31:29.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.869+0000 7fbaddffb640 1 -- 172.21.15.43:0/2073341213 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0049030 con 0x5643220de490 2024-09-11T23:31:29.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.869+0000 7fbaddffb640 1 -- 172.21.15.43:0/2073341213 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fbaf00b8690 con 0x5643220de490 2024-09-11T23:31:29.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.869+0000 7fbafe9c9740 1 -- 172.21.15.43:0/2073341213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaec011b80 con 0x5643220de490 2024-09-11T23:31:29.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.869+0000 7fbaddffb640 1 -- 172.21.15.43:0/2073341213 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbaf00f4040 con 0x5643220de490 2024-09-11T23:31:29.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.869+0000 7fbafe9c9740 10 client.16980 fetch_fsmap learned FSMap version 33 2024-09-11T23:31:29.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.869+0000 7fbafe9c9740 1 -- 172.21.15.43:0/2073341213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5643220f7c10 con 0x5643220de490 2024-09-11T23:31:29.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.869+0000 7fbaddffb640 1 -- 172.21.15.43:0/2073341213 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaec011b80 con 0x5643220de490 2024-09-11T23:31:29.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.869+0000 7fbaddffb640 1 -- 172.21.15.43:0/2073341213 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7fbaf00ed4d0 con 0x5643220de490 2024-09-11T23:31:29.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.869+0000 7fbafe9c9740 10 client.16980 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:31:29.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.869+0000 7fbafe9c9740 10 client.16980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:31:29.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.869+0000 7fbafe9c9740 20 client.16980 populate_metadata read hostname 'smithi043' 2024-09-11T23:31:29.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.869+0000 7fbafe9c9740 1 --2- 172.21.15.43:0/2073341213 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5643220f8f60 0x5643220fb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:29.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.869+0000 7fbafe9c9740 4 client.16980 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:31:29.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.869+0000 7fbafe9c9740 1 -- 172.21.15.43:0/2073341213 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564322051850 con 0x5643220f8f60 2024-09-11T23:31:29.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.869+0000 7fbaf9b8b640 1 --2- 172.21.15.43:0/2073341213 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5643220f8f60 0x5643220fb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:29.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.870+0000 7fbaf9b8b640 1 --2- 172.21.15.43:0/2073341213 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5643220f8f60 0x5643220fb3b0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b57fe2d9e80fe861 server_cookie=ca7915b87d211102 in_seq=0 out_seq=0 2024-09-11T23:31:29.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.870+0000 7fbaddffb640 10 client.16980 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:31:29.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.870+0000 7fbaddffb640 1 -- 172.21.15.43:0/2073341213 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564322051850 con 0x5643220f8f60 2024-09-11T23:31:29.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.870+0000 7fbaddffb640 10 client.16980 handle_command_reply: tid=0 2024-09-11T23:31:29.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.871+0000 7fbafe9c9740 1 client.16980 shutdown 2024-09-11T23:31:29.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.871+0000 7fbafe9c9740 1 -- 172.21.15.43:0/2073341213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220de490 msgr2=0x5643220e8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:29.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.871+0000 7fbafe9c9740 1 --2- 172.21.15.43:0/2073341213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220de490 0x5643220e8fc0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0031fd0 tx=0x7fbaf002a420 comp rx=0 tx=0).stop 2024-09-11T23:31:29.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.871+0000 7fbafe9c9740 1 -- 172.21.15.43:0/2073341213 shutdown_connections 2024-09-11T23:31:29.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.871+0000 7fbafe9c9740 1 --2- 172.21.15.43:0/2073341213 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5643220f8f60 0x5643220fb3b0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.871+0000 7fbafe9c9740 1 --2- 172.21.15.43:0/2073341213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643220e9670 0x5643220e67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.871+0000 7fbaddffb640 0 client.16980 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:31:29.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.871+0000 7fbafe9c9740 1 --2- 172.21.15.43:0/2073341213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220dedb0 0x5643220e6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.871+0000 7fbafe9c9740 1 --2- 172.21.15.43:0/2073341213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220de490 0x5643220e8fc0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.871+0000 7fbafe9c9740 1 -- 172.21.15.43:0/2073341213 >> 172.21.15.43:0/2073341213 conn(0x5643220daa20 msgr2=0x5643220d9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:29.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.871+0000 7fbafe9c9740 1 -- 172.21.15.43:0/2073341213 shutdown_connections 2024-09-11T23:31:29.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.871+0000 7fbafe9c9740 1 -- 172.21.15.43:0/2073341213 wait complete. 2024-09-11T23:31:29.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.871+0000 7fbafe9c9740 20 client.16980 trim_cache size 0 max 16384 2024-09-11T23:31:29.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.889+0000 7fbafe9c9740 1 Processor -- start 2024-09-11T23:31:29.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.889+0000 7fbafe9c9740 1 -- start start 2024-09-11T23:31:29.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.889+0000 7fbafe9c9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643220de490 0x564322100c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:29.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.889+0000 7fbafe9c9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220dedb0 0x5643221011c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:29.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.889+0000 7fbafe9c9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220e9670 0x5643220e34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:29.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.889+0000 7fbafe9c9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf41b2490 con 0x5643220e9670 2024-09-11T23:31:29.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.889+0000 7fbafe9c9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf41afd60 con 0x5643220dedb0 2024-09-11T23:31:29.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.889+0000 7fbafe9c9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf41adbd0 con 0x5643220de490 2024-09-11T23:31:29.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.889+0000 7fbafab8d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220e9670 0x5643220e34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:29.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.890+0000 7fbafab8d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220e9670 0x5643220e34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52830/0 (socket says 172.21.15.43:52830) 2024-09-11T23:31:29.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.890+0000 7fbafab8d640 1 -- 172.21.15.43:0/864075058 learned_addr learned my addr 172.21.15.43:0/864075058 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:29.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.890+0000 7fbafa38c640 1 --2- 172.21.15.43:0/864075058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643220de490 0x564322100c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:29.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.890+0000 7fbaf9b8b640 1 --2- 172.21.15.43:0/864075058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220dedb0 0x5643221011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:29.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.890+0000 7fbafab8d640 1 -- 172.21.15.43:0/864075058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643220de490 msgr2=0x564322100c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:29.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.890+0000 7fbafab8d640 1 --2- 172.21.15.43:0/864075058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643220de490 0x564322100c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.890+0000 7fbafab8d640 1 -- 172.21.15.43:0/864075058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220dedb0 msgr2=0x5643221011c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:29.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.890+0000 7fbafab8d640 1 --2- 172.21.15.43:0/864075058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220dedb0 0x5643221011c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.890+0000 7fbafab8d640 1 -- 172.21.15.43:0/864075058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643220e9330 con 0x5643220e9670 2024-09-11T23:31:29.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.890+0000 7fbafab8d640 1 --2- 172.21.15.43:0/864075058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220e9670 0x5643220e34d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fbaec010d60 tx=0x7fbaec028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de4915c7a76b754 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:29.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.890+0000 7fbadffff640 1 -- 172.21.15.43:0/864075058 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec039070 con 0x5643220e9670 2024-09-11T23:31:29.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.890+0000 7fbadffff640 1 -- 172.21.15.43:0/864075058 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaec003d70 con 0x5643220e9670 2024-09-11T23:31:29.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.890+0000 7fbadffff640 1 -- 172.21.15.43:0/864075058 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec011760 con 0x5643220e9670 2024-09-11T23:31:29.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.891+0000 7fbafe9c9740 1 -- 172.21.15.43:0/864075058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220e9670 msgr2=0x5643220e34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:29.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.891+0000 7fbafe9c9740 1 --2- 172.21.15.43:0/864075058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220e9670 0x5643220e34d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fbaec010d60 tx=0x7fbaec028000 comp rx=0 tx=0).stop 2024-09-11T23:31:29.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.891+0000 7fbafe9c9740 1 -- 172.21.15.43:0/864075058 shutdown_connections 2024-09-11T23:31:29.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.891+0000 7fbafe9c9740 1 --2- 172.21.15.43:0/864075058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220e9670 0x5643220e34d0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.891+0000 7fbafe9c9740 1 --2- 172.21.15.43:0/864075058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220dedb0 0x5643221011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.891+0000 7fbafe9c9740 1 --2- 172.21.15.43:0/864075058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643220de490 0x564322100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.891+0000 7fbafe9c9740 1 -- 172.21.15.43:0/864075058 >> 172.21.15.43:0/864075058 conn(0x5643220daa20 msgr2=0x5643220e82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:29.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.891+0000 7fbafe9c9740 1 -- 172.21.15.43:0/864075058 shutdown_connections 2024-09-11T23:31:29.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.891+0000 7fbafe9c9740 1 -- 172.21.15.43:0/864075058 wait complete. 2024-09-11T23:31:29.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.891+0000 7fbafe9c9740 1 -- start start 2024-09-11T23:31:29.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.891+0000 7fbafe9c9740 1 Processor -- start 2024-09-11T23:31:29.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.892+0000 7fbafe9c9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220de490 0x5643220ef820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:29.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.892+0000 7fbafe9c9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643220dedb0 0x5643220efd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:29.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.892+0000 7fbafe9c9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220e9670 0x564322162c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:29.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.892+0000 7fbafe9c9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf41ae220 con 0x5643220e9670 2024-09-11T23:31:29.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.892+0000 7fbafe9c9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf41af690 con 0x5643220de490 2024-09-11T23:31:29.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.892+0000 7fbafe9c9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf41ae8f0 con 0x5643220dedb0 2024-09-11T23:31:29.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.892+0000 7fbafe9c9740 10 client.0 _ll_register_callbacks cb 0x7fbaf00f3330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:31:29.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.892+0000 7fbaf9b8b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643220dedb0 0x5643220efd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:29.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.892+0000 7fbafab8d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220e9670 0x564322162c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:29.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.892+0000 7fbaf9b8b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643220dedb0 0x5643220efd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:44846/0 (socket says 172.21.15.43:44846) 2024-09-11T23:31:29.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.892+0000 7fbafa38c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220de490 0x5643220ef820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:29.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.892+0000 7fbaf9b8b640 1 -- 172.21.15.43:0/144203721 learned_addr learned my addr 172.21.15.43:0/144203721 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:31:29.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.892+0000 7fbafab8d640 1 -- 172.21.15.43:0/144203721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643220dedb0 msgr2=0x5643220efd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:29.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.892+0000 7fbafab8d640 1 --2- 172.21.15.43:0/144203721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643220dedb0 0x5643220efd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.892+0000 7fbafab8d640 1 -- 172.21.15.43:0/144203721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220de490 msgr2=0x5643220ef820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:31:29.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.892+0000 7fbafab8d640 1 --2- 172.21.15.43:0/144203721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220de490 0x5643220ef820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.892+0000 7fbafab8d640 1 -- 172.21.15.43:0/144203721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643221631a0 con 0x5643220e9670 2024-09-11T23:31:29.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.892+0000 7fbafab8d640 1 --2- 172.21.15.43:0/144203721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220e9670 0x564322162c60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fbaec009ee0 tx=0x7fbaec009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1814e77bab5092 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:31:29.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.892+0000 7fbaddffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:31:29.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.892+0000 7fbaddffb640 1 -- 172.21.15.43:0/144203721 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec042060 con 0x5643220e9670 2024-09-11T23:31:29.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.893+0000 7fbaddffb640 1 -- 172.21.15.43:0/144203721 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fbaec01e630 con 0x5643220e9670 2024-09-11T23:31:29.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.893+0000 7fbafe9c9740 1 -- 172.21.15.43:0/144203721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaec003d70 con 0x5643220e9670 2024-09-11T23:31:29.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.893+0000 7fbaddffb640 1 -- 172.21.15.43:0/144203721 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbaec039070 con 0x5643220e9670 2024-09-11T23:31:29.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.893+0000 7fbafe9c9740 10 client.16986 fetch_fsmap learned FSMap version 33 2024-09-11T23:31:29.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.893+0000 7fbafe9c9740 1 -- 172.21.15.43:0/144203721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564322164390 con 0x5643220e9670 2024-09-11T23:31:29.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.893+0000 7fbaddffb640 1 -- 172.21.15.43:0/144203721 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaec003d70 con 0x5643220e9670 2024-09-11T23:31:29.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.893+0000 7fbaddffb640 1 -- 172.21.15.43:0/144203721 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7fbaec033070 con 0x5643220e9670 2024-09-11T23:31:29.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.893+0000 7fbafe9c9740 10 client.16986 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T23:31:29.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.893+0000 7fbafe9c9740 10 client.16986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:31:29.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.893+0000 7fbafe9c9740 20 client.16986 populate_metadata read hostname 'smithi043' 2024-09-11T23:31:29.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.893+0000 7fbafe9c9740 1 --2- 172.21.15.43:0/144203721 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5643220f8f60 0x5643221668c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:31:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.893+0000 7fbafe9c9740 4 client.16986 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:31:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.893+0000 7fbafe9c9740 1 -- 172.21.15.43:0/144203721 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56432204bd60 con 0x5643220f8f60 2024-09-11T23:31:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.893+0000 7fbafa38c640 1 --2- 172.21.15.43:0/144203721 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5643220f8f60 0x5643221668c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:31:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.894+0000 7fbafa38c640 1 --2- 172.21.15.43:0/144203721 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5643220f8f60 0x5643221668c0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9e400f7486a6e72 server_cookie=eeb29a7064abfd00 in_seq=0 out_seq=0 2024-09-11T23:31:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.894+0000 7fbaddffb640 10 client.16986 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:31:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.894+0000 7fbaddffb640 1 -- 172.21.15.43:0/144203721 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56432204bd60 con 0x5643220f8f60 2024-09-11T23:31:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.894+0000 7fbaddffb640 10 client.16986 handle_command_reply: tid=0 2024-09-11T23:31:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.894+0000 7fbafe9c9740 1 client.16986 shutdown 2024-09-11T23:31:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.894+0000 7fbafe9c9740 1 -- 172.21.15.43:0/144203721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220e9670 msgr2=0x564322162c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.894+0000 7fbafe9c9740 1 --2- 172.21.15.43:0/144203721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220e9670 0x564322162c60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fbaec009ee0 tx=0x7fbaec009c70 comp rx=0 tx=0).stop 2024-09-11T23:31:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.894+0000 7fbafe9c9740 1 -- 172.21.15.43:0/144203721 shutdown_connections 2024-09-11T23:31:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.894+0000 7fbafe9c9740 1 --2- 172.21.15.43:0/144203721 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5643220f8f60 0x5643221668c0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.894+0000 7fbafe9c9740 1 --2- 172.21.15.43:0/144203721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643220e9670 0x564322162c60 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.894+0000 7fbaddffb640 0 client.16986 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:31:29.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.894+0000 7fbafe9c9740 1 --2- 172.21.15.43:0/144203721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5643220dedb0 0x5643220efd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.894+0000 7fbafe9c9740 1 --2- 172.21.15.43:0/144203721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643220de490 0x5643220ef820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.894+0000 7fbafe9c9740 1 -- 172.21.15.43:0/144203721 >> 172.21.15.43:0/144203721 conn(0x5643220daa20 msgr2=0x5643220d9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:29.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.894+0000 7fbafe9c9740 1 -- 172.21.15.43:0/144203721 shutdown_connections 2024-09-11T23:31:29.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.894+0000 7fbafe9c9740 1 -- 172.21.15.43:0/144203721 wait complete. 2024-09-11T23:31:29.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.894+0000 7fbafe9c9740 20 client.16986 trim_cache size 0 max 16384 2024-09-11T23:31:29.065 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:31:29.065 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:31:29.066 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-09-11T23:31:29.066 INFO:teuthology.orchestra.run.smithi043.stdout: "cbba29dc-7900-47db-86ba-e8bc90e2bd13": { 2024-09-11T23:31:29.066 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-09-11T23:31:29.066 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "cbba29dc-7900-47db-86ba-e8bc90e2bd13", 2024-09-11T23:31:29.066 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,force" 2024-09-11T23:31:29.066 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:31:29.066 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:31:29.066 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:31:29.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.895+0000 7fbafc5f0640 1 -- 172.21.15.43:0/2682012394 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fbae00a2cf0 msgr2=0x7fbae00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:29.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.895+0000 7fbafc5f0640 1 --2- 172.21.15.43:0/2682012394 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fbae00a2cf0 0x7fbae00a51b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbaec01f000 tx=0x7fbaec020000 comp rx=0 tx=0).stop 2024-09-11T23:31:29.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.895+0000 7fbafc5f0640 1 -- 172.21.15.43:0/2682012394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf4102d30 msgr2=0x7fbaf4100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:31:29.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.895+0000 7fbafc5f0640 1 --2- 172.21.15.43:0/2682012394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf4102d30 0x7fbaf4100990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbaf002a940 tx=0x7fbaf0003b20 comp rx=0 tx=0).stop 2024-09-11T23:31:29.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.895+0000 7fbafc5f0640 1 -- 172.21.15.43:0/2682012394 shutdown_connections 2024-09-11T23:31:29.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.895+0000 7fbafc5f0640 1 --2- 172.21.15.43:0/2682012394 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fbae00a2cf0 0x7fbae00a51b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.895+0000 7fbafc5f0640 1 --2- 172.21.15.43:0/2682012394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbaf41082a0 0x7fbaf4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.895+0000 7fbafc5f0640 1 --2- 172.21.15.43:0/2682012394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbaf41036e0 0x7fbaf4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.895+0000 7fbafc5f0640 1 --2- 172.21.15.43:0/2682012394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaf4102d30 0x7fbaf4100990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:31:29.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.895+0000 7fbafc5f0640 1 -- 172.21.15.43:0/2682012394 >> 172.21.15.43:0/2682012394 conn(0x7fbaf40fc7d0 msgr2=0x7fbaf4106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:31:29.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.895+0000 7fbafc5f0640 1 -- 172.21.15.43:0/2682012394 shutdown_connections 2024-09-11T23:31:29.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:31:28.895+0000 7fbafc5f0640 1 -- 172.21.15.43:0/2682012394 wait complete. 2024-09-11T23:31:29.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:29 smithi043 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:31:29.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:29 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:29.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:29 smithi090 ceph-mon[35130]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:31:29.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:29 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:29.300 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:29 smithi097 ceph-mon[35562]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:31:29.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:29 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:31:29.584 INFO:tasks.cephfs.filesystem:scrub status for tag:cbba29dc-7900-47db-86ba-e8bc90e2bd13 - {'path': '/', 'tag': 'cbba29dc-7900-47db-86ba-e8bc90e2bd13', 'options': 'recursive,force'} 2024-09-11T23:31:30.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:30 smithi090 ceph-mon[35130]: pgmap v299: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-11T23:31:30.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:30 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:30.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:30 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:31:30.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:30 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:31:30.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:30 smithi090 ceph-mon[35130]: scrub summary: idle 2024-09-11T23:31:30.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:30 smithi097 ceph-mon[35562]: pgmap v299: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-11T23:31:30.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:30 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:30.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:30 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:31:30.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:30 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:31:30.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:30 smithi097 ceph-mon[35562]: scrub summary: idle 2024-09-11T23:31:30.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:30 smithi043 ceph-mon[30950]: pgmap v299: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-11T23:31:30.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:30 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:31:30.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:30 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:31:30.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:30 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:31:30.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:30 smithi043 ceph-mon[30950]: scrub summary: idle 2024-09-11T23:31:32.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:32 smithi097 ceph-mon[35562]: pgmap v300: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-11T23:31:32.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:32 smithi043 ceph-mon[30950]: pgmap v300: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-11T23:31:32.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:32 smithi090 ceph-mon[35130]: pgmap v300: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-11T23:31:34.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:34 smithi097 ceph-mon[35562]: pgmap v301: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-11T23:31:34.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:34 smithi043 ceph-mon[30950]: pgmap v301: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-11T23:31:34.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:34 smithi090 ceph-mon[35130]: pgmap v301: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-11T23:31:36.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:36 smithi097 ceph-mon[35562]: pgmap v302: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-09-11T23:31:36.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:36 smithi043 ceph-mon[30950]: pgmap v302: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-09-11T23:31:36.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:36 smithi090 ceph-mon[35130]: pgmap v302: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-09-11T23:31:38.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:38 smithi097 ceph-mon[35562]: pgmap v303: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8 op/s 2024-09-11T23:31:38.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:38 smithi043 ceph-mon[30950]: pgmap v303: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8 op/s 2024-09-11T23:31:38.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:38 smithi090 ceph-mon[35130]: pgmap v303: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8 op/s 2024-09-11T23:31:40.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:40 smithi097 ceph-mon[35562]: pgmap v304: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-11T23:31:40.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:40 smithi043 ceph-mon[30950]: pgmap v304: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-11T23:31:40.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:40 smithi090 ceph-mon[35130]: pgmap v304: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-09-11T23:31:42.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:42 smithi097 ceph-mon[35562]: pgmap v305: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-09-11T23:31:42.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:42 smithi043 ceph-mon[30950]: pgmap v305: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-09-11T23:31:42.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:42 smithi090 ceph-mon[35130]: pgmap v305: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-09-11T23:31:44.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:44 smithi097 ceph-mon[35562]: pgmap v306: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:31:44.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:44 smithi043 ceph-mon[30950]: pgmap v306: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:31:44.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:44 smithi090 ceph-mon[35130]: pgmap v306: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:31:46.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:46 smithi097 ceph-mon[35562]: pgmap v307: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:31:46.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:46 smithi043 ceph-mon[30950]: pgmap v307: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:31:46.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:46 smithi090 ceph-mon[35130]: pgmap v307: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:31:48.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:48 smithi097 ceph-mon[35562]: pgmap v308: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T23:31:48.322 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:48 smithi090 ceph-mon[35130]: pgmap v308: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T23:31:48.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:48 smithi043 ceph-mon[30950]: pgmap v308: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T23:31:50.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:50 smithi097 ceph-mon[35562]: pgmap v309: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:31:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:50 smithi043 ceph-mon[30950]: pgmap v309: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:31:50.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:50 smithi090 ceph-mon[35130]: pgmap v309: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:31:52.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:52 smithi043 ceph-mon[30950]: pgmap v310: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T23:31:52.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:52 smithi090 ceph-mon[35130]: pgmap v310: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T23:31:52.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:52 smithi097 ceph-mon[35562]: pgmap v310: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T23:31:54.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:54 smithi043 ceph-mon[30950]: pgmap v311: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:31:54.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:54 smithi090 ceph-mon[35130]: pgmap v311: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:31:54.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:54 smithi097 ceph-mon[35562]: pgmap v311: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:31:56.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:56 smithi043 ceph-mon[30950]: pgmap v312: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:31:56.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:56 smithi090 ceph-mon[35130]: pgmap v312: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:31:56.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:56 smithi097 ceph-mon[35562]: pgmap v312: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:31:57.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:57 smithi043 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:31:57.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:57 smithi090 ceph-mon[35130]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:31:57.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:57 smithi097 ceph-mon[35562]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T23:31:58.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:31:58 smithi043 ceph-mon[30950]: pgmap v313: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T23:31:58.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:31:58 smithi090 ceph-mon[35130]: pgmap v313: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T23:31:58.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:31:58 smithi097 ceph-mon[35562]: pgmap v313: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T23:31:59.585 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:31:59.847 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:32:00.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:00 smithi043 ceph-mon[30950]: pgmap v314: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:32:00.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.469+0000 7f5f7ed91640 1 -- 172.21.15.43:0/4214459203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f78100a80 msgr2=0x7f5f78100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:00.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.469+0000 7f5f7ed91640 1 --2- 172.21.15.43:0/4214459203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f78100a80 0x7f5f78100ee0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009930 tx=0x7f5f6402a420 comp rx=0 tx=0).stop 2024-09-11T23:32:00.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.469+0000 7f5f7ed91640 1 -- 172.21.15.43:0/4214459203 shutdown_connections 2024-09-11T23:32:00.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.469+0000 7f5f7ed91640 1 --2- 172.21.15.43:0/4214459203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f78101420 0x7f5f7810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.469+0000 7f5f7ed91640 1 --2- 172.21.15.43:0/4214459203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f78100a80 0x7f5f78100ee0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.469+0000 7f5f7ed91640 1 --2- 172.21.15.43:0/4214459203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f78106a80 0x7f5f78106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.469+0000 7f5f7ed91640 1 -- 172.21.15.43:0/4214459203 >> 172.21.15.43:0/4214459203 conn(0x7f5f780fc7a0 msgr2=0x7f5f780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:00.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.469+0000 7f5f7ed91640 1 -- 172.21.15.43:0/4214459203 shutdown_connections 2024-09-11T23:32:00.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.469+0000 7f5f7ed91640 1 -- 172.21.15.43:0/4214459203 wait complete. 2024-09-11T23:32:00.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.470+0000 7f5f7ed91640 1 Processor -- start 2024-09-11T23:32:00.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.470+0000 7f5f7ed91640 1 -- start start 2024-09-11T23:32:00.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.470+0000 7f5f7ed91640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f78100a80 0x7f5f781984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:00.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.470+0000 7f5f7ed91640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f78101420 0x7f5f78198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:00.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.470+0000 7f5f7ed91640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f78106a80 0x7f5f7819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:00.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.470+0000 7f5f7ed91640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5f78110ec0 con 0x7f5f78100a80 2024-09-11T23:32:00.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.470+0000 7f5f7ed91640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5f78110d40 con 0x7f5f78106a80 2024-09-11T23:32:00.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.470+0000 7f5f7ed91640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5f78111040 con 0x7f5f78101420 2024-09-11T23:32:00.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.470+0000 7f5f7cb2d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f78100a80 0x7f5f781984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:00.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.470+0000 7f5f7cb2d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f78100a80 0x7f5f781984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38292/0 (socket says 172.21.15.43:38292) 2024-09-11T23:32:00.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.470+0000 7f5f7cb2d640 1 -- 172.21.15.43:0/3541156309 learned_addr learned my addr 172.21.15.43:0/3541156309 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:00.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.470+0000 7f5f7d32e640 1 --2- 172.21.15.43:0/3541156309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f78106a80 0x7f5f7819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:00.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.470+0000 7f5f77fff640 1 --2- 172.21.15.43:0/3541156309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f78101420 0x7f5f78198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:00.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.470+0000 7f5f7cb2d640 1 -- 172.21.15.43:0/3541156309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f78101420 msgr2=0x7f5f78198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:00.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.470+0000 7f5f7cb2d640 1 --2- 172.21.15.43:0/3541156309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f78101420 0x7f5f78198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.470+0000 7f5f7cb2d640 1 -- 172.21.15.43:0/3541156309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f78106a80 msgr2=0x7f5f7819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:00.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.470+0000 7f5f7cb2d640 1 --2- 172.21.15.43:0/3541156309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f78106a80 0x7f5f7819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.470+0000 7f5f7cb2d640 1 -- 172.21.15.43:0/3541156309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7819d480 con 0x7f5f78100a80 2024-09-11T23:32:00.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.471+0000 7f5f7cb2d640 1 --2- 172.21.15.43:0/3541156309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f78100a80 0x7f5f781984d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00b4a0 tx=0x7f5f6c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.471+0000 7f5f75ffb640 1 -- 172.21.15.43:0/3541156309 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c014070 con 0x7f5f78100a80 2024-09-11T23:32:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.471+0000 7f5f7ed91640 1 -- 172.21.15.43:0/3541156309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f7819d770 con 0x7f5f78100a80 2024-09-11T23:32:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.471+0000 7f5f75ffb640 1 -- 172.21.15.43:0/3541156309 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f6c010af0 con 0x7f5f78100a80 2024-09-11T23:32:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.471+0000 7f5f75ffb640 1 -- 172.21.15.43:0/3541156309 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c017d10 con 0x7f5f78100a80 2024-09-11T23:32:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.471+0000 7f5f7ed91640 1 -- 172.21.15.43:0/3541156309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f781a5050 con 0x7f5f78100a80 2024-09-11T23:32:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.473+0000 7f5f75ffb640 1 -- 172.21.15.43:0/3541156309 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5f6c017070 con 0x7f5f78100a80 2024-09-11T23:32:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.474+0000 7f5f8116a740 1 Processor -- start 2024-09-11T23:32:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.474+0000 7f5f8116a740 1 -- start start 2024-09-11T23:32:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.474+0000 7f5f8116a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651a32724a0 0x5651a3272880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.474+0000 7f5f8116a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a3272dc0 0x5651a327d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.474+0000 7f5f8116a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a327d680 0x5651a327fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.474+0000 7f5f8116a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5651a31d82e0 con 0x5651a3272dc0 2024-09-11T23:32:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.474+0000 7f5f8116a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5651a317fce0 con 0x5651a32724a0 2024-09-11T23:32:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.474+0000 7f5f8116a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5651a31d42a0 con 0x5651a327d680 2024-09-11T23:32:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.474+0000 7f5f7cb2d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a3272dc0 0x5651a327d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.474+0000 7f5f7cb2d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a3272dc0 0x5651a327d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38298/0 (socket says 172.21.15.43:38298) 2024-09-11T23:32:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.474+0000 7f5f7cb2d640 1 -- 172.21.15.43:0/2225898433 learned_addr learned my addr 172.21.15.43:0/2225898433 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.474+0000 7f5f75ffb640 1 --2- 172.21.15.43:0/3541156309 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5f680a2cf0 0x7f5f680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.474+0000 7f5f7d32e640 1 --2- 172.21.15.43:0/2225898433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651a32724a0 0x5651a3272880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.474+0000 7f5f77fff640 1 --2- 172.21.15.43:0/2225898433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a327d680 0x5651a327fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.474+0000 7f5f7d32e640 1 --2- 172.21.15.43:0/3541156309 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5f680a2cf0 0x7f5f680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.474+0000 7f5f75ffb640 1 -- 172.21.15.43:0/3541156309 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f5f6c0ce850 con 0x7f5f78100a80 2024-09-11T23:32:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.475+0000 7f5f7cb2d640 1 -- 172.21.15.43:0/2225898433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a327d680 msgr2=0x5651a327fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.475+0000 7f5f7cb2d640 1 --2- 172.21.15.43:0/2225898433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a327d680 0x5651a327fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.475+0000 7f5f7cb2d640 1 -- 172.21.15.43:0/2225898433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651a32724a0 msgr2=0x5651a3272880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.475+0000 7f5f7cb2d640 1 --2- 172.21.15.43:0/2225898433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651a32724a0 0x5651a3272880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.475+0000 7f5f7cb2d640 1 -- 172.21.15.43:0/2225898433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a3164440 con 0x5651a3272dc0 2024-09-11T23:32:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.475+0000 7f5f7d32e640 1 --2- 172.21.15.43:0/2225898433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651a32724a0 0x5651a3272880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:32:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.475+0000 7f5f7cb2d640 1 --2- 172.21.15.43:0/2225898433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a3272dc0 0x5651a327d140 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c09aaf0 tx=0x7f5f6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95624756f8ec88ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.475+0000 7f5f62ffd640 1 -- 172.21.15.43:0/2225898433 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c01e070 con 0x5651a3272dc0 2024-09-11T23:32:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.475+0000 7f5f7d32e640 1 --2- 172.21.15.43:0/3541156309 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5f680a2cf0 0x7f5f680a51b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000bfd0 tx=0x7f5f70007220 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.475+0000 7f5f62ffd640 1 -- 172.21.15.43:0/2225898433 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f6c09adf0 con 0x5651a3272dc0 2024-09-11T23:32:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.475+0000 7f5f62ffd640 1 -- 172.21.15.43:0/2225898433 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c010e00 con 0x5651a3272dc0 2024-09-11T23:32:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.476+0000 7f5f8116a740 1 -- 172.21.15.43:0/2225898433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a3272dc0 msgr2=0x5651a327d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.476+0000 7f5f8116a740 1 --2- 172.21.15.43:0/2225898433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a3272dc0 0x5651a327d140 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c09aaf0 tx=0x7f5f6c0046c0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.476+0000 7f5f8116a740 1 -- 172.21.15.43:0/2225898433 shutdown_connections 2024-09-11T23:32:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.476+0000 7f5f8116a740 1 --2- 172.21.15.43:0/2225898433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a327d680 0x5651a327fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.476+0000 7f5f8116a740 1 --2- 172.21.15.43:0/2225898433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a3272dc0 0x5651a327d140 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.476+0000 7f5f8116a740 1 --2- 172.21.15.43:0/2225898433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651a32724a0 0x5651a3272880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.476+0000 7f5f8116a740 1 -- 172.21.15.43:0/2225898433 >> 172.21.15.43:0/2225898433 conn(0x5651a326ea30 msgr2=0x5651a326ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.476+0000 7f5f8116a740 1 -- 172.21.15.43:0/2225898433 shutdown_connections 2024-09-11T23:32:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.476+0000 7f5f8116a740 1 -- 172.21.15.43:0/2225898433 wait complete. 2024-09-11T23:32:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.476+0000 7f5f8116a740 1 -- start start 2024-09-11T23:32:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.476+0000 7f5f8116a740 1 Processor -- start 2024-09-11T23:32:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.477+0000 7f5f8116a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a32724a0 0x5651a327cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.477+0000 7f5f8116a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651a3272dc0 0x5651a327a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.477+0000 7f5f8116a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a327d680 0x5651a327a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.477+0000 7f5f8116a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5651a32801f0 con 0x5651a327d680 2024-09-11T23:32:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.477+0000 7f5f8116a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5651a3193050 con 0x5651a3272dc0 2024-09-11T23:32:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.477+0000 7f5f8116a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5651a3280370 con 0x5651a32724a0 2024-09-11T23:32:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.477+0000 7f5f8116a740 10 client.0 _ll_register_callbacks cb 0x5651a31d3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:32:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.477+0000 7f5f7d32e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a327d680 0x5651a327a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.477+0000 7f5f7cb2d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a32724a0 0x5651a327cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:00.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.477+0000 7f5f7d32e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a327d680 0x5651a327a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38312/0 (socket says 172.21.15.43:38312) 2024-09-11T23:32:00.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.477+0000 7f5f7d32e640 1 -- 172.21.15.43:0/2328583863 learned_addr learned my addr 172.21.15.43:0/2328583863 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:00.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.477+0000 7f5f7cb2d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a32724a0 0x5651a327cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:34642/0 (socket says 172.21.15.43:34642) 2024-09-11T23:32:00.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.477+0000 7f5f77fff640 1 --2- 172.21.15.43:0/2328583863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651a3272dc0 0x5651a327a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:00.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.477+0000 7f5f7d32e640 1 -- 172.21.15.43:0/2328583863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a32724a0 msgr2=0x5651a327cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:00.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.477+0000 7f5f7d32e640 1 --2- 172.21.15.43:0/2328583863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a32724a0 0x5651a327cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.477+0000 7f5f7d32e640 1 -- 172.21.15.43:0/2328583863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651a3272dc0 msgr2=0x5651a327a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:00.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.477+0000 7f5f7d32e640 1 --2- 172.21.15.43:0/2328583863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651a3272dc0 0x5651a327a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.477+0000 7f5f7d32e640 1 -- 172.21.15.43:0/2328583863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a2f493c0 con 0x5651a327d680 2024-09-11T23:32:00.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.477+0000 7f5f7d32e640 1 --2- 172.21.15.43:0/2328583863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a327d680 0x5651a327a780 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000ef00 tx=0x7f5f70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ca8b5bc5e02ce7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:00.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.477+0000 7f5f57fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:32:00.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.477+0000 7f5f57fff640 1 -- 172.21.15.43:0/2328583863 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f70022070 con 0x5651a327d680 2024-09-11T23:32:00.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.478+0000 7f5f57fff640 1 -- 172.21.15.43:0/2328583863 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f5f70010c80 con 0x5651a327d680 2024-09-11T23:32:00.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.478+0000 7f5f8116a740 1 -- 172.21.15.43:0/2328583863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f6c09adf0 con 0x5651a327d680 2024-09-11T23:32:00.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.478+0000 7f5f57fff640 1 -- 172.21.15.43:0/2328583863 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5f7002e050 con 0x5651a327d680 2024-09-11T23:32:00.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.478+0000 7f5f8116a740 10 client.16998 fetch_fsmap learned FSMap version 34 2024-09-11T23:32:00.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.478+0000 7f5f8116a740 1 -- 172.21.15.43:0/2328583863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5651a328bbd0 con 0x5651a327d680 2024-09-11T23:32:00.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.478+0000 7f5f57fff640 1 -- 172.21.15.43:0/2328583863 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f6c09adf0 con 0x5651a327d680 2024-09-11T23:32:00.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.478+0000 7f5f57fff640 1 -- 172.21.15.43:0/2328583863 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f5f7001d040 con 0x5651a327d680 2024-09-11T23:32:00.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.478+0000 7f5f8116a740 10 client.16998 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T23:32:00.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.478+0000 7f5f8116a740 10 client.16998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:32:00.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.478+0000 7f5f8116a740 20 client.16998 populate_metadata read hostname 'smithi043' 2024-09-11T23:32:00.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.478+0000 7f5f8116a740 1 --2- 172.21.15.43:0/2328583863 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5651a328cfe0 0x5651a328f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:00.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.478+0000 7f5f8116a740 4 client.16998 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:32:00.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.478+0000 7f5f8116a740 1 -- 172.21.15.43:0/2328583863 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651a31e5850 con 0x5651a328cfe0 2024-09-11T23:32:00.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.478+0000 7f5f7cb2d640 1 --2- 172.21.15.43:0/2328583863 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5651a328cfe0 0x5651a328f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:00.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.479+0000 7f5f7cb2d640 1 --2- 172.21.15.43:0/2328583863 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5651a328cfe0 0x5651a328f440 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=985f4f3b511ac18a server_cookie=33e2af8bff5c1afd in_seq=0 out_seq=0 2024-09-11T23:32:00.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.479+0000 7f5f57fff640 10 client.16998 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:32:00.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.480+0000 7f5f57fff640 1 -- 172.21.15.43:0/2328583863 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5651a31e5850 con 0x5651a328cfe0 2024-09-11T23:32:00.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.480+0000 7f5f57fff640 10 client.16998 handle_command_reply: tid=0 2024-09-11T23:32:00.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.480+0000 7f5f8116a740 1 client.16998 shutdown 2024-09-11T23:32:00.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.480+0000 7f5f8116a740 1 -- 172.21.15.43:0/2328583863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a327d680 msgr2=0x5651a327a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.480+0000 7f5f8116a740 1 --2- 172.21.15.43:0/2328583863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a327d680 0x5651a327a780 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000ef00 tx=0x7f5f70004a40 comp rx=0 tx=0).stop 2024-09-11T23:32:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.480+0000 7f5f8116a740 1 -- 172.21.15.43:0/2328583863 shutdown_connections 2024-09-11T23:32:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.480+0000 7f5f8116a740 1 --2- 172.21.15.43:0/2328583863 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5651a328cfe0 0x5651a328f440 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.480+0000 7f5f8116a740 1 --2- 172.21.15.43:0/2328583863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a327d680 0x5651a327a780 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.480+0000 7f5f8116a740 1 --2- 172.21.15.43:0/2328583863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651a3272dc0 0x5651a327a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.480+0000 7f5f57fff640 0 client.16998 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:32:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.480+0000 7f5f8116a740 1 --2- 172.21.15.43:0/2328583863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a32724a0 0x5651a327cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.480+0000 7f5f8116a740 1 -- 172.21.15.43:0/2328583863 >> 172.21.15.43:0/2328583863 conn(0x5651a326ea30 msgr2=0x5651a3274760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.480+0000 7f5f8116a740 1 -- 172.21.15.43:0/2328583863 shutdown_connections 2024-09-11T23:32:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.480+0000 7f5f8116a740 1 -- 172.21.15.43:0/2328583863 wait complete. 2024-09-11T23:32:00.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.480+0000 7f5f8116a740 20 client.16998 trim_cache size 0 max 16384 2024-09-11T23:32:00.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.499+0000 7f5f8116a740 1 Processor -- start 2024-09-11T23:32:00.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.499+0000 7f5f8116a740 1 -- start start 2024-09-11T23:32:00.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.499+0000 7f5f8116a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651a326ea30 0x5651a327c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:00.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.499+0000 7f5f8116a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a32724a0 0x5651a3294d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.499+0000 7f5f8116a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a3272dc0 0x5651a32952a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.499+0000 7f5f8116a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5f781b03a0 con 0x5651a32724a0 2024-09-11T23:32:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.499+0000 7f5f8116a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5f781adc70 con 0x5651a326ea30 2024-09-11T23:32:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.499+0000 7f5f8116a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5f781abae0 con 0x5651a3272dc0 2024-09-11T23:32:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.499+0000 7f5f77fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a32724a0 0x5651a3294d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.499+0000 7f5f77fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a32724a0 0x5651a3294d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38328/0 (socket says 172.21.15.43:38328) 2024-09-11T23:32:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.499+0000 7f5f77fff640 1 -- 172.21.15.43:0/1179096237 learned_addr learned my addr 172.21.15.43:0/1179096237 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.499+0000 7f5f7cb2d640 1 --2- 172.21.15.43:0/1179096237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651a326ea30 0x5651a327c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.499+0000 7f5f7d32e640 1 --2- 172.21.15.43:0/1179096237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a3272dc0 0x5651a32952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.500+0000 7f5f7cb2d640 1 -- 172.21.15.43:0/1179096237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a3272dc0 msgr2=0x5651a32952a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.500+0000 7f5f7cb2d640 1 --2- 172.21.15.43:0/1179096237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a3272dc0 0x5651a32952a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.500+0000 7f5f7cb2d640 1 -- 172.21.15.43:0/1179096237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a32724a0 msgr2=0x5651a3294d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:00.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.500+0000 7f5f7cb2d640 1 --2- 172.21.15.43:0/1179096237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a32724a0 0x5651a3294d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.500+0000 7f5f7cb2d640 1 -- 172.21.15.43:0/1179096237 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a3272bf0 con 0x5651a326ea30 2024-09-11T23:32:00.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.500+0000 7f5f77fff640 1 --2- 172.21.15.43:0/1179096237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a32724a0 0x5651a3294d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:32:00.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.500+0000 7f5f7cb2d640 1 --2- 172.21.15.43:0/1179096237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651a326ea30 0x5651a327c1b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c096ad0 tx=0x7f5f6c096a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edafacd59a3cfc28 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:00.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.500+0000 7f5f627fc640 1 -- 172.21.15.43:0/1179096237 <== mon.2 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c01e040 con 0x5651a326ea30 2024-09-11T23:32:00.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.500+0000 7f5f627fc640 1 -- 172.21.15.43:0/1179096237 <== mon.2 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f6c010ca0 con 0x5651a326ea30 2024-09-11T23:32:00.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.500+0000 7f5f627fc640 1 -- 172.21.15.43:0/1179096237 <== mon.2 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c0cb4e0 con 0x5651a326ea30 2024-09-11T23:32:00.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.501+0000 7f5f8116a740 1 -- 172.21.15.43:0/1179096237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651a326ea30 msgr2=0x5651a327c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:00.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.501+0000 7f5f8116a740 1 --2- 172.21.15.43:0/1179096237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651a326ea30 0x5651a327c1b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c096ad0 tx=0x7f5f6c096a60 comp rx=0 tx=0).stop 2024-09-11T23:32:00.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.501+0000 7f5f8116a740 1 -- 172.21.15.43:0/1179096237 shutdown_connections 2024-09-11T23:32:00.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.501+0000 7f5f8116a740 1 --2- 172.21.15.43:0/1179096237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a3272dc0 0x5651a32952a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.501+0000 7f5f8116a740 1 --2- 172.21.15.43:0/1179096237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a32724a0 0x5651a3294d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.501+0000 7f5f8116a740 1 --2- 172.21.15.43:0/1179096237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651a326ea30 0x5651a327c1b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.501+0000 7f5f8116a740 1 -- 172.21.15.43:0/1179096237 >> 172.21.15.43:0/1179096237 conn(0x5651a327a8e0 msgr2=0x5651a327baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:00.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.501+0000 7f5f8116a740 1 -- 172.21.15.43:0/1179096237 shutdown_connections 2024-09-11T23:32:00.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.501+0000 7f5f8116a740 1 -- 172.21.15.43:0/1179096237 wait complete. 2024-09-11T23:32:00.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.502+0000 7f5f8116a740 1 -- start start 2024-09-11T23:32:00.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.502+0000 7f5f8116a740 1 Processor -- start 2024-09-11T23:32:00.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.502+0000 7f5f8116a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a326ea30 0x5651a3285b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:00.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.502+0000 7f5f8116a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651a32724a0 0x5651a3282ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:00.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.502+0000 7f5f8116a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a3272dc0 0x5651a3283400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:00.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.502+0000 7f5f8116a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5f781ac130 con 0x5651a3272dc0 2024-09-11T23:32:00.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.502+0000 7f5f8116a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5f781ad5a0 con 0x5651a32724a0 2024-09-11T23:32:00.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.502+0000 7f5f8116a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5f781ac800 con 0x5651a326ea30 2024-09-11T23:32:00.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.502+0000 7f5f8116a740 10 client.0 _ll_register_callbacks cb 0x7f5f7000b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:32:00.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.502+0000 7f5f7d32e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a3272dc0 0x5651a3283400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:00.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.502+0000 7f5f7cb2d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a326ea30 0x5651a3285b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:00.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.503+0000 7f5f7d32e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a3272dc0 0x5651a3283400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38340/0 (socket says 172.21.15.43:38340) 2024-09-11T23:32:00.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.503+0000 7f5f7d32e640 1 -- 172.21.15.43:0/3223980774 learned_addr learned my addr 172.21.15.43:0/3223980774 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:00.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.503+0000 7f5f7cb2d640 1 -- 172.21.15.43:0/3223980774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651a32724a0 msgr2=0x5651a3282ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T23:32:00.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.503+0000 7f5f7cb2d640 1 --2- 172.21.15.43:0/3223980774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651a32724a0 0x5651a3282ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.503+0000 7f5f7cb2d640 1 -- 172.21.15.43:0/3223980774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a3272dc0 msgr2=0x5651a3283400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:00.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.503+0000 7f5f7cb2d640 1 --2- 172.21.15.43:0/3223980774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a3272dc0 0x5651a3283400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.503+0000 7f5f7cb2d640 1 -- 172.21.15.43:0/3223980774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a3283a40 con 0x5651a326ea30 2024-09-11T23:32:00.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.503+0000 7f5f7d32e640 1 --2- 172.21.15.43:0/3223980774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a3272dc0 0x5651a3283400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:32:00.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.503+0000 7f5f7cb2d640 1 --2- 172.21.15.43:0/3223980774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a326ea30 0x5651a3285b20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c096ad0 tx=0x7f5f6c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f9eb16518a6fc66 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:00.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.503+0000 7f5f57fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-09-11T23:32:00.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.503+0000 7f5f57fff640 1 -- 172.21.15.43:0/3223980774 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c01e050 con 0x5651a326ea30 2024-09-11T23:32:00.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.503+0000 7f5f8116a740 1 -- 172.21.15.43:0/3223980774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f6c010ca0 con 0x5651a326ea30 2024-09-11T23:32:00.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.503+0000 7f5f57fff640 1 -- 172.21.15.43:0/3223980774 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f5f6c0cb4e0 con 0x5651a326ea30 2024-09-11T23:32:00.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.504+0000 7f5f57fff640 1 -- 172.21.15.43:0/3223980774 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5f6c09adf0 con 0x5651a326ea30 2024-09-11T23:32:00.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.504+0000 7f5f8116a740 10 client.26410 fetch_fsmap learned FSMap version 34 2024-09-11T23:32:00.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.504+0000 7f5f8116a740 1 -- 172.21.15.43:0/3223980774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5651a32fbf70 con 0x5651a326ea30 2024-09-11T23:32:00.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.504+0000 7f5f57fff640 1 -- 172.21.15.43:0/3223980774 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f6c010ca0 con 0x5651a326ea30 2024-09-11T23:32:00.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.504+0000 7f5f57fff640 1 -- 172.21.15.43:0/3223980774 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f5f6c061030 con 0x5651a326ea30 2024-09-11T23:32:00.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.504+0000 7f5f8116a740 10 client.26410 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T23:32:00.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.504+0000 7f5f8116a740 10 client.26410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:32:00.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.504+0000 7f5f8116a740 20 client.26410 populate_metadata read hostname 'smithi043' 2024-09-11T23:32:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.504+0000 7f5f8116a740 1 --2- 172.21.15.43:0/3223980774 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5651a327d680 0x5651a32fe210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.504+0000 7f5f8116a740 4 client.26410 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:32:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.504+0000 7f5f8116a740 1 -- 172.21.15.43:0/3223980774 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5651a31dfd60 con 0x5651a327d680 2024-09-11T23:32:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.505+0000 7f5f77fff640 1 --2- 172.21.15.43:0/3223980774 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5651a327d680 0x5651a32fe210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.505+0000 7f5f57fff640 10 client.26410 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:32:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.505+0000 7f5f77fff640 1 --2- 172.21.15.43:0/3223980774 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5651a327d680 0x5651a32fe210 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9b677c62e0c50a1 server_cookie=e190255df6cfef9 in_seq=0 out_seq=0 2024-09-11T23:32:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.505+0000 7f5f57fff640 1 -- 172.21.15.43:0/3223980774 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5651a31dfd60 con 0x5651a327d680 2024-09-11T23:32:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.505+0000 7f5f57fff640 10 client.26410 handle_command_reply: tid=0 2024-09-11T23:32:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.505+0000 7f5f8116a740 1 client.26410 shutdown 2024-09-11T23:32:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.505+0000 7f5f8116a740 1 -- 172.21.15.43:0/3223980774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a326ea30 msgr2=0x5651a3285b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.505+0000 7f5f8116a740 1 --2- 172.21.15.43:0/3223980774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a326ea30 0x5651a3285b20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c096ad0 tx=0x7f5f6c000bc0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.505+0000 7f5f8116a740 1 -- 172.21.15.43:0/3223980774 shutdown_connections 2024-09-11T23:32:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.505+0000 7f5f8116a740 1 --2- 172.21.15.43:0/3223980774 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x5651a327d680 0x5651a32fe210 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.506+0000 7f5f8116a740 1 --2- 172.21.15.43:0/3223980774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5651a3272dc0 0x5651a3283400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.506+0000 7f5f8116a740 1 --2- 172.21.15.43:0/3223980774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651a32724a0 0x5651a3282ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.506+0000 7f5f8116a740 1 --2- 172.21.15.43:0/3223980774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651a326ea30 0x5651a3285b20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.506+0000 7f5f8116a740 1 -- 172.21.15.43:0/3223980774 >> 172.21.15.43:0/3223980774 conn(0x5651a327a8e0 msgr2=0x5651a327c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:00.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.506+0000 7f5f57fff640 0 client.26410 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:32:00.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.506+0000 7f5f8116a740 1 -- 172.21.15.43:0/3223980774 shutdown_connections 2024-09-11T23:32:00.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.506+0000 7f5f8116a740 1 -- 172.21.15.43:0/3223980774 wait complete. 2024-09-11T23:32:00.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.506+0000 7f5f8116a740 20 client.26410 trim_cache size 0 max 16384 2024-09-11T23:32:00.510 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:32:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-09-11T23:32:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-09-11T23:32:00.511 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:32:00.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.507+0000 7f5f7ed91640 1 -- 172.21.15.43:0/3541156309 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5f680a2cf0 msgr2=0x7f5f680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.507+0000 7f5f7ed91640 1 --2- 172.21.15.43:0/3541156309 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5f680a2cf0 0x7f5f680a51b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000bfd0 tx=0x7f5f70007220 comp rx=0 tx=0).stop 2024-09-11T23:32:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.507+0000 7f5f7ed91640 1 -- 172.21.15.43:0/3541156309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f78100a80 msgr2=0x7f5f781984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.507+0000 7f5f7ed91640 1 --2- 172.21.15.43:0/3541156309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f78100a80 0x7f5f781984d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00b4a0 tx=0x7f5f6c00b960 comp rx=0 tx=0).stop 2024-09-11T23:32:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.508+0000 7f5f7ed91640 1 -- 172.21.15.43:0/3541156309 shutdown_connections 2024-09-11T23:32:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.508+0000 7f5f7ed91640 1 --2- 172.21.15.43:0/3541156309 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f5f680a2cf0 0x7f5f680a51b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.508+0000 7f5f7ed91640 1 --2- 172.21.15.43:0/3541156309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f78106a80 0x7f5f7819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.508+0000 7f5f7ed91640 1 --2- 172.21.15.43:0/3541156309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f78101420 0x7f5f78198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.508+0000 7f5f7ed91640 1 --2- 172.21.15.43:0/3541156309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f78100a80 0x7f5f781984d0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.508+0000 7f5f7ed91640 1 -- 172.21.15.43:0/3541156309 >> 172.21.15.43:0/3541156309 conn(0x7f5f780fc7a0 msgr2=0x7f5f78104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.508+0000 7f5f7ed91640 1 -- 172.21.15.43:0/3541156309 shutdown_connections 2024-09-11T23:32:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:00.508+0000 7f5f7ed91640 1 -- 172.21.15.43:0/3541156309 wait complete. 2024-09-11T23:32:00.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:00 smithi090 ceph-mon[35130]: pgmap v314: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:32:00.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:00 smithi097 ceph-mon[35562]: pgmap v314: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:32:01.199 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:32:01.199 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:32:01.468 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:32:01.923 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-11T23:32:01.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.991+0000 7f56dbe96640 1 -- 172.21.15.43:0/4218037223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56d4102570 msgr2=0x7f56d41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:01.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.991+0000 7f56dbe96640 1 --2- 172.21.15.43:0/4218037223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56d4102570 0x7f56d41029b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009900 tx=0x7f56c802a480 comp rx=0 tx=0).stop 2024-09-11T23:32:01.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.991+0000 7f56dbe96640 1 -- 172.21.15.43:0/4218037223 shutdown_connections 2024-09-11T23:32:01.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.991+0000 7f56dbe96640 1 --2- 172.21.15.43:0/4218037223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56d4102ef0 0x7f56d410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:01.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.991+0000 7f56dbe96640 1 --2- 172.21.15.43:0/4218037223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56d4102570 0x7f56d41029b0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:01.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.991+0000 7f56dbe96640 1 --2- 172.21.15.43:0/4218037223 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56d4100e20 0x7f56d4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:01.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.991+0000 7f56dbe96640 1 -- 172.21.15.43:0/4218037223 >> 172.21.15.43:0/4218037223 conn(0x7f56d40fc7d0 msgr2=0x7f56d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:01.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.991+0000 7f56dbe96640 1 -- 172.21.15.43:0/4218037223 shutdown_connections 2024-09-11T23:32:01.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.991+0000 7f56dbe96640 1 -- 172.21.15.43:0/4218037223 wait complete. 2024-09-11T23:32:01.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.992+0000 7f56dbe96640 1 Processor -- start 2024-09-11T23:32:01.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.992+0000 7f56dbe96640 1 -- start start 2024-09-11T23:32:01.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.992+0000 7f56dbe96640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56d4100e20 0x7f56d4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:01.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.992+0000 7f56dbe96640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56d4102570 0x7f56d41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:01.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.992+0000 7f56dbe96640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56d4102ef0 0x7f56d419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:01.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.992+0000 7f56dbe96640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f56d4110f80 con 0x7f56d4102ef0 2024-09-11T23:32:01.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.992+0000 7f56dbe96640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f56d4110e00 con 0x7f56d4100e20 2024-09-11T23:32:01.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.992+0000 7f56dbe96640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f56d4111100 con 0x7f56d4102570 2024-09-11T23:32:01.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.992+0000 7f56da433640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56d4102ef0 0x7f56d419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:01.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.992+0000 7f56da433640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56d4102ef0 0x7f56d419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38362/0 (socket says 172.21.15.43:38362) 2024-09-11T23:32:01.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.992+0000 7f56da433640 1 -- 172.21.15.43:0/161345110 learned_addr learned my addr 172.21.15.43:0/161345110 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:01.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.992+0000 7f56d9c32640 1 --2- 172.21.15.43:0/161345110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56d4100e20 0x7f56d4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:01.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.992+0000 7f56d9431640 1 --2- 172.21.15.43:0/161345110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56d4102570 0x7f56d41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:01.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.992+0000 7f56da433640 1 -- 172.21.15.43:0/161345110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56d4102570 msgr2=0x7f56d41989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:01.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.992+0000 7f56da433640 1 --2- 172.21.15.43:0/161345110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56d4102570 0x7f56d41989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:01.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.992+0000 7f56da433640 1 -- 172.21.15.43:0/161345110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56d4100e20 msgr2=0x7f56d4198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:01.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.992+0000 7f56da433640 1 --2- 172.21.15.43:0/161345110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56d4100e20 0x7f56d4198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:01.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.992+0000 7f56da433640 1 -- 172.21.15.43:0/161345110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d419d430 con 0x7f56d4102ef0 2024-09-11T23:32:01.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.992+0000 7f56d9431640 1 --2- 172.21.15.43:0/161345110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56d4102570 0x7f56d41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:32:01.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.993+0000 7f56da433640 1 --2- 172.21.15.43:0/161345110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56d4102ef0 0x7f56d419cd50 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f56cc00e970 tx=0x7f56cc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:01.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.993+0000 7f56d2ffd640 1 -- 172.21.15.43:0/161345110 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc00ccc0 con 0x7f56d4102ef0 2024-09-11T23:32:01.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.993+0000 7f56dbe96640 1 -- 172.21.15.43:0/161345110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d419d720 con 0x7f56d4102ef0 2024-09-11T23:32:01.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.993+0000 7f56d2ffd640 1 -- 172.21.15.43:0/161345110 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56cc00ce60 con 0x7f56d4102ef0 2024-09-11T23:32:01.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.993+0000 7f56d2ffd640 1 -- 172.21.15.43:0/161345110 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc010490 con 0x7f56d4102ef0 2024-09-11T23:32:01.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.993+0000 7f56dbe96640 1 -- 172.21.15.43:0/161345110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d41a5000 con 0x7f56d4102ef0 2024-09-11T23:32:01.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.994+0000 7f56dbe96640 1 -- 172.21.15.43:0/161345110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d4101200 con 0x7f56d4102ef0 2024-09-11T23:32:01.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.996+0000 7f56d2ffd640 1 -- 172.21.15.43:0/161345110 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f56cc005000 con 0x7f56d4102ef0 2024-09-11T23:32:01.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.996+0000 7f56d2ffd640 1 --2- 172.21.15.43:0/161345110 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56bc0a2d70 0x7f56bc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:01.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.996+0000 7f56d9c32640 1 --2- 172.21.15.43:0/161345110 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56bc0a2d70 0x7f56bc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:01.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.996+0000 7f56d2ffd640 1 -- 172.21.15.43:0/161345110 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f56cc0cfb80 con 0x7f56d4102ef0 2024-09-11T23:32:01.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.997+0000 7f56d9c32640 1 --2- 172.21.15.43:0/161345110 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56bc0a2d70 0x7f56bc0a5230 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f56c4005fd0 tx=0x7f56c4009680 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:01.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:01.999+0000 7f56d2ffd640 1 -- 172.21.15.43:0/161345110 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56cc0541b0 con 0x7f56d4102ef0 2024-09-11T23:32:02.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.038+0000 7f0e5f465640 1 -- 172.21.15.43:0/1649038965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e58107ad0 msgr2=0x7f0e58107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:02.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.038+0000 7f0e5f465640 1 --2- 172.21.15.43:0/1649038965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e58107ad0 0x7f0e58107eb0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009930 tx=0x7f0e5402a420 comp rx=0 tx=0).stop 2024-09-11T23:32:02.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.038+0000 7f0e5f465640 1 -- 172.21.15.43:0/1649038965 shutdown_connections 2024-09-11T23:32:02.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.038+0000 7f0e5f465640 1 --2- 172.21.15.43:0/1649038965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e58102460 0x7f0e5810e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.038+0000 7f0e5f465640 1 --2- 172.21.15.43:0/1649038965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e58101ac0 0x7f0e58101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.038+0000 7f0e5f465640 1 --2- 172.21.15.43:0/1649038965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e58107ad0 0x7f0e58107eb0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.038+0000 7f0e5f465640 1 -- 172.21.15.43:0/1649038965 >> 172.21.15.43:0/1649038965 conn(0x7f0e580fd7a0 msgr2=0x7f0e580ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:02.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.038+0000 7f0e5f465640 1 -- 172.21.15.43:0/1649038965 shutdown_connections 2024-09-11T23:32:02.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.038+0000 7f0e5f465640 1 -- 172.21.15.43:0/1649038965 wait complete. 2024-09-11T23:32:02.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.039+0000 7f0e5f465640 1 Processor -- start 2024-09-11T23:32:02.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.039+0000 7f0e5f465640 1 -- start start 2024-09-11T23:32:02.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.039+0000 7f0e5f465640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e58101ac0 0x7f0e5819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.039+0000 7f0e5f465640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e58102460 0x7f0e5819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.039+0000 7f0e5f465640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e58107ad0 0x7f0e5819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.039+0000 7f0e5f465640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0e58111ed0 con 0x7f0e58101ac0 2024-09-11T23:32:02.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.039+0000 7f0e5f465640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0e58111d50 con 0x7f0e58102460 2024-09-11T23:32:02.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.039+0000 7f0e5f465640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0e58112050 con 0x7f0e58107ad0 2024-09-11T23:32:02.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.039+0000 7f0e5d201640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e58101ac0 0x7f0e5819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.039+0000 7f0e5d201640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e58101ac0 0x7f0e5819a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38388/0 (socket says 172.21.15.43:38388) 2024-09-11T23:32:02.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.039+0000 7f0e5d201640 1 -- 172.21.15.43:0/4096611113 learned_addr learned my addr 172.21.15.43:0/4096611113 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:02.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.039+0000 7f0e5ca00640 1 --2- 172.21.15.43:0/4096611113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e58102460 0x7f0e5819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.039+0000 7f0e5da02640 1 --2- 172.21.15.43:0/4096611113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e58107ad0 0x7f0e5819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.039+0000 7f0e5d201640 1 -- 172.21.15.43:0/4096611113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e58107ad0 msgr2=0x7f0e5819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:02.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.039+0000 7f0e5d201640 1 --2- 172.21.15.43:0/4096611113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e58107ad0 0x7f0e5819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.039+0000 7f0e5d201640 1 -- 172.21.15.43:0/4096611113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e58102460 msgr2=0x7f0e5819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:02.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.039+0000 7f0e5d201640 1 --2- 172.21.15.43:0/4096611113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e58102460 0x7f0e5819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.040+0000 7f0e5d201640 1 -- 172.21.15.43:0/4096611113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5819f650 con 0x7f0e58101ac0 2024-09-11T23:32:02.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.040+0000 7f0e5da02640 1 --2- 172.21.15.43:0/4096611113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e58107ad0 0x7f0e5819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:32:02.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.040+0000 7f0e5d201640 1 --2- 172.21.15.43:0/4096611113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e58101ac0 0x7f0e5819a660 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009230 tx=0x7f0e54004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:02.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.040+0000 7f0e4e7fc640 1 -- 172.21.15.43:0/4096611113 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5403f030 con 0x7f0e58101ac0 2024-09-11T23:32:02.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.040+0000 7f0e5f465640 1 -- 172.21.15.43:0/4096611113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e5819f8e0 con 0x7f0e58101ac0 2024-09-11T23:32:02.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.040+0000 7f0e4e7fc640 1 -- 172.21.15.43:0/4096611113 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e54003ad0 con 0x7f0e58101ac0 2024-09-11T23:32:02.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.040+0000 7f0e4e7fc640 1 -- 172.21.15.43:0/4096611113 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54035070 con 0x7f0e58101ac0 2024-09-11T23:32:02.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.040+0000 7f0e5f465640 1 -- 172.21.15.43:0/4096611113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e581a71d0 con 0x7f0e58101ac0 2024-09-11T23:32:02.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.041+0000 7f0e5f465640 1 -- 172.21.15.43:0/4096611113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e58101f20 con 0x7f0e58101ac0 2024-09-11T23:32:02.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.042+0000 7f0e4e7fc640 1 -- 172.21.15.43:0/4096611113 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0e54042030 con 0x7f0e58101ac0 2024-09-11T23:32:02.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.043+0000 7f0e4e7fc640 1 --2- 172.21.15.43:0/4096611113 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f0e440a2cf0 0x7f0e440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.043+0000 7f0e4e7fc640 1 -- 172.21.15.43:0/4096611113 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f0e540efe90 con 0x7f0e58101ac0 2024-09-11T23:32:02.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.043+0000 7f0e5ca00640 1 --2- 172.21.15.43:0/4096611113 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f0e440a2cf0 0x7f0e440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.043+0000 7f0e5ca00640 1 --2- 172.21.15.43:0/4096611113 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f0e440a2cf0 0x7f0e440a51b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0e5810eb30 tx=0x7f0e4800b410 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:02.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.046+0000 7f0e4e7fc640 1 -- 172.21.15.43:0/4096611113 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0e540bca80 con 0x7f0e58101ac0 2024-09-11T23:32:02.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.214+0000 7f56dbe96640 1 -- 172.21.15.43:0/161345110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56d41997e0 con 0x7f56d4102ef0 2024-09-11T23:32:02.214 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:02 smithi043 ceph-mon[30950]: pgmap v315: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T23:32:02.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.215+0000 7f56d2ffd640 1 -- 172.21.15.43:0/161345110 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11893 (secure 0 0 0) 0x7f56d41997e0 con 0x7f56d4102ef0 2024-09-11T23:32:02.215 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:32:02.216 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":34,"btime":"2024-09-11T23:31:56:056494+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:31:55.493313+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25571":{"gid":25571,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/568491708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":568491708},{"type":"v1","addr":"172.21.15.90:6835","nonce":568491708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:32:02.216 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 34 2024-09-11T23:32:02.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.217+0000 7f56dbe96640 1 -- 172.21.15.43:0/161345110 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56bc0a2d70 msgr2=0x7f56bc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:02.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.217+0000 7f56dbe96640 1 --2- 172.21.15.43:0/161345110 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56bc0a2d70 0x7f56bc0a5230 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f56c4005fd0 tx=0x7f56c4009680 comp rx=0 tx=0).stop 2024-09-11T23:32:02.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.217+0000 7f56dbe96640 1 -- 172.21.15.43:0/161345110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56d4102ef0 msgr2=0x7f56d419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:02.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.217+0000 7f56dbe96640 1 --2- 172.21.15.43:0/161345110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56d4102ef0 0x7f56d419cd50 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f56cc00e970 tx=0x7f56cc00ed70 comp rx=0 tx=0).stop 2024-09-11T23:32:02.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.217+0000 7f56dbe96640 1 -- 172.21.15.43:0/161345110 shutdown_connections 2024-09-11T23:32:02.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.217+0000 7f56dbe96640 1 --2- 172.21.15.43:0/161345110 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f56bc0a2d70 0x7f56bc0a5230 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.217+0000 7f56dbe96640 1 --2- 172.21.15.43:0/161345110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56d4102ef0 0x7f56d419cd50 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.217+0000 7f56dbe96640 1 --2- 172.21.15.43:0/161345110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56d4102570 0x7f56d41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.217+0000 7f56dbe96640 1 --2- 172.21.15.43:0/161345110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56d4100e20 0x7f56d4198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.218+0000 7f56dbe96640 1 -- 172.21.15.43:0/161345110 >> 172.21.15.43:0/161345110 conn(0x7f56d40fc7d0 msgr2=0x7f56d406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:02.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.218+0000 7f56dbe96640 1 -- 172.21.15.43:0/161345110 shutdown_connections 2024-09-11T23:32:02.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.218+0000 7f56dbe96640 1 -- 172.21.15.43:0/161345110 wait complete. 2024-09-11T23:32:02.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.245+0000 7f0e5f465640 1 -- 172.21.15.43:0/4096611113 --> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f0e580fee50 con 0x7f0e440a2cf0 2024-09-11T23:32:02.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.253+0000 7f0e4e7fc640 1 -- 172.21.15.43:0/4096611113 <== mgr.15126 v2:172.21.15.43:6800/2510386404 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f0e580fee50 con 0x7f0e440a2cf0 2024-09-11T23:32:02.253 INFO:teuthology.orchestra.run.smithi043.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-11T00:00:00", "created": "2024-09-11T23:22:00", "first": "2024-09-11T23:23:00", "last": "2024-09-11T23:32:00", "last_pruned": "2024-09-11T23:32:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-09-11T23:32:02.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.255+0000 7f0e5f465640 1 -- 172.21.15.43:0/4096611113 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f0e440a2cf0 msgr2=0x7f0e440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:02.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.255+0000 7f0e5f465640 1 --2- 172.21.15.43:0/4096611113 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f0e440a2cf0 0x7f0e440a51b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0e5810eb30 tx=0x7f0e4800b410 comp rx=0 tx=0).stop 2024-09-11T23:32:02.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.255+0000 7f0e5f465640 1 -- 172.21.15.43:0/4096611113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e58101ac0 msgr2=0x7f0e5819a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:02.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.255+0000 7f0e5f465640 1 --2- 172.21.15.43:0/4096611113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e58101ac0 0x7f0e5819a660 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009230 tx=0x7f0e54004270 comp rx=0 tx=0).stop 2024-09-11T23:32:02.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.255+0000 7f0e5f465640 1 -- 172.21.15.43:0/4096611113 shutdown_connections 2024-09-11T23:32:02.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.255+0000 7f0e5f465640 1 --2- 172.21.15.43:0/4096611113 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f0e440a2cf0 0x7f0e440a51b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.255+0000 7f0e5f465640 1 --2- 172.21.15.43:0/4096611113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e58107ad0 0x7f0e5819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.255+0000 7f0e5f465640 1 --2- 172.21.15.43:0/4096611113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e58102460 0x7f0e5819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.255+0000 7f0e5f465640 1 --2- 172.21.15.43:0/4096611113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e58101ac0 0x7f0e5819a660 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.255+0000 7f0e5f465640 1 -- 172.21.15.43:0/4096611113 >> 172.21.15.43:0/4096611113 conn(0x7f0e580fd7a0 msgr2=0x7f0e5810d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:02.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.255+0000 7f0e5f465640 1 -- 172.21.15.43:0/4096611113 shutdown_connections 2024-09-11T23:32:02.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.255+0000 7f0e5f465640 1 -- 172.21.15.43:0/4096611113 wait complete. 2024-09-11T23:32:02.265 DEBUG:teuthology.orchestra.run.smithi043:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-11T23:32:02.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.371+0000 7f490fe1e640 1 -- 172.21.15.43:0/2804745215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4908103640 msgr2=0x7f490810bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:02.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.371+0000 7f490fe1e640 1 --2- 172.21.15.43:0/2804745215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4908103640 0x7f490810bb20 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009900 tx=0x7f48f802a480 comp rx=0 tx=0).stop 2024-09-11T23:32:02.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.371+0000 7f490fe1e640 1 -- 172.21.15.43:0/2804745215 shutdown_connections 2024-09-11T23:32:02.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.371+0000 7f490fe1e640 1 --2- 172.21.15.43:0/2804745215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f490810c0f0 0x7f490810e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.371+0000 7f490fe1e640 1 --2- 172.21.15.43:0/2804745215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4908103640 0x7f490810bb20 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.371+0000 7f490fe1e640 1 --2- 172.21.15.43:0/2804745215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4908102d20 0x7f4908103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.371+0000 7f490fe1e640 1 -- 172.21.15.43:0/2804745215 >> 172.21.15.43:0/2804745215 conn(0x7f49080779a0 msgr2=0x7f49080fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:02.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.371+0000 7f490fe1e640 1 -- 172.21.15.43:0/2804745215 shutdown_connections 2024-09-11T23:32:02.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.371+0000 7f490fe1e640 1 -- 172.21.15.43:0/2804745215 wait complete. 2024-09-11T23:32:02.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.372+0000 7f490fe1e640 1 Processor -- start 2024-09-11T23:32:02.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.372+0000 7f490fe1e640 1 -- start start 2024-09-11T23:32:02.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.372+0000 7f490fe1e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4908102d20 0x7f49081983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.372+0000 7f490fe1e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4908103640 0x7f4908198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.372+0000 7f490fe1e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f490810c0f0 0x7f490819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.372+0000 7f490fe1e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4908111ea0 con 0x7f490810c0f0 2024-09-11T23:32:02.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.372+0000 7f490fe1e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4908111d20 con 0x7f4908103640 2024-09-11T23:32:02.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.372+0000 7f490fe1e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4908112020 con 0x7f4908102d20 2024-09-11T23:32:02.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.372+0000 7f490e3bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f490810c0f0 0x7f490819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.372+0000 7f490e3bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f490810c0f0 0x7f490819cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38404/0 (socket says 172.21.15.43:38404) 2024-09-11T23:32:02.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.372+0000 7f490e3bb640 1 -- 172.21.15.43:0/1382866040 learned_addr learned my addr 172.21.15.43:0/1382866040 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:02.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.372+0000 7f490d3b9640 1 --2- 172.21.15.43:0/1382866040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4908103640 0x7f4908198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.372+0000 7f490dbba640 1 --2- 172.21.15.43:0/1382866040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4908102d20 0x7f49081983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.372+0000 7f490e3bb640 1 -- 172.21.15.43:0/1382866040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4908102d20 msgr2=0x7f49081983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:02.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.372+0000 7f490e3bb640 1 --2- 172.21.15.43:0/1382866040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4908102d20 0x7f49081983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.372+0000 7f490e3bb640 1 -- 172.21.15.43:0/1382866040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4908103640 msgr2=0x7f4908198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:02.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.372+0000 7f490e3bb640 1 --2- 172.21.15.43:0/1382866040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4908103640 0x7f4908198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.372+0000 7f490e3bb640 1 -- 172.21.15.43:0/1382866040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490819d3b0 con 0x7f490810c0f0 2024-09-11T23:32:02.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.372+0000 7f490e3bb640 1 --2- 172.21.15.43:0/1382866040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f490810c0f0 0x7f490819cc80 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f490000cce0 tx=0x7f490000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:02.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.373+0000 7f48feffd640 1 -- 172.21.15.43:0/1382866040 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490000f450 con 0x7f490810c0f0 2024-09-11T23:32:02.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.373+0000 7f48feffd640 1 -- 172.21.15.43:0/1382866040 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490000fad0 con 0x7f490810c0f0 2024-09-11T23:32:02.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.373+0000 7f490fe1e640 1 -- 172.21.15.43:0/1382866040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490819d6a0 con 0x7f490810c0f0 2024-09-11T23:32:02.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.373+0000 7f48feffd640 1 -- 172.21.15.43:0/1382866040 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900018c10 con 0x7f490810c0f0 2024-09-11T23:32:02.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.373+0000 7f490fe1e640 1 -- 172.21.15.43:0/1382866040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49081a4f30 con 0x7f490810c0f0 2024-09-11T23:32:02.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.374+0000 7f490fe1e640 1 -- 172.21.15.43:0/1382866040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4908103100 con 0x7f490810c0f0 2024-09-11T23:32:02.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.375+0000 7f48feffd640 1 -- 172.21.15.43:0/1382866040 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f490000f5f0 con 0x7f490810c0f0 2024-09-11T23:32:02.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.375+0000 7f48feffd640 1 --2- 172.21.15.43:0/1382866040 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f48f40a2cf0 0x7f48f40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:02.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.376+0000 7f490dbba640 1 --2- 172.21.15.43:0/1382866040 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f48f40a2cf0 0x7f48f40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:02.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.376+0000 7f48feffd640 1 -- 172.21.15.43:0/1382866040 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f49000cfbd0 con 0x7f490810c0f0 2024-09-11T23:32:02.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.376+0000 7f490dbba640 1 --2- 172.21.15.43:0/1382866040 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f48f40a2cf0 0x7f48f40a51b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4904005fd0 tx=0x7f4904005e20 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:02.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.379+0000 7f48feffd640 1 -- 172.21.15.43:0/1382866040 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4900014030 con 0x7f490810c0f0 2024-09-11T23:32:02.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:02 smithi090 ceph-mon[35130]: pgmap v315: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T23:32:02.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:02 smithi097 ceph-mon[35562]: pgmap v315: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T23:32:02.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.579+0000 7f490fe1e640 1 -- 172.21.15.43:0/1382866040 --> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f4908062630 con 0x7f48f40a2cf0 2024-09-11T23:32:02.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.586+0000 7f48feffd640 1 -- 172.21.15.43:0/1382866040 <== mgr.15126 v2:172.21.15.43:6800/2510386404 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f4908062630 con 0x7f48f40a2cf0 2024-09-11T23:32:02.586 INFO:teuthology.orchestra.run.smithi043.stdout:/ 1m 6m3h 2024-09-11T23:32:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.588+0000 7f490fe1e640 1 -- 172.21.15.43:0/1382866040 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f48f40a2cf0 msgr2=0x7f48f40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.588+0000 7f490fe1e640 1 --2- 172.21.15.43:0/1382866040 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f48f40a2cf0 0x7f48f40a51b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4904005fd0 tx=0x7f4904005e20 comp rx=0 tx=0).stop 2024-09-11T23:32:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.588+0000 7f490fe1e640 1 -- 172.21.15.43:0/1382866040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f490810c0f0 msgr2=0x7f490819cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.588+0000 7f490fe1e640 1 --2- 172.21.15.43:0/1382866040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f490810c0f0 0x7f490819cc80 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f490000cce0 tx=0x7f490000a620 comp rx=0 tx=0).stop 2024-09-11T23:32:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.588+0000 7f490fe1e640 1 -- 172.21.15.43:0/1382866040 shutdown_connections 2024-09-11T23:32:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.588+0000 7f490fe1e640 1 --2- 172.21.15.43:0/1382866040 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f48f40a2cf0 0x7f48f40a51b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.588+0000 7f490fe1e640 1 --2- 172.21.15.43:0/1382866040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f490810c0f0 0x7f490819cc80 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.588+0000 7f490fe1e640 1 --2- 172.21.15.43:0/1382866040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4908103640 0x7f4908198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.588+0000 7f490fe1e640 1 --2- 172.21.15.43:0/1382866040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4908102d20 0x7f49081983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.588+0000 7f490fe1e640 1 -- 172.21.15.43:0/1382866040 >> 172.21.15.43:0/1382866040 conn(0x7f49080779a0 msgr2=0x7f49080fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.589+0000 7f490fe1e640 1 -- 172.21.15.43:0/1382866040 shutdown_connections 2024-09-11T23:32:02.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:02.589+0000 7f490fe1e640 1 -- 172.21.15.43:0/1382866040 wait complete. 2024-09-11T23:32:02.599 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-11T23:32:02.609 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-11T23:32:02.853 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:32:03.108 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:32:03.343 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:03 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/161345110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:32:03.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:03 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/161345110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:32:03.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:03 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/161345110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:32:03.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.560+0000 7f202e327640 1 -- 172.21.15.43:0/4227274442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2028101420 msgr2=0x7f202810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:03.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.560+0000 7f202e327640 1 --2- 172.21.15.43:0/4227274442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2028101420 0x7f202810d950 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f2020009cd0 tx=0x7f202002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.560+0000 7f202e327640 1 -- 172.21.15.43:0/4227274442 shutdown_connections 2024-09-11T23:32:03.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.560+0000 7f202e327640 1 --2- 172.21.15.43:0/4227274442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2028101420 0x7f202810d950 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.560+0000 7f202e327640 1 --2- 172.21.15.43:0/4227274442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2028100a80 0x7f2028100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.560+0000 7f202e327640 1 --2- 172.21.15.43:0/4227274442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2028106a80 0x7f2028106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.560+0000 7f202e327640 1 -- 172.21.15.43:0/4227274442 >> 172.21.15.43:0/4227274442 conn(0x7f20280fc7a0 msgr2=0x7f20280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:03.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.560+0000 7f202e327640 1 -- 172.21.15.43:0/4227274442 shutdown_connections 2024-09-11T23:32:03.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.560+0000 7f202e327640 1 -- 172.21.15.43:0/4227274442 wait complete. 2024-09-11T23:32:03.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.560+0000 7f202e327640 1 Processor -- start 2024-09-11T23:32:03.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.560+0000 7f202e327640 1 -- start start 2024-09-11T23:32:03.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.561+0000 7f202e327640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2028100a80 0x7f20281983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:03.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.561+0000 7f202e327640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2028101420 0x7f2028198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:03.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.561+0000 7f202e327640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2028106a80 0x7f202819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:03.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.561+0000 7f202e327640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2028110e80 con 0x7f2028106a80 2024-09-11T23:32:03.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.561+0000 7f202e327640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2028110d00 con 0x7f2028101420 2024-09-11T23:32:03.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.561+0000 7f202e327640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2028111000 con 0x7f2028100a80 2024-09-11T23:32:03.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.561+0000 7f202c8c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2028106a80 0x7f202819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:03.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.561+0000 7f2027fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2028100a80 0x7f20281983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:03.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.561+0000 7f202c8c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2028106a80 0x7f202819ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38426/0 (socket says 172.21.15.43:38426) 2024-09-11T23:32:03.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.561+0000 7f202c8c4640 1 -- 172.21.15.43:0/1571567219 learned_addr learned my addr 172.21.15.43:0/1571567219 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:03.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.561+0000 7f20277fe640 1 --2- 172.21.15.43:0/1571567219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2028101420 0x7f2028198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:03.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.561+0000 7f202c8c4640 1 -- 172.21.15.43:0/1571567219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2028100a80 msgr2=0x7f20281983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:03.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.561+0000 7f202c8c4640 1 --2- 172.21.15.43:0/1571567219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2028100a80 0x7f20281983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.561+0000 7f202c8c4640 1 -- 172.21.15.43:0/1571567219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2028101420 msgr2=0x7f2028198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:03.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.561+0000 7f202c8c4640 1 --2- 172.21.15.43:0/1571567219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2028101420 0x7f2028198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.561+0000 7f202c8c4640 1 -- 172.21.15.43:0/1571567219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202819d3a0 con 0x7f2028106a80 2024-09-11T23:32:03.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.561+0000 7f202c8c4640 1 --2- 172.21.15.43:0/1571567219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2028106a80 0x7f202819ccc0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2020005cc0 tx=0x7f2020003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:03.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.562+0000 7f20257fa640 1 -- 172.21.15.43:0/1571567219 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020004040 con 0x7f2028106a80 2024-09-11T23:32:03.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.562+0000 7f202e327640 1 -- 172.21.15.43:0/1571567219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f202819d630 con 0x7f2028106a80 2024-09-11T23:32:03.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.562+0000 7f20257fa640 1 -- 172.21.15.43:0/1571567219 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20200046c0 con 0x7f2028106a80 2024-09-11T23:32:03.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.562+0000 7f20257fa640 1 -- 172.21.15.43:0/1571567219 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020046480 con 0x7f2028106a80 2024-09-11T23:32:03.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.562+0000 7f202e327640 1 -- 172.21.15.43:0/1571567219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20281a4f70 con 0x7f2028106a80 2024-09-11T23:32:03.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.564+0000 7f20257fa640 1 -- 172.21.15.43:0/1571567219 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f20200305a0 con 0x7f2028106a80 2024-09-11T23:32:03.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.564+0000 7f2030700740 1 Processor -- start 2024-09-11T23:32:03.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.564+0000 7f2030700740 1 -- start start 2024-09-11T23:32:03.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.565+0000 7f2030700740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56376000cb40 0x56376000b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:03.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.565+0000 7f2030700740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56376000b5d0 0x56376000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:03.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.565+0000 7f2030700740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563760016370 0x563760018760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:03.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.565+0000 7f2030700740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56375ff712f0 con 0x563760016370 2024-09-11T23:32:03.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.565+0000 7f2030700740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56375ff18cf0 con 0x56376000b5d0 2024-09-11T23:32:03.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.565+0000 7f2030700740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56375ff6d2b0 con 0x56376000cb40 2024-09-11T23:32:03.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.565+0000 7f20277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563760016370 0x563760018760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:03.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.565+0000 7f20277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563760016370 0x563760018760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38438/0 (socket says 172.21.15.43:38438) 2024-09-11T23:32:03.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.565+0000 7f20277fe640 1 -- 172.21.15.43:0/4082350553 learned_addr learned my addr 172.21.15.43:0/4082350553 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:03.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.565+0000 7f20257fa640 1 --2- 172.21.15.43:0/1571567219 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f20100a2cf0 0x7f20100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:03.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.565+0000 7f20277fe640 1 --2- 172.21.15.43:0/4082350553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56376000cb40 0x56376000b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:03.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.565+0000 7f2027fff640 1 --2- 172.21.15.43:0/4082350553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56376000b5d0 0x56376000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:03.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.565+0000 7f202c8c4640 1 --2- 172.21.15.43:0/1571567219 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f20100a2cf0 0x7f20100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:03.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.565+0000 7f20277fe640 1 -- 172.21.15.43:0/4082350553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56376000cb40 msgr2=0x56376000b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:03.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.565+0000 7f20277fe640 1 --2- 172.21.15.43:0/4082350553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56376000cb40 0x56376000b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.565+0000 7f20277fe640 1 -- 172.21.15.43:0/4082350553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56376000b5d0 msgr2=0x56376000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:03.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.565+0000 7f20277fe640 1 --2- 172.21.15.43:0/4082350553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56376000b5d0 0x56376000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.565+0000 7f20277fe640 1 -- 172.21.15.43:0/4082350553 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56375fefd3a0 con 0x563760016370 2024-09-11T23:32:03.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.565+0000 7f20257fa640 1 -- 172.21.15.43:0/1571567219 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f202002e030 con 0x7f2028106a80 2024-09-11T23:32:03.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.565+0000 7f20277fe640 1 --2- 172.21.15.43:0/4082350553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563760016370 0x563760018760 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f201c011490 tx=0x7f201c00a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d688ce0a05152dd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:03.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.566+0000 7f200e7fc640 1 -- 172.21.15.43:0/4082350553 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c005220 con 0x563760016370 2024-09-11T23:32:03.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.566+0000 7f202c8c4640 1 --2- 172.21.15.43:0/1571567219 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f20100a2cf0 0x7f20100a51b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f20200b9ac0 tx=0x7f20200c1040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:03.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.566+0000 7f200e7fc640 1 -- 172.21.15.43:0/4082350553 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f201c004c80 con 0x563760016370 2024-09-11T23:32:03.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.566+0000 7f200e7fc640 1 -- 172.21.15.43:0/4082350553 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c00fc80 con 0x563760016370 2024-09-11T23:32:03.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.566+0000 7f2030700740 1 -- 172.21.15.43:0/4082350553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563760016370 msgr2=0x563760018760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:03.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.566+0000 7f2030700740 1 --2- 172.21.15.43:0/4082350553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563760016370 0x563760018760 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f201c011490 tx=0x7f201c00a750 comp rx=0 tx=0).stop 2024-09-11T23:32:03.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.566+0000 7f2030700740 1 -- 172.21.15.43:0/4082350553 shutdown_connections 2024-09-11T23:32:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.566+0000 7f2030700740 1 --2- 172.21.15.43:0/4082350553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563760016370 0x563760018760 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.566+0000 7f2030700740 1 --2- 172.21.15.43:0/4082350553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56376000b5d0 0x56376000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.566+0000 7f2030700740 1 --2- 172.21.15.43:0/4082350553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56376000cb40 0x56376000b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.566+0000 7f2030700740 1 -- 172.21.15.43:0/4082350553 >> 172.21.15.43:0/4082350553 conn(0x56375ffd4310 msgr2=0x563760007c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.567+0000 7f2030700740 1 -- 172.21.15.43:0/4082350553 shutdown_connections 2024-09-11T23:32:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.567+0000 7f2030700740 1 -- 172.21.15.43:0/4082350553 wait complete. 2024-09-11T23:32:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.567+0000 7f2030700740 1 -- start start 2024-09-11T23:32:03.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.567+0000 7f2030700740 1 Processor -- start 2024-09-11T23:32:03.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.567+0000 7f2030700740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56376000b5d0 0x563760015dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:03.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.567+0000 7f2030700740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56376000cb40 0x563760013070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:03.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.567+0000 7f2030700740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563760016370 0x563760013800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:03.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.567+0000 7f2030700740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563760019070 con 0x56376000b5d0 2024-09-11T23:32:03.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.567+0000 7f2030700740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56375ff2c060 con 0x56376000cb40 2024-09-11T23:32:03.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.567+0000 7f2030700740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5637600191f0 con 0x563760016370 2024-09-11T23:32:03.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.567+0000 7f2030700740 10 client.0 _ll_register_callbacks cb 0x56375fc7f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:32:03.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.568+0000 7f20277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56376000b5d0 0x563760015dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:03.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.568+0000 7f20277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56376000b5d0 0x563760015dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38450/0 (socket says 172.21.15.43:38450) 2024-09-11T23:32:03.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.568+0000 7f20277fe640 1 -- 172.21.15.43:0/1386890259 learned_addr learned my addr 172.21.15.43:0/1386890259 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:03.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.568+0000 7f2027fff640 1 --2- 172.21.15.43:0/1386890259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56376000cb40 0x563760013070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.568+0000 7f20277fe640 1 -- 172.21.15.43:0/1386890259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563760016370 msgr2=0x563760013800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.568+0000 7f20277fe640 1 --2- 172.21.15.43:0/1386890259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563760016370 0x563760013800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.568+0000 7f20277fe640 1 -- 172.21.15.43:0/1386890259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56376000cb40 msgr2=0x563760013070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.568+0000 7f20277fe640 1 --2- 172.21.15.43:0/1386890259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56376000cb40 0x563760013070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.568+0000 7f20277fe640 1 -- 172.21.15.43:0/1386890259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56375fce23c0 con 0x56376000b5d0 2024-09-11T23:32:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.568+0000 7f2027fff640 1 --2- 172.21.15.43:0/1386890259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56376000cb40 0x563760013070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:32:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.568+0000 7f20277fe640 1 --2- 172.21.15.43:0/1386890259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56376000b5d0 0x563760015dc0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f201c00b010 tx=0x7f201c005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e04c874fff82e0a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.568+0000 7f20077fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:32:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.568+0000 7f20077fe640 1 -- 172.21.15.43:0/1386890259 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c00a350 con 0x56376000b5d0 2024-09-11T23:32:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.568+0000 7f20077fe640 1 -- 172.21.15.43:0/1386890259 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f201c005000 con 0x56376000b5d0 2024-09-11T23:32:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.568+0000 7f2030700740 1 -- 172.21.15.43:0/1386890259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f201c004c80 con 0x56376000b5d0 2024-09-11T23:32:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.569+0000 7f20077fe640 1 -- 172.21.15.43:0/1386890259 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f201c005200 con 0x56376000b5d0 2024-09-11T23:32:03.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.569+0000 7f2030700740 10 client.17034 fetch_fsmap learned FSMap version 34 2024-09-11T23:32:03.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.569+0000 7f2030700740 1 -- 172.21.15.43:0/1386890259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563760024b80 con 0x56376000b5d0 2024-09-11T23:32:03.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.569+0000 7f20077fe640 1 -- 172.21.15.43:0/1386890259 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f201c004c80 con 0x56376000b5d0 2024-09-11T23:32:03.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.569+0000 7f20077fe640 1 -- 172.21.15.43:0/1386890259 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f201c01b030 con 0x56376000b5d0 2024-09-11T23:32:03.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.569+0000 7f2030700740 10 client.17034 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T23:32:03.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.569+0000 7f2030700740 10 client.17034 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T23:32:03.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.569+0000 7f2030700740 20 client.17034 populate_metadata read hostname 'smithi043' 2024-09-11T23:32:03.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.569+0000 7f2030700740 1 --2- 172.21.15.43:0/1386890259 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x563760025fe0 0x5637600283d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:03.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.569+0000 7f2030700740 4 client.17034 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:32:03.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.569+0000 7f2030700740 1 -- 172.21.15.43:0/1386890259 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56375ff7e860 con 0x563760025fe0 2024-09-11T23:32:03.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.569+0000 7f2027fff640 1 --2- 172.21.15.43:0/1386890259 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x563760025fe0 0x5637600283d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:03.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.570+0000 7f2027fff640 1 --2- 172.21.15.43:0/1386890259 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x563760025fe0 0x5637600283d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be32791a4e926172 server_cookie=49fe61128f37a4f in_seq=0 out_seq=0 2024-09-11T23:32:03.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.570+0000 7f20077fe640 10 client.17034 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:32:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.570+0000 7f20077fe640 1 -- 172.21.15.43:0/1386890259 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56375ff7e860 con 0x563760025fe0 2024-09-11T23:32:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.570+0000 7f20077fe640 10 client.17034 handle_command_reply: tid=0 2024-09-11T23:32:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.570+0000 7f2030700740 1 client.17034 shutdown 2024-09-11T23:32:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.571+0000 7f2030700740 1 -- 172.21.15.43:0/1386890259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56376000b5d0 msgr2=0x563760015dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.571+0000 7f2030700740 1 --2- 172.21.15.43:0/1386890259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56376000b5d0 0x563760015dc0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f201c00b010 tx=0x7f201c005a50 comp rx=0 tx=0).stop 2024-09-11T23:32:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.571+0000 7f2030700740 1 -- 172.21.15.43:0/1386890259 shutdown_connections 2024-09-11T23:32:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.571+0000 7f2030700740 1 --2- 172.21.15.43:0/1386890259 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x563760025fe0 0x5637600283d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.571+0000 7f2030700740 1 --2- 172.21.15.43:0/1386890259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563760016370 0x563760013800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.571+0000 7f2030700740 1 --2- 172.21.15.43:0/1386890259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56376000cb40 0x563760013070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.571+0000 7f2030700740 1 --2- 172.21.15.43:0/1386890259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56376000b5d0 0x563760015dc0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.571+0000 7f2030700740 1 -- 172.21.15.43:0/1386890259 >> 172.21.15.43:0/1386890259 conn(0x56375ffd4310 msgr2=0x56376000d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.571+0000 7f20077fe640 0 client.17034 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:32:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.571+0000 7f2030700740 1 -- 172.21.15.43:0/1386890259 shutdown_connections 2024-09-11T23:32:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.571+0000 7f2030700740 1 -- 172.21.15.43:0/1386890259 wait complete. 2024-09-11T23:32:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.571+0000 7f2030700740 20 client.17034 trim_cache size 0 max 16384 2024-09-11T23:32:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.584+0000 7f2030700740 1 Processor -- start 2024-09-11T23:32:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.584+0000 7f2030700740 1 -- start start 2024-09-11T23:32:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.584+0000 7f2030700740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56376000b5d0 0x563760012930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.584+0000 7f2030700740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56376000cb40 0x56376000d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.584+0000 7f2030700740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563760016370 0x56376000dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.584+0000 7f2030700740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20281b0170 con 0x563760016370 2024-09-11T23:32:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.584+0000 7f2030700740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f20281ada40 con 0x56376000b5d0 2024-09-11T23:32:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.584+0000 7f2030700740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f20281ab8b0 con 0x56376000cb40 2024-09-11T23:32:03.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.584+0000 7f20277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563760016370 0x56376000dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:03.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.584+0000 7f20277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563760016370 0x56376000dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38464/0 (socket says 172.21.15.43:38464) 2024-09-11T23:32:03.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.584+0000 7f20277fe640 1 -- 172.21.15.43:0/3723805616 learned_addr learned my addr 172.21.15.43:0/3723805616 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:03.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.584+0000 7f20277fe640 1 --2- 172.21.15.43:0/3723805616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56376000b5d0 0x563760012930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:03.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.585+0000 7f2027fff640 1 --2- 172.21.15.43:0/3723805616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56376000cb40 0x56376000d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:03.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.585+0000 7f20277fe640 1 -- 172.21.15.43:0/3723805616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56376000cb40 msgr2=0x56376000d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:03.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.585+0000 7f20277fe640 1 --2- 172.21.15.43:0/3723805616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56376000cb40 0x56376000d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.585+0000 7f20277fe640 1 -- 172.21.15.43:0/3723805616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56376000b5d0 msgr2=0x563760012930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:03.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.585+0000 7f20277fe640 1 --2- 172.21.15.43:0/3723805616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56376000b5d0 0x563760012930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.585+0000 7f20277fe640 1 -- 172.21.15.43:0/3723805616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563760012020 con 0x563760016370 2024-09-11T23:32:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.585+0000 7f2027fff640 1 --2- 172.21.15.43:0/3723805616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56376000cb40 0x56376000d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:32:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.585+0000 7f20277fe640 1 --2- 172.21.15.43:0/3723805616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563760016370 0x56376000dd60 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f201c0058c0 tx=0x7f201c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0029b7fd65686cc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.585+0000 7f200dffb640 1 -- 172.21.15.43:0/3723805616 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c005530 con 0x563760016370 2024-09-11T23:32:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.585+0000 7f200dffb640 1 -- 172.21.15.43:0/3723805616 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f201c023070 con 0x563760016370 2024-09-11T23:32:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.585+0000 7f200dffb640 1 -- 172.21.15.43:0/3723805616 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c0277a0 con 0x563760016370 2024-09-11T23:32:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.586+0000 7f2030700740 1 -- 172.21.15.43:0/3723805616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563760016370 msgr2=0x56376000dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.586+0000 7f2030700740 1 --2- 172.21.15.43:0/3723805616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563760016370 0x56376000dd60 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f201c0058c0 tx=0x7f201c00f690 comp rx=0 tx=0).stop 2024-09-11T23:32:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.586+0000 7f2030700740 1 -- 172.21.15.43:0/3723805616 shutdown_connections 2024-09-11T23:32:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.586+0000 7f2030700740 1 --2- 172.21.15.43:0/3723805616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563760016370 0x56376000dd60 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.586+0000 7f2030700740 1 --2- 172.21.15.43:0/3723805616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56376000cb40 0x56376000d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.586+0000 7f2030700740 1 --2- 172.21.15.43:0/3723805616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56376000b5d0 0x563760012930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.586+0000 7f2030700740 1 -- 172.21.15.43:0/3723805616 >> 172.21.15.43:0/3723805616 conn(0x56375ffd4310 msgr2=0x56376000d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.586+0000 7f2030700740 1 -- 172.21.15.43:0/3723805616 shutdown_connections 2024-09-11T23:32:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.586+0000 7f2030700740 1 -- 172.21.15.43:0/3723805616 wait complete. 2024-09-11T23:32:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.586+0000 7f2030700740 1 -- start start 2024-09-11T23:32:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.586+0000 7f2030700740 1 Processor -- start 2024-09-11T23:32:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.587+0000 7f2030700740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56376000b5d0 0x56376001e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.587+0000 7f2030700740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56376000cb40 0x56376001eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.587+0000 7f2030700740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563760016370 0x56376000f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.587+0000 7f2030700740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f20281abf00 con 0x56376000b5d0 2024-09-11T23:32:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.587+0000 7f2030700740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f20281ad370 con 0x563760016370 2024-09-11T23:32:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.587+0000 7f2030700740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f20281ac5d0 con 0x56376000cb40 2024-09-11T23:32:03.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.587+0000 7f2030700740 10 client.0 _ll_register_callbacks cb 0x7f201c00af20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:32:03.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.587+0000 7f20277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56376000b5d0 0x56376001e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:03.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.587+0000 7f20277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56376000b5d0 0x56376001e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38470/0 (socket says 172.21.15.43:38470) 2024-09-11T23:32:03.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.587+0000 7f20277fe640 1 -- 172.21.15.43:0/4252268720 learned_addr learned my addr 172.21.15.43:0/4252268720 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:03.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.587+0000 7f2027fff640 1 --2- 172.21.15.43:0/4252268720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56376000cb40 0x56376001eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:03.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.587+0000 7f20277fe640 1 -- 172.21.15.43:0/4252268720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56376000cb40 msgr2=0x56376001eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:03.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.587+0000 7f20277fe640 1 --2- 172.21.15.43:0/4252268720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56376000cb40 0x56376001eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.587+0000 7f20277fe640 1 -- 172.21.15.43:0/4252268720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563760016370 msgr2=0x56376000f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:03.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.587+0000 7f20277fe640 1 --2- 172.21.15.43:0/4252268720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563760016370 0x56376000f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.587+0000 7f20277fe640 1 -- 172.21.15.43:0/4252268720 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637600100b0 con 0x56376000b5d0 2024-09-11T23:32:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.587+0000 7f20277fe640 1 --2- 172.21.15.43:0/4252268720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56376000b5d0 0x56376001e660 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f201c00a810 tx=0x7f201c027c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcedca16cb28fea2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.587+0000 7f20077fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:32:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.587+0000 7f20077fe640 1 -- 172.21.15.43:0/4252268720 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c049060 con 0x56376000b5d0 2024-09-11T23:32:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.587+0000 7f2030700740 1 -- 172.21.15.43:0/4252268720 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f201c023070 con 0x56376000b5d0 2024-09-11T23:32:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.587+0000 7f20077fe640 1 -- 172.21.15.43:0/4252268720 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f201c005460 con 0x56376000b5d0 2024-09-11T23:32:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.588+0000 7f20077fe640 1 -- 172.21.15.43:0/4252268720 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f201c027410 con 0x56376000b5d0 2024-09-11T23:32:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.588+0000 7f2030700740 10 client.17040 fetch_fsmap learned FSMap version 34 2024-09-11T23:32:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.588+0000 7f2030700740 1 -- 172.21.15.43:0/4252268720 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5637600905f0 con 0x56376000b5d0 2024-09-11T23:32:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.588+0000 7f20077fe640 1 -- 172.21.15.43:0/4252268720 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f201c023070 con 0x56376000b5d0 2024-09-11T23:32:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.588+0000 7f20077fe640 1 -- 172.21.15.43:0/4252268720 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f201c01b040 con 0x56376000b5d0 2024-09-11T23:32:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.588+0000 7f2030700740 10 client.17040 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T23:32:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.588+0000 7f2030700740 10 client.17040 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T23:32:03.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.588+0000 7f2030700740 20 client.17040 populate_metadata read hostname 'smithi043' 2024-09-11T23:32:03.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.588+0000 7f2030700740 1 --2- 172.21.15.43:0/4252268720 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x563760090ca0 0x5637600930c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:03.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.588+0000 7f2030700740 4 client.17040 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:32:03.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.588+0000 7f2030700740 1 -- 172.21.15.43:0/4252268720 --> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56375ff78d70 con 0x563760090ca0 2024-09-11T23:32:03.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.588+0000 7f2027fff640 1 --2- 172.21.15.43:0/4252268720 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x563760090ca0 0x5637600930c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:03.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.589+0000 7f2027fff640 1 --2- 172.21.15.43:0/4252268720 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x563760090ca0 0x5637600930c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7503c9c4c390816 server_cookie=9879956eca9f6708 in_seq=0 out_seq=0 2024-09-11T23:32:03.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.589+0000 7f20077fe640 10 client.17040 ms_handle_connect on v2:172.21.15.97:6838/1424670024 2024-09-11T23:32:03.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.589+0000 7f20077fe640 1 -- 172.21.15.43:0/4252268720 <== mds.2 v2:172.21.15.97:6838/1424670024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56375ff78d70 con 0x563760090ca0 2024-09-11T23:32:03.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.589+0000 7f20077fe640 10 client.17040 handle_command_reply: tid=0 2024-09-11T23:32:03.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.589+0000 7f2030700740 1 client.17040 shutdown 2024-09-11T23:32:03.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.589+0000 7f2030700740 1 -- 172.21.15.43:0/4252268720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56376000b5d0 msgr2=0x56376001e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:03.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.589+0000 7f2030700740 1 --2- 172.21.15.43:0/4252268720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56376000b5d0 0x56376001e660 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f201c00a810 tx=0x7f201c027c70 comp rx=0 tx=0).stop 2024-09-11T23:32:03.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.589+0000 7f2030700740 1 -- 172.21.15.43:0/4252268720 shutdown_connections 2024-09-11T23:32:03.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.589+0000 7f2030700740 1 --2- 172.21.15.43:0/4252268720 >> [v2:172.21.15.97:6838/1424670024,v1:172.21.15.97:6839/1424670024] conn(0x563760090ca0 0x5637600930c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.589+0000 7f20077fe640 0 client.17040 ms_handle_reset on v2:172.21.15.97:6838/1424670024 2024-09-11T23:32:03.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.589+0000 7f2030700740 1 --2- 172.21.15.43:0/4252268720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563760016370 0x56376000f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.589+0000 7f2030700740 1 --2- 172.21.15.43:0/4252268720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56376000cb40 0x56376001eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.589+0000 7f2030700740 1 --2- 172.21.15.43:0/4252268720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56376000b5d0 0x56376001e660 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.589+0000 7f2030700740 1 -- 172.21.15.43:0/4252268720 >> 172.21.15.43:0/4252268720 conn(0x56375ffd4310 msgr2=0x56376000d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:03.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.589+0000 7f2030700740 1 -- 172.21.15.43:0/4252268720 shutdown_connections 2024-09-11T23:32:03.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.589+0000 7f2030700740 1 -- 172.21.15.43:0/4252268720 wait complete. 2024-09-11T23:32:03.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.589+0000 7f2030700740 20 client.17040 trim_cache size 0 max 16384 2024-09-11T23:32:03.630 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:32:03.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.590+0000 7f202e327640 1 -- 172.21.15.43:0/1571567219 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f20100a2cf0 msgr2=0x7f20100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:03.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.590+0000 7f202e327640 1 --2- 172.21.15.43:0/1571567219 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f20100a2cf0 0x7f20100a51b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f20200b9ac0 tx=0x7f20200c1040 comp rx=0 tx=0).stop 2024-09-11T23:32:03.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.590+0000 7f202e327640 1 -- 172.21.15.43:0/1571567219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2028106a80 msgr2=0x7f202819ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:03.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.590+0000 7f202e327640 1 --2- 172.21.15.43:0/1571567219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2028106a80 0x7f202819ccc0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2020005cc0 tx=0x7f2020003970 comp rx=0 tx=0).stop 2024-09-11T23:32:03.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.590+0000 7f202e327640 1 -- 172.21.15.43:0/1571567219 shutdown_connections 2024-09-11T23:32:03.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.590+0000 7f202e327640 1 --2- 172.21.15.43:0/1571567219 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f20100a2cf0 0x7f20100a51b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.590+0000 7f202e327640 1 --2- 172.21.15.43:0/1571567219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2028106a80 0x7f202819ccc0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.590+0000 7f202e327640 1 --2- 172.21.15.43:0/1571567219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2028101420 0x7f2028198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.590+0000 7f202e327640 1 --2- 172.21.15.43:0/1571567219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2028100a80 0x7f20281983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:03.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.590+0000 7f202e327640 1 -- 172.21.15.43:0/1571567219 >> 172.21.15.43:0/1571567219 conn(0x7f20280fc7a0 msgr2=0x7f2028104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:03.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.590+0000 7f202e327640 1 -- 172.21.15.43:0/1571567219 shutdown_connections 2024-09-11T23:32:03.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:03.590+0000 7f202e327640 1 -- 172.21.15.43:0/1571567219 wait complete. 2024-09-11T23:32:04.349 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:04 smithi043 ceph-mon[30950]: from='client.17016 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:32:04.350 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:04 smithi043 ceph-mon[30950]: pgmap v316: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:32:04.350 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:04 smithi043 ceph-mon[30950]: from='client.17022 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:32:04.380 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:32:04.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:04 smithi090 ceph-mon[35130]: from='client.17016 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:32:04.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:04 smithi090 ceph-mon[35130]: pgmap v316: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:32:04.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:04 smithi090 ceph-mon[35130]: from='client.17022 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:32:04.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:04 smithi097 ceph-mon[35562]: from='client.17016 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:32:04.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:04 smithi097 ceph-mon[35562]: pgmap v316: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T23:32:04.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:04 smithi097 ceph-mon[35562]: from='client.17022 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:32:04.638 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:32:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.123+0000 7fadfa39e640 1 -- 172.21.15.43:0/1236805622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf4100e20 msgr2=0x7fadf4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.123+0000 7fadfa39e640 1 --2- 172.21.15.43:0/1236805622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf4100e20 0x7fadf4101200 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fade4009900 tx=0x7fade402a480 comp rx=0 tx=0).stop 2024-09-11T23:32:05.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.123+0000 7fadfa39e640 1 -- 172.21.15.43:0/1236805622 shutdown_connections 2024-09-11T23:32:05.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.123+0000 7fadfa39e640 1 --2- 172.21.15.43:0/1236805622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadf4102ef0 0x7fadf410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.123+0000 7fadfa39e640 1 --2- 172.21.15.43:0/1236805622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadf4102570 0x7fadf41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.123+0000 7fadfa39e640 1 --2- 172.21.15.43:0/1236805622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf4100e20 0x7fadf4101200 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.123+0000 7fadfa39e640 1 -- 172.21.15.43:0/1236805622 >> 172.21.15.43:0/1236805622 conn(0x7fadf40fc7d0 msgr2=0x7fadf40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:05.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.123+0000 7fadfa39e640 1 -- 172.21.15.43:0/1236805622 shutdown_connections 2024-09-11T23:32:05.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.123+0000 7fadfa39e640 1 -- 172.21.15.43:0/1236805622 wait complete. 2024-09-11T23:32:05.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.124+0000 7fadfa39e640 1 Processor -- start 2024-09-11T23:32:05.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.124+0000 7fadfa39e640 1 -- start start 2024-09-11T23:32:05.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.124+0000 7fadfa39e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadf4100e20 0x7fadf419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:05.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.124+0000 7fadfa39e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf4102570 0x7fadf419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:05.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.124+0000 7fadfa39e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadf4102ef0 0x7fadf419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:05.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.124+0000 7fadfa39e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fadf4110ff0 con 0x7fadf4102570 2024-09-11T23:32:05.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.124+0000 7fadfa39e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fadf4110e70 con 0x7fadf4102ef0 2024-09-11T23:32:05.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.124+0000 7fadfa39e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fadf4111170 con 0x7fadf4100e20 2024-09-11T23:32:05.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.124+0000 7fadebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf4102570 0x7fadf419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:05.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.124+0000 7fadebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf4102570 0x7fadf419ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38478/0 (socket says 172.21.15.43:38478) 2024-09-11T23:32:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.124+0000 7fadebfff640 1 -- 172.21.15.43:0/3973871057 learned_addr learned my addr 172.21.15.43:0/3973871057 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.124+0000 7fadf3fff640 1 --2- 172.21.15.43:0/3973871057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadf4100e20 0x7fadf419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.124+0000 7fadf893b640 1 --2- 172.21.15.43:0/3973871057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadf4102ef0 0x7fadf419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.124+0000 7fadebfff640 1 -- 172.21.15.43:0/3973871057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadf4100e20 msgr2=0x7fadf419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.124+0000 7fadebfff640 1 --2- 172.21.15.43:0/3973871057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadf4100e20 0x7fadf419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.124+0000 7fadebfff640 1 -- 172.21.15.43:0/3973871057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadf4102ef0 msgr2=0x7fadf419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.124+0000 7fadebfff640 1 --2- 172.21.15.43:0/3973871057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadf4102ef0 0x7fadf419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.124+0000 7fadebfff640 1 -- 172.21.15.43:0/3973871057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf419f6e0 con 0x7fadf4102570 2024-09-11T23:32:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.125+0000 7fadebfff640 1 --2- 172.21.15.43:0/3973871057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf4102570 0x7fadf419ac70 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7faddc00b4a0 tx=0x7faddc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:05.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.125+0000 7fadf1ffb640 1 -- 172.21.15.43:0/3973871057 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc014070 con 0x7fadf4102570 2024-09-11T23:32:05.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.125+0000 7fadf1ffb640 1 -- 172.21.15.43:0/3973871057 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faddc010af0 con 0x7fadf4102570 2024-09-11T23:32:05.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.125+0000 7fadf1ffb640 1 -- 172.21.15.43:0/3973871057 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc017d10 con 0x7fadf4102570 2024-09-11T23:32:05.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.125+0000 7fadfa39e640 1 -- 172.21.15.43:0/3973871057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf419f9d0 con 0x7fadf4102570 2024-09-11T23:32:05.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.125+0000 7fadfa39e640 1 -- 172.21.15.43:0/3973871057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf41a72b0 con 0x7fadf4102570 2024-09-11T23:32:05.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.127+0000 7fadf1ffb640 1 -- 172.21.15.43:0/3973871057 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7faddc017070 con 0x7fadf4102570 2024-09-11T23:32:05.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.127+0000 7fadfc777740 1 Processor -- start 2024-09-11T23:32:05.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.127+0000 7fadfc777740 1 -- start start 2024-09-11T23:32:05.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.128+0000 7fadfc777740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc1520cb40 0x55dc1520b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:05.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.128+0000 7fadfc777740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc1520b5d0 0x55dc1520ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.128+0000 7fadfc777740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc15216370 0x55dc15218760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.128+0000 7fadfc777740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dc151712f0 con 0x55dc1520b5d0 2024-09-11T23:32:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.128+0000 7fadfc777740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dc15118cf0 con 0x55dc1520cb40 2024-09-11T23:32:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.128+0000 7fadfc777740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dc1516d2b0 con 0x55dc15216370 2024-09-11T23:32:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.128+0000 7fadf3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc1520b5d0 0x55dc1520ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.128+0000 7fadf3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc1520b5d0 0x55dc1520ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38486/0 (socket says 172.21.15.43:38486) 2024-09-11T23:32:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.128+0000 7fadf3fff640 1 -- 172.21.15.43:0/3126218217 learned_addr learned my addr 172.21.15.43:0/3126218217 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.128+0000 7fadf1ffb640 1 --2- 172.21.15.43:0/3973871057 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fade00a2d70 0x7fade00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.128+0000 7fadebfff640 1 --2- 172.21.15.43:0/3126218217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc15216370 0x55dc15218760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.128+0000 7fadf893b640 1 --2- 172.21.15.43:0/3126218217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc1520cb40 0x55dc1520b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.128+0000 7fadf893b640 1 --2- 172.21.15.43:0/3973871057 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fade00a2d70 0x7fade00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.128+0000 7fadf3fff640 1 -- 172.21.15.43:0/3126218217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc15216370 msgr2=0x55dc15218760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.128+0000 7fadf3fff640 1 --2- 172.21.15.43:0/3126218217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc15216370 0x55dc15218760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.128+0000 7fadf3fff640 1 -- 172.21.15.43:0/3126218217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc1520cb40 msgr2=0x55dc1520b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:05.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.128+0000 7fadf3fff640 1 --2- 172.21.15.43:0/3126218217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc1520cb40 0x55dc1520b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.128+0000 7fadf3fff640 1 -- 172.21.15.43:0/3126218217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc150fd3a0 con 0x55dc1520b5d0 2024-09-11T23:32:05.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.128+0000 7fadebfff640 1 --2- 172.21.15.43:0/3126218217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc15216370 0x55dc15218760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:32:05.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.128+0000 7fadf1ffb640 1 -- 172.21.15.43:0/3973871057 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7faddc0ce680 con 0x7fadf4102570 2024-09-11T23:32:05.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.128+0000 7fadf3fff640 1 --2- 172.21.15.43:0/3126218217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc1520b5d0 0x55dc1520ba10 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fade4009900 tx=0x7fade4004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cf044fe90ef4c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:05.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.129+0000 7fadf893b640 1 --2- 172.21.15.43:0/3973871057 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fade00a2d70 0x7fade00a5230 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fadec005fd0 tx=0x7fadec009600 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:05.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.129+0000 7fadea7fc640 1 -- 172.21.15.43:0/3126218217 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade4035040 con 0x55dc1520b5d0 2024-09-11T23:32:05.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.129+0000 7fadea7fc640 1 -- 172.21.15.43:0/3126218217 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade4005520 con 0x55dc1520b5d0 2024-09-11T23:32:05.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.129+0000 7fadea7fc640 1 -- 172.21.15.43:0/3126218217 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade40038e0 con 0x55dc1520b5d0 2024-09-11T23:32:05.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.129+0000 7fadfc777740 1 -- 172.21.15.43:0/3126218217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc1520b5d0 msgr2=0x55dc1520ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:05.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.129+0000 7fadfc777740 1 --2- 172.21.15.43:0/3126218217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc1520b5d0 0x55dc1520ba10 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fade4009900 tx=0x7fade4004180 comp rx=0 tx=0).stop 2024-09-11T23:32:05.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.129+0000 7fadfc777740 1 -- 172.21.15.43:0/3126218217 shutdown_connections 2024-09-11T23:32:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.129+0000 7fadfc777740 1 --2- 172.21.15.43:0/3126218217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc15216370 0x55dc15218760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.129+0000 7fadfc777740 1 --2- 172.21.15.43:0/3126218217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc1520b5d0 0x55dc1520ba10 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.129+0000 7fadfc777740 1 --2- 172.21.15.43:0/3126218217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc1520cb40 0x55dc1520b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.129+0000 7fadfc777740 1 -- 172.21.15.43:0/3126218217 >> 172.21.15.43:0/3126218217 conn(0x55dc151d4310 msgr2=0x55dc15207c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.129+0000 7fadfc777740 1 -- 172.21.15.43:0/3126218217 shutdown_connections 2024-09-11T23:32:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.129+0000 7fadfc777740 1 -- 172.21.15.43:0/3126218217 wait complete. 2024-09-11T23:32:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.130+0000 7fadfc777740 1 -- start start 2024-09-11T23:32:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.130+0000 7fadfc777740 1 Processor -- start 2024-09-11T23:32:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.130+0000 7fadfc777740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc1520b5d0 0x55dc15215d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.130+0000 7fadfc777740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc1520cb40 0x55dc15213010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.130+0000 7fadfc777740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc15216370 0x55dc15213600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.130+0000 7fadfc777740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55dc15219010 con 0x55dc15216370 2024-09-11T23:32:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.130+0000 7fadfc777740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dc1512c060 con 0x55dc1520cb40 2024-09-11T23:32:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.130+0000 7fadfc777740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dc15219190 con 0x55dc1520b5d0 2024-09-11T23:32:05.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.130+0000 7fadfc777740 10 client.0 _ll_register_callbacks cb 0x55dc14e7f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:32:05.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.130+0000 7fadf893b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc15216370 0x55dc15213600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:05.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.131+0000 7fadf893b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc15216370 0x55dc15213600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38492/0 (socket says 172.21.15.43:38492) 2024-09-11T23:32:05.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.131+0000 7fadf893b640 1 -- 172.21.15.43:0/58366020 learned_addr learned my addr 172.21.15.43:0/58366020 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:05.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.131+0000 7fadebfff640 1 --2- 172.21.15.43:0/58366020 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc1520cb40 0x55dc15213010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:05.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.131+0000 7fadf3fff640 1 --2- 172.21.15.43:0/58366020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc1520b5d0 0x55dc15215d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:05.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.131+0000 7fadf893b640 1 -- 172.21.15.43:0/58366020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc1520b5d0 msgr2=0x55dc15215d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:05.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.131+0000 7fadf893b640 1 --2- 172.21.15.43:0/58366020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc1520b5d0 0x55dc15215d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.131+0000 7fadf893b640 1 -- 172.21.15.43:0/58366020 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc1520cb40 msgr2=0x55dc15213010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:05.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.131+0000 7fadf893b640 1 --2- 172.21.15.43:0/58366020 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc1520cb40 0x55dc15213010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.131+0000 7fadf893b640 1 -- 172.21.15.43:0/58366020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc14ee23c0 con 0x55dc15216370 2024-09-11T23:32:05.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.131+0000 7fadf893b640 1 --2- 172.21.15.43:0/58366020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc15216370 0x55dc15213600 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fadec010a10 tx=0x7fadec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7521e034326ebc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:05.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.131+0000 7fadd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:32:05.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.131+0000 7fadd37fe640 1 -- 172.21.15.43:0/58366020 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec025070 con 0x55dc15216370 2024-09-11T23:32:05.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.131+0000 7fadd37fe640 1 -- 172.21.15.43:0/58366020 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fadec004d20 con 0x55dc15216370 2024-09-11T23:32:05.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.131+0000 7fadfc777740 1 -- 172.21.15.43:0/58366020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fade4005520 con 0x55dc15216370 2024-09-11T23:32:05.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.131+0000 7fadd37fe640 1 -- 172.21.15.43:0/58366020 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fadec018070 con 0x55dc15216370 2024-09-11T23:32:05.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.131+0000 7fadfc777740 10 client.17052 fetch_fsmap learned FSMap version 34 2024-09-11T23:32:05.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.131+0000 7fadfc777740 1 -- 172.21.15.43:0/58366020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55dc15224b10 con 0x55dc15216370 2024-09-11T23:32:05.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.132+0000 7fadd37fe640 1 -- 172.21.15.43:0/58366020 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade4005520 con 0x55dc15216370 2024-09-11T23:32:05.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.132+0000 7fadd37fe640 1 -- 172.21.15.43:0/58366020 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fadec00f040 con 0x55dc15216370 2024-09-11T23:32:05.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.132+0000 7fadfc777740 10 client.17052 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T23:32:05.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.132+0000 7fadfc777740 10 client.17052 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T23:32:05.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.132+0000 7fadfc777740 20 client.17052 populate_metadata read hostname 'smithi043' 2024-09-11T23:32:05.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.132+0000 7fadfc777740 1 --2- 172.21.15.43:0/58366020 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55dc15225f20 0x55dc15228390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:05.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.132+0000 7fadfc777740 4 client.17052 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:32:05.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.132+0000 7fadfc777740 1 -- 172.21.15.43:0/58366020 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc1517e860 con 0x55dc15225f20 2024-09-11T23:32:05.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.132+0000 7fadf3fff640 1 --2- 172.21.15.43:0/58366020 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55dc15225f20 0x55dc15228390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:05.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.133+0000 7fadf3fff640 1 --2- 172.21.15.43:0/58366020 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55dc15225f20 0x55dc15228390 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=74c1d398ef50353c server_cookie=4f5975e2b73052f6 in_seq=0 out_seq=0 2024-09-11T23:32:05.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.133+0000 7fadd37fe640 10 client.17052 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:32:05.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.133+0000 7fadd37fe640 1 -- 172.21.15.43:0/58366020 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc1517e860 con 0x55dc15225f20 2024-09-11T23:32:05.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.133+0000 7fadd37fe640 10 client.17052 handle_command_reply: tid=0 2024-09-11T23:32:05.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.133+0000 7fadfc777740 1 client.17052 shutdown 2024-09-11T23:32:05.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.133+0000 7fadfc777740 1 -- 172.21.15.43:0/58366020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc15216370 msgr2=0x55dc15213600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:05.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.133+0000 7fadfc777740 1 --2- 172.21.15.43:0/58366020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc15216370 0x55dc15213600 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fadec010a10 tx=0x7fadec004a40 comp rx=0 tx=0).stop 2024-09-11T23:32:05.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.134+0000 7fadfc777740 1 -- 172.21.15.43:0/58366020 shutdown_connections 2024-09-11T23:32:05.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.134+0000 7fadfc777740 1 --2- 172.21.15.43:0/58366020 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55dc15225f20 0x55dc15228390 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.134+0000 7fadfc777740 1 --2- 172.21.15.43:0/58366020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc15216370 0x55dc15213600 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.134+0000 7fadfc777740 1 --2- 172.21.15.43:0/58366020 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc1520cb40 0x55dc15213010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.134+0000 7fadfc777740 1 --2- 172.21.15.43:0/58366020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc1520b5d0 0x55dc15215d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.134+0000 7fadd37fe640 0 client.17052 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:32:05.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.134+0000 7fadfc777740 1 -- 172.21.15.43:0/58366020 >> 172.21.15.43:0/58366020 conn(0x55dc151d4310 msgr2=0x55dc1520d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:05.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.134+0000 7fadfc777740 1 -- 172.21.15.43:0/58366020 shutdown_connections 2024-09-11T23:32:05.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.134+0000 7fadfc777740 1 -- 172.21.15.43:0/58366020 wait complete. 2024-09-11T23:32:05.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.134+0000 7fadfc777740 20 client.17052 trim_cache size 0 max 16384 2024-09-11T23:32:05.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.147+0000 7fadfc777740 1 Processor -- start 2024-09-11T23:32:05.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.147+0000 7fadfc777740 1 -- start start 2024-09-11T23:32:05.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.147+0000 7fadfc777740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc1520b5d0 0x55dc15207e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:05.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.147+0000 7fadfc777740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc1520cb40 0x55dc152128d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:05.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.147+0000 7fadfc777740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc15216370 0x55dc152222b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:05.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.147+0000 7fadfc777740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fadf41b24a0 con 0x55dc15216370 2024-09-11T23:32:05.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.147+0000 7fadfc777740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fadf41afd70 con 0x55dc1520cb40 2024-09-11T23:32:05.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.147+0000 7fadfc777740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fadf41adbe0 con 0x55dc1520b5d0 2024-09-11T23:32:05.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.147+0000 7fadf893b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc15216370 0x55dc152222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:05.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.147+0000 7fadf893b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc15216370 0x55dc152222b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38506/0 (socket says 172.21.15.43:38506) 2024-09-11T23:32:05.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.147+0000 7fadf893b640 1 -- 172.21.15.43:0/1222594968 learned_addr learned my addr 172.21.15.43:0/1222594968 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:05.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.147+0000 7fadf893b640 1 -- 172.21.15.43:0/1222594968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc1520b5d0 msgr2=0x55dc15207e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:05.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.147+0000 7fadebfff640 1 --2- 172.21.15.43:0/1222594968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc1520cb40 0x55dc152128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:05.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.147+0000 7fadf3fff640 1 --2- 172.21.15.43:0/1222594968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc1520b5d0 0x55dc15207e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:05.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.147+0000 7fadf893b640 1 --2- 172.21.15.43:0/1222594968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc1520b5d0 0x55dc15207e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.147+0000 7fadf893b640 1 -- 172.21.15.43:0/1222594968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc1520cb40 msgr2=0x55dc152128d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:05.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.147+0000 7fadf893b640 1 --2- 172.21.15.43:0/1222594968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc1520cb40 0x55dc152128d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.147+0000 7fadf893b640 1 -- 172.21.15.43:0/1222594968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc15211fc0 con 0x55dc15216370 2024-09-11T23:32:05.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.148+0000 7fadf3fff640 1 --2- 172.21.15.43:0/1222594968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc1520b5d0 0x55dc15207e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:32:05.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.148+0000 7fadebfff640 1 --2- 172.21.15.43:0/1222594968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc1520cb40 0x55dc152128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:32:05.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.148+0000 7fadf893b640 1 --2- 172.21.15.43:0/1222594968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc15216370 0x55dc152222b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fadec00c730 tx=0x7fadec0216a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d8b65b0df59b43 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:05.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.148+0000 7fade9ffb640 1 -- 172.21.15.43:0/1222594968 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec039070 con 0x55dc15216370 2024-09-11T23:32:05.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.148+0000 7fade9ffb640 1 -- 172.21.15.43:0/1222594968 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadec01a040 con 0x55dc15216370 2024-09-11T23:32:05.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.148+0000 7fade9ffb640 1 -- 172.21.15.43:0/1222594968 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec025040 con 0x55dc15216370 2024-09-11T23:32:05.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.148+0000 7fadfc777740 1 -- 172.21.15.43:0/1222594968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc15216370 msgr2=0x55dc152222b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:05.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.148+0000 7fadfc777740 1 --2- 172.21.15.43:0/1222594968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc15216370 0x55dc152222b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fadec00c730 tx=0x7fadec0216a0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.148+0000 7fadfc777740 1 -- 172.21.15.43:0/1222594968 shutdown_connections 2024-09-11T23:32:05.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.148+0000 7fadfc777740 1 --2- 172.21.15.43:0/1222594968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc15216370 0x55dc152222b0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.148+0000 7fadfc777740 1 --2- 172.21.15.43:0/1222594968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc1520cb40 0x55dc152128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.148+0000 7fadfc777740 1 --2- 172.21.15.43:0/1222594968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc1520b5d0 0x55dc15207e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.148+0000 7fadfc777740 1 -- 172.21.15.43:0/1222594968 >> 172.21.15.43:0/1222594968 conn(0x55dc151d4310 msgr2=0x55dc15207700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:05.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.149+0000 7fadfc777740 1 -- 172.21.15.43:0/1222594968 shutdown_connections 2024-09-11T23:32:05.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.149+0000 7fadfc777740 1 -- 172.21.15.43:0/1222594968 wait complete. 2024-09-11T23:32:05.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.149+0000 7fadfc777740 1 -- start start 2024-09-11T23:32:05.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.149+0000 7fadfc777740 1 Processor -- start 2024-09-11T23:32:05.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.149+0000 7fadfc777740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc1520b5d0 0x55dc1521e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:05.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.149+0000 7fadfc777740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc1520cb40 0x55dc1521e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:05.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.149+0000 7fadfc777740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc15216370 0x55dc1521b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:05.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.149+0000 7fadfc777740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fadf41ae230 con 0x55dc1520cb40 2024-09-11T23:32:05.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.149+0000 7fadfc777740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fadf41af6a0 con 0x55dc15216370 2024-09-11T23:32:05.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.149+0000 7fadfc777740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fadf41ae900 con 0x55dc1520b5d0 2024-09-11T23:32:05.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.149+0000 7fadfc777740 10 client.0 _ll_register_callbacks cb 0x7fadec0106c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:32:05.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.149+0000 7fadebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc1520cb40 0x55dc1521e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:05.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.149+0000 7fadebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc1520cb40 0x55dc1521e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38518/0 (socket says 172.21.15.43:38518) 2024-09-11T23:32:05.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.150+0000 7fadebfff640 1 -- 172.21.15.43:0/2679430515 learned_addr learned my addr 172.21.15.43:0/2679430515 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:05.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.150+0000 7fadf893b640 1 --2- 172.21.15.43:0/2679430515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc15216370 0x55dc1521b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:05.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.150+0000 7fadf3fff640 1 --2- 172.21.15.43:0/2679430515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc1520b5d0 0x55dc1521e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:05.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.150+0000 7fadebfff640 1 -- 172.21.15.43:0/2679430515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc1520b5d0 msgr2=0x55dc1521e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:05.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.150+0000 7fadebfff640 1 --2- 172.21.15.43:0/2679430515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc1520b5d0 0x55dc1521e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.150+0000 7fadebfff640 1 -- 172.21.15.43:0/2679430515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc15216370 msgr2=0x55dc1521b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:05.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.150+0000 7fadebfff640 1 --2- 172.21.15.43:0/2679430515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc15216370 0x55dc1521b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.150+0000 7fadebfff640 1 -- 172.21.15.43:0/2679430515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc152160d0 con 0x55dc1520cb40 2024-09-11T23:32:05.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.150+0000 7fadf893b640 1 --2- 172.21.15.43:0/2679430515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc15216370 0x55dc1521b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:32:05.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.150+0000 7fadf3fff640 1 --2- 172.21.15.43:0/2679430515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc1520b5d0 0x55dc1521e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:32:05.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.150+0000 7fadebfff640 1 --2- 172.21.15.43:0/2679430515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc1520cb40 0x55dc1521e960 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7faddc0176e0 tx=0x7faddc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbcb624fdc4678f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:05.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.150+0000 7fadd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:32:05.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.150+0000 7fadd37fe640 1 -- 172.21.15.43:0/2679430515 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc01e040 con 0x55dc1520cb40 2024-09-11T23:32:05.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.150+0000 7fadd37fe640 1 -- 172.21.15.43:0/2679430515 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7faddc010ca0 con 0x55dc1520cb40 2024-09-11T23:32:05.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.150+0000 7fadfc777740 1 -- 172.21.15.43:0/2679430515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadec01a040 con 0x55dc1520cb40 2024-09-11T23:32:05.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.150+0000 7fadd37fe640 1 -- 172.21.15.43:0/2679430515 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7faddc0a3890 con 0x55dc1520cb40 2024-09-11T23:32:05.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.150+0000 7fadfc777740 10 client.17058 fetch_fsmap learned FSMap version 34 2024-09-11T23:32:05.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.150+0000 7fadfc777740 1 -- 172.21.15.43:0/2679430515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55dc15290fe0 con 0x55dc1520cb40 2024-09-11T23:32:05.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.151+0000 7fadd37fe640 1 -- 172.21.15.43:0/2679430515 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadec01a040 con 0x55dc1520cb40 2024-09-11T23:32:05.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.151+0000 7fadd37fe640 1 -- 172.21.15.43:0/2679430515 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7faddc09a2b0 con 0x55dc1520cb40 2024-09-11T23:32:05.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.151+0000 7fadfc777740 10 client.17058 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T23:32:05.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.151+0000 7fadfc777740 10 client.17058 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T23:32:05.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.151+0000 7fadfc777740 20 client.17058 populate_metadata read hostname 'smithi043' 2024-09-11T23:32:05.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.151+0000 7fadfc777740 1 --2- 172.21.15.43:0/2679430515 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55dc152917b0 0x55dc15293bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:05.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.151+0000 7fadfc777740 4 client.17058 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:32:05.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.151+0000 7fadfc777740 1 -- 172.21.15.43:0/2679430515 --> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc15178d70 con 0x55dc152917b0 2024-09-11T23:32:05.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.151+0000 7fadf3fff640 1 --2- 172.21.15.43:0/2679430515 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55dc152917b0 0x55dc15293bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:05.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.151+0000 7fadd37fe640 10 client.17058 ms_handle_connect on v2:172.21.15.90:6836/2946725567 2024-09-11T23:32:05.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.151+0000 7fadf3fff640 1 --2- 172.21.15.43:0/2679430515 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55dc152917b0 0x55dc15293bd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=25a96f77970508f9 server_cookie=5056ea2732f2fd47 in_seq=0 out_seq=0 2024-09-11T23:32:05.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.152+0000 7fadd37fe640 1 -- 172.21.15.43:0/2679430515 <== mds.1 v2:172.21.15.90:6836/2946725567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc15178d70 con 0x55dc152917b0 2024-09-11T23:32:05.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.152+0000 7fadd37fe640 10 client.17058 handle_command_reply: tid=0 2024-09-11T23:32:05.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.152+0000 7fadfc777740 1 client.17058 shutdown 2024-09-11T23:32:05.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.152+0000 7fadfc777740 1 -- 172.21.15.43:0/2679430515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc1520cb40 msgr2=0x55dc1521e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:05.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.152+0000 7fadfc777740 1 --2- 172.21.15.43:0/2679430515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc1520cb40 0x55dc1521e960 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7faddc0176e0 tx=0x7faddc0046c0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.152+0000 7fadfc777740 1 -- 172.21.15.43:0/2679430515 shutdown_connections 2024-09-11T23:32:05.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.152+0000 7fadfc777740 1 --2- 172.21.15.43:0/2679430515 >> [v2:172.21.15.90:6836/2946725567,v1:172.21.15.90:6837/2946725567] conn(0x55dc152917b0 0x55dc15293bd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.152+0000 7fadfc777740 1 --2- 172.21.15.43:0/2679430515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc15216370 0x55dc1521b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.152+0000 7fadfc777740 1 --2- 172.21.15.43:0/2679430515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dc1520cb40 0x55dc1521e960 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.152+0000 7fadfc777740 1 --2- 172.21.15.43:0/2679430515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dc1520b5d0 0x55dc1521e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.152+0000 7fadfc777740 1 -- 172.21.15.43:0/2679430515 >> 172.21.15.43:0/2679430515 conn(0x55dc151d4310 msgr2=0x55dc1520d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:05.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.152+0000 7fadd37fe640 0 client.17058 ms_handle_reset on v2:172.21.15.90:6836/2946725567 2024-09-11T23:32:05.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.152+0000 7fadfc777740 1 -- 172.21.15.43:0/2679430515 shutdown_connections 2024-09-11T23:32:05.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.152+0000 7fadfc777740 1 -- 172.21.15.43:0/2679430515 wait complete. 2024-09-11T23:32:05.206 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:32:05.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.152+0000 7fadfc777740 20 client.17058 trim_cache size 0 max 16384 2024-09-11T23:32:05.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.152+0000 7fadfa39e640 1 -- 172.21.15.43:0/3973871057 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fade00a2d70 msgr2=0x7fade00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:05.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.152+0000 7fadfa39e640 1 --2- 172.21.15.43:0/3973871057 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fade00a2d70 0x7fade00a5230 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fadec005fd0 tx=0x7fadec009600 comp rx=0 tx=0).stop 2024-09-11T23:32:05.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.152+0000 7fadfa39e640 1 -- 172.21.15.43:0/3973871057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf4102570 msgr2=0x7fadf419ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:05.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.152+0000 7fadfa39e640 1 --2- 172.21.15.43:0/3973871057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf4102570 0x7fadf419ac70 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7faddc00b4a0 tx=0x7faddc00b960 comp rx=0 tx=0).stop 2024-09-11T23:32:05.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.153+0000 7fadfa39e640 1 -- 172.21.15.43:0/3973871057 shutdown_connections 2024-09-11T23:32:05.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.153+0000 7fadfa39e640 1 --2- 172.21.15.43:0/3973871057 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fade00a2d70 0x7fade00a5230 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.153+0000 7fadfa39e640 1 --2- 172.21.15.43:0/3973871057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadf4102ef0 0x7fadf419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.153+0000 7fadfa39e640 1 --2- 172.21.15.43:0/3973871057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf4102570 0x7fadf419ac70 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.153+0000 7fadfa39e640 1 --2- 172.21.15.43:0/3973871057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadf4100e20 0x7fadf419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:05.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.153+0000 7fadfa39e640 1 -- 172.21.15.43:0/3973871057 >> 172.21.15.43:0/3973871057 conn(0x7fadf40fc7d0 msgr2=0x7fadf406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:05.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.153+0000 7fadfa39e640 1 -- 172.21.15.43:0/3973871057 shutdown_connections 2024-09-11T23:32:05.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:05.153+0000 7fadfa39e640 1 -- 172.21.15.43:0/3973871057 wait complete. 2024-09-11T23:32:06.025 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:32:06.285 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:32:06.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:06 smithi043 ceph-mon[30950]: pgmap v317: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:32:06.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:06 smithi090 ceph-mon[35130]: pgmap v317: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:32:06.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:06 smithi097 ceph-mon[35562]: pgmap v317: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:32:06.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.783+0000 7f4945791640 1 -- 172.21.15.43:0/3629550416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4940102da0 msgr2=0x7f4940103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:06.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.783+0000 7f4945791640 1 --2- 172.21.15.43:0/3629550416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4940102da0 0x7f4940103180 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4930009900 tx=0x7f493002a480 comp rx=0 tx=0).stop 2024-09-11T23:32:06.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.783+0000 7f4945791640 1 -- 172.21.15.43:0/3629550416 shutdown_connections 2024-09-11T23:32:06.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.783+0000 7f4945791640 1 --2- 172.21.15.43:0/3629550416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f494010c180 0x7f494010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.783+0000 7f4945791640 1 --2- 172.21.15.43:0/3629550416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49401036c0 0x7f494010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.783+0000 7f4945791640 1 --2- 172.21.15.43:0/3629550416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4940102da0 0x7f4940103180 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.783+0000 7f4945791640 1 -- 172.21.15.43:0/3629550416 >> 172.21.15.43:0/3629550416 conn(0x7f49400fc7a0 msgr2=0x7f49400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:06.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.783+0000 7f4945791640 1 -- 172.21.15.43:0/3629550416 shutdown_connections 2024-09-11T23:32:06.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.783+0000 7f4945791640 1 -- 172.21.15.43:0/3629550416 wait complete. 2024-09-11T23:32:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.783+0000 7f4945791640 1 Processor -- start 2024-09-11T23:32:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.783+0000 7f4945791640 1 -- start start 2024-09-11T23:32:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.784+0000 7f4945791640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4940102da0 0x7f4940198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.784+0000 7f4945791640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49401036c0 0x7f4940198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.784+0000 7f4945791640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f494010c180 0x7f494019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.784+0000 7f4945791640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4940111f60 con 0x7f494010c180 2024-09-11T23:32:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.784+0000 7f4945791640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4940111de0 con 0x7f4940102da0 2024-09-11T23:32:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.784+0000 7f4945791640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f49401120e0 con 0x7f49401036c0 2024-09-11T23:32:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.784+0000 7f493f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f494010c180 0x7f494019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.784+0000 7f493f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f494010c180 0x7f494019ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38526/0 (socket says 172.21.15.43:38526) 2024-09-11T23:32:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.784+0000 7f493f7fe640 1 -- 172.21.15.43:0/3116297051 learned_addr learned my addr 172.21.15.43:0/3116297051 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.784+0000 7f4937fff640 1 --2- 172.21.15.43:0/3116297051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49401036c0 0x7f4940198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:06.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.784+0000 7f493effd640 1 --2- 172.21.15.43:0/3116297051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4940102da0 0x7f4940198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.784+0000 7f493f7fe640 1 -- 172.21.15.43:0/3116297051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49401036c0 msgr2=0x7f4940198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.784+0000 7f493f7fe640 1 --2- 172.21.15.43:0/3116297051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49401036c0 0x7f4940198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.784+0000 7f493f7fe640 1 -- 172.21.15.43:0/3116297051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4940102da0 msgr2=0x7f4940198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.784+0000 7f493f7fe640 1 --2- 172.21.15.43:0/3116297051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4940102da0 0x7f4940198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.784+0000 7f493f7fe640 1 -- 172.21.15.43:0/3116297051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494019d3d0 con 0x7f494010c180 2024-09-11T23:32:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.784+0000 7f493effd640 1 --2- 172.21.15.43:0/3116297051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4940102da0 0x7f4940198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:32:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.784+0000 7f493f7fe640 1 --2- 172.21.15.43:0/3116297051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f494010c180 0x7f494019ccf0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f493800eaa0 tx=0x7f493800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.784+0000 7f493cff9640 1 -- 172.21.15.43:0/3116297051 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493800cdb0 con 0x7f494010c180 2024-09-11T23:32:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.784+0000 7f493cff9640 1 -- 172.21.15.43:0/3116297051 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4938005020 con 0x7f494010c180 2024-09-11T23:32:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.784+0000 7f4945791640 1 -- 172.21.15.43:0/3116297051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f494019d6c0 con 0x7f494010c180 2024-09-11T23:32:06.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.785+0000 7f493cff9640 1 -- 172.21.15.43:0/3116297051 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49380116b0 con 0x7f494010c180 2024-09-11T23:32:06.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.785+0000 7f4945791640 1 -- 172.21.15.43:0/3116297051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49401a4fa0 con 0x7f494010c180 2024-09-11T23:32:06.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.787+0000 7f493cff9640 1 -- 172.21.15.43:0/3116297051 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4938005560 con 0x7f494010c180 2024-09-11T23:32:06.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.787+0000 7f4947b6a740 1 Processor -- start 2024-09-11T23:32:06.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.787+0000 7f4947b6a740 1 -- start start 2024-09-11T23:32:06.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.787+0000 7f4947b6a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c7db40 0x55bb89c7c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:06.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.787+0000 7f4947b6a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c7c5d0 0x55bb89c7ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:06.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.787+0000 7f4947b6a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb89c8b780 0x55bb89c8db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:06.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.787+0000 7f4947b6a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bb89be22f0 con 0x55bb89c7db40 2024-09-11T23:32:06.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.787+0000 7f4947b6a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bb89b89cf0 con 0x55bb89c7c5d0 2024-09-11T23:32:06.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.787+0000 7f4947b6a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bb89bde2b0 con 0x55bb89c8b780 2024-09-11T23:32:06.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.787+0000 7f4937fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c7db40 0x55bb89c7c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:06.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.787+0000 7f493cff9640 1 --2- 172.21.15.43:0/3116297051 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f492c0a2cf0 0x7f492c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:06.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.787+0000 7f4937fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c7db40 0x55bb89c7c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38534/0 (socket says 172.21.15.43:38534) 2024-09-11T23:32:06.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.787+0000 7f4937fff640 1 -- 172.21.15.43:0/4271420309 learned_addr learned my addr 172.21.15.43:0/4271420309 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:06.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.787+0000 7f493effd640 1 --2- 172.21.15.43:0/4271420309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c7c5d0 0x55bb89c7ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:06.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.788+0000 7f493f7fe640 1 --2- 172.21.15.43:0/3116297051 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f492c0a2cf0 0x7f492c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:06.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.788+0000 7f4937fff640 1 --2- 172.21.15.43:0/4271420309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb89c8b780 0x55bb89c8db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:06.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.788+0000 7f493cff9640 1 -- 172.21.15.43:0/3116297051 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f49380cf680 con 0x7f494010c180 2024-09-11T23:32:06.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.788+0000 7f4937fff640 1 -- 172.21.15.43:0/4271420309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb89c8b780 msgr2=0x55bb89c8db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:06.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.788+0000 7f4937fff640 1 --2- 172.21.15.43:0/4271420309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb89c8b780 0x55bb89c8db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.788+0000 7f4937fff640 1 -- 172.21.15.43:0/4271420309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c7c5d0 msgr2=0x55bb89c7ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:06.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.788+0000 7f4937fff640 1 --2- 172.21.15.43:0/4271420309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c7c5d0 0x55bb89c7ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.788+0000 7f4937fff640 1 -- 172.21.15.43:0/4271420309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb89b6e3a0 con 0x55bb89c7db40 2024-09-11T23:32:06.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.788+0000 7f493effd640 1 --2- 172.21.15.43:0/4271420309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c7c5d0 0x55bb89c7ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:32:06.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.788+0000 7f4937fff640 1 --2- 172.21.15.43:0/4271420309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c7db40 0x55bb89c7c090 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f49280058a0 tx=0x7f492800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595baacd3ce7ea29 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:06.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.788+0000 7f493f7fe640 1 --2- 172.21.15.43:0/3116297051 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f492c0a2cf0 0x7f492c0a51b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f493809b810 tx=0x7f4938090fa0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:06.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.788+0000 7f49357fa640 1 -- 172.21.15.43:0/4271420309 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928004030 con 0x55bb89c7db40 2024-09-11T23:32:06.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.788+0000 7f49357fa640 1 -- 172.21.15.43:0/4271420309 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49280041d0 con 0x55bb89c7db40 2024-09-11T23:32:06.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.788+0000 7f49357fa640 1 -- 172.21.15.43:0/4271420309 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49280106c0 con 0x55bb89c7db40 2024-09-11T23:32:06.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.789+0000 7f4947b6a740 1 -- 172.21.15.43:0/4271420309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c7db40 msgr2=0x55bb89c7c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:06.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.789+0000 7f4947b6a740 1 --2- 172.21.15.43:0/4271420309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c7db40 0x55bb89c7c090 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f49280058a0 tx=0x7f492800b840 comp rx=0 tx=0).stop 2024-09-11T23:32:06.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.789+0000 7f4947b6a740 1 -- 172.21.15.43:0/4271420309 shutdown_connections 2024-09-11T23:32:06.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.789+0000 7f4947b6a740 1 --2- 172.21.15.43:0/4271420309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb89c8b780 0x55bb89c8db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.789+0000 7f4947b6a740 1 --2- 172.21.15.43:0/4271420309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c7c5d0 0x55bb89c7ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.789+0000 7f4947b6a740 1 --2- 172.21.15.43:0/4271420309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c7db40 0x55bb89c7c090 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.789+0000 7f4947b6a740 1 -- 172.21.15.43:0/4271420309 >> 172.21.15.43:0/4271420309 conn(0x55bb89c45310 msgr2=0x55bb89c78c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:06.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.789+0000 7f4947b6a740 1 -- 172.21.15.43:0/4271420309 shutdown_connections 2024-09-11T23:32:06.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.789+0000 7f4947b6a740 1 -- 172.21.15.43:0/4271420309 wait complete. 2024-09-11T23:32:06.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.789+0000 7f4947b6a740 1 -- start start 2024-09-11T23:32:06.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.789+0000 7f4947b6a740 1 Processor -- start 2024-09-11T23:32:06.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f4947b6a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c7c5d0 0x55bb89c8b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:06.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f4947b6a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb89c7db40 0x55bb89c883f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:06.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f4947b6a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c8b780 0x55bb89c889e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:06.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f4947b6a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55bb89c8e480 con 0x55bb89c8b780 2024-09-11T23:32:06.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f4947b6a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bb89b9d060 con 0x55bb89c7c5d0 2024-09-11T23:32:06.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f4947b6a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bb89c8e600 con 0x55bb89c7db40 2024-09-11T23:32:06.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f4947b6a740 10 client.0 _ll_register_callbacks cb 0x55bb898f08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:32:06.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f4937fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c8b780 0x55bb89c889e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:06.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f4937fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c8b780 0x55bb89c889e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38544/0 (socket says 172.21.15.43:38544) 2024-09-11T23:32:06.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f4937fff640 1 -- 172.21.15.43:0/3843849915 learned_addr learned my addr 172.21.15.43:0/3843849915 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:06.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f4937fff640 1 --2- 172.21.15.43:0/3843849915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c7c5d0 0x55bb89c8b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:06.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f493effd640 1 --2- 172.21.15.43:0/3843849915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb89c7db40 0x55bb89c883f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:06.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f4937fff640 1 -- 172.21.15.43:0/3843849915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb89c7db40 msgr2=0x55bb89c883f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:06.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f4937fff640 1 --2- 172.21.15.43:0/3843849915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb89c7db40 0x55bb89c883f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f4937fff640 1 -- 172.21.15.43:0/3843849915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c7c5d0 msgr2=0x55bb89c8b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:06.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f4937fff640 1 --2- 172.21.15.43:0/3843849915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c7c5d0 0x55bb89c8b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f4937fff640 1 -- 172.21.15.43:0/3843849915 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb899533c0 con 0x55bb89c8b780 2024-09-11T23:32:06.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f4937fff640 1 --2- 172.21.15.43:0/3843849915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c8b780 0x55bb89c889e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f492800b450 tx=0x7f4928003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c03a89258540d03 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:06.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f49267fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:32:06.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f49267fc640 1 -- 172.21.15.43:0/3843849915 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928028040 con 0x55bb89c8b780 2024-09-11T23:32:06.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f4947b6a740 1 -- 172.21.15.43:0/3843849915 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49280041d0 con 0x55bb89c8b780 2024-09-11T23:32:06.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.790+0000 7f49267fc640 1 -- 172.21.15.43:0/3843849915 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f4928010460 con 0x55bb89c8b780 2024-09-11T23:32:06.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.791+0000 7f49267fc640 1 -- 172.21.15.43:0/3843849915 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4928010660 con 0x55bb89c8b780 2024-09-11T23:32:06.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.791+0000 7f4947b6a740 10 client.17070 fetch_fsmap learned FSMap version 34 2024-09-11T23:32:06.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.791+0000 7f4947b6a740 1 -- 172.21.15.43:0/3843849915 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bb89c99ee0 con 0x55bb89c8b780 2024-09-11T23:32:06.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.791+0000 7f49267fc640 1 -- 172.21.15.43:0/3843849915 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49280041d0 con 0x55bb89c8b780 2024-09-11T23:32:06.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.791+0000 7f49267fc640 1 -- 172.21.15.43:0/3843849915 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f492800e490 con 0x55bb89c8b780 2024-09-11T23:32:06.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.791+0000 7f4947b6a740 10 client.17070 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T23:32:06.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.791+0000 7f4947b6a740 10 client.17070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:32:06.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.791+0000 7f4947b6a740 20 client.17070 populate_metadata read hostname 'smithi043' 2024-09-11T23:32:06.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.791+0000 7f4947b6a740 1 --2- 172.21.15.43:0/3843849915 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bb89c9b230 0x55bb89c9d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:06.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.791+0000 7f4947b6a740 4 client.17070 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:32:06.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.791+0000 7f4947b6a740 1 -- 172.21.15.43:0/3843849915 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb89bef860 con 0x55bb89c9b230 2024-09-11T23:32:06.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.791+0000 7f493effd640 1 --2- 172.21.15.43:0/3843849915 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bb89c9b230 0x55bb89c9d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:06.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.792+0000 7f493effd640 1 --2- 172.21.15.43:0/3843849915 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bb89c9b230 0x55bb89c9d620 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10be888956f22713 server_cookie=674195dae8a61477 in_seq=0 out_seq=0 2024-09-11T23:32:06.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.792+0000 7f49267fc640 10 client.17070 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:32:06.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.792+0000 7f49267fc640 1 -- 172.21.15.43:0/3843849915 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb89bef860 con 0x55bb89c9b230 2024-09-11T23:32:06.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.792+0000 7f49267fc640 10 client.17070 handle_command_reply: tid=0 2024-09-11T23:32:06.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.792+0000 7f4947b6a740 1 client.17070 shutdown 2024-09-11T23:32:06.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.792+0000 7f4947b6a740 1 -- 172.21.15.43:0/3843849915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c8b780 msgr2=0x55bb89c889e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:06.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.792+0000 7f4947b6a740 1 --2- 172.21.15.43:0/3843849915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c8b780 0x55bb89c889e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f492800b450 tx=0x7f4928003c30 comp rx=0 tx=0).stop 2024-09-11T23:32:06.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.793+0000 7f4947b6a740 1 -- 172.21.15.43:0/3843849915 shutdown_connections 2024-09-11T23:32:06.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.793+0000 7f4947b6a740 1 --2- 172.21.15.43:0/3843849915 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bb89c9b230 0x55bb89c9d620 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.793+0000 7f4947b6a740 1 --2- 172.21.15.43:0/3843849915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c8b780 0x55bb89c889e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.793+0000 7f4947b6a740 1 --2- 172.21.15.43:0/3843849915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb89c7db40 0x55bb89c883f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.793+0000 7f4947b6a740 1 --2- 172.21.15.43:0/3843849915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c7c5d0 0x55bb89c8b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.793+0000 7f4947b6a740 1 -- 172.21.15.43:0/3843849915 >> 172.21.15.43:0/3843849915 conn(0x55bb89c45310 msgr2=0x55bb89c7e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:06.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.793+0000 7f49267fc640 0 client.17070 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:32:06.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.793+0000 7f4947b6a740 1 -- 172.21.15.43:0/3843849915 shutdown_connections 2024-09-11T23:32:06.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.793+0000 7f4947b6a740 1 -- 172.21.15.43:0/3843849915 wait complete. 2024-09-11T23:32:06.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.793+0000 7f4947b6a740 20 client.17070 trim_cache size 0 max 16384 2024-09-11T23:32:06.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.806+0000 7f4947b6a740 1 Processor -- start 2024-09-11T23:32:06.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.806+0000 7f4947b6a740 1 -- start start 2024-09-11T23:32:06.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.806+0000 7f4947b6a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb89c7c5d0 0x55bb89c97720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:06.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.806+0000 7f4947b6a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c7db40 0x55bb89c83720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:06.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.806+0000 7f4947b6a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c8b780 0x55bb89c8a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:06.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.806+0000 7f4947b6a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49401ae1a0 con 0x55bb89c7db40 2024-09-11T23:32:06.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.806+0000 7f4947b6a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f49401aba70 con 0x55bb89c8b780 2024-09-11T23:32:06.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.806+0000 7f4947b6a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f49401a98e0 con 0x55bb89c7c5d0 2024-09-11T23:32:06.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.806+0000 7f493effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c7db40 0x55bb89c83720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:06.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.806+0000 7f493effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c7db40 0x55bb89c83720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38548/0 (socket says 172.21.15.43:38548) 2024-09-11T23:32:06.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.806+0000 7f493effd640 1 -- 172.21.15.43:0/3109735661 learned_addr learned my addr 172.21.15.43:0/3109735661 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:06.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.806+0000 7f4937fff640 1 --2- 172.21.15.43:0/3109735661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c8b780 0x55bb89c8a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:06.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.806+0000 7f493effd640 1 -- 172.21.15.43:0/3109735661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb89c7c5d0 msgr2=0x55bb89c97720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:06.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.806+0000 7f493effd640 1 --2- 172.21.15.43:0/3109735661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb89c7c5d0 0x55bb89c97720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.806+0000 7f493effd640 1 -- 172.21.15.43:0/3109735661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c8b780 msgr2=0x55bb89c8a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:06.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.806+0000 7f493effd640 1 --2- 172.21.15.43:0/3109735661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c8b780 0x55bb89c8a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.806+0000 7f493effd640 1 -- 172.21.15.43:0/3109735661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb89c78ed0 con 0x55bb89c7db40 2024-09-11T23:32:06.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.806+0000 7f4937fff640 1 --2- 172.21.15.43:0/3109735661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c8b780 0x55bb89c8a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:32:06.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.807+0000 7f493effd640 1 --2- 172.21.15.43:0/3109735661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c7db40 0x55bb89c83720 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f49300092a0 tx=0x7f4930005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17fd11c36f6895a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:06.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.807+0000 7f4934ff9640 1 -- 172.21.15.43:0/3109735661 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930048020 con 0x55bb89c7db40 2024-09-11T23:32:06.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.807+0000 7f4934ff9640 1 -- 172.21.15.43:0/3109735661 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4930032880 con 0x55bb89c7db40 2024-09-11T23:32:06.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.807+0000 7f4934ff9640 1 -- 172.21.15.43:0/3109735661 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930035040 con 0x55bb89c7db40 2024-09-11T23:32:06.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.807+0000 7f4947b6a740 1 -- 172.21.15.43:0/3109735661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c7db40 msgr2=0x55bb89c83720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:06.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.807+0000 7f4947b6a740 1 --2- 172.21.15.43:0/3109735661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c7db40 0x55bb89c83720 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f49300092a0 tx=0x7f4930005b50 comp rx=0 tx=0).stop 2024-09-11T23:32:06.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.807+0000 7f4947b6a740 1 -- 172.21.15.43:0/3109735661 shutdown_connections 2024-09-11T23:32:06.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.807+0000 7f4947b6a740 1 --2- 172.21.15.43:0/3109735661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c8b780 0x55bb89c8a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.807+0000 7f4947b6a740 1 --2- 172.21.15.43:0/3109735661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c7db40 0x55bb89c83720 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.807+0000 7f4947b6a740 1 --2- 172.21.15.43:0/3109735661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb89c7c5d0 0x55bb89c97720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.807+0000 7f4947b6a740 1 -- 172.21.15.43:0/3109735661 >> 172.21.15.43:0/3109735661 conn(0x55bb89c45310 msgr2=0x55bb89c89c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:06.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.808+0000 7f4947b6a740 1 -- 172.21.15.43:0/3109735661 shutdown_connections 2024-09-11T23:32:06.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.808+0000 7f4947b6a740 1 -- 172.21.15.43:0/3109735661 wait complete. 2024-09-11T23:32:06.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.808+0000 7f4947b6a740 1 -- start start 2024-09-11T23:32:06.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.808+0000 7f4947b6a740 1 Processor -- start 2024-09-11T23:32:06.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.808+0000 7f4947b6a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb89c7c5d0 0x55bb89c93d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:06.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.808+0000 7f4947b6a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c7db40 0x55bb89c8f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:06.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.808+0000 7f4947b6a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c8b780 0x55bb89c8f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:06.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.808+0000 7f4947b6a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49401a9f30 con 0x55bb89c7db40 2024-09-11T23:32:06.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.808+0000 7f4947b6a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f49401ab3a0 con 0x55bb89c8b780 2024-09-11T23:32:06.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.808+0000 7f4947b6a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f49401aa600 con 0x55bb89c7c5d0 2024-09-11T23:32:06.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.808+0000 7f4947b6a740 10 client.0 _ll_register_callbacks cb 0x7f4930009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:32:06.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.808+0000 7f493effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c7db40 0x55bb89c8f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:06.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.808+0000 7f493effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c7db40 0x55bb89c8f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38560/0 (socket says 172.21.15.43:38560) 2024-09-11T23:32:06.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.808+0000 7f493effd640 1 -- 172.21.15.43:0/8645258 learned_addr learned my addr 172.21.15.43:0/8645258 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:06.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.808+0000 7f4937fff640 1 --2- 172.21.15.43:0/8645258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c8b780 0x55bb89c8f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:06.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.809+0000 7f4937fff640 1 --2- 172.21.15.43:0/8645258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb89c7c5d0 0x55bb89c93d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:06.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.809+0000 7f493effd640 1 -- 172.21.15.43:0/8645258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb89c7c5d0 msgr2=0x55bb89c93d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:06.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.809+0000 7f493effd640 1 --2- 172.21.15.43:0/8645258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb89c7c5d0 0x55bb89c93d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.809+0000 7f493effd640 1 -- 172.21.15.43:0/8645258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c8b780 msgr2=0x55bb89c8f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:32:06.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.809+0000 7f493effd640 1 --2- 172.21.15.43:0/8645258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c8b780 0x55bb89c8f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.809+0000 7f493effd640 1 -- 172.21.15.43:0/8645258 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb89c8fbb0 con 0x55bb89c7db40 2024-09-11T23:32:06.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.809+0000 7f4937fff640 1 --2- 172.21.15.43:0/8645258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb89c7c5d0 0x55bb89c93d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:32:06.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.809+0000 7f493effd640 1 --2- 172.21.15.43:0/8645258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c7db40 0x55bb89c8f0b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4930031550 tx=0x7f4930000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254b1ac8b66b0102 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:06.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.809+0000 7f49267fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-09-11T23:32:06.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.809+0000 7f49267fc640 1 -- 172.21.15.43:0/8645258 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930051070 con 0x55bb89c7db40 2024-09-11T23:32:06.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.809+0000 7f49267fc640 1 -- 172.21.15.43:0/8645258 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f4930004040 con 0x55bb89c7db40 2024-09-11T23:32:06.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.809+0000 7f4947b6a740 1 -- 172.21.15.43:0/8645258 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4930032880 con 0x55bb89c7db40 2024-09-11T23:32:06.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.809+0000 7f49267fc640 1 -- 172.21.15.43:0/8645258 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f493005d050 con 0x55bb89c7db40 2024-09-11T23:32:06.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.809+0000 7f4947b6a740 10 client.17076 fetch_fsmap learned FSMap version 34 2024-09-11T23:32:06.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.809+0000 7f4947b6a740 1 -- 172.21.15.43:0/8645258 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bb89d05a10 con 0x55bb89c7db40 2024-09-11T23:32:06.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.809+0000 7f49267fc640 1 -- 172.21.15.43:0/8645258 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4930032880 con 0x55bb89c7db40 2024-09-11T23:32:06.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.810+0000 7f49267fc640 1 -- 172.21.15.43:0/8645258 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f4930004750 con 0x55bb89c7db40 2024-09-11T23:32:06.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.810+0000 7f4947b6a740 10 client.17076 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T23:32:06.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.810+0000 7f4947b6a740 10 client.17076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:32:06.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.810+0000 7f4947b6a740 20 client.17076 populate_metadata read hostname 'smithi043' 2024-09-11T23:32:06.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.810+0000 7f4947b6a740 1 --2- 172.21.15.43:0/8645258 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bb89d060b0 0x55bb89d084d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:06.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.810+0000 7f4947b6a740 4 client.17076 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:32:06.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.810+0000 7f4947b6a740 1 -- 172.21.15.43:0/8645258 --> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb89be9d70 con 0x55bb89d060b0 2024-09-11T23:32:06.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.810+0000 7f4937fff640 1 --2- 172.21.15.43:0/8645258 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bb89d060b0 0x55bb89d084d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:06.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.810+0000 7f49267fc640 10 client.17076 ms_handle_connect on v2:172.21.15.90:6838/3234895964 2024-09-11T23:32:06.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.810+0000 7f4937fff640 1 --2- 172.21.15.43:0/8645258 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bb89d060b0 0x55bb89d084d0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba97bd8815dc6f8c server_cookie=4dd7e3ba79f199db in_seq=0 out_seq=0 2024-09-11T23:32:06.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.810+0000 7f49267fc640 1 -- 172.21.15.43:0/8645258 <== mds.0 v2:172.21.15.90:6838/3234895964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb89be9d70 con 0x55bb89d060b0 2024-09-11T23:32:06.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.810+0000 7f49267fc640 10 client.17076 handle_command_reply: tid=0 2024-09-11T23:32:06.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.810+0000 7f4947b6a740 1 client.17076 shutdown 2024-09-11T23:32:06.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.811+0000 7f4947b6a740 1 -- 172.21.15.43:0/8645258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c7db40 msgr2=0x55bb89c8f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:06.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.811+0000 7f4947b6a740 1 --2- 172.21.15.43:0/8645258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c7db40 0x55bb89c8f0b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4930031550 tx=0x7f4930000bc0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.811+0000 7f4947b6a740 1 -- 172.21.15.43:0/8645258 shutdown_connections 2024-09-11T23:32:06.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.811+0000 7f4947b6a740 1 --2- 172.21.15.43:0/8645258 >> [v2:172.21.15.90:6838/3234895964,v1:172.21.15.90:6839/3234895964] conn(0x55bb89d060b0 0x55bb89d084d0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.811+0000 7f4947b6a740 1 --2- 172.21.15.43:0/8645258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb89c8b780 0x55bb89c8f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.811+0000 7f4947b6a740 1 --2- 172.21.15.43:0/8645258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bb89c7db40 0x55bb89c8f0b0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.811+0000 7f4947b6a740 1 --2- 172.21.15.43:0/8645258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bb89c7c5d0 0x55bb89c93d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.811+0000 7f4947b6a740 1 -- 172.21.15.43:0/8645258 >> 172.21.15.43:0/8645258 conn(0x55bb89c45310 msgr2=0x55bb89c8a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.811+0000 7f49267fc640 0 client.17076 ms_handle_reset on v2:172.21.15.90:6838/3234895964 2024-09-11T23:32:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.811+0000 7f4947b6a740 1 -- 172.21.15.43:0/8645258 shutdown_connections 2024-09-11T23:32:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.811+0000 7f4947b6a740 1 -- 172.21.15.43:0/8645258 wait complete. 2024-09-11T23:32:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.811+0000 7f4947b6a740 20 client.17076 trim_cache size 0 max 16384 2024-09-11T23:32:06.833 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-09-11T23:32:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.811+0000 7f4945791640 1 -- 172.21.15.43:0/3116297051 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f492c0a2cf0 msgr2=0x7f492c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.811+0000 7f4945791640 1 --2- 172.21.15.43:0/3116297051 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f492c0a2cf0 0x7f492c0a51b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f493809b810 tx=0x7f4938090fa0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.811+0000 7f4945791640 1 -- 172.21.15.43:0/3116297051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f494010c180 msgr2=0x7f494019ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.811+0000 7f4945791640 1 --2- 172.21.15.43:0/3116297051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f494010c180 0x7f494019ccf0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f493800eaa0 tx=0x7f493800ef60 comp rx=0 tx=0).stop 2024-09-11T23:32:06.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.812+0000 7f4945791640 1 -- 172.21.15.43:0/3116297051 shutdown_connections 2024-09-11T23:32:06.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.812+0000 7f4945791640 1 --2- 172.21.15.43:0/3116297051 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f492c0a2cf0 0x7f492c0a51b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.812+0000 7f4945791640 1 --2- 172.21.15.43:0/3116297051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f494010c180 0x7f494019ccf0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.812+0000 7f4945791640 1 --2- 172.21.15.43:0/3116297051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49401036c0 0x7f4940198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.812+0000 7f4945791640 1 --2- 172.21.15.43:0/3116297051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4940102da0 0x7f4940198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:06.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.812+0000 7f4945791640 1 -- 172.21.15.43:0/3116297051 >> 172.21.15.43:0/3116297051 conn(0x7f49400fc7a0 msgr2=0x7f49400fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:06.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.812+0000 7f4945791640 1 -- 172.21.15.43:0/3116297051 shutdown_connections 2024-09-11T23:32:06.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:06.812+0000 7f4945791640 1 -- 172.21.15.43:0/3116297051 wait complete. 2024-09-11T23:32:07.512 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-11T23:32:07.513 INFO:tasks.fwd_scrub:done joining 2024-09-11T23:32:07.513 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-11T23:32:08.766 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T23:32:08.768 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:08 smithi097 ceph-mon[35562]: pgmap v318: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-11T23:32:08.768 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:08 smithi090 ceph-mon[35130]: pgmap v318: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-11T23:32:08.768 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:08 smithi043 ceph-mon[30950]: pgmap v318: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-11T23:32:08.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.885+0000 7f716ef33640 1 -- 172.21.15.43:0/3598634079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7168102d00 msgr2=0x7f71681030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:08.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.885+0000 7f716ef33640 1 --2- 172.21.15.43:0/3598634079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7168102d00 0x7f71681030e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7150009900 tx=0x7f715002a480 comp rx=0 tx=0).stop 2024-09-11T23:32:08.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.886+0000 7f716ef33640 1 -- 172.21.15.43:0/3598634079 shutdown_connections 2024-09-11T23:32:08.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.886+0000 7f716ef33640 1 --2- 172.21.15.43:0/3598634079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f716810c0d0 0x7f716810e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:08.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.886+0000 7f716ef33640 1 --2- 172.21.15.43:0/3598634079 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7168103620 0x7f716810bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:08.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.886+0000 7f716ef33640 1 --2- 172.21.15.43:0/3598634079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7168102d00 0x7f71681030e0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:08.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.886+0000 7f716ef33640 1 -- 172.21.15.43:0/3598634079 >> 172.21.15.43:0/3598634079 conn(0x7f71680fc6b0 msgr2=0x7f71680fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:08.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.886+0000 7f716ef33640 1 -- 172.21.15.43:0/3598634079 shutdown_connections 2024-09-11T23:32:08.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.886+0000 7f716ef33640 1 -- 172.21.15.43:0/3598634079 wait complete. 2024-09-11T23:32:08.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.886+0000 7f716ef33640 1 Processor -- start 2024-09-11T23:32:08.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.886+0000 7f716ef33640 1 -- start start 2024-09-11T23:32:08.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f716ef33640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7168102d00 0x7f7168198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:08.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f716ef33640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7168103620 0x7f71681988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:08.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f716ef33640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f716810c0d0 0x7f716819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:08.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f716ef33640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7168111ed0 con 0x7f716810c0d0 2024-09-11T23:32:08.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f716ef33640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7168111d50 con 0x7f7168102d00 2024-09-11T23:32:08.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f716ef33640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7168112050 con 0x7f7168103620 2024-09-11T23:32:08.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f716d4d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f716810c0d0 0x7f716819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:08.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f716d4d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f716810c0d0 0x7f716819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38584/0 (socket says 172.21.15.43:38584) 2024-09-11T23:32:08.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f716d4d0640 1 -- 172.21.15.43:0/3082894039 learned_addr learned my addr 172.21.15.43:0/3082894039 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:08.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f7167fff640 1 --2- 172.21.15.43:0/3082894039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7168103620 0x7f71681988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:08.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f716cccf640 1 --2- 172.21.15.43:0/3082894039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7168102d00 0x7f7168198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f716d4d0640 1 -- 172.21.15.43:0/3082894039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7168103620 msgr2=0x7f71681988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f716d4d0640 1 --2- 172.21.15.43:0/3082894039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7168103620 0x7f71681988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f716d4d0640 1 -- 172.21.15.43:0/3082894039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7168102d00 msgr2=0x7f7168198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f716d4d0640 1 --2- 172.21.15.43:0/3082894039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7168102d00 0x7f7168198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f716d4d0640 1 -- 172.21.15.43:0/3082894039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716819d370 con 0x7f716810c0d0 2024-09-11T23:32:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f716cccf640 1 --2- 172.21.15.43:0/3082894039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7168102d00 0x7f7168198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:32:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f7167fff640 1 --2- 172.21.15.43:0/3082894039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7168103620 0x7f71681988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:32:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f716d4d0640 1 --2- 172.21.15.43:0/3082894039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f716810c0d0 0x7f716819cc40 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f716000c970 tx=0x7f716000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f7165ffb640 1 -- 172.21.15.43:0/3082894039 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160007bb0 con 0x7f716810c0d0 2024-09-11T23:32:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f7165ffb640 1 -- 172.21.15.43:0/3082894039 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7160007d50 con 0x7f716810c0d0 2024-09-11T23:32:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f716ef33640 1 -- 172.21.15.43:0/3082894039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f716819d660 con 0x7f716810c0d0 2024-09-11T23:32:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.887+0000 7f7165ffb640 1 -- 172.21.15.43:0/3082894039 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160018800 con 0x7f716810c0d0 2024-09-11T23:32:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.888+0000 7f716ef33640 1 -- 172.21.15.43:0/3082894039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71681a4ef0 con 0x7f716810c0d0 2024-09-11T23:32:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.889+0000 7f716ef33640 1 -- 172.21.15.43:0/3082894039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71681030e0 con 0x7f716810c0d0 2024-09-11T23:32:08.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.890+0000 7f7165ffb640 1 -- 172.21.15.43:0/3082894039 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f716000fa50 con 0x7f716810c0d0 2024-09-11T23:32:08.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.890+0000 7f7165ffb640 1 --2- 172.21.15.43:0/3082894039 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f71540a2d80 0x7f71540a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:08.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.890+0000 7f716cccf640 1 --2- 172.21.15.43:0/3082894039 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f71540a2d80 0x7f71540a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:08.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.890+0000 7f7165ffb640 1 -- 172.21.15.43:0/3082894039 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f71600cf490 con 0x7f716810c0d0 2024-09-11T23:32:08.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.891+0000 7f716cccf640 1 --2- 172.21.15.43:0/3082894039 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f71540a2d80 0x7f71540a5240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7150000c00 tx=0x7f7150032c30 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:08.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:08.895+0000 7f7165ffb640 1 -- 172.21.15.43:0/3082894039 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7160014030 con 0x7f716810c0d0 2024-09-11T23:32:09.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.074+0000 7f716ef33640 1 -- 172.21.15.43:0/3082894039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71681071e0 con 0x7f716810c0d0 2024-09-11T23:32:09.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.076+0000 7f7165ffb640 1 -- 172.21.15.43:0/3082894039 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+23138 (secure 0 0 0) 0x7f716009c080 con 0x7f716810c0d0 2024-09-11T23:32:09.076 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:32:09.076 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":92,"fsid":"25c0cd84-7093-11ef-bcea-c7b262605968","created":"2024-09-11T23:12:03.504406+0000","modified":"2024-09-11T23:31:23.880308+0000","last_up_change":"2024-09-11T23:19:07.797873+0000","last_in_change":"2024-09-11T23:18:44.595176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:15:17.058713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:19:56.221406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:19:58.705875+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":13,"snap_epoch":91,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24076596-3909-4c87-85f1-cbf168070dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6803","nonce":43839997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6805","nonce":43839997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6809","nonce":43839997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6807","nonce":43839997}]},"public_addr":"172.21.15.43:6803/43839997","cluster_addr":"172.21.15.43:6805/43839997","heartbeat_back_addr":"172.21.15.43:6809/43839997","heartbeat_front_addr":"172.21.15.43:6807/43839997","state":["exists","up"]},{"osd":1,"uuid":"a0472a57-243c-489d-a247-972ee10a71f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6801","nonce":1825149246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6803","nonce":1825149246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6807","nonce":1825149246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6805","nonce":1825149246}]},"public_addr":"172.21.15.90:6801/1825149246","cluster_addr":"172.21.15.90:6803/1825149246","heartbeat_back_addr":"172.21.15.90:6807/1825149246","heartbeat_front_addr":"172.21.15.90:6805/1825149246","state":["exists","up"]},{"osd":2,"uuid":"40366536-0fcf-45a0-b8e3-49178401dc34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6801","nonce":1456846443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6803","nonce":1456846443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6807","nonce":1456846443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6805","nonce":1456846443}]},"public_addr":"172.21.15.97:6801/1456846443","cluster_addr":"172.21.15.97:6803/1456846443","heartbeat_back_addr":"172.21.15.97:6807/1456846443","heartbeat_front_addr":"172.21.15.97:6805/1456846443","state":["exists","up"]},{"osd":3,"uuid":"3062133e-000b-4421-9d53-ac434b9ed9b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6811","nonce":1962837804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6813","nonce":1962837804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6817","nonce":1962837804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6815","nonce":1962837804}]},"public_addr":"172.21.15.43:6811/1962837804","cluster_addr":"172.21.15.43:6813/1962837804","heartbeat_back_addr":"172.21.15.43:6817/1962837804","heartbeat_front_addr":"172.21.15.43:6815/1962837804","state":["exists","up"]},{"osd":4,"uuid":"2e73fa4b-f22c-4319-828d-87b91b9f19f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6809","nonce":181614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6811","nonce":181614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6815","nonce":181614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6813","nonce":181614173}]},"public_addr":"172.21.15.90:6809/181614173","cluster_addr":"172.21.15.90:6811/181614173","heartbeat_back_addr":"172.21.15.90:6815/181614173","heartbeat_front_addr":"172.21.15.90:6813/181614173","state":["exists","up"]},{"osd":5,"uuid":"1e1ad4f9-e724-440f-b515-3ec1ad97d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6809","nonce":1540039543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6811","nonce":1540039543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6815","nonce":1540039543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6813","nonce":1540039543}]},"public_addr":"172.21.15.97:6809/1540039543","cluster_addr":"172.21.15.97:6811/1540039543","heartbeat_back_addr":"172.21.15.97:6815/1540039543","heartbeat_front_addr":"172.21.15.97:6813/1540039543","state":["exists","up"]},{"osd":6,"uuid":"d34df8df-f40d-4159-8b29-5401b1b0e431","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6819","nonce":3785677637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6821","nonce":3785677637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6825","nonce":3785677637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6823","nonce":3785677637}]},"public_addr":"172.21.15.43:6819/3785677637","cluster_addr":"172.21.15.43:6821/3785677637","heartbeat_back_addr":"172.21.15.43:6825/3785677637","heartbeat_front_addr":"172.21.15.43:6823/3785677637","state":["exists","up"]},{"osd":7,"uuid":"b2e01078-528b-49fd-b000-9f584f784b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6817","nonce":236874962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6819","nonce":236874962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6823","nonce":236874962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6821","nonce":236874962}]},"public_addr":"172.21.15.90:6817/236874962","cluster_addr":"172.21.15.90:6819/236874962","heartbeat_back_addr":"172.21.15.90:6823/236874962","heartbeat_front_addr":"172.21.15.90:6821/236874962","state":["exists","up"]},{"osd":8,"uuid":"abef8f27-bf9b-4d63-bfbe-f967fc3d8b17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6817","nonce":1707327301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6819","nonce":1707327301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6823","nonce":1707327301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6821","nonce":1707327301}]},"public_addr":"172.21.15.97:6817/1707327301","cluster_addr":"172.21.15.97:6819/1707327301","heartbeat_back_addr":"172.21.15.97:6823/1707327301","heartbeat_front_addr":"172.21.15.97:6821/1707327301","state":["exists","up"]},{"osd":9,"uuid":"190ab8b2-a835-47e6-9b39-16a58331c58a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6827","nonce":3170306011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6829","nonce":3170306011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6833","nonce":3170306011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6831","nonce":3170306011}]},"public_addr":"172.21.15.43:6827/3170306011","cluster_addr":"172.21.15.43:6829/3170306011","heartbeat_back_addr":"172.21.15.43:6833/3170306011","heartbeat_front_addr":"172.21.15.43:6831/3170306011","state":["exists","up"]},{"osd":10,"uuid":"b32ab5fe-466f-4c8e-8696-cd9f820a3263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6825","nonce":59487041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6827","nonce":59487041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6831","nonce":59487041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6829","nonce":59487041}]},"public_addr":"172.21.15.90:6825/59487041","cluster_addr":"172.21.15.90:6827/59487041","heartbeat_back_addr":"172.21.15.90:6831/59487041","heartbeat_front_addr":"172.21.15.90:6829/59487041","state":["exists","up"]},{"osd":11,"uuid":"d0faa589-21c8-4a9f-8a73-813aafe6d33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6825","nonce":194699315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6827","nonce":194699315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6831","nonce":194699315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6829","nonce":194699315}]},"public_addr":"172.21.15.97:6825/194699315","cluster_addr":"172.21.15.97:6827/194699315","heartbeat_back_addr":"172.21.15.97:6831/194699315","heartbeat_front_addr":"172.21.15.97:6829/194699315","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:23.381247+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:48.843190+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:13.538979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:40.136983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:05.513790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:31.247090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:56.635221+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:22.667327+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:47.910422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:14.353914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:40.584503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:19:06.476766+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/2262294782":"2024-09-12T23:21:50.205362+0000","172.21.15.43:0/2398981254":"2024-09-12T23:21:50.205362+0000","172.21.15.43:0/481605249":"2024-09-12T23:21:50.205362+0000","172.21.15.43:0/1375379108":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1512201905":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/206044413":"2024-09-12T23:12:35.756598+0000","172.21.15.43:6800/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1579053924":"2024-09-12T23:21:50.205362+0000","172.21.15.43:6801/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:6800/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/2370495020":"2024-09-12T23:21:50.205362+0000","172.21.15.43:0/4006124758":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/76575371":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/120495444":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6800/2370495020":"2024-09-12T23:21:50.205362+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":4,"length":1},{"begin":13,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:32:09.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.078+0000 7f716ef33640 1 -- 172.21.15.43:0/3082894039 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f71540a2d80 msgr2=0x7f71540a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:09.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.078+0000 7f716ef33640 1 --2- 172.21.15.43:0/3082894039 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f71540a2d80 0x7f71540a5240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7150000c00 tx=0x7f7150032c30 comp rx=0 tx=0).stop 2024-09-11T23:32:09.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.078+0000 7f716ef33640 1 -- 172.21.15.43:0/3082894039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f716810c0d0 msgr2=0x7f716819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:09.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.078+0000 7f716ef33640 1 --2- 172.21.15.43:0/3082894039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f716810c0d0 0x7f716819cc40 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f716000c970 tx=0x7f716000ce30 comp rx=0 tx=0).stop 2024-09-11T23:32:09.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.078+0000 7f716ef33640 1 -- 172.21.15.43:0/3082894039 shutdown_connections 2024-09-11T23:32:09.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.078+0000 7f716ef33640 1 --2- 172.21.15.43:0/3082894039 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f71540a2d80 0x7f71540a5240 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.078+0000 7f716ef33640 1 --2- 172.21.15.43:0/3082894039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f716810c0d0 0x7f716819cc40 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.078+0000 7f716ef33640 1 --2- 172.21.15.43:0/3082894039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7168103620 0x7f71681988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.078+0000 7f716ef33640 1 --2- 172.21.15.43:0/3082894039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7168102d00 0x7f7168198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.078+0000 7f716ef33640 1 -- 172.21.15.43:0/3082894039 >> 172.21.15.43:0/3082894039 conn(0x7f71680fc6b0 msgr2=0x7f71680fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:09.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.078+0000 7f716ef33640 1 -- 172.21.15.43:0/3082894039 shutdown_connections 2024-09-11T23:32:09.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.078+0000 7f716ef33640 1 -- 172.21.15.43:0/3082894039 wait complete. 2024-09-11T23:32:09.090 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T23:15:17.058713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-11T23:19:56.221406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-11T23:19:58.705875+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 13, 'snap_epoch': 91, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T23:32:09.090 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T23:32:09.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.197+0000 7f8adb4af640 1 -- 172.21.15.43:0/758598226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad4100e00 msgr2=0x7f8ad4101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.197+0000 7f8adb4af640 1 --2- 172.21.15.43:0/758598226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad4100e00 0x7f8ad4101260 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009930 tx=0x7f8ac402a420 comp rx=0 tx=0).stop 2024-09-11T23:32:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.197+0000 7f8adb4af640 1 -- 172.21.15.43:0/758598226 shutdown_connections 2024-09-11T23:32:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.197+0000 7f8adb4af640 1 --2- 172.21.15.43:0/758598226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad41017a0 0x7f8ad410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.197+0000 7f8adb4af640 1 --2- 172.21.15.43:0/758598226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad4100e00 0x7f8ad4101260 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.197+0000 7f8adb4af640 1 --2- 172.21.15.43:0/758598226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ad41069b0 0x7f8ad4106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.197+0000 7f8adb4af640 1 -- 172.21.15.43:0/758598226 >> 172.21.15.43:0/758598226 conn(0x7f8ad40766f0 msgr2=0x7f8ad4076b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.197+0000 7f8adb4af640 1 -- 172.21.15.43:0/758598226 shutdown_connections 2024-09-11T23:32:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.197+0000 7f8adb4af640 1 -- 172.21.15.43:0/758598226 wait complete. 2024-09-11T23:32:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.198+0000 7f8adb4af640 1 Processor -- start 2024-09-11T23:32:09.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.198+0000 7f8adb4af640 1 -- start start 2024-09-11T23:32:09.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.198+0000 7f8adb4af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad4100e00 0x7f8ad41992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:09.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.198+0000 7f8adb4af640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad41017a0 0x7f8ad4199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:09.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.198+0000 7f8adb4af640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ad41069b0 0x7f8ad41a4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:09.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.198+0000 7f8adb4af640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad4110df0 con 0x7f8ad41017a0 2024-09-11T23:32:09.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.198+0000 7f8adb4af640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad4110c70 con 0x7f8ad41069b0 2024-09-11T23:32:09.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.198+0000 7f8adb4af640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad4110f70 con 0x7f8ad4100e00 2024-09-11T23:32:09.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.198+0000 7f8ad8a4a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad41017a0 0x7f8ad4199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.198+0000 7f8ad8a4a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad41017a0 0x7f8ad4199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38606/0 (socket says 172.21.15.43:38606) 2024-09-11T23:32:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.198+0000 7f8ad8a4a640 1 -- 172.21.15.43:0/809359240 learned_addr learned my addr 172.21.15.43:0/809359240 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.198+0000 7f8ad924b640 1 --2- 172.21.15.43:0/809359240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad4100e00 0x7f8ad41992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.198+0000 7f8ad9a4c640 1 --2- 172.21.15.43:0/809359240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ad41069b0 0x7f8ad41a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.199+0000 7f8ad8a4a640 1 -- 172.21.15.43:0/809359240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad4100e00 msgr2=0x7f8ad41992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.199+0000 7f8ad8a4a640 1 --2- 172.21.15.43:0/809359240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad4100e00 0x7f8ad41992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.199+0000 7f8ad8a4a640 1 -- 172.21.15.43:0/809359240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ad41069b0 msgr2=0x7f8ad41a4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.199+0000 7f8ad8a4a640 1 --2- 172.21.15.43:0/809359240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ad41069b0 0x7f8ad41a4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.199+0000 7f8ad8a4a640 1 -- 172.21.15.43:0/809359240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad41a4880 con 0x7f8ad41017a0 2024-09-11T23:32:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.199+0000 7f8ad9a4c640 1 --2- 172.21.15.43:0/809359240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ad41069b0 0x7f8ad41a4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:32:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.199+0000 7f8ad8a4a640 1 --2- 172.21.15.43:0/809359240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad41017a0 0x7f8ad4199800 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009230 tx=0x7f8ac4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.199+0000 7f8aca7fc640 1 -- 172.21.15.43:0/809359240 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac403f030 con 0x7f8ad41017a0 2024-09-11T23:32:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.199+0000 7f8aca7fc640 1 -- 172.21.15.43:0/809359240 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac4003ad0 con 0x7f8ad41017a0 2024-09-11T23:32:09.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.199+0000 7f8adb4af640 1 -- 172.21.15.43:0/809359240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad41a4a50 con 0x7f8ad41017a0 2024-09-11T23:32:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.199+0000 7f8aca7fc640 1 -- 172.21.15.43:0/809359240 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4035070 con 0x7f8ad41017a0 2024-09-11T23:32:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.199+0000 7f8adb4af640 1 -- 172.21.15.43:0/809359240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad41a4eb0 con 0x7f8ad41017a0 2024-09-11T23:32:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.200+0000 7f8adb4af640 1 -- 172.21.15.43:0/809359240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad4101260 con 0x7f8ad41017a0 2024-09-11T23:32:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.201+0000 7f8aca7fc640 1 -- 172.21.15.43:0/809359240 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8ac4042030 con 0x7f8ad41017a0 2024-09-11T23:32:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.202+0000 7f8aca7fc640 1 --2- 172.21.15.43:0/809359240 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f8ac00a2d00 0x7f8ac00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.202+0000 7f8aca7fc640 1 -- 172.21.15.43:0/809359240 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f8ac40efe40 con 0x7f8ad41017a0 2024-09-11T23:32:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.202+0000 7f8ad924b640 1 --2- 172.21.15.43:0/809359240 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f8ac00a2d00 0x7f8ac00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.202+0000 7f8ad924b640 1 --2- 172.21.15.43:0/809359240 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f8ac00a2d00 0x7f8ac00a51c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0005fd0 tx=0x7f8ad0005850 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:09.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.206+0000 7f8aca7fc640 1 -- 172.21.15.43:0/809359240 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ac40bc9d0 con 0x7f8ad41017a0 2024-09-11T23:32:09.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:09 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3082894039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:32:09.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.381+0000 7f8adb4af640 1 -- 172.21.15.43:0/809359240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8ad41089a0 con 0x7f8ad41017a0 2024-09-11T23:32:09.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.382+0000 7f8aca7fc640 1 -- 172.21.15.43:0/809359240 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f8ac40bc2d0 con 0x7f8ad41017a0 2024-09-11T23:32:09.382 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-09-11T23:32:09.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.384+0000 7f8adb4af640 1 -- 172.21.15.43:0/809359240 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f8ac00a2d00 msgr2=0x7f8ac00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:09.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.384+0000 7f8adb4af640 1 --2- 172.21.15.43:0/809359240 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f8ac00a2d00 0x7f8ac00a51c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0005fd0 tx=0x7f8ad0005850 comp rx=0 tx=0).stop 2024-09-11T23:32:09.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.384+0000 7f8adb4af640 1 -- 172.21.15.43:0/809359240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad41017a0 msgr2=0x7f8ad4199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:09.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.384+0000 7f8adb4af640 1 --2- 172.21.15.43:0/809359240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad41017a0 0x7f8ad4199800 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009230 tx=0x7f8ac4004270 comp rx=0 tx=0).stop 2024-09-11T23:32:09.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.384+0000 7f8adb4af640 1 -- 172.21.15.43:0/809359240 shutdown_connections 2024-09-11T23:32:09.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.384+0000 7f8adb4af640 1 --2- 172.21.15.43:0/809359240 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f8ac00a2d00 0x7f8ac00a51c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.384+0000 7f8adb4af640 1 --2- 172.21.15.43:0/809359240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ad41069b0 0x7f8ad41a4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.384+0000 7f8adb4af640 1 --2- 172.21.15.43:0/809359240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad41017a0 0x7f8ad4199800 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.384+0000 7f8adb4af640 1 --2- 172.21.15.43:0/809359240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad4100e00 0x7f8ad41992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.384+0000 7f8adb4af640 1 -- 172.21.15.43:0/809359240 >> 172.21.15.43:0/809359240 conn(0x7f8ad40766f0 msgr2=0x7f8ad40fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:09.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.384+0000 7f8adb4af640 1 -- 172.21.15.43:0/809359240 shutdown_connections 2024-09-11T23:32:09.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.384+0000 7f8adb4af640 1 -- 172.21.15.43:0/809359240 wait complete. 2024-09-11T23:32:09.395 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-11T23:32:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.502+0000 7fdc4c847640 1 -- 172.21.15.43:0/2652632263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc48100e00 msgr2=0x7fdc48101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.502+0000 7fdc4c847640 1 --2- 172.21.15.43:0/2652632263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc48100e00 0x7fdc48101260 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c009930 tx=0x7fdc3c02a420 comp rx=0 tx=0).stop 2024-09-11T23:32:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.503+0000 7fdc4c847640 1 -- 172.21.15.43:0/2652632263 shutdown_connections 2024-09-11T23:32:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.503+0000 7fdc4c847640 1 --2- 172.21.15.43:0/2652632263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc481017a0 0x7fdc4810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.503+0000 7fdc4c847640 1 --2- 172.21.15.43:0/2652632263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc48100e00 0x7fdc48101260 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.503+0000 7fdc4c847640 1 --2- 172.21.15.43:0/2652632263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc481069b0 0x7fdc48106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.503+0000 7fdc4c847640 1 -- 172.21.15.43:0/2652632263 >> 172.21.15.43:0/2652632263 conn(0x7fdc480766f0 msgr2=0x7fdc48076b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.503+0000 7fdc4c847640 1 -- 172.21.15.43:0/2652632263 shutdown_connections 2024-09-11T23:32:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.503+0000 7fdc4c847640 1 -- 172.21.15.43:0/2652632263 wait complete. 2024-09-11T23:32:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.503+0000 7fdc4c847640 1 Processor -- start 2024-09-11T23:32:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.503+0000 7fdc4c847640 1 -- start start 2024-09-11T23:32:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.504+0000 7fdc4c847640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc48100e00 0x7fdc481983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.504+0000 7fdc4c847640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc481017a0 0x7fdc481988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.504+0000 7fdc4c847640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc481069b0 0x7fdc4819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.504+0000 7fdc4c847640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdc48110df0 con 0x7fdc481017a0 2024-09-11T23:32:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.504+0000 7fdc4c847640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdc48110c70 con 0x7fdc481069b0 2024-09-11T23:32:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.504+0000 7fdc4c847640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdc48110f70 con 0x7fdc48100e00 2024-09-11T23:32:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.504+0000 7fdc45d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc481017a0 0x7fdc481988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.504+0000 7fdc45d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc481017a0 0x7fdc481988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38628/0 (socket says 172.21.15.43:38628) 2024-09-11T23:32:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.504+0000 7fdc45d9b640 1 -- 172.21.15.43:0/1049542108 learned_addr learned my addr 172.21.15.43:0/1049542108 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.504+0000 7fdc4659c640 1 --2- 172.21.15.43:0/1049542108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc48100e00 0x7fdc481983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.504+0000 7fdc46d9d640 1 --2- 172.21.15.43:0/1049542108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc481069b0 0x7fdc4819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.504+0000 7fdc45d9b640 1 -- 172.21.15.43:0/1049542108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc48100e00 msgr2=0x7fdc481983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.504+0000 7fdc45d9b640 1 --2- 172.21.15.43:0/1049542108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc48100e00 0x7fdc481983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.504+0000 7fdc45d9b640 1 -- 172.21.15.43:0/1049542108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc481069b0 msgr2=0x7fdc4819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.504+0000 7fdc45d9b640 1 --2- 172.21.15.43:0/1049542108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc481069b0 0x7fdc4819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.504+0000 7fdc45d9b640 1 -- 172.21.15.43:0/1049542108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4819d390 con 0x7fdc481017a0 2024-09-11T23:32:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.504+0000 7fdc46d9d640 1 --2- 172.21.15.43:0/1049542108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc481069b0 0x7fdc4819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:32:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.504+0000 7fdc45d9b640 1 --2- 172.21.15.43:0/1049542108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc481017a0 0x7fdc481988e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c009230 tx=0x7fdc3c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.505+0000 7fdc377fe640 1 -- 172.21.15.43:0/1049542108 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c03f030 con 0x7fdc481017a0 2024-09-11T23:32:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.505+0000 7fdc377fe640 1 -- 172.21.15.43:0/1049542108 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc3c003ad0 con 0x7fdc481017a0 2024-09-11T23:32:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.505+0000 7fdc4c847640 1 -- 172.21.15.43:0/1049542108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc4819d620 con 0x7fdc481017a0 2024-09-11T23:32:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.505+0000 7fdc377fe640 1 -- 172.21.15.43:0/1049542108 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c035070 con 0x7fdc481017a0 2024-09-11T23:32:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.505+0000 7fdc4c847640 1 -- 172.21.15.43:0/1049542108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc481a4f10 con 0x7fdc481017a0 2024-09-11T23:32:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.506+0000 7fdc4c847640 1 -- 172.21.15.43:0/1049542108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc48101260 con 0x7fdc481017a0 2024-09-11T23:32:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.507+0000 7fdc377fe640 1 -- 172.21.15.43:0/1049542108 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fdc3c042030 con 0x7fdc481017a0 2024-09-11T23:32:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.508+0000 7fdc377fe640 1 --2- 172.21.15.43:0/1049542108 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdc300a2d00 0x7fdc300a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.508+0000 7fdc4659c640 1 --2- 172.21.15.43:0/1049542108 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdc300a2d00 0x7fdc300a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.508+0000 7fdc377fe640 1 -- 172.21.15.43:0/1049542108 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fdc3c0ef9e0 con 0x7fdc481017a0 2024-09-11T23:32:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.508+0000 7fdc4659c640 1 --2- 172.21.15.43:0/1049542108 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdc300a2d00 0x7fdc300a51c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdc38007e40 tx=0x7fdc380075b0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:09.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.512+0000 7fdc377fe640 1 -- 172.21.15.43:0/1049542108 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdc3c0bc570 con 0x7fdc481017a0 2024-09-11T23:32:09.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:09 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3082894039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:32:09.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:09 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3082894039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:32:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.693+0000 7fdc4c847640 1 -- 172.21.15.43:0/1049542108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdc48109f70 con 0x7fdc481017a0 2024-09-11T23:32:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.694+0000 7fdc377fe640 1 -- 172.21.15.43:0/1049542108 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7fdc3c0bbe70 con 0x7fdc481017a0 2024-09-11T23:32:09.694 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-09-11T23:32:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.696+0000 7fdc4c847640 1 -- 172.21.15.43:0/1049542108 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdc300a2d00 msgr2=0x7fdc300a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.696+0000 7fdc4c847640 1 --2- 172.21.15.43:0/1049542108 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdc300a2d00 0x7fdc300a51c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdc38007e40 tx=0x7fdc380075b0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.696+0000 7fdc4c847640 1 -- 172.21.15.43:0/1049542108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc481017a0 msgr2=0x7fdc481988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.696+0000 7fdc4c847640 1 --2- 172.21.15.43:0/1049542108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc481017a0 0x7fdc481988e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c009230 tx=0x7fdc3c004270 comp rx=0 tx=0).stop 2024-09-11T23:32:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.696+0000 7fdc4c847640 1 -- 172.21.15.43:0/1049542108 shutdown_connections 2024-09-11T23:32:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.696+0000 7fdc4c847640 1 --2- 172.21.15.43:0/1049542108 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fdc300a2d00 0x7fdc300a51c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.696+0000 7fdc4c847640 1 --2- 172.21.15.43:0/1049542108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc481069b0 0x7fdc4819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.696+0000 7fdc4c847640 1 --2- 172.21.15.43:0/1049542108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc481017a0 0x7fdc481988e0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.696+0000 7fdc4c847640 1 --2- 172.21.15.43:0/1049542108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc48100e00 0x7fdc481983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.696+0000 7fdc4c847640 1 -- 172.21.15.43:0/1049542108 >> 172.21.15.43:0/1049542108 conn(0x7fdc480766f0 msgr2=0x7fdc48069770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.696+0000 7fdc4c847640 1 -- 172.21.15.43:0/1049542108 shutdown_connections 2024-09-11T23:32:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.696+0000 7fdc4c847640 1 -- 172.21.15.43:0/1049542108 wait complete. 2024-09-11T23:32:09.707 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-11T23:32:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.814+0000 7fab11237640 1 -- 172.21.15.43:0/521756160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab0c1069b0 msgr2=0x7fab0c106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.814+0000 7fab11237640 1 --2- 172.21.15.43:0/521756160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab0c1069b0 0x7fab0c106d90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7faafc009930 tx=0x7faafc02a420 comp rx=0 tx=0).stop 2024-09-11T23:32:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.814+0000 7fab11237640 1 -- 172.21.15.43:0/521756160 shutdown_connections 2024-09-11T23:32:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.814+0000 7fab11237640 1 --2- 172.21.15.43:0/521756160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab0c1017a0 0x7fab0c10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.814+0000 7fab11237640 1 --2- 172.21.15.43:0/521756160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab0c100e00 0x7fab0c101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.814+0000 7fab11237640 1 --2- 172.21.15.43:0/521756160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab0c1069b0 0x7fab0c106d90 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.814+0000 7fab11237640 1 -- 172.21.15.43:0/521756160 >> 172.21.15.43:0/521756160 conn(0x7fab0c0766f0 msgr2=0x7fab0c076b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.814+0000 7fab11237640 1 -- 172.21.15.43:0/521756160 shutdown_connections 2024-09-11T23:32:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.814+0000 7fab11237640 1 -- 172.21.15.43:0/521756160 wait complete. 2024-09-11T23:32:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.815+0000 7fab11237640 1 Processor -- start 2024-09-11T23:32:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.815+0000 7fab11237640 1 -- start start 2024-09-11T23:32:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.816+0000 7fab11237640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab0c100e00 0x7fab0c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.816+0000 7fab11237640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab0c1017a0 0x7fab0c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.816+0000 7fab11237640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab0c1069b0 0x7fab0c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.816+0000 7fab11237640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c110df0 con 0x7fab0c1017a0 2024-09-11T23:32:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.816+0000 7fab11237640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c110c70 con 0x7fab0c1069b0 2024-09-11T23:32:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.816+0000 7fab11237640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c110f70 con 0x7fab0c100e00 2024-09-11T23:32:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.816+0000 7fab0a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab0c1017a0 0x7fab0c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.816+0000 7fab0a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab0c1017a0 0x7fab0c198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38636/0 (socket says 172.21.15.43:38636) 2024-09-11T23:32:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.816+0000 7fab0a59c640 1 -- 172.21.15.43:0/1301410952 learned_addr learned my addr 172.21.15.43:0/1301410952 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.816+0000 7fab0b59e640 1 --2- 172.21.15.43:0/1301410952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab0c1069b0 0x7fab0c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.816+0000 7fab0ad9d640 1 --2- 172.21.15.43:0/1301410952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab0c100e00 0x7fab0c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.816+0000 7fab0a59c640 1 -- 172.21.15.43:0/1301410952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab0c100e00 msgr2=0x7fab0c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.816+0000 7fab0a59c640 1 --2- 172.21.15.43:0/1301410952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab0c100e00 0x7fab0c198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.816+0000 7fab0a59c640 1 -- 172.21.15.43:0/1301410952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab0c1069b0 msgr2=0x7fab0c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.816+0000 7fab0a59c640 1 --2- 172.21.15.43:0/1301410952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab0c1069b0 0x7fab0c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:09.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.816+0000 7fab0a59c640 1 -- 172.21.15.43:0/1301410952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c19d3f0 con 0x7fab0c1017a0 2024-09-11T23:32:09.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.816+0000 7fab0a59c640 1 --2- 172.21.15.43:0/1301410952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab0c1017a0 0x7fab0c198940 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fab0000c890 tx=0x7fab0000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:09.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.816+0000 7faaf3fff640 1 -- 172.21.15.43:0/1301410952 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00004220 con 0x7fab0c1017a0 2024-09-11T23:32:09.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.817+0000 7faaf3fff640 1 -- 172.21.15.43:0/1301410952 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab000043c0 con 0x7fab0c1017a0 2024-09-11T23:32:09.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.817+0000 7fab11237640 1 -- 172.21.15.43:0/1301410952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0c19d6e0 con 0x7fab0c1017a0 2024-09-11T23:32:09.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.817+0000 7faaf3fff640 1 -- 172.21.15.43:0/1301410952 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab000049e0 con 0x7fab0c1017a0 2024-09-11T23:32:09.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.817+0000 7fab11237640 1 -- 172.21.15.43:0/1301410952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab0c1a4f70 con 0x7fab0c1017a0 2024-09-11T23:32:09.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.817+0000 7fab11237640 1 -- 172.21.15.43:0/1301410952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab0c101260 con 0x7fab0c1017a0 2024-09-11T23:32:09.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.819+0000 7faaf3fff640 1 -- 172.21.15.43:0/1301410952 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fab000079b0 con 0x7fab0c1017a0 2024-09-11T23:32:09.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.819+0000 7faaf3fff640 1 --2- 172.21.15.43:0/1301410952 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7faaf40a2d00 0x7faaf40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:09.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.819+0000 7fab0ad9d640 1 --2- 172.21.15.43:0/1301410952 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7faaf40a2d00 0x7faaf40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:09.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.819+0000 7faaf3fff640 1 -- 172.21.15.43:0/1301410952 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fab000cf430 con 0x7fab0c1017a0 2024-09-11T23:32:09.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.819+0000 7fab0ad9d640 1 --2- 172.21.15.43:0/1301410952 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7faaf40a2d00 0x7faaf40a51c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faafc009900 tx=0x7faafc0058d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.821+0000 7faaf3fff640 1 -- 172.21.15.43:0/1301410952 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab00014030 con 0x7fab0c1017a0 2024-09-11T23:32:10.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.997+0000 7fab11237640 1 -- 172.21.15.43:0/1301410952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fab0c1a52a0 con 0x7fab0c1017a0 2024-09-11T23:32:10.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.997+0000 7faaf3fff640 1 -- 172.21.15.43:0/1301410952 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7fab0009b9d0 con 0x7fab0c1017a0 2024-09-11T23:32:10.001 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-09-11T23:32:10.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.999+0000 7fab11237640 1 -- 172.21.15.43:0/1301410952 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7faaf40a2d00 msgr2=0x7faaf40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:10.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.999+0000 7fab11237640 1 --2- 172.21.15.43:0/1301410952 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7faaf40a2d00 0x7faaf40a51c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faafc009900 tx=0x7faafc0058d0 comp rx=0 tx=0).stop 2024-09-11T23:32:10.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.999+0000 7fab11237640 1 -- 172.21.15.43:0/1301410952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab0c1017a0 msgr2=0x7fab0c198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:10.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.999+0000 7fab11237640 1 --2- 172.21.15.43:0/1301410952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab0c1017a0 0x7fab0c198940 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fab0000c890 tx=0x7fab0000cd50 comp rx=0 tx=0).stop 2024-09-11T23:32:10.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.999+0000 7fab11237640 1 -- 172.21.15.43:0/1301410952 shutdown_connections 2024-09-11T23:32:10.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.999+0000 7fab11237640 1 --2- 172.21.15.43:0/1301410952 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7faaf40a2d00 0x7faaf40a51c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:10.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.999+0000 7fab11237640 1 --2- 172.21.15.43:0/1301410952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab0c1069b0 0x7fab0c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:10.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.999+0000 7fab11237640 1 --2- 172.21.15.43:0/1301410952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab0c1017a0 0x7fab0c198940 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:10.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:09.999+0000 7fab11237640 1 --2- 172.21.15.43:0/1301410952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab0c100e00 0x7fab0c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:10.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.000+0000 7fab11237640 1 -- 172.21.15.43:0/1301410952 >> 172.21.15.43:0/1301410952 conn(0x7fab0c0766f0 msgr2=0x7fab0c104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:10.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.000+0000 7fab11237640 1 -- 172.21.15.43:0/1301410952 shutdown_connections 2024-09-11T23:32:10.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.000+0000 7fab11237640 1 -- 172.21.15.43:0/1301410952 wait complete. 2024-09-11T23:32:10.011 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-11T23:32:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.131+0000 7fac76b18640 1 -- 172.21.15.43:0/1643919071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac680ab1f0 msgr2=0x7fac680ab630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.131+0000 7fac76b18640 1 --2- 172.21.15.43:0/1643919071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac680ab1f0 0x7fac680ab630 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009930 tx=0x7fac5c02a460 comp rx=0 tx=0).stop 2024-09-11T23:32:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.133+0000 7fac76b18640 1 -- 172.21.15.43:0/1643919071 shutdown_connections 2024-09-11T23:32:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.133+0000 7fac76b18640 1 --2- 172.21.15.43:0/1643919071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac680abb70 0x7fac680b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.133+0000 7fac76b18640 1 --2- 172.21.15.43:0/1643919071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac680ab1f0 0x7fac680ab630 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.133+0000 7fac76b18640 1 --2- 172.21.15.43:0/1643919071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac680aa610 0x7fac680aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.133+0000 7fac76b18640 1 -- 172.21.15.43:0/1643919071 >> 172.21.15.43:0/1643919071 conn(0x7fac6801a0c0 msgr2=0x7fac6801a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.134+0000 7fac76b18640 1 -- 172.21.15.43:0/1643919071 shutdown_connections 2024-09-11T23:32:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.135+0000 7fac76b18640 1 -- 172.21.15.43:0/1643919071 wait complete. 2024-09-11T23:32:10.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.136+0000 7fac76b18640 1 Processor -- start 2024-09-11T23:32:10.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.136+0000 7fac76b18640 1 -- start start 2024-09-11T23:32:10.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.136+0000 7fac76b18640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac680aa610 0x7fac6813ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:10.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.136+0000 7fac76b18640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac680ab1f0 0x7fac6813b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:10.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.136+0000 7fac76b18640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac680abb70 0x7fac6813f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:10.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.136+0000 7fac76b18640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fac680b49d0 con 0x7fac680abb70 2024-09-11T23:32:10.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.136+0000 7fac76b18640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac680b4850 con 0x7fac680ab1f0 2024-09-11T23:32:10.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.136+0000 7fac76b18640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fac680b4b50 con 0x7fac680aa610 2024-09-11T23:32:10.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.136+0000 7fac748b4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac680aa610 0x7fac6813ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:10.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.136+0000 7fac748b4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac680aa610 0x7fac6813ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:34986/0 (socket says 172.21.15.43:34986) 2024-09-11T23:32:10.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.137+0000 7fac748b4640 1 -- 172.21.15.43:0/3805089278 learned_addr learned my addr 172.21.15.43:0/3805089278 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:32:10.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.137+0000 7fac750b5640 1 --2- 172.21.15.43:0/3805089278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac680abb70 0x7fac6813f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:10.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.137+0000 7fac748b4640 1 -- 172.21.15.43:0/3805089278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac680ab1f0 msgr2=0x7fac6813b3c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T23:32:10.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.138+0000 7fac748b4640 1 --2- 172.21.15.43:0/3805089278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac680ab1f0 0x7fac6813b3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:10.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.138+0000 7fac748b4640 1 -- 172.21.15.43:0/3805089278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac680abb70 msgr2=0x7fac6813f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:10.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.138+0000 7fac748b4640 1 --2- 172.21.15.43:0/3805089278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac680abb70 0x7fac6813f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:10.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.138+0000 7fac748b4640 1 -- 172.21.15.43:0/3805089278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6813fe70 con 0x7fac680aa610 2024-09-11T23:32:10.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.138+0000 7fac750b5640 1 --2- 172.21.15.43:0/3805089278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac680abb70 0x7fac6813f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:32:10.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.138+0000 7fac748b4640 1 --2- 172.21.15.43:0/3805089278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac680aa610 0x7fac6813ae80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fac6400b4a0 tx=0x7fac6400b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:10.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.138+0000 7fac6dffb640 1 -- 172.21.15.43:0/3805089278 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64014070 con 0x7fac680aa610 2024-09-11T23:32:10.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.139+0000 7fac76b18640 1 -- 172.21.15.43:0/3805089278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac68140160 con 0x7fac680aa610 2024-09-11T23:32:10.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.139+0000 7fac76b18640 1 -- 172.21.15.43:0/3805089278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac6800b790 con 0x7fac680aa610 2024-09-11T23:32:10.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.139+0000 7fac6dffb640 1 -- 172.21.15.43:0/3805089278 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac64010af0 con 0x7fac680aa610 2024-09-11T23:32:10.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.139+0000 7fac6dffb640 1 -- 172.21.15.43:0/3805089278 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64017d10 con 0x7fac680aa610 2024-09-11T23:32:10.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.141+0000 7fac76b18640 1 -- 172.21.15.43:0/3805089278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac54005290 con 0x7fac680aa610 2024-09-11T23:32:10.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.144+0000 7fac6dffb640 1 -- 172.21.15.43:0/3805089278 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fac64017070 con 0x7fac680aa610 2024-09-11T23:32:10.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.145+0000 7fac6dffb640 1 --2- 172.21.15.43:0/3805089278 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fac600a7310 0x7fac600a97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:32:10.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.145+0000 7fac6ffff640 1 --2- 172.21.15.43:0/3805089278 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fac600a7310 0x7fac600a97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:32:10.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.146+0000 7fac6dffb640 1 -- 172.21.15.43:0/3805089278 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8615+0+0 (secure 0 0 0) 0x7fac640ce6f0 con 0x7fac680aa610 2024-09-11T23:32:10.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.147+0000 7fac6ffff640 1 --2- 172.21.15.43:0/3805089278 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fac600a7310 0x7fac600a97d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fac5c02aa50 tx=0x7fac5c005fd0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:32:10.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.153+0000 7fac6dffb640 1 -- 172.21.15.43:0/3805089278 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac6409b3e0 con 0x7fac680aa610 2024-09-11T23:32:10.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:10 smithi043 ceph-mon[30950]: pgmap v319: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-11T23:32:10.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:10 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/809359240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:32:10.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:10 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1049542108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:32:10.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:10 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1301410952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:32:10.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:10 smithi090 ceph-mon[35130]: pgmap v319: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-11T23:32:10.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:10 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/809359240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:32:10.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:10 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1049542108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:32:10.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:10 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1301410952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:32:10.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:10 smithi097 ceph-mon[35562]: pgmap v319: 97 pgs: 97 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-11T23:32:10.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:10 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/809359240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:32:10.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:10 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1049542108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:32:10.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:10 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1301410952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:32:10.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.606+0000 7fac76b18640 1 -- 172.21.15.43:0/3805089278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fac54005850 con 0x7fac680aa610 2024-09-11T23:32:10.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.619+0000 7fac6dffb640 1 -- 172.21.15.43:0/3805089278 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+740900 (secure 0 0 0) 0x7fac6409ace0 con 0x7fac680aa610 2024-09-11T23:32:10.631 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:32:10.631 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:32:10.632 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 26, 2024-09-11T23:32:10.632 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.632 INFO:teuthology.orchestra.run.smithi043.stdout: "active_gid": 15126, 2024-09-11T23:32:10.632 INFO:teuthology.orchestra.run.smithi043.stdout: "active_name": "x", 2024-09-11T23:32:10.632 INFO:teuthology.orchestra.run.smithi043.stdout: "active_addrs": { 2024-09-11T23:32:10.632 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-11T23:32:10.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.632 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-11T23:32:10.632 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6800", 2024-09-11T23:32:10.633 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2510386404 2024-09-11T23:32:10.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.633 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-09-11T23:32:10.633 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6801", 2024-09-11T23:32:10.633 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2510386404 2024-09-11T23:32:10.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.633 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-11T23:32:10.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.633 INFO:teuthology.orchestra.run.smithi043.stdout: "active_addr": "172.21.15.43:6801/2510386404", 2024-09-11T23:32:10.633 INFO:teuthology.orchestra.run.smithi043.stdout: "active_change": "2024-09-11T23:21:50.205571+0000", 2024-09-11T23:32:10.634 INFO:teuthology.orchestra.run.smithi043.stdout: "active_mgr_features": 4540701547738038271, 2024-09-11T23:32:10.634 INFO:teuthology.orchestra.run.smithi043.stdout: "available": true, 2024-09-11T23:32:10.634 INFO:teuthology.orchestra.run.smithi043.stdout: "standbys": [ 2024-09-11T23:32:10.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.634 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 24808, 2024-09-11T23:32:10.634 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "z", 2024-09-11T23:32:10.634 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_features": 4540701547738038271, 2024-09-11T23:32:10.634 INFO:teuthology.orchestra.run.smithi043.stdout: "available_modules": [ 2024-09-11T23:32:10.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.635 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "alerts", 2024-09-11T23:32:10.635 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:10.635 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:10.635 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:10.635 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-11T23:32:10.635 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-11T23:32:10.635 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:10.635 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.635 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.635 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-11T23:32:10.635 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.635 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.635 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.636 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T23:32:10.636 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.636 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.636 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.636 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:10.636 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:10.636 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.637 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.637 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.637 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.637 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.637 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.637 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.637 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.637 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.637 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.637 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.637 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.637 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.638 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.638 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.639 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.639 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.639 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.639 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:10.639 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:10.639 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.639 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.639 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.640 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.640 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.640 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.640 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.640 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.640 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.640 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.640 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.640 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:10.640 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:10.640 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.640 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.641 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.641 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:10.641 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.641 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.641 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.641 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.641 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.642 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.642 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.642 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.642 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.642 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.642 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.642 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.642 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.642 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.643 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:10.643 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:10.643 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.643 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.643 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.643 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.643 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.643 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.643 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.643 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.643 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.643 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_destination": { 2024-09-11T23:32:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_destination", 2024-09-11T23:32:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.644 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email address to send alerts to", 2024-09-11T23:32:10.645 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.645 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.645 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.645 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_from_name": { 2024-09-11T23:32:10.645 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_from_name", 2024-09-11T23:32:10.645 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.645 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.645 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.645 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "Ceph", 2024-09-11T23:32:10.645 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email From: name", 2024-09-11T23:32:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_host": { 2024-09-11T23:32:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_host", 2024-09-11T23:32:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.646 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.647 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.647 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.647 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.647 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.647 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP server", 2024-09-11T23:32:10.647 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.647 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.647 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_password": { 2024-09-11T23:32:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_password", 2024-09-11T23:32:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Password to authenticate with", 2024-09-11T23:32:10.648 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.649 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.649 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.649 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_port": { 2024-09-11T23:32:10.649 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_port", 2024-09-11T23:32:10.650 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.650 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.650 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "465", 2024-09-11T23:32:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP port", 2024-09-11T23:32:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.651 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_sender": { 2024-09-11T23:32:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_sender", 2024-09-11T23:32:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP envelope sender", 2024-09-11T23:32:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.652 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.653 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_ssl": { 2024-09-11T23:32:10.653 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_ssl", 2024-09-11T23:32:10.653 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.653 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.654 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.654 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.654 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.654 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.654 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.654 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T23:32:10.654 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.654 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.654 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.654 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_user": { 2024-09-11T23:32:10.654 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_user", 2024-09-11T23:32:10.655 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.655 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.655 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.655 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.655 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.655 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.655 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.655 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "User to authenticate as", 2024-09-11T23:32:10.655 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.655 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.655 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.655 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.655 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.656 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "balancer", 2024-09-11T23:32:10.656 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:10.656 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:10.656 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:10.656 INFO:teuthology.orchestra.run.smithi043.stdout: "active": { 2024-09-11T23:32:10.656 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "active", 2024-09-11T23:32:10.656 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.656 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.656 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.657 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.657 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.657 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.657 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.657 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T23:32:10.657 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.657 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.657 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.657 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_time": { 2024-09-11T23:32:10.657 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_time", 2024-09-11T23:32:10.657 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.658 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.658 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.658 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0000", 2024-09-11T23:32:10.658 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.658 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.658 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.658 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T23:32:10.658 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:32:10.658 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.658 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.658 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_weekday": { 2024-09-11T23:32:10.659 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_weekday", 2024-09-11T23:32:10.659 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-11T23:32:10.659 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.659 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.659 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-11T23:32:10.659 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-09-11T23:32:10.659 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-09-11T23:32:10.660 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.660 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T23:32:10.660 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:32:10.660 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.660 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.660 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_max_iterations": { 2024-09-11T23:32:10.660 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_max_iterations", 2024-09-11T23:32:10.660 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-11T23:32:10.661 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.661 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.661 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "25", 2024-09-11T23:32:10.661 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-11T23:32:10.661 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "250", 2024-09-11T23:32:10.661 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.661 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T23:32:10.661 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.661 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.661 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.661 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_metrics": { 2024-09-11T23:32:10.661 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_metrics", 2024-09-11T23:32:10.662 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.662 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.662 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.662 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T23:32:10.662 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.662 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.662 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.662 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T23:32:10.662 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T23:32:10.662 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.663 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.663 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_step": { 2024-09-11T23:32:10.663 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_step", 2024-09-11T23:32:10.663 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-11T23:32:10.663 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.663 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.663 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.5", 2024-09-11T23:32:10.663 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0.001", 2024-09-11T23:32:10.663 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "0.999", 2024-09-11T23:32:10.663 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.664 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "aggressiveness of optimization", 2024-09-11T23:32:10.664 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T23:32:10.664 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.664 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.664 INFO:teuthology.orchestra.run.smithi043.stdout: "end_time": { 2024-09-11T23:32:10.664 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_time", 2024-09-11T23:32:10.664 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.664 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.664 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.665 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2359", 2024-09-11T23:32:10.665 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.665 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.665 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.666 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T23:32:10.666 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:32:10.666 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.666 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.666 INFO:teuthology.orchestra.run.smithi043.stdout: "end_weekday": { 2024-09-11T23:32:10.667 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_weekday", 2024-09-11T23:32:10.667 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-11T23:32:10.667 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.667 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.667 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-11T23:32:10.668 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-09-11T23:32:10.668 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-09-11T23:32:10.668 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.668 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T23:32:10.668 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:32:10.669 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.669 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.669 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:10.669 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:10.670 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.670 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.670 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.670 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.670 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.670 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.671 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.671 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.671 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.671 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.671 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.672 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.672 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.672 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.672 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.672 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.673 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.673 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.673 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:10.673 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:10.673 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.674 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.674 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.674 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.674 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.675 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.675 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.675 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.675 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.675 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.676 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.676 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:10.676 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:10.676 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.677 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.677 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.677 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:10.677 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.677 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.677 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.678 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.678 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.678 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.678 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.678 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.679 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.679 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.679 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.679 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.679 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.680 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.680 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:10.680 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:10.680 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.681 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.681 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.681 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.681 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.681 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.681 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.682 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.682 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.682 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.682 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.683 INFO:teuthology.orchestra.run.smithi043.stdout: "min_score": { 2024-09-11T23:32:10.683 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_score", 2024-09-11T23:32:10.683 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-11T23:32:10.683 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.683 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.683 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-11T23:32:10.684 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.684 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.684 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.684 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T23:32:10.684 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.685 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.685 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.685 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-09-11T23:32:10.685 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-09-11T23:32:10.686 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.686 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.686 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.686 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "upmap", 2024-09-11T23:32:10.686 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.686 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.687 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.687 INFO:teuthology.orchestra.run.smithi043.stdout: "crush-compat", 2024-09-11T23:32:10.687 INFO:teuthology.orchestra.run.smithi043.stdout: "none", 2024-09-11T23:32:10.687 INFO:teuthology.orchestra.run.smithi043.stdout: "read", 2024-09-11T23:32:10.687 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap", 2024-09-11T23:32:10.688 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap-read" 2024-09-11T23:32:10.688 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.688 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Balancer mode", 2024-09-11T23:32:10.688 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.688 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.689 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.689 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ids": { 2024-09-11T23:32:10.689 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_ids", 2024-09-11T23:32:10.689 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.689 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.689 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.689 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.689 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.689 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.690 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.690 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T23:32:10.690 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.690 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.690 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.690 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-11T23:32:10.691 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-11T23:32:10.691 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:10.691 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.691 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.691 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-11T23:32:10.691 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.691 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.691 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.691 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T23:32:10.691 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.691 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.691 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.691 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_deviation": { 2024-09-11T23:32:10.692 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_deviation", 2024-09-11T23:32:10.692 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.692 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.692 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.692 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-11T23:32:10.692 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-11T23:32:10.692 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.692 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.692 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T23:32:10.692 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T23:32:10.692 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.692 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.693 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_optimizations": { 2024-09-11T23:32:10.693 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_optimizations", 2024-09-11T23:32:10.693 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-11T23:32:10.693 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.693 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.693 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:10.693 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.693 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.693 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.693 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T23:32:10.693 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.693 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.694 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.694 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.694 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.694 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm", 2024-09-11T23:32:10.694 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:10.694 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:10.694 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:10.694 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_down_multiplier": { 2024-09-11T23:32:10.694 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_down_multiplier", 2024-09-11T23:32:10.694 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-11T23:32:10.695 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.695 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.695 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-09-11T23:32:10.695 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.695 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.695 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.695 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T23:32:10.695 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.695 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.695 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.696 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_refresh_rate": { 2024-09-11T23:32:10.696 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_refresh_rate", 2024-09-11T23:32:10.696 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:10.696 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.696 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.696 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "20", 2024-09-11T23:32:10.696 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.696 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.696 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.697 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T23:32:10.697 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.697 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.697 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.697 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_starting_port": { 2024-09-11T23:32:10.697 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_starting_port", 2024-09-11T23:32:10.697 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.697 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.697 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.697 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4721", 2024-09-11T23:32:10.697 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.697 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.697 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.698 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T23:32:10.698 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.698 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.698 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.698 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_ptrace": { 2024-09-11T23:32:10.698 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_ptrace", 2024-09-11T23:32:10.698 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.699 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.699 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.699 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.699 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.699 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.699 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.700 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T23:32:10.700 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T23:32:10.700 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.700 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.700 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_interval": { 2024-09-11T23:32:10.700 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_interval", 2024-09-11T23:32:10.700 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:10.700 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.700 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.700 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-11T23:32:10.700 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T23:32:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_memory_target_ratio": { 2024-09-11T23:32:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T23:32:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-11T23:32:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.701 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.7", 2024-09-11T23:32:10.702 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.702 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.702 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.702 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T23:32:10.702 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.702 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.702 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.702 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm_log_destination": { 2024-09-11T23:32:10.702 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm_log_destination", 2024-09-11T23:32:10.703 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.703 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.703 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.703 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.703 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.703 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.703 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.703 INFO:teuthology.orchestra.run.smithi043.stdout: "file", 2024-09-11T23:32:10.703 INFO:teuthology.orchestra.run.smithi043.stdout: "file,syslog", 2024-09-11T23:32:10.703 INFO:teuthology.orchestra.run.smithi043.stdout: "syslog" 2024-09-11T23:32:10.703 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.703 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T23:32:10.703 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.704 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.704 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.704 INFO:teuthology.orchestra.run.smithi043.stdout: "cgroups_split": { 2024-09-11T23:32:10.704 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cgroups_split", 2024-09-11T23:32:10.704 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.704 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.704 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.704 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.704 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.704 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.704 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.705 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T23:32:10.705 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.705 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.705 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.705 INFO:teuthology.orchestra.run.smithi043.stdout: "config_checks_enabled": { 2024-09-11T23:32:10.705 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_checks_enabled", 2024-09-11T23:32:10.705 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.705 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.705 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.706 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.706 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.706 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.706 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.706 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T23:32:10.706 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.706 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.706 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.706 INFO:teuthology.orchestra.run.smithi043.stdout: "config_dashboard": { 2024-09-11T23:32:10.706 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_dashboard", 2024-09-11T23:32:10.706 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.706 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.707 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.707 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.707 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.707 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.707 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.707 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T23:32:10.707 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.707 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.707 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.707 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_alertmanager": { 2024-09-11T23:32:10.707 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_alertmanager", 2024-09-11T23:32:10.707 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.708 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.708 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.708 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-11T23:32:10.708 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.708 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.708 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.708 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-11T23:32:10.708 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.708 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.708 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.708 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_base": { 2024-09-11T23:32:10.709 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_base", 2024-09-11T23:32:10.709 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.709 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.709 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.709 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T23:32:10.709 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.709 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.709 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.709 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Container image name, without the tag", 2024-09-11T23:32:10.709 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.709 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.709 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.710 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_elasticsearch": { 2024-09-11T23:32:10.710 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_elasticsearch", 2024-09-11T23:32:10.710 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.710 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.710 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.710 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T23:32:10.710 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.710 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.710 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.710 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "elasticsearch container image", 2024-09-11T23:32:10.710 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.710 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.710 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.711 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_grafana": { 2024-09-11T23:32:10.711 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_grafana", 2024-09-11T23:32:10.711 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.711 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.711 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.711 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-11T23:32:10.711 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.711 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.711 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.711 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-11T23:32:10.711 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.711 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.712 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.712 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_haproxy": { 2024-09-11T23:32:10.712 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_haproxy", 2024-09-11T23:32:10.712 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.712 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.712 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.712 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T23:32:10.712 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.713 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.713 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.713 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "HAproxy container image", 2024-09-11T23:32:10.713 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.713 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.713 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.713 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_agent": { 2024-09-11T23:32:10.713 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_agent", 2024-09-11T23:32:10.713 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.713 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.713 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.714 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T23:32:10.714 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.714 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.714 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.714 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger agent container image", 2024-09-11T23:32:10.714 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.714 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.714 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.714 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_collector": { 2024-09-11T23:32:10.714 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_collector", 2024-09-11T23:32:10.714 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.714 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.715 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.715 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T23:32:10.715 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.715 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.715 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.715 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger collector container image", 2024-09-11T23:32:10.715 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.715 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.715 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.715 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_query": { 2024-09-11T23:32:10.715 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_query", 2024-09-11T23:32:10.716 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.716 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.716 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.716 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T23:32:10.716 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.716 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.716 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.716 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger query container image", 2024-09-11T23:32:10.716 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.717 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.717 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.717 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_keepalived": { 2024-09-11T23:32:10.717 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_keepalived", 2024-09-11T23:32:10.717 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.717 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.717 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.717 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T23:32:10.717 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.717 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.717 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.718 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Keepalived container image", 2024-09-11T23:32:10.718 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.718 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.718 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.718 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_loki": { 2024-09-11T23:32:10.718 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_loki", 2024-09-11T23:32:10.718 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.718 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.718 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.718 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T23:32:10.718 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.718 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.719 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.719 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Loki container image", 2024-09-11T23:32:10.719 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.719 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.719 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.719 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_node_exporter": { 2024-09-11T23:32:10.719 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_node_exporter", 2024-09-11T23:32:10.719 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.719 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.719 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.719 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-11T23:32:10.720 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.720 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.720 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.720 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-11T23:32:10.720 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.720 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.720 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.720 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_nvmeof": { 2024-09-11T23:32:10.720 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_nvmeof", 2024-09-11T23:32:10.720 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.720 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.721 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.721 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T23:32:10.721 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.721 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.721 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.721 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Nvme-of container image", 2024-09-11T23:32:10.721 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.721 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.721 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.721 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_prometheus": { 2024-09-11T23:32:10.721 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_prometheus", 2024-09-11T23:32:10.722 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.722 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.722 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.722 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-11T23:32:10.722 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.722 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.722 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.722 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-11T23:32:10.722 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.722 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.723 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.723 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_promtail": { 2024-09-11T23:32:10.723 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_promtail", 2024-09-11T23:32:10.723 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.723 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.723 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.723 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T23:32:10.723 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.723 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.723 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.723 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Promtail container image", 2024-09-11T23:32:10.723 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.724 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.724 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.724 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_samba": { 2024-09-11T23:32:10.724 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_samba", 2024-09-11T23:32:10.724 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.724 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.724 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.725 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T23:32:10.725 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.725 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.725 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.725 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Samba/SMB container image", 2024-09-11T23:32:10.725 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.725 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.725 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.725 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_snmp_gateway": { 2024-09-11T23:32:10.726 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_snmp_gateway", 2024-09-11T23:32:10.726 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.726 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.726 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.726 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T23:32:10.726 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.726 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.726 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.726 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SNMP Gateway container image", 2024-09-11T23:32:10.726 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.726 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.726 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.727 INFO:teuthology.orchestra.run.smithi043.stdout: "container_init": { 2024-09-11T23:32:10.727 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_init", 2024-09-11T23:32:10.727 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.727 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.727 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.727 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.727 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.727 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.727 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.727 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T23:32:10.727 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.728 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.728 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.728 INFO:teuthology.orchestra.run.smithi043.stdout: "daemon_cache_timeout": { 2024-09-11T23:32:10.728 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "daemon_cache_timeout", 2024-09-11T23:32:10.728 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:10.728 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.728 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.728 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-11T23:32:10.728 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.728 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.728 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.729 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T23:32:10.729 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.729 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.729 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.729 INFO:teuthology.orchestra.run.smithi043.stdout: "default_cephadm_command_timeout": { 2024-09-11T23:32:10.729 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T23:32:10.729 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.729 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.729 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.729 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "900", 2024-09-11T23:32:10.729 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.730 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.730 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.730 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T23:32:10.730 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.730 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.730 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.730 INFO:teuthology.orchestra.run.smithi043.stdout: "default_registry": { 2024-09-11T23:32:10.730 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_registry", 2024-09-11T23:32:10.730 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.730 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.730 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.731 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io", 2024-09-11T23:32:10.731 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.731 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.731 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.731 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T23:32:10.731 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.731 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.731 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.731 INFO:teuthology.orchestra.run.smithi043.stdout: "device_cache_timeout": { 2024-09-11T23:32:10.731 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_cache_timeout", 2024-09-11T23:32:10.731 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1800", 2024-09-11T23:32:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache device inventory", 2024-09-11T23:32:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: "device_enhanced_scan": { 2024-09-11T23:32:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_enhanced_scan", 2024-09-11T23:32:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T23:32:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.734 INFO:teuthology.orchestra.run.smithi043.stdout: "facts_cache_timeout": { 2024-09-11T23:32:10.734 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "facts_cache_timeout", 2024-09-11T23:32:10.734 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:10.734 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.734 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.734 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-11T23:32:10.734 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.734 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache host facts data", 2024-09-11T23:32:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: "grafana_dashboards_path": { 2024-09-11T23:32:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "grafana_dashboards_path", 2024-09-11T23:32:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T23:32:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T23:32:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: "host_check_interval": { 2024-09-11T23:32:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "host_check_interval", 2024-09-11T23:32:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-11T23:32:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to perform a host check", 2024-09-11T23:32:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: "hw_monitoring": { 2024-09-11T23:32:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hw_monitoring", 2024-09-11T23:32:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T23:32:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: "inventory_list_all": { 2024-09-11T23:32:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "inventory_list_all", 2024-09-11T23:32:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T23:32:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: "log_refresh_metadata": { 2024-09-11T23:32:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_refresh_metadata", 2024-09-11T23:32:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T23:32:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T23:32:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T23:32:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T23:32:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T23:32:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T23:32:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T23:32:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "*", 2024-09-11T23:32:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T23:32:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "max_count_per_host": { 2024-09-11T23:32:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_count_per_host", 2024-09-11T23:32:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of daemons per service per host", 2024-09-11T23:32:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "max_osd_draining_count": { 2024-09-11T23:32:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_osd_draining_count", 2024-09-11T23:32:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T23:32:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: "migration_current": { 2024-09-11T23:32:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "migration_current", 2024-09-11T23:32:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "internal - do not modify", 2024-09-11T23:32:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-09-11T23:32:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-09-11T23:32:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "root", 2024-09-11T23:32:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm-package", 2024-09-11T23:32:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: "root" 2024-09-11T23:32:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T23:32:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: "oob_default_addr": { 2024-09-11T23:32:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "oob_default_addr", 2024-09-11T23:32:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "169.254.1.1", 2024-09-11T23:32:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T23:32:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "prometheus_alerts_path": { 2024-09-11T23:32:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus_alerts_path", 2024-09-11T23:32:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T23:32:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T23:32:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_insecure": { 2024-09-11T23:32:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_insecure", 2024-09-11T23:32:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T23:32:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_password": { 2024-09-11T23:32:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_password", 2024-09-11T23:32:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T23:32:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.758 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_url": { 2024-09-11T23:32:10.758 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_url", 2024-09-11T23:32:10.758 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.758 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.759 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.759 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.759 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.759 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.759 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.759 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T23:32:10.759 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.760 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.760 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.760 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_username": { 2024-09-11T23:32:10.760 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_username", 2024-09-11T23:32:10.760 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.760 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T23:32:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: "secure_monitoring_stack": { 2024-09-11T23:32:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "secure_monitoring_stack", 2024-09-11T23:32:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T23:32:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: "service_discovery_port": { 2024-09-11T23:32:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "service_discovery_port", 2024-09-11T23:32:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8765", 2024-09-11T23:32:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "cephadm service discovery port", 2024-09-11T23:32:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: "ssh_config_file": { 2024-09-11T23:32:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssh_config_file", 2024-09-11T23:32:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T23:32:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: "use_agent": { 2024-09-11T23:32:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_agent", 2024-09-11T23:32:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T23:32:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "use_repo_digest": { 2024-09-11T23:32:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_repo_digest", 2024-09-11T23:32:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T23:32:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_failed_host_check": { 2024-09-11T23:32:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_failed_host_check", 2024-09-11T23:32:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T23:32:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_daemons": { 2024-09-11T23:32:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_daemons", 2024-09-11T23:32:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T23:32:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_hosts": { 2024-09-11T23:32:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_hosts", 2024-09-11T23:32:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T23:32:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.771 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.771 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.771 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crash", 2024-09-11T23:32:10.771 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:10.771 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:10.771 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:10.771 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:10.771 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:10.771 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.771 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: "retain_interval": { 2024-09-11T23:32:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retain_interval", 2024-09-11T23:32:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "31536000", 2024-09-11T23:32:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T23:32:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_recent_interval": { 2024-09-11T23:32:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_recent_interval", 2024-09-11T23:32:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1209600", 2024-09-11T23:32:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T23:32:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dashboard", 2024-09-11T23:32:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T23:32:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T23:32:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T23:32:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T23:32:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.782 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.782 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.782 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.782 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T23:32:10.782 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T23:32:10.782 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.782 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.782 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.782 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.782 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.782 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.782 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_ENABLED": { 2024-09-11T23:32:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T23:32:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T23:32:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T23:32:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T23:32:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T23:32:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T23:32:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T23:32:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T23:32:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T23:32:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T23:32:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T23:32:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T23:32:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T23:32:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T23:32:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T23:32:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.793 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.793 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.793 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T23:32:10.793 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T23:32:10.793 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.793 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.793 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.793 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.793 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.793 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.793 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.794 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.794 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.794 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.794 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.794 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T23:32:10.794 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T23:32:10.794 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.794 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.794 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.794 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.794 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.795 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.795 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.795 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.795 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.795 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.795 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.795 INFO:teuthology.orchestra.run.smithi043.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T23:32:10.795 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T23:32:10.795 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.795 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.795 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.795 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.796 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.796 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.796 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.796 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.796 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.796 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.796 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.796 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T23:32:10.796 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T23:32:10.796 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.796 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.796 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-09-11T23:32:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T23:32:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T23:32:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_URL": { 2024-09-11T23:32:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_URL", 2024-09-11T23:32:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T23:32:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T23:32:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-09-11T23:32:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T23:32:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T23:32:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T23:32:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T23:32:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T23:32:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T23:32:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T23:32:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T23:32:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T23:32:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T23:32:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T23:32:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T23:32:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T23:32:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T23:32:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T23:32:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T23:32:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T23:32:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T23:32:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T23:32:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T23:32:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T23:32:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T23:32:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T23:32:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T23:32:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T23:32:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T23:32:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T23:32:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T23:32:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T23:32:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T23:32:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T23:32:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T23:32:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T23:32:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T23:32:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T23:32:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8", 2024-09-11T23:32:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T23:32:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T23:32:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "45", 2024-09-11T23:32:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T23:32:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T23:32:10.825 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.825 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.825 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.825 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.825 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.825 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.825 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T23:32:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T23:32:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-09-11T23:32:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T23:32:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T23:32:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T23:32:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T23:32:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T23:32:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T23:32:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-11T23:32:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T23:32:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T23:32:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T23:32:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T23:32:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-11T23:32:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: "cross_origin_url": { 2024-09-11T23:32:10.833 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cross_origin_url", 2024-09-11T23:32:10.833 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.833 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.833 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.833 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.833 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.833 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.833 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.833 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.833 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.833 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.833 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.834 INFO:teuthology.orchestra.run.smithi043.stdout: "crt_file": { 2024-09-11T23:32:10.834 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crt_file", 2024-09-11T23:32:10.834 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.834 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.834 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.834 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.834 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.834 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.834 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.834 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.834 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.834 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.834 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.835 INFO:teuthology.orchestra.run.smithi043.stdout: "debug": { 2024-09-11T23:32:10.835 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "debug", 2024-09-11T23:32:10.835 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.835 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.835 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.835 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.835 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.835 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.835 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.835 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable/disable debug options", 2024-09-11T23:32:10.835 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.836 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.836 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.836 INFO:teuthology.orchestra.run.smithi043.stdout: "jwt_token_ttl": { 2024-09-11T23:32:10.836 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "jwt_token_ttl", 2024-09-11T23:32:10.836 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.836 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.836 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.836 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "28800", 2024-09-11T23:32:10.836 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.836 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.836 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.837 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.837 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.837 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.837 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.837 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-09-11T23:32:10.837 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-09-11T23:32:10.837 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.837 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.837 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.837 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.837 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.837 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.838 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.838 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.838 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.838 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.838 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.838 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:10.838 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:10.838 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.838 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.838 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.838 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.838 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.839 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.839 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.839 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.839 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.839 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.839 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.839 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.839 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.839 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.839 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.839 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.839 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.839 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.840 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:10.840 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:10.840 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.840 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.840 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.840 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.840 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.840 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.840 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.840 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.840 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.840 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.840 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.841 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:10.841 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:10.841 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.841 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.841 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.841 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:10.841 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.841 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.841 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.841 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.841 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.842 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.842 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.842 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.842 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.842 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.842 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.842 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.842 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.842 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.842 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:10.842 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:10.842 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.842 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.843 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.843 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.843 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.843 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.843 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.843 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.843 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.843 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.843 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.843 INFO:teuthology.orchestra.run.smithi043.stdout: "motd": { 2024-09-11T23:32:10.843 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "motd", 2024-09-11T23:32:10.843 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.844 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.844 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.844 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.844 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.844 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.844 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.844 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "The message of the day", 2024-09-11T23:32:10.844 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.844 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.844 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.844 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect_resolve_ip_addr": { 2024-09-11T23:32:10.844 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T23:32:10.845 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.845 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.845 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.845 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.845 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.845 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.845 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.845 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.845 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.845 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.845 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.845 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-09-11T23:32:10.845 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-09-11T23:32:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-09-11T23:32:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.846 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-09-11T23:32:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-09-11T23:32:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8080", 2024-09-11T23:32:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.847 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-09-11T23:32:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-09-11T23:32:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.848 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl_server_port": { 2024-09-11T23:32:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl_server_port", 2024-09-11T23:32:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8443", 2024-09-11T23:32:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.849 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-09-11T23:32:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-09-11T23:32:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.850 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "redirect", 2024-09-11T23:32:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect" 2024-09-11T23:32:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-09-11T23:32:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-09-11T23:32:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-09-11T23:32:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-09-11T23:32:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-09-11T23:32:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: "url_prefix": { 2024-09-11T23:32:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url_prefix", 2024-09-11T23:32:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.853 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "devicehealth", 2024-09-11T23:32:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_monitoring": { 2024-09-11T23:32:10.854 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_monitoring", 2024-09-11T23:32:10.855 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.855 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.855 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.855 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.855 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.855 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.855 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.855 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "monitor device health metrics", 2024-09-11T23:32:10.855 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.855 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.855 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.855 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:10.856 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:10.856 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.856 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.856 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.856 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.856 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.856 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.856 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.856 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.856 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.857 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.857 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.857 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.857 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.857 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.857 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.858 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.858 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.858 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.858 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:10.858 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:10.859 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.859 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.859 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.859 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.859 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.859 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.859 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.860 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.860 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.860 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.860 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.860 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:10.860 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:10.861 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.861 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.861 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.861 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:10.861 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.861 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.861 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.862 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.862 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.862 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.862 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.862 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.862 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.863 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.863 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.863 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.863 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.863 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.864 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:10.864 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:10.864 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.864 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.864 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.864 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.865 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.865 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.865 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.865 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.865 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.865 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.865 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.866 INFO:teuthology.orchestra.run.smithi043.stdout: "mark_out_threshold": { 2024-09-11T23:32:10.866 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mark_out_threshold", 2024-09-11T23:32:10.866 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:10.866 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.866 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.866 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2419200", 2024-09-11T23:32:10.866 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.866 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.866 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.866 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T23:32:10.866 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.866 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.867 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.867 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": { 2024-09-11T23:32:10.867 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_name", 2024-09-11T23:32:10.867 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.867 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.867 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.867 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "device_health_metrics", 2024-09-11T23:32:10.867 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.867 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.867 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.867 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T23:32:10.867 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.868 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.868 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.868 INFO:teuthology.orchestra.run.smithi043.stdout: "retention_period": { 2024-09-11T23:32:10.868 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retention_period", 2024-09-11T23:32:10.868 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:10.868 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.868 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.868 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15552000", 2024-09-11T23:32:10.868 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.868 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.868 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.868 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain device health metrics", 2024-09-11T23:32:10.869 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.869 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.869 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.869 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_frequency": { 2024-09-11T23:32:10.869 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_frequency", 2024-09-11T23:32:10.869 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:10.869 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.869 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.869 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-09-11T23:32:10.869 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.870 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.870 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.870 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T23:32:10.870 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.870 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.870 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.870 INFO:teuthology.orchestra.run.smithi043.stdout: "self_heal": { 2024-09-11T23:32:10.870 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "self_heal", 2024-09-11T23:32:10.870 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.870 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.870 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.871 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.871 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.871 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.871 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.871 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T23:32:10.871 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.871 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.871 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.871 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-11T23:32:10.871 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-11T23:32:10.871 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:10.872 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.872 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.872 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-11T23:32:10.872 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.872 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.872 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.872 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T23:32:10.872 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.872 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.872 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.872 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_threshold": { 2024-09-11T23:32:10.873 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_threshold", 2024-09-11T23:32:10.873 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:10.873 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.873 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.873 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7257600", 2024-09-11T23:32:10.873 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.873 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.873 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.873 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T23:32:10.873 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.873 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.873 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.873 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.873 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.874 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "diskprediction_local", 2024-09-11T23:32:10.874 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:10.874 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:10.874 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:10.874 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:10.874 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:10.874 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.874 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.874 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.874 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.875 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.875 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.875 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.875 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.875 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.875 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.875 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.875 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.875 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.875 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.875 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.875 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.875 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.876 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.876 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:10.876 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:10.876 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.876 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.876 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.876 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.876 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.876 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.876 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.876 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.876 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.877 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.877 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.877 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:10.877 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:10.877 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.877 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.877 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.877 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:10.877 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.877 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.877 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.877 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.878 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.878 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.878 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.878 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.878 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.878 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.878 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.878 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.878 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.878 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.879 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:10.879 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:10.879 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.879 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.879 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.879 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.879 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.879 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.879 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.879 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.879 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.879 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.880 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.880 INFO:teuthology.orchestra.run.smithi043.stdout: "predict_interval": { 2024-09-11T23:32:10.880 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predict_interval", 2024-09-11T23:32:10.880 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.880 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.880 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.880 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-09-11T23:32:10.880 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.880 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.880 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.880 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.881 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.881 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.881 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.881 INFO:teuthology.orchestra.run.smithi043.stdout: "predictor_model": { 2024-09-11T23:32:10.881 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predictor_model", 2024-09-11T23:32:10.881 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.881 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.881 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.881 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "prophetstor", 2024-09-11T23:32:10.881 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.881 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.882 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.882 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.882 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.882 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.882 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.882 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-11T23:32:10.882 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-11T23:32:10.882 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.882 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.882 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.882 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-11T23:32:10.883 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.883 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.883 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.883 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.883 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.883 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.883 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.883 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.883 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.884 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "influx", 2024-09-11T23:32:10.884 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": false, 2024-09-11T23:32:10.884 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "influxdb python module not found", 2024-09-11T23:32:10.884 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:10.884 INFO:teuthology.orchestra.run.smithi043.stdout: "batch_size": { 2024-09-11T23:32:10.884 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "batch_size", 2024-09-11T23:32:10.884 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.884 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.884 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.884 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5000", 2024-09-11T23:32:10.884 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.884 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.885 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.885 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T23:32:10.885 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.885 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.885 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.885 INFO:teuthology.orchestra.run.smithi043.stdout: "database": { 2024-09-11T23:32:10.885 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "database", 2024-09-11T23:32:10.885 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.885 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.885 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.886 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "ceph", 2024-09-11T23:32:10.886 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.886 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.886 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.886 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T23:32:10.886 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.886 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.886 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.886 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": { 2024-09-11T23:32:10.886 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hostname", 2024-09-11T23:32:10.886 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.886 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.887 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.887 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.887 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.887 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.887 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.887 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server hostname", 2024-09-11T23:32:10.887 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.887 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.887 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.887 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-11T23:32:10.887 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-11T23:32:10.887 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:10.888 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.888 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.888 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "30", 2024-09-11T23:32:10.888 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "5", 2024-09-11T23:32:10.888 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.888 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.888 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T23:32:10.888 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.888 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.888 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.888 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:10.889 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:10.889 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.889 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.889 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.889 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.889 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.889 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.889 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.889 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.889 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.890 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.890 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.890 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.890 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.890 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.890 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.890 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.890 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.890 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.890 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:10.890 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:10.890 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.891 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.891 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.891 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.891 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.891 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.891 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.891 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.891 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.891 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.891 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.891 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:10.892 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:10.892 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.892 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.892 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.892 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:10.892 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.892 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.892 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.892 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.892 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.892 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.893 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.893 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.893 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.893 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.893 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.893 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.893 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.893 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.893 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:10.893 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:10.893 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.893 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.894 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.894 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.894 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.894 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.894 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.894 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.894 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.894 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.894 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.894 INFO:teuthology.orchestra.run.smithi043.stdout: "password": { 2024-09-11T23:32:10.894 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "password", 2024-09-11T23:32:10.894 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.895 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.895 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.895 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.895 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.895 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.895 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.895 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "password of InfluxDB server user", 2024-09-11T23:32:10.895 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.895 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.895 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.895 INFO:teuthology.orchestra.run.smithi043.stdout: "port": { 2024-09-11T23:32:10.895 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "port", 2024-09-11T23:32:10.896 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.896 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.896 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.896 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8086", 2024-09-11T23:32:10.896 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.896 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.896 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.896 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server port", 2024-09-11T23:32:10.896 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.896 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.896 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.896 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-09-11T23:32:10.897 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-09-11T23:32:10.897 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.897 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.897 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.897 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "false", 2024-09-11T23:32:10.897 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.897 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.897 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.897 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:32:10.897 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.897 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.898 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.898 INFO:teuthology.orchestra.run.smithi043.stdout: "threads": { 2024-09-11T23:32:10.898 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threads", 2024-09-11T23:32:10.898 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.898 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.898 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.898 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-11T23:32:10.898 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-11T23:32:10.898 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "32", 2024-09-11T23:32:10.898 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.898 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T23:32:10.898 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.899 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.899 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.899 INFO:teuthology.orchestra.run.smithi043.stdout: "username": { 2024-09-11T23:32:10.899 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "username", 2024-09-11T23:32:10.899 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.899 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.899 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.899 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.899 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.899 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.899 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.899 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "username of InfluxDB server user", 2024-09-11T23:32:10.900 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.900 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.900 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.900 INFO:teuthology.orchestra.run.smithi043.stdout: "verify_ssl": { 2024-09-11T23:32:10.900 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "verify_ssl", 2024-09-11T23:32:10.900 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.900 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.900 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.900 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "true", 2024-09-11T23:32:10.900 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.901 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.901 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.901 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:32:10.901 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.901 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.901 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.901 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.901 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.901 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "insights", 2024-09-11T23:32:10.901 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:10.902 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:10.902 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:10.902 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:10.902 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:10.902 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.902 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.902 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.902 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.902 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.902 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.902 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.902 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.902 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.903 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.903 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.903 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.903 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.903 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.903 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.903 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.903 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.903 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.903 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:10.903 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:10.903 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.904 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.904 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.904 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.904 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.904 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.904 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.904 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.904 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.904 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.904 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.904 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:10.905 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:10.905 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.905 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.905 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.905 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:10.905 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.905 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.905 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.905 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.905 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.905 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.905 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.905 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.906 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.906 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.906 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.906 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.906 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.906 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.906 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:10.906 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:10.906 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.906 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.906 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.906 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.907 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.907 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.907 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.907 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.907 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.907 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.907 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.907 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.907 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.908 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "iostat", 2024-09-11T23:32:10.908 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:10.908 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:10.908 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:10.908 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:10.908 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:10.908 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.908 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.908 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.908 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.908 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.908 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.909 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.909 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.909 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.909 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.909 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.909 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.909 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.909 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.909 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.909 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.909 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.909 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.910 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:10.910 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:10.910 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.910 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.910 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.910 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.910 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.910 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.910 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.910 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.910 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.910 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.910 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.911 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:10.911 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:10.911 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.911 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.911 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.911 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:10.911 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.911 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.911 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.911 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.911 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.911 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.912 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.912 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.912 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.912 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.912 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.912 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.912 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.912 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.912 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:10.912 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:10.912 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.912 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.913 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.913 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.913 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.913 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.913 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.913 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.913 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.913 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.913 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.914 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.914 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "k8sevents", 2024-09-11T23:32:10.914 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:10.914 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:10.914 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:10.914 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_event_retention_days": { 2024-09-11T23:32:10.914 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ceph_event_retention_days", 2024-09-11T23:32:10.914 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.914 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.914 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.914 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7", 2024-09-11T23:32:10.914 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.915 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.915 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.915 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T23:32:10.915 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.915 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.915 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.915 INFO:teuthology.orchestra.run.smithi043.stdout: "config_check_secs": { 2024-09-11T23:32:10.915 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_check_secs", 2024-09-11T23:32:10.915 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.915 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.915 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.915 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:10.916 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "10", 2024-09-11T23:32:10.916 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.916 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.916 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T23:32:10.916 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.916 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.916 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.916 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:10.916 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:10.916 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.917 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.917 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.917 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.917 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.917 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.917 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.917 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.917 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.917 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.917 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.917 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.917 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.917 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.918 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.918 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.918 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.918 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.918 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:10.918 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:10.918 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.918 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.918 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.918 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.918 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.918 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.919 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.919 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.919 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.919 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.919 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.919 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:10.919 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:10.919 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.919 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.919 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.919 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:10.919 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.920 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.920 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.920 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.920 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.920 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.920 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.920 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.920 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.920 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.920 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.920 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.920 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.921 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.921 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:10.921 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:10.921 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.921 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.921 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.921 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.921 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.921 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.921 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.921 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.921 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.921 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.922 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.922 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.922 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.922 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "localpool", 2024-09-11T23:32:10.922 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:10.922 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:10.922 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:10.922 INFO:teuthology.orchestra.run.smithi043.stdout: "failure_domain": { 2024-09-11T23:32:10.922 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "failure_domain", 2024-09-11T23:32:10.922 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.922 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.923 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.923 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "host", 2024-09-11T23:32:10.923 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.923 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.923 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.923 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "failure domain for any created local pool", 2024-09-11T23:32:10.923 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T23:32:10.923 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.923 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.923 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:10.924 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:10.924 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.924 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.924 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.924 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.924 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.924 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.924 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.924 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.924 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.924 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.924 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.925 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.925 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.925 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.925 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.925 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.925 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.925 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.925 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:10.925 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:10.925 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.925 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.925 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.926 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.926 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.926 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.926 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.926 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.926 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.926 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.926 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.926 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:10.927 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:10.927 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.927 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.927 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.927 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:10.927 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.927 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.927 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.927 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.927 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.927 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.927 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.927 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.928 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.928 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.928 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.928 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.928 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.928 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.928 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:10.928 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:10.928 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.928 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.928 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.929 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.929 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.929 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.929 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.929 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.929 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.929 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.929 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.929 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": { 2024-09-11T23:32:10.929 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_size", 2024-09-11T23:32:10.929 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.929 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.930 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.930 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.930 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.930 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.930 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.930 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default min_size for any created local pool", 2024-09-11T23:32:10.930 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T23:32:10.930 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.930 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.930 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rep": { 2024-09-11T23:32:10.930 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "num_rep", 2024-09-11T23:32:10.931 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.931 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.931 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.931 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3", 2024-09-11T23:32:10.931 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.931 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.931 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.931 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default replica count for any created local pool", 2024-09-11T23:32:10.931 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.931 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.931 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.932 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": { 2024-09-11T23:32:10.932 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_num", 2024-09-11T23:32:10.932 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.932 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.932 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.932 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "128", 2024-09-11T23:32:10.932 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.932 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.932 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.932 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T23:32:10.932 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.932 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.932 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.933 INFO:teuthology.orchestra.run.smithi043.stdout: "prefix": { 2024-09-11T23:32:10.933 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prefix", 2024-09-11T23:32:10.933 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.933 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.933 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.933 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.933 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.933 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.933 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.933 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name prefix for any created local pool", 2024-09-11T23:32:10.934 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.934 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.934 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.934 INFO:teuthology.orchestra.run.smithi043.stdout: "subtree": { 2024-09-11T23:32:10.934 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "subtree", 2024-09-11T23:32:10.934 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.934 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.934 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.934 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "rack", 2024-09-11T23:32:10.934 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.934 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.934 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.935 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T23:32:10.935 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T23:32:10.935 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.935 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.935 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.935 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.935 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mds_autoscaler", 2024-09-11T23:32:10.935 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:10.935 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:10.935 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:10.935 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:10.936 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:10.936 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.936 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.936 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.936 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.936 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.936 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.936 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.936 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.936 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.936 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.937 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.937 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.937 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.937 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.937 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.937 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.937 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.937 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.937 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:10.937 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:10.937 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.937 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.938 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.938 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.938 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.938 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.938 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.938 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.938 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.938 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.938 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.938 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:10.938 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:10.938 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.939 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.939 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.939 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:10.939 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.939 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.939 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.939 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.939 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.939 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.939 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.939 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.939 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.939 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.940 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.940 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.940 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.940 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.940 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:10.940 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:10.940 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.940 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.940 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.940 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.940 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.940 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.941 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.941 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.941 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.941 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.941 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.941 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.941 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.941 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirroring", 2024-09-11T23:32:10.941 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:10.941 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:10.941 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:10.942 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:10.942 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:10.942 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.942 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.942 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.942 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.942 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.942 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.942 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.942 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.942 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.942 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.942 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.943 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.943 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.943 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.943 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.943 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.943 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.943 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.943 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:10.943 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:10.943 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.943 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.943 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.944 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.944 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.944 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.944 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.944 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.944 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.944 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.944 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.944 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:10.944 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:10.944 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.944 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.945 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.945 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:10.945 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.945 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.945 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.945 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.945 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.945 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.945 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.945 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.945 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.945 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.945 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.946 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.946 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.946 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.946 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:10.946 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:10.946 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.946 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.946 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.947 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.947 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.947 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.947 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.947 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.947 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.947 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.947 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.947 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.947 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.948 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "nfs", 2024-09-11T23:32:10.948 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:10.948 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:10.948 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:10.948 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:10.948 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:10.948 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.948 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.948 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.948 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.948 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.948 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.949 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.949 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.949 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.949 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.949 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.949 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.949 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.949 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.949 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.949 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.949 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.949 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.949 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:10.950 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:10.950 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.950 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.950 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.950 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.950 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.950 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.950 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.950 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.950 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.950 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.950 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.951 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:10.951 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:10.951 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.951 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.951 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.951 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:10.951 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.951 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.951 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.951 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.951 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.951 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.952 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.952 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.952 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.952 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.952 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.952 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.952 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.952 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.952 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:10.952 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:10.953 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.953 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.953 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.953 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.953 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.953 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.953 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.953 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.953 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.953 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.954 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.954 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.954 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.954 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-09-11T23:32:10.954 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:10.954 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:10.954 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:10.954 INFO:teuthology.orchestra.run.smithi043.stdout: "fail_fs": { 2024-09-11T23:32:10.954 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "fail_fs", 2024-09-11T23:32:10.954 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.954 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.954 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.955 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.955 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.955 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.955 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.955 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T23:32:10.955 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.955 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.955 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.955 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:10.955 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:10.955 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.956 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.956 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.956 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.956 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.956 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.956 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.956 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.956 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.956 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.956 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.956 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.956 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.956 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.957 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.957 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.957 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.957 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.957 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:10.957 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:10.957 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.957 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.957 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.957 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.957 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.958 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.958 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.958 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.958 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.958 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.958 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.958 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:10.958 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:10.958 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.958 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.958 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.958 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:10.959 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.959 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.959 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.959 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.959 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.959 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.959 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.959 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.959 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.959 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.959 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.959 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.959 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.960 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.960 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:10.960 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:10.960 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.960 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.960 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.960 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.960 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.960 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.960 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.960 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.960 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.961 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.961 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.961 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator": { 2024-09-11T23:32:10.961 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-09-11T23:32:10.961 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.961 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.961 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.961 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.962 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.962 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.962 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.962 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm", 2024-09-11T23:32:10.962 INFO:teuthology.orchestra.run.smithi043.stdout: "rook", 2024-09-11T23:32:10.962 INFO:teuthology.orchestra.run.smithi043.stdout: "test_orchestrator" 2024-09-11T23:32:10.962 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.962 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Orchestrator backend", 2024-09-11T23:32:10.962 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.962 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.962 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.962 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.963 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.963 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_perf_query", 2024-09-11T23:32:10.963 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:10.963 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:10.963 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:10.963 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:10.963 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:10.963 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.963 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.964 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.964 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.964 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.964 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.964 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.964 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.964 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.964 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.964 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.965 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.965 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.965 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.965 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.965 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.965 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.965 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.965 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:10.965 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:10.965 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.966 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.966 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.966 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.966 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.966 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.966 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.966 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.966 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.966 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.966 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.966 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:10.967 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:10.967 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.967 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.967 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.967 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:10.967 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.967 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.967 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.967 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.967 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.967 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.967 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.967 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.968 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.968 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.968 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.968 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.968 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.968 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.968 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:10.968 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:10.968 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.968 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.968 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.968 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.969 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.969 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.969 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.969 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.969 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.969 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.969 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.969 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.969 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.969 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_support", 2024-09-11T23:32:10.969 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:10.970 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:10.970 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:10.970 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:10.970 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:10.970 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.970 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.970 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.970 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.970 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.970 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.970 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.970 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.971 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.971 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.971 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.971 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.971 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.971 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.971 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.971 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.971 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.971 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.971 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:10.971 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:10.972 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.972 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.972 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.972 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.972 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.972 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.972 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.972 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.972 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.972 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.972 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.972 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:10.973 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:10.973 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.973 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.973 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.973 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:10.973 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.973 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.973 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.973 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.973 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.973 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.973 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.973 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.974 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.974 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.974 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.974 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.974 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.974 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.974 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:10.974 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:10.974 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.974 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.974 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.974 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.975 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.975 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.975 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.975 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.975 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.975 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.975 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.975 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.975 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.975 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_autoscaler", 2024-09-11T23:32:10.975 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:10.976 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:10.976 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:10.976 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:10.976 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:10.976 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.976 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.976 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.976 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.976 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.976 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.976 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.976 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.977 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.977 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.977 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.977 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.977 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.977 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.977 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.977 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.977 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.977 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.977 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:10.978 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:10.978 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.978 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.978 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.978 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.978 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.978 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.978 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.978 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.978 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.978 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.978 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.979 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:10.979 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:10.979 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.979 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.979 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.979 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:10.979 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.979 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.979 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.980 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.980 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.980 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.980 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.980 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.980 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.980 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.980 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.980 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.980 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.980 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.980 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:10.981 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:10.981 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.981 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.981 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.981 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.981 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.981 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.981 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.981 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.981 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.981 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.981 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.982 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-11T23:32:10.982 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-11T23:32:10.982 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:10.982 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.982 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.982 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-11T23:32:10.982 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.982 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.982 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.982 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.982 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.982 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.983 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.983 INFO:teuthology.orchestra.run.smithi043.stdout: "threshold": { 2024-09-11T23:32:10.983 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threshold", 2024-09-11T23:32:10.983 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-11T23:32:10.983 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.983 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.983 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-09-11T23:32:10.983 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1.0", 2024-09-11T23:32:10.983 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.983 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.983 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "scaling threshold", 2024-09-11T23:32:10.984 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T23:32:10.984 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.984 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.984 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.984 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.984 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "progress", 2024-09-11T23:32:10.984 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:10.984 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:10.984 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:10.985 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_pg_recovery_event": { 2024-09-11T23:32:10.985 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_pg_recovery_event", 2024-09-11T23:32:10.985 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.985 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.985 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.985 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.985 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.985 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.985 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.985 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T23:32:10.985 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.985 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.985 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.986 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-09-11T23:32:10.986 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-09-11T23:32:10.986 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.986 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.986 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.986 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.986 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.986 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.986 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.986 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.986 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.986 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.987 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.987 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:10.987 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:10.987 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.987 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.987 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.987 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.987 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.987 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.987 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.987 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.988 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.988 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.988 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.988 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.988 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.988 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.988 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.988 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.988 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.988 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.988 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:10.988 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:10.989 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.989 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.989 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.989 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.989 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.989 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.989 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.989 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.989 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.989 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.989 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.990 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:10.990 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:10.990 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.990 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.990 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.990 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:10.990 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.990 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.990 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.990 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.990 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.991 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.991 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.991 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.991 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.991 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.991 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.991 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.991 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.991 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.991 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:10.991 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:10.991 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.992 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.992 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.992 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:10.992 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.992 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.992 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.992 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.992 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.992 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.992 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.993 INFO:teuthology.orchestra.run.smithi043.stdout: "max_completed_events": { 2024-09-11T23:32:10.993 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_completed_events", 2024-09-11T23:32:10.993 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:10.993 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.993 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.993 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "50", 2024-09-11T23:32:10.993 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.993 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.993 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.993 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "number of past completed events to remember", 2024-09-11T23:32:10.993 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.993 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.994 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.994 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-11T23:32:10.994 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-11T23:32:10.994 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:10.994 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.994 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.994 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-11T23:32:10.994 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.994 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.994 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.994 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long the module is going to sleep", 2024-09-11T23:32:10.995 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.995 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.995 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.995 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.995 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:10.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:10.995 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus", 2024-09-11T23:32:10.995 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:10.995 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:10.995 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:10.995 INFO:teuthology.orchestra.run.smithi043.stdout: "cache": { 2024-09-11T23:32:10.996 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cache", 2024-09-11T23:32:10.996 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.996 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.996 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:10.996 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.996 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.996 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.996 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.996 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.996 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.996 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.996 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.997 INFO:teuthology.orchestra.run.smithi043.stdout: "exclude_perf_counters": { 2024-09-11T23:32:10.997 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "exclude_perf_counters", 2024-09-11T23:32:10.997 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:10.997 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.997 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.997 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:10.997 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.997 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.997 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:10.997 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T23:32:10.997 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T23:32:10.997 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:10.998 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:10.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:10.998 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:10.998 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:10.998 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:10.998 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:10.998 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:10.998 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:10.998 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:10.998 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:10.999 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:10.999 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:10.999 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:10.999 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:10.999 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:10.999 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:10.999 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:10.999 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:10.999 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:10.999 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:10.999 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.000 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.000 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.000 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.000 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.000 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.000 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.000 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.000 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.000 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.000 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.000 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.001 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.001 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.001 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.001 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.001 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.001 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.001 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.001 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.001 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.001 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.001 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.001 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.002 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.002 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.002 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.002 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.002 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.002 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.002 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.002 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.002 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.002 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.002 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.002 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.002 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.003 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.003 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.003 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.003 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.003 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.003 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.003 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.003 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.003 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.003 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.003 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.003 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools": { 2024-09-11T23:32:11.004 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools", 2024-09-11T23:32:11.004 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.004 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.004 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.004 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.004 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.004 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.004 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.004 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.004 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.004 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.004 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.005 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T23:32:11.005 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T23:32:11.005 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.005 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.005 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.005 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "300", 2024-09-11T23:32:11.005 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.005 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.005 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.005 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.005 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.005 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.005 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.006 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_interval": { 2024-09-11T23:32:11.006 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_interval", 2024-09-11T23:32:11.006 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-11T23:32:11.006 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.006 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.006 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15.0", 2024-09-11T23:32:11.006 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.006 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.006 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.006 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.006 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.006 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.007 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.007 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-09-11T23:32:11.007 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-09-11T23:32:11.007 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.007 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.007 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.007 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-09-11T23:32:11.007 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.007 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.007 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.007 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T23:32:11.007 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.007 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.008 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.008 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-09-11T23:32:11.008 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-09-11T23:32:11.008 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.008 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.008 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.008 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "9283", 2024-09-11T23:32:11.008 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.009 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.009 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.009 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T23:32:11.009 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.009 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.009 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.009 INFO:teuthology.orchestra.run.smithi043.stdout: "stale_cache_strategy": { 2024-09-11T23:32:11.009 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stale_cache_strategy", 2024-09-11T23:32:11.009 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.009 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.009 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.010 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "log", 2024-09-11T23:32:11.010 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.010 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.010 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.010 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.010 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.010 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.010 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.010 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-09-11T23:32:11.010 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-09-11T23:32:11.010 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.010 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.011 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.011 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "default", 2024-09-11T23:32:11.011 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.011 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.011 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.011 INFO:teuthology.orchestra.run.smithi043.stdout: "default", 2024-09-11T23:32:11.011 INFO:teuthology.orchestra.run.smithi043.stdout: "error" 2024-09-11T23:32:11.011 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.011 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.011 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.011 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.011 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.012 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-09-11T23:32:11.012 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-09-11T23:32:11.012 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.012 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.012 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.012 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-09-11T23:32:11.012 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-09-11T23:32:11.012 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-09-11T23:32:11.012 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.012 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.012 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.012 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.012 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.013 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.013 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.013 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_support", 2024-09-11T23:32:11.013 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.013 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.013 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.013 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.013 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.013 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.014 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.014 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.014 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.014 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.014 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.014 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.014 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.014 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.014 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.014 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.014 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.014 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.015 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.015 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.015 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.015 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.015 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.015 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.015 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.015 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.015 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.015 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.015 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.015 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.016 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.016 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.016 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.016 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.016 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.016 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.016 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.016 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.016 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.016 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.016 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.016 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.017 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.017 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.017 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.017 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.017 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.017 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.017 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.017 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.017 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.017 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.017 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.017 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.017 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.018 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.018 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.018 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.018 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.018 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.018 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.019 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.020 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.020 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.020 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.020 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.020 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.020 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.020 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.020 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_snap_create": { 2024-09-11T23:32:11.020 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_snap_create", 2024-09-11T23:32:11.020 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.021 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.021 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.021 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:11.021 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.021 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.021 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.021 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.021 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.021 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.021 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.021 INFO:teuthology.orchestra.run.smithi043.stdout: "mirror_snapshot_schedule": { 2024-09-11T23:32:11.021 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T23:32:11.022 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.022 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.022 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.022 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.022 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.022 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.022 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.022 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.022 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.022 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.022 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.022 INFO:teuthology.orchestra.run.smithi043.stdout: "trash_purge_schedule": { 2024-09-11T23:32:11.022 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "trash_purge_schedule", 2024-09-11T23:32:11.023 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.023 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.023 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.023 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.023 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.023 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.023 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.023 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.023 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.023 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.023 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.024 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.024 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.024 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "restful", 2024-09-11T23:32:11.024 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.024 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.024 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.024 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_auth": { 2024-09-11T23:32:11.024 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_auth", 2024-09-11T23:32:11.024 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.024 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.025 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.025 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.025 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.025 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.025 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.025 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.025 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.025 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.025 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.025 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-09-11T23:32:11.025 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-09-11T23:32:11.025 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.026 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.026 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.026 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.026 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.026 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.026 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.026 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.026 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.027 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.027 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.027 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.027 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.027 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.027 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.027 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.027 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.027 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.027 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.028 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.028 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.028 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.028 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.028 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.028 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.028 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.028 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.028 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.028 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.028 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.028 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.029 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.029 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.029 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.029 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.029 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.029 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.029 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.029 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.029 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.029 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.029 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.029 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.030 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.030 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.030 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.030 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.030 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.030 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.030 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.030 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.030 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.030 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.030 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.030 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.030 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.031 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.031 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.031 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.031 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.031 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.031 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.031 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.031 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.031 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.031 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.031 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.031 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.032 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.032 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.032 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.032 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.032 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.032 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.032 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.032 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.032 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.032 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-09-11T23:32:11.032 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-09-11T23:32:11.032 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.033 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.033 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.033 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.033 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.033 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.033 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.033 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.033 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.033 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.033 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.033 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-09-11T23:32:11.033 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-09-11T23:32:11.034 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.034 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.034 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.034 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.034 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.034 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.034 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.034 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.034 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.034 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.034 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.034 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.034 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.035 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rgw", 2024-09-11T23:32:11.035 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.035 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.035 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.035 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.035 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.035 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.035 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.035 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.035 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.035 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.036 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.036 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.036 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.036 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.036 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.036 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.036 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.036 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.036 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.036 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.036 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.036 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.037 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.037 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.037 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.037 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.037 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.037 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.037 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.037 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.037 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.037 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.037 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.037 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.038 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.038 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.038 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.038 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.038 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.038 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.038 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.038 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.038 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.038 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.038 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.039 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.039 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.039 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.039 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.039 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.039 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.039 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.039 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.039 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.039 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.039 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.039 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.040 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.040 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.040 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.040 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.040 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.040 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.040 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.040 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.040 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.040 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.040 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.040 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.040 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.041 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.041 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rook", 2024-09-11T23:32:11.041 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.041 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.041 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.041 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.041 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.041 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.041 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.041 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.041 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.042 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.042 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.042 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.042 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.042 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.042 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.042 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.042 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.042 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.042 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.042 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.042 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.043 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.043 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.043 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.043 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.043 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.043 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.043 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.043 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.043 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.043 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.043 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.043 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.044 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.044 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.044 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.044 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.044 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.044 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.044 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.044 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.044 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.044 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.044 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.044 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.045 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.045 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.045 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.045 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.045 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.045 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.045 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.045 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.045 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.045 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.045 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.045 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.046 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.046 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.046 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.046 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.046 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.046 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.046 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.046 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.046 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.046 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.046 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.046 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.047 INFO:teuthology.orchestra.run.smithi043.stdout: "storage_class": { 2024-09-11T23:32:11.047 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "storage_class", 2024-09-11T23:32:11.047 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.047 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.047 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.047 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "local", 2024-09-11T23:32:11.047 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.047 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.047 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.047 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T23:32:11.048 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.048 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.048 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.048 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.048 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.048 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "selftest", 2024-09-11T23:32:11.048 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.048 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.048 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.048 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.049 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.049 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.049 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.049 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.049 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.049 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.049 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.049 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.049 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.049 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.049 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.049 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.050 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.050 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.050 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.050 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.050 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.050 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.050 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.050 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.050 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.051 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.051 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.051 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.051 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.051 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.051 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.051 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.051 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.051 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.051 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.051 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.052 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.052 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.052 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.052 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.052 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.052 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.052 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.052 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.052 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.053 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.053 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.053 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.053 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.053 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.053 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.053 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.053 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.053 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.053 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.053 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.053 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.054 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.054 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.054 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.054 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.054 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.054 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.054 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.054 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.054 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.055 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.055 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.055 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.055 INFO:teuthology.orchestra.run.smithi043.stdout: "roption1": { 2024-09-11T23:32:11.055 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption1", 2024-09-11T23:32:11.055 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.055 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.055 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.055 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.055 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.056 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.056 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.056 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.056 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.056 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.056 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.056 INFO:teuthology.orchestra.run.smithi043.stdout: "roption2": { 2024-09-11T23:32:11.056 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption2", 2024-09-11T23:32:11.056 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.056 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.056 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.057 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "xyz", 2024-09-11T23:32:11.057 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.057 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.057 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.057 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.057 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.057 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.057 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.057 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption1": { 2024-09-11T23:32:11.057 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption1", 2024-09-11T23:32:11.057 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.057 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.058 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.058 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.058 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.058 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.058 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.058 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.058 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.058 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.058 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.058 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption2": { 2024-09-11T23:32:11.058 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption2", 2024-09-11T23:32:11.058 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.058 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.059 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.059 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.059 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.059 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.059 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.059 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.059 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.059 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.059 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.059 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption3": { 2024-09-11T23:32:11.060 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption3", 2024-09-11T23:32:11.060 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-11T23:32:11.060 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.060 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.060 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.060 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.060 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.060 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.060 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.060 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.060 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.060 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.061 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption4": { 2024-09-11T23:32:11.061 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption4", 2024-09-11T23:32:11.061 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.061 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.061 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.061 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.061 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.061 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.061 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.061 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.061 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.061 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.061 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.062 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption5": { 2024-09-11T23:32:11.062 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption5", 2024-09-11T23:32:11.062 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.062 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.062 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.062 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.062 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.062 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.062 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption6": { 2024-09-11T23:32:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption6", 2024-09-11T23:32:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption7": { 2024-09-11T23:32:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption7", 2024-09-11T23:32:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.065 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-11T23:32:11.065 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "42", 2024-09-11T23:32:11.065 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.065 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.065 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.065 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.065 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.066 INFO:teuthology.orchestra.run.smithi043.stdout: "testkey": { 2024-09-11T23:32:11.066 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testkey", 2024-09-11T23:32:11.066 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.066 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.066 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.066 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.066 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.066 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.066 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.066 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.066 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.067 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.067 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.067 INFO:teuthology.orchestra.run.smithi043.stdout: "testlkey": { 2024-09-11T23:32:11.067 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testlkey", 2024-09-11T23:32:11.067 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.067 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.067 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.067 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.067 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.067 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.067 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.068 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.068 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.068 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.068 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.068 INFO:teuthology.orchestra.run.smithi043.stdout: "testnewline": { 2024-09-11T23:32:11.068 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testnewline", 2024-09-11T23:32:11.068 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.068 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.068 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.068 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.068 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.069 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.069 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.069 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.069 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.069 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.069 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.069 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.069 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.069 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-09-11T23:32:11.069 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.069 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.070 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.070 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_m_granularity": { 2024-09-11T23:32:11.070 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_m_granularity", 2024-09-11T23:32:11.070 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.070 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.070 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.070 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.070 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.070 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.070 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.070 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T23:32:11.070 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.070 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.070 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.071 INFO:teuthology.orchestra.run.smithi043.stdout: "dump_on_update": { 2024-09-11T23:32:11.071 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dump_on_update", 2024-09-11T23:32:11.071 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.071 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.071 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.071 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.071 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.071 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.071 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.071 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "dump database to debug log on update", 2024-09-11T23:32:11.071 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.071 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.072 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.072 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.072 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.072 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.072 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.072 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.072 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.072 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.072 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.072 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.072 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.072 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.073 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.073 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.073 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.073 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.073 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.073 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.073 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.073 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.073 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.073 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.073 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.073 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.074 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.074 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.074 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.074 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.074 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.074 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.074 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.074 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.074 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.074 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.074 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.074 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.075 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.075 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.075 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.075 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.075 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.075 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.075 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.075 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.075 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.075 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.075 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.075 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.076 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.076 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.076 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.076 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.076 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.076 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.076 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.076 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.076 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.076 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.076 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.077 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.077 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.077 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.077 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.077 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.077 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.077 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.077 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.077 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.077 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.077 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stats", 2024-09-11T23:32:11.078 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.078 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.078 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.078 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.078 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.078 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.078 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.078 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.078 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.078 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.078 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.078 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.078 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.079 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.079 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.079 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.079 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.079 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.079 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.079 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.079 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.079 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.079 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.080 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.080 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.080 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.080 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.080 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.080 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.080 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.080 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.080 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.080 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.080 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.080 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.081 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.081 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.081 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.081 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.081 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.081 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.081 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.081 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.081 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.081 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.081 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.081 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.082 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.082 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.082 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.082 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.082 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.082 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.082 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.082 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.082 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.082 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.082 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.082 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.082 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.083 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.083 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.083 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.083 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.083 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.083 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.083 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.083 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.083 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.083 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.083 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.084 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "status", 2024-09-11T23:32:11.084 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.084 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.084 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.084 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.084 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.084 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.084 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.084 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.085 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.085 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.085 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.085 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.085 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.085 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.085 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.085 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.085 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.085 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.085 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.085 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.085 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.086 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.086 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.086 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.086 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.086 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.086 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.086 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.086 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.086 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.086 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.086 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.086 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.087 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.087 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.087 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.087 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.087 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.087 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.087 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.087 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.087 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.087 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.087 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.087 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.088 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.088 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.088 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.088 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.088 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.088 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.088 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.088 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.088 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.088 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.088 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.088 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.089 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.089 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.089 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.089 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.089 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.089 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.089 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.089 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.089 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.089 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.089 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.089 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.089 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.090 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.090 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telegraf", 2024-09-11T23:32:11.090 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.090 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.090 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.090 INFO:teuthology.orchestra.run.smithi043.stdout: "address": { 2024-09-11T23:32:11.090 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "address", 2024-09-11T23:32:11.090 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.090 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.091 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.091 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T23:32:11.091 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.091 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.091 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.091 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.091 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.091 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.091 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.092 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-11T23:32:11.092 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-11T23:32:11.092 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.092 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.092 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.092 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15", 2024-09-11T23:32:11.092 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.092 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.092 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.092 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.092 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.092 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.092 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.093 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.093 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.093 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.093 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.093 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.093 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.093 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.093 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.094 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.094 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.094 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.094 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.094 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.094 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.094 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.094 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.094 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.094 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.094 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.094 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.094 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.095 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.095 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.095 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.095 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.095 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.095 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.095 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.095 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.095 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.095 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.095 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.095 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.096 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.096 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.096 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.096 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.096 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.096 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.096 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.096 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.096 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.096 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.096 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.096 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.097 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.097 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.097 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.097 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.097 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.097 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.097 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.097 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.097 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.097 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.097 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.098 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.098 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.098 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.098 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.098 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.098 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.098 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.098 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.098 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.098 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.098 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.099 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.099 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telemetry", 2024-09-11T23:32:11.099 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.099 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.099 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.099 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_basic": { 2024-09-11T23:32:11.099 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_basic", 2024-09-11T23:32:11.099 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.099 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.099 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.099 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.100 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.100 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.100 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.100 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T23:32:11.100 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.100 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.100 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.100 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_crash": { 2024-09-11T23:32:11.100 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_crash", 2024-09-11T23:32:11.100 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.100 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.101 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.101 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.101 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.101 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.101 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.101 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T23:32:11.101 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.101 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.101 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.101 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_device": { 2024-09-11T23:32:11.101 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_device", 2024-09-11T23:32:11.101 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.102 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.102 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.102 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.102 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.102 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.102 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.102 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T23:32:11.102 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.102 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.102 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.102 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_ident": { 2024-09-11T23:32:11.102 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_ident", 2024-09-11T23:32:11.103 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.103 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.103 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.103 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.103 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.103 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.103 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.103 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T23:32:11.103 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.103 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.103 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.103 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_perf": { 2024-09-11T23:32:11.104 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_perf", 2024-09-11T23:32:11.104 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.104 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.104 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.104 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.104 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.104 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.104 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.104 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T23:32:11.104 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.104 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.104 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.105 INFO:teuthology.orchestra.run.smithi043.stdout: "contact": { 2024-09-11T23:32:11.105 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "contact", 2024-09-11T23:32:11.105 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.105 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.105 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.105 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.105 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.105 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.105 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.105 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.105 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.105 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.106 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.106 INFO:teuthology.orchestra.run.smithi043.stdout: "description": { 2024-09-11T23:32:11.106 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "description", 2024-09-11T23:32:11.106 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.106 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.106 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.106 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.106 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.106 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.106 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.107 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.107 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.107 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.107 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.107 INFO:teuthology.orchestra.run.smithi043.stdout: "device_url": { 2024-09-11T23:32:11.107 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_url", 2024-09-11T23:32:11.107 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.107 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.107 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.107 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T23:32:11.107 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.107 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.108 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.108 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.108 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.108 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.108 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.108 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-09-11T23:32:11.108 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-09-11T23:32:11.108 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.108 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.108 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.108 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.109 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.109 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.109 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.109 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.109 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.109 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.109 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.109 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-11T23:32:11.109 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-11T23:32:11.109 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.110 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.110 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.110 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "24", 2024-09-11T23:32:11.110 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "8", 2024-09-11T23:32:11.110 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.110 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.110 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.110 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.110 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.110 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.110 INFO:teuthology.orchestra.run.smithi043.stdout: "last_opt_revision": { 2024-09-11T23:32:11.110 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "last_opt_revision", 2024-09-11T23:32:11.111 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.111 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.111 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.111 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1", 2024-09-11T23:32:11.111 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.111 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.111 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.111 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.111 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.111 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.111 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.111 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard": { 2024-09-11T23:32:11.111 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard", 2024-09-11T23:32:11.112 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.112 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.112 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.112 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.112 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.112 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.112 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.112 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.112 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.112 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.112 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.113 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard_description": { 2024-09-11T23:32:11.113 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard_description", 2024-09-11T23:32:11.113 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.113 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.113 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.113 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.113 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.113 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.113 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.113 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.114 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.114 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.114 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.114 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.114 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.114 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.114 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.114 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.114 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.114 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.114 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.114 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.114 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.115 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.115 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.115 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.115 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.115 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.115 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.115 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.115 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.115 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.115 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.115 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.115 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.116 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.116 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.116 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.116 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.116 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.116 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.116 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.116 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.116 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.116 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.116 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.116 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.117 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.117 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.117 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.117 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.117 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.117 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.117 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.117 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.117 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.117 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.117 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.117 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.117 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.118 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.118 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.118 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.118 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.118 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.118 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.118 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.118 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.118 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.118 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.118 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.118 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.118 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.119 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.119 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.119 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.119 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.119 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.119 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.119 INFO:teuthology.orchestra.run.smithi043.stdout: "organization": { 2024-09-11T23:32:11.119 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "organization", 2024-09-11T23:32:11.119 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.119 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.119 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.119 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.120 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.120 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.120 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.120 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.120 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.120 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.120 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.120 INFO:teuthology.orchestra.run.smithi043.stdout: "proxy": { 2024-09-11T23:32:11.120 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "proxy", 2024-09-11T23:32:11.120 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.120 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.120 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.121 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.121 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.121 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.121 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.121 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.121 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.121 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.121 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.121 INFO:teuthology.orchestra.run.smithi043.stdout: "url": { 2024-09-11T23:32:11.121 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url", 2024-09-11T23:32:11.121 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.122 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.122 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.122 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T23:32:11.122 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.122 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.122 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.122 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.122 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.122 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.122 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.122 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.122 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.123 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "test_orchestrator", 2024-09-11T23:32:11.123 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.123 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.123 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.123 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.123 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.123 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.123 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.123 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.123 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.123 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.124 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.124 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.124 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.124 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.124 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.124 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.124 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.124 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.124 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.124 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.124 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.124 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.124 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.125 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.125 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.125 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.125 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.125 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.125 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.125 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.125 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.125 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.125 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.125 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.125 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.126 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.126 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.126 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.126 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.126 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.126 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.126 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.126 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.126 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.126 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.126 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.127 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.127 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.127 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.127 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.127 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.127 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.128 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.128 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.128 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.128 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.128 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.129 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.129 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.129 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.129 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.129 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.129 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.129 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.130 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.130 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.130 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.130 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.130 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.130 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.131 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.131 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "volumes", 2024-09-11T23:32:11.131 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.131 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.131 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.131 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.132 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.132 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.132 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.132 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.132 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.132 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.133 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.133 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.133 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.133 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.133 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.133 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.134 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.134 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.134 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.134 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.134 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.134 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.134 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.135 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.135 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.135 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.135 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.135 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.135 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.135 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.135 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.136 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.136 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.136 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.136 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.136 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.136 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.136 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.136 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.136 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.136 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.136 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.137 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.137 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.137 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.137 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.137 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.137 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.137 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.137 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.137 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.137 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.137 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.137 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.137 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.138 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.138 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.138 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.138 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.138 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.138 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.138 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.138 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.138 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.138 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.138 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.138 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.139 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.139 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.139 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_clones": { 2024-09-11T23:32:11.139 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_clones", 2024-09-11T23:32:11.139 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.139 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.139 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.139 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4", 2024-09-11T23:32:11.139 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.139 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.139 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.139 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T23:32:11.140 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.140 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.140 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.140 INFO:teuthology.orchestra.run.smithi043.stdout: "periodic_async_work": { 2024-09-11T23:32:11.140 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "periodic_async_work", 2024-09-11T23:32:11.140 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.140 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.140 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.140 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.140 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.140 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.140 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.141 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Periodically check for async work", 2024-09-11T23:32:11.141 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.141 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.141 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.141 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_delay": { 2024-09-11T23:32:11.141 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_delay", 2024-09-11T23:32:11.141 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.141 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.141 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.141 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-11T23:32:11.141 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.142 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.142 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.142 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T23:32:11.142 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.142 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.142 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.142 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_no_wait": { 2024-09-11T23:32:11.142 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T23:32:11.142 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.142 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.142 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.142 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.143 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.143 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.143 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.143 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T23:32:11.143 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.143 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.143 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.143 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.143 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.143 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix", 2024-09-11T23:32:11.143 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.144 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.144 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.144 INFO:teuthology.orchestra.run.smithi043.stdout: "discovery_interval": { 2024-09-11T23:32:11.144 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "discovery_interval", 2024-09-11T23:32:11.144 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-11T23:32:11.144 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.144 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.144 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "100", 2024-09-11T23:32:11.144 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.144 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.144 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.144 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.144 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.145 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.145 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.145 INFO:teuthology.orchestra.run.smithi043.stdout: "identifier": { 2024-09-11T23:32:11.145 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "identifier", 2024-09-11T23:32:11.145 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.145 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.145 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.145 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.145 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.145 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.145 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.145 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.146 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.146 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.146 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.146 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-11T23:32:11.146 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-11T23:32:11.146 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.146 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.146 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.146 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-11T23:32:11.146 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.146 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.146 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.147 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.147 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.147 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.147 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.147 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.147 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.147 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.147 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.147 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.147 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.148 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.148 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.148 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.148 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.148 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.148 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.148 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.148 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.148 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.148 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.148 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.148 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.149 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.149 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.149 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.149 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.149 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.149 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.149 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.149 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.149 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.149 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.149 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.149 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.150 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.150 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.150 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.150 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.150 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.150 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.150 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.150 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.151 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.151 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.151 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.151 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.151 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.151 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.151 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.151 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.151 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.151 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.151 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.151 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.151 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.152 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.152 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.152 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.152 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.152 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.152 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.152 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.152 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.152 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.152 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.153 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.153 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.153 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.153 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.153 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.153 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_host": { 2024-09-11T23:32:11.153 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_host", 2024-09-11T23:32:11.153 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.153 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.153 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.153 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.153 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.154 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.154 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.154 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.154 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.154 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.154 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.154 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_port": { 2024-09-11T23:32:11.154 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_port", 2024-09-11T23:32:11.154 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.154 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.154 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.155 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10051", 2024-09-11T23:32:11.155 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.155 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.155 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.155 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.155 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.155 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.155 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.155 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_sender": { 2024-09-11T23:32:11.155 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_sender", 2024-09-11T23:32:11.155 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.155 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.156 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.156 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T23:32:11.156 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.156 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.156 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.156 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.156 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.156 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.156 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.156 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.156 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.156 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.157 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-11T23:32:11.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.157 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 24818, 2024-09-11T23:32:11.157 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "y", 2024-09-11T23:32:11.157 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_features": 4540701547738038271, 2024-09-11T23:32:11.157 INFO:teuthology.orchestra.run.smithi043.stdout: "available_modules": [ 2024-09-11T23:32:11.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.157 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "alerts", 2024-09-11T23:32:11.157 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.157 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.157 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.158 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-11T23:32:11.158 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-11T23:32:11.158 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.158 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.158 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.158 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-11T23:32:11.158 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.158 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.158 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.158 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T23:32:11.158 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.158 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.158 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.159 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.159 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.159 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.159 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.159 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.159 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.159 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.159 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.159 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.159 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.159 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.159 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.159 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.160 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.160 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.160 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.160 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.160 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.160 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.160 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.160 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.160 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.160 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.160 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.161 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.161 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.161 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.161 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.161 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.161 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.161 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.161 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.161 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.161 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.161 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.161 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.162 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.162 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.162 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.162 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.162 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.162 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.162 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.162 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.162 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.162 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.163 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.163 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.163 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.163 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.163 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.163 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.163 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.163 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.163 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.163 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.163 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.163 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.164 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.164 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.164 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.164 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.164 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.164 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.164 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.164 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.164 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_destination": { 2024-09-11T23:32:11.164 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_destination", 2024-09-11T23:32:11.164 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.164 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.165 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.165 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.165 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.165 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.165 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.165 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email address to send alerts to", 2024-09-11T23:32:11.165 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.165 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.165 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.166 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_from_name": { 2024-09-11T23:32:11.166 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_from_name", 2024-09-11T23:32:11.166 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.166 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.166 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.166 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "Ceph", 2024-09-11T23:32:11.166 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.166 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.166 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.166 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email From: name", 2024-09-11T23:32:11.167 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.167 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.167 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.167 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_host": { 2024-09-11T23:32:11.167 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_host", 2024-09-11T23:32:11.167 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.167 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.167 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.167 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.167 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.167 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.167 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.167 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP server", 2024-09-11T23:32:11.168 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.168 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.168 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.168 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_password": { 2024-09-11T23:32:11.168 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_password", 2024-09-11T23:32:11.168 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.168 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.168 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.168 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.168 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.168 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.168 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.169 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Password to authenticate with", 2024-09-11T23:32:11.169 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.169 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.169 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.169 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_port": { 2024-09-11T23:32:11.169 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_port", 2024-09-11T23:32:11.169 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.169 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.169 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.169 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "465", 2024-09-11T23:32:11.169 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.169 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.170 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.170 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP port", 2024-09-11T23:32:11.170 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.170 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.170 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.170 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_sender": { 2024-09-11T23:32:11.170 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_sender", 2024-09-11T23:32:11.170 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.170 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.170 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.170 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.170 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.170 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.171 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.171 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP envelope sender", 2024-09-11T23:32:11.171 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.171 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.171 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.171 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_ssl": { 2024-09-11T23:32:11.171 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_ssl", 2024-09-11T23:32:11.171 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.171 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.171 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.171 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.171 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.172 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.172 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.172 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T23:32:11.172 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.172 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.172 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.172 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_user": { 2024-09-11T23:32:11.172 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_user", 2024-09-11T23:32:11.172 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.172 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.172 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.173 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.173 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.173 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.173 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.173 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "User to authenticate as", 2024-09-11T23:32:11.173 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.173 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.173 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.173 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.173 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.173 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "balancer", 2024-09-11T23:32:11.173 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.174 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.174 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.174 INFO:teuthology.orchestra.run.smithi043.stdout: "active": { 2024-09-11T23:32:11.174 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "active", 2024-09-11T23:32:11.174 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.174 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.174 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.174 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.174 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.174 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.174 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.174 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T23:32:11.174 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.175 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.175 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.175 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_time": { 2024-09-11T23:32:11.175 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_time", 2024-09-11T23:32:11.175 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.175 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.175 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.175 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0000", 2024-09-11T23:32:11.175 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.175 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.175 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.175 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T23:32:11.176 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:32:11.176 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.176 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.176 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_weekday": { 2024-09-11T23:32:11.176 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_weekday", 2024-09-11T23:32:11.176 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-11T23:32:11.176 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.176 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.176 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-11T23:32:11.176 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-09-11T23:32:11.176 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-09-11T23:32:11.176 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.177 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T23:32:11.177 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:32:11.177 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.177 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.177 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_max_iterations": { 2024-09-11T23:32:11.177 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_max_iterations", 2024-09-11T23:32:11.177 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-11T23:32:11.177 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.177 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.177 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "25", 2024-09-11T23:32:11.177 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-11T23:32:11.178 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "250", 2024-09-11T23:32:11.178 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.178 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T23:32:11.178 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.178 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.178 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.178 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_metrics": { 2024-09-11T23:32:11.178 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_metrics", 2024-09-11T23:32:11.178 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.178 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.178 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.178 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T23:32:11.178 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.179 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.179 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.179 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T23:32:11.179 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T23:32:11.179 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.179 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.179 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_step": { 2024-09-11T23:32:11.179 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_step", 2024-09-11T23:32:11.179 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-11T23:32:11.179 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.180 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.180 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.5", 2024-09-11T23:32:11.180 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0.001", 2024-09-11T23:32:11.180 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "0.999", 2024-09-11T23:32:11.180 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.180 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "aggressiveness of optimization", 2024-09-11T23:32:11.180 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T23:32:11.180 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.180 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.180 INFO:teuthology.orchestra.run.smithi043.stdout: "end_time": { 2024-09-11T23:32:11.180 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_time", 2024-09-11T23:32:11.181 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.181 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.181 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.181 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2359", 2024-09-11T23:32:11.181 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.181 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.181 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.181 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T23:32:11.181 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:32:11.181 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.181 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.181 INFO:teuthology.orchestra.run.smithi043.stdout: "end_weekday": { 2024-09-11T23:32:11.182 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_weekday", 2024-09-11T23:32:11.182 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-11T23:32:11.182 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.182 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.182 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-11T23:32:11.182 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-09-11T23:32:11.182 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-09-11T23:32:11.182 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.182 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T23:32:11.182 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:32:11.182 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.182 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.183 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.183 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.183 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.183 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.183 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.183 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.183 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.183 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.183 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.183 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.183 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.183 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.183 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.184 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.184 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.184 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.184 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.184 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.184 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.184 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.184 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.184 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.184 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.185 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.185 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.185 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.185 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.185 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.185 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.185 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.185 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.185 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.185 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.186 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.186 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.186 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.186 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.186 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.186 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.186 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.186 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.186 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.186 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.186 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.186 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.187 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.187 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.187 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.187 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.187 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.187 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.187 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.187 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.187 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.187 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.187 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.187 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.188 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.188 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.188 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.188 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.188 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.188 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.188 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.188 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.188 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.188 INFO:teuthology.orchestra.run.smithi043.stdout: "min_score": { 2024-09-11T23:32:11.188 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_score", 2024-09-11T23:32:11.189 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-11T23:32:11.189 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.189 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.189 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-11T23:32:11.189 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.189 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.189 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.189 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T23:32:11.189 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.189 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.189 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.189 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-09-11T23:32:11.190 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-09-11T23:32:11.190 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.190 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.190 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.190 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "upmap", 2024-09-11T23:32:11.190 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.190 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.190 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.190 INFO:teuthology.orchestra.run.smithi043.stdout: "crush-compat", 2024-09-11T23:32:11.190 INFO:teuthology.orchestra.run.smithi043.stdout: "none", 2024-09-11T23:32:11.190 INFO:teuthology.orchestra.run.smithi043.stdout: "read", 2024-09-11T23:32:11.190 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap", 2024-09-11T23:32:11.190 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap-read" 2024-09-11T23:32:11.191 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.191 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Balancer mode", 2024-09-11T23:32:11.191 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.191 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.191 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.191 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ids": { 2024-09-11T23:32:11.191 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_ids", 2024-09-11T23:32:11.191 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.191 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.192 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.192 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.192 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.192 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.192 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.192 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T23:32:11.192 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.192 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.192 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.192 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-11T23:32:11.192 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-11T23:32:11.193 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.193 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.193 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.193 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-11T23:32:11.193 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.193 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.193 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.193 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T23:32:11.193 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.194 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.194 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.194 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_deviation": { 2024-09-11T23:32:11.194 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_deviation", 2024-09-11T23:32:11.195 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.195 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.195 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.195 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-11T23:32:11.195 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-11T23:32:11.195 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.195 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.195 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T23:32:11.195 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T23:32:11.195 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.195 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.196 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_optimizations": { 2024-09-11T23:32:11.196 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_optimizations", 2024-09-11T23:32:11.196 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-11T23:32:11.196 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.196 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.196 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:11.196 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.196 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.196 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.196 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T23:32:11.196 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.196 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.197 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.197 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.197 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.197 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm", 2024-09-11T23:32:11.197 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.197 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.197 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.197 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_down_multiplier": { 2024-09-11T23:32:11.197 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_down_multiplier", 2024-09-11T23:32:11.198 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-11T23:32:11.198 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.198 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.198 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-09-11T23:32:11.198 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.198 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.199 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.199 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T23:32:11.199 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.199 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.199 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.199 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_refresh_rate": { 2024-09-11T23:32:11.199 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_refresh_rate", 2024-09-11T23:32:11.199 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.199 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.200 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.200 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "20", 2024-09-11T23:32:11.200 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.200 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.200 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.200 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T23:32:11.200 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.200 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.200 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.201 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_starting_port": { 2024-09-11T23:32:11.201 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_starting_port", 2024-09-11T23:32:11.201 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.201 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.201 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.201 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4721", 2024-09-11T23:32:11.201 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.201 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.201 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.201 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T23:32:11.201 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.201 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.202 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.202 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_ptrace": { 2024-09-11T23:32:11.202 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_ptrace", 2024-09-11T23:32:11.202 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.202 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.202 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.202 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.202 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.202 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.202 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.203 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T23:32:11.203 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T23:32:11.203 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.203 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.203 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_interval": { 2024-09-11T23:32:11.203 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_interval", 2024-09-11T23:32:11.203 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.203 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.203 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.203 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-11T23:32:11.203 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.203 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.204 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.204 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T23:32:11.204 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.204 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.204 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.204 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_memory_target_ratio": { 2024-09-11T23:32:11.204 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T23:32:11.204 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-11T23:32:11.204 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.204 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.204 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.7", 2024-09-11T23:32:11.205 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.205 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.205 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.205 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T23:32:11.205 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.205 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.205 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.205 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm_log_destination": { 2024-09-11T23:32:11.205 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm_log_destination", 2024-09-11T23:32:11.205 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.205 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.205 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.206 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.206 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.206 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.206 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.206 INFO:teuthology.orchestra.run.smithi043.stdout: "file", 2024-09-11T23:32:11.206 INFO:teuthology.orchestra.run.smithi043.stdout: "file,syslog", 2024-09-11T23:32:11.206 INFO:teuthology.orchestra.run.smithi043.stdout: "syslog" 2024-09-11T23:32:11.206 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.206 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T23:32:11.206 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.206 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.206 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.206 INFO:teuthology.orchestra.run.smithi043.stdout: "cgroups_split": { 2024-09-11T23:32:11.207 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cgroups_split", 2024-09-11T23:32:11.207 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.207 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.207 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.207 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.207 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.207 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.207 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.207 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T23:32:11.207 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.207 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.207 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.208 INFO:teuthology.orchestra.run.smithi043.stdout: "config_checks_enabled": { 2024-09-11T23:32:11.208 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_checks_enabled", 2024-09-11T23:32:11.208 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.208 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.208 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.208 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.208 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.208 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.208 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.208 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T23:32:11.208 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.208 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.208 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.209 INFO:teuthology.orchestra.run.smithi043.stdout: "config_dashboard": { 2024-09-11T23:32:11.209 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_dashboard", 2024-09-11T23:32:11.209 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.209 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.209 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.209 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.209 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.209 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.209 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.209 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T23:32:11.209 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.209 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.209 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.210 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_alertmanager": { 2024-09-11T23:32:11.210 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_alertmanager", 2024-09-11T23:32:11.210 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.210 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.210 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.210 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-11T23:32:11.210 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.210 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.210 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.210 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-11T23:32:11.211 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.211 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.211 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.211 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_base": { 2024-09-11T23:32:11.211 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_base", 2024-09-11T23:32:11.211 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.211 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.211 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.211 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T23:32:11.211 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.211 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.211 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.211 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Container image name, without the tag", 2024-09-11T23:32:11.212 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.212 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.212 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.212 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_elasticsearch": { 2024-09-11T23:32:11.212 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_elasticsearch", 2024-09-11T23:32:11.212 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.212 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.212 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.212 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T23:32:11.212 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.212 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.213 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.213 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "elasticsearch container image", 2024-09-11T23:32:11.213 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.213 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.213 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.213 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_grafana": { 2024-09-11T23:32:11.213 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_grafana", 2024-09-11T23:32:11.213 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.213 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.213 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.213 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-11T23:32:11.214 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.214 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.214 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.214 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-11T23:32:11.214 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.214 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.214 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.214 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_haproxy": { 2024-09-11T23:32:11.214 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_haproxy", 2024-09-11T23:32:11.214 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.214 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.214 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.215 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T23:32:11.215 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.215 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.215 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.215 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "HAproxy container image", 2024-09-11T23:32:11.215 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.215 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.215 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.215 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_agent": { 2024-09-11T23:32:11.215 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_agent", 2024-09-11T23:32:11.215 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.216 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.216 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.216 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T23:32:11.216 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.216 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.216 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.216 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger agent container image", 2024-09-11T23:32:11.216 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.216 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.216 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.216 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_collector": { 2024-09-11T23:32:11.217 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_collector", 2024-09-11T23:32:11.217 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.217 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.217 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.217 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T23:32:11.217 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.217 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.217 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.217 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger collector container image", 2024-09-11T23:32:11.217 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.217 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.217 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.218 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_query": { 2024-09-11T23:32:11.218 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_query", 2024-09-11T23:32:11.218 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.218 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.218 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.218 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T23:32:11.218 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.218 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.218 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.218 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger query container image", 2024-09-11T23:32:11.218 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.218 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.219 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.219 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_keepalived": { 2024-09-11T23:32:11.219 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_keepalived", 2024-09-11T23:32:11.219 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.219 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.219 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.219 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T23:32:11.219 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.219 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.219 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.219 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Keepalived container image", 2024-09-11T23:32:11.219 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.220 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.220 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.220 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_loki": { 2024-09-11T23:32:11.220 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_loki", 2024-09-11T23:32:11.220 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.220 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.220 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.220 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T23:32:11.220 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.220 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.220 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.220 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Loki container image", 2024-09-11T23:32:11.221 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.221 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.221 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.221 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_node_exporter": { 2024-09-11T23:32:11.221 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_node_exporter", 2024-09-11T23:32:11.221 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.221 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.221 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.221 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-11T23:32:11.221 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.221 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.221 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.222 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-11T23:32:11.222 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.222 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.222 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.222 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_nvmeof": { 2024-09-11T23:32:11.222 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_nvmeof", 2024-09-11T23:32:11.222 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.222 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.222 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.222 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T23:32:11.222 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.222 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.223 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.223 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Nvme-of container image", 2024-09-11T23:32:11.223 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.223 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.223 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.223 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_prometheus": { 2024-09-11T23:32:11.223 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_prometheus", 2024-09-11T23:32:11.223 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.223 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.223 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.223 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-11T23:32:11.223 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.224 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.224 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.224 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-11T23:32:11.224 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.224 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.224 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.224 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_promtail": { 2024-09-11T23:32:11.224 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_promtail", 2024-09-11T23:32:11.224 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.224 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.225 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.225 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T23:32:11.225 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.225 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.225 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.225 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Promtail container image", 2024-09-11T23:32:11.225 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.225 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.225 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.225 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_samba": { 2024-09-11T23:32:11.225 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_samba", 2024-09-11T23:32:11.225 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.226 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.226 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.226 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T23:32:11.226 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.226 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.226 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.226 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Samba/SMB container image", 2024-09-11T23:32:11.226 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.226 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.226 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.226 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_snmp_gateway": { 2024-09-11T23:32:11.226 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_snmp_gateway", 2024-09-11T23:32:11.227 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.228 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.228 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.228 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T23:32:11.228 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.228 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.228 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.228 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SNMP Gateway container image", 2024-09-11T23:32:11.228 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.228 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.228 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.229 INFO:teuthology.orchestra.run.smithi043.stdout: "container_init": { 2024-09-11T23:32:11.229 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_init", 2024-09-11T23:32:11.229 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.229 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.229 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.229 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.229 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.229 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.229 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.229 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T23:32:11.229 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.229 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.229 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.230 INFO:teuthology.orchestra.run.smithi043.stdout: "daemon_cache_timeout": { 2024-09-11T23:32:11.230 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "daemon_cache_timeout", 2024-09-11T23:32:11.230 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.230 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.230 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.230 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-11T23:32:11.230 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.230 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.230 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.230 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T23:32:11.230 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.231 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.231 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.231 INFO:teuthology.orchestra.run.smithi043.stdout: "default_cephadm_command_timeout": { 2024-09-11T23:32:11.231 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T23:32:11.231 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.231 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.231 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.231 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "900", 2024-09-11T23:32:11.231 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.231 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.231 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.231 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T23:32:11.232 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.232 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.232 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.232 INFO:teuthology.orchestra.run.smithi043.stdout: "default_registry": { 2024-09-11T23:32:11.232 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_registry", 2024-09-11T23:32:11.232 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.232 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.232 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.232 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io", 2024-09-11T23:32:11.232 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.232 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.232 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.233 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T23:32:11.233 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.233 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.233 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.233 INFO:teuthology.orchestra.run.smithi043.stdout: "device_cache_timeout": { 2024-09-11T23:32:11.233 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_cache_timeout", 2024-09-11T23:32:11.233 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.233 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.233 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.233 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1800", 2024-09-11T23:32:11.233 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.233 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.234 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.234 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache device inventory", 2024-09-11T23:32:11.234 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.234 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.234 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.234 INFO:teuthology.orchestra.run.smithi043.stdout: "device_enhanced_scan": { 2024-09-11T23:32:11.234 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_enhanced_scan", 2024-09-11T23:32:11.234 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.234 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.234 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.234 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.235 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.235 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.235 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.235 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T23:32:11.235 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.235 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.235 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.235 INFO:teuthology.orchestra.run.smithi043.stdout: "facts_cache_timeout": { 2024-09-11T23:32:11.235 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "facts_cache_timeout", 2024-09-11T23:32:11.235 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.235 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.235 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.235 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-11T23:32:11.236 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.236 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.236 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.236 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache host facts data", 2024-09-11T23:32:11.236 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.236 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.236 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.236 INFO:teuthology.orchestra.run.smithi043.stdout: "grafana_dashboards_path": { 2024-09-11T23:32:11.237 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "grafana_dashboards_path", 2024-09-11T23:32:11.237 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.237 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.237 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.237 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T23:32:11.237 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.237 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.237 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.237 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T23:32:11.238 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.238 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.238 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.238 INFO:teuthology.orchestra.run.smithi043.stdout: "host_check_interval": { 2024-09-11T23:32:11.238 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "host_check_interval", 2024-09-11T23:32:11.238 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.238 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.238 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.238 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-11T23:32:11.238 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.238 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.238 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.239 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to perform a host check", 2024-09-11T23:32:11.239 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.239 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.239 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.239 INFO:teuthology.orchestra.run.smithi043.stdout: "hw_monitoring": { 2024-09-11T23:32:11.239 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hw_monitoring", 2024-09-11T23:32:11.239 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.239 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.239 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.239 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.239 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.239 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.240 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.240 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T23:32:11.240 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.240 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.240 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.240 INFO:teuthology.orchestra.run.smithi043.stdout: "inventory_list_all": { 2024-09-11T23:32:11.240 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "inventory_list_all", 2024-09-11T23:32:11.240 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.240 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.240 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.240 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.240 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.241 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.241 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.241 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T23:32:11.241 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.241 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.241 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.241 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.241 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.241 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.241 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.241 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.241 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.242 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.242 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.242 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.242 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.242 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.242 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.242 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.242 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.242 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.242 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.242 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.242 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.242 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.243 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.243 INFO:teuthology.orchestra.run.smithi043.stdout: "log_refresh_metadata": { 2024-09-11T23:32:11.243 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_refresh_metadata", 2024-09-11T23:32:11.243 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.243 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.243 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.243 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.243 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.243 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.243 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.243 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T23:32:11.243 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.244 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.244 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.244 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.244 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.244 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.244 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.244 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.244 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.244 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.244 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.244 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T23:32:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.248 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T23:32:11.248 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T23:32:11.248 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.248 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.248 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.248 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.248 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.248 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.248 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.249 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T23:32:11.249 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.249 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.249 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.249 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T23:32:11.249 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T23:32:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "*", 2024-09-11T23:32:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T23:32:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.251 INFO:teuthology.orchestra.run.smithi043.stdout: "max_count_per_host": { 2024-09-11T23:32:11.251 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_count_per_host", 2024-09-11T23:32:11.251 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.251 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.251 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.251 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:11.251 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.251 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.251 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of daemons per service per host", 2024-09-11T23:32:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: "max_osd_draining_count": { 2024-09-11T23:32:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_osd_draining_count", 2024-09-11T23:32:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T23:32:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: "migration_current": { 2024-09-11T23:32:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "migration_current", 2024-09-11T23:32:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "internal - do not modify", 2024-09-11T23:32:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-09-11T23:32:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-09-11T23:32:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.255 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.255 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "root", 2024-09-11T23:32:11.255 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.255 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.255 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.255 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm-package", 2024-09-11T23:32:11.255 INFO:teuthology.orchestra.run.smithi043.stdout: "root" 2024-09-11T23:32:11.255 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.255 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T23:32:11.255 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.255 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.255 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.256 INFO:teuthology.orchestra.run.smithi043.stdout: "oob_default_addr": { 2024-09-11T23:32:11.256 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "oob_default_addr", 2024-09-11T23:32:11.256 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.256 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.256 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.256 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "169.254.1.1", 2024-09-11T23:32:11.256 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.256 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.256 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.256 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T23:32:11.256 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.256 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.257 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.257 INFO:teuthology.orchestra.run.smithi043.stdout: "prometheus_alerts_path": { 2024-09-11T23:32:11.257 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus_alerts_path", 2024-09-11T23:32:11.257 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.257 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.257 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.257 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T23:32:11.257 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.257 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T23:32:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_insecure": { 2024-09-11T23:32:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_insecure", 2024-09-11T23:32:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.259 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.259 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.259 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.259 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T23:32:11.259 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.259 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.259 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.259 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_password": { 2024-09-11T23:32:11.260 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_password", 2024-09-11T23:32:11.260 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.260 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.260 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.260 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.260 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.260 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.260 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.260 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T23:32:11.260 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.260 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.260 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_url": { 2024-09-11T23:32:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_url", 2024-09-11T23:32:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T23:32:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.262 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_username": { 2024-09-11T23:32:11.262 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_username", 2024-09-11T23:32:11.262 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.262 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.262 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.262 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.262 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.262 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.262 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.263 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T23:32:11.263 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.263 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.263 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.263 INFO:teuthology.orchestra.run.smithi043.stdout: "secure_monitoring_stack": { 2024-09-11T23:32:11.263 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "secure_monitoring_stack", 2024-09-11T23:32:11.263 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.263 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.263 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.264 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.264 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.264 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.264 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.264 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T23:32:11.264 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.264 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.264 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.264 INFO:teuthology.orchestra.run.smithi043.stdout: "service_discovery_port": { 2024-09-11T23:32:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "service_discovery_port", 2024-09-11T23:32:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8765", 2024-09-11T23:32:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "cephadm service discovery port", 2024-09-11T23:32:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.266 INFO:teuthology.orchestra.run.smithi043.stdout: "ssh_config_file": { 2024-09-11T23:32:11.266 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssh_config_file", 2024-09-11T23:32:11.266 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.266 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.266 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.266 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.266 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.266 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.266 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T23:32:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: "use_agent": { 2024-09-11T23:32:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_agent", 2024-09-11T23:32:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T23:32:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: "use_repo_digest": { 2024-09-11T23:32:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_repo_digest", 2024-09-11T23:32:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T23:32:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_failed_host_check": { 2024-09-11T23:32:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_failed_host_check", 2024-09-11T23:32:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.270 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.270 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.270 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.270 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.270 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.270 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T23:32:11.270 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.270 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.270 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_daemons": { 2024-09-11T23:32:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_daemons", 2024-09-11T23:32:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T23:32:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_hosts": { 2024-09-11T23:32:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_hosts", 2024-09-11T23:32:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T23:32:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crash", 2024-09-11T23:32:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: "retain_interval": { 2024-09-11T23:32:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retain_interval", 2024-09-11T23:32:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "31536000", 2024-09-11T23:32:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T23:32:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_recent_interval": { 2024-09-11T23:32:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_recent_interval", 2024-09-11T23:32:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1209600", 2024-09-11T23:32:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T23:32:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dashboard", 2024-09-11T23:32:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T23:32:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T23:32:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T23:32:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T23:32:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T23:32:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T23:32:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.285 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.285 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.285 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.285 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.285 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.285 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.285 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_ENABLED": { 2024-09-11T23:32:11.285 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T23:32:11.285 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T23:32:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T23:32:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T23:32:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T23:32:11.288 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.288 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.288 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.288 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.288 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.288 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.288 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.288 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.288 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.288 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.288 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.289 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T23:32:11.289 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T23:32:11.289 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.289 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.289 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.289 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.289 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.289 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.289 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.289 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.289 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.289 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T23:32:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T23:32:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T23:32:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T23:32:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.292 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.292 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.292 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.292 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T23:32:11.292 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T23:32:11.292 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.292 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.292 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.292 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T23:32:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T23:32:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.294 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.294 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.294 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.294 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.294 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.294 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.294 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.294 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.294 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.294 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T23:32:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T23:32:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.296 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T23:32:11.296 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T23:32:11.296 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.296 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.296 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.296 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.296 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.296 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T23:32:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T23:32:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T23:32:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T23:32:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-09-11T23:32:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T23:32:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T23:32:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.301 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.301 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.301 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.301 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.301 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_URL": { 2024-09-11T23:32:11.301 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_URL", 2024-09-11T23:32:11.301 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.301 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.301 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.301 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T23:32:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T23:32:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.303 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.303 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.303 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-09-11T23:32:11.303 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.303 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.303 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.303 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.303 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.303 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.303 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T23:32:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T23:32:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.305 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.305 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.305 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T23:32:11.305 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T23:32:11.305 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.305 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.305 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.305 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.305 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.305 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.306 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.306 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.306 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.306 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.306 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.306 INFO:teuthology.orchestra.run.smithi043.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T23:32:11.306 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T23:32:11.306 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.306 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.306 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T23:32:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T23:32:11.308 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.308 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.308 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.308 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.308 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.308 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.308 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.308 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.308 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.308 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.308 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T23:32:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T23:32:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T23:32:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T23:32:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.311 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.311 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.311 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.311 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.311 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.311 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T23:32:11.311 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T23:32:11.311 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.311 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.311 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.312 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.312 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.312 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.312 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.312 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.312 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.312 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.312 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.312 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T23:32:11.312 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T23:32:11.313 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.313 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.313 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.313 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.313 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.313 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.313 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.313 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.313 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.313 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.313 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.314 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T23:32:11.314 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T23:32:11.314 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.314 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.314 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.314 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.314 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.314 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.314 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.314 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.314 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.315 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.315 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.315 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T23:32:11.315 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T23:32:11.315 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.315 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.315 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.315 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.315 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.315 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.315 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.316 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.316 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.316 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.316 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.316 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T23:32:11.316 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T23:32:11.316 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.316 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.316 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.316 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.317 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.317 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.317 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.317 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.317 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.317 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.317 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.317 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T23:32:11.317 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T23:32:11.317 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T23:32:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T23:32:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T23:32:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T23:32:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.321 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.321 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.321 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.321 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.321 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T23:32:11.321 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T23:32:11.321 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.321 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.321 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.322 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T23:32:11.322 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.322 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.322 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.322 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.322 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.322 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.322 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.322 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T23:32:11.323 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T23:32:11.323 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.323 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.323 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.323 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:11.323 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.323 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.323 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.323 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.323 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.323 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T23:32:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T23:32:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8", 2024-09-11T23:32:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.325 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.325 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.325 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.325 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.325 INFO:teuthology.orchestra.run.smithi043.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T23:32:11.325 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T23:32:11.325 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.325 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.326 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.326 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "45", 2024-09-11T23:32:11.326 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.326 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.326 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.326 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.326 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.326 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.326 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.326 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T23:32:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T23:32:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.328 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.328 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.328 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T23:32:11.328 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T23:32:11.328 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.328 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.328 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.328 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-09-11T23:32:11.328 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.329 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.329 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.329 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.329 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.329 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.329 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.332 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T23:32:11.332 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T23:32:11.332 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.332 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.332 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.332 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.332 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.332 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.332 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.332 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T23:32:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T23:32:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.334 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.334 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.334 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.334 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.334 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.334 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.334 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T23:32:11.334 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T23:32:11.334 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.334 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-11T23:32:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T23:32:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T23:32:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.337 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.337 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T23:32:11.337 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T23:32:11.337 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.337 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.337 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.337 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-11T23:32:11.337 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.337 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.337 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.338 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.338 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.338 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.338 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.338 INFO:teuthology.orchestra.run.smithi043.stdout: "cross_origin_url": { 2024-09-11T23:32:11.338 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cross_origin_url", 2024-09-11T23:32:11.338 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.338 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.338 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.338 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: "crt_file": { 2024-09-11T23:32:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crt_file", 2024-09-11T23:32:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: "debug": { 2024-09-11T23:32:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "debug", 2024-09-11T23:32:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable/disable debug options", 2024-09-11T23:32:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: "jwt_token_ttl": { 2024-09-11T23:32:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "jwt_token_ttl", 2024-09-11T23:32:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "28800", 2024-09-11T23:32:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-09-11T23:32:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-09-11T23:32:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.346 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.346 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.346 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.346 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.346 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.346 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.346 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.346 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.346 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.346 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.349 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.349 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.349 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.349 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.349 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.349 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.349 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.349 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.349 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.349 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: "motd": { 2024-09-11T23:32:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "motd", 2024-09-11T23:32:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "The message of the day", 2024-09-11T23:32:11.352 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.352 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.352 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.352 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect_resolve_ip_addr": { 2024-09-11T23:32:11.352 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T23:32:11.352 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.352 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.352 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.352 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-09-11T23:32:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-09-11T23:32:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-09-11T23:32:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-09-11T23:32:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-09-11T23:32:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8080", 2024-09-11T23:32:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-09-11T23:32:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-09-11T23:32:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.357 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.357 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.357 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl_server_port": { 2024-09-11T23:32:11.357 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl_server_port", 2024-09-11T23:32:11.357 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.357 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.357 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.357 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8443", 2024-09-11T23:32:11.357 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.357 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-09-11T23:32:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-09-11T23:32:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.359 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "redirect", 2024-09-11T23:32:11.359 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.359 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.359 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.359 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.359 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect" 2024-09-11T23:32:11.359 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.359 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.359 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.359 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-09-11T23:32:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-09-11T23:32:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-09-11T23:32:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-09-11T23:32:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-09-11T23:32:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.361 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.361 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.361 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.361 INFO:teuthology.orchestra.run.smithi043.stdout: "url_prefix": { 2024-09-11T23:32:11.361 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url_prefix", 2024-09-11T23:32:11.361 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.361 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.361 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.361 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "devicehealth", 2024-09-11T23:32:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_monitoring": { 2024-09-11T23:32:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_monitoring", 2024-09-11T23:32:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "monitor device health metrics", 2024-09-11T23:32:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.366 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.366 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.366 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.366 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.366 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.366 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.366 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.366 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.366 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.366 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.367 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.367 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.367 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.367 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.367 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.367 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.367 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.367 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.367 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.367 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "mark_out_threshold": { 2024-09-11T23:32:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mark_out_threshold", 2024-09-11T23:32:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2419200", 2024-09-11T23:32:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T23:32:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": { 2024-09-11T23:32:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_name", 2024-09-11T23:32:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "device_health_metrics", 2024-09-11T23:32:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T23:32:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "retention_period": { 2024-09-11T23:32:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retention_period", 2024-09-11T23:32:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15552000", 2024-09-11T23:32:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain device health metrics", 2024-09-11T23:32:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_frequency": { 2024-09-11T23:32:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_frequency", 2024-09-11T23:32:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-09-11T23:32:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T23:32:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "self_heal": { 2024-09-11T23:32:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "self_heal", 2024-09-11T23:32:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T23:32:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-11T23:32:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-11T23:32:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-11T23:32:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T23:32:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_threshold": { 2024-09-11T23:32:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_threshold", 2024-09-11T23:32:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7257600", 2024-09-11T23:32:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T23:32:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "diskprediction_local", 2024-09-11T23:32:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.385 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.385 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.385 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.385 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.385 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.385 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.385 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.385 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.385 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.385 INFO:teuthology.orchestra.run.smithi043.stdout: "predict_interval": { 2024-09-11T23:32:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predict_interval", 2024-09-11T23:32:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-09-11T23:32:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "predictor_model": { 2024-09-11T23:32:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predictor_model", 2024-09-11T23:32:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "prophetstor", 2024-09-11T23:32:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-11T23:32:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-11T23:32:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-11T23:32:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "influx", 2024-09-11T23:32:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": false, 2024-09-11T23:32:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "influxdb python module not found", 2024-09-11T23:32:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "batch_size": { 2024-09-11T23:32:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "batch_size", 2024-09-11T23:32:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5000", 2024-09-11T23:32:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T23:32:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "database": { 2024-09-11T23:32:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "database", 2024-09-11T23:32:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "ceph", 2024-09-11T23:32:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T23:32:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": { 2024-09-11T23:32:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hostname", 2024-09-11T23:32:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server hostname", 2024-09-11T23:32:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-11T23:32:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-11T23:32:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "30", 2024-09-11T23:32:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "5", 2024-09-11T23:32:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T23:32:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "password": { 2024-09-11T23:32:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "password", 2024-09-11T23:32:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "password of InfluxDB server user", 2024-09-11T23:32:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "port": { 2024-09-11T23:32:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "port", 2024-09-11T23:32:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8086", 2024-09-11T23:32:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server port", 2024-09-11T23:32:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-09-11T23:32:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-09-11T23:32:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "false", 2024-09-11T23:32:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:32:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "threads": { 2024-09-11T23:32:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threads", 2024-09-11T23:32:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-11T23:32:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-11T23:32:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "32", 2024-09-11T23:32:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T23:32:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "username": { 2024-09-11T23:32:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "username", 2024-09-11T23:32:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "username of InfluxDB server user", 2024-09-11T23:32:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "verify_ssl": { 2024-09-11T23:32:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "verify_ssl", 2024-09-11T23:32:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "true", 2024-09-11T23:32:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:32:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "insights", 2024-09-11T23:32:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "iostat", 2024-09-11T23:32:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "k8sevents", 2024-09-11T23:32:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_event_retention_days": { 2024-09-11T23:32:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ceph_event_retention_days", 2024-09-11T23:32:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7", 2024-09-11T23:32:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.419 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.419 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.419 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T23:32:11.419 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.419 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.419 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.419 INFO:teuthology.orchestra.run.smithi043.stdout: "config_check_secs": { 2024-09-11T23:32:11.419 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_check_secs", 2024-09-11T23:32:11.419 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.419 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.419 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.419 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "10", 2024-09-11T23:32:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T23:32:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "localpool", 2024-09-11T23:32:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: "failure_domain": { 2024-09-11T23:32:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "failure_domain", 2024-09-11T23:32:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "host", 2024-09-11T23:32:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "failure domain for any created local pool", 2024-09-11T23:32:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T23:32:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": { 2024-09-11T23:32:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_size", 2024-09-11T23:32:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default min_size for any created local pool", 2024-09-11T23:32:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T23:32:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rep": { 2024-09-11T23:32:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "num_rep", 2024-09-11T23:32:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3", 2024-09-11T23:32:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default replica count for any created local pool", 2024-09-11T23:32:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": { 2024-09-11T23:32:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_num", 2024-09-11T23:32:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "128", 2024-09-11T23:32:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T23:32:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "prefix": { 2024-09-11T23:32:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prefix", 2024-09-11T23:32:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name prefix for any created local pool", 2024-09-11T23:32:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "subtree": { 2024-09-11T23:32:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "subtree", 2024-09-11T23:32:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "rack", 2024-09-11T23:32:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T23:32:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T23:32:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mds_autoscaler", 2024-09-11T23:32:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.439 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.439 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.439 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.439 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.439 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.439 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.439 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.439 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.439 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.439 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.439 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirroring", 2024-09-11T23:32:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "nfs", 2024-09-11T23:32:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-09-11T23:32:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: "fail_fs": { 2024-09-11T23:32:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "fail_fs", 2024-09-11T23:32:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T23:32:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator": { 2024-09-11T23:32:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-09-11T23:32:11.463 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.463 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.463 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.463 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.463 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.463 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.463 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.463 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm", 2024-09-11T23:32:11.463 INFO:teuthology.orchestra.run.smithi043.stdout: "rook", 2024-09-11T23:32:11.463 INFO:teuthology.orchestra.run.smithi043.stdout: "test_orchestrator" 2024-09-11T23:32:11.463 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.463 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Orchestrator backend", 2024-09-11T23:32:11.464 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.464 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.464 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.464 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.464 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.464 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_perf_query", 2024-09-11T23:32:11.464 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.464 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.464 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.464 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.464 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.464 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.465 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.465 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.465 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.465 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.465 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.465 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.465 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.465 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.465 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.465 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.465 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.465 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.466 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.466 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.466 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.466 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.466 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.466 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.466 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.466 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.466 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.466 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.466 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.466 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.467 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.467 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.467 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.467 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.467 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.467 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.467 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.467 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.467 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.467 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.467 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.467 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.468 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.468 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.468 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.468 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.468 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.468 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.468 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.468 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.468 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.468 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.468 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.468 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.469 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.469 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.469 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.469 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.469 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.469 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.469 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.469 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.469 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.469 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.469 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.469 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.469 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.470 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.470 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.470 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.470 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.470 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_support", 2024-09-11T23:32:11.470 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.470 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.470 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.470 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.470 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.470 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.471 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.471 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.471 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.471 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.471 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.471 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.471 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.471 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.471 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.471 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.471 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.471 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.471 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.472 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.472 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.472 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.472 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.472 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.472 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.472 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.472 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.472 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.472 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.472 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.472 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.473 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.473 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.473 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.473 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.473 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.473 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.473 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.473 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.473 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.473 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.473 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.473 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.474 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.474 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.474 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.474 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.474 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.474 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.474 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.474 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.474 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.474 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.474 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.474 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.474 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.475 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.475 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.475 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.475 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.475 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.475 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.475 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.475 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.475 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.475 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.475 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.475 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.475 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.476 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.476 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.476 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_autoscaler", 2024-09-11T23:32:11.476 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.476 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.476 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.476 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.476 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.476 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.476 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.476 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.477 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.477 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.477 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.477 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.477 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.477 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.477 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.477 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.477 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.477 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.477 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.477 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.477 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.477 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.478 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.478 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.478 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.478 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.478 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.478 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.478 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.478 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.478 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.478 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.478 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.479 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.479 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.479 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.479 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.479 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.479 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.479 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.479 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.479 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.479 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.479 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.479 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.479 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.480 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.480 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.480 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.480 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.480 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.480 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.480 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.480 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.480 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.480 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.480 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.480 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.481 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.481 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.481 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.481 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.481 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.481 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.481 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.481 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.481 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.481 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.481 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.481 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-11T23:32:11.482 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-11T23:32:11.482 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.482 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.482 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.482 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-11T23:32:11.482 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.482 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.482 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.482 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.482 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.482 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.482 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.482 INFO:teuthology.orchestra.run.smithi043.stdout: "threshold": { 2024-09-11T23:32:11.483 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threshold", 2024-09-11T23:32:11.483 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-11T23:32:11.483 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.483 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.483 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-09-11T23:32:11.483 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1.0", 2024-09-11T23:32:11.483 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.483 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.483 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "scaling threshold", 2024-09-11T23:32:11.483 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T23:32:11.483 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.483 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.483 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.484 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.484 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "progress", 2024-09-11T23:32:11.484 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.484 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.484 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.484 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_pg_recovery_event": { 2024-09-11T23:32:11.484 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_pg_recovery_event", 2024-09-11T23:32:11.484 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.484 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.484 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.484 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.485 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.485 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.485 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.485 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T23:32:11.485 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.485 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.485 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.485 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-09-11T23:32:11.485 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-09-11T23:32:11.485 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.485 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.485 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.485 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.486 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.486 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.486 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.486 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.486 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.486 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.486 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.486 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.486 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.486 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.486 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.486 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.487 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.487 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.487 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.487 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.487 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.487 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.487 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.487 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.487 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.487 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.487 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.487 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.487 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.487 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.488 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.488 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.488 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.488 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.488 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.488 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.488 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.488 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.488 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.488 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.488 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.488 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.489 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.489 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.489 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.489 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.489 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.489 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.489 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.489 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.489 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.489 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.489 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.489 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.490 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.490 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.490 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.490 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.490 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.490 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.490 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.490 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.490 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.490 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.490 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.490 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.491 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.491 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.491 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.491 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.491 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.491 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.491 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.491 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.491 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.491 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.491 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.492 INFO:teuthology.orchestra.run.smithi043.stdout: "max_completed_events": { 2024-09-11T23:32:11.492 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_completed_events", 2024-09-11T23:32:11.492 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.492 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.492 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.492 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "50", 2024-09-11T23:32:11.492 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.492 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.492 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.492 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "number of past completed events to remember", 2024-09-11T23:32:11.492 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.492 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.492 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.493 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-11T23:32:11.493 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-11T23:32:11.493 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.493 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.493 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.493 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-11T23:32:11.493 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.493 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.493 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.493 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long the module is going to sleep", 2024-09-11T23:32:11.493 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.493 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.494 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.494 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.494 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.494 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus", 2024-09-11T23:32:11.494 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.494 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.494 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.494 INFO:teuthology.orchestra.run.smithi043.stdout: "cache": { 2024-09-11T23:32:11.494 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cache", 2024-09-11T23:32:11.494 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.495 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.495 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.495 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.495 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.495 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.495 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.495 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.495 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.495 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.495 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.495 INFO:teuthology.orchestra.run.smithi043.stdout: "exclude_perf_counters": { 2024-09-11T23:32:11.495 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "exclude_perf_counters", 2024-09-11T23:32:11.496 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.496 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.496 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.496 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.496 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.496 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.496 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.496 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T23:32:11.496 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T23:32:11.496 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.496 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.496 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.497 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.497 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.497 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.497 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.497 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.497 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.497 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.497 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.497 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.497 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.497 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.497 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.497 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.497 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.498 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.498 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.498 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.498 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.498 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.498 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.498 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.498 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.498 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.498 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.498 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.498 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.498 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.499 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.499 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.499 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.499 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.499 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.499 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.499 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.499 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.499 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.499 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.499 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.499 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.500 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.500 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.500 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.500 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.500 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.500 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.500 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.500 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.500 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.500 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.500 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.500 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.500 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.501 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.501 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.501 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.501 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.501 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.501 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.501 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.501 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.501 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.501 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.501 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.501 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.501 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.502 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools": { 2024-09-11T23:32:11.502 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools", 2024-09-11T23:32:11.502 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.502 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.502 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.502 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.502 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.502 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.502 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.502 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.502 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.502 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.502 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.503 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T23:32:11.503 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T23:32:11.503 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.503 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.503 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.503 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "300", 2024-09-11T23:32:11.503 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.503 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.503 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.503 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.503 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.503 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.504 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.504 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_interval": { 2024-09-11T23:32:11.504 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_interval", 2024-09-11T23:32:11.504 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-11T23:32:11.504 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.504 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.504 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15.0", 2024-09-11T23:32:11.504 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.504 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.504 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.504 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.505 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.505 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.505 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.505 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-09-11T23:32:11.505 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-09-11T23:32:11.505 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.505 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.505 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.505 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-09-11T23:32:11.505 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.505 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.505 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.506 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T23:32:11.506 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.506 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.506 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.506 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-09-11T23:32:11.506 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-09-11T23:32:11.506 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.506 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.506 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.506 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "9283", 2024-09-11T23:32:11.506 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.506 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.507 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.507 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T23:32:11.507 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.507 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.507 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.507 INFO:teuthology.orchestra.run.smithi043.stdout: "stale_cache_strategy": { 2024-09-11T23:32:11.507 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stale_cache_strategy", 2024-09-11T23:32:11.507 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.507 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.507 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.507 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "log", 2024-09-11T23:32:11.507 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.508 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.508 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.508 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.508 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.508 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.508 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.508 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-09-11T23:32:11.508 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-09-11T23:32:11.508 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.508 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.508 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.509 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "default", 2024-09-11T23:32:11.509 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.509 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.509 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.509 INFO:teuthology.orchestra.run.smithi043.stdout: "default", 2024-09-11T23:32:11.509 INFO:teuthology.orchestra.run.smithi043.stdout: "error" 2024-09-11T23:32:11.509 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.509 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.509 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.509 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.509 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.509 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-09-11T23:32:11.510 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-09-11T23:32:11.510 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.510 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.510 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.510 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-09-11T23:32:11.510 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-09-11T23:32:11.510 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-09-11T23:32:11.510 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.510 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.510 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.510 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.510 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.510 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.511 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.511 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_support", 2024-09-11T23:32:11.511 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.511 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.511 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.511 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.511 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.511 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.511 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.511 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.511 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.511 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.512 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.512 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.512 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.512 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.512 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.512 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.512 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.512 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.512 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.512 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.512 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.512 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.512 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.513 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.513 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.513 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.513 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.513 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.513 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.513 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.513 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.513 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.513 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.513 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.514 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.514 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.514 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.514 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.514 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.514 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.514 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.514 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.514 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.514 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.514 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.514 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.514 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.515 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.515 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.515 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.515 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.515 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.515 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.515 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.515 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.515 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.515 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.515 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.515 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.516 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.516 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.516 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.516 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.516 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.516 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.516 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.516 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.516 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.516 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.516 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_snap_create": { 2024-09-11T23:32:11.516 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_snap_create", 2024-09-11T23:32:11.517 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.517 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.517 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.517 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:11.517 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.517 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.517 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.517 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.517 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.517 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.517 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.517 INFO:teuthology.orchestra.run.smithi043.stdout: "mirror_snapshot_schedule": { 2024-09-11T23:32:11.518 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T23:32:11.518 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.518 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.518 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.518 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.518 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.518 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.518 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.518 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.518 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.518 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.518 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.519 INFO:teuthology.orchestra.run.smithi043.stdout: "trash_purge_schedule": { 2024-09-11T23:32:11.519 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "trash_purge_schedule", 2024-09-11T23:32:11.519 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.519 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.519 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.519 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.519 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.519 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.519 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.519 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.519 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.519 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.519 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.520 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "restful", 2024-09-11T23:32:11.520 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.520 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.520 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.520 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_auth": { 2024-09-11T23:32:11.520 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_auth", 2024-09-11T23:32:11.520 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.520 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.521 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.521 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.521 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.521 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.521 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.521 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.521 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.521 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.521 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.521 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-09-11T23:32:11.521 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-09-11T23:32:11.521 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.522 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.522 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.522 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.522 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.522 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.522 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.522 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.522 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.522 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.522 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.522 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.522 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.522 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.523 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.523 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.523 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.523 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.523 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.523 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.523 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.523 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.523 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.523 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.523 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.523 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.523 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.524 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.524 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.524 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.524 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.524 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.524 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.524 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.524 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.524 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.524 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.524 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.524 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.525 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.525 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.525 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.525 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.525 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.525 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.525 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.525 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.525 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.525 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.525 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.525 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.526 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.526 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.526 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.526 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.526 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.526 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.526 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.526 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.526 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.526 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.526 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.526 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.526 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.527 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.527 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.527 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.527 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.527 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.527 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.527 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.527 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.527 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.527 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.527 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.527 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.528 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.528 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-09-11T23:32:11.528 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-09-11T23:32:11.528 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.528 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.528 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.528 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.528 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.528 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.528 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.528 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.528 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.529 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.529 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.529 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-09-11T23:32:11.529 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-09-11T23:32:11.529 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.529 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.529 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.529 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.529 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.529 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.529 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.529 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.530 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.530 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.530 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.530 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.530 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.530 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rgw", 2024-09-11T23:32:11.530 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.530 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.530 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.530 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.530 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.530 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.531 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.531 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.531 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.531 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.531 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.531 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.531 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.531 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.531 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.531 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.531 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.531 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.531 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.532 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.532 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.532 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.532 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.532 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.532 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.532 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.532 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.532 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.532 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.532 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.532 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.533 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.533 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.533 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.533 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.533 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.533 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.533 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.533 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.533 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.533 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.533 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.533 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.533 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.534 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.534 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.534 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.534 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.534 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.534 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.534 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.534 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.534 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.534 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.534 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.534 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.535 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.535 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.535 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.535 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.535 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.535 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.535 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.535 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.535 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.535 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.535 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.535 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.535 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.536 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.536 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.536 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rook", 2024-09-11T23:32:11.536 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.536 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.536 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.536 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.536 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.536 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.536 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.536 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.537 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.537 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.537 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.537 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.537 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.537 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.537 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.537 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.537 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.537 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.537 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.537 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.537 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.538 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.538 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.538 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.538 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.538 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.538 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.538 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.538 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.538 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.538 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.538 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.538 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.539 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.539 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.539 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.539 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.539 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.539 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.539 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.539 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.539 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.539 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.539 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.539 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.540 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.540 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.540 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.540 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.540 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.540 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.540 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.540 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.540 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.540 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.540 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.540 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.541 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.541 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.541 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.541 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.541 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.541 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.541 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.541 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.541 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.541 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.541 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.541 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.542 INFO:teuthology.orchestra.run.smithi043.stdout: "storage_class": { 2024-09-11T23:32:11.542 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "storage_class", 2024-09-11T23:32:11.542 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.542 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.542 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.542 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "local", 2024-09-11T23:32:11.542 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.542 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.542 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.542 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T23:32:11.542 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.542 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.542 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.542 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.543 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "selftest", 2024-09-11T23:32:11.543 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.543 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.543 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.543 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.543 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.543 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.543 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.543 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.543 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.544 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.544 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.544 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.544 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.544 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.544 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.544 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.544 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.544 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.544 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.544 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.544 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.544 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.545 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.545 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.545 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.545 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.545 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.545 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.545 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.545 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.545 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.545 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.545 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.545 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.545 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.546 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.546 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.546 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.546 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.546 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.546 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.546 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.546 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.546 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.546 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.546 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.547 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.547 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.547 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.547 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.547 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.547 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.547 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.547 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.547 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.547 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.547 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.547 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.547 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.548 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.548 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.548 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.548 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.548 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.548 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.548 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.548 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.548 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.548 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.548 INFO:teuthology.orchestra.run.smithi043.stdout: "roption1": { 2024-09-11T23:32:11.548 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption1", 2024-09-11T23:32:11.549 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.549 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.549 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.549 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.549 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.549 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.549 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.549 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.549 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.549 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.549 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.549 INFO:teuthology.orchestra.run.smithi043.stdout: "roption2": { 2024-09-11T23:32:11.550 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption2", 2024-09-11T23:32:11.550 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.550 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.550 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.550 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "xyz", 2024-09-11T23:32:11.550 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.550 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.550 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.550 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.550 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.550 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.550 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.551 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption1": { 2024-09-11T23:32:11.551 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption1", 2024-09-11T23:32:11.551 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.551 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.551 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.551 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.551 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.551 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.551 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.551 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.551 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.551 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.551 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.552 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption2": { 2024-09-11T23:32:11.552 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption2", 2024-09-11T23:32:11.552 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.552 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.552 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.552 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.552 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.552 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.552 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.552 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.552 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.552 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.552 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.553 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption3": { 2024-09-11T23:32:11.553 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption3", 2024-09-11T23:32:11.553 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-11T23:32:11.553 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.553 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.553 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.553 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.553 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.553 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.553 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.553 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.553 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.553 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.554 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption4": { 2024-09-11T23:32:11.554 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption4", 2024-09-11T23:32:11.554 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.554 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.554 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.554 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.554 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.554 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.554 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.554 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.554 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.554 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.555 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.555 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption5": { 2024-09-11T23:32:11.555 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption5", 2024-09-11T23:32:11.555 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.555 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.555 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.555 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.555 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.555 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.555 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.555 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.555 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.555 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.556 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.556 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption6": { 2024-09-11T23:32:11.556 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption6", 2024-09-11T23:32:11.556 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.556 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.556 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.556 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.556 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.556 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.556 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.556 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.556 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.557 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.557 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.557 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption7": { 2024-09-11T23:32:11.557 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption7", 2024-09-11T23:32:11.557 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.557 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.557 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.557 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.557 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-11T23:32:11.557 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "42", 2024-09-11T23:32:11.557 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.557 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.558 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.558 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.558 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.558 INFO:teuthology.orchestra.run.smithi043.stdout: "testkey": { 2024-09-11T23:32:11.558 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testkey", 2024-09-11T23:32:11.558 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.558 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.558 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.558 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.558 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.558 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.558 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.559 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.559 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.559 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.559 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.559 INFO:teuthology.orchestra.run.smithi043.stdout: "testlkey": { 2024-09-11T23:32:11.559 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testlkey", 2024-09-11T23:32:11.559 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.559 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.559 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.559 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.559 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.559 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.560 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.560 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.560 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.560 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.560 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.560 INFO:teuthology.orchestra.run.smithi043.stdout: "testnewline": { 2024-09-11T23:32:11.560 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testnewline", 2024-09-11T23:32:11.560 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.560 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.560 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.560 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.560 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.560 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.561 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.561 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.561 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.561 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.561 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.561 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.561 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.561 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-09-11T23:32:11.561 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.561 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.561 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.562 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_m_granularity": { 2024-09-11T23:32:11.562 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_m_granularity", 2024-09-11T23:32:11.562 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.562 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.562 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.562 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.562 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.562 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.562 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.562 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T23:32:11.562 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.562 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.562 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.563 INFO:teuthology.orchestra.run.smithi043.stdout: "dump_on_update": { 2024-09-11T23:32:11.563 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dump_on_update", 2024-09-11T23:32:11.563 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.563 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.563 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.563 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.563 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.563 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.563 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.563 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "dump database to debug log on update", 2024-09-11T23:32:11.563 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.563 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.563 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.564 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.564 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.564 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.564 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.564 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.564 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.564 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.564 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.564 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.564 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.564 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.564 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.564 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.565 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.565 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.565 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.565 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.565 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.565 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.565 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.565 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.565 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.565 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.565 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.565 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.566 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.566 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.566 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.566 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.566 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.566 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.566 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.566 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.566 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.566 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.566 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.566 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.566 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.567 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.567 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.567 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.567 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.567 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.567 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.567 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.567 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.567 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.567 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.567 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.567 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.568 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.568 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.568 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.568 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.568 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.568 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.568 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.568 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.568 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.568 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.568 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.568 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.569 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.569 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.569 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.569 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.569 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stats", 2024-09-11T23:32:11.569 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.569 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.569 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.569 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.570 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.570 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.570 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.570 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.570 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.570 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.570 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.570 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.570 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.570 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.570 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.570 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.570 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.570 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.571 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.571 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.571 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.571 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.571 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.571 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.571 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.571 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.571 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.571 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.571 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.571 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.571 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.572 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.572 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.572 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.572 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.572 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.572 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.572 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.572 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.572 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.572 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.572 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.572 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.573 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.573 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.573 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.573 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.573 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.573 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.573 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.573 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.573 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.573 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.573 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.573 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.573 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.574 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.574 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.574 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.574 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.574 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.574 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.574 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.574 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.574 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.574 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.574 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.574 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.574 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.575 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.575 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.575 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "status", 2024-09-11T23:32:11.575 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.575 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.575 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.575 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.575 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.575 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.575 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.575 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.575 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.576 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.576 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.576 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.576 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.576 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.576 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.576 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.576 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.576 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.576 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.576 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.576 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.576 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.577 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.577 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.577 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.577 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.577 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.577 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.577 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.577 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.577 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.577 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.577 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.577 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.578 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.578 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.578 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.578 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.578 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.578 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.578 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.578 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.578 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.578 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.578 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.578 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.579 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.579 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.579 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.579 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.579 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.579 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.579 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.579 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.579 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.579 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.579 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.579 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.579 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.580 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.580 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.580 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.580 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.580 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.580 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.580 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.580 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.580 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.580 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.580 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.580 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.581 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telegraf", 2024-09-11T23:32:11.581 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.581 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.581 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.581 INFO:teuthology.orchestra.run.smithi043.stdout: "address": { 2024-09-11T23:32:11.581 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "address", 2024-09-11T23:32:11.581 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.581 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.581 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.581 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T23:32:11.581 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.581 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.582 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.582 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.582 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.582 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.582 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.582 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-11T23:32:11.582 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-11T23:32:11.582 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.582 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.582 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.582 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15", 2024-09-11T23:32:11.583 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.583 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.583 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.583 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.583 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.583 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.583 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.583 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.583 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.583 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.583 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.583 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.584 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.584 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.584 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.584 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.584 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.584 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.584 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.584 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.584 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.584 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.584 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.584 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.584 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.585 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.585 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.585 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.585 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.585 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.585 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.585 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.585 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.585 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.585 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.585 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.585 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.586 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.586 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.586 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.586 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.586 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.586 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.586 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.586 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.586 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.586 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.586 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.586 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.587 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.587 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.587 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.587 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.587 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.587 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.587 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.587 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.587 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.587 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.587 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.587 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.588 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.588 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.588 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.588 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.588 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.588 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.588 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.588 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.588 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.588 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.588 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.588 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.589 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telemetry", 2024-09-11T23:32:11.589 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.589 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.589 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.589 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_basic": { 2024-09-11T23:32:11.589 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_basic", 2024-09-11T23:32:11.589 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.589 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.589 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.590 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.590 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.590 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.590 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.590 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T23:32:11.590 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.590 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.590 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.590 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_crash": { 2024-09-11T23:32:11.590 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_crash", 2024-09-11T23:32:11.590 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.590 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.590 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.591 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.591 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.591 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.591 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.591 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T23:32:11.591 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.591 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.591 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.591 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_device": { 2024-09-11T23:32:11.591 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_device", 2024-09-11T23:32:11.591 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.591 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.592 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.592 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.592 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.592 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.592 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.592 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T23:32:11.592 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.592 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.592 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.592 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_ident": { 2024-09-11T23:32:11.592 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_ident", 2024-09-11T23:32:11.593 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.593 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.593 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.593 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.593 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.593 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.593 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.593 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T23:32:11.593 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.593 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.593 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.593 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_perf": { 2024-09-11T23:32:11.593 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_perf", 2024-09-11T23:32:11.594 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.594 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.594 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.594 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.594 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.594 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.594 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.594 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T23:32:11.594 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.594 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.594 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.594 INFO:teuthology.orchestra.run.smithi043.stdout: "contact": { 2024-09-11T23:32:11.595 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "contact", 2024-09-11T23:32:11.595 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.595 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.595 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.595 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.595 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.595 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.595 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.595 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.595 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.595 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.595 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.595 INFO:teuthology.orchestra.run.smithi043.stdout: "description": { 2024-09-11T23:32:11.596 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "description", 2024-09-11T23:32:11.596 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.596 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.596 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.596 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.596 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.596 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.596 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.596 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.596 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.596 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.596 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.597 INFO:teuthology.orchestra.run.smithi043.stdout: "device_url": { 2024-09-11T23:32:11.597 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_url", 2024-09-11T23:32:11.597 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.597 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.597 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.597 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T23:32:11.597 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.597 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.597 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.597 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.597 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.597 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.597 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.598 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-09-11T23:32:11.598 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-09-11T23:32:11.598 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.598 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.598 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.598 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.598 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.598 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.598 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.598 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.598 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.598 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.599 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.599 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-11T23:32:11.599 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-11T23:32:11.599 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.599 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.599 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.599 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "24", 2024-09-11T23:32:11.599 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "8", 2024-09-11T23:32:11.599 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.599 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.599 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.599 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.599 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.600 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.600 INFO:teuthology.orchestra.run.smithi043.stdout: "last_opt_revision": { 2024-09-11T23:32:11.600 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "last_opt_revision", 2024-09-11T23:32:11.600 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.600 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.600 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.600 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1", 2024-09-11T23:32:11.600 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.600 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.600 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.600 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.601 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.601 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.601 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.601 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard": { 2024-09-11T23:32:11.601 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard", 2024-09-11T23:32:11.601 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.601 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.601 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.601 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.601 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.601 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.601 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.602 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.602 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.602 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.602 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.602 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard_description": { 2024-09-11T23:32:11.602 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard_description", 2024-09-11T23:32:11.602 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.602 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.602 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.602 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.602 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.602 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.603 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.603 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.603 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.603 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.603 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.603 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.603 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.603 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.603 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.603 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.603 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.603 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.604 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.604 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.604 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.604 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.604 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.604 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.604 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.604 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.604 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.604 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.604 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.604 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.604 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.605 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.605 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.605 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.605 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.605 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.605 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.605 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.605 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.605 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.605 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.605 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.605 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.606 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.606 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.606 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.606 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.606 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.606 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.606 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.606 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.606 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.606 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.606 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.606 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.607 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.607 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.607 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.607 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.607 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.607 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.607 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.607 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.607 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.607 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.607 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.607 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.607 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.608 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.608 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.608 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.608 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.608 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.608 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.608 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.608 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.608 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.608 INFO:teuthology.orchestra.run.smithi043.stdout: "organization": { 2024-09-11T23:32:11.608 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "organization", 2024-09-11T23:32:11.608 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.609 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.609 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.609 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.609 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.609 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.609 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.609 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.609 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.609 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.609 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.609 INFO:teuthology.orchestra.run.smithi043.stdout: "proxy": { 2024-09-11T23:32:11.609 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "proxy", 2024-09-11T23:32:11.610 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.610 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.610 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.610 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.610 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.610 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.610 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.610 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.610 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.610 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.610 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.610 INFO:teuthology.orchestra.run.smithi043.stdout: "url": { 2024-09-11T23:32:11.611 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url", 2024-09-11T23:32:11.611 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.611 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.611 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.611 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T23:32:11.611 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.611 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.611 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.611 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.611 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.611 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.611 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.612 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.612 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.612 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "test_orchestrator", 2024-09-11T23:32:11.612 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.612 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.612 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.612 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.612 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.612 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.612 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.612 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.613 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.613 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.613 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.613 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.613 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.613 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.613 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.613 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.613 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.613 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.613 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.613 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.613 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.614 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.614 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.614 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.614 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.614 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.614 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.614 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.614 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.614 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.614 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.614 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.614 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.615 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.615 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.615 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.615 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.615 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.615 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.615 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.615 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.615 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.615 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.615 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.615 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.616 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.616 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.616 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.616 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.616 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.616 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.616 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.616 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.616 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.616 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.616 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.616 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.617 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.617 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.617 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.617 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.617 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.617 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.617 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.617 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.617 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.617 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.617 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.617 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.617 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.618 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.618 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "volumes", 2024-09-11T23:32:11.618 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.618 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.618 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.618 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.618 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.618 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.618 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.618 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.618 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.619 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.619 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.619 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.619 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.619 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.619 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.619 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.619 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.619 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.619 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.619 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.619 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.619 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.620 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.620 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.620 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.620 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.620 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.620 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.620 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.620 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.620 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.620 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.620 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.620 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.621 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.621 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.621 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.621 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.621 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.621 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.621 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.621 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.621 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.621 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.621 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.621 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.622 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.622 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.622 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.622 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.622 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.622 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.622 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.622 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.622 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.622 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.622 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.622 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.623 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.623 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.623 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.623 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.623 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.623 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.623 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.623 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.623 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.623 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.623 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.623 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_clones": { 2024-09-11T23:32:11.624 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_clones", 2024-09-11T23:32:11.624 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.624 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.624 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.624 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4", 2024-09-11T23:32:11.624 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.624 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.624 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.624 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T23:32:11.624 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.624 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.624 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.625 INFO:teuthology.orchestra.run.smithi043.stdout: "periodic_async_work": { 2024-09-11T23:32:11.625 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "periodic_async_work", 2024-09-11T23:32:11.625 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.625 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.625 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.625 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.625 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.625 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.625 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.625 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Periodically check for async work", 2024-09-11T23:32:11.625 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.625 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.625 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.626 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_delay": { 2024-09-11T23:32:11.626 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_delay", 2024-09-11T23:32:11.626 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.626 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.626 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.626 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-11T23:32:11.626 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.626 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.626 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.626 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T23:32:11.626 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.626 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.627 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.627 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_no_wait": { 2024-09-11T23:32:11.627 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T23:32:11.627 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.627 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.627 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.627 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.627 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.627 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.627 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.627 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T23:32:11.627 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.627 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.628 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.628 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix", 2024-09-11T23:32:11.628 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.628 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.628 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.628 INFO:teuthology.orchestra.run.smithi043.stdout: "discovery_interval": { 2024-09-11T23:32:11.628 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "discovery_interval", 2024-09-11T23:32:11.628 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-11T23:32:11.628 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.629 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.629 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "100", 2024-09-11T23:32:11.629 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.629 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.629 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.629 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.629 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.629 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.632 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.632 INFO:teuthology.orchestra.run.smithi043.stdout: "identifier": { 2024-09-11T23:32:11.632 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "identifier", 2024-09-11T23:32:11.632 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.632 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.632 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.632 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.632 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.632 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.632 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.633 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.633 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.633 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.633 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.633 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-11T23:32:11.633 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-11T23:32:11.633 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.633 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.633 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.633 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-11T23:32:11.633 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.633 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.634 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.634 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.634 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.634 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.634 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.634 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.634 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.634 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.634 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.634 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.634 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.634 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.635 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.635 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.635 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.635 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.635 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.635 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.635 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.635 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.635 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.635 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.635 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.635 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.635 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.636 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.636 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.636 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.636 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.636 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.636 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.636 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.636 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.636 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.636 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.636 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.636 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.636 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.637 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.637 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.637 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.637 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.637 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.637 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.637 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.637 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.637 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.637 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.637 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.637 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.638 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.638 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.638 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.638 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.638 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.638 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.638 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.638 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.638 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.638 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.638 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.638 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.638 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.639 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.639 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.639 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.639 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.639 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.639 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.639 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.639 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.639 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_host": { 2024-09-11T23:32:11.639 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_host", 2024-09-11T23:32:11.639 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.639 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.640 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.640 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.640 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.640 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.640 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.640 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.640 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.640 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.640 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.640 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_port": { 2024-09-11T23:32:11.640 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_port", 2024-09-11T23:32:11.640 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.641 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.641 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.641 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10051", 2024-09-11T23:32:11.641 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.641 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.641 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.641 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.641 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.641 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.641 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.641 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_sender": { 2024-09-11T23:32:11.641 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_sender", 2024-09-11T23:32:11.642 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.642 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.642 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.642 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T23:32:11.642 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.642 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.642 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.642 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.642 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.642 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.642 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.642 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.642 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.642 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.643 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-11T23:32:11.643 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.643 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.643 INFO:teuthology.orchestra.run.smithi043.stdout: "modules": [ 2024-09-11T23:32:11.643 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm", 2024-09-11T23:32:11.643 INFO:teuthology.orchestra.run.smithi043.stdout: "dashboard", 2024-09-11T23:32:11.643 INFO:teuthology.orchestra.run.smithi043.stdout: "iostat", 2024-09-11T23:32:11.643 INFO:teuthology.orchestra.run.smithi043.stdout: "nfs", 2024-09-11T23:32:11.643 INFO:teuthology.orchestra.run.smithi043.stdout: "restful", 2024-09-11T23:32:11.643 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_schedule" 2024-09-11T23:32:11.643 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.643 INFO:teuthology.orchestra.run.smithi043.stdout: "available_modules": [ 2024-09-11T23:32:11.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.644 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "alerts", 2024-09-11T23:32:11.644 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.644 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.644 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.644 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-11T23:32:11.644 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-11T23:32:11.644 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.644 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.644 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.644 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-11T23:32:11.644 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.645 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.645 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.645 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T23:32:11.645 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.645 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.645 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.645 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.645 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.645 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.645 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.645 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.645 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.646 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.646 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.646 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.646 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.646 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.646 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.646 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.646 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.646 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.646 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.646 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.646 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.646 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.646 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.647 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.647 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.647 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.647 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.647 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.647 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.647 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.647 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.647 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.647 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.647 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.647 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.647 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.648 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.648 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.648 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.648 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.648 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.648 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.648 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.648 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.648 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.648 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.648 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.649 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.649 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.649 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.649 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.649 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.649 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.649 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.649 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.649 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.649 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.649 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.650 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.650 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.650 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.650 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.650 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.650 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.650 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.650 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.650 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.650 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.650 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.650 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_destination": { 2024-09-11T23:32:11.651 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_destination", 2024-09-11T23:32:11.651 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.651 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.651 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.651 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.651 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.651 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.651 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.651 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email address to send alerts to", 2024-09-11T23:32:11.651 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.651 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.651 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.651 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_from_name": { 2024-09-11T23:32:11.652 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_from_name", 2024-09-11T23:32:11.652 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.652 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.652 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.652 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "Ceph", 2024-09-11T23:32:11.652 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.652 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.652 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.652 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email From: name", 2024-09-11T23:32:11.652 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.652 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.652 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.653 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_host": { 2024-09-11T23:32:11.653 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_host", 2024-09-11T23:32:11.653 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.653 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.653 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.653 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.653 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.653 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.653 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.653 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP server", 2024-09-11T23:32:11.653 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.653 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.653 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.654 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_password": { 2024-09-11T23:32:11.654 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_password", 2024-09-11T23:32:11.654 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.654 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.654 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.654 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.654 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.654 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.654 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.654 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Password to authenticate with", 2024-09-11T23:32:11.654 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.654 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.655 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.655 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_port": { 2024-09-11T23:32:11.655 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_port", 2024-09-11T23:32:11.655 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.655 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.655 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.655 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "465", 2024-09-11T23:32:11.655 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.655 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.655 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.655 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP port", 2024-09-11T23:32:11.655 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.655 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.656 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.656 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_sender": { 2024-09-11T23:32:11.656 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_sender", 2024-09-11T23:32:11.656 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.656 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.656 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.656 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.656 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.656 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.656 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.656 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP envelope sender", 2024-09-11T23:32:11.657 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.657 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.657 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.657 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_ssl": { 2024-09-11T23:32:11.657 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_ssl", 2024-09-11T23:32:11.657 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.657 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.657 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.657 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.657 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.657 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.657 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.658 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T23:32:11.658 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.658 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.658 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.658 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_user": { 2024-09-11T23:32:11.658 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_user", 2024-09-11T23:32:11.658 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.658 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.658 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.658 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.658 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.658 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.658 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.659 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "User to authenticate as", 2024-09-11T23:32:11.659 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.659 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.659 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.659 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.659 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.659 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "balancer", 2024-09-11T23:32:11.659 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.659 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.659 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.659 INFO:teuthology.orchestra.run.smithi043.stdout: "active": { 2024-09-11T23:32:11.660 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "active", 2024-09-11T23:32:11.660 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.660 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.660 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.660 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.660 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.660 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.660 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.660 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T23:32:11.660 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.660 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.660 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.660 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_time": { 2024-09-11T23:32:11.661 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_time", 2024-09-11T23:32:11.661 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.661 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.661 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.661 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0000", 2024-09-11T23:32:11.661 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.661 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.661 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.661 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T23:32:11.661 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:32:11.661 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.661 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.662 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_weekday": { 2024-09-11T23:32:11.662 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_weekday", 2024-09-11T23:32:11.662 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-11T23:32:11.662 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.662 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.662 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-11T23:32:11.662 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-09-11T23:32:11.662 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-09-11T23:32:11.662 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.662 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T23:32:11.662 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:32:11.662 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.662 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.663 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_max_iterations": { 2024-09-11T23:32:11.663 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_max_iterations", 2024-09-11T23:32:11.663 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-11T23:32:11.663 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.663 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.663 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "25", 2024-09-11T23:32:11.663 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-11T23:32:11.663 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "250", 2024-09-11T23:32:11.663 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.663 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T23:32:11.663 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.664 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.664 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.664 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_metrics": { 2024-09-11T23:32:11.664 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_metrics", 2024-09-11T23:32:11.664 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.664 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.664 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.664 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T23:32:11.664 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.664 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.664 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.664 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T23:32:11.664 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T23:32:11.665 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.665 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.665 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_step": { 2024-09-11T23:32:11.665 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_step", 2024-09-11T23:32:11.665 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-11T23:32:11.665 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.665 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.665 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.5", 2024-09-11T23:32:11.665 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0.001", 2024-09-11T23:32:11.665 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "0.999", 2024-09-11T23:32:11.665 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.665 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "aggressiveness of optimization", 2024-09-11T23:32:11.666 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T23:32:11.666 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.666 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.666 INFO:teuthology.orchestra.run.smithi043.stdout: "end_time": { 2024-09-11T23:32:11.666 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_time", 2024-09-11T23:32:11.666 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.666 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.666 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.666 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2359", 2024-09-11T23:32:11.666 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.666 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.666 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.666 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T23:32:11.667 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:32:11.667 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.667 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.667 INFO:teuthology.orchestra.run.smithi043.stdout: "end_weekday": { 2024-09-11T23:32:11.667 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_weekday", 2024-09-11T23:32:11.667 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-11T23:32:11.667 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.667 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.667 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-11T23:32:11.667 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-09-11T23:32:11.667 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-09-11T23:32:11.667 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.668 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T23:32:11.668 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:32:11.668 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.668 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.668 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.668 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.668 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.668 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.668 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.668 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.668 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.669 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.669 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.669 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.669 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.669 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.669 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.669 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.669 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.669 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.669 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.669 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.669 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.669 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.670 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.670 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.670 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.670 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.670 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.670 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.670 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.670 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.670 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.670 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.670 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.670 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.670 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.671 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.671 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.671 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.671 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.671 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.671 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.671 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.671 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.671 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.671 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.671 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.671 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.671 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.671 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.672 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.672 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.672 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.672 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.672 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.672 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.672 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.672 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.672 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.672 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.672 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.672 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.672 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.673 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.673 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.673 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.673 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.673 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.673 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.673 INFO:teuthology.orchestra.run.smithi043.stdout: "min_score": { 2024-09-11T23:32:11.673 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_score", 2024-09-11T23:32:11.673 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-11T23:32:11.673 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.673 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.673 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-11T23:32:11.674 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.674 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.674 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.674 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T23:32:11.674 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.674 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.674 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.674 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-09-11T23:32:11.674 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-09-11T23:32:11.674 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.674 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.674 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.674 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "upmap", 2024-09-11T23:32:11.675 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.675 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.675 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.675 INFO:teuthology.orchestra.run.smithi043.stdout: "crush-compat", 2024-09-11T23:32:11.675 INFO:teuthology.orchestra.run.smithi043.stdout: "none", 2024-09-11T23:32:11.675 INFO:teuthology.orchestra.run.smithi043.stdout: "read", 2024-09-11T23:32:11.675 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap", 2024-09-11T23:32:11.675 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap-read" 2024-09-11T23:32:11.675 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.675 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Balancer mode", 2024-09-11T23:32:11.675 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.675 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.675 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.676 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ids": { 2024-09-11T23:32:11.676 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_ids", 2024-09-11T23:32:11.676 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.676 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.676 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.676 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.676 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.676 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.676 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.676 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T23:32:11.676 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.676 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.677 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.677 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-11T23:32:11.677 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-11T23:32:11.677 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.677 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.677 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.677 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-11T23:32:11.677 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.677 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.677 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.677 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T23:32:11.677 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.677 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.678 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.678 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_deviation": { 2024-09-11T23:32:11.678 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_deviation", 2024-09-11T23:32:11.678 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.678 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.678 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.678 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-11T23:32:11.678 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-11T23:32:11.678 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.678 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.678 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T23:32:11.679 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T23:32:11.679 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.679 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.679 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_optimizations": { 2024-09-11T23:32:11.679 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_optimizations", 2024-09-11T23:32:11.679 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-11T23:32:11.679 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.679 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.679 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:11.679 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.679 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.679 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.680 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T23:32:11.680 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.680 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.680 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.680 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.680 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.680 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm", 2024-09-11T23:32:11.680 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.680 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.680 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.680 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_down_multiplier": { 2024-09-11T23:32:11.680 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_down_multiplier", 2024-09-11T23:32:11.681 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-11T23:32:11.681 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.681 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.681 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-09-11T23:32:11.681 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.681 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.681 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.681 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T23:32:11.681 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.681 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.681 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.681 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_refresh_rate": { 2024-09-11T23:32:11.682 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_refresh_rate", 2024-09-11T23:32:11.682 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.682 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.682 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.682 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "20", 2024-09-11T23:32:11.682 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.682 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.682 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.682 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T23:32:11.682 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.682 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.682 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.682 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_starting_port": { 2024-09-11T23:32:11.683 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_starting_port", 2024-09-11T23:32:11.683 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.683 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.683 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.683 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4721", 2024-09-11T23:32:11.683 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.683 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.683 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.683 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T23:32:11.683 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.683 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.683 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.683 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_ptrace": { 2024-09-11T23:32:11.684 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_ptrace", 2024-09-11T23:32:11.684 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.684 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.684 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.684 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.684 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.684 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.684 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.684 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T23:32:11.684 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T23:32:11.684 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.684 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.685 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_interval": { 2024-09-11T23:32:11.685 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_interval", 2024-09-11T23:32:11.685 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.685 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.685 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.685 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-11T23:32:11.685 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.685 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.685 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.685 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T23:32:11.685 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.685 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.685 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.686 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_memory_target_ratio": { 2024-09-11T23:32:11.686 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T23:32:11.686 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-11T23:32:11.686 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.686 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.686 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.7", 2024-09-11T23:32:11.686 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.686 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.686 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.686 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T23:32:11.686 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.686 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.687 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.687 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm_log_destination": { 2024-09-11T23:32:11.687 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm_log_destination", 2024-09-11T23:32:11.687 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.687 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.687 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.687 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.687 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.687 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.687 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.687 INFO:teuthology.orchestra.run.smithi043.stdout: "file", 2024-09-11T23:32:11.687 INFO:teuthology.orchestra.run.smithi043.stdout: "file,syslog", 2024-09-11T23:32:11.687 INFO:teuthology.orchestra.run.smithi043.stdout: "syslog" 2024-09-11T23:32:11.688 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.688 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T23:32:11.688 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.688 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.688 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.688 INFO:teuthology.orchestra.run.smithi043.stdout: "cgroups_split": { 2024-09-11T23:32:11.688 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cgroups_split", 2024-09-11T23:32:11.688 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.688 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.688 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.688 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.688 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.689 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.689 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.689 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T23:32:11.689 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.689 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.689 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.689 INFO:teuthology.orchestra.run.smithi043.stdout: "config_checks_enabled": { 2024-09-11T23:32:11.689 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_checks_enabled", 2024-09-11T23:32:11.689 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.689 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.689 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.689 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.690 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.690 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.690 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.690 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T23:32:11.690 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.690 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.690 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.690 INFO:teuthology.orchestra.run.smithi043.stdout: "config_dashboard": { 2024-09-11T23:32:11.690 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_dashboard", 2024-09-11T23:32:11.690 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.690 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.690 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.691 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.691 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.691 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.691 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.691 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T23:32:11.691 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.691 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.691 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.691 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_alertmanager": { 2024-09-11T23:32:11.691 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_alertmanager", 2024-09-11T23:32:11.691 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.691 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.691 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.692 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-11T23:32:11.692 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.692 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.692 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.692 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-11T23:32:11.692 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.692 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.692 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.692 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_base": { 2024-09-11T23:32:11.692 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_base", 2024-09-11T23:32:11.692 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.692 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.693 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.693 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T23:32:11.693 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.693 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.693 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.693 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Container image name, without the tag", 2024-09-11T23:32:11.693 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.693 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.693 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.693 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_elasticsearch": { 2024-09-11T23:32:11.693 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_elasticsearch", 2024-09-11T23:32:11.694 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.694 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.694 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.694 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T23:32:11.694 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.694 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.694 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.694 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "elasticsearch container image", 2024-09-11T23:32:11.694 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.694 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.694 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.694 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_grafana": { 2024-09-11T23:32:11.694 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_grafana", 2024-09-11T23:32:11.695 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.695 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.695 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.695 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-11T23:32:11.695 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.695 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.695 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.695 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-11T23:32:11.695 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.695 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.695 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.695 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_haproxy": { 2024-09-11T23:32:11.695 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_haproxy", 2024-09-11T23:32:11.696 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.696 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.696 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.696 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T23:32:11.696 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.696 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.696 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.696 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "HAproxy container image", 2024-09-11T23:32:11.696 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.696 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.696 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.696 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_agent": { 2024-09-11T23:32:11.697 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_agent", 2024-09-11T23:32:11.697 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.697 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.697 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.697 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T23:32:11.697 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.697 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.697 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.697 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger agent container image", 2024-09-11T23:32:11.697 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.697 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.697 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.697 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_collector": { 2024-09-11T23:32:11.698 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_collector", 2024-09-11T23:32:11.698 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.698 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.698 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.698 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T23:32:11.698 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.698 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.698 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.698 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger collector container image", 2024-09-11T23:32:11.698 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.698 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.698 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.699 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_query": { 2024-09-11T23:32:11.699 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_query", 2024-09-11T23:32:11.699 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.699 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.699 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.699 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T23:32:11.699 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.699 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.699 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.699 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger query container image", 2024-09-11T23:32:11.699 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.699 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.699 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.700 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_keepalived": { 2024-09-11T23:32:11.700 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_keepalived", 2024-09-11T23:32:11.700 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.700 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.700 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.700 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T23:32:11.700 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.700 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.700 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.700 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Keepalived container image", 2024-09-11T23:32:11.700 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.700 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.701 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.701 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_loki": { 2024-09-11T23:32:11.701 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_loki", 2024-09-11T23:32:11.701 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.701 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.701 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.701 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T23:32:11.701 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.701 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.701 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.701 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Loki container image", 2024-09-11T23:32:11.701 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.701 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.702 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.702 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_node_exporter": { 2024-09-11T23:32:11.702 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_node_exporter", 2024-09-11T23:32:11.702 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.702 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.702 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.702 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-11T23:32:11.702 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.702 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.702 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.702 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-11T23:32:11.702 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.703 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.703 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.703 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_nvmeof": { 2024-09-11T23:32:11.703 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_nvmeof", 2024-09-11T23:32:11.703 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.703 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.703 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.703 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T23:32:11.703 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.703 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.703 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.703 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Nvme-of container image", 2024-09-11T23:32:11.704 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.704 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.704 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.704 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_prometheus": { 2024-09-11T23:32:11.704 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_prometheus", 2024-09-11T23:32:11.704 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.704 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.704 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.704 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-11T23:32:11.704 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.704 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.704 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.705 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-09-11T23:32:11.705 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.705 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.705 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.705 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_promtail": { 2024-09-11T23:32:11.705 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_promtail", 2024-09-11T23:32:11.705 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.705 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.705 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.705 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T23:32:11.705 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.705 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.706 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.706 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Promtail container image", 2024-09-11T23:32:11.706 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.706 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.706 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.706 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_samba": { 2024-09-11T23:32:11.706 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_samba", 2024-09-11T23:32:11.706 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.706 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.706 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.706 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T23:32:11.706 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.706 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.707 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.707 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Samba/SMB container image", 2024-09-11T23:32:11.707 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.707 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.707 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.707 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_snmp_gateway": { 2024-09-11T23:32:11.707 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_snmp_gateway", 2024-09-11T23:32:11.707 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.707 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.707 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.707 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T23:32:11.707 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.708 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.708 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.708 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SNMP Gateway container image", 2024-09-11T23:32:11.708 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.708 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.708 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.708 INFO:teuthology.orchestra.run.smithi043.stdout: "container_init": { 2024-09-11T23:32:11.708 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_init", 2024-09-11T23:32:11.708 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.708 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.708 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.708 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.709 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.709 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.709 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.709 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T23:32:11.709 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.709 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.709 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.709 INFO:teuthology.orchestra.run.smithi043.stdout: "daemon_cache_timeout": { 2024-09-11T23:32:11.709 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "daemon_cache_timeout", 2024-09-11T23:32:11.709 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.709 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.710 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.710 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-11T23:32:11.710 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.710 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.710 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.710 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T23:32:11.710 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.710 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.710 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.710 INFO:teuthology.orchestra.run.smithi043.stdout: "default_cephadm_command_timeout": { 2024-09-11T23:32:11.710 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T23:32:11.710 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.710 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.711 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.711 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "900", 2024-09-11T23:32:11.711 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.711 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.711 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.711 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T23:32:11.711 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.711 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.711 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.711 INFO:teuthology.orchestra.run.smithi043.stdout: "default_registry": { 2024-09-11T23:32:11.711 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_registry", 2024-09-11T23:32:11.712 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.712 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.712 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.712 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io", 2024-09-11T23:32:11.712 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.712 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.712 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.712 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T23:32:11.712 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.712 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.712 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.712 INFO:teuthology.orchestra.run.smithi043.stdout: "device_cache_timeout": { 2024-09-11T23:32:11.713 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_cache_timeout", 2024-09-11T23:32:11.713 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.713 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.713 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.713 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1800", 2024-09-11T23:32:11.713 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.713 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.713 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.713 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache device inventory", 2024-09-11T23:32:11.713 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.713 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.713 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.713 INFO:teuthology.orchestra.run.smithi043.stdout: "device_enhanced_scan": { 2024-09-11T23:32:11.714 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_enhanced_scan", 2024-09-11T23:32:11.714 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.714 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.714 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.714 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.714 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.714 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.714 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.714 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T23:32:11.714 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.714 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.714 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.715 INFO:teuthology.orchestra.run.smithi043.stdout: "facts_cache_timeout": { 2024-09-11T23:32:11.715 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "facts_cache_timeout", 2024-09-11T23:32:11.715 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.715 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.715 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.715 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-11T23:32:11.715 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.715 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.715 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.715 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache host facts data", 2024-09-11T23:32:11.715 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.715 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.715 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.716 INFO:teuthology.orchestra.run.smithi043.stdout: "grafana_dashboards_path": { 2024-09-11T23:32:11.716 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "grafana_dashboards_path", 2024-09-11T23:32:11.716 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.716 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.716 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.716 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T23:32:11.716 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.716 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.716 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.716 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T23:32:11.716 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.716 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.717 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.717 INFO:teuthology.orchestra.run.smithi043.stdout: "host_check_interval": { 2024-09-11T23:32:11.717 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "host_check_interval", 2024-09-11T23:32:11.717 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.717 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.717 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.717 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-11T23:32:11.717 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.717 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.717 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.717 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to perform a host check", 2024-09-11T23:32:11.717 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.717 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.718 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.718 INFO:teuthology.orchestra.run.smithi043.stdout: "hw_monitoring": { 2024-09-11T23:32:11.718 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hw_monitoring", 2024-09-11T23:32:11.718 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.718 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.718 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.718 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.718 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.718 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.718 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.718 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T23:32:11.719 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.719 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.719 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.719 INFO:teuthology.orchestra.run.smithi043.stdout: "inventory_list_all": { 2024-09-11T23:32:11.719 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "inventory_list_all", 2024-09-11T23:32:11.719 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.719 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.719 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.719 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.719 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.719 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.720 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.720 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T23:32:11.720 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.720 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.720 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.720 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.720 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.720 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.720 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.720 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.720 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.720 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.721 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.721 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.721 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.721 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.721 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.721 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.721 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.721 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.721 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.721 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.721 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.721 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.721 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.722 INFO:teuthology.orchestra.run.smithi043.stdout: "log_refresh_metadata": { 2024-09-11T23:32:11.722 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_refresh_metadata", 2024-09-11T23:32:11.722 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.722 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.722 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.722 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.722 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.722 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.722 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.722 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T23:32:11.722 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.723 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.723 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.723 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.723 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.723 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.723 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.723 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.723 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.723 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.723 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.723 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.724 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T23:32:11.724 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.724 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.724 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.724 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.724 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.724 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.724 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.724 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.724 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.724 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.725 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.725 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.725 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.725 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.725 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.725 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.725 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.725 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.725 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.725 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.725 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.726 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.726 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.726 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.726 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.726 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.726 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.726 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.726 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.726 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.726 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.726 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.726 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.727 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.727 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.727 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.727 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T23:32:11.727 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T23:32:11.727 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.727 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.727 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.727 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.727 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.727 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.727 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.728 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T23:32:11.728 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.728 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.728 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.728 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T23:32:11.728 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T23:32:11.728 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.728 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.728 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.728 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "*", 2024-09-11T23:32:11.728 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.728 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.729 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.729 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T23:32:11.729 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.729 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.729 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.729 INFO:teuthology.orchestra.run.smithi043.stdout: "max_count_per_host": { 2024-09-11T23:32:11.729 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_count_per_host", 2024-09-11T23:32:11.729 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.729 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.729 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.729 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:11.729 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of daemons per service per host", 2024-09-11T23:32:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: "max_osd_draining_count": { 2024-09-11T23:32:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_osd_draining_count", 2024-09-11T23:32:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.731 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.731 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.731 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T23:32:11.731 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.731 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.731 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.731 INFO:teuthology.orchestra.run.smithi043.stdout: "migration_current": { 2024-09-11T23:32:11.731 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "migration_current", 2024-09-11T23:32:11.731 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.731 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.731 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.731 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.731 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.732 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.732 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.732 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "internal - do not modify", 2024-09-11T23:32:11.732 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.732 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.732 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.732 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-09-11T23:32:11.732 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-09-11T23:32:11.732 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.732 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.732 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.732 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "root", 2024-09-11T23:32:11.733 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.733 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.733 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.733 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm-package", 2024-09-11T23:32:11.733 INFO:teuthology.orchestra.run.smithi043.stdout: "root" 2024-09-11T23:32:11.733 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.733 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T23:32:11.733 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.733 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.733 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.733 INFO:teuthology.orchestra.run.smithi043.stdout: "oob_default_addr": { 2024-09-11T23:32:11.733 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "oob_default_addr", 2024-09-11T23:32:11.733 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.734 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.734 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.734 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "169.254.1.1", 2024-09-11T23:32:11.734 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.734 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.734 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.734 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T23:32:11.734 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.734 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.734 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.734 INFO:teuthology.orchestra.run.smithi043.stdout: "prometheus_alerts_path": { 2024-09-11T23:32:11.734 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus_alerts_path", 2024-09-11T23:32:11.734 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.735 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.735 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.735 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T23:32:11.735 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.735 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.735 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.735 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T23:32:11.735 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.735 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.735 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.735 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_insecure": { 2024-09-11T23:32:11.735 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_insecure", 2024-09-11T23:32:11.736 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.736 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.736 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.736 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.736 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.736 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.736 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.736 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T23:32:11.736 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.736 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.736 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.736 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_password": { 2024-09-11T23:32:11.736 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_password", 2024-09-11T23:32:11.737 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.737 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.737 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.737 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.737 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.737 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.737 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.737 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T23:32:11.737 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.737 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.737 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.737 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_url": { 2024-09-11T23:32:11.737 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_url", 2024-09-11T23:32:11.738 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.738 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.738 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.738 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.738 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.738 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.738 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.738 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T23:32:11.738 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.738 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.738 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.738 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_username": { 2024-09-11T23:32:11.739 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_username", 2024-09-11T23:32:11.739 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.739 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.739 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.739 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.739 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.739 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.739 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.739 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T23:32:11.739 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.739 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.739 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.739 INFO:teuthology.orchestra.run.smithi043.stdout: "secure_monitoring_stack": { 2024-09-11T23:32:11.740 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "secure_monitoring_stack", 2024-09-11T23:32:11.740 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.740 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.740 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.740 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.740 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.740 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.740 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.740 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T23:32:11.740 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.740 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.740 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.741 INFO:teuthology.orchestra.run.smithi043.stdout: "service_discovery_port": { 2024-09-11T23:32:11.741 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "service_discovery_port", 2024-09-11T23:32:11.741 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.741 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.741 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.741 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8765", 2024-09-11T23:32:11.741 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.741 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.741 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.741 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "cephadm service discovery port", 2024-09-11T23:32:11.741 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.741 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.741 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.742 INFO:teuthology.orchestra.run.smithi043.stdout: "ssh_config_file": { 2024-09-11T23:32:11.742 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssh_config_file", 2024-09-11T23:32:11.742 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.742 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.742 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.742 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.742 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.742 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.742 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.742 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T23:32:11.742 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.742 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.743 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.743 INFO:teuthology.orchestra.run.smithi043.stdout: "use_agent": { 2024-09-11T23:32:11.743 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_agent", 2024-09-11T23:32:11.743 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.743 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.743 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.743 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.743 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.743 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.743 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.743 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T23:32:11.743 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.744 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.744 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.744 INFO:teuthology.orchestra.run.smithi043.stdout: "use_repo_digest": { 2024-09-11T23:32:11.744 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_repo_digest", 2024-09-11T23:32:11.744 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.744 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.744 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.744 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.744 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.744 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.744 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.744 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T23:32:11.744 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.745 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.745 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.745 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_failed_host_check": { 2024-09-11T23:32:11.745 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_failed_host_check", 2024-09-11T23:32:11.745 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.745 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.745 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.745 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.745 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.745 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.745 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.745 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T23:32:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_daemons": { 2024-09-11T23:32:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_daemons", 2024-09-11T23:32:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T23:32:11.747 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.747 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.747 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.747 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_hosts": { 2024-09-11T23:32:11.747 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_hosts", 2024-09-11T23:32:11.747 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.747 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.747 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.747 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.747 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.747 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.747 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T23:32:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crash", 2024-09-11T23:32:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.749 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.749 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.749 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.749 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.749 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.749 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.749 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.749 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.749 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.749 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.749 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.749 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.749 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.750 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.750 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.750 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.750 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.750 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.750 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.750 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.750 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.750 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.750 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.750 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.750 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.750 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.751 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.751 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.751 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.751 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.751 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.751 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.751 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.751 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.751 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.751 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.751 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.751 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.752 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.752 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.752 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.752 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.752 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.752 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.752 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.752 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.752 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.752 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.752 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.752 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.752 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.753 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.753 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.753 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.753 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.753 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.753 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.753 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.753 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.753 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.753 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.753 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.753 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.753 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.754 INFO:teuthology.orchestra.run.smithi043.stdout: "retain_interval": { 2024-09-11T23:32:11.754 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retain_interval", 2024-09-11T23:32:11.754 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.754 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.754 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.754 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "31536000", 2024-09-11T23:32:11.754 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.754 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.754 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.754 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T23:32:11.754 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.754 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.754 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.755 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_recent_interval": { 2024-09-11T23:32:11.755 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_recent_interval", 2024-09-11T23:32:11.755 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.755 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.755 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.755 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1209600", 2024-09-11T23:32:11.755 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.755 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.755 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.755 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T23:32:11.755 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.755 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.755 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.756 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dashboard", 2024-09-11T23:32:11.756 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.756 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.756 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.756 INFO:teuthology.orchestra.run.smithi043.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T23:32:11.756 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T23:32:11.756 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.756 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.756 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.756 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:11.757 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.757 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.757 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.757 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.757 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.757 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.757 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.757 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T23:32:11.757 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T23:32:11.757 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.757 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.757 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.758 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.758 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.758 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.758 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.758 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.758 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.758 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.758 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.758 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T23:32:11.758 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T23:32:11.758 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.758 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.759 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.759 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.759 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.759 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.759 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.759 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.759 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.759 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.759 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.759 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_ENABLED": { 2024-09-11T23:32:11.759 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T23:32:11.759 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.760 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.760 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.760 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.760 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.760 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.760 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.760 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.760 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.760 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.760 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.760 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T23:32:11.760 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T23:32:11.761 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.761 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.761 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.761 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.761 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.761 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.761 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.761 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.761 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.761 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.761 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.761 INFO:teuthology.orchestra.run.smithi043.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T23:32:11.761 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T23:32:11.762 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.762 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.762 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.762 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.762 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.762 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.762 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.762 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.762 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.762 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.762 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.762 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T23:32:11.763 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T23:32:11.763 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.763 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.763 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.763 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.763 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.763 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.763 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.763 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.763 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.763 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.763 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.763 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T23:32:11.764 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T23:32:11.764 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.764 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.764 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.764 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.764 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.764 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.764 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.764 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.764 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.764 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.764 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.765 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T23:32:11.765 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T23:32:11.765 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.765 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.765 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.765 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.765 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.765 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.765 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.765 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.765 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.765 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.765 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.766 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T23:32:11.766 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T23:32:11.766 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.766 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.766 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.766 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.766 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.766 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.766 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.766 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.766 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.766 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.766 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.767 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T23:32:11.767 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T23:32:11.767 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.767 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.767 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.767 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.767 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.767 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.767 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.767 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.767 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.767 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.767 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.768 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T23:32:11.768 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T23:32:11.768 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.768 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.768 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.768 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.768 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.768 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.768 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.768 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.768 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.768 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.768 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.769 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T23:32:11.769 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T23:32:11.769 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.769 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.769 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.769 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.769 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.769 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.769 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.769 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.769 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.769 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.769 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.770 INFO:teuthology.orchestra.run.smithi043.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T23:32:11.770 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T23:32:11.770 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.770 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.770 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.770 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.770 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.770 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.770 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.770 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.770 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.770 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.770 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.771 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T23:32:11.771 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T23:32:11.771 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.771 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.771 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.771 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-09-11T23:32:11.771 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.771 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.771 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.771 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.771 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.771 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.772 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.772 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T23:32:11.772 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T23:32:11.772 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.772 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.772 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.772 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.772 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.772 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.772 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.772 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.772 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.773 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.773 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.773 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_URL": { 2024-09-11T23:32:11.773 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_URL", 2024-09-11T23:32:11.773 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.773 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.773 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.773 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.773 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.773 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.773 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.773 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.773 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.774 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.774 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.774 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T23:32:11.774 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T23:32:11.774 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.774 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.774 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.774 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-09-11T23:32:11.774 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.774 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.774 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.774 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.775 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.775 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.775 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.775 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T23:32:11.775 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T23:32:11.775 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.775 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.775 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.775 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.775 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.775 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.775 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.775 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.776 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.776 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.776 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.776 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T23:32:11.776 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T23:32:11.776 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.776 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.776 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.776 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.776 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.776 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.776 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.777 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.777 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.777 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.777 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.777 INFO:teuthology.orchestra.run.smithi043.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T23:32:11.777 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T23:32:11.777 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.777 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.777 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.777 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.777 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.777 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.778 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.778 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.778 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.778 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.778 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.778 INFO:teuthology.orchestra.run.smithi043.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T23:32:11.778 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T23:32:11.778 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.778 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.778 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.778 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.778 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.779 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.779 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.779 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.779 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.779 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.779 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.779 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T23:32:11.779 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T23:32:11.779 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.779 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.779 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.779 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.779 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.780 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.780 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.780 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.780 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.780 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.780 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.780 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T23:32:11.780 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T23:32:11.780 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.780 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.780 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.780 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.781 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.781 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.781 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.781 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.781 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.781 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.781 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.781 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T23:32:11.781 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T23:32:11.781 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.781 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.781 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.782 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.782 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.782 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.782 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.782 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.782 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.782 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.782 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.782 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T23:32:11.782 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T23:32:11.782 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.782 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.783 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.783 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.783 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.783 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.783 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.783 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.783 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.783 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.783 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.783 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T23:32:11.783 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T23:32:11.783 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.783 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.784 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.784 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.784 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.784 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.784 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.784 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.784 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.784 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.784 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.784 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T23:32:11.784 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T23:32:11.785 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.785 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.785 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.785 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.785 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.785 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.785 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.785 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.785 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.785 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.785 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.785 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T23:32:11.786 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T23:32:11.786 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.786 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.786 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.786 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.786 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.786 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.786 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.786 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.786 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.786 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.786 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.787 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T23:32:11.787 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T23:32:11.787 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.787 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.787 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.787 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.787 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.787 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.787 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.787 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.787 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.787 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.787 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.788 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T23:32:11.788 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T23:32:11.788 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.788 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.788 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.788 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.788 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.788 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.788 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.788 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.788 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.788 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.789 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.789 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T23:32:11.789 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T23:32:11.789 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.789 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.789 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.789 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.789 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.789 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.789 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.789 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.789 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.789 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.790 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.790 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T23:32:11.790 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T23:32:11.790 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.790 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.790 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.790 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T23:32:11.790 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.790 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.790 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.790 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.790 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.791 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.791 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.791 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T23:32:11.791 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T23:32:11.791 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.791 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.791 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.791 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:11.791 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.791 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.791 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.791 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.792 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.792 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.792 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.792 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T23:32:11.792 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T23:32:11.792 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.792 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.792 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.792 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8", 2024-09-11T23:32:11.792 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.792 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.792 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.793 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.793 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.793 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.793 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.793 INFO:teuthology.orchestra.run.smithi043.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T23:32:11.793 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T23:32:11.793 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.793 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.793 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.793 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "45", 2024-09-11T23:32:11.793 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.793 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.794 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.794 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.794 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.794 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.794 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.794 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T23:32:11.794 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T23:32:11.794 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.794 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.794 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.794 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.794 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.794 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.795 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.795 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.795 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.795 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.795 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.795 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T23:32:11.795 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T23:32:11.795 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.795 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.795 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.795 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-09-11T23:32:11.795 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.796 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.796 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.796 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.796 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.796 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.796 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.796 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T23:32:11.796 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T23:32:11.796 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.796 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.796 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.796 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.797 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.797 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.797 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.797 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.797 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.797 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.797 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.797 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T23:32:11.797 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T23:32:11.797 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.797 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.797 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.798 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.798 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.798 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.798 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.798 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.798 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.798 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.798 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.798 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T23:32:11.798 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T23:32:11.798 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.798 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.799 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.799 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-11T23:32:11.799 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.799 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.799 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.799 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.799 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.799 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.799 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.799 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T23:32:11.799 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T23:32:11.799 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.799 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.800 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.800 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:11.800 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.800 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.800 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.800 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.800 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.800 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.800 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.800 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T23:32:11.800 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T23:32:11.800 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.801 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.801 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.801 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-11T23:32:11.801 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.801 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.801 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.801 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.801 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.801 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.801 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.801 INFO:teuthology.orchestra.run.smithi043.stdout: "cross_origin_url": { 2024-09-11T23:32:11.801 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cross_origin_url", 2024-09-11T23:32:11.802 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.802 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.802 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.802 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.802 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.802 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.802 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.802 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.802 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.802 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.802 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.802 INFO:teuthology.orchestra.run.smithi043.stdout: "crt_file": { 2024-09-11T23:32:11.802 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crt_file", 2024-09-11T23:32:11.803 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.803 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.803 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.803 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.803 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.803 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.803 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.803 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.803 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.803 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.803 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.803 INFO:teuthology.orchestra.run.smithi043.stdout: "debug": { 2024-09-11T23:32:11.803 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "debug", 2024-09-11T23:32:11.804 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.804 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.804 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.804 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.804 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.804 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.804 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.804 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable/disable debug options", 2024-09-11T23:32:11.804 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.804 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.804 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.804 INFO:teuthology.orchestra.run.smithi043.stdout: "jwt_token_ttl": { 2024-09-11T23:32:11.805 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "jwt_token_ttl", 2024-09-11T23:32:11.805 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.805 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.805 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.805 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "28800", 2024-09-11T23:32:11.805 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.805 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.805 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.805 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.805 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.805 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.805 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.806 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-09-11T23:32:11.806 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-09-11T23:32:11.806 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.806 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.806 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.806 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.806 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.806 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.806 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.806 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.806 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.806 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.806 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.807 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.807 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.807 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.807 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.807 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.807 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.807 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.807 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.807 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.807 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.807 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.807 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.807 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.808 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.808 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.808 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.808 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.808 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.808 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.808 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.808 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.808 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.808 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.808 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.808 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.809 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.809 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.809 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.809 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.809 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.809 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.809 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.809 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.809 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.809 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.809 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.809 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.810 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.810 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.810 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.810 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.810 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.810 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.810 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.810 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.810 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.810 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.810 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.810 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.810 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.810 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.811 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.811 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.811 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.811 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.811 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.811 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.811 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.811 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.811 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.811 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.811 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.811 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.812 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.812 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.812 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.812 INFO:teuthology.orchestra.run.smithi043.stdout: "motd": { 2024-09-11T23:32:11.812 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "motd", 2024-09-11T23:32:11.812 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.812 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.812 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.812 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.812 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.812 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.812 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.812 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "The message of the day", 2024-09-11T23:32:11.813 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.813 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.813 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.813 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect_resolve_ip_addr": { 2024-09-11T23:32:11.813 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T23:32:11.813 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.813 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.813 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.813 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.813 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.813 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.813 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.814 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.814 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.814 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.814 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.814 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-09-11T23:32:11.814 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-09-11T23:32:11.814 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.814 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.814 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.814 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-09-11T23:32:11.814 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.814 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.814 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.815 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.815 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.815 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.815 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.815 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-09-11T23:32:11.815 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-09-11T23:32:11.815 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.815 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.815 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.815 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8080", 2024-09-11T23:32:11.815 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.815 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.815 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.816 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.816 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.816 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.816 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.816 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-09-11T23:32:11.816 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-09-11T23:32:11.816 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.816 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.816 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.816 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.816 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.816 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.817 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.817 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.817 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.817 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.817 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.817 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl_server_port": { 2024-09-11T23:32:11.817 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl_server_port", 2024-09-11T23:32:11.817 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.817 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.817 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.817 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8443", 2024-09-11T23:32:11.817 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.817 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.818 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.818 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.818 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.818 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.818 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.818 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-09-11T23:32:11.818 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-09-11T23:32:11.818 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.818 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.818 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.818 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "redirect", 2024-09-11T23:32:11.818 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.818 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.819 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.819 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.819 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect" 2024-09-11T23:32:11.819 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.819 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.819 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.819 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.819 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.819 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-09-11T23:32:11.819 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-09-11T23:32:11.819 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.819 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.819 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.820 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-09-11T23:32:11.820 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-09-11T23:32:11.820 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-09-11T23:32:11.820 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.820 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.820 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.820 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.820 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.820 INFO:teuthology.orchestra.run.smithi043.stdout: "url_prefix": { 2024-09-11T23:32:11.820 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url_prefix", 2024-09-11T23:32:11.820 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.820 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.821 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.821 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.821 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.821 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.821 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.821 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.821 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.821 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.821 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.821 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.821 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.822 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "devicehealth", 2024-09-11T23:32:11.822 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.822 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.822 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.822 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_monitoring": { 2024-09-11T23:32:11.822 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_monitoring", 2024-09-11T23:32:11.822 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.822 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.822 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.822 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.822 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.822 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.822 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.822 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "monitor device health metrics", 2024-09-11T23:32:11.823 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.823 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.823 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.823 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.823 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.823 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.823 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.823 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.823 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.823 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.823 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.823 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.823 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.824 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.824 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.824 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.824 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.824 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.824 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.824 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.824 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.824 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.824 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.824 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.824 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.825 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.825 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.825 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.825 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.825 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.825 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.825 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.825 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.825 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.825 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.825 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.825 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.826 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.826 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.826 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.826 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.826 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.826 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.826 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.826 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.826 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.826 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.826 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.826 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.826 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.827 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.827 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.827 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.827 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.827 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.827 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.827 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.827 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.827 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.827 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.827 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.827 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.828 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.828 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.828 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.828 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.828 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.828 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.828 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.828 INFO:teuthology.orchestra.run.smithi043.stdout: "mark_out_threshold": { 2024-09-11T23:32:11.828 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mark_out_threshold", 2024-09-11T23:32:11.828 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.828 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.828 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.828 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2419200", 2024-09-11T23:32:11.829 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.829 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.829 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.829 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T23:32:11.829 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.829 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.829 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.829 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": { 2024-09-11T23:32:11.829 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_name", 2024-09-11T23:32:11.829 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.829 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.829 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.830 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "device_health_metrics", 2024-09-11T23:32:11.830 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.830 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.830 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.830 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T23:32:11.830 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.830 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.830 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.830 INFO:teuthology.orchestra.run.smithi043.stdout: "retention_period": { 2024-09-11T23:32:11.830 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retention_period", 2024-09-11T23:32:11.830 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.830 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.831 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.831 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15552000", 2024-09-11T23:32:11.831 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.831 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.831 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.831 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain device health metrics", 2024-09-11T23:32:11.831 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.831 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.831 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.831 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_frequency": { 2024-09-11T23:32:11.831 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_frequency", 2024-09-11T23:32:11.831 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.832 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.832 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.832 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-09-11T23:32:11.832 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.832 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.832 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.832 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T23:32:11.832 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.832 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.832 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.832 INFO:teuthology.orchestra.run.smithi043.stdout: "self_heal": { 2024-09-11T23:32:11.832 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "self_heal", 2024-09-11T23:32:11.833 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.833 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.833 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.833 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.833 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.833 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.833 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.833 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T23:32:11.833 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.833 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.833 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.833 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-11T23:32:11.833 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-11T23:32:11.834 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.834 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.834 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.834 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-11T23:32:11.834 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.834 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.834 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.834 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T23:32:11.834 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.834 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.834 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.834 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_threshold": { 2024-09-11T23:32:11.835 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_threshold", 2024-09-11T23:32:11.835 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.835 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.835 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.835 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7257600", 2024-09-11T23:32:11.835 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.835 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.835 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.835 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T23:32:11.835 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.835 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.835 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.835 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.835 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.836 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "diskprediction_local", 2024-09-11T23:32:11.836 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.836 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.836 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.836 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.836 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.836 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.836 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.836 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.836 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.836 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.837 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.837 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.837 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.837 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.837 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.837 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.837 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.837 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.837 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.837 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.837 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.837 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.838 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.838 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.838 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.838 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.838 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.838 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.838 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.839 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.843 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.843 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.843 INFO:teuthology.orchestra.run.smithi043.stdout: "predict_interval": { 2024-09-11T23:32:11.843 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predict_interval", 2024-09-11T23:32:11.843 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.843 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.843 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.843 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-09-11T23:32:11.843 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: "predictor_model": { 2024-09-11T23:32:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predictor_model", 2024-09-11T23:32:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "prophetstor", 2024-09-11T23:32:11.845 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.845 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.845 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.845 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.845 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.845 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.845 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.845 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-11T23:32:11.845 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-11T23:32:11.845 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.845 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.845 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-09-11T23:32:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "influx", 2024-09-11T23:32:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": false, 2024-09-11T23:32:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "influxdb python module not found", 2024-09-11T23:32:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "batch_size": { 2024-09-11T23:32:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "batch_size", 2024-09-11T23:32:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5000", 2024-09-11T23:32:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T23:32:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: "database": { 2024-09-11T23:32:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "database", 2024-09-11T23:32:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "ceph", 2024-09-11T23:32:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T23:32:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": { 2024-09-11T23:32:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hostname", 2024-09-11T23:32:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server hostname", 2024-09-11T23:32:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-11T23:32:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-11T23:32:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "30", 2024-09-11T23:32:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "5", 2024-09-11T23:32:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T23:32:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: "password": { 2024-09-11T23:32:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "password", 2024-09-11T23:32:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "password of InfluxDB server user", 2024-09-11T23:32:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "port": { 2024-09-11T23:32:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "port", 2024-09-11T23:32:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8086", 2024-09-11T23:32:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server port", 2024-09-11T23:32:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-09-11T23:32:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-09-11T23:32:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "false", 2024-09-11T23:32:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:32:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: "threads": { 2024-09-11T23:32:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threads", 2024-09-11T23:32:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-11T23:32:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-11T23:32:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "32", 2024-09-11T23:32:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T23:32:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: "username": { 2024-09-11T23:32:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "username", 2024-09-11T23:32:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "username of InfluxDB server user", 2024-09-11T23:32:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "verify_ssl": { 2024-09-11T23:32:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "verify_ssl", 2024-09-11T23:32:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "true", 2024-09-11T23:32:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:32:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "insights", 2024-09-11T23:32:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.867 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.867 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.867 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.867 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.867 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.867 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.867 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.867 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.867 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.867 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.867 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.867 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.867 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.868 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.868 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.868 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.868 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.868 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.868 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.868 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.868 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.868 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.868 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.868 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.868 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.868 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.869 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.869 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.869 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "iostat", 2024-09-11T23:32:11.869 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.869 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.869 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.869 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.869 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.869 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.869 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.870 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.870 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.870 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.870 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.870 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.870 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.870 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.870 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.870 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.870 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.870 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.870 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.870 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.870 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.871 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.871 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.871 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.871 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.871 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.871 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.871 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.871 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.871 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.871 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.871 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.871 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.871 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.872 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.872 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.872 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.872 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.872 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.872 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.872 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.872 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.872 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.872 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.872 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.872 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.873 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.873 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.873 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.873 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.873 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.873 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.873 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.873 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.873 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.873 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.873 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.873 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.873 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.874 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.874 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.874 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.874 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.874 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.874 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.874 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.874 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.874 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.874 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.874 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.874 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.875 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "k8sevents", 2024-09-11T23:32:11.875 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.875 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.875 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.875 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_event_retention_days": { 2024-09-11T23:32:11.875 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ceph_event_retention_days", 2024-09-11T23:32:11.875 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.875 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.875 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.875 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7", 2024-09-11T23:32:11.875 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.875 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.876 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.876 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T23:32:11.876 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.876 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.876 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.876 INFO:teuthology.orchestra.run.smithi043.stdout: "config_check_secs": { 2024-09-11T23:32:11.876 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_check_secs", 2024-09-11T23:32:11.876 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.876 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.876 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.876 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:11.876 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "10", 2024-09-11T23:32:11.876 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.877 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.877 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T23:32:11.877 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.877 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.877 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.877 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.877 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.877 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.877 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.877 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.877 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.877 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.878 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.878 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.878 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.878 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.878 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.878 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.878 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.878 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.878 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.878 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.878 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.878 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.878 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.879 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.879 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.879 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.879 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.879 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.879 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.879 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.879 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.879 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.879 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.879 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.880 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.880 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.880 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.880 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.880 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.880 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.880 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.880 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.880 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.880 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.880 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.880 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.881 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.881 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.881 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.881 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.881 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.881 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.881 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.881 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.881 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.881 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.881 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.881 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.882 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.882 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.882 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.882 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.882 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.882 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.882 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.882 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.882 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.882 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.882 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.882 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.882 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.883 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "localpool", 2024-09-11T23:32:11.883 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.883 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.883 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.883 INFO:teuthology.orchestra.run.smithi043.stdout: "failure_domain": { 2024-09-11T23:32:11.883 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "failure_domain", 2024-09-11T23:32:11.883 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.883 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.883 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.883 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "host", 2024-09-11T23:32:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "failure domain for any created local pool", 2024-09-11T23:32:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T23:32:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.885 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.885 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.885 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.885 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.885 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.885 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.885 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.885 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.885 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.885 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.885 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.885 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.885 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.886 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.886 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.886 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.886 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.886 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.886 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.886 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.886 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.886 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.886 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.886 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.886 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.887 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.887 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.887 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.887 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.887 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.887 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.887 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.887 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.887 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.887 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.887 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.888 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.888 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.888 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.888 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.888 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.888 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.888 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.888 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.888 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.888 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.888 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.888 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.889 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.889 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.889 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.889 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.889 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.889 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.889 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.889 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.889 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.889 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.889 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.889 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.889 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.890 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": { 2024-09-11T23:32:11.890 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_size", 2024-09-11T23:32:11.890 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.890 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.890 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.890 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.890 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.890 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.890 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.890 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default min_size for any created local pool", 2024-09-11T23:32:11.890 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T23:32:11.890 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.890 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.891 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rep": { 2024-09-11T23:32:11.891 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "num_rep", 2024-09-11T23:32:11.891 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.891 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.891 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.891 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3", 2024-09-11T23:32:11.891 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.891 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.891 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.891 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default replica count for any created local pool", 2024-09-11T23:32:11.891 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.892 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.892 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.892 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": { 2024-09-11T23:32:11.892 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_num", 2024-09-11T23:32:11.892 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.892 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.892 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.892 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "128", 2024-09-11T23:32:11.892 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.892 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.892 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.892 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T23:32:11.892 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.893 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.893 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.893 INFO:teuthology.orchestra.run.smithi043.stdout: "prefix": { 2024-09-11T23:32:11.893 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prefix", 2024-09-11T23:32:11.893 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.893 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.893 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.893 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.893 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.893 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.893 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.894 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name prefix for any created local pool", 2024-09-11T23:32:11.894 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.894 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.894 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.894 INFO:teuthology.orchestra.run.smithi043.stdout: "subtree": { 2024-09-11T23:32:11.894 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "subtree", 2024-09-11T23:32:11.894 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.894 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.894 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.894 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "rack", 2024-09-11T23:32:11.894 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.894 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T23:32:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T23:32:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mds_autoscaler", 2024-09-11T23:32:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.896 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.896 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.896 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.896 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.896 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.896 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.896 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.896 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.896 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.896 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.896 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.896 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.896 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirroring", 2024-09-11T23:32:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "nfs", 2024-09-11T23:32:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-09-11T23:32:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: "fail_fs": { 2024-09-11T23:32:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "fail_fs", 2024-09-11T23:32:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T23:32:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.917 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.917 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.917 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.917 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.917 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.917 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.917 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.917 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.917 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.917 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.917 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.917 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator": { 2024-09-11T23:32:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-09-11T23:32:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm", 2024-09-11T23:32:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "rook", 2024-09-11T23:32:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "test_orchestrator" 2024-09-11T23:32:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Orchestrator backend", 2024-09-11T23:32:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_perf_query", 2024-09-11T23:32:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_support", 2024-09-11T23:32:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_autoscaler", 2024-09-11T23:32:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-11T23:32:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-11T23:32:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-11T23:32:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: "threshold": { 2024-09-11T23:32:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threshold", 2024-09-11T23:32:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-11T23:32:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-09-11T23:32:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1.0", 2024-09-11T23:32:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "scaling threshold", 2024-09-11T23:32:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T23:32:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "progress", 2024-09-11T23:32:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_pg_recovery_event": { 2024-09-11T23:32:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_pg_recovery_event", 2024-09-11T23:32:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T23:32:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-09-11T23:32:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-09-11T23:32:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: "max_completed_events": { 2024-09-11T23:32:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_completed_events", 2024-09-11T23:32:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "50", 2024-09-11T23:32:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "number of past completed events to remember", 2024-09-11T23:32:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-09-11T23:32:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-09-11T23:32:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-09-11T23:32:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long the module is going to sleep", 2024-09-11T23:32:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus", 2024-09-11T23:32:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: "cache": { 2024-09-11T23:32:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cache", 2024-09-11T23:32:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.952 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.952 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.952 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.952 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.952 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.952 INFO:teuthology.orchestra.run.smithi043.stdout: "exclude_perf_counters": { 2024-09-11T23:32:11.952 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "exclude_perf_counters", 2024-09-11T23:32:11.952 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.952 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.952 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.952 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.952 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.953 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.953 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.953 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T23:32:11.953 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T23:32:11.953 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.953 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.953 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.953 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.953 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.953 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.953 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.953 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.954 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.954 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.954 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.954 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.954 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.954 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.954 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.954 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.954 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.954 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.954 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.954 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.955 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.955 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.955 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.955 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.955 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.955 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.955 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.955 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.955 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.955 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.955 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.955 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.955 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.956 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.956 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.956 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.956 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.956 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.956 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.956 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.956 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.956 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.956 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.956 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.956 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.956 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.957 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.957 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.957 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.957 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.957 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.957 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.957 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.957 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.957 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.957 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.957 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.957 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.958 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.958 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.958 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.958 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.958 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.958 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.958 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.958 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.958 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.958 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.958 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools": { 2024-09-11T23:32:11.959 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools", 2024-09-11T23:32:11.959 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.959 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.959 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.959 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.959 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.959 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.959 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.959 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.959 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.959 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.959 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.959 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T23:32:11.960 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T23:32:11.960 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.960 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.960 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.960 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "300", 2024-09-11T23:32:11.960 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.960 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.960 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.960 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.960 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.960 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.960 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.961 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_interval": { 2024-09-11T23:32:11.961 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_interval", 2024-09-11T23:32:11.961 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-11T23:32:11.961 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.961 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.961 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15.0", 2024-09-11T23:32:11.961 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.961 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.961 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.961 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.961 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.961 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.962 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.962 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-09-11T23:32:11.962 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-09-11T23:32:11.962 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.962 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.962 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.962 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-09-11T23:32:11.962 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.962 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.962 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.962 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T23:32:11.962 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.963 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.963 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.963 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-09-11T23:32:11.963 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-09-11T23:32:11.963 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.963 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.963 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.963 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "9283", 2024-09-11T23:32:11.963 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.963 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.963 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.963 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T23:32:11.964 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.964 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.964 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.964 INFO:teuthology.orchestra.run.smithi043.stdout: "stale_cache_strategy": { 2024-09-11T23:32:11.964 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stale_cache_strategy", 2024-09-11T23:32:11.964 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.964 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.964 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.964 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "log", 2024-09-11T23:32:11.964 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.964 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.964 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.964 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.965 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.965 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.965 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.965 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-09-11T23:32:11.965 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-09-11T23:32:11.965 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.965 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.965 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.965 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "default", 2024-09-11T23:32:11.965 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.965 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.965 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.966 INFO:teuthology.orchestra.run.smithi043.stdout: "default", 2024-09-11T23:32:11.966 INFO:teuthology.orchestra.run.smithi043.stdout: "error" 2024-09-11T23:32:11.966 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.966 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.966 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.966 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.966 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.966 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-09-11T23:32:11.966 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-09-11T23:32:11.966 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.966 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.966 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.967 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-09-11T23:32:11.967 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-09-11T23:32:11.967 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-09-11T23:32:11.967 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.967 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.967 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.967 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.967 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.967 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.967 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.967 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_support", 2024-09-11T23:32:11.968 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.968 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.968 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.968 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.968 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.968 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.968 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.968 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.968 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.968 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.968 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.968 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.969 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.969 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.969 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.969 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.969 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.969 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.969 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.969 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.969 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.969 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.969 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.969 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.970 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.970 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.970 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.970 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.970 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.970 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.970 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.970 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.970 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.970 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.970 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.970 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.970 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.971 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.971 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.971 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.971 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.971 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.971 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.971 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.971 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.971 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.971 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.971 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.971 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.971 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.972 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.972 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.972 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.972 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.972 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.972 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.972 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.972 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.972 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.972 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.972 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.973 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.973 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.973 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.973 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.973 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.973 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.973 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.973 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.973 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_snap_create": { 2024-09-11T23:32:11.973 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_snap_create", 2024-09-11T23:32:11.973 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:11.973 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.973 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.974 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-09-11T23:32:11.974 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.974 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.974 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.974 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.974 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.974 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.974 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.974 INFO:teuthology.orchestra.run.smithi043.stdout: "mirror_snapshot_schedule": { 2024-09-11T23:32:11.974 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T23:32:11.974 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.974 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.974 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.975 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.975 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.975 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.975 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.975 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.975 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.975 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.975 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.975 INFO:teuthology.orchestra.run.smithi043.stdout: "trash_purge_schedule": { 2024-09-11T23:32:11.975 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "trash_purge_schedule", 2024-09-11T23:32:11.975 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.975 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.975 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.976 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.976 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.976 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.976 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.976 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.976 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.976 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.976 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.976 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.976 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.977 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "restful", 2024-09-11T23:32:11.977 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.977 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.977 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.977 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_auth": { 2024-09-11T23:32:11.977 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_auth", 2024-09-11T23:32:11.977 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.977 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.977 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.977 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:11.977 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.977 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.977 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.977 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.978 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.978 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.978 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.978 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-09-11T23:32:11.978 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-09-11T23:32:11.978 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.978 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.978 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.978 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.978 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.978 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.978 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.978 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.979 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.979 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.979 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.979 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.979 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.979 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.979 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.979 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.979 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.979 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.979 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.979 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.980 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.980 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.980 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.980 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.980 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.980 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.980 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.980 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.980 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.980 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.980 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.980 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.980 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.981 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.981 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.981 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.981 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.981 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.981 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.981 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.981 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.981 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.981 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.981 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.981 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.982 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.982 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.982 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.982 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.982 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.982 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.982 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.982 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.982 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.982 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.982 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.982 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.982 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.983 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.983 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.983 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.983 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.983 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.983 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.983 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.983 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.983 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.983 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.983 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.983 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.984 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.984 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.984 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.984 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.984 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.984 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.984 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.984 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-09-11T23:32:11.984 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-09-11T23:32:11.984 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.984 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.984 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.985 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.985 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.985 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.985 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.985 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.985 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.985 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.985 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.985 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-09-11T23:32:11.985 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-09-11T23:32:11.985 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.985 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.986 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.986 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.986 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.986 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.986 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.986 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.986 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.986 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.986 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.986 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.986 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.986 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rgw", 2024-09-11T23:32:11.986 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.987 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.987 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.987 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.987 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.987 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.987 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.987 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.987 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.987 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.987 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.987 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.987 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.988 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.988 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.988 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.988 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.988 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.988 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.988 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.988 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.988 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.988 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.988 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.988 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.989 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.989 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.989 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.989 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.989 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.989 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.989 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.989 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.989 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.989 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.989 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.989 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.989 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.990 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.990 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.990 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.990 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.990 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.990 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.990 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.990 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.990 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.990 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.990 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.990 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.990 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.991 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.991 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.991 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.991 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.991 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.991 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.991 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.991 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.991 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.991 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.991 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.991 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.992 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.992 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.992 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.992 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.992 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.992 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.992 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.992 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.992 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rook", 2024-09-11T23:32:11.992 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.993 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.993 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:11.993 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:11.993 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:11.993 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.993 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.993 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.993 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:11.993 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.993 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.993 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.993 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.993 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.994 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.994 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.994 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.994 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.994 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.994 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.994 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.994 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.994 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.994 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:11.994 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:11.994 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.995 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.995 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.995 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.995 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.995 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.995 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.995 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.995 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.995 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.995 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.995 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:11.995 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:11.995 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.996 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.996 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.996 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:11.996 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.996 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.996 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:11.996 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:11.996 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:11.996 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:11.996 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:11.996 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:11.996 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:11.996 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:11.997 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.997 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.997 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.997 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.997 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:11.997 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:11.997 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:11.997 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.997 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:11.997 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:11.997 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.997 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.998 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.998 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:11.998 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.998 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.998 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.998 INFO:teuthology.orchestra.run.smithi043.stdout: "storage_class": { 2024-09-11T23:32:11.998 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "storage_class", 2024-09-11T23:32:11.998 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:11.998 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:11.998 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:11.998 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "local", 2024-09-11T23:32:11.998 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:11.998 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:11.999 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:11.999 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T23:32:11.999 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:11.999 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:11.999 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:11.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:11.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:11.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:11.999 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "selftest", 2024-09-11T23:32:11.999 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:11.999 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:11.999 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:12.000 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:12.000 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:12.000 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.000 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.000 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.000 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.000 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.000 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.000 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:12.000 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:12.000 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:12.000 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:12.000 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:12.000 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:12.001 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:12.001 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.001 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.001 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.001 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.001 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.001 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:12.001 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:12.001 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.001 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.001 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.001 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.001 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.002 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.002 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.002 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.002 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.002 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.002 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.002 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:12.002 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:12.002 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.002 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.002 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.002 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:12.003 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.003 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.003 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:12.003 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:12.003 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:12.003 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:12.003 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:12.003 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:12.003 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:12.003 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.003 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.003 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.003 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.003 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.004 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:12.004 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:12.004 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.004 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.004 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.004 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.004 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.004 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.004 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.004 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.004 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.004 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.005 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.005 INFO:teuthology.orchestra.run.smithi043.stdout: "roption1": { 2024-09-11T23:32:12.005 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption1", 2024-09-11T23:32:12.005 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.005 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.005 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.005 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.005 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.005 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.005 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.005 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.005 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.006 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.006 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.006 INFO:teuthology.orchestra.run.smithi043.stdout: "roption2": { 2024-09-11T23:32:12.006 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption2", 2024-09-11T23:32:12.006 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.006 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.006 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.006 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "xyz", 2024-09-11T23:32:12.006 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.006 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.006 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.006 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.006 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.007 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.007 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.007 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption1": { 2024-09-11T23:32:12.007 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption1", 2024-09-11T23:32:12.007 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.007 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.007 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.007 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.007 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.007 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.007 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.008 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.008 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.008 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.008 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.008 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption2": { 2024-09-11T23:32:12.008 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption2", 2024-09-11T23:32:12.008 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:12.008 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.008 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.008 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.008 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.008 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.009 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.009 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.009 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.009 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.009 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.009 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption3": { 2024-09-11T23:32:12.009 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption3", 2024-09-11T23:32:12.009 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-09-11T23:32:12.009 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.009 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.009 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.009 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.009 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.010 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.010 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.010 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.010 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.010 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.010 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption4": { 2024-09-11T23:32:12.010 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption4", 2024-09-11T23:32:12.010 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.010 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.010 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.010 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.010 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.010 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.011 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.011 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.011 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.011 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.011 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.011 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption5": { 2024-09-11T23:32:12.011 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption5", 2024-09-11T23:32:12.011 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.011 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.011 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.011 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.011 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.012 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.012 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.012 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.012 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.012 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.012 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.012 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption6": { 2024-09-11T23:32:12.012 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption6", 2024-09-11T23:32:12.012 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.012 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.012 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.012 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:12.012 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.013 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.013 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.013 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.013 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.013 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.013 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.013 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption7": { 2024-09-11T23:32:12.013 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption7", 2024-09-11T23:32:12.013 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:12.013 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.013 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.013 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.014 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-09-11T23:32:12.014 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "42", 2024-09-11T23:32:12.014 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.014 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.014 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.014 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.014 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.014 INFO:teuthology.orchestra.run.smithi043.stdout: "testkey": { 2024-09-11T23:32:12.014 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testkey", 2024-09-11T23:32:12.014 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.014 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.014 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.014 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.015 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.015 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.015 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.015 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.015 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.015 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.015 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.015 INFO:teuthology.orchestra.run.smithi043.stdout: "testlkey": { 2024-09-11T23:32:12.015 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testlkey", 2024-09-11T23:32:12.015 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.015 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.015 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.015 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.016 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.016 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.016 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.016 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.016 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.016 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.016 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.016 INFO:teuthology.orchestra.run.smithi043.stdout: "testnewline": { 2024-09-11T23:32:12.016 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testnewline", 2024-09-11T23:32:12.016 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.016 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.016 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.017 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.017 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.017 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.017 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.017 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.017 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.017 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.017 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.017 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.017 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:12.017 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-09-11T23:32:12.017 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:12.018 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:12.018 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:12.018 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_m_granularity": { 2024-09-11T23:32:12.018 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_m_granularity", 2024-09-11T23:32:12.018 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.018 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.018 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.018 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.018 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.018 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.018 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.018 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T23:32:12.018 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.018 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.019 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.019 INFO:teuthology.orchestra.run.smithi043.stdout: "dump_on_update": { 2024-09-11T23:32:12.019 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dump_on_update", 2024-09-11T23:32:12.019 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.019 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.019 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.019 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.019 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.019 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.019 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.019 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "dump database to debug log on update", 2024-09-11T23:32:12.019 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.019 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.020 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.020 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:12.020 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:12.020 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.020 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.020 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.020 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.020 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.020 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.020 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:12.020 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:12.020 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:12.020 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:12.021 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:12.021 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:12.021 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:12.021 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.021 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.021 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.021 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.021 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.021 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:12.021 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:12.021 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.021 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.022 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.022 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.022 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.022 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.022 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.022 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.022 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.022 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.022 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.022 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:12.022 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:12.022 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.023 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.023 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.023 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:12.023 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.023 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.023 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:12.023 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:12.023 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:12.023 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:12.023 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:12.023 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:12.023 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:12.023 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.023 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.024 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.024 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.024 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.024 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:12.024 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:12.024 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.024 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.024 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.024 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.024 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.024 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.024 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.025 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.025 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.025 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.025 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.025 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.025 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:12.025 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stats", 2024-09-11T23:32:12.025 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:12.025 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:12.025 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:12.025 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:12.026 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:12.026 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.026 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.026 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.026 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.026 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.026 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.026 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:12.026 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:12.026 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:12.026 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:12.026 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:12.026 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:12.026 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:12.027 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.027 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.027 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.027 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.027 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.027 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:12.027 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:12.027 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.027 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.027 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.027 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.027 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.028 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.028 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.028 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.028 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.028 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.028 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.028 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:12.028 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:12.028 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.028 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.028 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.029 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:12.029 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.029 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.029 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:12.029 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:12.029 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:12.029 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:12.029 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:12.029 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:12.029 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:12.029 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.029 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.029 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.030 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.030 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.030 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:12.030 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:12.030 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.030 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.030 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.030 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.030 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.030 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.030 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.030 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.031 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.031 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.031 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.031 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.031 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:12.031 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "status", 2024-09-11T23:32:12.031 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:12.031 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:12.031 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:12.031 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:12.031 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:12.032 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.032 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.032 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.032 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.032 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.032 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.032 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:12.032 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:12.032 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:12.032 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:12.032 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:12.032 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:12.032 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:12.033 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.033 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.033 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.033 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.033 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.033 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:12.033 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:12.033 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.033 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.033 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.033 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.034 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.034 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.034 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.034 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.034 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.034 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.034 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.034 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:12.034 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:12.034 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.034 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.034 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.034 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:12.035 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.035 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.035 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:12.035 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:12.035 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:12.035 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:12.035 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:12.035 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:12.035 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:12.035 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.035 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.035 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.035 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.036 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.036 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:12.036 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:12.036 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.036 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.036 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.036 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.036 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.036 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.036 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.036 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.036 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.037 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.037 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:12.037 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telegraf", 2024-09-11T23:32:12.037 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:12.037 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:12.037 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:12.037 INFO:teuthology.orchestra.run.smithi043.stdout: "address": { 2024-09-11T23:32:12.037 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "address", 2024-09-11T23:32:12.037 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.038 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.038 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.038 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T23:32:12.038 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.038 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.038 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.038 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.038 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.038 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.038 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.038 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-11T23:32:12.038 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-11T23:32:12.039 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:12.039 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.039 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.039 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15", 2024-09-11T23:32:12.039 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.039 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.039 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.039 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.039 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.039 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.039 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.039 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:12.039 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:12.040 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.040 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.040 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.040 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.040 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.040 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.040 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:12.040 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:12.040 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:12.040 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:12.040 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:12.040 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:12.040 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:12.041 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.041 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.041 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.041 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.041 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.041 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:12.041 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:12.041 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.041 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.044 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.044 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.044 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.044 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.044 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.044 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.044 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.044 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.044 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.044 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:12.044 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:12.044 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.045 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.045 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.045 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:12.045 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.045 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.045 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:12.045 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:12.045 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:12.045 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:12.045 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:12.045 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:12.045 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:12.045 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.046 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.046 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.046 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.046 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.046 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:12.046 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:12.046 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.046 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.046 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.046 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.046 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.046 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.047 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.047 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.047 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.047 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.047 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.047 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.047 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:12.047 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telemetry", 2024-09-11T23:32:12.047 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:12.047 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:12.047 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:12.048 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_basic": { 2024-09-11T23:32:12.048 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_basic", 2024-09-11T23:32:12.048 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.048 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.048 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.048 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:12.048 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.048 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.048 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.048 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T23:32:12.048 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.048 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.048 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.049 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_crash": { 2024-09-11T23:32:12.049 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_crash", 2024-09-11T23:32:12.049 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.049 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.049 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.049 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:12.049 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.049 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.049 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.049 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T23:32:12.049 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.049 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.050 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.050 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_device": { 2024-09-11T23:32:12.050 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_device", 2024-09-11T23:32:12.050 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.050 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.050 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.050 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:12.050 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.050 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.050 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.050 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T23:32:12.050 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.051 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.051 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.051 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_ident": { 2024-09-11T23:32:12.051 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_ident", 2024-09-11T23:32:12.051 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.051 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.051 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.051 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.051 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.051 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.051 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.051 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T23:32:12.052 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.052 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.052 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.052 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_perf": { 2024-09-11T23:32:12.052 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_perf", 2024-09-11T23:32:12.052 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.052 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.052 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.052 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.052 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.052 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.052 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.052 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T23:32:12.053 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.053 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.053 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.053 INFO:teuthology.orchestra.run.smithi043.stdout: "contact": { 2024-09-11T23:32:12.053 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "contact", 2024-09-11T23:32:12.053 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.053 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.053 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.053 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.053 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.053 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.053 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.054 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.054 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.054 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.054 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.054 INFO:teuthology.orchestra.run.smithi043.stdout: "description": { 2024-09-11T23:32:12.054 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "description", 2024-09-11T23:32:12.054 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.054 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.054 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.054 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.054 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.054 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.055 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.055 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.055 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.055 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.055 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.055 INFO:teuthology.orchestra.run.smithi043.stdout: "device_url": { 2024-09-11T23:32:12.055 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_url", 2024-09-11T23:32:12.055 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.055 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.055 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.055 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T23:32:12.055 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.056 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.056 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.056 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.056 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.056 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.056 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.056 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-09-11T23:32:12.056 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-09-11T23:32:12.056 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.056 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.056 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.056 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.056 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.057 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.057 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.057 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.057 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.057 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.057 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.057 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-11T23:32:12.057 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-11T23:32:12.057 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:12.057 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.057 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.057 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "24", 2024-09-11T23:32:12.057 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "8", 2024-09-11T23:32:12.058 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.058 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.058 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.058 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.058 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.058 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.058 INFO:teuthology.orchestra.run.smithi043.stdout: "last_opt_revision": { 2024-09-11T23:32:12.058 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "last_opt_revision", 2024-09-11T23:32:12.058 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:12.058 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.058 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.058 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1", 2024-09-11T23:32:12.059 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.059 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.059 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.059 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.059 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.059 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.059 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.059 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard": { 2024-09-11T23:32:12.059 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard", 2024-09-11T23:32:12.059 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.059 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.059 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.059 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.060 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.060 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.060 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.060 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.060 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.060 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.060 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.060 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard_description": { 2024-09-11T23:32:12.060 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard_description", 2024-09-11T23:32:12.060 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.060 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.060 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.061 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.061 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.061 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.061 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.061 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.061 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.061 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.061 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.061 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:12.061 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:12.061 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.061 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.062 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.062 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.062 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.062 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.062 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:12.062 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:12.062 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:12.062 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:12.062 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:12.062 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:12.062 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:12.062 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.062 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.062 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.063 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.063 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.063 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:12.063 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:12.063 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.063 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.063 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.063 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.063 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.063 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.063 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.063 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.063 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.064 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.064 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.064 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:12.064 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:12.064 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.064 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.064 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.064 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:12.064 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.064 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.064 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:12.064 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:12.065 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:12.065 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:12.065 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:12.065 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:12.065 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:12.065 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.065 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.065 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.065 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.065 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.065 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:12.065 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:12.065 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.066 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.066 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.066 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.066 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.066 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.066 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.066 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.066 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.066 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.066 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.066 INFO:teuthology.orchestra.run.smithi043.stdout: "organization": { 2024-09-11T23:32:12.067 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "organization", 2024-09-11T23:32:12.067 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.067 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.067 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.067 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.067 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.067 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.067 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.067 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.067 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.067 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.067 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.067 INFO:teuthology.orchestra.run.smithi043.stdout: "proxy": { 2024-09-11T23:32:12.068 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "proxy", 2024-09-11T23:32:12.068 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.068 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.068 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.068 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.068 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.068 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.068 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.068 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.068 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.068 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.068 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.069 INFO:teuthology.orchestra.run.smithi043.stdout: "url": { 2024-09-11T23:32:12.069 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url", 2024-09-11T23:32:12.069 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.069 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.069 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.069 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T23:32:12.069 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.069 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.069 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.069 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.069 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.069 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.069 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.069 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.070 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:12.070 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "test_orchestrator", 2024-09-11T23:32:12.070 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:12.070 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:12.070 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:12.070 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:12.070 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:12.070 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.070 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.070 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.070 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.070 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.071 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.071 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:12.071 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:12.071 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:12.071 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:12.071 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:12.071 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:12.071 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:12.071 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.071 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.071 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.071 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.071 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.072 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:12.072 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:12.072 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.072 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.072 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.072 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.072 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.072 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.072 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.072 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.072 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.072 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.072 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.073 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:12.073 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:12.073 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.073 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.073 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.073 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:12.073 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.073 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.073 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:12.073 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:12.073 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:12.073 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:12.073 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:12.074 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:12.074 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:12.074 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.074 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.074 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.074 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.074 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.074 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:12.074 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:12.074 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.074 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.074 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.074 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.075 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.075 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.075 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.075 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.075 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.075 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.075 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.075 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.075 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:12.075 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "volumes", 2024-09-11T23:32:12.075 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:12.075 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:12.076 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:12.076 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:12.076 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:12.076 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.076 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.076 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.076 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.076 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.076 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.076 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:12.076 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:12.076 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:12.076 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:12.077 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:12.077 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:12.077 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:12.077 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.077 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.077 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.077 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.077 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.077 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:12.077 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:12.077 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.077 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.077 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.078 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.078 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.078 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.078 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.078 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.078 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.078 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.078 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.078 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:12.078 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:12.078 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.078 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.078 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.079 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:12.079 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.079 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.079 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:12.079 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:12.079 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:12.079 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:12.079 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:12.079 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:12.079 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:12.079 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.079 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.079 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.080 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.080 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.080 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:12.080 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:12.080 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.080 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.080 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.080 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.080 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.080 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.080 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.080 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.080 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.081 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.081 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.081 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_clones": { 2024-09-11T23:32:12.081 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_clones", 2024-09-11T23:32:12.081 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:12.081 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.081 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.081 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4", 2024-09-11T23:32:12.081 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.081 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.081 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.081 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T23:32:12.082 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.082 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.082 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.082 INFO:teuthology.orchestra.run.smithi043.stdout: "periodic_async_work": { 2024-09-11T23:32:12.082 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "periodic_async_work", 2024-09-11T23:32:12.082 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.082 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.082 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.082 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.082 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.082 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.082 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.083 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Periodically check for async work", 2024-09-11T23:32:12.083 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.083 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.083 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.083 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_delay": { 2024-09-11T23:32:12.083 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_delay", 2024-09-11T23:32:12.083 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:12.083 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.083 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.083 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-09-11T23:32:12.083 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.083 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.084 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.084 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T23:32:12.084 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.084 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.084 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.084 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_no_wait": { 2024-09-11T23:32:12.084 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T23:32:12.084 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.084 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.084 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.084 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-09-11T23:32:12.084 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.084 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.085 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.085 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T23:32:12.085 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.085 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.085 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.085 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.085 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:12.085 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix", 2024-09-11T23:32:12.085 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-09-11T23:32:12.085 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-09-11T23:32:12.085 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-09-11T23:32:12.085 INFO:teuthology.orchestra.run.smithi043.stdout: "discovery_interval": { 2024-09-11T23:32:12.086 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "discovery_interval", 2024-09-11T23:32:12.086 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-09-11T23:32:12.086 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.086 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.086 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "100", 2024-09-11T23:32:12.086 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.086 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.086 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.086 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.086 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.086 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.086 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.087 INFO:teuthology.orchestra.run.smithi043.stdout: "identifier": { 2024-09-11T23:32:12.087 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "identifier", 2024-09-11T23:32:12.087 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.087 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.087 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.087 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.087 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.087 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.087 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.087 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.087 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.087 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.087 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.087 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-09-11T23:32:12.088 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-09-11T23:32:12.088 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-09-11T23:32:12.088 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.088 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.088 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-09-11T23:32:12.088 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.088 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.088 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.088 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.088 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.088 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.088 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.089 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-09-11T23:32:12.089 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-09-11T23:32:12.089 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.089 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.089 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.089 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.089 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.089 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.089 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:12.089 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:12.089 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:12.089 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:12.089 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:12.090 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:12.090 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:12.090 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.090 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.090 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.090 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.090 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.090 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-09-11T23:32:12.090 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-09-11T23:32:12.090 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.090 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.090 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.090 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.091 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.091 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.091 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.091 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.091 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.091 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.091 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.091 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-09-11T23:32:12.091 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-09-11T23:32:12.091 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.091 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.091 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.092 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-09-11T23:32:12.092 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.092 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.092 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-09-11T23:32:12.092 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-09-11T23:32:12.092 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-09-11T23:32:12.092 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-09-11T23:32:12.092 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-09-11T23:32:12.092 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-09-11T23:32:12.092 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-09-11T23:32:12.092 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.092 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.092 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.092 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.093 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.093 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-09-11T23:32:12.093 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-09-11T23:32:12.093 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-09-11T23:32:12.093 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.093 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-11T23:32:12.093 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-09-11T23:32:12.093 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.093 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.093 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.093 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.093 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.093 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.094 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.094 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_host": { 2024-09-11T23:32:12.094 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_host", 2024-09-11T23:32:12.094 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.094 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.094 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.094 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-09-11T23:32:12.094 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.094 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.094 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.094 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.094 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.094 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.095 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.095 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_port": { 2024-09-11T23:32:12.095 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_port", 2024-09-11T23:32:12.095 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-09-11T23:32:12.095 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.095 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.095 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10051", 2024-09-11T23:32:12.095 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.095 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.095 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.095 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.095 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.095 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.096 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.096 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_sender": { 2024-09-11T23:32:12.096 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_sender", 2024-09-11T23:32:12.096 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-09-11T23:32:12.096 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-09-11T23:32:12.096 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-09-11T23:32:12.096 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T23:32:12.096 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-09-11T23:32:12.096 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-09-11T23:32:12.096 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-09-11T23:32:12.096 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-09-11T23:32:12.096 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-09-11T23:32:12.097 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-09-11T23:32:12.097 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-09-11T23:32:12.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.097 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.097 INFO:teuthology.orchestra.run.smithi043.stdout: "services": { 2024-09-11T23:32:12.097 INFO:teuthology.orchestra.run.smithi043.stdout: "dashboard": "https://172.21.15.43:8443/" 2024-09-11T23:32:12.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.097 INFO:teuthology.orchestra.run.smithi043.stdout: "always_on_modules": { 2024-09-11T23:32:12.097 INFO:teuthology.orchestra.run.smithi043.stdout: "octopus": [ 2024-09-11T23:32:12.097 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-09-11T23:32:12.097 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-09-11T23:32:12.098 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-09-11T23:32:12.098 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-09-11T23:32:12.098 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-09-11T23:32:12.098 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-09-11T23:32:12.098 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-09-11T23:32:12.098 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-09-11T23:32:12.098 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-09-11T23:32:12.098 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-09-11T23:32:12.098 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.098 INFO:teuthology.orchestra.run.smithi043.stdout: "pacific": [ 2024-09-11T23:32:12.098 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-09-11T23:32:12.098 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-09-11T23:32:12.098 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-09-11T23:32:12.098 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-09-11T23:32:12.099 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-09-11T23:32:12.099 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-09-11T23:32:12.099 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-09-11T23:32:12.099 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-09-11T23:32:12.099 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-09-11T23:32:12.099 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-09-11T23:32:12.099 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.099 INFO:teuthology.orchestra.run.smithi043.stdout: "quincy": [ 2024-09-11T23:32:12.099 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-09-11T23:32:12.099 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-09-11T23:32:12.099 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-09-11T23:32:12.099 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-09-11T23:32:12.099 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-09-11T23:32:12.100 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-09-11T23:32:12.100 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-09-11T23:32:12.100 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-09-11T23:32:12.100 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-09-11T23:32:12.100 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-09-11T23:32:12.100 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.100 INFO:teuthology.orchestra.run.smithi043.stdout: "reef": [ 2024-09-11T23:32:12.100 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-09-11T23:32:12.100 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-09-11T23:32:12.100 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-09-11T23:32:12.100 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-09-11T23:32:12.100 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-09-11T23:32:12.100 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-09-11T23:32:12.101 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-09-11T23:32:12.101 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-09-11T23:32:12.101 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-09-11T23:32:12.101 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-09-11T23:32:12.101 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-11T23:32:12.101 INFO:teuthology.orchestra.run.smithi043.stdout: "squid": [ 2024-09-11T23:32:12.101 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-09-11T23:32:12.101 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-09-11T23:32:12.101 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-09-11T23:32:12.101 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-09-11T23:32:12.101 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-09-11T23:32:12.101 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-09-11T23:32:12.101 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-09-11T23:32:12.102 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-09-11T23:32:12.102 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-09-11T23:32:12.102 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-09-11T23:32:12.102 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-11T23:32:12.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.102 INFO:teuthology.orchestra.run.smithi043.stdout: "last_failure_osd_epoch": 85, 2024-09-11T23:32:12.102 INFO:teuthology.orchestra.run.smithi043.stdout: "active_clients": [ 2024-09-11T23:32:12.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:12.102 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "devicehealth", 2024-09-11T23:32:12.102 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-11T23:32:12.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:12.102 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-11T23:32:12.102 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-09-11T23:32:12.102 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 449604297 2024-09-11T23:32:12.103 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.103 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-11T23:32:12.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:12.103 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "libcephsqlite", 2024-09-11T23:32:12.103 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-11T23:32:12.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:12.103 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-11T23:32:12.103 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-09-11T23:32:12.103 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 428098360 2024-09-11T23:32:12.103 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.103 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-11T23:32:12.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:12.104 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_support", 2024-09-11T23:32:12.104 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-11T23:32:12.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:12.104 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-11T23:32:12.104 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-09-11T23:32:12.104 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 558849949 2024-09-11T23:32:12.104 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.104 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-11T23:32:12.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:12.104 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-09-11T23:32:12.104 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-11T23:32:12.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:12.105 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-11T23:32:12.105 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-09-11T23:32:12.105 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1982004841 2024-09-11T23:32:12.105 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.105 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-11T23:32:12.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:12.105 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-09-11T23:32:12.105 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-11T23:32:12.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:12.105 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-11T23:32:12.105 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-09-11T23:32:12.105 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2241782492 2024-09-11T23:32:12.105 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.106 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-11T23:32:12.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:12.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:12.106 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "volumes", 2024-09-11T23:32:12.106 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-11T23:32:12.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-11T23:32:12.106 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-11T23:32:12.106 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-09-11T23:32:12.106 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1845489665 2024-09-11T23:32:12.106 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.106 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-11T23:32:12.106 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:12.106 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-11T23:32:12.106 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:32:12.108 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:11 smithi097 ceph-mon[35562]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T23:32:12.108 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:11 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3805089278' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-11T23:32:12.108 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:11 smithi090 ceph-mon[35130]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T23:32:12.108 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:11 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3805089278' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-11T23:32:12.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.628+0000 7fac76b18640 1 -- 172.21.15.43:0/3805089278 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fac600a7310 msgr2=0x7fac600a97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.628+0000 7fac76b18640 1 --2- 172.21.15.43:0/3805089278 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fac600a7310 0x7fac600a97d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fac5c02aa50 tx=0x7fac5c005fd0 comp rx=0 tx=0).stop 2024-09-11T23:32:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.628+0000 7fac76b18640 1 -- 172.21.15.43:0/3805089278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac680aa610 msgr2=0x7fac6813ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:32:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.628+0000 7fac76b18640 1 --2- 172.21.15.43:0/3805089278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac680aa610 0x7fac6813ae80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fac6400b4a0 tx=0x7fac6400b960 comp rx=0 tx=0).stop 2024-09-11T23:32:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.628+0000 7fac76b18640 1 -- 172.21.15.43:0/3805089278 shutdown_connections 2024-09-11T23:32:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.628+0000 7fac76b18640 1 --2- 172.21.15.43:0/3805089278 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fac600a7310 0x7fac600a97d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.628+0000 7fac76b18640 1 --2- 172.21.15.43:0/3805089278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac680abb70 0x7fac6813f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.628+0000 7fac76b18640 1 --2- 172.21.15.43:0/3805089278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac680ab1f0 0x7fac6813b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:32:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.628+0000 7fac76b18640 1 --2- 172.21.15.43:0/3805089278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac680aa610 0x7fac6813ae80 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fac6400b4a0 tx=0x7fac6400b960 comp rx=0 tx=0).stop 2024-09-11T23:32:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.628+0000 7fac76b18640 1 -- 172.21.15.43:0/3805089278 >> 172.21.15.43:0/3805089278 conn(0x7fac6801a0c0 msgr2=0x7fac680a3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:32:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.629+0000 7fac76b18640 1 -- 172.21.15.43:0/3805089278 shutdown_connections 2024-09-11T23:32:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:32:10.629+0000 7fac76b18640 1 -- 172.21.15.43:0/3805089278 wait complete. 2024-09-11T23:32:12.117 DEBUG:tasks.check_counter:Getting stats from a 2024-09-11T23:32:12.117 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-11T23:32:12.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:11 smithi043 ceph-mon[30950]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T23:32:12.118 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:11 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3805089278' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-11T23:32:12.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:12 smithi043 ceph-mon[30950]: pgmap v321: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-09-11T23:32:12.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:12 smithi090 ceph-mon[35130]: pgmap v321: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-09-11T23:32:12.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:12 smithi097 ceph-mon[35562]: pgmap v321: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-09-11T23:32:12.622 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:32:14.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:14 smithi090 ceph-mon[35130]: pgmap v322: 97 pgs: 8 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 72 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-09-11T23:32:14.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:14 smithi097 ceph-mon[35562]: pgmap v322: 97 pgs: 8 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 72 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-09-11T23:32:14.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:14 smithi043 ceph-mon[30950]: pgmap v322: 97 pgs: 8 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 72 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-09-11T23:32:16.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:16 smithi043 ceph-mon[30950]: pgmap v323: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-09-11T23:32:16.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:16 smithi090 ceph-mon[35130]: pgmap v323: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-09-11T23:32:16.552 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:16 smithi097 ceph-mon[35562]: pgmap v323: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 16 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-09-11T23:32:18.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:18 smithi090 ceph-mon[35130]: pgmap v324: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 14 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-11T23:32:18.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:18 smithi097 ceph-mon[35562]: pgmap v324: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 14 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-11T23:32:18.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:18 smithi043 ceph-mon[30950]: pgmap v324: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 14 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-11T23:32:18.958 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:32:18.958 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:32:18.958 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 0, 2024-09-11T23:32:18.958 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 2, 2024-09-11T23:32:18.958 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 0, 2024-09-11T23:32:18.958 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 0, 2024-09-11T23:32:18.958 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 4, 2024-09-11T23:32:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 0, 2024-09-11T23:32:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.002020609, 2024-09-11T23:32:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.000000000, 2024-09-11T23:32:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.000277623, 2024-09-11T23:32:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:32:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-11T23:32:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-11T23:32:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-11T23:32:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-11T23:32:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-11T23:32:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-11T23:32:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:32:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:32:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:32:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 217, 2024-09-11T23:32:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 204, 2024-09-11T23:32:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1581600, 2024-09-11T23:32:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 124512, 2024-09-11T23:32:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 2, 2024-09-11T23:32:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 1, 2024-09-11T23:32:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.042850672, 2024-09-11T23:32:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.013860050, 2024-09-11T23:32:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.041638390, 2024-09-11T23:32:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:32:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 204, 2024-09-11T23:32:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.014144307, 2024-09-11T23:32:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000069334 2024-09-11T23:32:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-11T23:32:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-11T23:32:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-11T23:32:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 1581600, 2024-09-11T23:32:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 124512 2024-09-11T23:32:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:32:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 2, 2024-09-11T23:32:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 155, 2024-09-11T23:32:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 320, 2024-09-11T23:32:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 184608, 2024-09-11T23:32:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 5, 2024-09-11T23:32:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 3, 2024-09-11T23:32:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.013277672, 2024-09-11T23:32:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.008968624, 2024-09-11T23:32:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.002105060, 2024-09-11T23:32:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:32:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 152, 2024-09-11T23:32:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.016770724, 2024-09-11T23:32:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000110333 2024-09-11T23:32:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-11T23:32:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-11T23:32:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-11T23:32:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T23:32:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 184608 2024-09-11T23:32:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-09-11T23:32:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 0, 2024-09-11T23:32:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-09-11T23:32:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-09-11T23:32:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:32:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-09-11T23:32:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:32:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:32:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:32:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:32:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:32:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:32:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:32:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:32:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:32:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:32:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:32:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:32:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:32:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-09-11T23:32:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:32:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-09-11T23:32:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:32:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:32:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:32:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:32:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:32:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:32:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:32:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-09-11T23:32:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:32:18.967 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:32:18.967 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:32:18.967 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-09-11T23:32:18.967 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-09-11T23:32:18.967 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-09-11T23:32:18.967 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:32:18.968 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:32:18.968 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:32:18.968 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:32:18.968 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 149017, 2024-09-11T23:32:18.968 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 31, 2024-09-11T23:32:18.968 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:32:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-09-11T23:32:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-09-11T23:32:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-09-11T23:32:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:32:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-09-11T23:32:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:32:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-09-11T23:32:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 0, 2024-09-11T23:32:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 0, 2024-09-11T23:32:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:32:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:32:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-09-11T23:32:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-09-11T23:32:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 928, 2024-09-11T23:32:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 2, 2024-09-11T23:32:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-09-11T23:32:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-09-11T23:32:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-09-11T23:32:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-09-11T23:32:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:32:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-09-11T23:32:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-09-11T23:32:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-09-11T23:32:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 215, 2024-09-11T23:32:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 1548529, 2024-09-11T23:32:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 215, 2024-09-11T23:32:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-09-11T23:32:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-09-11T23:32:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 215, 2024-09-11T23:32:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 1548529, 2024-09-11T23:32:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-09-11T23:32:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-11T23:32:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-11T23:32:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-11T23:32:18.972 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:18.972 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:18.972 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:32:20.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:20 smithi090 ceph-mon[35130]: pgmap v325: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 14 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-11T23:32:20.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:20 smithi097 ceph-mon[35562]: pgmap v325: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 14 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-11T23:32:20.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:20 smithi043 ceph-mon[30950]: pgmap v325: 97 pgs: 21 active+clean+snaptrim, 39 active+clean+snaptrim_wait, 37 active+clean; 14 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-11T23:32:22.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:22 smithi090 ceph-mon[35130]: pgmap v326: 97 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 63 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 198 B/s wr, 6 op/s 2024-09-11T23:32:22.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:22 smithi097 ceph-mon[35562]: pgmap v326: 97 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 63 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 198 B/s wr, 6 op/s 2024-09-11T23:32:22.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:22 smithi043 ceph-mon[30950]: pgmap v326: 97 pgs: 11 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 63 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 198 B/s wr, 6 op/s 2024-09-11T23:32:24.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:24 smithi090 ceph-mon[35130]: pgmap v327: 97 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 80 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-11T23:32:24.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:24 smithi097 ceph-mon[35562]: pgmap v327: 97 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 80 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-11T23:32:24.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:24 smithi043 ceph-mon[30950]: pgmap v327: 97 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 80 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-11T23:32:26.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:26 smithi090 ceph-mon[35130]: pgmap v328: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-11T23:32:26.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:26 smithi097 ceph-mon[35562]: pgmap v328: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-11T23:32:26.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:26 smithi043 ceph-mon[30950]: pgmap v328: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-11T23:32:28.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:28 smithi043 ceph-mon[30950]: pgmap v329: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-11T23:32:28.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:28 smithi090 ceph-mon[35130]: pgmap v329: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-11T23:32:28.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:28 smithi097 ceph-mon[35562]: pgmap v329: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-11T23:32:29.943 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-11T23:32:29.943 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-11T23:32:29.943 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-09-11T23:32:29.943 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-09-11T23:32:29.943 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-11T23:32:29.943 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-11T23:32:29.943 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-11T23:32:29.944 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-09-11T23:32:29.944 DEBUG:tasks.check_counter:Getting stats from d 2024-09-11T23:32:29.944 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-11T23:32:30.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:29 smithi090 ceph-mon[35130]: pgmap v330: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-11T23:32:30.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:29 smithi090 ceph-mon[35130]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T23:32:30.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:29 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:32:30.300 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:29 smithi097 ceph-mon[35562]: pgmap v330: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-11T23:32:30.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:29 smithi097 ceph-mon[35562]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T23:32:30.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:29 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:32:30.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:29 smithi043 ceph-mon[30950]: pgmap v330: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-11T23:32:30.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:29 smithi043 ceph-mon[30950]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T23:32:30.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:29 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:32:32.181 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:31 smithi043 ceph-mon[30950]: pgmap v332: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T23:32:32.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:31 smithi090 ceph-mon[35130]: pgmap v332: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T23:32:32.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:31 smithi097 ceph-mon[35562]: pgmap v332: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T23:32:34.200 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:33 smithi043 ceph-mon[30950]: pgmap v333: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T23:32:34.265 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:33 smithi090 ceph-mon[35130]: pgmap v333: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T23:32:34.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:33 smithi097 ceph-mon[35562]: pgmap v333: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T23:32:35.201 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:32:36.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:35 smithi043 ceph-mon[30950]: pgmap v334: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T23:32:36.153 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:32:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:32:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 0, 2024-09-11T23:32:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 2, 2024-09-11T23:32:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 0, 2024-09-11T23:32:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 0, 2024-09-11T23:32:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 4, 2024-09-11T23:32:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 0, 2024-09-11T23:32:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.001678623, 2024-09-11T23:32:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.000000000, 2024-09-11T23:32:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.000245928, 2024-09-11T23:32:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:32:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:36.154 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-11T23:32:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-11T23:32:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-11T23:32:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-11T23:32:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-11T23:32:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-11T23:32:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:32:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:32:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:32:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 228, 2024-09-11T23:32:36.155 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 215, 2024-09-11T23:32:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1584656, 2024-09-11T23:32:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 131072, 2024-09-11T23:32:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 2, 2024-09-11T23:32:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 1, 2024-09-11T23:32:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.043356157, 2024-09-11T23:32:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.013399944, 2024-09-11T23:32:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.043408862, 2024-09-11T23:32:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:32:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 215, 2024-09-11T23:32:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.016658331, 2024-09-11T23:32:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000077480 2024-09-11T23:32:36.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-11T23:32:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-11T23:32:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-11T23:32:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 1584656, 2024-09-11T23:32:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 131072 2024-09-11T23:32:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:32:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 2, 2024-09-11T23:32:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 162, 2024-09-11T23:32:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 320, 2024-09-11T23:32:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 185840, 2024-09-11T23:32:36.157 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 5, 2024-09-11T23:32:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 3, 2024-09-11T23:32:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.013657449, 2024-09-11T23:32:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.009110804, 2024-09-11T23:32:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.002096252, 2024-09-11T23:32:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:32:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 159, 2024-09-11T23:32:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.010424647, 2024-09-11T23:32:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000065563 2024-09-11T23:32:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:36.158 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-11T23:32:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-11T23:32:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-11T23:32:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T23:32:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 185840 2024-09-11T23:32:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-09-11T23:32:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 0, 2024-09-11T23:32:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-09-11T23:32:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:36.159 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-09-11T23:32:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:32:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-09-11T23:32:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:32:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:32:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:32:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:32:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:32:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:32:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:32:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:32:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:32:36.160 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:32:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:32:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:32:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:32:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-09-11T23:32:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:32:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-09-11T23:32:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:32:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:32:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:32:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:32:36.161 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:32:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:32:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:32:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-09-11T23:32:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:32:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:32:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:32:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-09-11T23:32:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-09-11T23:32:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-09-11T23:32:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:32:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:32:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:32:36.162 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:32:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 149011, 2024-09-11T23:32:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 32, 2024-09-11T23:32:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:32:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-09-11T23:32:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-09-11T23:32:36.163 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-09-11T23:32:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:32:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-09-11T23:32:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:32:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-09-11T23:32:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 0, 2024-09-11T23:32:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 0, 2024-09-11T23:32:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:32:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:32:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-09-11T23:32:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-09-11T23:32:36.164 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 928, 2024-09-11T23:32:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 2, 2024-09-11T23:32:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-09-11T23:32:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-09-11T23:32:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-09-11T23:32:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-09-11T23:32:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:32:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-09-11T23:32:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-09-11T23:32:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-09-11T23:32:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 226, 2024-09-11T23:32:36.165 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 1550023, 2024-09-11T23:32:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 226, 2024-09-11T23:32:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-09-11T23:32:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-09-11T23:32:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 226, 2024-09-11T23:32:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 1550023, 2024-09-11T23:32:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-09-11T23:32:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-11T23:32:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-11T23:32:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-11T23:32:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:36.166 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:36.167 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:32:36.244 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:35 smithi090 ceph-mon[35130]: pgmap v334: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T23:32:36.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:35 smithi097 ceph-mon[35562]: pgmap v334: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T23:32:36.986 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-11T23:32:36.986 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-11T23:32:36.986 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-11T23:32:36.986 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-11T23:32:36.987 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-11T23:32:36.987 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-11T23:32:36.987 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-11T23:32:36.987 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-11T23:32:36.987 DEBUG:tasks.check_counter:Getting stats from g 2024-09-11T23:32:36.987 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-11T23:32:37.275 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:32:37.963 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:37 smithi043 ceph-mon[30950]: pgmap v335: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T23:32:37.963 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:37 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:32:37.963 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:37 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:32:37.964 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:37 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:37.964 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:37 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:37.964 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:37 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:37.964 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:37 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:37.970 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:32:37.970 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:32:37.970 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 220, 2024-09-11T23:32:37.971 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 209, 2024-09-11T23:32:37.971 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1580496, 2024-09-11T23:32:37.971 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 128064, 2024-09-11T23:32:37.971 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 4, 2024-09-11T23:32:37.971 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 0, 2024-09-11T23:32:37.971 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.042552616, 2024-09-11T23:32:37.971 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.013892703, 2024-09-11T23:32:37.971 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.040452207, 2024-09-11T23:32:37.971 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:32:37.971 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:37.971 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 208, 2024-09-11T23:32:37.971 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.010462254, 2024-09-11T23:32:37.971 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000050299 2024-09-11T23:32:37.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:37.972 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:37.972 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-11T23:32:37.972 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-11T23:32:37.972 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-11T23:32:37.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:37.972 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 1580496, 2024-09-11T23:32:37.972 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 128064 2024-09-11T23:32:37.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:37.972 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:32:37.972 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 6, 2024-09-11T23:32:37.972 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 162, 2024-09-11T23:32:37.972 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 3392, 2024-09-11T23:32:37.972 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 185936, 2024-09-11T23:32:37.973 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 5, 2024-09-11T23:32:37.973 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 3, 2024-09-11T23:32:37.973 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.015640688, 2024-09-11T23:32:37.973 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.009366277, 2024-09-11T23:32:37.973 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.003703576, 2024-09-11T23:32:37.973 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:32:37.973 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:37.973 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 160, 2024-09-11T23:32:37.973 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.016954196, 2024-09-11T23:32:37.973 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000105963 2024-09-11T23:32:37.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:37.974 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:37.974 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-11T23:32:37.974 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-11T23:32:37.974 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-11T23:32:37.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:37.974 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-09-11T23:32:37.974 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 185936 2024-09-11T23:32:37.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:37.974 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:32:37.974 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 0, 2024-09-11T23:32:37.974 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 2, 2024-09-11T23:32:37.974 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 0, 2024-09-11T23:32:37.975 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 0, 2024-09-11T23:32:37.975 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 2, 2024-09-11T23:32:37.975 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 1, 2024-09-11T23:32:37.975 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.000361333, 2024-09-11T23:32:37.975 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.000000000, 2024-09-11T23:32:37.975 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.000261042, 2024-09-11T23:32:37.975 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:32:37.975 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:37.975 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-11T23:32:37.975 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-11T23:32:37.975 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-11T23:32:37.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:37.976 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:37.976 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-11T23:32:37.976 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-11T23:32:37.976 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-11T23:32:37.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:37.976 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:32:37.976 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:32:37.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:37.976 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-09-11T23:32:37.976 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 0, 2024-09-11T23:32:37.976 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-09-11T23:32:37.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:37.976 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-09-11T23:32:37.977 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:32:37.977 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-09-11T23:32:37.977 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:32:37.977 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:32:37.977 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:32:37.977 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:32:37.977 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:32:37.977 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:32:37.977 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:32:37.977 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:32:37.977 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:32:37.977 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:32:37.977 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:32:37.977 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:32:37.978 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:32:37.978 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-09-11T23:32:37.978 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:32:37.978 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-09-11T23:32:37.978 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:32:37.978 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:32:37.978 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:32:37.978 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:32:37.978 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:32:37.978 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:32:37.978 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:32:37.978 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-09-11T23:32:37.978 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:32:37.978 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:32:37.979 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:32:37.979 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-09-11T23:32:37.979 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-09-11T23:32:37.979 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-09-11T23:32:37.979 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:32:37.979 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:32:37.979 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:32:37.979 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:32:37.979 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 149007, 2024-09-11T23:32:37.979 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 30, 2024-09-11T23:32:37.979 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:32:37.979 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-09-11T23:32:37.979 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-09-11T23:32:37.980 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-09-11T23:32:37.980 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:32:37.980 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-09-11T23:32:37.980 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:32:37.980 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-09-11T23:32:37.980 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 0, 2024-09-11T23:32:37.980 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 0, 2024-09-11T23:32:37.980 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:32:37.980 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:32:37.980 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-09-11T23:32:37.980 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-09-11T23:32:37.980 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 928, 2024-09-11T23:32:37.980 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 2, 2024-09-11T23:32:37.981 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-09-11T23:32:37.981 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-09-11T23:32:37.981 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-09-11T23:32:37.981 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-09-11T23:32:37.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:37.981 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:32:37.981 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-09-11T23:32:37.981 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-09-11T23:32:37.981 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-09-11T23:32:37.981 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 222, 2024-09-11T23:32:37.981 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 1549503, 2024-09-11T23:32:37.981 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:37.981 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 222, 2024-09-11T23:32:37.982 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-09-11T23:32:37.982 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-09-11T23:32:37.982 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 222, 2024-09-11T23:32:37.982 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 1549503, 2024-09-11T23:32:37.982 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-09-11T23:32:37.982 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-11T23:32:37.982 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-11T23:32:37.982 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-11T23:32:37.982 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:37.982 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:37.982 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:32:38.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:37 smithi090 ceph-mon[35130]: pgmap v335: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T23:32:39.020 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:37 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:32:39.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:37 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:32:39.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:37 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:39.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:37 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:39.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:37 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:39.021 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:37 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:39.022 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:37 smithi097 ceph-mon[35562]: pgmap v335: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T23:32:39.022 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:37 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:32:39.022 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:37 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:32:39.022 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:37 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:39.022 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:37 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:39.022 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:37 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:39.022 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:37 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:39.023 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-11T23:32:39.023 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-11T23:32:39.023 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-09-11T23:32:39.023 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-09-11T23:32:39.023 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-11T23:32:39.023 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-11T23:32:39.024 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-11T23:32:39.024 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-09-11T23:32:39.024 DEBUG:tasks.check_counter:Getting stats from j 2024-09-11T23:32:39.024 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-11T23:32:39.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:38 smithi043 ceph-mon[30950]: Adjusting osd_memory_target on smithi043 to 177.0M 2024-09-11T23:32:39.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:38 smithi043 ceph-mon[30950]: Unable to set osd_memory_target on smithi043 to 185687756: error parsing value: Value '185687756' is below minimum 939524096 2024-09-11T23:32:39.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:38 smithi090 ceph-mon[35130]: Adjusting osd_memory_target on smithi043 to 177.0M 2024-09-11T23:32:39.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:39 smithi090 ceph-mon[35130]: Unable to set osd_memory_target on smithi043 to 185687756: error parsing value: Value '185687756' is below minimum 939524096 2024-09-11T23:32:39.296 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:32:39.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:38 smithi097 ceph-mon[35562]: Adjusting osd_memory_target on smithi043 to 177.0M 2024-09-11T23:32:39.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:38 smithi097 ceph-mon[35562]: Unable to set osd_memory_target on smithi043 to 185687756: error parsing value: Value '185687756' is below minimum 939524096 2024-09-11T23:32:39.806 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-11T23:32:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:32:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 4, 2024-09-11T23:32:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 4, 2024-09-11T23:32:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 3072, 2024-09-11T23:32:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 448, 2024-09-11T23:32:39.808 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 4, 2024-09-11T23:32:39.808 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 0, 2024-09-11T23:32:39.808 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.003305384, 2024-09-11T23:32:39.808 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.000047097, 2024-09-11T23:32:39.808 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.002302630, 2024-09-11T23:32:39.808 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:32:39.809 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:39.809 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 3, 2024-09-11T23:32:39.809 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.004215059, 2024-09-11T23:32:39.809 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.001405019 2024-09-11T23:32:39.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:39.809 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:39.809 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-11T23:32:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-11T23:32:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-11T23:32:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-11T23:32:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T23:32:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:32:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 226, 2024-09-11T23:32:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 215, 2024-09-11T23:32:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1582128, 2024-09-11T23:32:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 131904, 2024-09-11T23:32:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 2, 2024-09-11T23:32:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 1, 2024-09-11T23:32:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.041194477, 2024-09-11T23:32:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.015774930, 2024-09-11T23:32:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.039015395, 2024-09-11T23:32:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:32:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 214, 2024-09-11T23:32:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.014741123, 2024-09-11T23:32:39.813 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000068883 2024-09-11T23:32:39.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:39.813 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:39.813 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-11T23:32:39.813 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-11T23:32:39.813 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-11T23:32:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 1582128, 2024-09-11T23:32:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 131904 2024-09-11T23:32:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:32:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 2, 2024-09-11T23:32:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 164, 2024-09-11T23:32:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 320, 2024-09-11T23:32:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 186192, 2024-09-11T23:32:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 5, 2024-09-11T23:32:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 3, 2024-09-11T23:32:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.014010610, 2024-09-11T23:32:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.008981267, 2024-09-11T23:32:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.002508734, 2024-09-11T23:32:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:32:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 161, 2024-09-11T23:32:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.013947398, 2024-09-11T23:32:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000086629 2024-09-11T23:32:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-11T23:32:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-11T23:32:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-11T23:32:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T23:32:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 186192 2024-09-11T23:32:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-09-11T23:32:39.818 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 0, 2024-09-11T23:32:39.818 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-09-11T23:32:39.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:39.818 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-09-11T23:32:39.819 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:32:39.819 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-09-11T23:32:39.819 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:32:39.819 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:32:39.819 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:32:39.820 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:32:39.820 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:32:39.820 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:32:39.820 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:32:39.820 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:32:39.821 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:32:39.821 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:32:39.821 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:32:39.821 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:32:39.821 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:32:39.822 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-09-11T23:32:39.822 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:32:39.822 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-09-11T23:32:39.822 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:32:39.822 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:32:39.822 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:32:39.822 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:32:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:32:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:32:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:32:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-09-11T23:32:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:32:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:32:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:32:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-09-11T23:32:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-09-11T23:32:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-09-11T23:32:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:32:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:32:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:32:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:32:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 149017, 2024-09-11T23:32:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 31, 2024-09-11T23:32:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:32:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-09-11T23:32:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-09-11T23:32:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-09-11T23:32:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:32:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-09-11T23:32:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:32:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-09-11T23:32:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 0, 2024-09-11T23:32:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 0, 2024-09-11T23:32:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:32:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:32:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-09-11T23:32:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-09-11T23:32:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 928, 2024-09-11T23:32:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 2, 2024-09-11T23:32:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-09-11T23:32:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-09-11T23:32:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-09-11T23:32:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-09-11T23:32:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-11T23:32:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:32:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-09-11T23:32:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-09-11T23:32:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-09-11T23:32:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 228, 2024-09-11T23:32:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 1550283, 2024-09-11T23:32:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 228, 2024-09-11T23:32:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-09-11T23:32:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-09-11T23:32:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 228, 2024-09-11T23:32:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 1550283, 2024-09-11T23:32:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-09-11T23:32:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-09-11T23:32:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-09-11T23:32:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-09-11T23:32:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-11T23:32:39.828 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-11T23:32:40.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:39 smithi043 ceph-mon[30950]: pgmap v336: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T23:32:40.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:39 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:32:40.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:39 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:32:40.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:39 smithi090 ceph-mon[35130]: pgmap v336: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T23:32:40.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:39 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:32:40.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:39 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:32:40.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:39 smithi097 ceph-mon[35562]: pgmap v336: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T23:32:40.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:39 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:32:40.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:39 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:32:40.633 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-11T23:32:40.634 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-11T23:32:40.634 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-11T23:32:40.634 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-11T23:32:40.634 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-11T23:32:40.634 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-11T23:32:40.634 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-11T23:32:40.634 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-11T23:32:40.634 DEBUG:tasks.check_counter:Getting stats from b 2024-09-11T23:32:40.634 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-11T23:32:41.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:40 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:41.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:40 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:41.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:40 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:41.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:40 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:41.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:40 smithi090 ceph-mon[35130]: Adjusting osd_memory_target on smithi097 to 176.0M 2024-09-11T23:32:41.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:40 smithi090 ceph-mon[35130]: Unable to set osd_memory_target on smithi097 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-11T23:32:41.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:40 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:41.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:40 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:41.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:40 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:41.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:40 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:41.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:40 smithi097 ceph-mon[35562]: Adjusting osd_memory_target on smithi097 to 176.0M 2024-09-11T23:32:41.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:40 smithi097 ceph-mon[35562]: Unable to set osd_memory_target on smithi097 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-11T23:32:41.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:40 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:41.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:40 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:41.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:40 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:41.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:40 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:41.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:40 smithi043 ceph-mon[30950]: Adjusting osd_memory_target on smithi097 to 176.0M 2024-09-11T23:32:41.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:40 smithi043 ceph-mon[30950]: Unable to set osd_memory_target on smithi097 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-11T23:32:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:32:42.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:41 smithi090 ceph-mon[35130]: pgmap v337: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2024-09-11T23:32:42.300 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:41 smithi097 ceph-mon[35562]: pgmap v337: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2024-09-11T23:32:42.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:41 smithi043 ceph-mon[30950]: pgmap v337: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s 2024-09-11T23:32:44.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:43 smithi090 ceph-mon[35130]: pgmap v338: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-11T23:32:44.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:43 smithi097 ceph-mon[35562]: pgmap v338: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-11T23:32:44.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:43 smithi043 ceph-mon[30950]: pgmap v338: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-11T23:32:46.117 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:45 smithi090 ceph-mon[35130]: pgmap v339: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-09-11T23:32:46.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:45 smithi097 ceph-mon[35562]: pgmap v339: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-09-11T23:32:46.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:45 smithi043 ceph-mon[30950]: pgmap v339: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-09-11T23:32:47.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:47 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:32:47.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:47 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:32:47.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:47 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:32:47.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:47 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:32:47.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:47 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:32:47.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:47 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:32:47.870 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-11T23:32:47.870 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:32:47.870 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 5113, 2024-09-11T23:32:47.870 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 4324, 2024-09-11T23:32:47.870 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 1627323, 2024-09-11T23:32:47.870 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 38459699, 2024-09-11T23:32:47.870 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 139, 2024-09-11T23:32:47.870 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 6, 2024-09-11T23:32:47.870 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.706132199, 2024-09-11T23:32:47.870 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.205023920, 2024-09-11T23:32:47.871 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.159913617, 2024-09-11T23:32:47.871 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.299864868, 2024-09-11T23:32:47.871 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:47.871 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 4318, 2024-09-11T23:32:47.871 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 1.056950080, 2024-09-11T23:32:47.871 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000244777 2024-09-11T23:32:47.871 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.871 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:47.871 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1338, 2024-09-11T23:32:47.871 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000215388, 2024-09-11T23:32:47.871 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000160 2024-09-11T23:32:47.871 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.872 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 446528, 2024-09-11T23:32:47.872 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 121824 2024-09-11T23:32:47.872 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.872 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:32:47.872 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 6569, 2024-09-11T23:32:47.872 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 8617, 2024-09-11T23:32:47.872 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 3524379, 2024-09-11T23:32:47.872 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 11303135, 2024-09-11T23:32:47.872 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 12, 2024-09-11T23:32:47.872 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 6, 2024-09-11T23:32:47.872 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.545102275, 2024-09-11T23:32:47.873 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.201859121, 2024-09-11T23:32:47.873 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.235522728, 2024-09-11T23:32:47.873 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.112315231, 2024-09-11T23:32:47.873 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:47.873 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 8619, 2024-09-11T23:32:47.873 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 1.338013203, 2024-09-11T23:32:47.873 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000155239 2024-09-11T23:32:47.873 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.873 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:47.873 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 6150, 2024-09-11T23:32:47.873 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.001194828, 2024-09-11T23:32:47.873 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000194 2024-09-11T23:32:47.873 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.874 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 1675744, 2024-09-11T23:32:47.874 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 478032 2024-09-11T23:32:47.874 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.874 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:32:47.874 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 30574, 2024-09-11T23:32:47.874 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 38033, 2024-09-11T23:32:47.874 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 10448271, 2024-09-11T23:32:47.874 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 29723114, 2024-09-11T23:32:47.874 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 34, 2024-09-11T23:32:47.874 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 6, 2024-09-11T23:32:47.874 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 1.876826804, 2024-09-11T23:32:47.874 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.728712559, 2024-09-11T23:32:47.875 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 1.810251809, 2024-09-11T23:32:47.875 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.196105384, 2024-09-11T23:32:47.875 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:47.875 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 38032, 2024-09-11T23:32:47.875 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 10.967295875, 2024-09-11T23:32:47.875 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000288370 2024-09-11T23:32:47.875 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.875 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:47.875 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 50804, 2024-09-11T23:32:47.875 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.014157984, 2024-09-11T23:32:47.875 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000278 2024-09-11T23:32:47.876 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.876 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 47904, 2024-09-11T23:32:47.876 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 132960 2024-09-11T23:32:47.876 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.876 INFO:teuthology.orchestra.run.smithi090.stdout: "cct": { 2024-09-11T23:32:47.876 INFO:teuthology.orchestra.run.smithi090.stdout: "total_workers": 1, 2024-09-11T23:32:47.876 INFO:teuthology.orchestra.run.smithi090.stdout: "unhealthy_workers": 0 2024-09-11T23:32:47.876 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.876 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-MDSRank": { 2024-09-11T23:32:47.876 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-09-11T23:32:47.876 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-09-11T23:32:47.876 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 7894, 2024-09-11T23:32:47.877 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 10.909603831, 2024-09-11T23:32:47.877 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.001382012 2024-09-11T23:32:47.877 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:47.877 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.877 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-PurgeQueue": { 2024-09-11T23:32:47.877 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-09-11T23:32:47.877 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-09-11T23:32:47.877 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 4519, 2024-09-11T23:32:47.877 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.228496026, 2024-09-11T23:32:47.877 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000050563 2024-09-11T23:32:47.877 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:47.877 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.877 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-09-11T23:32:47.878 INFO:teuthology.orchestra.run.smithi090.stdout: "request": 3068, 2024-09-11T23:32:47.878 INFO:teuthology.orchestra.run.smithi090.stdout: "reply": 3065, 2024-09-11T23:32:47.878 INFO:teuthology.orchestra.run.smithi090.stdout: "reply_latency": { 2024-09-11T23:32:47.878 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 3065, 2024-09-11T23:32:47.878 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 71.918690369, 2024-09-11T23:32:47.878 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.023464499 2024-09-11T23:32:47.878 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.878 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_reply": 0, 2024-09-11T23:32:47.878 INFO:teuthology.orchestra.run.smithi090.stdout: "forward": 3, 2024-09-11T23:32:47.878 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_complete": 4, 2024-09-11T23:32:47.878 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_keys": 1, 2024-09-11T23:32:47.878 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_commit": 577, 2024-09-11T23:32:47.878 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_split": 3, 2024-09-11T23:32:47.878 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_merge": 0, 2024-09-11T23:32:47.879 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes": 1385, 2024-09-11T23:32:47.879 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_top": 902, 2024-09-11T23:32:47.879 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_bottom": 472, 2024-09-11T23:32:47.879 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pin_tail": 11, 2024-09-11T23:32:47.879 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pinned": 96, 2024-09-11T23:32:47.879 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_expired": 104, 2024-09-11T23:32:47.879 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_with_caps": 2, 2024-09-11T23:32:47.879 INFO:teuthology.orchestra.run.smithi090.stdout: "caps": 3, 2024-09-11T23:32:47.879 INFO:teuthology.orchestra.run.smithi090.stdout: "subtrees": 5, 2024-09-11T23:32:47.879 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse": 4180, 2024-09-11T23:32:47.879 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_hit": 3841, 2024-09-11T23:32:47.879 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_forward": 0, 2024-09-11T23:32:47.879 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_discover": 27, 2024-09-11T23:32:47.880 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:32:47.880 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_remote_ino": 0, 2024-09-11T23:32:47.880 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_lock": 26, 2024-09-11T23:32:47.880 INFO:teuthology.orchestra.run.smithi090.stdout: "load_cent": 0, 2024-09-11T23:32:47.880 INFO:teuthology.orchestra.run.smithi090.stdout: "q": 0, 2024-09-11T23:32:47.880 INFO:teuthology.orchestra.run.smithi090.stdout: "exported": 4, 2024-09-11T23:32:47.880 INFO:teuthology.orchestra.run.smithi090.stdout: "exported_inodes": 27, 2024-09-11T23:32:47.880 INFO:teuthology.orchestra.run.smithi090.stdout: "imported": 4, 2024-09-11T23:32:47.880 INFO:teuthology.orchestra.run.smithi090.stdout: "imported_inodes": 66, 2024-09-11T23:32:47.880 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_dir_fetch": 0, 2024-09-11T23:32:47.880 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_backtrace_fetch": 1175, 2024-09-11T23:32:47.880 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_peer_discover": 0, 2024-09-11T23:32:47.880 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rfiles": 4, 2024-09-11T23:32:47.881 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rbytes": 234, 2024-09-11T23:32:47.881 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rsnaps": 6, 2024-09-11T23:32:47.881 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_fetch": 1175, 2024-09-11T23:32:47.881 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_set_tag": 0, 2024-09-11T23:32:47.881 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:32:47.881 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:32:47.881 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_inodes": 294, 2024-09-11T23:32:47.881 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_base_inodes": 8, 2024-09-11T23:32:47.881 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dirfrag_rstats": 286, 2024-09-11T23:32:47.881 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_file_inodes": 881, 2024-09-11T23:32:47.881 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_inode_file_caps": 12, 2024-09-11T23:32:47.882 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_revoke": 1368, 2024-09-11T23:32:47.882 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_grant": 5251, 2024-09-11T23:32:47.882 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_trunc": 6, 2024-09-11T23:32:47.882 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:32:47.882 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:32:47.882 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps": 20849, 2024-09-11T23:32:47.882 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps_dirty": 19130, 2024-09-11T23:32:47.882 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_cap_release": 163, 2024-09-11T23:32:47.882 INFO:teuthology.orchestra.run.smithi090.stdout: "process_request_cap_release": 1441 2024-09-11T23:32:47.882 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.882 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache": { 2024-09-11T23:32:47.882 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update": 179, 2024-09-11T23:32:47.882 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update_receipt": 21, 2024-09-11T23:32:47.883 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_try_discover": 15, 2024-09-11T23:32:47.883 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_send_discover": 30, 2024-09-11T23:32:47.883 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_handle_discover": 353, 2024-09-11T23:32:47.883 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays": 79, 2024-09-11T23:32:47.883 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_delayed": 0, 2024-09-11T23:32:47.883 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:32:47.883 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_created": 890, 2024-09-11T23:32:47.883 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_enqueued": 750, 2024-09-11T23:32:47.883 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_reintegrated": 61, 2024-09-11T23:32:47.883 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_migrated": 0, 2024-09-11T23:32:47.883 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_processing": 0, 2024-09-11T23:32:47.883 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:32:47.883 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:32:47.884 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_started": 0, 2024-09-11T23:32:47.884 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_completed": 0, 2024-09-11T23:32:47.884 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_quiesce_path": 5, 2024-09-11T23:32:47.884 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_quiesce_inode": 1297, 2024-09-11T23:32:47.884 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_enqueue_scrub": 8, 2024-09-11T23:32:47.884 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_exportdir": 4, 2024-09-11T23:32:47.884 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_flush": 0, 2024-09-11T23:32:47.884 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragmentdir": 3, 2024-09-11T23:32:47.884 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragstats": 0, 2024-09-11T23:32:47.884 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_inodestats": 0 2024-09-11T23:32:47.884 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.884 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log": { 2024-09-11T23:32:47.884 INFO:teuthology.orchestra.run.smithi090.stdout: "evlrg": 0, 2024-09-11T23:32:47.884 INFO:teuthology.orchestra.run.smithi090.stdout: "evadd": 23167, 2024-09-11T23:32:47.885 INFO:teuthology.orchestra.run.smithi090.stdout: "evex": 23127, 2024-09-11T23:32:47.885 INFO:teuthology.orchestra.run.smithi090.stdout: "evtrm": 23127, 2024-09-11T23:32:47.885 INFO:teuthology.orchestra.run.smithi090.stdout: "ev": 40, 2024-09-11T23:32:47.885 INFO:teuthology.orchestra.run.smithi090.stdout: "evexg": 0, 2024-09-11T23:32:47.885 INFO:teuthology.orchestra.run.smithi090.stdout: "evexd": 109, 2024-09-11T23:32:47.885 INFO:teuthology.orchestra.run.smithi090.stdout: "segadd": 54, 2024-09-11T23:32:47.885 INFO:teuthology.orchestra.run.smithi090.stdout: "segex": 53, 2024-09-11T23:32:47.885 INFO:teuthology.orchestra.run.smithi090.stdout: "segtrm": 53, 2024-09-11T23:32:47.885 INFO:teuthology.orchestra.run.smithi090.stdout: "seg": 1, 2024-09-11T23:32:47.885 INFO:teuthology.orchestra.run.smithi090.stdout: "segmjr": 1, 2024-09-11T23:32:47.885 INFO:teuthology.orchestra.run.smithi090.stdout: "segexg": 0, 2024-09-11T23:32:47.885 INFO:teuthology.orchestra.run.smithi090.stdout: "segexd": 2, 2024-09-11T23:32:47.885 INFO:teuthology.orchestra.run.smithi090.stdout: "expos": 67433080, 2024-09-11T23:32:47.886 INFO:teuthology.orchestra.run.smithi090.stdout: "wrpos": 67528635, 2024-09-11T23:32:47.886 INFO:teuthology.orchestra.run.smithi090.stdout: "rdpos": 0, 2024-09-11T23:32:47.886 INFO:teuthology.orchestra.run.smithi090.stdout: "jlat": { 2024-09-11T23:32:47.886 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 3880, 2024-09-11T23:32:47.886 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 1547.539630090, 2024-09-11T23:32:47.886 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.398850420 2024-09-11T23:32:47.886 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.886 INFO:teuthology.orchestra.run.smithi090.stdout: "replayed": 0 2024-09-11T23:32:47.886 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.886 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_mem": { 2024-09-11T23:32:47.886 INFO:teuthology.orchestra.run.smithi090.stdout: "ino": 888, 2024-09-11T23:32:47.886 INFO:teuthology.orchestra.run.smithi090.stdout: "ino+": 1697, 2024-09-11T23:32:47.886 INFO:teuthology.orchestra.run.smithi090.stdout: "ino-": 809, 2024-09-11T23:32:47.887 INFO:teuthology.orchestra.run.smithi090.stdout: "dir": 122, 2024-09-11T23:32:47.887 INFO:teuthology.orchestra.run.smithi090.stdout: "dir+": 150, 2024-09-11T23:32:47.887 INFO:teuthology.orchestra.run.smithi090.stdout: "dir-": 28, 2024-09-11T23:32:47.887 INFO:teuthology.orchestra.run.smithi090.stdout: "dn": 1385, 2024-09-11T23:32:47.887 INFO:teuthology.orchestra.run.smithi090.stdout: "dn+": 2525, 2024-09-11T23:32:47.887 INFO:teuthology.orchestra.run.smithi090.stdout: "dn-": 1140, 2024-09-11T23:32:47.887 INFO:teuthology.orchestra.run.smithi090.stdout: "cap": 3, 2024-09-11T23:32:47.887 INFO:teuthology.orchestra.run.smithi090.stdout: "cap+": 1454, 2024-09-11T23:32:47.887 INFO:teuthology.orchestra.run.smithi090.stdout: "cap-": 1451, 2024-09-11T23:32:47.887 INFO:teuthology.orchestra.run.smithi090.stdout: "rss": 81828, 2024-09-11T23:32:47.887 INFO:teuthology.orchestra.run.smithi090.stdout: "heap": 231712 2024-09-11T23:32:47.887 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.887 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_server": { 2024-09-11T23:32:47.887 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_client_request": 4100, 2024-09-11T23:32:47.888 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_server_request": 103, 2024-09-11T23:32:47.888 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_request": 3068, 2024-09-11T23:32:47.888 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_session": 39, 2024-09-11T23:32:47.888 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_peer_request": 171, 2024-09-11T23:32:47.888 INFO:teuthology.orchestra.run.smithi090.stdout: "req_create_latency": { 2024-09-11T23:32:47.888 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 745, 2024-09-11T23:32:47.888 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.763649470, 2024-09-11T23:32:47.888 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.001025032 2024-09-11T23:32:47.888 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.888 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getattr_latency": { 2024-09-11T23:32:47.888 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 136, 2024-09-11T23:32:47.888 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.033690002, 2024-09-11T23:32:47.888 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000247720 2024-09-11T23:32:47.889 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.889 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getfilelock_latency": { 2024-09-11T23:32:47.889 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:47.889 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:47.889 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:47.889 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.889 INFO:teuthology.orchestra.run.smithi090.stdout: "req_link_latency": { 2024-09-11T23:32:47.889 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 61, 2024-09-11T23:32:47.889 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.077288765, 2024-09-11T23:32:47.889 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.001267028 2024-09-11T23:32:47.889 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.889 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookup_latency": { 2024-09-11T23:32:47.889 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 424, 2024-09-11T23:32:47.890 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 3.091890881, 2024-09-11T23:32:47.890 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.007292195 2024-09-11T23:32:47.890 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.890 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookuphash_latency": { 2024-09-11T23:32:47.890 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:47.890 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:47.890 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:47.890 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.890 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupino_latency": { 2024-09-11T23:32:47.890 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1, 2024-09-11T23:32:47.890 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000180289, 2024-09-11T23:32:47.890 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000180289 2024-09-11T23:32:47.890 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.891 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupname_latency": { 2024-09-11T23:32:47.891 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:47.891 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:47.891 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:47.891 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.891 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupparent_latency": { 2024-09-11T23:32:47.891 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:47.891 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:47.891 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:47.891 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.891 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupsnap_latency": { 2024-09-11T23:32:47.891 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:47.891 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:47.892 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:47.892 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.892 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lssnap_latency": { 2024-09-11T23:32:47.892 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 10, 2024-09-11T23:32:47.892 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.001495666, 2024-09-11T23:32:47.892 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000149566 2024-09-11T23:32:47.892 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.892 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mkdir_latency": { 2024-09-11T23:32:47.892 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 103, 2024-09-11T23:32:47.892 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.120182379, 2024-09-11T23:32:47.892 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.001166819 2024-09-11T23:32:47.893 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.893 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mknod_latency": { 2024-09-11T23:32:47.893 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:47.893 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:47.893 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:47.893 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.893 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mksnap_latency": { 2024-09-11T23:32:47.893 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 10, 2024-09-11T23:32:47.893 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 14.329704080, 2024-09-11T23:32:47.893 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 1.432970408 2024-09-11T23:32:47.893 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.893 INFO:teuthology.orchestra.run.smithi090.stdout: "req_open_latency": { 2024-09-11T23:32:47.893 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 300, 2024-09-11T23:32:47.894 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 1.796108450, 2024-09-11T23:32:47.894 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.005987028 2024-09-11T23:32:47.894 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.894 INFO:teuthology.orchestra.run.smithi090.stdout: "req_readdir_latency": { 2024-09-11T23:32:47.894 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 271, 2024-09-11T23:32:47.894 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.803351432, 2024-09-11T23:32:47.894 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.002964396 2024-09-11T23:32:47.894 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.894 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rename_latency": { 2024-09-11T23:32:47.894 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 93, 2024-09-11T23:32:47.894 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 4.198481806, 2024-09-11T23:32:47.894 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.045144965 2024-09-11T23:32:47.895 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.895 INFO:teuthology.orchestra.run.smithi090.stdout: "req_renamesnap_latency": { 2024-09-11T23:32:47.895 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:47.895 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:47.895 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:47.895 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.895 INFO:teuthology.orchestra.run.smithi090.stdout: "req_snapdiff_latency": { 2024-09-11T23:32:47.895 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:47.895 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:47.895 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:47.895 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.895 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmdir_latency": { 2024-09-11T23:32:47.895 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 95, 2024-09-11T23:32:47.896 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 44.088665667, 2024-09-11T23:32:47.896 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.464091217 2024-09-11T23:32:47.896 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.896 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmsnap_latency": { 2024-09-11T23:32:47.896 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 4, 2024-09-11T23:32:47.896 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 1.422314586, 2024-09-11T23:32:47.896 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.355578646 2024-09-11T23:32:47.896 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.896 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmxattr_latency": { 2024-09-11T23:32:47.896 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:47.896 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:47.896 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:47.896 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.897 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setattr_latency": { 2024-09-11T23:32:47.897 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 7, 2024-09-11T23:32:47.897 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.031377576, 2024-09-11T23:32:47.897 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.004482510 2024-09-11T23:32:47.897 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.897 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setdirlayout_latency": { 2024-09-11T23:32:47.897 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:47.900 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:47.901 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:47.901 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.901 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setfilelock_latency": { 2024-09-11T23:32:47.901 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 4, 2024-09-11T23:32:47.901 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000812152, 2024-09-11T23:32:47.901 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000203038 2024-09-11T23:32:47.901 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.901 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setlayout_latency": { 2024-09-11T23:32:47.901 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:47.901 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:47.901 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:47.901 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.902 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setxattr_latency": { 2024-09-11T23:32:47.902 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 8, 2024-09-11T23:32:47.902 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.020683458, 2024-09-11T23:32:47.902 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.002585432 2024-09-11T23:32:47.902 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.902 INFO:teuthology.orchestra.run.smithi090.stdout: "req_symlink_latency": { 2024-09-11T23:32:47.902 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 13, 2024-09-11T23:32:47.902 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.019719962, 2024-09-11T23:32:47.902 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.001516920 2024-09-11T23:32:47.902 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.902 INFO:teuthology.orchestra.run.smithi090.stdout: "req_unlink_latency": { 2024-09-11T23:32:47.902 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 780, 2024-09-11T23:32:47.902 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 1.119093748, 2024-09-11T23:32:47.903 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.001434735 2024-09-11T23:32:47.903 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.903 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:32:47.903 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:32:47.903 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getvxattr_latency": { 2024-09-11T23:32:47.903 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:47.903 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:47.903 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:47.903 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:47.903 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.903 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_sessions": { 2024-09-11T23:32:47.903 INFO:teuthology.orchestra.run.smithi090.stdout: "session_count": 2, 2024-09-11T23:32:47.903 INFO:teuthology.orchestra.run.smithi090.stdout: "session_add": 6, 2024-09-11T23:32:47.904 INFO:teuthology.orchestra.run.smithi090.stdout: "session_remove": 4, 2024-09-11T23:32:47.904 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_open": 2, 2024-09-11T23:32:47.904 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_stale": 0, 2024-09-11T23:32:47.904 INFO:teuthology.orchestra.run.smithi090.stdout: "total_load": 144, 2024-09-11T23:32:47.904 INFO:teuthology.orchestra.run.smithi090.stdout: "average_load": 72, 2024-09-11T23:32:47.904 INFO:teuthology.orchestra.run.smithi090.stdout: "avg_session_uptime": 620, 2024-09-11T23:32:47.904 INFO:teuthology.orchestra.run.smithi090.stdout: "mdthresh_evicted": 0 2024-09-11T23:32:47.904 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.904 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool": { 2024-09-11T23:32:47.904 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:32:47.904 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_items": 0, 2024-09-11T23:32:47.904 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:32:47.904 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:32:47.904 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:32:47.905 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:32:47.905 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:32:47.905 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:32:47.905 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:32:47.905 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:32:47.905 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:32:47.905 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:32:47.905 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:32:47.905 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:32:47.905 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:32:47.905 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_items": 0, 2024-09-11T23:32:47.905 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:32:47.905 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_items": 0, 2024-09-11T23:32:47.906 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:32:47.906 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:32:47.906 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:32:47.906 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:32:47.906 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:32:47.906 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:32:47.906 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:32:47.906 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_items": 0, 2024-09-11T23:32:47.906 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:32:47.906 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:32:47.906 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:32:47.906 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_items": 0, 2024-09-11T23:32:47.907 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_bytes": 0, 2024-09-11T23:32:47.907 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_items": 0, 2024-09-11T23:32:47.907 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:32:47.907 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:32:47.907 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:32:47.907 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:32:47.907 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_bytes": 958781, 2024-09-11T23:32:47.907 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_items": 662, 2024-09-11T23:32:47.907 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:32:47.907 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_items": 0, 2024-09-11T23:32:47.907 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bytes": 0, 2024-09-11T23:32:47.907 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_items": 0, 2024-09-11T23:32:47.907 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:32:47.908 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_items": 0, 2024-09-11T23:32:47.908 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:32:47.908 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_items": 0, 2024-09-11T23:32:47.908 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_bytes": 8320, 2024-09-11T23:32:47.908 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_items": 183, 2024-09-11T23:32:47.908 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:32:47.908 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:32:47.908 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_bytes": 0, 2024-09-11T23:32:47.908 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_items": 0, 2024-09-11T23:32:47.908 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_bytes": 3323535, 2024-09-11T23:32:47.908 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_items": 45102, 2024-09-11T23:32:47.908 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_bytes": 0, 2024-09-11T23:32:47.908 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_items": 0, 2024-09-11T23:32:47.909 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_bytes": 0, 2024-09-11T23:32:47.909 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_items": 0 2024-09-11T23:32:47.909 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.909 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter": { 2024-09-11T23:32:47.909 INFO:teuthology.orchestra.run.smithi090.stdout: "op_active": 0, 2024-09-11T23:32:47.909 INFO:teuthology.orchestra.run.smithi090.stdout: "op_laggy": 0, 2024-09-11T23:32:47.909 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send": 10974, 2024-09-11T23:32:47.909 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send_bytes": 66235507, 2024-09-11T23:32:47.909 INFO:teuthology.orchestra.run.smithi090.stdout: "op_resend": 37, 2024-09-11T23:32:47.909 INFO:teuthology.orchestra.run.smithi090.stdout: "op_reply": 10974, 2024-09-11T23:32:47.909 INFO:teuthology.orchestra.run.smithi090.stdout: "op_latency": { 2024-09-11T23:32:47.909 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 10974, 2024-09-11T23:32:47.909 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 2753.368723160, 2024-09-11T23:32:47.909 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.250899282 2024-09-11T23:32:47.910 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.910 INFO:teuthology.orchestra.run.smithi090.stdout: "op_inflight": 0, 2024-09-11T23:32:47.910 INFO:teuthology.orchestra.run.smithi090.stdout: "oplen_avg": { 2024-09-11T23:32:47.910 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 10974, 2024-09-11T23:32:47.910 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 13366 2024-09-11T23:32:47.910 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.910 INFO:teuthology.orchestra.run.smithi090.stdout: "op": 10974, 2024-09-11T23:32:47.910 INFO:teuthology.orchestra.run.smithi090.stdout: "op_r": 1222, 2024-09-11T23:32:47.910 INFO:teuthology.orchestra.run.smithi090.stdout: "op_w": 9752, 2024-09-11T23:32:47.910 INFO:teuthology.orchestra.run.smithi090.stdout: "op_rmw": 0, 2024-09-11T23:32:47.911 INFO:teuthology.orchestra.run.smithi090.stdout: "op_pg": 0, 2024-09-11T23:32:47.911 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_stat": 457, 2024-09-11T23:32:47.911 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_create": 501, 2024-09-11T23:32:47.911 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_read": 34, 2024-09-11T23:32:47.911 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_write": 3906, 2024-09-11T23:32:47.911 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writefull": 135, 2024-09-11T23:32:47.911 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writesame": 0, 2024-09-11T23:32:47.911 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_append": 0, 2024-09-11T23:32:47.911 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_zero": 0, 2024-09-11T23:32:47.911 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_truncate": 1, 2024-09-11T23:32:47.911 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_delete": 4392, 2024-09-11T23:32:47.911 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_mapext": 0, 2024-09-11T23:32:47.912 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_sparse_read": 0, 2024-09-11T23:32:47.912 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_clonerange": 0, 2024-09-11T23:32:47.912 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_getxattr": 1188, 2024-09-11T23:32:47.912 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_setxattr": 1003, 2024-09-11T23:32:47.912 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:32:47.912 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_rmxattr": 0, 2024-09-11T23:32:47.912 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:32:47.912 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_call": 0, 2024-09-11T23:32:47.912 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_watch": 0, 2024-09-11T23:32:47.912 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_notify": 0, 2024-09-11T23:32:47.912 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:32:47.912 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls": 0, 2024-09-11T23:32:47.913 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:32:47.913 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_other": 6, 2024-09-11T23:32:47.913 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_active": 0, 2024-09-11T23:32:47.913 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_send": 0, 2024-09-11T23:32:47.913 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_resend": 0, 2024-09-11T23:32:47.913 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_ping": 0, 2024-09-11T23:32:47.913 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_active": 0, 2024-09-11T23:32:47.913 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_send": 0, 2024-09-11T23:32:47.913 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_resend": 0, 2024-09-11T23:32:47.913 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_active": 0, 2024-09-11T23:32:47.913 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_send": 0, 2024-09-11T23:32:47.913 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_resend": 0, 2024-09-11T23:32:47.914 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_active": 0, 2024-09-11T23:32:47.914 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_send": 0, 2024-09-11T23:32:47.914 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_resend": 0, 2024-09-11T23:32:47.914 INFO:teuthology.orchestra.run.smithi090.stdout: "command_active": 0, 2024-09-11T23:32:47.914 INFO:teuthology.orchestra.run.smithi090.stdout: "command_send": 0, 2024-09-11T23:32:47.914 INFO:teuthology.orchestra.run.smithi090.stdout: "command_resend": 0, 2024-09-11T23:32:47.914 INFO:teuthology.orchestra.run.smithi090.stdout: "map_epoch": 94, 2024-09-11T23:32:47.914 INFO:teuthology.orchestra.run.smithi090.stdout: "map_full": 0, 2024-09-11T23:32:47.914 INFO:teuthology.orchestra.run.smithi090.stdout: "map_inc": 12, 2024-09-11T23:32:47.915 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_sessions": 12, 2024-09-11T23:32:47.915 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_open": 12, 2024-09-11T23:32:47.915 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_close": 0, 2024-09-11T23:32:47.915 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_laggy": 0, 2024-09-11T23:32:47.915 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_wr": 1421, 2024-09-11T23:32:47.915 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_rd": 10, 2024-09-11T23:32:47.915 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_del": 312 2024-09-11T23:32:47.915 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.915 INFO:teuthology.orchestra.run.smithi090.stdout: "oft": { 2024-09-11T23:32:47.915 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_objs": 1, 2024-09-11T23:32:47.915 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:32:47.915 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_updates": 0, 2024-09-11T23:32:47.915 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_removes": 0 2024-09-11T23:32:47.916 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.916 INFO:teuthology.orchestra.run.smithi090.stdout: "osd-slow-ops": { 2024-09-11T23:32:47.916 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_ops_count": 1305 2024-09-11T23:32:47.916 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.916 INFO:teuthology.orchestra.run.smithi090.stdout: "purge_queue": { 2024-09-11T23:32:47.916 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops": 0, 2024-09-11T23:32:47.916 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops_high_water": 51, 2024-09-11T23:32:47.916 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing": 0, 2024-09-11T23:32:47.916 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_high_water": 10, 2024-09-11T23:32:47.916 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed_ops": 3848, 2024-09-11T23:32:47.916 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed": 750, 2024-09-11T23:32:47.916 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_item_in_journal": 0 2024-09-11T23:32:47.916 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.917 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:32:47.917 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-11T23:32:47.917 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-09-11T23:32:47.917 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-11T23:32:47.917 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 42252, 2024-09-11T23:32:47.917 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 11727152, 2024-09-11T23:32:47.917 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:47.917 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 42252, 2024-09-11T23:32:47.917 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-09-11T23:32:47.917 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-09-11T23:32:47.917 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 42252, 2024-09-11T23:32:47.917 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 11727152, 2024-09-11T23:32:47.917 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-11T23:32:47.918 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:47.918 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:47.918 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:47.918 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:47.918 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.918 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_bytes": { 2024-09-11T23:32:47.918 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-11T23:32:47.918 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-09-11T23:32:47.918 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-11T23:32:47.918 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-09-11T23:32:47.918 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-09-11T23:32:47.918 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:47.918 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-09-11T23:32:47.918 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 10974, 2024-09-11T23:32:47.919 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 66323517, 2024-09-11T23:32:47.919 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 10974, 2024-09-11T23:32:47.919 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 66323517, 2024-09-11T23:32:47.919 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-11T23:32:47.919 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:47.919 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:47.919 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:47.919 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:47.919 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.919 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_ops": { 2024-09-11T23:32:47.919 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-11T23:32:47.920 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 1024, 2024-09-11T23:32:47.920 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-11T23:32:47.920 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-09-11T23:32:47.920 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-09-11T23:32:47.920 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:47.920 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-09-11T23:32:47.920 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 10974, 2024-09-11T23:32:47.920 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 10974, 2024-09-11T23:32:47.920 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 10974, 2024-09-11T23:32:47.920 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 10974, 2024-09-11T23:32:47.920 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-11T23:32:47.920 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:47.920 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:47.921 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:47.921 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:47.921 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.921 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:32:47.921 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-11T23:32:47.921 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-09-11T23:32:47.921 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-11T23:32:47.921 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 750, 2024-09-11T23:32:47.921 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 88078, 2024-09-11T23:32:47.921 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:47.922 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 750, 2024-09-11T23:32:47.922 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-09-11T23:32:47.922 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-09-11T23:32:47.922 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 26, 2024-09-11T23:32:47.922 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 88078, 2024-09-11T23:32:47.922 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-11T23:32:47.922 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:47.922 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:47.922 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:47.922 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:47.922 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:47.922 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle-0x55b6eebe8500": { 2024-09-11T23:32:47.923 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-11T23:32:47.923 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-09-11T23:32:47.923 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-11T23:32:47.923 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 23167, 2024-09-11T23:32:47.923 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 63334331, 2024-09-11T23:32:47.923 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:47.923 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 23167, 2024-09-11T23:32:47.923 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-09-11T23:32:47.923 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-09-11T23:32:47.923 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 3880, 2024-09-11T23:32:47.924 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 63334331, 2024-09-11T23:32:47.924 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-11T23:32:47.924 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:47.924 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:47.924 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:47.924 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:47.924 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:47.924 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-11T23:32:48.316 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:48 smithi090 ceph-mon[35130]: pgmap v340: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-11T23:32:48.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:48 smithi043 ceph-mon[30950]: pgmap v340: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-11T23:32:48.452 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:48 smithi097 ceph-mon[35562]: pgmap v340: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-11T23:32:48.886 INFO:tasks.check_counter:Daemon mds.b mds.exported=4 2024-09-11T23:32:48.886 INFO:tasks.check_counter:Daemon mds.b mds.imported=4 2024-09-11T23:32:48.886 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=179 2024-09-11T23:32:48.886 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=21 2024-09-11T23:32:48.886 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-11T23:32:48.886 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-09-11T23:32:48.886 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-09-11T23:32:48.887 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=3 2024-09-11T23:32:48.887 DEBUG:tasks.check_counter:Getting stats from e 2024-09-11T23:32:48.887 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-11T23:32:49.164 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:32:49.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:49 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:49.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:49 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:49.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:49 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:49.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:49 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:49.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:49 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:49.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:49 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:32:49.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:49 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:32:49.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:49 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:49.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:49 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:49.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:49 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:49.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:49 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:49.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:49 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:49.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:49 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:32:49.361 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:49 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:32:49.472 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:49 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:49.472 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:49 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:49.473 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:49 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:49.473 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:49 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:32:49.473 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:49 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:32:49.473 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:49 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:32:49.473 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:49 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' 2024-09-11T23:32:49.810 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-11T23:32:49.810 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:32:49.810 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 633, 2024-09-11T23:32:49.810 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 629, 2024-09-11T23:32:49.810 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 4570246, 2024-09-11T23:32:49.810 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 232099, 2024-09-11T23:32:49.810 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 6, 2024-09-11T23:32:49.811 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 2, 2024-09-11T23:32:49.811 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.071244235, 2024-09-11T23:32:49.811 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.026464083, 2024-09-11T23:32:49.811 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.027617470, 2024-09-11T23:32:49.811 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.015551285, 2024-09-11T23:32:49.811 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:49.811 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 628, 2024-09-11T23:32:49.811 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.040194297, 2024-09-11T23:32:49.811 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000064003 2024-09-11T23:32:49.811 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.811 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:49.811 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.811 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.812 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.812 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.812 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 219952, 2024-09-11T23:32:49.812 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 75408 2024-09-11T23:32:49.812 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.812 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:32:49.812 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 284, 2024-09-11T23:32:49.812 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 374, 2024-09-11T23:32:49.812 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 4161378, 2024-09-11T23:32:49.812 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 336131, 2024-09-11T23:32:49.812 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 5, 2024-09-11T23:32:49.812 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 4, 2024-09-11T23:32:49.813 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.031857592, 2024-09-11T23:32:49.813 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.011405400, 2024-09-11T23:32:49.813 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.009407657, 2024-09-11T23:32:49.813 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.008477035, 2024-09-11T23:32:49.813 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:49.813 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 372, 2024-09-11T23:32:49.813 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.014902803, 2024-09-11T23:32:49.813 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000040061 2024-09-11T23:32:49.813 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.813 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:49.813 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.814 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.814 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.814 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.814 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T23:32:49.814 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 246080 2024-09-11T23:32:49.814 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.814 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:32:49.814 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 420, 2024-09-11T23:32:49.814 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 420, 2024-09-11T23:32:49.814 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 8918536, 2024-09-11T23:32:49.814 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 132752, 2024-09-11T23:32:49.814 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 5, 2024-09-11T23:32:49.814 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 4, 2024-09-11T23:32:49.815 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.041453963, 2024-09-11T23:32:49.815 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.008964711, 2024-09-11T23:32:49.815 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.018613294, 2024-09-11T23:32:49.815 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.011299455, 2024-09-11T23:32:49.815 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:49.815 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 419, 2024-09-11T23:32:49.815 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.021714134, 2024-09-11T23:32:49.815 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000051823 2024-09-11T23:32:49.815 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.815 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:49.815 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.815 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.815 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.816 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.816 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-11T23:32:49.816 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T23:32:49.816 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.816 INFO:teuthology.orchestra.run.smithi090.stdout: "cct": { 2024-09-11T23:32:49.816 INFO:teuthology.orchestra.run.smithi090.stdout: "total_workers": 1, 2024-09-11T23:32:49.816 INFO:teuthology.orchestra.run.smithi090.stdout: "unhealthy_workers": 0 2024-09-11T23:32:49.816 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.816 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-MDSRank": { 2024-09-11T23:32:49.816 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-09-11T23:32:49.816 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-09-11T23:32:49.816 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 2003, 2024-09-11T23:32:49.816 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.058674178, 2024-09-11T23:32:49.817 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000029293 2024-09-11T23:32:49.817 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:49.817 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.817 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-PurgeQueue": { 2024-09-11T23:32:49.817 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-09-11T23:32:49.817 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-09-11T23:32:49.817 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.817 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.817 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.817 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:49.817 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.817 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-09-11T23:32:49.817 INFO:teuthology.orchestra.run.smithi090.stdout: "request": 0, 2024-09-11T23:32:49.818 INFO:teuthology.orchestra.run.smithi090.stdout: "reply": 0, 2024-09-11T23:32:49.818 INFO:teuthology.orchestra.run.smithi090.stdout: "reply_latency": { 2024-09-11T23:32:49.818 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.818 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.818 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.818 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.818 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_reply": 0, 2024-09-11T23:32:49.818 INFO:teuthology.orchestra.run.smithi090.stdout: "forward": 0, 2024-09-11T23:32:49.818 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_complete": 0, 2024-09-11T23:32:49.818 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_keys": 0, 2024-09-11T23:32:49.818 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_commit": 0, 2024-09-11T23:32:49.818 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_split": 0, 2024-09-11T23:32:49.818 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_merge": 0, 2024-09-11T23:32:49.819 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes": 1519, 2024-09-11T23:32:49.819 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_top": 1002, 2024-09-11T23:32:49.819 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_bottom": 517, 2024-09-11T23:32:49.819 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pin_tail": 0, 2024-09-11T23:32:49.819 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pinned": 88, 2024-09-11T23:32:49.819 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_expired": 0, 2024-09-11T23:32:49.819 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_with_caps": 0, 2024-09-11T23:32:49.819 INFO:teuthology.orchestra.run.smithi090.stdout: "caps": 0, 2024-09-11T23:32:49.819 INFO:teuthology.orchestra.run.smithi090.stdout: "subtrees": 2, 2024-09-11T23:32:49.819 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse": 0, 2024-09-11T23:32:49.819 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_hit": 0, 2024-09-11T23:32:49.819 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_forward": 0, 2024-09-11T23:32:49.819 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_discover": 0, 2024-09-11T23:32:49.820 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:32:49.820 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_remote_ino": 0, 2024-09-11T23:32:49.820 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_lock": 0, 2024-09-11T23:32:49.820 INFO:teuthology.orchestra.run.smithi090.stdout: "load_cent": 0, 2024-09-11T23:32:49.820 INFO:teuthology.orchestra.run.smithi090.stdout: "q": 0, 2024-09-11T23:32:49.820 INFO:teuthology.orchestra.run.smithi090.stdout: "exported": 0, 2024-09-11T23:32:49.820 INFO:teuthology.orchestra.run.smithi090.stdout: "exported_inodes": 0, 2024-09-11T23:32:49.820 INFO:teuthology.orchestra.run.smithi090.stdout: "imported": 0, 2024-09-11T23:32:49.820 INFO:teuthology.orchestra.run.smithi090.stdout: "imported_inodes": 0, 2024-09-11T23:32:49.820 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_dir_fetch": 0, 2024-09-11T23:32:49.820 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:32:49.820 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_peer_discover": 0, 2024-09-11T23:32:49.821 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rfiles": 4, 2024-09-11T23:32:49.821 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rbytes": 234, 2024-09-11T23:32:49.821 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rsnaps": 6, 2024-09-11T23:32:49.821 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:32:49.821 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_set_tag": 0, 2024-09-11T23:32:49.821 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:32:49.821 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:32:49.821 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:32:49.821 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:32:49.821 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:32:49.821 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_file_inodes": 0, 2024-09-11T23:32:49.821 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:32:49.821 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T23:32:49.821 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:32:49.822 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:32:49.822 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:32:49.822 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:32:49.822 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps": 0, 2024-09-11T23:32:49.822 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:32:49.822 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_cap_release": 0, 2024-09-11T23:32:49.822 INFO:teuthology.orchestra.run.smithi090.stdout: "process_request_cap_release": 0 2024-09-11T23:32:49.822 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.822 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache": { 2024-09-11T23:32:49.822 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update": 0, 2024-09-11T23:32:49.822 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update_receipt": 0, 2024-09-11T23:32:49.822 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_try_discover": 0, 2024-09-11T23:32:49.822 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_send_discover": 0, 2024-09-11T23:32:49.823 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_handle_discover": 0, 2024-09-11T23:32:49.823 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays": 0, 2024-09-11T23:32:49.823 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_delayed": 0, 2024-09-11T23:32:49.823 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:32:49.823 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_created": 0, 2024-09-11T23:32:49.823 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_enqueued": 0, 2024-09-11T23:32:49.823 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_reintegrated": 0, 2024-09-11T23:32:49.823 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_migrated": 0, 2024-09-11T23:32:49.823 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_processing": 0, 2024-09-11T23:32:49.823 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:32:49.823 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:32:49.823 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_started": 0, 2024-09-11T23:32:49.823 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_completed": 0, 2024-09-11T23:32:49.824 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_quiesce_path": 0, 2024-09-11T23:32:49.824 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_quiesce_inode": 0, 2024-09-11T23:32:49.824 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:32:49.824 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_exportdir": 0, 2024-09-11T23:32:49.824 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_flush": 0, 2024-09-11T23:32:49.824 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:32:49.824 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragstats": 0, 2024-09-11T23:32:49.824 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_inodestats": 0 2024-09-11T23:32:49.824 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.824 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log": { 2024-09-11T23:32:49.824 INFO:teuthology.orchestra.run.smithi090.stdout: "evlrg": 0, 2024-09-11T23:32:49.824 INFO:teuthology.orchestra.run.smithi090.stdout: "evadd": 0, 2024-09-11T23:32:49.824 INFO:teuthology.orchestra.run.smithi090.stdout: "evex": 0, 2024-09-11T23:32:49.825 INFO:teuthology.orchestra.run.smithi090.stdout: "evtrm": 0, 2024-09-11T23:32:49.825 INFO:teuthology.orchestra.run.smithi090.stdout: "ev": 3604, 2024-09-11T23:32:49.825 INFO:teuthology.orchestra.run.smithi090.stdout: "evexg": 0, 2024-09-11T23:32:49.825 INFO:teuthology.orchestra.run.smithi090.stdout: "evexd": 0, 2024-09-11T23:32:49.825 INFO:teuthology.orchestra.run.smithi090.stdout: "segadd": 0, 2024-09-11T23:32:49.825 INFO:teuthology.orchestra.run.smithi090.stdout: "segex": 0, 2024-09-11T23:32:49.825 INFO:teuthology.orchestra.run.smithi090.stdout: "segtrm": 0, 2024-09-11T23:32:49.825 INFO:teuthology.orchestra.run.smithi090.stdout: "seg": 2, 2024-09-11T23:32:49.825 INFO:teuthology.orchestra.run.smithi090.stdout: "segmjr": 15, 2024-09-11T23:32:49.825 INFO:teuthology.orchestra.run.smithi090.stdout: "segexg": 0, 2024-09-11T23:32:49.825 INFO:teuthology.orchestra.run.smithi090.stdout: "segexd": 0, 2024-09-11T23:32:49.825 INFO:teuthology.orchestra.run.smithi090.stdout: "expos": 67433080, 2024-09-11T23:32:49.825 INFO:teuthology.orchestra.run.smithi090.stdout: "wrpos": 67528635, 2024-09-11T23:32:49.826 INFO:teuthology.orchestra.run.smithi090.stdout: "rdpos": 67528523, 2024-09-11T23:32:49.826 INFO:teuthology.orchestra.run.smithi090.stdout: "jlat": { 2024-09-11T23:32:49.826 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.826 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.826 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.826 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.826 INFO:teuthology.orchestra.run.smithi090.stdout: "replayed": 3604 2024-09-11T23:32:49.826 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.826 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_mem": { 2024-09-11T23:32:49.826 INFO:teuthology.orchestra.run.smithi090.stdout: "ino": 720, 2024-09-11T23:32:49.826 INFO:teuthology.orchestra.run.smithi090.stdout: "ino+": 1019, 2024-09-11T23:32:49.826 INFO:teuthology.orchestra.run.smithi090.stdout: "ino-": 299, 2024-09-11T23:32:49.826 INFO:teuthology.orchestra.run.smithi090.stdout: "dir": 105, 2024-09-11T23:32:49.827 INFO:teuthology.orchestra.run.smithi090.stdout: "dir+": 107, 2024-09-11T23:32:49.827 INFO:teuthology.orchestra.run.smithi090.stdout: "dir-": 2, 2024-09-11T23:32:49.827 INFO:teuthology.orchestra.run.smithi090.stdout: "dn": 1519, 2024-09-11T23:32:49.827 INFO:teuthology.orchestra.run.smithi090.stdout: "dn+": 1519, 2024-09-11T23:32:49.827 INFO:teuthology.orchestra.run.smithi090.stdout: "dn-": 0, 2024-09-11T23:32:49.827 INFO:teuthology.orchestra.run.smithi090.stdout: "cap": 0, 2024-09-11T23:32:49.827 INFO:teuthology.orchestra.run.smithi090.stdout: "cap+": 0, 2024-09-11T23:32:49.827 INFO:teuthology.orchestra.run.smithi090.stdout: "cap-": 0, 2024-09-11T23:32:49.827 INFO:teuthology.orchestra.run.smithi090.stdout: "rss": 52600, 2024-09-11T23:32:49.827 INFO:teuthology.orchestra.run.smithi090.stdout: "heap": 215328 2024-09-11T23:32:49.827 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.827 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_server": { 2024-09-11T23:32:49.827 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_client_request": 0, 2024-09-11T23:32:49.828 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_server_request": 0, 2024-09-11T23:32:49.828 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_request": 0, 2024-09-11T23:32:49.828 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_session": 0, 2024-09-11T23:32:49.828 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_peer_request": 0, 2024-09-11T23:32:49.828 INFO:teuthology.orchestra.run.smithi090.stdout: "req_create_latency": { 2024-09-11T23:32:49.828 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.828 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.828 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.828 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.828 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getattr_latency": { 2024-09-11T23:32:49.828 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.828 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.828 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.829 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.829 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getfilelock_latency": { 2024-09-11T23:32:49.829 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.829 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.829 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.829 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.829 INFO:teuthology.orchestra.run.smithi090.stdout: "req_link_latency": { 2024-09-11T23:32:49.829 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.829 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.829 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.829 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.829 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookup_latency": { 2024-09-11T23:32:49.829 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.830 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.830 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.830 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.830 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookuphash_latency": { 2024-09-11T23:32:49.830 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.830 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.830 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.830 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.830 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupino_latency": { 2024-09-11T23:32:49.830 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.830 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.830 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.830 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.830 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupname_latency": { 2024-09-11T23:32:49.831 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.831 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.831 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.831 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.831 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupparent_latency": { 2024-09-11T23:32:49.831 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.831 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.831 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.831 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.831 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupsnap_latency": { 2024-09-11T23:32:49.831 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.831 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.831 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.832 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.832 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lssnap_latency": { 2024-09-11T23:32:49.832 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.832 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.832 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.832 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.832 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mkdir_latency": { 2024-09-11T23:32:49.832 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.832 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.832 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.832 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.832 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mknod_latency": { 2024-09-11T23:32:49.833 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.833 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.833 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.833 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.833 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mksnap_latency": { 2024-09-11T23:32:49.833 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.833 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.833 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.833 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.833 INFO:teuthology.orchestra.run.smithi090.stdout: "req_open_latency": { 2024-09-11T23:32:49.833 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.833 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.833 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.834 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.834 INFO:teuthology.orchestra.run.smithi090.stdout: "req_readdir_latency": { 2024-09-11T23:32:49.834 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.834 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.834 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.834 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.834 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rename_latency": { 2024-09-11T23:32:49.834 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.834 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.834 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.834 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.834 INFO:teuthology.orchestra.run.smithi090.stdout: "req_renamesnap_latency": { 2024-09-11T23:32:49.834 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.834 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.835 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.835 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.835 INFO:teuthology.orchestra.run.smithi090.stdout: "req_snapdiff_latency": { 2024-09-11T23:32:49.835 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.835 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.835 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.835 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.835 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmdir_latency": { 2024-09-11T23:32:49.835 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.835 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.835 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.835 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.835 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmsnap_latency": { 2024-09-11T23:32:49.836 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.836 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.836 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.836 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.836 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmxattr_latency": { 2024-09-11T23:32:49.836 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.836 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.836 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.836 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.836 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setattr_latency": { 2024-09-11T23:32:49.836 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.836 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.836 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.837 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.837 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setdirlayout_latency": { 2024-09-11T23:32:49.837 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.837 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.837 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.837 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.837 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setfilelock_latency": { 2024-09-11T23:32:49.837 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.837 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.837 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.837 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.837 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setlayout_latency": { 2024-09-11T23:32:49.837 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.838 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.838 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.838 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.838 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setxattr_latency": { 2024-09-11T23:32:49.838 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.838 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.838 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.838 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.838 INFO:teuthology.orchestra.run.smithi090.stdout: "req_symlink_latency": { 2024-09-11T23:32:49.838 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.838 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.838 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.838 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.838 INFO:teuthology.orchestra.run.smithi090.stdout: "req_unlink_latency": { 2024-09-11T23:32:49.839 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.839 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.839 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.839 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.839 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:32:49.839 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:32:49.839 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getvxattr_latency": { 2024-09-11T23:32:49.839 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.839 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.839 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.839 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:49.839 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.839 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_sessions": { 2024-09-11T23:32:49.840 INFO:teuthology.orchestra.run.smithi090.stdout: "session_count": 2, 2024-09-11T23:32:49.840 INFO:teuthology.orchestra.run.smithi090.stdout: "session_add": 2, 2024-09-11T23:32:49.840 INFO:teuthology.orchestra.run.smithi090.stdout: "session_remove": 0, 2024-09-11T23:32:49.840 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_open": 0, 2024-09-11T23:32:49.840 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_stale": 0, 2024-09-11T23:32:49.840 INFO:teuthology.orchestra.run.smithi090.stdout: "total_load": 0, 2024-09-11T23:32:49.840 INFO:teuthology.orchestra.run.smithi090.stdout: "average_load": 0, 2024-09-11T23:32:49.840 INFO:teuthology.orchestra.run.smithi090.stdout: "avg_session_uptime": 1884, 2024-09-11T23:32:49.840 INFO:teuthology.orchestra.run.smithi090.stdout: "mdthresh_evicted": 0 2024-09-11T23:32:49.840 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.840 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool": { 2024-09-11T23:32:49.840 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:32:49.841 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_items": 0, 2024-09-11T23:32:49.841 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:32:49.841 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:32:49.841 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:32:49.841 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:32:49.841 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:32:49.841 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:32:49.841 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:32:49.841 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:32:49.841 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:32:49.841 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:32:49.841 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:32:49.841 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:32:49.842 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:32:49.842 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_items": 0, 2024-09-11T23:32:49.842 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:32:49.842 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_items": 0, 2024-09-11T23:32:49.842 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:32:49.842 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:32:49.842 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:32:49.842 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:32:49.842 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:32:49.842 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:32:49.842 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:32:49.842 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_items": 0, 2024-09-11T23:32:49.843 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:32:49.843 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:32:49.843 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:32:49.843 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_items": 0, 2024-09-11T23:32:49.843 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_bytes": 0, 2024-09-11T23:32:49.843 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_items": 0, 2024-09-11T23:32:49.843 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:32:49.843 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:32:49.843 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:32:49.843 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:32:49.843 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_bytes": 238953, 2024-09-11T23:32:49.843 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_items": 936, 2024-09-11T23:32:49.843 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:32:49.844 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_items": 0, 2024-09-11T23:32:49.844 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bytes": 0, 2024-09-11T23:32:49.844 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_items": 0, 2024-09-11T23:32:49.844 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:32:49.844 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_items": 0, 2024-09-11T23:32:49.844 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:32:49.844 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_items": 0, 2024-09-11T23:32:49.844 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_bytes": 8320, 2024-09-11T23:32:49.844 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_items": 183, 2024-09-11T23:32:49.844 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:32:49.844 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:32:49.844 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_bytes": 0, 2024-09-11T23:32:49.844 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_items": 0, 2024-09-11T23:32:49.845 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_bytes": 3052534, 2024-09-11T23:32:49.845 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_items": 44421, 2024-09-11T23:32:49.845 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_bytes": 0, 2024-09-11T23:32:49.845 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_items": 0, 2024-09-11T23:32:49.845 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_bytes": 0, 2024-09-11T23:32:49.845 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_items": 0 2024-09-11T23:32:49.845 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.845 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter": { 2024-09-11T23:32:49.845 INFO:teuthology.orchestra.run.smithi090.stdout: "op_active": 0, 2024-09-11T23:32:49.845 INFO:teuthology.orchestra.run.smithi090.stdout: "op_laggy": 0, 2024-09-11T23:32:49.845 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send": 1191, 2024-09-11T23:32:49.845 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send_bytes": 26, 2024-09-11T23:32:49.845 INFO:teuthology.orchestra.run.smithi090.stdout: "op_resend": 4, 2024-09-11T23:32:49.846 INFO:teuthology.orchestra.run.smithi090.stdout: "op_reply": 1191, 2024-09-11T23:32:49.846 INFO:teuthology.orchestra.run.smithi090.stdout: "op_latency": { 2024-09-11T23:32:49.846 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1191, 2024-09-11T23:32:49.846 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 84.765676944, 2024-09-11T23:32:49.846 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.071171853 2024-09-11T23:32:49.846 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.846 INFO:teuthology.orchestra.run.smithi090.stdout: "op_inflight": 0, 2024-09-11T23:32:49.846 INFO:teuthology.orchestra.run.smithi090.stdout: "oplen_avg": { 2024-09-11T23:32:49.846 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1191, 2024-09-11T23:32:49.846 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 1192 2024-09-11T23:32:49.846 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.846 INFO:teuthology.orchestra.run.smithi090.stdout: "op": 1191, 2024-09-11T23:32:49.846 INFO:teuthology.orchestra.run.smithi090.stdout: "op_r": 1191, 2024-09-11T23:32:49.847 INFO:teuthology.orchestra.run.smithi090.stdout: "op_w": 0, 2024-09-11T23:32:49.847 INFO:teuthology.orchestra.run.smithi090.stdout: "op_rmw": 0, 2024-09-11T23:32:49.847 INFO:teuthology.orchestra.run.smithi090.stdout: "op_pg": 0, 2024-09-11T23:32:49.847 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_stat": 718, 2024-09-11T23:32:49.847 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_create": 0, 2024-09-11T23:32:49.847 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_read": 470, 2024-09-11T23:32:49.847 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_write": 0, 2024-09-11T23:32:49.847 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writefull": 0, 2024-09-11T23:32:49.847 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writesame": 0, 2024-09-11T23:32:49.847 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_append": 0, 2024-09-11T23:32:49.847 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_zero": 0, 2024-09-11T23:32:49.847 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_truncate": 0, 2024-09-11T23:32:49.847 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_delete": 0, 2024-09-11T23:32:49.847 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_mapext": 0, 2024-09-11T23:32:49.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_sparse_read": 0, 2024-09-11T23:32:49.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_clonerange": 0, 2024-09-11T23:32:49.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_getxattr": 2, 2024-09-11T23:32:49.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_setxattr": 0, 2024-09-11T23:32:49.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:32:49.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_rmxattr": 0, 2024-09-11T23:32:49.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:32:49.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_call": 0, 2024-09-11T23:32:49.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_watch": 0, 2024-09-11T23:32:49.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_notify": 0, 2024-09-11T23:32:49.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:32:49.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls": 0, 2024-09-11T23:32:49.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:32:49.849 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_other": 0, 2024-09-11T23:32:49.849 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_active": 0, 2024-09-11T23:32:49.849 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_send": 0, 2024-09-11T23:32:49.849 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_resend": 0, 2024-09-11T23:32:49.849 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_ping": 0, 2024-09-11T23:32:49.849 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_active": 0, 2024-09-11T23:32:49.849 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_send": 0, 2024-09-11T23:32:49.849 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_resend": 0, 2024-09-11T23:32:49.849 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_active": 0, 2024-09-11T23:32:49.849 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_send": 0, 2024-09-11T23:32:49.849 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_resend": 0, 2024-09-11T23:32:49.849 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_active": 0, 2024-09-11T23:32:49.849 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_send": 0, 2024-09-11T23:32:49.850 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_resend": 0, 2024-09-11T23:32:49.850 INFO:teuthology.orchestra.run.smithi090.stdout: "command_active": 0, 2024-09-11T23:32:49.850 INFO:teuthology.orchestra.run.smithi090.stdout: "command_send": 0, 2024-09-11T23:32:49.850 INFO:teuthology.orchestra.run.smithi090.stdout: "command_resend": 0, 2024-09-11T23:32:49.850 INFO:teuthology.orchestra.run.smithi090.stdout: "map_epoch": 94, 2024-09-11T23:32:49.850 INFO:teuthology.orchestra.run.smithi090.stdout: "map_full": 0, 2024-09-11T23:32:49.850 INFO:teuthology.orchestra.run.smithi090.stdout: "map_inc": 8, 2024-09-11T23:32:49.850 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_sessions": 7, 2024-09-11T23:32:49.850 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_open": 7, 2024-09-11T23:32:49.850 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_close": 0, 2024-09-11T23:32:49.850 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_laggy": 0, 2024-09-11T23:32:49.850 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_wr": 0, 2024-09-11T23:32:49.850 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_rd": 2, 2024-09-11T23:32:49.851 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_del": 0 2024-09-11T23:32:49.851 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.851 INFO:teuthology.orchestra.run.smithi090.stdout: "oft": { 2024-09-11T23:32:49.851 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_objs": 0, 2024-09-11T23:32:49.851 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:32:49.851 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_updates": 0, 2024-09-11T23:32:49.851 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_removes": 0 2024-09-11T23:32:49.851 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.851 INFO:teuthology.orchestra.run.smithi090.stdout: "osd-slow-ops": { 2024-09-11T23:32:49.851 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_ops_count": 0 2024-09-11T23:32:49.851 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.851 INFO:teuthology.orchestra.run.smithi090.stdout: "purge_queue": { 2024-09-11T23:32:49.851 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops": 0, 2024-09-11T23:32:49.851 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:32:49.852 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing": 0, 2024-09-11T23:32:49.852 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_high_water": 0, 2024-09-11T23:32:49.852 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed_ops": 0, 2024-09-11T23:32:49.852 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed": 0, 2024-09-11T23:32:49.852 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_item_in_journal": 0 2024-09-11T23:32:49.852 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.852 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:32:49.852 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-11T23:32:49.852 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-09-11T23:32:49.852 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-11T23:32:49.852 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 1333, 2024-09-11T23:32:49.852 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 17519927, 2024-09-11T23:32:49.853 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:49.853 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 1333, 2024-09-11T23:32:49.853 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-09-11T23:32:49.853 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-09-11T23:32:49.853 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 1333, 2024-09-11T23:32:49.853 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 17519927, 2024-09-11T23:32:49.853 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-11T23:32:49.853 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.853 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.853 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.853 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:49.854 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.854 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_bytes": { 2024-09-11T23:32:49.854 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-11T23:32:49.854 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-09-11T23:32:49.854 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-11T23:32:49.854 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-09-11T23:32:49.854 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-09-11T23:32:49.854 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:49.854 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-09-11T23:32:49.854 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 1191, 2024-09-11T23:32:49.854 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 17091518, 2024-09-11T23:32:49.854 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 1191, 2024-09-11T23:32:49.854 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 17091518, 2024-09-11T23:32:49.855 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-11T23:32:49.855 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.855 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.855 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.855 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:49.855 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.855 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_ops": { 2024-09-11T23:32:49.855 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-11T23:32:49.855 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 1024, 2024-09-11T23:32:49.855 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-11T23:32:49.855 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-09-11T23:32:49.855 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-09-11T23:32:49.855 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:49.856 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-09-11T23:32:49.856 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 1191, 2024-09-11T23:32:49.856 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 1191, 2024-09-11T23:32:49.856 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 1191, 2024-09-11T23:32:49.856 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 1191, 2024-09-11T23:32:49.856 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-11T23:32:49.856 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.856 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.856 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.856 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:49.856 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.856 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:32:49.856 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-11T23:32:49.857 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-09-11T23:32:49.857 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-11T23:32:49.857 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-09-11T23:32:49.857 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-09-11T23:32:49.857 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:49.857 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-09-11T23:32:49.857 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-09-11T23:32:49.857 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-09-11T23:32:49.857 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 0, 2024-09-11T23:32:49.857 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 0, 2024-09-11T23:32:49.857 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-11T23:32:49.857 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.857 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.858 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.858 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:49.858 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:49.858 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle-0x5576382b06e0": { 2024-09-11T23:32:49.858 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-11T23:32:49.858 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-09-11T23:32:49.858 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-11T23:32:49.858 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-09-11T23:32:49.858 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-09-11T23:32:49.858 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:49.858 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-09-11T23:32:49.858 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-09-11T23:32:49.858 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-09-11T23:32:49.859 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 0, 2024-09-11T23:32:49.859 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 0, 2024-09-11T23:32:49.859 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-11T23:32:49.859 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:49.859 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:49.859 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:49.859 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:49.859 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:49.859 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-11T23:32:50.350 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:50 smithi090 ceph-mon[35130]: Adjusting osd_memory_target on smithi090 to 176.0M 2024-09-11T23:32:50.350 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:50 smithi090 ceph-mon[35130]: Unable to set osd_memory_target on smithi090 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-11T23:32:50.350 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:50 smithi090 ceph-mon[35130]: pgmap v341: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-11T23:32:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:50 smithi043 ceph-mon[30950]: Adjusting osd_memory_target on smithi090 to 176.0M 2024-09-11T23:32:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:50 smithi043 ceph-mon[30950]: Unable to set osd_memory_target on smithi090 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-11T23:32:50.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:50 smithi043 ceph-mon[30950]: pgmap v341: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-11T23:32:50.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:50 smithi097 ceph-mon[35562]: Adjusting osd_memory_target on smithi090 to 176.0M 2024-09-11T23:32:50.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:50 smithi097 ceph-mon[35562]: Unable to set osd_memory_target on smithi090 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-11T23:32:50.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:50 smithi097 ceph-mon[35562]: pgmap v341: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-11T23:32:50.931 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-11T23:32:50.931 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-11T23:32:50.932 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-11T23:32:50.932 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-11T23:32:50.932 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-09-11T23:32:50.932 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:32:50.932 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:32:50.932 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-09-11T23:32:50.932 DEBUG:tasks.check_counter:Getting stats from h 2024-09-11T23:32:50.932 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-11T23:32:51.197 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:32:51.801 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-11T23:32:51.801 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:32:51.801 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 169, 2024-09-11T23:32:51.801 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 173, 2024-09-11T23:32:51.801 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 44692, 2024-09-11T23:32:51.801 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 456966, 2024-09-11T23:32:51.801 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 19, 2024-09-11T23:32:51.801 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 6, 2024-09-11T23:32:51.802 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.021475215, 2024-09-11T23:32:51.802 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.003494421, 2024-09-11T23:32:51.802 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.009456466, 2024-09-11T23:32:51.802 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.003948145, 2024-09-11T23:32:51.802 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:51.802 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 169, 2024-09-11T23:32:51.802 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.048402655, 2024-09-11T23:32:51.802 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000286406 2024-09-11T23:32:51.802 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.802 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:51.802 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 14, 2024-09-11T23:32:51.802 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000001449, 2024-09-11T23:32:51.802 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000103 2024-09-11T23:32:51.803 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.803 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 4000, 2024-09-11T23:32:51.803 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 1424 2024-09-11T23:32:51.803 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.803 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:32:51.803 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 4587, 2024-09-11T23:32:51.803 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 3234, 2024-09-11T23:32:51.803 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 1794570, 2024-09-11T23:32:51.803 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 1664190, 2024-09-11T23:32:51.803 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 27, 2024-09-11T23:32:51.803 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 9, 2024-09-11T23:32:51.803 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.265283670, 2024-09-11T23:32:51.803 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.126731321, 2024-09-11T23:32:51.803 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.192634606, 2024-09-11T23:32:51.804 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.005035705, 2024-09-11T23:32:51.804 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:51.804 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 3234, 2024-09-11T23:32:51.804 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.258210895, 2024-09-11T23:32:51.804 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000079842 2024-09-11T23:32:51.804 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.804 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:51.804 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 7305, 2024-09-11T23:32:51.804 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.001092009, 2024-09-11T23:32:51.804 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000149 2024-09-11T23:32:51.804 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.804 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 3856, 2024-09-11T23:32:51.804 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 462464 2024-09-11T23:32:51.805 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.805 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:32:51.805 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 630, 2024-09-11T23:32:51.805 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 710, 2024-09-11T23:32:51.805 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 1805066, 2024-09-11T23:32:51.805 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 359836, 2024-09-11T23:32:51.805 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 7, 2024-09-11T23:32:51.805 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 6, 2024-09-11T23:32:51.805 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.069877168, 2024-09-11T23:32:51.805 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.025102444, 2024-09-11T23:32:51.805 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.060755069, 2024-09-11T23:32:51.805 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.000815716, 2024-09-11T23:32:51.805 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:51.805 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 709, 2024-09-11T23:32:51.806 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.054816358, 2024-09-11T23:32:51.806 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000077315 2024-09-11T23:32:51.806 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.806 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:51.806 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 616, 2024-09-11T23:32:51.806 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000094109, 2024-09-11T23:32:51.806 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000152 2024-09-11T23:32:51.806 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.806 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 1669456, 2024-09-11T23:32:51.806 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 144336 2024-09-11T23:32:51.806 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.806 INFO:teuthology.orchestra.run.smithi090.stdout: "cct": { 2024-09-11T23:32:51.806 INFO:teuthology.orchestra.run.smithi090.stdout: "total_workers": 1, 2024-09-11T23:32:51.806 INFO:teuthology.orchestra.run.smithi090.stdout: "unhealthy_workers": 0 2024-09-11T23:32:51.807 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.807 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-MDSRank": { 2024-09-11T23:32:51.807 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-09-11T23:32:51.807 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-09-11T23:32:51.807 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 228, 2024-09-11T23:32:51.807 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.066639497, 2024-09-11T23:32:51.807 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000292278 2024-09-11T23:32:51.807 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:51.807 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.807 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-PurgeQueue": { 2024-09-11T23:32:51.807 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-09-11T23:32:51.807 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-09-11T23:32:51.807 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 90, 2024-09-11T23:32:51.808 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.004118857, 2024-09-11T23:32:51.808 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000045765 2024-09-11T23:32:51.808 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:51.808 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.808 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-09-11T23:32:51.808 INFO:teuthology.orchestra.run.smithi090.stdout: "request": 136, 2024-09-11T23:32:51.808 INFO:teuthology.orchestra.run.smithi090.stdout: "reply": 101, 2024-09-11T23:32:51.808 INFO:teuthology.orchestra.run.smithi090.stdout: "reply_latency": { 2024-09-11T23:32:51.808 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 101, 2024-09-11T23:32:51.808 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.197310528, 2024-09-11T23:32:51.808 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.001953569 2024-09-11T23:32:51.808 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.808 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_reply": 0, 2024-09-11T23:32:51.809 INFO:teuthology.orchestra.run.smithi090.stdout: "forward": 35, 2024-09-11T23:32:51.809 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_complete": 0, 2024-09-11T23:32:51.809 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_keys": 0, 2024-09-11T23:32:51.809 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_commit": 11, 2024-09-11T23:32:51.809 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_split": 0, 2024-09-11T23:32:51.809 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_merge": 0, 2024-09-11T23:32:51.809 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes": 18, 2024-09-11T23:32:51.809 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_top": 2, 2024-09-11T23:32:51.809 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_bottom": 6, 2024-09-11T23:32:51.809 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pin_tail": 10, 2024-09-11T23:32:51.809 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pinned": 13, 2024-09-11T23:32:51.809 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_expired": 249, 2024-09-11T23:32:51.810 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_with_caps": 2, 2024-09-11T23:32:51.810 INFO:teuthology.orchestra.run.smithi090.stdout: "caps": 2, 2024-09-11T23:32:51.810 INFO:teuthology.orchestra.run.smithi090.stdout: "subtrees": 4, 2024-09-11T23:32:51.810 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse": 551, 2024-09-11T23:32:51.810 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_hit": 343, 2024-09-11T23:32:51.810 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_forward": 33, 2024-09-11T23:32:51.810 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_discover": 106, 2024-09-11T23:32:51.810 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:32:51.810 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_remote_ino": 0, 2024-09-11T23:32:51.810 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_lock": 0, 2024-09-11T23:32:51.810 INFO:teuthology.orchestra.run.smithi090.stdout: "load_cent": 0, 2024-09-11T23:32:51.810 INFO:teuthology.orchestra.run.smithi090.stdout: "q": 0, 2024-09-11T23:32:51.810 INFO:teuthology.orchestra.run.smithi090.stdout: "exported": 3, 2024-09-11T23:32:51.810 INFO:teuthology.orchestra.run.smithi090.stdout: "exported_inodes": 60, 2024-09-11T23:32:51.811 INFO:teuthology.orchestra.run.smithi090.stdout: "imported": 3, 2024-09-11T23:32:51.811 INFO:teuthology.orchestra.run.smithi090.stdout: "imported_inodes": 22, 2024-09-11T23:32:51.811 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_dir_fetch": 0, 2024-09-11T23:32:51.811 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_backtrace_fetch": 60, 2024-09-11T23:32:51.811 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_peer_discover": 0, 2024-09-11T23:32:51.811 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rfiles": 4, 2024-09-11T23:32:51.811 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rbytes": 234, 2024-09-11T23:32:51.811 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rsnaps": 6, 2024-09-11T23:32:51.811 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_fetch": 60, 2024-09-11T23:32:51.811 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_set_tag": 0, 2024-09-11T23:32:51.811 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:32:51.811 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:32:51.811 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:32:51.811 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:32:51.812 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:32:51.812 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_file_inodes": 60, 2024-09-11T23:32:51.812 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:32:51.812 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_revoke": 13, 2024-09-11T23:32:51.812 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_grant": 61, 2024-09-11T23:32:51.812 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:32:51.812 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:32:51.812 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:32:51.812 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps": 28, 2024-09-11T23:32:51.812 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps_dirty": 18, 2024-09-11T23:32:51.812 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_cap_release": 6, 2024-09-11T23:32:51.812 INFO:teuthology.orchestra.run.smithi090.stdout: "process_request_cap_release": 115 2024-09-11T23:32:51.812 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.812 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache": { 2024-09-11T23:32:51.813 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update": 4, 2024-09-11T23:32:51.813 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update_receipt": 308, 2024-09-11T23:32:51.813 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_try_discover": 207, 2024-09-11T23:32:51.813 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_send_discover": 196, 2024-09-11T23:32:51.813 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_handle_discover": 22, 2024-09-11T23:32:51.813 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays": 0, 2024-09-11T23:32:51.813 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_delayed": 0, 2024-09-11T23:32:51.813 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:32:51.813 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_created": 38, 2024-09-11T23:32:51.813 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_enqueued": 38, 2024-09-11T23:32:51.813 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_reintegrated": 0, 2024-09-11T23:32:51.813 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_migrated": 0, 2024-09-11T23:32:51.813 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_processing": 0, 2024-09-11T23:32:51.814 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:32:51.814 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:32:51.814 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_started": 0, 2024-09-11T23:32:51.814 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_completed": 0, 2024-09-11T23:32:51.814 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_quiesce_path": 5, 2024-09-11T23:32:51.814 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_quiesce_inode": 324, 2024-09-11T23:32:51.814 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:32:51.814 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_exportdir": 3, 2024-09-11T23:32:51.814 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_flush": 0, 2024-09-11T23:32:51.814 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:32:51.814 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragstats": 0, 2024-09-11T23:32:51.814 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_inodestats": 0 2024-09-11T23:32:51.814 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.814 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log": { 2024-09-11T23:32:51.814 INFO:teuthology.orchestra.run.smithi090.stdout: "evlrg": 0, 2024-09-11T23:32:51.815 INFO:teuthology.orchestra.run.smithi090.stdout: "evadd": 180, 2024-09-11T23:32:51.815 INFO:teuthology.orchestra.run.smithi090.stdout: "evex": 0, 2024-09-11T23:32:51.815 INFO:teuthology.orchestra.run.smithi090.stdout: "evtrm": 0, 2024-09-11T23:32:51.815 INFO:teuthology.orchestra.run.smithi090.stdout: "ev": 180, 2024-09-11T23:32:51.815 INFO:teuthology.orchestra.run.smithi090.stdout: "evexg": 0, 2024-09-11T23:32:51.815 INFO:teuthology.orchestra.run.smithi090.stdout: "evexd": 0, 2024-09-11T23:32:51.815 INFO:teuthology.orchestra.run.smithi090.stdout: "segadd": 2, 2024-09-11T23:32:51.815 INFO:teuthology.orchestra.run.smithi090.stdout: "segex": 0, 2024-09-11T23:32:51.815 INFO:teuthology.orchestra.run.smithi090.stdout: "segtrm": 0, 2024-09-11T23:32:51.815 INFO:teuthology.orchestra.run.smithi090.stdout: "seg": 2, 2024-09-11T23:32:51.815 INFO:teuthology.orchestra.run.smithi090.stdout: "segmjr": 2, 2024-09-11T23:32:51.815 INFO:teuthology.orchestra.run.smithi090.stdout: "segexg": 0, 2024-09-11T23:32:51.815 INFO:teuthology.orchestra.run.smithi090.stdout: "segexd": 0, 2024-09-11T23:32:51.815 INFO:teuthology.orchestra.run.smithi090.stdout: "expos": 4194304, 2024-09-11T23:32:51.816 INFO:teuthology.orchestra.run.smithi090.stdout: "wrpos": 4513967, 2024-09-11T23:32:51.816 INFO:teuthology.orchestra.run.smithi090.stdout: "rdpos": 0, 2024-09-11T23:32:51.816 INFO:teuthology.orchestra.run.smithi090.stdout: "jlat": { 2024-09-11T23:32:51.816 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 47, 2024-09-11T23:32:51.816 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.364586504, 2024-09-11T23:32:51.816 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.007757159 2024-09-11T23:32:51.816 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.816 INFO:teuthology.orchestra.run.smithi090.stdout: "replayed": 0 2024-09-11T23:32:51.816 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.816 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_mem": { 2024-09-11T23:32:51.816 INFO:teuthology.orchestra.run.smithi090.stdout: "ino": 23, 2024-09-11T23:32:51.816 INFO:teuthology.orchestra.run.smithi090.stdout: "ino+": 184, 2024-09-11T23:32:51.816 INFO:teuthology.orchestra.run.smithi090.stdout: "ino-": 161, 2024-09-11T23:32:51.817 INFO:teuthology.orchestra.run.smithi090.stdout: "dir": 22, 2024-09-11T23:32:51.817 INFO:teuthology.orchestra.run.smithi090.stdout: "dir+": 151, 2024-09-11T23:32:51.817 INFO:teuthology.orchestra.run.smithi090.stdout: "dir-": 129, 2024-09-11T23:32:51.817 INFO:teuthology.orchestra.run.smithi090.stdout: "dn": 18, 2024-09-11T23:32:51.817 INFO:teuthology.orchestra.run.smithi090.stdout: "dn+": 307, 2024-09-11T23:32:51.817 INFO:teuthology.orchestra.run.smithi090.stdout: "dn-": 289, 2024-09-11T23:32:51.817 INFO:teuthology.orchestra.run.smithi090.stdout: "cap": 2, 2024-09-11T23:32:51.817 INFO:teuthology.orchestra.run.smithi090.stdout: "cap+": 63, 2024-09-11T23:32:51.817 INFO:teuthology.orchestra.run.smithi090.stdout: "cap-": 61, 2024-09-11T23:32:51.817 INFO:teuthology.orchestra.run.smithi090.stdout: "rss": 55936, 2024-09-11T23:32:51.817 INFO:teuthology.orchestra.run.smithi090.stdout: "heap": 223520 2024-09-11T23:32:51.817 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.817 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_server": { 2024-09-11T23:32:51.817 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_client_request": 349, 2024-09-11T23:32:51.818 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_server_request": 6, 2024-09-11T23:32:51.818 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_request": 136, 2024-09-11T23:32:51.818 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_session": 21, 2024-09-11T23:32:51.818 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_peer_request": 108, 2024-09-11T23:32:51.818 INFO:teuthology.orchestra.run.smithi090.stdout: "req_create_latency": { 2024-09-11T23:32:51.818 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 18, 2024-09-11T23:32:51.818 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.022493384, 2024-09-11T23:32:51.818 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.001249632 2024-09-11T23:32:51.818 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.818 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getattr_latency": { 2024-09-11T23:32:51.818 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 3, 2024-09-11T23:32:51.818 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000447264, 2024-09-11T23:32:51.818 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000149088 2024-09-11T23:32:51.818 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.819 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getfilelock_latency": { 2024-09-11T23:32:51.819 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.819 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.819 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.819 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.819 INFO:teuthology.orchestra.run.smithi090.stdout: "req_link_latency": { 2024-09-11T23:32:51.819 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.819 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.819 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.819 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.819 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookup_latency": { 2024-09-11T23:32:51.819 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 37, 2024-09-11T23:32:51.819 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.008200756, 2024-09-11T23:32:51.819 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000221642 2024-09-11T23:32:51.820 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.820 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookuphash_latency": { 2024-09-11T23:32:51.820 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.820 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.820 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.820 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.820 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupino_latency": { 2024-09-11T23:32:51.820 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1, 2024-09-11T23:32:51.820 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000152481, 2024-09-11T23:32:51.820 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000152481 2024-09-11T23:32:51.820 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.820 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupname_latency": { 2024-09-11T23:32:51.820 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.820 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.821 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.821 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.821 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupparent_latency": { 2024-09-11T23:32:51.821 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.821 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.821 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.821 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.821 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupsnap_latency": { 2024-09-11T23:32:51.821 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.821 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.821 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.821 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.821 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lssnap_latency": { 2024-09-11T23:32:51.822 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.822 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.822 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.822 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.822 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mkdir_latency": { 2024-09-11T23:32:51.822 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.822 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.822 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.822 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.822 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mknod_latency": { 2024-09-11T23:32:51.822 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.822 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.822 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.823 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.823 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mksnap_latency": { 2024-09-11T23:32:51.823 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.823 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.823 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.823 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.823 INFO:teuthology.orchestra.run.smithi090.stdout: "req_open_latency": { 2024-09-11T23:32:51.823 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.823 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.823 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.823 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.823 INFO:teuthology.orchestra.run.smithi090.stdout: "req_readdir_latency": { 2024-09-11T23:32:51.823 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 4, 2024-09-11T23:32:51.823 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.002687206, 2024-09-11T23:32:51.824 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000671801 2024-09-11T23:32:51.824 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.824 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rename_latency": { 2024-09-11T23:32:51.824 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 18, 2024-09-11T23:32:51.824 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.120871700, 2024-09-11T23:32:51.824 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.006715094 2024-09-11T23:32:51.824 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.824 INFO:teuthology.orchestra.run.smithi090.stdout: "req_renamesnap_latency": { 2024-09-11T23:32:51.824 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.824 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.824 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.824 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.824 INFO:teuthology.orchestra.run.smithi090.stdout: "req_snapdiff_latency": { 2024-09-11T23:32:51.825 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.825 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.825 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.825 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.825 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmdir_latency": { 2024-09-11T23:32:51.825 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.825 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.825 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.825 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.825 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmsnap_latency": { 2024-09-11T23:32:51.825 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.825 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.825 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.825 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.826 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmxattr_latency": { 2024-09-11T23:32:51.826 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.826 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.826 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.826 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.826 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setattr_latency": { 2024-09-11T23:32:51.826 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.826 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.826 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.826 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.826 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setdirlayout_latency": { 2024-09-11T23:32:51.826 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.826 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.826 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.827 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.827 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setfilelock_latency": { 2024-09-11T23:32:51.827 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.827 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.827 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.827 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.827 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setlayout_latency": { 2024-09-11T23:32:51.827 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.827 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.827 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.827 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.827 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setxattr_latency": { 2024-09-11T23:32:51.827 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.827 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.828 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.828 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.828 INFO:teuthology.orchestra.run.smithi090.stdout: "req_symlink_latency": { 2024-09-11T23:32:51.828 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.828 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.828 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.828 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.828 INFO:teuthology.orchestra.run.smithi090.stdout: "req_unlink_latency": { 2024-09-11T23:32:51.828 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 20, 2024-09-11T23:32:51.828 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.042457737, 2024-09-11T23:32:51.828 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.002122886 2024-09-11T23:32:51.828 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.828 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:32:51.829 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:32:51.829 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getvxattr_latency": { 2024-09-11T23:32:51.829 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.829 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.829 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.829 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:51.829 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.829 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_sessions": { 2024-09-11T23:32:51.829 INFO:teuthology.orchestra.run.smithi090.stdout: "session_count": 2, 2024-09-11T23:32:51.829 INFO:teuthology.orchestra.run.smithi090.stdout: "session_add": 3, 2024-09-11T23:32:51.829 INFO:teuthology.orchestra.run.smithi090.stdout: "session_remove": 1, 2024-09-11T23:32:51.829 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_open": 2, 2024-09-11T23:32:51.829 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_stale": 0, 2024-09-11T23:32:51.830 INFO:teuthology.orchestra.run.smithi090.stdout: "total_load": 40, 2024-09-11T23:32:51.830 INFO:teuthology.orchestra.run.smithi090.stdout: "average_load": 20, 2024-09-11T23:32:51.830 INFO:teuthology.orchestra.run.smithi090.stdout: "avg_session_uptime": 610, 2024-09-11T23:32:51.830 INFO:teuthology.orchestra.run.smithi090.stdout: "mdthresh_evicted": 0 2024-09-11T23:32:51.830 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.830 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool": { 2024-09-11T23:32:51.830 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:32:51.830 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_items": 0, 2024-09-11T23:32:51.830 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:32:51.830 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:32:51.830 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:32:51.830 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:32:51.830 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:32:51.831 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:32:51.831 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:32:51.831 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:32:51.831 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:32:51.831 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:32:51.831 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:32:51.831 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:32:51.831 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:32:51.831 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_items": 0, 2024-09-11T23:32:51.831 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:32:51.831 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_items": 0, 2024-09-11T23:32:51.831 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:32:51.831 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:32:51.831 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:32:51.832 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:32:51.832 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:32:51.832 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:32:51.832 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:32:51.832 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_items": 0, 2024-09-11T23:32:51.832 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:32:51.832 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:32:51.832 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:32:51.832 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_items": 0, 2024-09-11T23:32:51.832 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_bytes": 0, 2024-09-11T23:32:51.832 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_items": 0, 2024-09-11T23:32:51.832 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:32:51.832 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:32:51.832 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:32:51.833 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:32:51.833 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_bytes": 295831, 2024-09-11T23:32:51.833 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_items": 108, 2024-09-11T23:32:51.833 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:32:51.833 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_items": 0, 2024-09-11T23:32:51.833 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bytes": 0, 2024-09-11T23:32:51.833 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_items": 0, 2024-09-11T23:32:51.833 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:32:51.833 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_items": 0, 2024-09-11T23:32:51.833 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:32:51.833 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_items": 0, 2024-09-11T23:32:51.833 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_bytes": 8320, 2024-09-11T23:32:51.833 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_items": 183, 2024-09-11T23:32:51.833 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:32:51.834 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:32:51.834 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_bytes": 0, 2024-09-11T23:32:51.834 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_items": 0, 2024-09-11T23:32:51.834 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_bytes": 102045, 2024-09-11T23:32:51.834 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_items": 407, 2024-09-11T23:32:51.834 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_bytes": 0, 2024-09-11T23:32:51.834 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_items": 0, 2024-09-11T23:32:51.834 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_bytes": 0, 2024-09-11T23:32:51.834 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_items": 0 2024-09-11T23:32:51.834 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.834 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter": { 2024-09-11T23:32:51.834 INFO:teuthology.orchestra.run.smithi090.stdout: "op_active": 0, 2024-09-11T23:32:51.834 INFO:teuthology.orchestra.run.smithi090.stdout: "op_laggy": 0, 2024-09-11T23:32:51.835 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send": 226, 2024-09-11T23:32:51.835 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send_bytes": 342954, 2024-09-11T23:32:51.835 INFO:teuthology.orchestra.run.smithi090.stdout: "op_resend": 33, 2024-09-11T23:32:51.835 INFO:teuthology.orchestra.run.smithi090.stdout: "op_reply": 226, 2024-09-11T23:32:51.835 INFO:teuthology.orchestra.run.smithi090.stdout: "op_latency": { 2024-09-11T23:32:51.835 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 226, 2024-09-11T23:32:51.835 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 4.631927136, 2024-09-11T23:32:51.835 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.020495252 2024-09-11T23:32:51.835 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.835 INFO:teuthology.orchestra.run.smithi090.stdout: "op_inflight": 0, 2024-09-11T23:32:51.835 INFO:teuthology.orchestra.run.smithi090.stdout: "oplen_avg": { 2024-09-11T23:32:51.835 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 226, 2024-09-11T23:32:51.835 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 267 2024-09-11T23:32:51.835 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.836 INFO:teuthology.orchestra.run.smithi090.stdout: "op": 226, 2024-09-11T23:32:51.836 INFO:teuthology.orchestra.run.smithi090.stdout: "op_r": 66, 2024-09-11T23:32:51.836 INFO:teuthology.orchestra.run.smithi090.stdout: "op_w": 160, 2024-09-11T23:32:51.836 INFO:teuthology.orchestra.run.smithi090.stdout: "op_rmw": 0, 2024-09-11T23:32:51.836 INFO:teuthology.orchestra.run.smithi090.stdout: "op_pg": 0, 2024-09-11T23:32:51.836 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_stat": 0, 2024-09-11T23:32:51.836 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_create": 10, 2024-09-11T23:32:51.836 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_read": 6, 2024-09-11T23:32:51.836 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_write": 53, 2024-09-11T23:32:51.836 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writefull": 18, 2024-09-11T23:32:51.836 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writesame": 0, 2024-09-11T23:32:51.836 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_append": 0, 2024-09-11T23:32:51.836 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_zero": 0, 2024-09-11T23:32:51.836 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_truncate": 0, 2024-09-11T23:32:51.837 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_delete": 50, 2024-09-11T23:32:51.837 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_mapext": 0, 2024-09-11T23:32:51.837 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_sparse_read": 0, 2024-09-11T23:32:51.837 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_clonerange": 0, 2024-09-11T23:32:51.837 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_getxattr": 60, 2024-09-11T23:32:51.837 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_setxattr": 20, 2024-09-11T23:32:51.837 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:32:51.837 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_rmxattr": 0, 2024-09-11T23:32:51.837 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:32:51.838 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_call": 0, 2024-09-11T23:32:51.838 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_watch": 0, 2024-09-11T23:32:51.838 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_notify": 0, 2024-09-11T23:32:51.838 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:32:51.838 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls": 0, 2024-09-11T23:32:51.838 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:32:51.838 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_other": 0, 2024-09-11T23:32:51.838 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_active": 0, 2024-09-11T23:32:51.838 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_send": 0, 2024-09-11T23:32:51.838 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_resend": 0, 2024-09-11T23:32:51.838 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_ping": 0, 2024-09-11T23:32:51.838 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_active": 0, 2024-09-11T23:32:51.838 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_send": 0, 2024-09-11T23:32:51.838 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_resend": 0, 2024-09-11T23:32:51.839 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_active": 0, 2024-09-11T23:32:51.839 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_send": 0, 2024-09-11T23:32:51.839 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_resend": 0, 2024-09-11T23:32:51.839 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_active": 0, 2024-09-11T23:32:51.839 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_send": 0, 2024-09-11T23:32:51.839 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_resend": 0, 2024-09-11T23:32:51.839 INFO:teuthology.orchestra.run.smithi090.stdout: "command_active": 0, 2024-09-11T23:32:51.839 INFO:teuthology.orchestra.run.smithi090.stdout: "command_send": 0, 2024-09-11T23:32:51.839 INFO:teuthology.orchestra.run.smithi090.stdout: "command_resend": 0, 2024-09-11T23:32:51.839 INFO:teuthology.orchestra.run.smithi090.stdout: "map_epoch": 94, 2024-09-11T23:32:51.839 INFO:teuthology.orchestra.run.smithi090.stdout: "map_full": 0, 2024-09-11T23:32:51.839 INFO:teuthology.orchestra.run.smithi090.stdout: "map_inc": 12, 2024-09-11T23:32:51.839 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_sessions": 12, 2024-09-11T23:32:51.839 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_open": 12, 2024-09-11T23:32:51.840 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_close": 0, 2024-09-11T23:32:51.840 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_laggy": 0, 2024-09-11T23:32:51.840 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_wr": 41, 2024-09-11T23:32:51.840 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_rd": 0, 2024-09-11T23:32:51.840 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_del": 9 2024-09-11T23:32:51.840 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.840 INFO:teuthology.orchestra.run.smithi090.stdout: "oft": { 2024-09-11T23:32:51.840 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_objs": 1, 2024-09-11T23:32:51.840 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:32:51.840 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_updates": 0, 2024-09-11T23:32:51.840 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_removes": 0 2024-09-11T23:32:51.840 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.840 INFO:teuthology.orchestra.run.smithi090.stdout: "osd-slow-ops": { 2024-09-11T23:32:51.840 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_ops_count": 329 2024-09-11T23:32:51.841 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.841 INFO:teuthology.orchestra.run.smithi090.stdout: "purge_queue": { 2024-09-11T23:32:51.841 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops": 0, 2024-09-11T23:32:51.841 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops_high_water": 10, 2024-09-11T23:32:51.841 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing": 0, 2024-09-11T23:32:51.841 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_high_water": 10, 2024-09-11T23:32:51.841 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed_ops": 38, 2024-09-11T23:32:51.841 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed": 38, 2024-09-11T23:32:51.841 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_item_in_journal": 0 2024-09-11T23:32:51.841 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.841 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:32:51.841 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-11T23:32:51.841 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-09-11T23:32:51.841 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-11T23:32:51.842 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 5382, 2024-09-11T23:32:51.842 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 3141168, 2024-09-11T23:32:51.842 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:51.842 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 5382, 2024-09-11T23:32:51.842 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-09-11T23:32:51.842 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-09-11T23:32:51.842 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 5382, 2024-09-11T23:32:51.842 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 3141168, 2024-09-11T23:32:51.842 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-11T23:32:51.842 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.842 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.842 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.842 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:51.843 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.843 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_bytes": { 2024-09-11T23:32:51.843 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-11T23:32:51.843 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-09-11T23:32:51.843 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-11T23:32:51.843 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-09-11T23:32:51.843 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-09-11T23:32:51.843 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:51.843 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-09-11T23:32:51.843 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 226, 2024-09-11T23:32:51.843 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 347592, 2024-09-11T23:32:51.843 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 226, 2024-09-11T23:32:51.843 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 347592, 2024-09-11T23:32:51.843 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-11T23:32:51.844 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.844 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.844 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.844 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:51.844 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.844 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_ops": { 2024-09-11T23:32:51.844 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-11T23:32:51.844 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 1024, 2024-09-11T23:32:51.844 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-11T23:32:51.844 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-09-11T23:32:51.844 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-09-11T23:32:51.844 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:51.844 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-09-11T23:32:51.844 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 226, 2024-09-11T23:32:51.845 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 226, 2024-09-11T23:32:51.845 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 226, 2024-09-11T23:32:51.845 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 226, 2024-09-11T23:32:51.845 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-11T23:32:51.845 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.845 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.845 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.845 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:51.845 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.845 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:32:51.845 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-11T23:32:51.845 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-09-11T23:32:51.845 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-11T23:32:51.845 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 38, 2024-09-11T23:32:51.846 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 4638, 2024-09-11T23:32:51.846 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:51.846 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 38, 2024-09-11T23:32:51.846 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-09-11T23:32:51.846 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-09-11T23:32:51.846 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 6, 2024-09-11T23:32:51.846 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 4638, 2024-09-11T23:32:51.846 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-11T23:32:51.846 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.846 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.846 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.846 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:51.846 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:51.846 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle-0x555b8fd48500": { 2024-09-11T23:32:51.847 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-11T23:32:51.847 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-09-11T23:32:51.847 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-11T23:32:51.847 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 180, 2024-09-11T23:32:51.847 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 319663, 2024-09-11T23:32:51.847 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:51.847 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 180, 2024-09-11T23:32:51.847 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-09-11T23:32:51.847 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-09-11T23:32:51.847 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 47, 2024-09-11T23:32:51.847 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 319663, 2024-09-11T23:32:51.847 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-11T23:32:51.847 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:51.847 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:51.848 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:51.848 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:51.848 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:51.848 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-11T23:32:52.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:52 smithi090 ceph-mon[35130]: pgmap v342: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-09-11T23:32:52.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:52 smithi043 ceph-mon[30950]: pgmap v342: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-09-11T23:32:52.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:52 smithi097 ceph-mon[35562]: pgmap v342: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-09-11T23:32:52.712 INFO:tasks.check_counter:Daemon mds.h mds.exported=3 2024-09-11T23:32:52.712 INFO:tasks.check_counter:Daemon mds.h mds.imported=3 2024-09-11T23:32:52.712 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=4 2024-09-11T23:32:52.712 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=308 2024-09-11T23:32:52.712 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-11T23:32:52.713 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:32:52.713 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:32:52.713 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-09-11T23:32:52.713 DEBUG:tasks.check_counter:Getting stats from k 2024-09-11T23:32:52.713 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-11T23:32:52.962 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:32:53.493 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-11T23:32:53.493 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:32:53.493 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 0, 2024-09-11T23:32:53.493 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 2, 2024-09-11T23:32:53.493 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 0, 2024-09-11T23:32:53.494 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 0, 2024-09-11T23:32:53.494 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 4, 2024-09-11T23:32:53.494 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 0, 2024-09-11T23:32:53.494 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.000969775, 2024-09-11T23:32:53.494 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.000000000, 2024-09-11T23:32:53.494 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.000192583, 2024-09-11T23:32:53.494 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:32:53.494 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:53.494 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:53.494 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:53.494 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:53.494 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:53.494 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:53.495 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:53.495 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:53.495 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:53.495 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:53.495 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:32:53.495 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:32:53.495 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:53.495 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:32:53.495 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 234, 2024-09-11T23:32:53.495 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 221, 2024-09-11T23:32:53.495 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 1586288, 2024-09-11T23:32:53.495 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 134912, 2024-09-11T23:32:53.495 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 2, 2024-09-11T23:32:53.496 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 1, 2024-09-11T23:32:53.496 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.042446828, 2024-09-11T23:32:53.496 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.014388147, 2024-09-11T23:32:53.496 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.041354973, 2024-09-11T23:32:53.496 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:32:53.496 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:53.496 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 221, 2024-09-11T23:32:53.496 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.018786512, 2024-09-11T23:32:53.496 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000085006 2024-09-11T23:32:53.496 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:53.496 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:53.496 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:53.497 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:53.497 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:53.497 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:53.497 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 1586288, 2024-09-11T23:32:53.497 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 134912 2024-09-11T23:32:53.497 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:53.497 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:32:53.497 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 2, 2024-09-11T23:32:53.497 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 168, 2024-09-11T23:32:53.497 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 320, 2024-09-11T23:32:53.497 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 186896, 2024-09-11T23:32:53.497 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 5, 2024-09-11T23:32:53.498 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 3, 2024-09-11T23:32:53.498 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.010689403, 2024-09-11T23:32:53.498 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.006929234, 2024-09-11T23:32:53.498 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.001790181, 2024-09-11T23:32:53.498 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:32:53.498 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:53.498 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 165, 2024-09-11T23:32:53.498 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.020331278, 2024-09-11T23:32:53.498 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000123219 2024-09-11T23:32:53.498 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:53.498 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:53.498 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:53.499 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:53.499 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:53.499 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:53.499 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T23:32:53.499 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 186896 2024-09-11T23:32:53.499 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:53.499 INFO:teuthology.orchestra.run.smithi090.stdout: "cct": { 2024-09-11T23:32:53.499 INFO:teuthology.orchestra.run.smithi090.stdout: "total_workers": 0, 2024-09-11T23:32:53.499 INFO:teuthology.orchestra.run.smithi090.stdout: "unhealthy_workers": 0 2024-09-11T23:32:53.500 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:53.500 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool": { 2024-09-11T23:32:53.500 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:32:53.500 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_items": 0, 2024-09-11T23:32:53.500 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:32:53.500 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:32:53.500 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:32:53.500 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:32:53.500 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:32:53.500 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:32:53.500 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:32:53.500 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:32:53.500 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:32:53.501 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:32:53.501 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:32:53.501 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:32:53.501 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:32:53.501 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_items": 0, 2024-09-11T23:32:53.501 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:32:53.501 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_items": 0, 2024-09-11T23:32:53.501 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:32:53.501 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:32:53.501 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:32:53.501 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:32:53.501 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:32:53.501 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:32:53.502 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:32:53.502 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_items": 0, 2024-09-11T23:32:53.502 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:32:53.502 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:32:53.502 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:32:53.502 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_items": 0, 2024-09-11T23:32:53.502 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_bytes": 0, 2024-09-11T23:32:53.502 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_items": 0, 2024-09-11T23:32:53.502 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:32:53.502 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:32:53.502 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:32:53.502 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:32:53.502 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_bytes": 149007, 2024-09-11T23:32:53.502 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_items": 30, 2024-09-11T23:32:53.503 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:32:53.503 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_items": 0, 2024-09-11T23:32:53.503 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bytes": 0, 2024-09-11T23:32:53.503 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_items": 0, 2024-09-11T23:32:53.503 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:32:53.503 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_items": 0, 2024-09-11T23:32:53.503 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:32:53.503 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_items": 0, 2024-09-11T23:32:53.503 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_bytes": 0, 2024-09-11T23:32:53.503 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_items": 0, 2024-09-11T23:32:53.503 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:32:53.503 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:32:53.504 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_bytes": 0, 2024-09-11T23:32:53.504 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_items": 0, 2024-09-11T23:32:53.504 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_bytes": 928, 2024-09-11T23:32:53.504 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_items": 2, 2024-09-11T23:32:53.504 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_bytes": 0, 2024-09-11T23:32:53.504 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_items": 0, 2024-09-11T23:32:53.504 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_bytes": 0, 2024-09-11T23:32:53.504 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_items": 0 2024-09-11T23:32:53.504 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-11T23:32:53.504 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:32:53.504 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-11T23:32:53.504 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-09-11T23:32:53.505 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-11T23:32:53.505 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 232, 2024-09-11T23:32:53.505 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 1550803, 2024-09-11T23:32:53.505 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:53.505 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 232, 2024-09-11T23:32:53.505 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-09-11T23:32:53.505 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-09-11T23:32:53.505 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 232, 2024-09-11T23:32:53.505 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 1550803, 2024-09-11T23:32:53.505 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-11T23:32:53.505 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-11T23:32:53.505 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-11T23:32:53.505 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-11T23:32:53.505 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:53.506 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-11T23:32:53.506 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-11T23:32:54.228 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-11T23:32:54.228 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-11T23:32:54.228 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-09-11T23:32:54.228 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-09-11T23:32:54.228 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-09-11T23:32:54.228 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-09-11T23:32:54.228 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-09-11T23:32:54.228 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-09-11T23:32:54.228 DEBUG:tasks.check_counter:Getting stats from c 2024-09-11T23:32:54.229 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-11T23:32:54.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:54 smithi043 ceph-mon[30950]: pgmap v343: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-11T23:32:54.497 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:32:54.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:54 smithi090 ceph-mon[35130]: pgmap v343: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-11T23:32:54.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:54 smithi097 ceph-mon[35562]: pgmap v343: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-11T23:32:55.113 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-11T23:32:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:32:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 984, 2024-09-11T23:32:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 961, 2024-09-11T23:32:55.113 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 1899264, 2024-09-11T23:32:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 366670, 2024-09-11T23:32:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 5, 2024-09-11T23:32:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 1, 2024-09-11T23:32:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.126138018, 2024-09-11T23:32:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.040012078, 2024-09-11T23:32:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.064665857, 2024-09-11T23:32:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.027882885, 2024-09-11T23:32:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 960, 2024-09-11T23:32:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.054975888, 2024-09-11T23:32:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000057266 2024-09-11T23:32:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.114 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1654320, 2024-09-11T23:32:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 134848 2024-09-11T23:32:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:32:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 8, 2024-09-11T23:32:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 168, 2024-09-11T23:32:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 3948, 2024-09-11T23:32:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 449914, 2024-09-11T23:32:55.115 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 6, 2024-09-11T23:32:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 4, 2024-09-11T23:32:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.014527318, 2024-09-11T23:32:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.006027619, 2024-09-11T23:32:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.003959015, 2024-09-11T23:32:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000071405, 2024-09-11T23:32:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 166, 2024-09-11T23:32:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.023114170, 2024-09-11T23:32:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000139241 2024-09-11T23:32:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.116 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-09-11T23:32:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 449216 2024-09-11T23:32:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:32:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 1380, 2024-09-11T23:32:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 1382, 2024-09-11T23:32:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 671452, 2024-09-11T23:32:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 438847, 2024-09-11T23:32:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2024-09-11T23:32:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2024-09-11T23:32:55.117 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.094759250, 2024-09-11T23:32:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.029937579, 2024-09-11T23:32:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.024400282, 2024-09-11T23:32:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.034193876, 2024-09-11T23:32:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1380, 2024-09-11T23:32:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.063392884, 2024-09-11T23:32:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000045936 2024-09-11T23:32:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.118 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:32:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:32:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-09-11T23:32:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-09-11T23:32:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-09-11T23:32:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-09-11T23:32:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-09-11T23:32:55.119 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-09-11T23:32:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3531, 2024-09-11T23:32:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.094716244, 2024-09-11T23:32:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000026824 2024-09-11T23:32:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-09-11T23:32:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-09-11T23:32:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-09-11T23:32:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:55.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-09-11T23:32:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 0, 2024-09-11T23:32:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 0, 2024-09-11T23:32:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-09-11T23:32:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-09-11T23:32:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-09-11T23:32:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-09-11T23:32:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-09-11T23:32:55.121 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 0, 2024-09-11T23:32:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-09-11T23:32:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-09-11T23:32:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 39, 2024-09-11T23:32:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 0, 2024-09-11T23:32:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 39, 2024-09-11T23:32:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-09-11T23:32:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 39, 2024-09-11T23:32:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-09-11T23:32:55.122 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 0, 2024-09-11T23:32:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 0, 2024-09-11T23:32:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 1, 2024-09-11T23:32:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 0, 2024-09-11T23:32:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 0, 2024-09-11T23:32:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-09-11T23:32:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2024-09-11T23:32:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:32:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-09-11T23:32:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2024-09-11T23:32:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-09-11T23:32:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-09-11T23:32:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-09-11T23:32:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-09-11T23:32:55.123 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-09-11T23:32:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-09-11T23:32:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-09-11T23:32:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:32:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-09-11T23:32:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 0, 2024-09-11T23:32:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 0, 2024-09-11T23:32:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-09-11T23:32:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:32:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-09-11T23:32:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:32:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:32:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:32:55.124 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:32:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:32:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-09-11T23:32:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:32:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T23:32:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:32:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:32:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:32:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:32:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 0, 2024-09-11T23:32:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:32:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 0, 2024-09-11T23:32:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 0 2024-09-11T23:32:55.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-09-11T23:32:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-09-11T23:32:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-09-11T23:32:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-09-11T23:32:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 0, 2024-09-11T23:32:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-09-11T23:32:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-09-11T23:32:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-09-11T23:32:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:32:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 0, 2024-09-11T23:32:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 0, 2024-09-11T23:32:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-09-11T23:32:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-09-11T23:32:55.126 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-09-11T23:32:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:32:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:32:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-09-11T23:32:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-09-11T23:32:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 0, 2024-09-11T23:32:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 0, 2024-09-11T23:32:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:32:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-09-11T23:32:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-09-11T23:32:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:32:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-09-11T23:32:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-09-11T23:32:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.127 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-09-11T23:32:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-09-11T23:32:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 0, 2024-09-11T23:32:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-09-11T23:32:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-09-11T23:32:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 180, 2024-09-11T23:32:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-09-11T23:32:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-09-11T23:32:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 0, 2024-09-11T23:32:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-09-11T23:32:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-09-11T23:32:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 2, 2024-09-11T23:32:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 2, 2024-09-11T23:32:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-09-11T23:32:55.128 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-09-11T23:32:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-09-11T23:32:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 4513967, 2024-09-11T23:32:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 4511406, 2024-09-11T23:32:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-09-11T23:32:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 180 2024-09-11T23:32:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-09-11T23:32:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 4, 2024-09-11T23:32:55.129 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 76, 2024-09-11T23:32:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 72, 2024-09-11T23:32:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 2, 2024-09-11T23:32:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 17, 2024-09-11T23:32:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 15, 2024-09-11T23:32:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 39, 2024-09-11T23:32:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 115, 2024-09-11T23:32:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 76, 2024-09-11T23:32:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 0, 2024-09-11T23:32:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 0, 2024-09-11T23:32:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 0, 2024-09-11T23:32:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 48624, 2024-09-11T23:32:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 215328 2024-09-11T23:32:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-09-11T23:32:55.130 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 0, 2024-09-11T23:32:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-09-11T23:32:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 0, 2024-09-11T23:32:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 0, 2024-09-11T23:32:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-09-11T23:32:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-09-11T23:32:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-09-11T23:32:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.131 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-09-11T23:32:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-09-11T23:32:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-09-11T23:32:55.132 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-09-11T23:32:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-09-11T23:32:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.133 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-09-11T23:32:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-09-11T23:32:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-09-11T23:32:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.135 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-09-11T23:32:55.135 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.135 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.135 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.135 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-09-11T23:32:55.135 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.136 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.136 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.136 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-09-11T23:32:55.136 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.136 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.136 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.136 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-09-11T23:32:55.136 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-09-11T23:32:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-09-11T23:32:55.137 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-09-11T23:32:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-09-11T23:32:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-09-11T23:32:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-09-11T23:32:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-09-11T23:32:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-09-11T23:32:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-09-11T23:32:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-09-11T23:32:55.140 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-09-11T23:32:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-09-11T23:32:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-09-11T23:32:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-09-11T23:32:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-09-11T23:32:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.142 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.143 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:32:55.143 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:32:55.143 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-09-11T23:32:55.143 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.143 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.143 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.143 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:55.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.143 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-09-11T23:32:55.143 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 2, 2024-09-11T23:32:55.143 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 3, 2024-09-11T23:32:55.143 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 1, 2024-09-11T23:32:55.143 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 3, 2024-09-11T23:32:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-09-11T23:32:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 0, 2024-09-11T23:32:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 0, 2024-09-11T23:32:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 2561, 2024-09-11T23:32:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-09-11T23:32:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-09-11T23:32:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:32:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-09-11T23:32:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:32:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:32:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:32:55.144 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:32:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:32:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:32:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:32:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:32:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:32:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:32:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:32:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:32:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:32:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-09-11T23:32:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:32:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-09-11T23:32:55.145 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:32:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:32:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:32:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:32:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:32:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:32:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:32:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-09-11T23:32:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:32:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:32:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:32:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-09-11T23:32:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-09-11T23:32:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-09-11T23:32:55.146 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:32:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:32:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:32:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:32:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 181320, 2024-09-11T23:32:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 47, 2024-09-11T23:32:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:32:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-09-11T23:32:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-09-11T23:32:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-09-11T23:32:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:32:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-09-11T23:32:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:32:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-09-11T23:32:55.147 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 8320, 2024-09-11T23:32:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 183, 2024-09-11T23:32:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:32:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:32:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-09-11T23:32:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-09-11T23:32:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 43424, 2024-09-11T23:32:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 192, 2024-09-11T23:32:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-09-11T23:32:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-09-11T23:32:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-09-11T23:32:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-09-11T23:32:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.148 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-09-11T23:32:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-09-11T23:32:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-09-11T23:32:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 2111, 2024-09-11T23:32:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 21, 2024-09-11T23:32:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 3, 2024-09-11T23:32:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 2111, 2024-09-11T23:32:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-09-11T23:32:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2111, 2024-09-11T23:32:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 64.217996822, 2024-09-11T23:32:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.030420652 2024-09-11T23:32:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-09-11T23:32:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-09-11T23:32:55.149 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2111, 2024-09-11T23:32:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2112 2024-09-11T23:32:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 2111, 2024-09-11T23:32:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 2111, 2024-09-11T23:32:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 0, 2024-09-11T23:32:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-09-11T23:32:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-09-11T23:32:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 1358, 2024-09-11T23:32:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 0, 2024-09-11T23:32:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 751, 2024-09-11T23:32:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 0, 2024-09-11T23:32:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 0, 2024-09-11T23:32:55.150 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-09-11T23:32:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-09-11T23:32:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-09-11T23:32:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-09-11T23:32:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 0, 2024-09-11T23:32:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-09-11T23:32:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-09-11T23:32:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-09-11T23:32:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 1, 2024-09-11T23:32:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 0, 2024-09-11T23:32:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:32:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-09-11T23:32:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:32:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-09-11T23:32:55.151 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-09-11T23:32:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-09-11T23:32:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:32:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-09-11T23:32:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:32:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-09-11T23:32:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-09-11T23:32:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-09-11T23:32:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-09-11T23:32:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-09-11T23:32:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-09-11T23:32:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-09-11T23:32:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-09-11T23:32:55.152 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-09-11T23:32:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-09-11T23:32:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-09-11T23:32:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-09-11T23:32:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-09-11T23:32:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-09-11T23:32:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-09-11T23:32:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-09-11T23:32:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-09-11T23:32:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 94, 2024-09-11T23:32:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-09-11T23:32:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 12, 2024-09-11T23:32:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 4, 2024-09-11T23:32:55.153 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 4, 2024-09-11T23:32:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-09-11T23:32:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-09-11T23:32:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 0, 2024-09-11T23:32:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 2, 2024-09-11T23:32:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 0 2024-09-11T23:32:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-09-11T23:32:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 0, 2024-09-11T23:32:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:32:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-09-11T23:32:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-09-11T23:32:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-09-11T23:32:55.154 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-09-11T23:32:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-09-11T23:32:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-09-11T23:32:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:32:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-09-11T23:32:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 0, 2024-09-11T23:32:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 0, 2024-09-11T23:32:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 0, 2024-09-11T23:32:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-09-11T23:32:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:32:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-11T23:32:55.155 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-09-11T23:32:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-11T23:32:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 2368, 2024-09-11T23:32:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 2345448, 2024-09-11T23:32:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 2368, 2024-09-11T23:32:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-11T23:32:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-11T23:32:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2368, 2024-09-11T23:32:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 2345448, 2024-09-11T23:32:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-11T23:32:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.156 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-09-11T23:32:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-11T23:32:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-09-11T23:32:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-11T23:32:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-11T23:32:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-11T23:32:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-11T23:32:55.157 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 2111, 2024-09-11T23:32:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 319668, 2024-09-11T23:32:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2111, 2024-09-11T23:32:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 319668, 2024-09-11T23:32:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-11T23:32:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:55.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-09-11T23:32:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-11T23:32:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-09-11T23:32:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-11T23:32:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-11T23:32:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-11T23:32:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-11T23:32:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 2111, 2024-09-11T23:32:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 2111, 2024-09-11T23:32:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2111, 2024-09-11T23:32:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 2111, 2024-09-11T23:32:55.159 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-11T23:32:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:32:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-11T23:32:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-09-11T23:32:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-11T23:32:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-11T23:32:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-11T23:32:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:55.160 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-11T23:32:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-11T23:32:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-11T23:32:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-09-11T23:32:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-09-11T23:32:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-11T23:32:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x56441847a6e0": { 2024-09-11T23:32:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-11T23:32:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-09-11T23:32:55.161 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-11T23:32:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-11T23:32:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-11T23:32:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-11T23:32:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-11T23:32:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-11T23:32:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-09-11T23:32:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-09-11T23:32:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-11T23:32:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:55.163 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-11T23:32:56.001 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-11T23:32:56.002 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-11T23:32:56.002 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-11T23:32:56.002 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-09-11T23:32:56.002 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-09-11T23:32:56.002 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:32:56.002 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:32:56.002 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-09-11T23:32:56.002 DEBUG:tasks.check_counter:Getting stats from f 2024-09-11T23:32:56.003 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-11T23:32:56.252 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:32:56.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:56 smithi097 ceph-mon[35562]: pgmap v344: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:32:56.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:56 smithi043 ceph-mon[30950]: pgmap v344: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:32:56.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:56 smithi090 ceph-mon[35130]: pgmap v344: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T23:32:56.786 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-11T23:32:56.786 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:32:56.786 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 753, 2024-09-11T23:32:56.786 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 755, 2024-09-11T23:32:56.786 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 283076, 2024-09-11T23:32:56.786 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 239518, 2024-09-11T23:32:56.786 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 6, 2024-09-11T23:32:56.786 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2024-09-11T23:32:56.786 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.065337930, 2024-09-11T23:32:56.787 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.019688932, 2024-09-11T23:32:56.787 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.018201132, 2024-09-11T23:32:56.787 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.020825391, 2024-09-11T23:32:56.787 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:56.787 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 753, 2024-09-11T23:32:56.787 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.032627411, 2024-09-11T23:32:56.787 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000043329 2024-09-11T23:32:56.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.787 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:56.787 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.787 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.787 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.788 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:32:56.788 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:32:56.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.788 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:32:56.788 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 658, 2024-09-11T23:32:56.788 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 822, 2024-09-11T23:32:56.788 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 161718, 2024-09-11T23:32:56.788 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 660965, 2024-09-11T23:32:56.788 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 7, 2024-09-11T23:32:56.788 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 5, 2024-09-11T23:32:56.788 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.075284954, 2024-09-11T23:32:56.788 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.021954629, 2024-09-11T23:32:56.789 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.016792621, 2024-09-11T23:32:56.789 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.025499764, 2024-09-11T23:32:56.789 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:56.789 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 819, 2024-09-11T23:32:56.789 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.034614169, 2024-09-11T23:32:56.789 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000042263 2024-09-11T23:32:56.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.789 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:56.789 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.789 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.789 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.789 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T23:32:56.790 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 452352 2024-09-11T23:32:56.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.790 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:32:56.790 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 995, 2024-09-11T23:32:56.790 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 970, 2024-09-11T23:32:56.790 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 1904560, 2024-09-11T23:32:56.790 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 369988, 2024-09-11T23:32:56.790 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 3, 2024-09-11T23:32:56.790 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2024-09-11T23:32:56.790 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.126293080, 2024-09-11T23:32:56.790 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.039924894, 2024-09-11T23:32:56.790 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.067527406, 2024-09-11T23:32:56.790 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.026993157, 2024-09-11T23:32:56.791 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:56.791 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 970, 2024-09-11T23:32:56.791 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.053730169, 2024-09-11T23:32:56.791 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000055391 2024-09-11T23:32:56.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.791 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:56.791 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.791 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.791 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.791 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1657936, 2024-09-11T23:32:56.791 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 136576 2024-09-11T23:32:56.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.792 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-09-11T23:32:56.792 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-09-11T23:32:56.792 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-09-11T23:32:56.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.792 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-09-11T23:32:56.792 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-09-11T23:32:56.792 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-09-11T23:32:56.792 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3560, 2024-09-11T23:32:56.792 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.102462064, 2024-09-11T23:32:56.792 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000028781 2024-09-11T23:32:56.792 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:56.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.792 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-09-11T23:32:56.793 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-09-11T23:32:56.793 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-09-11T23:32:56.793 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.793 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.793 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.793 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:56.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.793 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-09-11T23:32:56.793 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 0, 2024-09-11T23:32:56.793 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 0, 2024-09-11T23:32:56.793 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-09-11T23:32:56.793 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.793 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.794 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.794 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-09-11T23:32:56.794 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-09-11T23:32:56.794 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-09-11T23:32:56.794 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-09-11T23:32:56.794 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 0, 2024-09-11T23:32:56.794 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-09-11T23:32:56.794 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-09-11T23:32:56.794 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 15, 2024-09-11T23:32:56.794 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 0, 2024-09-11T23:32:56.794 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 15, 2024-09-11T23:32:56.794 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-09-11T23:32:56.795 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 15, 2024-09-11T23:32:56.795 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-09-11T23:32:56.795 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 0, 2024-09-11T23:32:56.795 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 0, 2024-09-11T23:32:56.795 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 1, 2024-09-11T23:32:56.795 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 0, 2024-09-11T23:32:56.795 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 0, 2024-09-11T23:32:56.795 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-09-11T23:32:56.795 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2024-09-11T23:32:56.795 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:32:56.796 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-09-11T23:32:56.796 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2024-09-11T23:32:56.796 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-09-11T23:32:56.796 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-09-11T23:32:56.796 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-09-11T23:32:56.796 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-09-11T23:32:56.796 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-09-11T23:32:56.796 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-09-11T23:32:56.796 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-09-11T23:32:56.797 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:32:56.797 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-09-11T23:32:56.797 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 0, 2024-09-11T23:32:56.797 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 0, 2024-09-11T23:32:56.797 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-09-11T23:32:56.797 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:32:56.797 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-09-11T23:32:56.797 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:32:56.797 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:32:56.797 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:32:56.797 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:32:56.797 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:32:56.797 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-09-11T23:32:56.797 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:32:56.798 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T23:32:56.798 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:32:56.798 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:32:56.798 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:32:56.798 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:32:56.798 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 0, 2024-09-11T23:32:56.798 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:32:56.798 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 0, 2024-09-11T23:32:56.798 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 0 2024-09-11T23:32:56.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.798 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-09-11T23:32:56.798 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-09-11T23:32:56.798 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-09-11T23:32:56.799 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-09-11T23:32:56.799 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 0, 2024-09-11T23:32:56.799 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-09-11T23:32:56.799 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-09-11T23:32:56.799 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-09-11T23:32:56.799 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:32:56.799 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 0, 2024-09-11T23:32:56.799 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 0, 2024-09-11T23:32:56.799 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-09-11T23:32:56.799 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-09-11T23:32:56.799 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-09-11T23:32:56.799 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:32:56.799 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:32:56.799 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-09-11T23:32:56.800 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-09-11T23:32:56.800 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 0, 2024-09-11T23:32:56.800 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 0, 2024-09-11T23:32:56.800 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:32:56.800 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-09-11T23:32:56.800 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-09-11T23:32:56.800 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:32:56.800 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-09-11T23:32:56.800 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-09-11T23:32:56.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.800 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-09-11T23:32:56.800 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-09-11T23:32:56.800 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 0, 2024-09-11T23:32:56.801 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-09-11T23:32:56.801 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-09-11T23:32:56.801 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 43, 2024-09-11T23:32:56.801 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-09-11T23:32:56.801 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-09-11T23:32:56.801 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 0, 2024-09-11T23:32:56.801 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-09-11T23:32:56.801 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-09-11T23:32:56.801 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 2, 2024-09-11T23:32:56.801 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 2, 2024-09-11T23:32:56.801 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-09-11T23:32:56.801 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-09-11T23:32:56.801 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-09-11T23:32:56.802 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 4279711, 2024-09-11T23:32:56.802 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 4276824, 2024-09-11T23:32:56.802 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-09-11T23:32:56.802 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.802 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.802 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.802 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 43 2024-09-11T23:32:56.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.802 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-09-11T23:32:56.802 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 10, 2024-09-11T23:32:56.802 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 23, 2024-09-11T23:32:56.802 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 13, 2024-09-11T23:32:56.802 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 6, 2024-09-11T23:32:56.803 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 18, 2024-09-11T23:32:56.803 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 12, 2024-09-11T23:32:56.803 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 15, 2024-09-11T23:32:56.803 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 35, 2024-09-11T23:32:56.803 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 20, 2024-09-11T23:32:56.803 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 0, 2024-09-11T23:32:56.803 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 0, 2024-09-11T23:32:56.803 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 0, 2024-09-11T23:32:56.803 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 49112, 2024-09-11T23:32:56.803 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 215328 2024-09-11T23:32:56.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.803 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-09-11T23:32:56.803 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 0, 2024-09-11T23:32:56.803 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-09-11T23:32:56.804 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 0, 2024-09-11T23:32:56.804 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 0, 2024-09-11T23:32:56.804 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-09-11T23:32:56.804 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-09-11T23:32:56.804 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.804 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.804 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.804 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-09-11T23:32:56.804 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.804 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.805 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.805 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-09-11T23:32:56.805 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.805 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.805 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.805 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-09-11T23:32:56.805 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.805 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.805 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.805 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-09-11T23:32:56.806 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.806 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.806 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.806 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-09-11T23:32:56.806 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.806 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.806 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.806 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-09-11T23:32:56.807 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.807 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.807 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.807 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-09-11T23:32:56.807 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.807 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.807 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.807 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-09-11T23:32:56.807 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.807 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.807 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.808 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-09-11T23:32:56.808 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.808 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.808 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.808 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-09-11T23:32:56.808 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.808 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.808 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.808 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-09-11T23:32:56.808 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.809 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.809 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.809 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-09-11T23:32:56.809 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.809 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.810 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.810 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-09-11T23:32:56.810 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.810 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.810 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.810 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-09-11T23:32:56.810 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.810 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.810 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.810 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-09-11T23:32:56.811 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.811 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.811 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.811 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-09-11T23:32:56.811 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.811 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.811 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.811 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-09-11T23:32:56.811 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.811 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.811 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.812 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-09-11T23:32:56.812 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.812 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.812 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.812 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-09-11T23:32:56.812 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.812 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.813 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.813 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-09-11T23:32:56.813 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.813 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.813 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.813 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-09-11T23:32:56.813 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.813 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.813 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.813 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-09-11T23:32:56.814 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.814 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.814 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.814 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-09-11T23:32:56.814 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.814 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.814 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.814 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-09-11T23:32:56.814 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.814 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.814 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.815 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-09-11T23:32:56.815 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.815 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.815 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.815 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-09-11T23:32:56.815 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.815 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-09-11T23:32:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-09-11T23:32:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:32:56.817 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:32:56.817 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-09-11T23:32:56.817 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.817 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.817 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.817 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:56.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.817 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-09-11T23:32:56.817 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 1, 2024-09-11T23:32:56.817 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 5, 2024-09-11T23:32:56.817 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 4, 2024-09-11T23:32:56.817 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 4, 2024-09-11T23:32:56.817 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-09-11T23:32:56.818 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 0, 2024-09-11T23:32:56.818 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 0, 2024-09-11T23:32:56.818 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 5736, 2024-09-11T23:32:56.818 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-09-11T23:32:56.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.818 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-09-11T23:32:56.818 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:32:56.818 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-09-11T23:32:56.818 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:32:56.818 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:32:56.818 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:32:56.818 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:32:56.818 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:32:56.819 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:32:56.819 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:32:56.819 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:32:56.819 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:32:56.819 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:32:56.819 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:32:56.819 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:32:56.819 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:32:56.819 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-09-11T23:32:56.819 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:32:56.819 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-09-11T23:32:56.819 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:32:56.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:32:56.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:32:56.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:32:56.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:32:56.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:32:56.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:32:56.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-09-11T23:32:56.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:32:56.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:32:56.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:32:56.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-09-11T23:32:56.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-09-11T23:32:56.820 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-09-11T23:32:56.821 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:32:56.821 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:32:56.821 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:32:56.821 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:32:56.821 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 189867, 2024-09-11T23:32:56.821 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 63, 2024-09-11T23:32:56.821 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:32:56.821 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-09-11T23:32:56.821 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-09-11T23:32:56.821 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-09-11T23:32:56.821 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:32:56.821 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-09-11T23:32:56.821 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:32:56.821 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-09-11T23:32:56.822 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 8320, 2024-09-11T23:32:56.822 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 183, 2024-09-11T23:32:56.822 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:32:56.822 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:32:56.822 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-09-11T23:32:56.822 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-09-11T23:32:56.822 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 49438, 2024-09-11T23:32:56.822 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 730, 2024-09-11T23:32:56.822 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-09-11T23:32:56.823 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-09-11T23:32:56.823 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-09-11T23:32:56.823 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-09-11T23:32:56.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.823 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-09-11T23:32:56.823 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-09-11T23:32:56.823 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-09-11T23:32:56.823 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 2143, 2024-09-11T23:32:56.823 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 21, 2024-09-11T23:32:56.823 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 3, 2024-09-11T23:32:56.823 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 2143, 2024-09-11T23:32:56.823 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-09-11T23:32:56.823 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2143, 2024-09-11T23:32:56.824 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 34.259459079, 2024-09-11T23:32:56.824 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.015986681 2024-09-11T23:32:56.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.824 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-09-11T23:32:56.824 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-09-11T23:32:56.824 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2143, 2024-09-11T23:32:56.824 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2144 2024-09-11T23:32:56.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.824 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 2143, 2024-09-11T23:32:56.824 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 2143, 2024-09-11T23:32:56.824 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 0, 2024-09-11T23:32:56.824 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-09-11T23:32:56.824 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-09-11T23:32:56.825 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 1388, 2024-09-11T23:32:56.825 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 0, 2024-09-11T23:32:56.825 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 753, 2024-09-11T23:32:56.825 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 0, 2024-09-11T23:32:56.825 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 0, 2024-09-11T23:32:56.825 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-09-11T23:32:56.825 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-09-11T23:32:56.825 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-09-11T23:32:56.825 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-09-11T23:32:56.825 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 0, 2024-09-11T23:32:56.825 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-09-11T23:32:56.825 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-09-11T23:32:56.825 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-09-11T23:32:56.826 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 1, 2024-09-11T23:32:56.826 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 0, 2024-09-11T23:32:56.826 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:32:56.826 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-09-11T23:32:56.826 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:32:56.826 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-09-11T23:32:56.826 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-09-11T23:32:56.827 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-09-11T23:32:56.827 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:32:56.827 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-09-11T23:32:56.827 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:32:56.827 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-09-11T23:32:56.827 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-09-11T23:32:56.827 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-09-11T23:32:56.827 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-09-11T23:32:56.827 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-09-11T23:32:56.827 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-09-11T23:32:56.827 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-09-11T23:32:56.828 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-09-11T23:32:56.828 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-09-11T23:32:56.828 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-09-11T23:32:56.828 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-09-11T23:32:56.828 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-09-11T23:32:56.829 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-09-11T23:32:56.829 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-09-11T23:32:56.829 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-09-11T23:32:56.829 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-09-11T23:32:56.829 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-09-11T23:32:56.829 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 94, 2024-09-11T23:32:56.829 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-09-11T23:32:56.829 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 12, 2024-09-11T23:32:56.829 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 5, 2024-09-11T23:32:56.829 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 5, 2024-09-11T23:32:56.829 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-09-11T23:32:56.829 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-09-11T23:32:56.829 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 0, 2024-09-11T23:32:56.830 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 2, 2024-09-11T23:32:56.830 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 0 2024-09-11T23:32:56.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.830 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-09-11T23:32:56.830 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 0, 2024-09-11T23:32:56.830 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:32:56.830 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-09-11T23:32:56.830 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-09-11T23:32:56.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.830 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-09-11T23:32:56.830 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-09-11T23:32:56.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.830 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-09-11T23:32:56.831 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-09-11T23:32:56.831 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:32:56.831 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-09-11T23:32:56.831 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 0, 2024-09-11T23:32:56.831 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 0, 2024-09-11T23:32:56.831 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 0, 2024-09-11T23:32:56.831 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-09-11T23:32:56.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.831 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:32:56.831 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-11T23:32:56.831 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-09-11T23:32:56.831 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-11T23:32:56.831 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 2402, 2024-09-11T23:32:56.831 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 2116974, 2024-09-11T23:32:56.832 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:56.832 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 2402, 2024-09-11T23:32:56.832 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-11T23:32:56.832 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-11T23:32:56.832 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2402, 2024-09-11T23:32:56.832 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 2116974, 2024-09-11T23:32:56.832 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-11T23:32:56.832 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.832 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.832 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.832 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:56.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.832 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-09-11T23:32:56.833 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-11T23:32:56.833 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-09-11T23:32:56.833 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-11T23:32:56.833 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-11T23:32:56.833 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-11T23:32:56.833 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:56.833 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-11T23:32:56.833 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 2143, 2024-09-11T23:32:56.833 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 85412, 2024-09-11T23:32:56.833 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2143, 2024-09-11T23:32:56.833 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 85412, 2024-09-11T23:32:56.833 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-11T23:32:56.833 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.834 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.834 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.834 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:56.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.834 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-09-11T23:32:56.834 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-11T23:32:56.834 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-09-11T23:32:56.834 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-11T23:32:56.834 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-11T23:32:56.834 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-11T23:32:56.834 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:56.834 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-11T23:32:56.834 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 2143, 2024-09-11T23:32:56.834 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 2143, 2024-09-11T23:32:56.835 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2143, 2024-09-11T23:32:56.835 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 2143, 2024-09-11T23:32:56.836 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-11T23:32:56.836 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.836 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.836 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.836 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:56.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.836 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:32:56.836 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-11T23:32:56.836 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-09-11T23:32:56.836 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-11T23:32:56.836 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-11T23:32:56.836 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-11T23:32:56.836 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:56.837 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-11T23:32:56.837 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-11T23:32:56.837 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-11T23:32:56.837 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-09-11T23:32:56.837 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-09-11T23:32:56.837 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-11T23:32:56.837 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.837 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.837 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.837 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:56.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:56.837 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x55cfe4ce86e0": { 2024-09-11T23:32:56.837 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-11T23:32:56.838 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-09-11T23:32:56.838 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-11T23:32:56.838 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-11T23:32:56.838 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-11T23:32:56.838 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:56.838 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-11T23:32:56.838 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-11T23:32:56.838 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-11T23:32:56.838 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-09-11T23:32:56.838 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-09-11T23:32:56.838 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-11T23:32:56.838 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:56.838 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:56.839 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:56.839 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:56.839 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:56.839 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-11T23:32:57.700 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-11T23:32:57.701 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-11T23:32:57.701 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-11T23:32:57.701 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-11T23:32:57.701 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-09-11T23:32:57.701 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:32:57.701 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:32:57.701 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-09-11T23:32:57.701 DEBUG:tasks.check_counter:Getting stats from i 2024-09-11T23:32:57.701 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-11T23:32:57.945 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:32:58.352 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:32:58 smithi097 ceph-mon[35562]: pgmap v345: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T23:32:58.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:32:58 smithi043 ceph-mon[30950]: pgmap v345: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T23:32:58.453 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-11T23:32:58.453 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:32:58.453 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 185, 2024-09-11T23:32:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 243, 2024-09-11T23:32:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 51656, 2024-09-11T23:32:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 311242, 2024-09-11T23:32:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 23, 2024-09-11T23:32:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 4, 2024-09-11T23:32:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.029971804, 2024-09-11T23:32:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.006832906, 2024-09-11T23:32:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.018046653, 2024-09-11T23:32:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.001829847, 2024-09-11T23:32:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 242, 2024-09-11T23:32:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.045695662, 2024-09-11T23:32:58.454 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000188825 2024-09-11T23:32:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 258, 2024-09-11T23:32:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000050563, 2024-09-11T23:32:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000195 2024-09-11T23:32:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 5584, 2024-09-11T23:32:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 2864 2024-09-11T23:32:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:32:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 116, 2024-09-11T23:32:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 336, 2024-09-11T23:32:58.455 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 42145, 2024-09-11T23:32:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 567865, 2024-09-11T23:32:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 12, 2024-09-11T23:32:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 5, 2024-09-11T23:32:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.021409509, 2024-09-11T23:32:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.009653375, 2024-09-11T23:32:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.008678724, 2024-09-11T23:32:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.001537416, 2024-09-11T23:32:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 333, 2024-09-11T23:32:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.041412696, 2024-09-11T23:32:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000124362 2024-09-11T23:32:58.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 204, 2024-09-11T23:32:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000042870, 2024-09-11T23:32:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000210 2024-09-11T23:32:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 2976, 2024-09-11T23:32:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 455856 2024-09-11T23:32:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.457 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:32:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 4568, 2024-09-11T23:32:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 2609, 2024-09-11T23:32:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 3362545, 2024-09-11T23:32:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 883855, 2024-09-11T23:32:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 17, 2024-09-11T23:32:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 5, 2024-09-11T23:32:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.256666279, 2024-09-11T23:32:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.102161589, 2024-09-11T23:32:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.213310643, 2024-09-11T23:32:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.004322464, 2024-09-11T23:32:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:32:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2608, 2024-09-11T23:32:58.458 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.130585359, 2024-09-11T23:32:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000050071 2024-09-11T23:32:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:32:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 6365, 2024-09-11T23:32:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.001055490, 2024-09-11T23:32:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000165 2024-09-11T23:32:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1665680, 2024-09-11T23:32:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 143296 2024-09-11T23:32:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-09-11T23:32:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-09-11T23:32:58.459 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-09-11T23:32:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-09-11T23:32:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-09-11T23:32:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-09-11T23:32:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 121, 2024-09-11T23:32:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.019628775, 2024-09-11T23:32:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000162221 2024-09-11T23:32:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-09-11T23:32:58.460 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-09-11T23:32:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-09-11T23:32:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 17, 2024-09-11T23:32:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000389173, 2024-09-11T23:32:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000022892 2024-09-11T23:32:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-09-11T23:32:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 73, 2024-09-11T23:32:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 31, 2024-09-11T23:32:58.461 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-09-11T23:32:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 31, 2024-09-11T23:32:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.073918267, 2024-09-11T23:32:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.002384460 2024-09-11T23:32:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-09-11T23:32:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 42, 2024-09-11T23:32:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-09-11T23:32:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-09-11T23:32:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 11, 2024-09-11T23:32:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-09-11T23:32:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-09-11T23:32:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 30, 2024-09-11T23:32:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 2, 2024-09-11T23:32:58.462 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 12, 2024-09-11T23:32:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 16, 2024-09-11T23:32:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 25, 2024-09-11T23:32:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 175, 2024-09-11T23:32:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 0, 2024-09-11T23:32:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 0, 2024-09-11T23:32:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 4, 2024-09-11T23:32:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 291, 2024-09-11T23:32:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 140, 2024-09-11T23:32:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 41, 2024-09-11T23:32:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 90, 2024-09-11T23:32:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:32:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-09-11T23:32:58.463 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2024-09-11T23:32:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-09-11T23:32:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-09-11T23:32:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 1, 2024-09-11T23:32:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 6, 2024-09-11T23:32:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 1, 2024-09-11T23:32:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 5, 2024-09-11T23:32:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-09-11T23:32:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 18, 2024-09-11T23:32:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-09-11T23:32:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 4, 2024-09-11T23:32:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 234, 2024-09-11T23:32:58.464 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 6, 2024-09-11T23:32:58.465 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 18, 2024-09-11T23:32:58.465 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-09-11T23:32:58.465 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:32:58.465 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:32:58.465 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 12, 2024-09-11T23:32:58.465 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:32:58.465 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 12, 2024-09-11T23:32:58.465 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 6, 2024-09-11T23:32:58.465 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:32:58.465 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 21, 2024-09-11T23:32:58.465 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 26, 2024-09-11T23:32:58.465 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:32:58.465 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:32:58.465 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:32:58.466 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 23, 2024-09-11T23:32:58.466 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 2, 2024-09-11T23:32:58.466 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 2, 2024-09-11T23:32:58.466 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 25 2024-09-11T23:32:58.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.466 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-09-11T23:32:58.466 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 14, 2024-09-11T23:32:58.466 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 269, 2024-09-11T23:32:58.466 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 179, 2024-09-11T23:32:58.466 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 175, 2024-09-11T23:32:58.466 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 26, 2024-09-11T23:32:58.466 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 4, 2024-09-11T23:32:58.466 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-09-11T23:32:58.467 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:32:58.467 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 6, 2024-09-11T23:32:58.467 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 2, 2024-09-11T23:32:58.467 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-09-11T23:32:58.467 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-09-11T23:32:58.467 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-09-11T23:32:58.467 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:32:58.467 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:32:58.467 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-09-11T23:32:58.467 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-09-11T23:32:58.467 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 5, 2024-09-11T23:32:58.467 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 282, 2024-09-11T23:32:58.468 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:32:58.468 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 1, 2024-09-11T23:32:58.468 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-09-11T23:32:58.468 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:32:58.468 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-09-11T23:32:58.468 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-09-11T23:32:58.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.468 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-09-11T23:32:58.468 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-09-11T23:32:58.468 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 43, 2024-09-11T23:32:58.468 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-09-11T23:32:58.468 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-09-11T23:32:58.468 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 43, 2024-09-11T23:32:58.469 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-09-11T23:32:58.469 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-09-11T23:32:58.469 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 2, 2024-09-11T23:32:58.469 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-09-11T23:32:58.469 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-09-11T23:32:58.469 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 2, 2024-09-11T23:32:58.469 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 2, 2024-09-11T23:32:58.469 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-09-11T23:32:58.469 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-09-11T23:32:58.470 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-09-11T23:32:58.470 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 4279711, 2024-09-11T23:32:58.470 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 0, 2024-09-11T23:32:58.470 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-09-11T23:32:58.470 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 30, 2024-09-11T23:32:58.470 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.411031792, 2024-09-11T23:32:58.470 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.013701059 2024-09-11T23:32:58.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.470 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 0 2024-09-11T23:32:58.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.470 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-09-11T23:32:58.470 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 28, 2024-09-11T23:32:58.471 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 116, 2024-09-11T23:32:58.471 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 88, 2024-09-11T23:32:58.471 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 25, 2024-09-11T23:32:58.471 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 139, 2024-09-11T23:32:58.471 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 114, 2024-09-11T23:32:58.471 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 30, 2024-09-11T23:32:58.471 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 213, 2024-09-11T23:32:58.471 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 183, 2024-09-11T23:32:58.471 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 0, 2024-09-11T23:32:58.471 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 14, 2024-09-11T23:32:58.471 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 14, 2024-09-11T23:32:58.471 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 56488, 2024-09-11T23:32:58.472 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 223520 2024-09-11T23:32:58.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.472 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-09-11T23:32:58.472 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 103, 2024-09-11T23:32:58.472 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 5, 2024-09-11T23:32:58.472 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 73, 2024-09-11T23:32:58.473 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 24, 2024-09-11T23:32:58.473 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 20, 2024-09-11T23:32:58.473 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-09-11T23:32:58.473 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2, 2024-09-11T23:32:58.473 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.001391732, 2024-09-11T23:32:58.473 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000695866 2024-09-11T23:32:58.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.473 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-09-11T23:32:58.473 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2, 2024-09-11T23:32:58.473 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.001283828, 2024-09-11T23:32:58.474 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000641914 2024-09-11T23:32:58.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.474 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-09-11T23:32:58.474 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.474 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.474 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.474 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-09-11T23:32:58.474 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.474 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.474 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.474 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-09-11T23:32:58.475 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 8, 2024-09-11T23:32:58.475 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.001720773, 2024-09-11T23:32:58.475 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000215096 2024-09-11T23:32:58.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.475 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-09-11T23:32:58.475 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.475 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.475 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.475 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-09-11T23:32:58.475 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.475 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.475 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.476 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-09-11T23:32:58.476 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.476 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.476 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.476 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-09-11T23:32:58.476 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.476 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.477 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.477 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-09-11T23:32:58.477 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.477 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.477 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.477 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-09-11T23:32:58.477 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.477 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.477 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.479 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-09-11T23:32:58.479 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2, 2024-09-11T23:32:58.479 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.016026188, 2024-09-11T23:32:58.479 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.008013094 2024-09-11T23:32:58.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.479 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-09-11T23:32:58.479 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.479 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.480 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.480 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-09-11T23:32:58.480 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.480 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.480 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.480 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-09-11T23:32:58.480 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.480 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.480 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.481 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-09-11T23:32:58.481 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 9, 2024-09-11T23:32:58.481 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.003677347, 2024-09-11T23:32:58.481 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000408594 2024-09-11T23:32:58.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.481 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-09-11T23:32:58.481 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2, 2024-09-11T23:32:58.482 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.001632702, 2024-09-11T23:32:58.482 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000816351 2024-09-11T23:32:58.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.482 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-09-11T23:32:58.482 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.482 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.482 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.482 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-09-11T23:32:58.482 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.482 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.482 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.483 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-09-11T23:32:58.483 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 4, 2024-09-11T23:32:58.483 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.045864807, 2024-09-11T23:32:58.483 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.011466201 2024-09-11T23:32:58.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.483 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-09-11T23:32:58.483 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.483 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.483 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.484 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-09-11T23:32:58.484 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.484 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.484 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.484 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-09-11T23:32:58.485 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.485 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.485 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.485 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-09-11T23:32:58.485 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.485 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.485 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.485 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-09-11T23:32:58.485 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.486 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.486 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.486 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-09-11T23:32:58.486 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.486 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.486 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.487 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-09-11T23:32:58.487 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.487 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.487 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.487 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-09-11T23:32:58.487 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.487 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.488 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.488 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-09-11T23:32:58.488 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2, 2024-09-11T23:32:58.488 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.002320890, 2024-09-11T23:32:58.488 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001160445 2024-09-11T23:32:58.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.489 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:32:58.489 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:32:58.489 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-09-11T23:32:58.489 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.489 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.489 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.489 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:58.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.489 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-09-11T23:32:58.489 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 1, 2024-09-11T23:32:58.489 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 5, 2024-09-11T23:32:58.489 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 4, 2024-09-11T23:32:58.490 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 1, 2024-09-11T23:32:58.490 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-09-11T23:32:58.490 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 12, 2024-09-11T23:32:58.490 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 12, 2024-09-11T23:32:58.490 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 580, 2024-09-11T23:32:58.490 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-09-11T23:32:58.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.491 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-09-11T23:32:58.491 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:32:58.491 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-09-11T23:32:58.491 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:32:58.491 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:32:58.491 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:32:58.491 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:32:58.491 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:32:58.491 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:32:58.491 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:32:58.491 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:32:58.491 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:32:58.491 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:32:58.492 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:32:58.492 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:32:58.492 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:32:58.492 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-09-11T23:32:58.492 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:32:58.492 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-09-11T23:32:58.492 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:32:58.492 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:32:58.492 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:32:58.492 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:32:58.492 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:32:58.492 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:32:58.493 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:32:58.493 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-09-11T23:32:58.493 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:32:58.493 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:32:58.493 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:32:58.493 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-09-11T23:32:58.493 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-09-11T23:32:58.493 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-09-11T23:32:58.493 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:32:58.493 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:32:58.493 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:32:58.493 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:32:58.493 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 295477, 2024-09-11T23:32:58.494 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 103, 2024-09-11T23:32:58.494 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:32:58.494 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-09-11T23:32:58.494 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-09-11T23:32:58.494 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-09-11T23:32:58.494 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:32:58.494 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-09-11T23:32:58.494 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:32:58.494 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-09-11T23:32:58.494 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 8320, 2024-09-11T23:32:58.494 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 183, 2024-09-11T23:32:58.494 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:32:58.494 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:32:58.494 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-09-11T23:32:58.495 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-09-11T23:32:58.495 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 131212, 2024-09-11T23:32:58.495 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 1213, 2024-09-11T23:32:58.495 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-09-11T23:32:58.495 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-09-11T23:32:58.495 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-09-11T23:32:58.495 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-09-11T23:32:58.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.495 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-09-11T23:32:58.495 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-09-11T23:32:58.495 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-09-11T23:32:58.495 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 108, 2024-09-11T23:32:58.495 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 98949, 2024-09-11T23:32:58.495 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 28, 2024-09-11T23:32:58.496 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 108, 2024-09-11T23:32:58.496 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-09-11T23:32:58.496 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 108, 2024-09-11T23:32:58.496 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1.690973174, 2024-09-11T23:32:58.496 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.015657159 2024-09-11T23:32:58.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.496 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-09-11T23:32:58.496 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-09-11T23:32:58.496 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 108, 2024-09-11T23:32:58.496 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 140 2024-09-11T23:32:58.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.496 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 108, 2024-09-11T23:32:58.496 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 19, 2024-09-11T23:32:58.497 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 89, 2024-09-11T23:32:58.497 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-09-11T23:32:58.497 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-09-11T23:32:58.497 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 0, 2024-09-11T23:32:58.497 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 10, 2024-09-11T23:32:58.497 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 1, 2024-09-11T23:32:58.497 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 31, 2024-09-11T23:32:58.497 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 13, 2024-09-11T23:32:58.497 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-09-11T23:32:58.497 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-09-11T23:32:58.497 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-09-11T23:32:58.497 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-09-11T23:32:58.497 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 14, 2024-09-11T23:32:58.498 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-09-11T23:32:58.498 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-09-11T23:32:58.498 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-09-11T23:32:58.498 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 18, 2024-09-11T23:32:58.498 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 20, 2024-09-11T23:32:58.498 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:32:58.498 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-09-11T23:32:58.498 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:32:58.498 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-09-11T23:32:58.498 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-09-11T23:32:58.498 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-09-11T23:32:58.498 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:32:58.498 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-09-11T23:32:58.499 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:32:58.499 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-09-11T23:32:58.499 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-09-11T23:32:58.499 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-09-11T23:32:58.499 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-09-11T23:32:58.499 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-09-11T23:32:58.499 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-09-11T23:32:58.500 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-09-11T23:32:58.500 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-09-11T23:32:58.500 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-09-11T23:32:58.500 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-09-11T23:32:58.500 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-09-11T23:32:58.500 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-09-11T23:32:58.500 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-09-11T23:32:58.500 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-09-11T23:32:58.500 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-09-11T23:32:58.500 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-09-11T23:32:58.501 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-09-11T23:32:58.501 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 94, 2024-09-11T23:32:58.501 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-09-11T23:32:58.501 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 12, 2024-09-11T23:32:58.501 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 9, 2024-09-11T23:32:58.501 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 9, 2024-09-11T23:32:58.501 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-09-11T23:32:58.501 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-09-11T23:32:58.501 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 27, 2024-09-11T23:32:58.501 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 0, 2024-09-11T23:32:58.501 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 6 2024-09-11T23:32:58.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.501 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-09-11T23:32:58.501 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 1, 2024-09-11T23:32:58.502 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:32:58.502 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-09-11T23:32:58.502 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-09-11T23:32:58.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.502 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-09-11T23:32:58.502 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 287 2024-09-11T23:32:58.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.502 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-09-11T23:32:58.502 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-09-11T23:32:58.503 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 2, 2024-09-11T23:32:58.503 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-09-11T23:32:58.503 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 2, 2024-09-11T23:32:58.503 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 2, 2024-09-11T23:32:58.503 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 2, 2024-09-11T23:32:58.503 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-09-11T23:32:58.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.503 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:32:58.503 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-11T23:32:58.503 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-09-11T23:32:58.503 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-11T23:32:58.503 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 4865, 2024-09-11T23:32:58.504 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 2999843, 2024-09-11T23:32:58.504 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:58.504 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 4865, 2024-09-11T23:32:58.504 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-11T23:32:58.504 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-11T23:32:58.504 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 4865, 2024-09-11T23:32:58.504 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 2999843, 2024-09-11T23:32:58.504 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-11T23:32:58.504 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.504 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.504 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.504 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:58.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.505 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-09-11T23:32:58.505 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-11T23:32:58.505 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-09-11T23:32:58.505 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-11T23:32:58.505 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-11T23:32:58.505 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-11T23:32:58.505 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:58.505 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-11T23:32:58.505 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 108, 2024-09-11T23:32:58.505 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 99247, 2024-09-11T23:32:58.505 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 108, 2024-09-11T23:32:58.506 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 99247, 2024-09-11T23:32:58.506 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-11T23:32:58.506 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.506 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.506 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.506 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:58.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.506 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-09-11T23:32:58.506 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-11T23:32:58.506 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-09-11T23:32:58.506 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-11T23:32:58.506 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-09-11T23:32:58.506 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-09-11T23:32:58.506 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:58.507 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-09-11T23:32:58.507 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 108, 2024-09-11T23:32:58.507 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 108, 2024-09-11T23:32:58.507 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 108, 2024-09-11T23:32:58.507 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 108, 2024-09-11T23:32:58.507 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-11T23:32:58.507 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.507 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.507 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.507 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:58.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.507 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:32:58.507 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-11T23:32:58.508 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-09-11T23:32:58.508 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-11T23:32:58.508 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 2, 2024-09-11T23:32:58.508 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 298, 2024-09-11T23:32:58.508 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:58.508 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 2, 2024-09-11T23:32:58.508 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-11T23:32:58.508 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-11T23:32:58.508 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 1, 2024-09-11T23:32:58.508 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 298, 2024-09-11T23:32:58.508 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-11T23:32:58.508 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.508 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.509 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.509 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:58.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:32:58.509 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x558302d68500": { 2024-09-11T23:32:58.509 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-11T23:32:58.509 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-09-11T23:32:58.509 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-11T23:32:58.509 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 43, 2024-09-11T23:32:58.509 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 85407, 2024-09-11T23:32:58.509 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-11T23:32:58.509 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 43, 2024-09-11T23:32:58.509 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-11T23:32:58.509 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-11T23:32:58.510 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 30, 2024-09-11T23:32:58.510 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 85407, 2024-09-11T23:32:58.510 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-11T23:32:58.510 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:32:58.510 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:32:58.510 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:32:58.510 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:58.510 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:32:58.510 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-11T23:32:58.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:32:58 smithi090 ceph-mon[35130]: pgmap v345: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T23:32:59.269 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-09-11T23:32:59.269 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-09-11T23:32:59.269 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=14 2024-09-11T23:32:59.269 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=269 2024-09-11T23:32:59.270 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-11T23:32:59.270 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:32:59.270 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:32:59.270 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-09-11T23:32:59.270 DEBUG:tasks.check_counter:Getting stats from l 2024-09-11T23:32:59.270 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-11T23:32:59.513 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:33:00.063 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-11T23:33:00.064 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:33:00.064 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 4, 2024-09-11T23:33:00.064 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 4, 2024-09-11T23:33:00.064 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 3072, 2024-09-11T23:33:00.064 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 448, 2024-09-11T23:33:00.064 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2024-09-11T23:33:00.064 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 0, 2024-09-11T23:33:00.064 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.003036802, 2024-09-11T23:33:00.064 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.000040598, 2024-09-11T23:33:00.064 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.002183907, 2024-09-11T23:33:00.064 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:33:00.065 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:33:00.065 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3, 2024-09-11T23:33:00.065 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.003955535, 2024-09-11T23:33:00.065 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001318511 2024-09-11T23:33:00.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:33:00.065 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:33:00.065 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:33:00.065 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:33:00.065 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:33:00.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:33:00.065 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-11T23:33:00.065 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T23:33:00.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:33:00.066 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:33:00.066 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 2, 2024-09-11T23:33:00.066 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 169, 2024-09-11T23:33:00.066 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 320, 2024-09-11T23:33:00.066 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 187072, 2024-09-11T23:33:00.066 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 5, 2024-09-11T23:33:00.066 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2024-09-11T23:33:00.066 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.010933858, 2024-09-11T23:33:00.066 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.007430765, 2024-09-11T23:33:00.066 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.001189470, 2024-09-11T23:33:00.066 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:33:00.066 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:33:00.067 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 166, 2024-09-11T23:33:00.067 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.015571463, 2024-09-11T23:33:00.067 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000093803 2024-09-11T23:33:00.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:33:00.067 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:33:00.067 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:33:00.067 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:33:00.067 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:33:00.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:33:00.067 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T23:33:00.067 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 187072 2024-09-11T23:33:00.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:33:00.067 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:33:00.068 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 232, 2024-09-11T23:33:00.068 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 221, 2024-09-11T23:33:00.068 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 1583760, 2024-09-11T23:33:00.068 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 135744, 2024-09-11T23:33:00.068 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 2, 2024-09-11T23:33:00.068 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 1, 2024-09-11T23:33:00.068 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.040841116, 2024-09-11T23:33:00.068 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.014427440, 2024-09-11T23:33:00.068 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.041078502, 2024-09-11T23:33:00.068 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:33:00.068 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:33:00.068 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 220, 2024-09-11T23:33:00.068 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.012886205, 2024-09-11T23:33:00.069 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000058573 2024-09-11T23:33:00.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:33:00.069 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:33:00.069 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:33:00.069 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:33:00.069 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:33:00.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:33:00.069 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1583760, 2024-09-11T23:33:00.069 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 135744 2024-09-11T23:33:00.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:33:00.069 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-09-11T23:33:00.069 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 0, 2024-09-11T23:33:00.070 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-09-11T23:33:00.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:33:00.070 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-09-11T23:33:00.070 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:33:00.070 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-09-11T23:33:00.070 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:33:00.070 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:33:00.070 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:33:00.070 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:33:00.070 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:33:00.070 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:33:00.070 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:33:00.070 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:33:00.070 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:33:00.071 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:33:00.071 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:33:00.071 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:33:00.071 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:33:00.071 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-09-11T23:33:00.071 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:33:00.071 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-09-11T23:33:00.071 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:33:00.071 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:33:00.071 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:33:00.071 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:33:00.071 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:33:00.071 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:33:00.072 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:33:00.072 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-09-11T23:33:00.072 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:33:00.072 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:33:00.072 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:33:00.072 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-09-11T23:33:00.072 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-09-11T23:33:00.072 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-09-11T23:33:00.072 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:33:00.072 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:33:00.072 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:33:00.072 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:33:00.072 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 148985, 2024-09-11T23:33:00.073 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 30, 2024-09-11T23:33:00.073 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:33:00.073 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-09-11T23:33:00.073 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-09-11T23:33:00.073 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-09-11T23:33:00.073 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:33:00.073 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-09-11T23:33:00.073 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:33:00.073 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-09-11T23:33:00.073 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 0, 2024-09-11T23:33:00.073 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 0, 2024-09-11T23:33:00.073 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:33:00.073 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:33:00.074 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-09-11T23:33:00.074 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-09-11T23:33:00.074 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 928, 2024-09-11T23:33:00.074 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 2, 2024-09-11T23:33:00.074 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-09-11T23:33:00.074 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-09-11T23:33:00.074 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-09-11T23:33:00.074 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-09-11T23:33:00.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-11T23:33:00.074 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:33:00.074 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-09-11T23:33:00.074 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-09-11T23:33:00.074 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-09-11T23:33:00.075 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 234, 2024-09-11T23:33:00.075 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 1551063, 2024-09-11T23:33:00.075 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-09-11T23:33:00.075 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 234, 2024-09-11T23:33:00.075 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-09-11T23:33:00.075 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-09-11T23:33:00.075 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 234, 2024-09-11T23:33:00.075 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 1551063, 2024-09-11T23:33:00.075 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-09-11T23:33:00.075 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-09-11T23:33:00.075 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-09-11T23:33:00.075 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-09-11T23:33:00.075 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:33:00.075 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-11T23:33:00.076 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-11T23:33:00.300 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:00 smithi097 ceph-mon[35562]: pgmap v346: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:33:00.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:00 smithi043 ceph-mon[30950]: pgmap v346: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:33:00.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:00 smithi090 ceph-mon[35130]: pgmap v346: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T23:33:00.753 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-11T23:33:00.753 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-11T23:33:00.753 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-09-11T23:33:00.753 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-09-11T23:33:00.753 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-09-11T23:33:00.753 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-09-11T23:33:00.753 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-09-11T23:33:00.753 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-09-11T23:33:00.753 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-11T23:33:00.769 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-11T23:33:00.770 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T23:33:00.770 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T23:33:00.792 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-11T23:33:00.792 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T23:33:00.850 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-11T23:33:01.149 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:33:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.651+0000 7f0f7308c640 1 -- 172.21.15.43:0/780786185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f6c106a80 msgr2=0x7f0f6c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.651+0000 7f0f7308c640 1 --2- 172.21.15.43:0/780786185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f6c106a80 0x7f0f6c106e60 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009930 tx=0x7f0f5c02a420 comp rx=0 tx=0).stop 2024-09-11T23:33:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.651+0000 7f0f7308c640 1 -- 172.21.15.43:0/780786185 shutdown_connections 2024-09-11T23:33:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.651+0000 7f0f7308c640 1 --2- 172.21.15.43:0/780786185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f6c101420 0x7f0f6c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.651+0000 7f0f7308c640 1 --2- 172.21.15.43:0/780786185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f6c100a80 0x7f0f6c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.651+0000 7f0f7308c640 1 --2- 172.21.15.43:0/780786185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f6c106a80 0x7f0f6c106e60 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.651+0000 7f0f7308c640 1 -- 172.21.15.43:0/780786185 >> 172.21.15.43:0/780786185 conn(0x7f0f6c0fc7a0 msgr2=0x7f0f6c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.651+0000 7f0f7308c640 1 -- 172.21.15.43:0/780786185 shutdown_connections 2024-09-11T23:33:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.651+0000 7f0f7308c640 1 -- 172.21.15.43:0/780786185 wait complete. 2024-09-11T23:33:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.652+0000 7f0f7308c640 1 Processor -- start 2024-09-11T23:33:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.652+0000 7f0f7308c640 1 -- start start 2024-09-11T23:33:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.652+0000 7f0f7308c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f6c100a80 0x7f0f6c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.652+0000 7f0f7308c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f6c101420 0x7f0f6c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.652+0000 7f0f7308c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f6c106a80 0x7f0f6c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:01.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.652+0000 7f0f7308c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c110e40 con 0x7f0f6c101420 2024-09-11T23:33:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.652+0000 7f0f7308c640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c110cc0 con 0x7f0f6c106a80 2024-09-11T23:33:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.652+0000 7f0f7308c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c110fc0 con 0x7f0f6c100a80 2024-09-11T23:33:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.652+0000 7f0f6bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f6c101420 0x7f0f6c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.652+0000 7f0f6bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f6c101420 0x7f0f6c198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53338/0 (socket says 172.21.15.43:53338) 2024-09-11T23:33:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.652+0000 7f0f6bfff640 1 -- 172.21.15.43:0/2800063795 learned_addr learned my addr 172.21.15.43:0/2800063795 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:33:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.652+0000 7f0f71629640 1 --2- 172.21.15.43:0/2800063795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f6c106a80 0x7f0f6c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.652+0000 7f0f70e28640 1 --2- 172.21.15.43:0/2800063795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f6c100a80 0x7f0f6c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.652+0000 7f0f6bfff640 1 -- 172.21.15.43:0/2800063795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f6c100a80 msgr2=0x7f0f6c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.652+0000 7f0f6bfff640 1 --2- 172.21.15.43:0/2800063795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f6c100a80 0x7f0f6c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.652+0000 7f0f6bfff640 1 -- 172.21.15.43:0/2800063795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f6c106a80 msgr2=0x7f0f6c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.652+0000 7f0f6bfff640 1 --2- 172.21.15.43:0/2800063795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f6c106a80 0x7f0f6c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.652+0000 7f0f6bfff640 1 -- 172.21.15.43:0/2800063795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6c19d3f0 con 0x7f0f6c101420 2024-09-11T23:33:01.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.653+0000 7f0f6bfff640 1 --2- 172.21.15.43:0/2800063795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f6c101420 0x7f0f6c198980 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400d950 tx=0x7f0f5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.653+0000 7f0f69ffb640 1 -- 172.21.15.43:0/2800063795 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54017030 con 0x7f0f6c101420 2024-09-11T23:33:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.653+0000 7f0f69ffb640 1 -- 172.21.15.43:0/2800063795 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f54004020 con 0x7f0f6c101420 2024-09-11T23:33:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.653+0000 7f0f7308c640 1 -- 172.21.15.43:0/2800063795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f6c19d6e0 con 0x7f0f6c101420 2024-09-11T23:33:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.653+0000 7f0f69ffb640 1 -- 172.21.15.43:0/2800063795 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f540047a0 con 0x7f0f6c101420 2024-09-11T23:33:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.653+0000 7f0f7308c640 1 -- 172.21.15.43:0/2800063795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f6c1a4fc0 con 0x7f0f6c101420 2024-09-11T23:33:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.654+0000 7f0f7308c640 1 -- 172.21.15.43:0/2800063795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f6c100ee0 con 0x7f0f6c101420 2024-09-11T23:33:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.655+0000 7f0f69ffb640 1 -- 172.21.15.43:0/2800063795 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0f540041d0 con 0x7f0f6c101420 2024-09-11T23:33:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.656+0000 7f0f69ffb640 1 --2- 172.21.15.43:0/2800063795 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f0f580a2cf0 0x7f0f580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.656+0000 7f0f70e28640 1 --2- 172.21.15.43:0/2800063795 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f0f580a2cf0 0x7f0f580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.656+0000 7f0f69ffb640 1 -- 172.21.15.43:0/2800063795 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f0f540cee10 con 0x7f0f6c101420 2024-09-11T23:33:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.656+0000 7f0f70e28640 1 --2- 172.21.15.43:0/2800063795 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f0f580a2cf0 0x7f0f580a51b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009900 tx=0x7f0f5c0058d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.660+0000 7f0f69ffb640 1 -- 172.21.15.43:0/2800063795 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0f540d2030 con 0x7f0f6c101420 2024-09-11T23:33:01.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.846+0000 7f0f7308c640 1 -- 172.21.15.43:0/2800063795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0f6c1997a0 con 0x7f0f6c101420 2024-09-11T23:33:01.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.846+0000 7f0f69ffb640 1 -- 172.21.15.43:0/2800063795 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v94) ==== 81+0+941 (secure 0 0 0) 0x7f0f6c1997a0 con 0x7f0f6c101420 2024-09-11T23:33:01.847 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/2262294782 2024-09-12T23:21:50.205362+0000 2024-09-11T23:33:01.847 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/2398981254 2024-09-12T23:21:50.205362+0000 2024-09-11T23:33:01.847 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/481605249 2024-09-12T23:21:50.205362+0000 2024-09-11T23:33:01.847 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/1375379108 2024-09-12T23:12:35.756598+0000 2024-09-11T23:33:01.847 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/1512201905 2024-09-12T23:12:35.756598+0000 2024-09-11T23:33:01.847 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/206044413 2024-09-12T23:12:35.756598+0000 2024-09-11T23:33:01.847 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/4015351527 2024-09-12T23:12:35.756598+0000 2024-09-11T23:33:01.848 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/1579053924 2024-09-12T23:21:50.205362+0000 2024-09-11T23:33:01.848 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/4015351527 2024-09-12T23:12:35.756598+0000 2024-09-11T23:33:01.848 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/3892590401 2024-09-12T23:13:00.548389+0000 2024-09-11T23:33:01.848 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/3892590401 2024-09-12T23:13:00.548389+0000 2024-09-11T23:33:01.848 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/2370495020 2024-09-12T23:21:50.205362+0000 2024-09-11T23:33:01.848 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/4006124758 2024-09-12T23:13:00.548389+0000 2024-09-11T23:33:01.848 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/76575371 2024-09-12T23:13:00.548389+0000 2024-09-11T23:33:01.848 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/120495444 2024-09-12T23:13:00.548389+0000 2024-09-11T23:33:01.848 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/2370495020 2024-09-12T23:21:50.205362+0000 2024-09-11T23:33:01.849 INFO:teuthology.orchestra.run.smithi043.stderr:listed 16 entries 2024-09-11T23:33:01.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.849+0000 7f0f7308c640 1 -- 172.21.15.43:0/2800063795 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f0f580a2cf0 msgr2=0x7f0f580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:01.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.849+0000 7f0f7308c640 1 --2- 172.21.15.43:0/2800063795 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f0f580a2cf0 0x7f0f580a51b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009900 tx=0x7f0f5c0058d0 comp rx=0 tx=0).stop 2024-09-11T23:33:01.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.849+0000 7f0f7308c640 1 -- 172.21.15.43:0/2800063795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f6c101420 msgr2=0x7f0f6c198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:01.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.849+0000 7f0f7308c640 1 --2- 172.21.15.43:0/2800063795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f6c101420 0x7f0f6c198980 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400d950 tx=0x7f0f5400de10 comp rx=0 tx=0).stop 2024-09-11T23:33:01.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.849+0000 7f0f7308c640 1 -- 172.21.15.43:0/2800063795 shutdown_connections 2024-09-11T23:33:01.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.849+0000 7f0f7308c640 1 --2- 172.21.15.43:0/2800063795 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f0f580a2cf0 0x7f0f580a51b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:01.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.849+0000 7f0f7308c640 1 --2- 172.21.15.43:0/2800063795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f6c106a80 0x7f0f6c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:01.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.849+0000 7f0f7308c640 1 --2- 172.21.15.43:0/2800063795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f6c101420 0x7f0f6c198980 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:01.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.849+0000 7f0f7308c640 1 --2- 172.21.15.43:0/2800063795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f6c100a80 0x7f0f6c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:01.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.849+0000 7f0f7308c640 1 -- 172.21.15.43:0/2800063795 >> 172.21.15.43:0/2800063795 conn(0x7f0f6c0fc7a0 msgr2=0x7f0f6c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:01.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.849+0000 7f0f7308c640 1 -- 172.21.15.43:0/2800063795 shutdown_connections 2024-09-11T23:33:01.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:01.849+0000 7f0f7308c640 1 -- 172.21.15.43:0/2800063795 wait complete. 2024-09-11T23:33:02.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:02 smithi043 ceph-mon[30950]: pgmap v347: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-11T23:33:02.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:02 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2800063795' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T23:33:02.451 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-11T23:33:02.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:33:02.451 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-11T23:33:02.509 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-09-11T23:33:02.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:33:02.510 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T23:33:02.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:02 smithi090 ceph-mon[35130]: pgmap v347: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-11T23:33:02.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:02 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2800063795' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T23:33:02.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:02 smithi097 ceph-mon[35562]: pgmap v347: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-11T23:33:02.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:02 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2800063795' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T23:33:02.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:33:02.562 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-11T23:33:02.623 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T23:33:02.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:33:02.624 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T23:33:02.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:33:02.699 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-09-11T23:33:02.797 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-11T23:33:02.808 INFO:tasks.cephadm:Teardown begin 2024-09-11T23:33:02.809 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T23:33:02.809 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:33:03.125 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:33:03.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.660+0000 7f385fc55640 1 -- 172.21.15.43:0/692432314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3858074f40 msgr2=0x7f385810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.660+0000 7f385fc55640 1 --2- 172.21.15.43:0/692432314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3858074f40 0x7f385810de80 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f3848009900 tx=0x7f384802a480 comp rx=0 tx=0).stop 2024-09-11T23:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.661+0000 7f385fc55640 1 -- 172.21.15.43:0/692432314 shutdown_connections 2024-09-11T23:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.661+0000 7f385fc55640 1 --2- 172.21.15.43:0/692432314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f385810e3c0 0x7f3858110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.661+0000 7f385fc55640 1 --2- 172.21.15.43:0/692432314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3858074f40 0x7f385810de80 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.661+0000 7f385fc55640 1 --2- 172.21.15.43:0/692432314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3858074620 0x7f3858074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.661+0000 7f385fc55640 1 -- 172.21.15.43:0/692432314 >> 172.21.15.43:0/692432314 conn(0x7f38580fd8c0 msgr2=0x7f38580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.661+0000 7f385fc55640 1 -- 172.21.15.43:0/692432314 shutdown_connections 2024-09-11T23:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.661+0000 7f385fc55640 1 -- 172.21.15.43:0/692432314 wait complete. 2024-09-11T23:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.661+0000 7f385fc55640 1 Processor -- start 2024-09-11T23:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.661+0000 7f385fc55640 1 -- start start 2024-09-11T23:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f385fc55640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3858074620 0x7f385819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f385fc55640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3858074f40 0x7f385819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:03.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f385fc55640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f385810e3c0 0x7f385819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:03.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f385fc55640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3858114140 con 0x7f3858074620 2024-09-11T23:33:03.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f385fc55640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3858113fc0 con 0x7f385810e3c0 2024-09-11T23:33:03.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f385fc55640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f38581142c0 con 0x7f3858074f40 2024-09-11T23:33:03.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f385d9f1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3858074620 0x7f385819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:03.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f385d9f1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3858074620 0x7f385819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53358/0 (socket says 172.21.15.43:53358) 2024-09-11T23:33:03.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f385d9f1640 1 -- 172.21.15.43:0/4214688131 learned_addr learned my addr 172.21.15.43:0/4214688131 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:33:03.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f385d1f0640 1 --2- 172.21.15.43:0/4214688131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3858074f40 0x7f385819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:03.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f385e1f2640 1 --2- 172.21.15.43:0/4214688131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f385810e3c0 0x7f385819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:03.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f385d9f1640 1 -- 172.21.15.43:0/4214688131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3858074f40 msgr2=0x7f385819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:03.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f385d9f1640 1 --2- 172.21.15.43:0/4214688131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3858074f40 0x7f385819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:03.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f385d9f1640 1 -- 172.21.15.43:0/4214688131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f385810e3c0 msgr2=0x7f385819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:03.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f385d9f1640 1 --2- 172.21.15.43:0/4214688131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f385810e3c0 0x7f385819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:03.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f385d9f1640 1 -- 172.21.15.43:0/4214688131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385819f5f0 con 0x7f3858074620 2024-09-11T23:33:03.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f385d9f1640 1 --2- 172.21.15.43:0/4214688131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3858074620 0x7f385819a640 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f385400e9f0 tx=0x7f385400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:03.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f384effd640 1 -- 172.21.15.43:0/4214688131 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385400cd80 con 0x7f3858074620 2024-09-11T23:33:03.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f384effd640 1 -- 172.21.15.43:0/4214688131 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3854010910 con 0x7f3858074620 2024-09-11T23:33:03.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f385fc55640 1 -- 172.21.15.43:0/4214688131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385819f8e0 con 0x7f3858074620 2024-09-11T23:33:03.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f384effd640 1 -- 172.21.15.43:0/4214688131 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3854018990 con 0x7f3858074620 2024-09-11T23:33:03.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.662+0000 7f385fc55640 1 -- 172.21.15.43:0/4214688131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38581a71c0 con 0x7f3858074620 2024-09-11T23:33:03.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.664+0000 7f385fc55640 1 -- 172.21.15.43:0/4214688131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3858074a00 con 0x7f3858074620 2024-09-11T23:33:03.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.665+0000 7f384effd640 1 -- 172.21.15.43:0/4214688131 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3854010ac0 con 0x7f3858074620 2024-09-11T23:33:03.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.665+0000 7f384effd640 1 --2- 172.21.15.43:0/4214688131 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f38440a2d70 0x7f38440a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:03.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.666+0000 7f384effd640 1 -- 172.21.15.43:0/4214688131 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f38540cf750 con 0x7f3858074620 2024-09-11T23:33:03.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.666+0000 7f385d1f0640 1 --2- 172.21.15.43:0/4214688131 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f38440a2d70 0x7f38440a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:03.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.666+0000 7f385d1f0640 1 --2- 172.21.15.43:0/4214688131 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f38440a2d70 0x7f38440a5230 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3848009230 tx=0x7f38480058d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:03.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.670+0000 7f384effd640 1 -- 172.21.15.43:0/4214688131 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f385409bc90 con 0x7f3858074620 2024-09-11T23:33:03.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.882+0000 7f385fc55640 1 -- 172.21.15.43:0/4214688131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f385802d7c0 con 0x7f3858074620 2024-09-11T23:33:03.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.883+0000 7f384effd640 1 -- 172.21.15.43:0/4214688131 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11893 (secure 0 0 0) 0x7f385409ff90 con 0x7f3858074620 2024-09-11T23:33:03.883 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:33:03.883 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":34,"btime":"2024-09-11T23:31:56:056494+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:31:55.493313+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25571":{"gid":25571,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/568491708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":568491708},{"type":"v1","addr":"172.21.15.90:6835","nonce":568491708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:33:03.883 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 34 2024-09-11T23:33:03.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.885+0000 7f385fc55640 1 -- 172.21.15.43:0/4214688131 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f38440a2d70 msgr2=0x7f38440a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:03.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.885+0000 7f385fc55640 1 --2- 172.21.15.43:0/4214688131 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f38440a2d70 0x7f38440a5230 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3848009230 tx=0x7f38480058d0 comp rx=0 tx=0).stop 2024-09-11T23:33:03.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.885+0000 7f385fc55640 1 -- 172.21.15.43:0/4214688131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3858074620 msgr2=0x7f385819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:03.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.885+0000 7f385fc55640 1 --2- 172.21.15.43:0/4214688131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3858074620 0x7f385819a640 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f385400e9f0 tx=0x7f385400eeb0 comp rx=0 tx=0).stop 2024-09-11T23:33:03.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.885+0000 7f385fc55640 1 -- 172.21.15.43:0/4214688131 shutdown_connections 2024-09-11T23:33:03.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.885+0000 7f385fc55640 1 --2- 172.21.15.43:0/4214688131 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f38440a2d70 0x7f38440a5230 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:03.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.885+0000 7f385fc55640 1 --2- 172.21.15.43:0/4214688131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f385810e3c0 0x7f385819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:03.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.885+0000 7f385fc55640 1 --2- 172.21.15.43:0/4214688131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3858074f40 0x7f385819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:03.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.885+0000 7f385fc55640 1 --2- 172.21.15.43:0/4214688131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3858074620 0x7f385819a640 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:03.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.885+0000 7f385fc55640 1 -- 172.21.15.43:0/4214688131 >> 172.21.15.43:0/4214688131 conn(0x7f38580fd8c0 msgr2=0x7f38580fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:03.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.885+0000 7f385fc55640 1 -- 172.21.15.43:0/4214688131 shutdown_connections 2024-09-11T23:33:03.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:03.885+0000 7f385fc55640 1 -- 172.21.15.43:0/4214688131 wait complete. 2024-09-11T23:33:04.358 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:04 smithi043 ceph-mon[30950]: pgmap v348: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-11T23:33:04.358 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:04 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/4214688131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:33:04.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:04 smithi090 ceph-mon[35130]: pgmap v348: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-11T23:33:04.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:04 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/4214688131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:33:04.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:04 smithi097 ceph-mon[35562]: pgmap v348: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-11T23:33:04.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:04 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/4214688131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:33:04.707 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:33:04.970 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:33:05.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.550+0000 7fc87effd640 1 -- 172.21.15.43:0/3129384077 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87402f300 con 0x7fc880101450 2024-09-11T23:33:05.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.551+0000 7fc886998640 1 -- 172.21.15.43:0/3129384077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc880101450 msgr2=0x7fc880105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:05.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.551+0000 7fc886998640 1 --2- 172.21.15.43:0/3129384077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc880101450 0x7fc880105920 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc874009860 tx=0x7fc87402a400 comp rx=0 tx=0).stop 2024-09-11T23:33:05.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.555+0000 7fc886998640 1 -- 172.21.15.43:0/3129384077 shutdown_connections 2024-09-11T23:33:05.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.555+0000 7fc886998640 1 --2- 172.21.15.43:0/3129384077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc880105f40 0x7fc8801063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.555+0000 7fc886998640 1 --2- 172.21.15.43:0/3129384077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc880101450 0x7fc880105920 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.555+0000 7fc886998640 1 --2- 172.21.15.43:0/3129384077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc880100b30 0x7fc880100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.555+0000 7fc886998640 1 -- 172.21.15.43:0/3129384077 >> 172.21.15.43:0/3129384077 conn(0x7fc8800fc7a0 msgr2=0x7fc8800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:05.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.558+0000 7fc886998640 1 -- 172.21.15.43:0/3129384077 shutdown_connections 2024-09-11T23:33:05.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.559+0000 7fc886998640 1 -- 172.21.15.43:0/3129384077 wait complete. 2024-09-11T23:33:05.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.560+0000 7fc886998640 1 Processor -- start 2024-09-11T23:33:05.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.560+0000 7fc886998640 1 -- start start 2024-09-11T23:33:05.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.561+0000 7fc886998640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc880100b30 0x7fc8801a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.561+0000 7fc886998640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc880101450 0x7fc8801a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.561+0000 7fc886998640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc880105f40 0x7fc88019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.561+0000 7fc886998640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc880118630 con 0x7fc880100b30 2024-09-11T23:33:05.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.561+0000 7fc886998640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc8801184b0 con 0x7fc880105f40 2024-09-11T23:33:05.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.561+0000 7fc886998640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc8801187b0 con 0x7fc880101450 2024-09-11T23:33:05.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.561+0000 7fc87ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc880100b30 0x7fc8801a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.561+0000 7fc87f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc880101450 0x7fc8801a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.561+0000 7fc87f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc880101450 0x7fc8801a4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.43:46236/0 (socket says 172.21.15.43:46236) 2024-09-11T23:33:05.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.561+0000 7fc87ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc880100b30 0x7fc8801a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53372/0 (socket says 172.21.15.43:53372) 2024-09-11T23:33:05.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.561+0000 7fc87f7fe640 1 -- 172.21.15.43:0/3139477491 learned_addr learned my addr 172.21.15.43:0/3139477491 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:33:05.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.561+0000 7fc884f35640 1 --2- 172.21.15.43:0/3139477491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc880105f40 0x7fc88019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.562+0000 7fc87f7fe640 1 -- 172.21.15.43:0/3139477491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc880105f40 msgr2=0x7fc88019e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:05.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.562+0000 7fc87f7fe640 1 --2- 172.21.15.43:0/3139477491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc880105f40 0x7fc88019e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.562+0000 7fc87f7fe640 1 -- 172.21.15.43:0/3139477491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc880100b30 msgr2=0x7fc8801a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:05.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.562+0000 7fc87f7fe640 1 --2- 172.21.15.43:0/3139477491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc880100b30 0x7fc8801a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:05.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.562+0000 7fc87f7fe640 1 -- 172.21.15.43:0/3139477491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88019f030 con 0x7fc880101450 2024-09-11T23:33:05.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.562+0000 7fc87f7fe640 1 --2- 172.21.15.43:0/3139477491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc880101450 0x7fc8801a4c80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc874030ff0 tx=0x7fc8740309b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:05.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.562+0000 7fc884f35640 1 --2- 172.21.15.43:0/3139477491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc880105f40 0x7fc88019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:33:05.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.564+0000 7fc87d7fa640 1 -- 172.21.15.43:0/3139477491 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874005610 con 0x7fc880101450 2024-09-11T23:33:05.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.564+0000 7fc886998640 1 -- 172.21.15.43:0/3139477491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc88019f2c0 con 0x7fc880101450 2024-09-11T23:33:05.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.564+0000 7fc886998640 1 -- 172.21.15.43:0/3139477491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc88006bc10 con 0x7fc880101450 2024-09-11T23:33:05.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.565+0000 7fc87d7fa640 1 -- 172.21.15.43:0/3139477491 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8740318e0 con 0x7fc880101450 2024-09-11T23:33:05.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.565+0000 7fc87d7fa640 1 -- 172.21.15.43:0/3139477491 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87405e690 con 0x7fc880101450 2024-09-11T23:33:05.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.568+0000 7fc886998640 1 -- 172.21.15.43:0/3139477491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc86c005290 con 0x7fc880101450 2024-09-11T23:33:05.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.570+0000 7fc87d7fa640 1 -- 172.21.15.43:0/3139477491 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc874039040 con 0x7fc880101450 2024-09-11T23:33:05.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.571+0000 7fc87d7fa640 1 --2- 172.21.15.43:0/3139477491 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc8680a2f00 0x7fc8680a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:05.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.571+0000 7fc87d7fa640 1 -- 172.21.15.43:0/3139477491 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7fc874112b80 con 0x7fc880101450 2024-09-11T23:33:05.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.572+0000 7fc87ffff640 1 --2- 172.21.15.43:0/3139477491 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc8680a2f00 0x7fc8680a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:05.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.572+0000 7fc87ffff640 1 --2- 172.21.15.43:0/3139477491 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc8680a2f00 0x7fc8680a53c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc88019fa40 tx=0x7fc87000d040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:05.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:05.580+0000 7fc87d7fa640 1 -- 172.21.15.43:0/3139477491 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc87406b060 con 0x7fc880101450 2024-09-11T23:33:06.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:06.006+0000 7fc886998640 1 -- 172.21.15.43:0/3139477491 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc86c005580 con 0x7fc880101450 2024-09-11T23:33:06.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:06.011+0000 7fc87d7fa640 1 -- 172.21.15.43:0/3139477491 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11893 (secure 0 0 0) 0x7fc874062030 con 0x7fc880101450 2024-09-11T23:33:06.011 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:33:06.012 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":34,"btime":"2024-09-11T23:31:56:056494+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/108369845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":108369845},{"type":"v1","addr":"172.21.15.43:6835","nonce":108369845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3739466479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3739466479},{"type":"v1","addr":"172.21.15.43:6837","nonce":3739466479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/2165493077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":2165493077},{"type":"v1","addr":"172.21.15.43:6839","nonce":2165493077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3592420960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3592420960},{"type":"v1","addr":"172.21.15.43:6841","nonce":3592420960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2769852264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2769852264},{"type":"v1","addr":"172.21.15.97:6833","nonce":2769852264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/2266906537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2266906537},{"type":"v1","addr":"172.21.15.90:6833","nonce":2266906537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T23:20:00.465830+0000","modified":"2024-09-11T23:31:55.493313+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24436":{"gid":24436,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/1640511441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1640511441},{"type":"v1","addr":"172.21.15.97:6835","nonce":1640511441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/191447096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":191447096},{"type":"v1","addr":"172.21.15.97:6837","nonce":191447096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.97:6839/1424670024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1424670024},{"type":"v1","addr":"172.21.15.97:6839","nonce":1424670024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.90:6837/2946725567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2946725567},{"type":"v1","addr":"172.21.15.90:6837","nonce":2946725567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6839/3234895964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3234895964},{"type":"v1","addr":"172.21.15.90:6839","nonce":3234895964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25571":{"gid":25571,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/568491708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":568491708},{"type":"v1","addr":"172.21.15.90:6835","nonce":568491708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24457]},"id":1}]} 2024-09-11T23:33:06.012 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 34 2024-09-11T23:33:06.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:06.017+0000 7fc866ffd640 1 -- 172.21.15.43:0/3139477491 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc8680a2f00 msgr2=0x7fc8680a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:06.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:06.017+0000 7fc866ffd640 1 --2- 172.21.15.43:0/3139477491 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc8680a2f00 0x7fc8680a53c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc88019fa40 tx=0x7fc87000d040 comp rx=0 tx=0).stop 2024-09-11T23:33:06.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:06.017+0000 7fc866ffd640 1 -- 172.21.15.43:0/3139477491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc880101450 msgr2=0x7fc8801a4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:06.017+0000 7fc866ffd640 1 --2- 172.21.15.43:0/3139477491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc880101450 0x7fc8801a4c80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc874030ff0 tx=0x7fc8740309b0 comp rx=0 tx=0).stop 2024-09-11T23:33:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:06.017+0000 7fc866ffd640 1 -- 172.21.15.43:0/3139477491 shutdown_connections 2024-09-11T23:33:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:06.017+0000 7fc866ffd640 1 --2- 172.21.15.43:0/3139477491 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc8680a2f00 0x7fc8680a53c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:06.017+0000 7fc866ffd640 1 --2- 172.21.15.43:0/3139477491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc880105f40 0x7fc88019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:06.017+0000 7fc866ffd640 1 --2- 172.21.15.43:0/3139477491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc880101450 0x7fc8801a4c80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:06.017+0000 7fc866ffd640 1 --2- 172.21.15.43:0/3139477491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc880100b30 0x7fc8801a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:06.017+0000 7fc866ffd640 1 -- 172.21.15.43:0/3139477491 >> 172.21.15.43:0/3139477491 conn(0x7fc8800fc7a0 msgr2=0x7fc8801038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:06.017+0000 7fc866ffd640 1 -- 172.21.15.43:0/3139477491 shutdown_connections 2024-09-11T23:33:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:06.017+0000 7fc866ffd640 1 -- 172.21.15.43:0/3139477491 wait complete. 2024-09-11T23:33:06.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:06 smithi043 ceph-mon[30950]: pgmap v349: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-11T23:33:06.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:06 smithi043 ceph-mon[30950]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T23:33:06.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:06 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3139477491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:33:06.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:06 smithi090 ceph-mon[35130]: pgmap v349: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-11T23:33:06.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:06 smithi090 ceph-mon[35130]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T23:33:06.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:06 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3139477491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:33:06.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:06 smithi097 ceph-mon[35562]: pgmap v349: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-11T23:33:06.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:06 smithi097 ceph-mon[35562]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T23:33:06.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:06 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3139477491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:33:08.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:08 smithi090 ceph-mon[35130]: pgmap v351: 97 pgs: 8 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 75 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-09-11T23:33:08.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:08 smithi097 ceph-mon[35562]: pgmap v351: 97 pgs: 8 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 75 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-09-11T23:33:08.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:08 smithi043 ceph-mon[30950]: pgmap v351: 97 pgs: 8 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 75 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-09-11T23:33:09.206 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:33:09.471 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:33:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.947+0000 7f5437be1640 1 -- 172.21.15.43:0/3023229628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430106f10 msgr2=0x7f543010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.947+0000 7f5437be1640 1 --2- 172.21.15.43:0/3023229628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430106f10 0x7f543010ea90 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5420009ca0 tx=0x7f542002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.947+0000 7f5437be1640 1 -- 172.21.15.43:0/3023229628 shutdown_connections 2024-09-11T23:33:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.947+0000 7f5437be1640 1 --2- 172.21.15.43:0/3023229628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430106f10 0x7f543010ea90 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.947+0000 7f5437be1640 1 --2- 172.21.15.43:0/3023229628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5430102790 0x7f54301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.947+0000 7f5437be1640 1 --2- 172.21.15.43:0/3023229628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5430101de0 0x7f54301021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.947+0000 7f5437be1640 1 -- 172.21.15.43:0/3023229628 >> 172.21.15.43:0/3023229628 conn(0x7f54300fd8c0 msgr2=0x7f54300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.947+0000 7f5437be1640 1 -- 172.21.15.43:0/3023229628 shutdown_connections 2024-09-11T23:33:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.947+0000 7f5437be1640 1 -- 172.21.15.43:0/3023229628 wait complete. 2024-09-11T23:33:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.948+0000 7f5437be1640 1 Processor -- start 2024-09-11T23:33:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.948+0000 7f5437be1640 1 -- start start 2024-09-11T23:33:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.948+0000 7f5437be1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430101de0 0x7f54300744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.948+0000 7f5437be1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5430102790 0x7f5430074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.948+0000 7f5437be1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5430106f10 0x7f5430074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.948+0000 7f5437be1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5430111fc0 con 0x7f5430101de0 2024-09-11T23:33:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.948+0000 7f5437be1640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5430111e40 con 0x7f5430102790 2024-09-11T23:33:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.948+0000 7f5437be1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5430112140 con 0x7f5430106f10 2024-09-11T23:33:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.948+0000 7f543597d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430101de0 0x7f54300744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.948+0000 7f543597d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430101de0 0x7f54300744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53390/0 (socket says 172.21.15.43:53390) 2024-09-11T23:33:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.948+0000 7f543597d640 1 -- 172.21.15.43:0/2340043940 learned_addr learned my addr 172.21.15.43:0/2340043940 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:33:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.948+0000 7f543517c640 1 --2- 172.21.15.43:0/2340043940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5430102790 0x7f5430074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.949+0000 7f543617e640 1 --2- 172.21.15.43:0/2340043940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5430106f10 0x7f5430074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.949+0000 7f543597d640 1 -- 172.21.15.43:0/2340043940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5430106f10 msgr2=0x7f5430074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.949+0000 7f543597d640 1 --2- 172.21.15.43:0/2340043940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5430106f10 0x7f5430074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.949+0000 7f543597d640 1 -- 172.21.15.43:0/2340043940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5430102790 msgr2=0x7f5430074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.949+0000 7f543597d640 1 --2- 172.21.15.43:0/2340043940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5430102790 0x7f5430074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.949+0000 7f543597d640 1 -- 172.21.15.43:0/2340043940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54301a6800 con 0x7f5430101de0 2024-09-11T23:33:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.949+0000 7f543597d640 1 --2- 172.21.15.43:0/2340043940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430101de0 0x7f54300744f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f542800c920 tx=0x7f542800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.949+0000 7f542effd640 1 -- 172.21.15.43:0/2340043940 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428007c70 con 0x7f5430101de0 2024-09-11T23:33:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.949+0000 7f5437be1640 1 -- 172.21.15.43:0/2340043940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54301a6af0 con 0x7f5430101de0 2024-09-11T23:33:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.949+0000 7f542effd640 1 -- 172.21.15.43:0/2340043940 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5428007e10 con 0x7f5430101de0 2024-09-11T23:33:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.949+0000 7f542effd640 1 -- 172.21.15.43:0/2340043940 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428018890 con 0x7f5430101de0 2024-09-11T23:33:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.949+0000 7f5437be1640 1 -- 172.21.15.43:0/2340043940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54301a7030 con 0x7f5430101de0 2024-09-11T23:33:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.950+0000 7f5437be1640 1 -- 172.21.15.43:0/2340043940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54301021c0 con 0x7f5430101de0 2024-09-11T23:33:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.951+0000 7f542effd640 1 -- 172.21.15.43:0/2340043940 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5428018a70 con 0x7f5430101de0 2024-09-11T23:33:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.952+0000 7f542effd640 1 --2- 172.21.15.43:0/2340043940 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f54180a2cf0 0x7f54180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.952+0000 7f543517c640 1 --2- 172.21.15.43:0/2340043940 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f54180a2cf0 0x7f54180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.952+0000 7f542effd640 1 -- 172.21.15.43:0/2340043940 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7f54280cf660 con 0x7f5430101de0 2024-09-11T23:33:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.952+0000 7f543517c640 1 --2- 172.21.15.43:0/2340043940 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f54180a2cf0 0x7f54180a51b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5430075f80 tx=0x7f542400b390 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:09.956+0000 7f542effd640 1 -- 172.21.15.43:0/2340043940 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5428014030 con 0x7f5430101de0 2024-09-11T23:33:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:10.133+0000 7f5437be1640 1 -- 172.21.15.43:0/2340043940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5430075a20 con 0x7f5430101de0 2024-09-11T23:33:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:10.135+0000 7f542effd640 1 -- 172.21.15.43:0/2340043940 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+23205 (secure 0 0 0) 0x7f542809c1c0 con 0x7f5430101de0 2024-09-11T23:33:10.135 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-11T23:33:10.135 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":95,"fsid":"25c0cd84-7093-11ef-bcea-c7b262605968","created":"2024-09-11T23:12:03.504406+0000","modified":"2024-09-11T23:33:05.110840+0000","last_up_change":"2024-09-11T23:19:07.797873+0000","last_in_change":"2024-09-11T23:18:44.595176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T23:15:17.058713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T23:19:56.221406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T23:19:58.705875+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"24076596-3909-4c87-85f1-cbf168070dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6803","nonce":43839997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6805","nonce":43839997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6809","nonce":43839997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":43839997},{"type":"v1","addr":"172.21.15.43:6807","nonce":43839997}]},"public_addr":"172.21.15.43:6803/43839997","cluster_addr":"172.21.15.43:6805/43839997","heartbeat_back_addr":"172.21.15.43:6809/43839997","heartbeat_front_addr":"172.21.15.43:6807/43839997","state":["exists","up"]},{"osd":1,"uuid":"a0472a57-243c-489d-a247-972ee10a71f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6801","nonce":1825149246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6803","nonce":1825149246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6807","nonce":1825149246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1825149246},{"type":"v1","addr":"172.21.15.90:6805","nonce":1825149246}]},"public_addr":"172.21.15.90:6801/1825149246","cluster_addr":"172.21.15.90:6803/1825149246","heartbeat_back_addr":"172.21.15.90:6807/1825149246","heartbeat_front_addr":"172.21.15.90:6805/1825149246","state":["exists","up"]},{"osd":2,"uuid":"40366536-0fcf-45a0-b8e3-49178401dc34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6801","nonce":1456846443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6803","nonce":1456846443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6807","nonce":1456846443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1456846443},{"type":"v1","addr":"172.21.15.97:6805","nonce":1456846443}]},"public_addr":"172.21.15.97:6801/1456846443","cluster_addr":"172.21.15.97:6803/1456846443","heartbeat_back_addr":"172.21.15.97:6807/1456846443","heartbeat_front_addr":"172.21.15.97:6805/1456846443","state":["exists","up"]},{"osd":3,"uuid":"3062133e-000b-4421-9d53-ac434b9ed9b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6811","nonce":1962837804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6813","nonce":1962837804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6817","nonce":1962837804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1962837804},{"type":"v1","addr":"172.21.15.43:6815","nonce":1962837804}]},"public_addr":"172.21.15.43:6811/1962837804","cluster_addr":"172.21.15.43:6813/1962837804","heartbeat_back_addr":"172.21.15.43:6817/1962837804","heartbeat_front_addr":"172.21.15.43:6815/1962837804","state":["exists","up"]},{"osd":4,"uuid":"2e73fa4b-f22c-4319-828d-87b91b9f19f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6809","nonce":181614173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6811","nonce":181614173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6815","nonce":181614173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":181614173},{"type":"v1","addr":"172.21.15.90:6813","nonce":181614173}]},"public_addr":"172.21.15.90:6809/181614173","cluster_addr":"172.21.15.90:6811/181614173","heartbeat_back_addr":"172.21.15.90:6815/181614173","heartbeat_front_addr":"172.21.15.90:6813/181614173","state":["exists","up"]},{"osd":5,"uuid":"1e1ad4f9-e724-440f-b515-3ec1ad97d6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6809","nonce":1540039543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6811","nonce":1540039543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6815","nonce":1540039543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":1540039543},{"type":"v1","addr":"172.21.15.97:6813","nonce":1540039543}]},"public_addr":"172.21.15.97:6809/1540039543","cluster_addr":"172.21.15.97:6811/1540039543","heartbeat_back_addr":"172.21.15.97:6815/1540039543","heartbeat_front_addr":"172.21.15.97:6813/1540039543","state":["exists","up"]},{"osd":6,"uuid":"d34df8df-f40d-4159-8b29-5401b1b0e431","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6819","nonce":3785677637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6821","nonce":3785677637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6825","nonce":3785677637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":3785677637},{"type":"v1","addr":"172.21.15.43:6823","nonce":3785677637}]},"public_addr":"172.21.15.43:6819/3785677637","cluster_addr":"172.21.15.43:6821/3785677637","heartbeat_back_addr":"172.21.15.43:6825/3785677637","heartbeat_front_addr":"172.21.15.43:6823/3785677637","state":["exists","up"]},{"osd":7,"uuid":"b2e01078-528b-49fd-b000-9f584f784b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6817","nonce":236874962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6819","nonce":236874962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6823","nonce":236874962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":236874962},{"type":"v1","addr":"172.21.15.90:6821","nonce":236874962}]},"public_addr":"172.21.15.90:6817/236874962","cluster_addr":"172.21.15.90:6819/236874962","heartbeat_back_addr":"172.21.15.90:6823/236874962","heartbeat_front_addr":"172.21.15.90:6821/236874962","state":["exists","up"]},{"osd":8,"uuid":"abef8f27-bf9b-4d63-bfbe-f967fc3d8b17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6817","nonce":1707327301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6819","nonce":1707327301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6823","nonce":1707327301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1707327301},{"type":"v1","addr":"172.21.15.97:6821","nonce":1707327301}]},"public_addr":"172.21.15.97:6817/1707327301","cluster_addr":"172.21.15.97:6819/1707327301","heartbeat_back_addr":"172.21.15.97:6823/1707327301","heartbeat_front_addr":"172.21.15.97:6821/1707327301","state":["exists","up"]},{"osd":9,"uuid":"190ab8b2-a835-47e6-9b39-16a58331c58a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6827","nonce":3170306011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6829","nonce":3170306011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6833","nonce":3170306011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3170306011},{"type":"v1","addr":"172.21.15.43:6831","nonce":3170306011}]},"public_addr":"172.21.15.43:6827/3170306011","cluster_addr":"172.21.15.43:6829/3170306011","heartbeat_back_addr":"172.21.15.43:6833/3170306011","heartbeat_front_addr":"172.21.15.43:6831/3170306011","state":["exists","up"]},{"osd":10,"uuid":"b32ab5fe-466f-4c8e-8696-cd9f820a3263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6825","nonce":59487041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6827","nonce":59487041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6831","nonce":59487041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":59487041},{"type":"v1","addr":"172.21.15.90:6829","nonce":59487041}]},"public_addr":"172.21.15.90:6825/59487041","cluster_addr":"172.21.15.90:6827/59487041","heartbeat_back_addr":"172.21.15.90:6831/59487041","heartbeat_front_addr":"172.21.15.90:6829/59487041","state":["exists","up"]},{"osd":11,"uuid":"d0faa589-21c8-4a9f-8a73-813aafe6d33c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6825","nonce":194699315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6827","nonce":194699315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6831","nonce":194699315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":194699315},{"type":"v1","addr":"172.21.15.97:6829","nonce":194699315}]},"public_addr":"172.21.15.97:6825/194699315","cluster_addr":"172.21.15.97:6827/194699315","heartbeat_back_addr":"172.21.15.97:6831/194699315","heartbeat_front_addr":"172.21.15.97:6829/194699315","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:23.381247+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:14:48.843190+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:13.538979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:15:40.136983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:05.513790+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:31.247090+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:16:56.635221+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:22.667327+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:17:47.910422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:14.353914+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:18:40.584503+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T23:19:06.476766+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/2262294782":"2024-09-12T23:21:50.205362+0000","172.21.15.43:0/2398981254":"2024-09-12T23:21:50.205362+0000","172.21.15.43:0/481605249":"2024-09-12T23:21:50.205362+0000","172.21.15.43:0/1375379108":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1512201905":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/206044413":"2024-09-12T23:12:35.756598+0000","172.21.15.43:6800/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:0/1579053924":"2024-09-12T23:21:50.205362+0000","172.21.15.43:6801/4015351527":"2024-09-12T23:12:35.756598+0000","172.21.15.43:6800/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/3892590401":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6801/2370495020":"2024-09-12T23:21:50.205362+0000","172.21.15.43:0/4006124758":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/76575371":"2024-09-12T23:13:00.548389+0000","172.21.15.43:0/120495444":"2024-09-12T23:13:00.548389+0000","172.21.15.43:6800/2370495020":"2024-09-12T23:21:50.205362+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:33:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:10.137+0000 7f5437be1640 1 -- 172.21.15.43:0/2340043940 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f54180a2cf0 msgr2=0x7f54180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:10.137+0000 7f5437be1640 1 --2- 172.21.15.43:0/2340043940 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f54180a2cf0 0x7f54180a51b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5430075f80 tx=0x7f542400b390 comp rx=0 tx=0).stop 2024-09-11T23:33:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:10.137+0000 7f5437be1640 1 -- 172.21.15.43:0/2340043940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430101de0 msgr2=0x7f54300744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:10.137+0000 7f5437be1640 1 --2- 172.21.15.43:0/2340043940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430101de0 0x7f54300744f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f542800c920 tx=0x7f542800cde0 comp rx=0 tx=0).stop 2024-09-11T23:33:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:10.137+0000 7f5437be1640 1 -- 172.21.15.43:0/2340043940 shutdown_connections 2024-09-11T23:33:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:10.137+0000 7f5437be1640 1 --2- 172.21.15.43:0/2340043940 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f54180a2cf0 0x7f54180a51b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:10.137+0000 7f5437be1640 1 --2- 172.21.15.43:0/2340043940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5430106f10 0x7f5430074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:10.137+0000 7f5437be1640 1 --2- 172.21.15.43:0/2340043940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5430102790 0x7f5430074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:10.137+0000 7f5437be1640 1 --2- 172.21.15.43:0/2340043940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430101de0 0x7f54300744f0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:10.137+0000 7f5437be1640 1 -- 172.21.15.43:0/2340043940 >> 172.21.15.43:0/2340043940 conn(0x7f54300fd8c0 msgr2=0x7f54300fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:10.137+0000 7f5437be1640 1 -- 172.21.15.43:0/2340043940 shutdown_connections 2024-09-11T23:33:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:10.137+0000 7f5437be1640 1 -- 172.21.15.43:0/2340043940 wait complete. 2024-09-11T23:33:10.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:10 smithi043 ceph-mon[30950]: pgmap v352: 97 pgs: 12 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 60 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-09-11T23:33:10.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:10 smithi090 ceph-mon[35130]: pgmap v352: 97 pgs: 12 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 60 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-09-11T23:33:10.551 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:10 smithi097 ceph-mon[35562]: pgmap v352: 97 pgs: 12 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 60 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-09-11T23:33:10.875 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-11T23:33:11.139 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:33:11.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:11 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/2340043940' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:33:11.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:11 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/2340043940' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:33:11.550 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:11 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/2340043940' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:33:11.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.660+0000 7f4942cd1640 1 -- 172.21.15.43:0/2755059698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f493c102d90 msgr2=0x7f493c103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:11.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.660+0000 7f4942cd1640 1 --2- 172.21.15.43:0/2755059698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f493c102d90 0x7f493c103170 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4930009930 tx=0x7f493002a420 comp rx=0 tx=0).stop 2024-09-11T23:33:11.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.660+0000 7f4942cd1640 1 -- 172.21.15.43:0/2755059698 shutdown_connections 2024-09-11T23:33:11.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.660+0000 7f4942cd1640 1 --2- 172.21.15.43:0/2755059698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f493c108290 0x7f493c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.660+0000 7f4942cd1640 1 --2- 172.21.15.43:0/2755059698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f493c103740 0x7f493c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.660+0000 7f4942cd1640 1 --2- 172.21.15.43:0/2755059698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f493c102d90 0x7f493c103170 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.660+0000 7f4942cd1640 1 -- 172.21.15.43:0/2755059698 >> 172.21.15.43:0/2755059698 conn(0x7f493c0fc7d0 msgr2=0x7f493c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:11.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.660+0000 7f4942cd1640 1 -- 172.21.15.43:0/2755059698 shutdown_connections 2024-09-11T23:33:11.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.660+0000 7f4942cd1640 1 -- 172.21.15.43:0/2755059698 wait complete. 2024-09-11T23:33:11.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.661+0000 7f4942cd1640 1 Processor -- start 2024-09-11T23:33:11.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.661+0000 7f4942cd1640 1 -- start start 2024-09-11T23:33:11.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.661+0000 7f4942cd1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f493c102d90 0x7f493c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.661+0000 7f4942cd1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f493c103740 0x7f493c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.661+0000 7f4942cd1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f493c108290 0x7f493c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.661+0000 7f4942cd1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f493c112090 con 0x7f493c108290 2024-09-11T23:33:11.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.661+0000 7f4942cd1640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f493c111f10 con 0x7f493c103740 2024-09-11T23:33:11.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.661+0000 7f4942cd1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f493c112210 con 0x7f493c102d90 2024-09-11T23:33:11.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.661+0000 7f494126e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f493c108290 0x7f493c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:11.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.661+0000 7f494126e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f493c108290 0x7f493c19ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54214/0 (socket says 172.21.15.43:54214) 2024-09-11T23:33:11.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.661+0000 7f494126e640 1 -- 172.21.15.43:0/1088827849 learned_addr learned my addr 172.21.15.43:0/1088827849 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:33:11.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.661+0000 7f4940a6d640 1 --2- 172.21.15.43:0/1088827849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f493c102d90 0x7f493c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:11.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.661+0000 7f493bfff640 1 --2- 172.21.15.43:0/1088827849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f493c103740 0x7f493c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:11.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.661+0000 7f494126e640 1 -- 172.21.15.43:0/1088827849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f493c102d90 msgr2=0x7f493c198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:11.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.661+0000 7f494126e640 1 --2- 172.21.15.43:0/1088827849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f493c102d90 0x7f493c198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.661+0000 7f494126e640 1 -- 172.21.15.43:0/1088827849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f493c103740 msgr2=0x7f493c198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:11.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.661+0000 7f494126e640 1 --2- 172.21.15.43:0/1088827849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f493c103740 0x7f493c198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.661+0000 7f494126e640 1 -- 172.21.15.43:0/1088827849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493c19d500 con 0x7f493c108290 2024-09-11T23:33:11.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.661+0000 7f493bfff640 1 --2- 172.21.15.43:0/1088827849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f493c103740 0x7f493c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:33:11.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.662+0000 7f494126e640 1 --2- 172.21.15.43:0/1088827849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f493c108290 0x7f493c19ce20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f493400cce0 tx=0x7f493400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:11.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.662+0000 7f4939ffb640 1 -- 172.21.15.43:0/1088827849 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493400f450 con 0x7f493c108290 2024-09-11T23:33:11.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.662+0000 7f4939ffb640 1 -- 172.21.15.43:0/1088827849 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f493400fad0 con 0x7f493c108290 2024-09-11T23:33:11.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.662+0000 7f4942cd1640 1 -- 172.21.15.43:0/1088827849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f493c19d7f0 con 0x7f493c108290 2024-09-11T23:33:11.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.662+0000 7f4939ffb640 1 -- 172.21.15.43:0/1088827849 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934018c10 con 0x7f493c108290 2024-09-11T23:33:11.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.663+0000 7f4942cd1640 1 -- 172.21.15.43:0/1088827849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f493c1a50d0 con 0x7f493c108290 2024-09-11T23:33:11.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.663+0000 7f4942cd1640 1 -- 172.21.15.43:0/1088827849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f493c06a7f0 con 0x7f493c108290 2024-09-11T23:33:11.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.665+0000 7f4939ffb640 1 -- 172.21.15.43:0/1088827849 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f493400f5f0 con 0x7f493c108290 2024-09-11T23:33:11.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.666+0000 7f4939ffb640 1 --2- 172.21.15.43:0/1088827849 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f492c0a2d70 0x7f492c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:11.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.666+0000 7f4940a6d640 1 --2- 172.21.15.43:0/1088827849 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f492c0a2d70 0x7f492c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:11.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.666+0000 7f4939ffb640 1 -- 172.21.15.43:0/1088827849 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7f49340cfae0 con 0x7f493c108290 2024-09-11T23:33:11.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.666+0000 7f4940a6d640 1 --2- 172.21.15.43:0/1088827849 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f492c0a2d70 0x7f492c0a5230 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4930009900 tx=0x7f49300058d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:11.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.670+0000 7f4939ffb640 1 -- 172.21.15.43:0/1088827849 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4934014030 con 0x7f493c108290 2024-09-11T23:33:11.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.890+0000 7f4942cd1640 1 -- 172.21.15.43:0/1088827849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f493c10a2a0 con 0x7f493c108290 2024-09-11T23:33:11.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.892+0000 7f4939ffb640 1 -- 172.21.15.43:0/1088827849 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) ==== 161+0+0 (secure 0 0 0) 0x7f493409c640 con 0x7f493c108290 2024-09-11T23:33:11.892 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T23:33:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.895+0000 7f4942cd1640 1 -- 172.21.15.43:0/1088827849 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f492c0a2d70 msgr2=0x7f492c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.895+0000 7f4942cd1640 1 --2- 172.21.15.43:0/1088827849 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f492c0a2d70 0x7f492c0a5230 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4930009900 tx=0x7f49300058d0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.895+0000 7f4942cd1640 1 -- 172.21.15.43:0/1088827849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f493c108290 msgr2=0x7f493c19ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.895+0000 7f4942cd1640 1 --2- 172.21.15.43:0/1088827849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f493c108290 0x7f493c19ce20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f493400cce0 tx=0x7f493400a620 comp rx=0 tx=0).stop 2024-09-11T23:33:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.895+0000 7f4942cd1640 1 -- 172.21.15.43:0/1088827849 shutdown_connections 2024-09-11T23:33:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.895+0000 7f4942cd1640 1 --2- 172.21.15.43:0/1088827849 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f492c0a2d70 0x7f492c0a5230 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.895+0000 7f4942cd1640 1 --2- 172.21.15.43:0/1088827849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f493c108290 0x7f493c19ce20 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.895+0000 7f4942cd1640 1 --2- 172.21.15.43:0/1088827849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f493c103740 0x7f493c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.895+0000 7f4942cd1640 1 --2- 172.21.15.43:0/1088827849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f493c102d90 0x7f493c198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.895+0000 7f4942cd1640 1 -- 172.21.15.43:0/1088827849 >> 172.21.15.43:0/1088827849 conn(0x7f493c0fc7d0 msgr2=0x7f493c105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.895+0000 7f4942cd1640 1 -- 172.21.15.43:0/1088827849 shutdown_connections 2024-09-11T23:33:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:11.895+0000 7f4942cd1640 1 -- 172.21.15.43:0/1088827849 wait complete. 2024-09-11T23:33:12.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:11 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-a[30946]: 2024-09-11T23:33:11.890+0000 7f935f9d6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T23:33:12.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:12 smithi043 ceph-mon[30950]: pgmap v353: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-09-11T23:33:12.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:12 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1088827849' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T23:33:12.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:12 smithi043 ceph-mon[30950]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T23:33:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:12 smithi043 ceph-mon[30950]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T23:33:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:12 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1088827849' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T23:33:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:12 smithi043 ceph-mon[30950]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T23:33:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:12 smithi043 ceph-mon[30950]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-11T23:33:12.269 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:33:11 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-b[65178]: ignoring --setuser ceph since I am not root 2024-09-11T23:33:12.270 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:33:11 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-b[65178]: ignoring --setgroup ceph since I am not root 2024-09-11T23:33:12.270 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:33:11 smithi090 ceph-mds[65182]: -- 172.21.15.90:0/3002371776 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558b55eec680 con 0x558b55ec0800 2024-09-11T23:33:12.270 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:33:11 smithi090 ceph-mds[65182]: -- 172.21.15.90:0/3002371776 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558b55ee3860 con 0x558b55ec0800 2024-09-11T23:33:12.270 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:33:11 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-b[65178]: starting mds.b at 2024-09-11T23:33:12.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:12 smithi090 ceph-mon[35130]: pgmap v353: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-09-11T23:33:12.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:12 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1088827849' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T23:33:12.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:12 smithi090 ceph-mon[35130]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T23:33:12.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:12 smithi090 ceph-mon[35130]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T23:33:12.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:12 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1088827849' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T23:33:12.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:12 smithi090 ceph-mon[35130]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T23:33:12.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:12 smithi090 ceph-mon[35130]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-11T23:33:12.271 INFO:journalctl@ceph.mds.e.smithi090.stdout:Sep 11 23:33:11 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-e[64566]: ignoring --setuser ceph since I am not root 2024-09-11T23:33:12.271 INFO:journalctl@ceph.mds.e.smithi090.stdout:Sep 11 23:33:11 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-e[64566]: ignoring --setgroup ceph since I am not root 2024-09-11T23:33:12.271 INFO:journalctl@ceph.mds.e.smithi090.stdout:Sep 11 23:33:11 smithi090 ceph-mds[64570]: -- 172.21.15.90:0/4050078234 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56227546b860 con 0x562275448800 2024-09-11T23:33:12.271 INFO:journalctl@ceph.mds.e.smithi090.stdout:Sep 11 23:33:11 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-e[64566]: starting mds.e at 2024-09-11T23:33:12.272 INFO:journalctl@ceph.mds.h.smithi090.stdout:Sep 11 23:33:11 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-h[64868]: ignoring --setuser ceph since I am not root 2024-09-11T23:33:12.272 INFO:journalctl@ceph.mds.h.smithi090.stdout:Sep 11 23:33:11 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-h[64868]: ignoring --setgroup ceph since I am not root 2024-09-11T23:33:12.272 INFO:journalctl@ceph.mds.h.smithi090.stdout:Sep 11 23:33:11 smithi090 ceph-mds[64872]: -- 172.21.15.90:0/1557296128 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55927bf27860 con 0x55927bf05000 2024-09-11T23:33:12.272 INFO:journalctl@ceph.mds.h.smithi090.stdout:Sep 11 23:33:11 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-h[64868]: starting mds.h at 2024-09-11T23:33:12.301 INFO:journalctl@ceph.mds.f.smithi097.stdout:Sep 11 23:33:12 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-f[65154]: ignoring --setuser ceph since I am not root 2024-09-11T23:33:12.301 INFO:journalctl@ceph.mds.f.smithi097.stdout:Sep 11 23:33:12 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-f[65154]: ignoring --setgroup ceph since I am not root 2024-09-11T23:33:12.301 INFO:journalctl@ceph.mds.f.smithi097.stdout:Sep 11 23:33:12 smithi097 ceph-mds[65158]: -- 172.21.15.97:0/868294859 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560f300bb860 con 0x560f30098800 2024-09-11T23:33:12.301 INFO:journalctl@ceph.mds.f.smithi097.stdout:Sep 11 23:33:12 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-f[65154]: starting mds.f at 2024-09-11T23:33:12.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:12 smithi097 ceph-mon[35562]: pgmap v353: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-09-11T23:33:12.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:12 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1088827849' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T23:33:12.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:12 smithi097 ceph-mon[35562]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T23:33:12.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:12 smithi097 ceph-mon[35562]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T23:33:12.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:12 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1088827849' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T23:33:12.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:12 smithi097 ceph-mon[35562]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T23:33:12.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:12 smithi097 ceph-mon[35562]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-11T23:33:12.302 INFO:journalctl@ceph.mds.c.smithi097.stdout:Sep 11 23:33:12 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-c[65456]: ignoring --setuser ceph since I am not root 2024-09-11T23:33:12.302 INFO:journalctl@ceph.mds.c.smithi097.stdout:Sep 11 23:33:12 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-c[65456]: ignoring --setgroup ceph since I am not root 2024-09-11T23:33:12.302 INFO:journalctl@ceph.mds.c.smithi097.stdout:Sep 11 23:33:12 smithi097 ceph-mds[65460]: -- 172.21.15.97:0/1378445832 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dfb3029860 con 0x55dfb3007000 2024-09-11T23:33:12.303 INFO:journalctl@ceph.mds.c.smithi097.stdout:Sep 11 23:33:12 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-c[65456]: starting mds.c at 2024-09-11T23:33:12.303 INFO:journalctl@ceph.mds.i.smithi097.stdout:Sep 11 23:33:12 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-i[65760]: ignoring --setuser ceph since I am not root 2024-09-11T23:33:12.303 INFO:journalctl@ceph.mds.i.smithi097.stdout:Sep 11 23:33:12 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-i[65760]: ignoring --setgroup ceph since I am not root 2024-09-11T23:33:12.303 INFO:journalctl@ceph.mds.i.smithi097.stdout:Sep 11 23:33:12 smithi097 ceph-mds[65764]: -- 172.21.15.97:0/2236578138 <== mon.2 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c657569860 con 0x55c657546800 2024-09-11T23:33:12.303 INFO:journalctl@ceph.mds.i.smithi097.stdout:Sep 11 23:33:12 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-i[65760]: starting mds.i at 2024-09-11T23:33:13.793 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T23:33:14.063 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:33:14.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:13 smithi043 ceph-mon[30950]: pgmap v355: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 67 KiB/s wr, 34 op/s 2024-09-11T23:33:14.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:13 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.90:6834/2231894920,v1:172.21.15.90:6837/2231894920] up:boot 2024-09-11T23:33:14.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:13 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.90:6835/3527335283,v1:172.21.15.90:6838/3527335283] up:boot 2024-09-11T23:33:14.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:13 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.90:6836/774399692,v1:172.21.15.90:6839/774399692] up:boot 2024-09-11T23:33:14.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:13 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.97:6836/1921005833,v1:172.21.15.97:6838/1921005833] up:boot 2024-09-11T23:33:14.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:13 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.97:6835/989361658,v1:172.21.15.97:6839/989361658] up:boot 2024-09-11T23:33:14.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:13 smithi043 ceph-mon[30950]: mds.? [v2:172.21.15.97:6834/523508537,v1:172.21.15.97:6837/523508537] up:boot 2024-09-11T23:33:14.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:13 smithi043 ceph-mon[30950]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-11T23:33:14.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:13 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:33:14.111 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:13 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:33:14.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:13 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:33:14.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:13 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:33:14.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:13 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:33:14.112 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:13 smithi043 ceph-mon[30950]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:33:14.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:13 smithi090 ceph-mon[35130]: pgmap v355: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 67 KiB/s wr, 34 op/s 2024-09-11T23:33:14.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:13 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.90:6834/2231894920,v1:172.21.15.90:6837/2231894920] up:boot 2024-09-11T23:33:14.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:13 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.90:6835/3527335283,v1:172.21.15.90:6838/3527335283] up:boot 2024-09-11T23:33:14.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:13 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.90:6836/774399692,v1:172.21.15.90:6839/774399692] up:boot 2024-09-11T23:33:14.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:13 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.97:6836/1921005833,v1:172.21.15.97:6838/1921005833] up:boot 2024-09-11T23:33:14.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:13 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.97:6835/989361658,v1:172.21.15.97:6839/989361658] up:boot 2024-09-11T23:33:14.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:13 smithi090 ceph-mon[35130]: mds.? [v2:172.21.15.97:6834/523508537,v1:172.21.15.97:6837/523508537] up:boot 2024-09-11T23:33:14.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:13 smithi090 ceph-mon[35130]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-11T23:33:14.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:13 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:33:14.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:13 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:33:14.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:13 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:33:14.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:13 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:33:14.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:13 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:33:14.271 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:13 smithi090 ceph-mon[35130]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:33:14.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:13 smithi097 ceph-mon[35562]: pgmap v355: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 67 KiB/s wr, 34 op/s 2024-09-11T23:33:14.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:13 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.90:6834/2231894920,v1:172.21.15.90:6837/2231894920] up:boot 2024-09-11T23:33:14.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:13 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.90:6835/3527335283,v1:172.21.15.90:6838/3527335283] up:boot 2024-09-11T23:33:14.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:13 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.90:6836/774399692,v1:172.21.15.90:6839/774399692] up:boot 2024-09-11T23:33:14.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:13 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.97:6836/1921005833,v1:172.21.15.97:6838/1921005833] up:boot 2024-09-11T23:33:14.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:13 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.97:6835/989361658,v1:172.21.15.97:6839/989361658] up:boot 2024-09-11T23:33:14.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:13 smithi097 ceph-mon[35562]: mds.? [v2:172.21.15.97:6834/523508537,v1:172.21.15.97:6837/523508537] up:boot 2024-09-11T23:33:14.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:13 smithi097 ceph-mon[35562]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-11T23:33:14.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:13 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:33:14.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:13 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:33:14.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:13 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:33:14.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:13 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:33:14.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:13 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:33:14.302 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:13 smithi097 ceph-mon[35562]: from='mgr.15126 172.21.15.43:0/264676996' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:33:14.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.561+0000 7f730f85e640 1 -- 172.21.15.43:0/1211769160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7308106f10 msgr2=0x7f730810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:14.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.561+0000 7f730f85e640 1 --2- 172.21.15.43:0/1211769160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7308106f10 0x7f730810ea90 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7300009ca0 tx=0x7f730002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:33:14.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.561+0000 7f730f85e640 1 -- 172.21.15.43:0/1211769160 shutdown_connections 2024-09-11T23:33:14.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.561+0000 7f730f85e640 1 --2- 172.21.15.43:0/1211769160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7308106f10 0x7f730810ea90 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:14.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.561+0000 7f730f85e640 1 --2- 172.21.15.43:0/1211769160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7308102790 0x7f73081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:14.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.561+0000 7f730f85e640 1 --2- 172.21.15.43:0/1211769160 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7308101de0 0x7f73081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:14.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.561+0000 7f730f85e640 1 -- 172.21.15.43:0/1211769160 >> 172.21.15.43:0/1211769160 conn(0x7f73080fd8c0 msgr2=0x7f73080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:14.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.561+0000 7f730f85e640 1 -- 172.21.15.43:0/1211769160 shutdown_connections 2024-09-11T23:33:14.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.561+0000 7f730f85e640 1 -- 172.21.15.43:0/1211769160 wait complete. 2024-09-11T23:33:14.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.562+0000 7f730f85e640 1 Processor -- start 2024-09-11T23:33:14.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.562+0000 7f730f85e640 1 -- start start 2024-09-11T23:33:14.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.562+0000 7f730f85e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7308101de0 0x7f73081983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:14.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.562+0000 7f730f85e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7308102790 0x7f7308198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:14.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.562+0000 7f730f85e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7308106f10 0x7f730819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:14.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.562+0000 7f730f85e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7308111f70 con 0x7f7308101de0 2024-09-11T23:33:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.562+0000 7f730f85e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7308111df0 con 0x7f7308106f10 2024-09-11T23:33:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.562+0000 7f730f85e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f73081120f0 con 0x7f7308102790 2024-09-11T23:33:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.562+0000 7f730d5fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7308101de0 0x7f73081983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.562+0000 7f730d5fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7308101de0 0x7f73081983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54232/0 (socket says 172.21.15.43:54232) 2024-09-11T23:33:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.562+0000 7f730d5fa640 1 -- 172.21.15.43:0/3345481987 learned_addr learned my addr 172.21.15.43:0/3345481987 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:33:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.562+0000 7f730cdf9640 1 --2- 172.21.15.43:0/3345481987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7308102790 0x7f7308198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.562+0000 7f730ddfb640 1 --2- 172.21.15.43:0/3345481987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7308106f10 0x7f730819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.563+0000 7f730d5fa640 1 -- 172.21.15.43:0/3345481987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7308102790 msgr2=0x7f7308198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.563+0000 7f730d5fa640 1 --2- 172.21.15.43:0/3345481987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7308102790 0x7f7308198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.563+0000 7f730d5fa640 1 -- 172.21.15.43:0/3345481987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7308106f10 msgr2=0x7f730819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:14.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.563+0000 7f730d5fa640 1 --2- 172.21.15.43:0/3345481987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7308106f10 0x7f730819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:14.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.563+0000 7f730d5fa640 1 -- 172.21.15.43:0/3345481987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730819d380 con 0x7f7308101de0 2024-09-11T23:33:14.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.563+0000 7f730d5fa640 1 --2- 172.21.15.43:0/3345481987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7308101de0 0x7f73081983d0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f730400e9f0 tx=0x7f730400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:14.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.563+0000 7f72fe7fc640 1 -- 172.21.15.43:0/3345481987 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730400cd80 con 0x7f7308101de0 2024-09-11T23:33:14.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.563+0000 7f72fe7fc640 1 -- 172.21.15.43:0/3345481987 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7304005500 con 0x7f7308101de0 2024-09-11T23:33:14.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.563+0000 7f730f85e640 1 -- 172.21.15.43:0/3345481987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f730819d670 con 0x7f7308101de0 2024-09-11T23:33:14.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.563+0000 7f72fe7fc640 1 -- 172.21.15.43:0/3345481987 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304010690 con 0x7f7308101de0 2024-09-11T23:33:14.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.563+0000 7f730f85e640 1 -- 172.21.15.43:0/3345481987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73081a4f50 con 0x7f7308101de0 2024-09-11T23:33:14.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.564+0000 7f730f85e640 1 -- 172.21.15.43:0/3345481987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73081021c0 con 0x7f7308101de0 2024-09-11T23:33:14.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.566+0000 7f72fe7fc640 1 -- 172.21.15.43:0/3345481987 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f73040046c0 con 0x7f7308101de0 2024-09-11T23:33:14.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.566+0000 7f72fe7fc640 1 --2- 172.21.15.43:0/3345481987 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f72f40a2d70 0x7f72f40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:14.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.566+0000 7f730cdf9640 1 --2- 172.21.15.43:0/3345481987 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f72f40a2d70 0x7f72f40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:14.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.566+0000 7f72fe7fc640 1 -- 172.21.15.43:0/3345481987 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8829+0+0 (secure 0 0 0) 0x7f73040cfb40 con 0x7f7308101de0 2024-09-11T23:33:14.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.567+0000 7f730cdf9640 1 --2- 172.21.15.43:0/3345481987 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f72f40a2d70 0x7f72f40a5230 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f73081088e0 tx=0x7f72f8005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:14.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.569+0000 7f72fe7fc640 1 -- 172.21.15.43:0/3345481987 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73040541b0 con 0x7f7308101de0 2024-09-11T23:33:14.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.780+0000 7f730f85e640 1 -- 172.21.15.43:0/3345481987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7308109bc0 con 0x7f7308101de0 2024-09-11T23:33:14.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.901+0000 7f72fe7fc640 1 -- 172.21.15.43:0/3345481987 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) ==== 104+0+0 (secure 0 0 0) 0x7f730409c5d0 con 0x7f7308101de0 2024-09-11T23:33:14.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.903+0000 7f730f85e640 1 -- 172.21.15.43:0/3345481987 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f72f40a2d70 msgr2=0x7f72f40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:14.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.903+0000 7f730f85e640 1 --2- 172.21.15.43:0/3345481987 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f72f40a2d70 0x7f72f40a5230 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f73081088e0 tx=0x7f72f8005f90 comp rx=0 tx=0).stop 2024-09-11T23:33:14.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.903+0000 7f730f85e640 1 -- 172.21.15.43:0/3345481987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7308101de0 msgr2=0x7f73081983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:14.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.903+0000 7f730f85e640 1 --2- 172.21.15.43:0/3345481987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7308101de0 0x7f73081983d0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f730400e9f0 tx=0x7f730400edf0 comp rx=0 tx=0).stop 2024-09-11T23:33:14.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.903+0000 7f730f85e640 1 -- 172.21.15.43:0/3345481987 shutdown_connections 2024-09-11T23:33:14.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.903+0000 7f730f85e640 1 --2- 172.21.15.43:0/3345481987 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f72f40a2d70 0x7f72f40a5230 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:14.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.903+0000 7f730f85e640 1 --2- 172.21.15.43:0/3345481987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7308106f10 0x7f730819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:14.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.903+0000 7f730f85e640 1 --2- 172.21.15.43:0/3345481987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7308102790 0x7f7308198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:14.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.903+0000 7f730f85e640 1 --2- 172.21.15.43:0/3345481987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7308101de0 0x7f73081983d0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:14.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.903+0000 7f730f85e640 1 -- 172.21.15.43:0/3345481987 >> 172.21.15.43:0/3345481987 conn(0x7f73080fd8c0 msgr2=0x7f73080feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:14.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.903+0000 7f730f85e640 1 -- 172.21.15.43:0/3345481987 shutdown_connections 2024-09-11T23:33:14.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:14.904+0000 7f730f85e640 1 -- 172.21.15.43:0/3345481987 wait complete. 2024-09-11T23:33:15.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:14 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3345481987' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:33:15.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:14 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3345481987' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:33:15.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:14 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3345481987' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:33:15.572 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T23:33:15.831 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:33:16.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:15 smithi043 ceph-mon[30950]: pgmap v356: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 KiB/s wr, 24 op/s 2024-09-11T23:33:16.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:15 smithi043 ceph-mon[30950]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T23:33:16.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:15 smithi043 ceph-mon[30950]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T23:33:16.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:15 smithi043 ceph-mon[30950]: Cluster is now healthy 2024-09-11T23:33:16.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:15 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/3345481987' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:33:16.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:15 smithi043 ceph-mon[30950]: fsmap 12 up:standby 2024-09-11T23:33:16.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:15 smithi090 ceph-mon[35130]: pgmap v356: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 KiB/s wr, 24 op/s 2024-09-11T23:33:16.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:15 smithi090 ceph-mon[35130]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T23:33:16.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:15 smithi090 ceph-mon[35130]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T23:33:16.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:15 smithi090 ceph-mon[35130]: Cluster is now healthy 2024-09-11T23:33:16.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:15 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/3345481987' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:33:16.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:15 smithi090 ceph-mon[35130]: fsmap 12 up:standby 2024-09-11T23:33:16.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:15 smithi097 ceph-mon[35562]: pgmap v356: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 KiB/s wr, 24 op/s 2024-09-11T23:33:16.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:15 smithi097 ceph-mon[35562]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T23:33:16.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:15 smithi097 ceph-mon[35562]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T23:33:16.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:15 smithi097 ceph-mon[35562]: Cluster is now healthy 2024-09-11T23:33:16.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:15 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/3345481987' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:33:16.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:15 smithi097 ceph-mon[35562]: fsmap 12 up:standby 2024-09-11T23:33:16.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.330+0000 7fc994d84640 1 -- 172.21.15.43:0/2701366513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9901036e0 msgr2=0x7fc990107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:16.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.330+0000 7fc994d84640 1 --2- 172.21.15.43:0/2701366513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9901036e0 0x7fc990107b90 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc984009930 tx=0x7fc98402a420 comp rx=0 tx=0).stop 2024-09-11T23:33:16.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.330+0000 7fc994d84640 1 -- 172.21.15.43:0/2701366513 shutdown_connections 2024-09-11T23:33:16.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.330+0000 7fc994d84640 1 --2- 172.21.15.43:0/2701366513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9901082a0 0x7fc99010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:16.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.330+0000 7fc994d84640 1 --2- 172.21.15.43:0/2701366513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9901036e0 0x7fc990107b90 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:16.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.330+0000 7fc994d84640 1 --2- 172.21.15.43:0/2701366513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc990102d30 0x7fc990103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:16.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.330+0000 7fc994d84640 1 -- 172.21.15.43:0/2701366513 >> 172.21.15.43:0/2701366513 conn(0x7fc9900fc7d0 msgr2=0x7fc9900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:16.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.330+0000 7fc994d84640 1 -- 172.21.15.43:0/2701366513 shutdown_connections 2024-09-11T23:33:16.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.330+0000 7fc994d84640 1 -- 172.21.15.43:0/2701366513 wait complete. 2024-09-11T23:33:16.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.331+0000 7fc994d84640 1 Processor -- start 2024-09-11T23:33:16.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.331+0000 7fc994d84640 1 -- start start 2024-09-11T23:33:16.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.332+0000 7fc994d84640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc990102d30 0x7fc9901986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:16.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.332+0000 7fc994d84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9901036e0 0x7fc990198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:16.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.332+0000 7fc994d84640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9901082a0 0x7fc99019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:16.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.332+0000 7fc994d84640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901120c0 con 0x7fc9901036e0 2024-09-11T23:33:16.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.332+0000 7fc994d84640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc990111f40 con 0x7fc9901082a0 2024-09-11T23:33:16.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.332+0000 7fc994d84640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc990112240 con 0x7fc990102d30 2024-09-11T23:33:16.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.332+0000 7fc98dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9901036e0 0x7fc990198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:16.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.332+0000 7fc98e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc990102d30 0x7fc9901986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:16.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.332+0000 7fc98dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9901036e0 0x7fc990198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54246/0 (socket says 172.21.15.43:54246) 2024-09-11T23:33:16.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.332+0000 7fc98dd9b640 1 -- 172.21.15.43:0/228957870 learned_addr learned my addr 172.21.15.43:0/228957870 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:33:16.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.332+0000 7fc98ed9d640 1 --2- 172.21.15.43:0/228957870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9901082a0 0x7fc99019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:16.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.332+0000 7fc98dd9b640 1 -- 172.21.15.43:0/228957870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc990102d30 msgr2=0x7fc9901986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:16.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.332+0000 7fc98dd9b640 1 --2- 172.21.15.43:0/228957870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc990102d30 0x7fc9901986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:16.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.332+0000 7fc98dd9b640 1 -- 172.21.15.43:0/228957870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9901082a0 msgr2=0x7fc99019cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:16.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.332+0000 7fc98dd9b640 1 --2- 172.21.15.43:0/228957870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9901082a0 0x7fc99019cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:16.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.332+0000 7fc98dd9b640 1 -- 172.21.15.43:0/228957870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99019d650 con 0x7fc9901036e0 2024-09-11T23:33:16.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.332+0000 7fc98dd9b640 1 --2- 172.21.15.43:0/228957870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9901036e0 0x7fc990198be0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc984005940 tx=0x7fc9840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:16.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.332+0000 7fc97f7fe640 1 -- 172.21.15.43:0/228957870 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98403f030 con 0x7fc9901036e0 2024-09-11T23:33:16.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.333+0000 7fc97f7fe640 1 -- 172.21.15.43:0/228957870 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc984003ce0 con 0x7fc9901036e0 2024-09-11T23:33:16.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.333+0000 7fc994d84640 1 -- 172.21.15.43:0/228957870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99019d8e0 con 0x7fc9901036e0 2024-09-11T23:33:16.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.333+0000 7fc97f7fe640 1 -- 172.21.15.43:0/228957870 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984035070 con 0x7fc9901036e0 2024-09-11T23:33:16.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.333+0000 7fc994d84640 1 -- 172.21.15.43:0/228957870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9901a5220 con 0x7fc9901036e0 2024-09-11T23:33:16.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.334+0000 7fc994d84640 1 -- 172.21.15.43:0/228957870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc99006a850 con 0x7fc9901036e0 2024-09-11T23:33:16.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.335+0000 7fc97f7fe640 1 -- 172.21.15.43:0/228957870 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc984042030 con 0x7fc9901036e0 2024-09-11T23:33:16.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.335+0000 7fc97f7fe640 1 --2- 172.21.15.43:0/228957870 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc9780a2cf0 0x7fc9780a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:16.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.336+0000 7fc98e59c640 1 --2- 172.21.15.43:0/228957870 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc9780a2cf0 0x7fc9780a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:16.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.336+0000 7fc97f7fe640 1 -- 172.21.15.43:0/228957870 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8829+0+0 (secure 0 0 0) 0x7fc9840efb40 con 0x7fc9901036e0 2024-09-11T23:33:16.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.336+0000 7fc98e59c640 1 --2- 172.21.15.43:0/228957870 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc9780a2cf0 0x7fc9780a51b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc9800058c0 tx=0x7fc98000a2d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:16.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.340+0000 7fc97f7fe640 1 -- 172.21.15.43:0/228957870 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc9840bc630 con 0x7fc9901036e0 2024-09-11T23:33:16.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.517+0000 7fc994d84640 1 -- 172.21.15.43:0/228957870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc99010a2b0 con 0x7fc9901036e0 2024-09-11T23:33:16.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.904+0000 7fc97f7fe640 1 -- 172.21.15.43:0/228957870 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v97) ==== 181+0+0 (secure 0 0 0) 0x7fc9840bbf30 con 0x7fc9901036e0 2024-09-11T23:33:16.904 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-09-11T23:33:16.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.907+0000 7fc994d84640 1 -- 172.21.15.43:0/228957870 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc9780a2cf0 msgr2=0x7fc9780a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:16.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.908+0000 7fc994d84640 1 --2- 172.21.15.43:0/228957870 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc9780a2cf0 0x7fc9780a51b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc9800058c0 tx=0x7fc98000a2d0 comp rx=0 tx=0).stop 2024-09-11T23:33:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.908+0000 7fc994d84640 1 -- 172.21.15.43:0/228957870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9901036e0 msgr2=0x7fc990198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.908+0000 7fc994d84640 1 --2- 172.21.15.43:0/228957870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9901036e0 0x7fc990198be0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc984005940 tx=0x7fc9840042f0 comp rx=0 tx=0).stop 2024-09-11T23:33:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.908+0000 7fc994d84640 1 -- 172.21.15.43:0/228957870 shutdown_connections 2024-09-11T23:33:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.908+0000 7fc994d84640 1 --2- 172.21.15.43:0/228957870 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7fc9780a2cf0 0x7fc9780a51b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.908+0000 7fc994d84640 1 --2- 172.21.15.43:0/228957870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9901082a0 0x7fc99019cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.908+0000 7fc994d84640 1 --2- 172.21.15.43:0/228957870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9901036e0 0x7fc990198be0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.908+0000 7fc994d84640 1 --2- 172.21.15.43:0/228957870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc990102d30 0x7fc9901986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.908+0000 7fc994d84640 1 -- 172.21.15.43:0/228957870 >> 172.21.15.43:0/228957870 conn(0x7fc9900fc7d0 msgr2=0x7fc990106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:16.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.909+0000 7fc994d84640 1 -- 172.21.15.43:0/228957870 shutdown_connections 2024-09-11T23:33:16.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:16.909+0000 7fc994d84640 1 -- 172.21.15.43:0/228957870 wait complete. 2024-09-11T23:33:17.206 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:16 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/228957870' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:33:17.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:16 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/228957870' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:33:17.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:16 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/228957870' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:33:17.542 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T23:33:17.806 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:33:18.169 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:17 smithi043 ceph-mon[30950]: pgmap v357: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 102 B/s wr, 4 op/s 2024-09-11T23:33:18.169 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:17 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/228957870' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:33:18.169 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:17 smithi043 ceph-mon[30950]: osdmap e97: 12 total, 12 up, 12 in 2024-09-11T23:33:18.170 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:17 smithi090 ceph-mon[35130]: pgmap v357: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 102 B/s wr, 4 op/s 2024-09-11T23:33:18.170 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:17 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/228957870' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:33:18.171 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:17 smithi090 ceph-mon[35130]: osdmap e97: 12 total, 12 up, 12 in 2024-09-11T23:33:18.202 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:17 smithi097 ceph-mon[35562]: pgmap v357: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 102 B/s wr, 4 op/s 2024-09-11T23:33:18.202 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:17 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/228957870' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:33:18.202 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:17 smithi097 ceph-mon[35562]: osdmap e97: 12 total, 12 up, 12 in 2024-09-11T23:33:18.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.366+0000 7f1deecad640 1 -- 172.21.15.43:0/1988207996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de8102570 msgr2=0x7f1de81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:18.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.366+0000 7f1deecad640 1 --2- 172.21.15.43:0/1988207996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de8102570 0x7f1de81029b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009900 tx=0x7f1dd402a480 comp rx=0 tx=0).stop 2024-09-11T23:33:18.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.366+0000 7f1deecad640 1 -- 172.21.15.43:0/1988207996 shutdown_connections 2024-09-11T23:33:18.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.366+0000 7f1deecad640 1 --2- 172.21.15.43:0/1988207996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1de8102ef0 0x7f1de810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:18.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.366+0000 7f1deecad640 1 --2- 172.21.15.43:0/1988207996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de8102570 0x7f1de81029b0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:18.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.366+0000 7f1deecad640 1 --2- 172.21.15.43:0/1988207996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1de8100e20 0x7f1de8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:18.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.366+0000 7f1deecad640 1 -- 172.21.15.43:0/1988207996 >> 172.21.15.43:0/1988207996 conn(0x7f1de80fc7d0 msgr2=0x7f1de80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:18.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.366+0000 7f1deecad640 1 -- 172.21.15.43:0/1988207996 shutdown_connections 2024-09-11T23:33:18.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.366+0000 7f1deecad640 1 -- 172.21.15.43:0/1988207996 wait complete. 2024-09-11T23:33:18.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.367+0000 7f1deecad640 1 Processor -- start 2024-09-11T23:33:18.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.367+0000 7f1deecad640 1 -- start start 2024-09-11T23:33:18.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.367+0000 7f1deecad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1de8100e20 0x7f1de81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:18.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.367+0000 7f1deecad640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1de8102570 0x7f1de8198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:18.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.367+0000 7f1deecad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de8102ef0 0x7f1de819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:18.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.367+0000 7f1deecad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1de8110f80 con 0x7f1de8102ef0 2024-09-11T23:33:18.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.367+0000 7f1deecad640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1de8110e00 con 0x7f1de8102570 2024-09-11T23:33:18.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.367+0000 7f1deecad640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1de8111100 con 0x7f1de8100e20 2024-09-11T23:33:18.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.367+0000 7f1ded24a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de8102ef0 0x7f1de819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:18.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.367+0000 7f1ded24a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de8102ef0 0x7f1de819ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54272/0 (socket says 172.21.15.43:54272) 2024-09-11T23:33:18.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.367+0000 7f1de7fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1de8102570 0x7f1de8198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:18.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.367+0000 7f1ded24a640 1 -- 172.21.15.43:0/1878381194 learned_addr learned my addr 172.21.15.43:0/1878381194 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-11T23:33:18.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.367+0000 7f1deca49640 1 --2- 172.21.15.43:0/1878381194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1de8100e20 0x7f1de81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:18.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.368+0000 7f1ded24a640 1 -- 172.21.15.43:0/1878381194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1de8100e20 msgr2=0x7f1de81985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:18.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.368+0000 7f1ded24a640 1 --2- 172.21.15.43:0/1878381194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1de8100e20 0x7f1de81985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:18.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.368+0000 7f1ded24a640 1 -- 172.21.15.43:0/1878381194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1de8102570 msgr2=0x7f1de8198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:18.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.368+0000 7f1ded24a640 1 --2- 172.21.15.43:0/1878381194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1de8102570 0x7f1de8198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:18.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.368+0000 7f1ded24a640 1 -- 172.21.15.43:0/1878381194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de819d560 con 0x7f1de8102ef0 2024-09-11T23:33:18.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.368+0000 7f1ded24a640 1 --2- 172.21.15.43:0/1878381194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de8102ef0 0x7f1de819ce80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1de000cad0 tx=0x7f1de000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:18.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.368+0000 7f1de5ffb640 1 -- 172.21.15.43:0/1878381194 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de000ae20 con 0x7f1de8102ef0 2024-09-11T23:33:18.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.368+0000 7f1deecad640 1 -- 172.21.15.43:0/1878381194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de819d850 con 0x7f1de8102ef0 2024-09-11T23:33:18.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.368+0000 7f1de5ffb640 1 -- 172.21.15.43:0/1878381194 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1de0005020 con 0x7f1de8102ef0 2024-09-11T23:33:18.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.368+0000 7f1de5ffb640 1 -- 172.21.15.43:0/1878381194 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de000f6f0 con 0x7f1de8102ef0 2024-09-11T23:33:18.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.368+0000 7f1deecad640 1 -- 172.21.15.43:0/1878381194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de81a5130 con 0x7f1de8102ef0 2024-09-11T23:33:18.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.369+0000 7f1deecad640 1 -- 172.21.15.43:0/1878381194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1de8101200 con 0x7f1de8102ef0 2024-09-11T23:33:18.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.371+0000 7f1de5ffb640 1 -- 172.21.15.43:0/1878381194 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1de00051d0 con 0x7f1de8102ef0 2024-09-11T23:33:18.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.372+0000 7f1de5ffb640 1 --2- 172.21.15.43:0/1878381194 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f1dd80a2cf0 0x7f1dd80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:33:18.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.372+0000 7f1de5ffb640 1 -- 172.21.15.43:0/1878381194 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8304+0+0 (secure 0 0 0) 0x7f1de00cf650 con 0x7f1de8102ef0 2024-09-11T23:33:18.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.372+0000 7f1deca49640 1 --2- 172.21.15.43:0/1878381194 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f1dd80a2cf0 0x7f1dd80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:33:18.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.372+0000 7f1deca49640 1 --2- 172.21.15.43:0/1878381194 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f1dd80a2cf0 0x7f1dd80a51b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc005fd0 tx=0x7f1ddc00b390 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:33:18.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.375+0000 7f1de5ffb640 1 -- 172.21.15.43:0/1878381194 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1de0014030 con 0x7f1de8102ef0 2024-09-11T23:33:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.550+0000 7f1deecad640 1 -- 172.21.15.43:0/1878381194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1de810a0a0 con 0x7f1de8102ef0 2024-09-11T23:33:18.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.909+0000 7f1de5ffb640 1 -- 172.21.15.43:0/1878381194 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v98) ==== 169+0+0 (secure 0 0 0) 0x7f1de009bd00 con 0x7f1de8102ef0 2024-09-11T23:33:18.910 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-09-11T23:33:18.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.914+0000 7f1deecad640 1 -- 172.21.15.43:0/1878381194 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f1dd80a2cf0 msgr2=0x7f1dd80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:18.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.914+0000 7f1deecad640 1 --2- 172.21.15.43:0/1878381194 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f1dd80a2cf0 0x7f1dd80a51b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc005fd0 tx=0x7f1ddc00b390 comp rx=0 tx=0).stop 2024-09-11T23:33:18.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.914+0000 7f1deecad640 1 -- 172.21.15.43:0/1878381194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de8102ef0 msgr2=0x7f1de819ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:33:18.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.914+0000 7f1deecad640 1 --2- 172.21.15.43:0/1878381194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de8102ef0 0x7f1de819ce80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1de000cad0 tx=0x7f1de000cf90 comp rx=0 tx=0).stop 2024-09-11T23:33:18.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.914+0000 7f1deecad640 1 -- 172.21.15.43:0/1878381194 shutdown_connections 2024-09-11T23:33:18.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.914+0000 7f1deecad640 1 --2- 172.21.15.43:0/1878381194 >> [v2:172.21.15.43:6800/2510386404,v1:172.21.15.43:6801/2510386404] conn(0x7f1dd80a2cf0 0x7f1dd80a51b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:18.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.914+0000 7f1deecad640 1 --2- 172.21.15.43:0/1878381194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de8102ef0 0x7f1de819ce80 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:18.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.914+0000 7f1deecad640 1 --2- 172.21.15.43:0/1878381194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1de8102570 0x7f1de8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:18.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.914+0000 7f1deecad640 1 --2- 172.21.15.43:0/1878381194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1de8100e20 0x7f1de81985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:33:18.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.914+0000 7f1deecad640 1 -- 172.21.15.43:0/1878381194 >> 172.21.15.43:0/1878381194 conn(0x7f1de80fc7d0 msgr2=0x7f1de806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:33:18.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.916+0000 7f1deecad640 1 -- 172.21.15.43:0/1878381194 shutdown_connections 2024-09-11T23:33:18.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-11T23:33:18.916+0000 7f1deecad640 1 -- 172.21.15.43:0/1878381194 wait complete. 2024-09-11T23:33:19.126 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:18 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1878381194' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:33:19.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:18 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1878381194' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:33:19.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:18 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1878381194' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:33:20.181 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:19 smithi090 ceph-mon[35130]: pgmap v359: 65 pgs: 65 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:33:20.181 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:19 smithi090 ceph-mon[35130]: from='client.? 172.21.15.43:0/1878381194' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:33:20.182 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:19 smithi090 ceph-mon[35130]: osdmap e98: 12 total, 12 up, 12 in 2024-09-11T23:33:20.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:19 smithi097 ceph-mon[35562]: pgmap v359: 65 pgs: 65 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:33:20.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:19 smithi097 ceph-mon[35562]: from='client.? 172.21.15.43:0/1878381194' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:33:20.301 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:19 smithi097 ceph-mon[35562]: osdmap e98: 12 total, 12 up, 12 in 2024-09-11T23:33:20.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:19 smithi043 ceph-mon[30950]: pgmap v359: 65 pgs: 65 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:33:20.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:19 smithi043 ceph-mon[30950]: from='client.? 172.21.15.43:0/1878381194' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:33:20.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:19 smithi043 ceph-mon[30950]: osdmap e98: 12 total, 12 up, 12 in 2024-09-11T23:33:20.361 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:33:20.557 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:33:20.592 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:33:20.831 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-11T23:33:20.832 DEBUG:teuthology.orchestra.run.smithi043:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T23:33:20.874 DEBUG:teuthology.orchestra.run.smithi090:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T23:33:20.892 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T23:33:20.906 INFO:tasks.cephadm:Stopping all daemons... 2024-09-11T23:33:20.907 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-11T23:33:20.907 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mon.a 2024-09-11T23:33:21.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:20 smithi043 systemd[1]: Stopping Ceph mon.a for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:33:22.183 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:21 smithi097 ceph-mon[35562]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:33:22.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:21 smithi090 ceph-mon[35130]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:33:22.348 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:21 smithi043 ceph-mon[30950]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:33:23.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:22 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-a[30946]: 2024-09-11T23:33:22.733+0000 7f93651e1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:33:23.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:22 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-a[30946]: 2024-09-11T23:33:22.733+0000 7f93651e1640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-11T23:33:23.110 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:22 smithi043 podman[105323]: 2024-09-11 23:33:22.837646302 +0000 UTC m=+1.330224467 container died 176c0fba3df58a2066e3348fbb3d216f502a92f6be69c391569422176a7baee5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-11T23:33:23.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:23 smithi043 podman[105323]: 2024-09-11 23:33:23.224921203 +0000 UTC m=+1.717499368 container cleanup 176c0fba3df58a2066e3348fbb3d216f502a92f6be69c391569422176a7baee5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-11T23:33:23.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:23 smithi043 bash[105323]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-a 2024-09-11T23:33:23.945 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mon.a.service' 2024-09-11T23:33:24.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:23 smithi043 podman[105358]: 2024-09-11 23:33:23.656251551 +0000 UTC m=+0.808978596 container remove 176c0fba3df58a2066e3348fbb3d216f502a92f6be69c391569422176a7baee5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:33:24.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:23 smithi043 systemd[1]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T23:33:24.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:23 smithi043 systemd[1]: Stopped Ceph mon.a for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:33:24.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 11 23:33:23 smithi043 systemd[1]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mon.a.service: Consumed 21.378s CPU time. 2024-09-11T23:33:24.582 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:33:24.583 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-11T23:33:24.583 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-11T23:33:24.583 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mon.b 2024-09-11T23:33:25.019 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:24 smithi090 systemd[1]: Stopping Ceph mon.b for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:33:28.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:27 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-b[35126]: 2024-09-11T23:33:27.873+0000 7f9296ffd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:33:28.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:27 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-b[35126]: 2024-09-11T23:33:27.873+0000 7f9296ffd640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-11T23:33:28.270 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:28 smithi090 podman[70773]: 2024-09-11 23:33:28.093933641 +0000 UTC m=+2.734944373 container died 172cb0b311f0fa00e6cf1d9b273c04a111c4cc04939f268b0e9bfe563bfa647d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-b, RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T23:33:36.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:36 smithi090 podman[70773]: 2024-09-11 23:33:36.407230639 +0000 UTC m=+11.048241364 container cleanup 172cb0b311f0fa00e6cf1d9b273c04a111c4cc04939f268b0e9bfe563bfa647d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-11T23:33:36.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:36 smithi090 bash[70773]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-b 2024-09-11T23:33:41.559 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mon.b.service' 2024-09-11T23:33:41.580 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:41 smithi090 podman[70783]: 2024-09-11 23:33:41.326973059 +0000 UTC m=+13.229823241 container remove 172cb0b311f0fa00e6cf1d9b273c04a111c4cc04939f268b0e9bfe563bfa647d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T23:33:41.581 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:41 smithi090 systemd[1]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-11T23:33:41.581 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:41 smithi090 systemd[1]: Stopped Ceph mon.b for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:33:41.581 INFO:journalctl@ceph.mon.b.smithi090.stdout:Sep 11 23:33:41 smithi090 systemd[1]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mon.b.service: Consumed 12.440s CPU time. 2024-09-11T23:33:41.673 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:33:41.673 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-11T23:33:41.673 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-11T23:33:41.674 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mon.c 2024-09-11T23:33:42.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:41 smithi097 systemd[1]: Stopping Ceph mon.c for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:33:44.050 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:43 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-c[35550]: 2024-09-11T23:33:43.765+0000 7fe50625d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:33:44.051 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:43 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-c[35550]: 2024-09-11T23:33:43.765+0000 7fe50625d640 -1 mon.c@1(probing) e3 *** Got Signal Terminated *** 2024-09-11T23:33:44.395 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:44 smithi097 podman[71179]: 2024-09-11 23:33:44.061447571 +0000 UTC m=+1.522088162 container stop fba50890eb516b71025d6dc58dcaeba7aa072005f64b9571c16610ab3f9d03ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-c, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:33:44.395 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:44 smithi097 podman[71179]: 2024-09-11 23:33:44.087057727 +0000 UTC m=+1.547698335 container died fba50890eb516b71025d6dc58dcaeba7aa072005f64b9571c16610ab3f9d03ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-11T23:33:44.723 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:44 smithi097 podman[71179]: 2024-09-11 23:33:44.506163759 +0000 UTC m=+1.966804347 container cleanup fba50890eb516b71025d6dc58dcaeba7aa072005f64b9571c16610ab3f9d03ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-c, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-11T23:33:44.723 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:44 smithi097 bash[71179]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-c 2024-09-11T23:33:44.959 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mon.c.service' 2024-09-11T23:33:44.981 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:44 smithi097 podman[71191]: 2024-09-11 23:33:44.724049294 +0000 UTC m=+0.652703930 container remove fba50890eb516b71025d6dc58dcaeba7aa072005f64b9571c16610ab3f9d03ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mon-c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T23:33:44.981 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:44 smithi097 systemd[1]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mon.c.service: Deactivated successfully. 2024-09-11T23:33:44.981 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:44 smithi097 systemd[1]: Stopped Ceph mon.c for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:33:44.981 INFO:journalctl@ceph.mon.c.smithi097.stdout:Sep 11 23:33:44 smithi097 systemd[1]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mon.c.service: Consumed 12.864s CPU time. 2024-09-11T23:33:45.016 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:33:45.016 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-11T23:33:45.017 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-11T23:33:45.017 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mgr.x 2024-09-11T23:33:45.360 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:33:45 smithi043 systemd[1]: Stopping Ceph mgr.x for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:33:45.773 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:33:45 smithi043 podman[105517]: 2024-09-11 23:33:45.421738214 +0000 UTC m=+0.161207612 container died 9dc9cd17074c4580028d6d23dac45d73c2f88368b73dea84a9d79b8ec200e462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T23:33:46.038 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:33:46 smithi043 podman[105517]: 2024-09-11 23:33:46.01473066 +0000 UTC m=+0.754200050 container cleanup 9dc9cd17074c4580028d6d23dac45d73c2f88368b73dea84a9d79b8ec200e462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-11T23:33:46.038 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Sep 11 23:33:46 smithi043 bash[105517]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-x 2024-09-11T23:33:46.418 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-11T23:33:46.458 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:33:46.459 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-11T23:33:46.459 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-11T23:33:46.459 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mgr.y 2024-09-11T23:33:46.769 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:33:46 smithi090 systemd[1]: Stopping Ceph mgr.y for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:33:48.769 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:33:48 smithi090 podman[70892]: 2024-09-11 23:33:48.412490074 +0000 UTC m=+1.752778571 container died 98d6ff6839de7980bb97cbbfd861cf53d533c04f9547fe84a7923c0529f855f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731) 2024-09-11T23:33:54.520 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:33:54 smithi090 podman[70892]: 2024-09-11 23:33:54.204899183 +0000 UTC m=+7.545187689 container cleanup 98d6ff6839de7980bb97cbbfd861cf53d533c04f9547fe84a7923c0529f855f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-11T23:33:54.520 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:33:54 smithi090 bash[70892]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y 2024-09-11T23:33:57.816 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-11T23:33:57.839 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:33:57 smithi090 podman[70907]: 2024-09-11 23:33:57.580806871 +0000 UTC m=+8.800618826 container remove 98d6ff6839de7980bb97cbbfd861cf53d533c04f9547fe84a7923c0529f855f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-11T23:33:57.839 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:33:57 smithi090 systemd[1]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-11T23:33:57.839 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:33:57 smithi090 systemd[1]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-09-11T23:33:57.839 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:33:57 smithi090 systemd[1]: Stopped Ceph mgr.y for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:33:57.839 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 11 23:33:57 smithi090 systemd[1]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mgr.y.service: Consumed 11.495s CPU time. 2024-09-11T23:33:57.854 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:33:57.854 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-11T23:33:57.854 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-11T23:33:57.854 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mgr.z 2024-09-11T23:33:58.199 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:33:57 smithi097 systemd[1]: Stopping Ceph mgr.z for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:33:58.534 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:33:58 smithi097 podman[71320]: 2024-09-11 23:33:58.199508098 +0000 UTC m=+0.158832054 container died 907fd4394ca1a2d325b2e95b3837646d7db9c0e1439c7d4c8f65db487b7a36ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:33:58.800 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:33:58 smithi097 podman[71320]: 2024-09-11 23:33:58.643990541 +0000 UTC m=+0.603314487 container cleanup 907fd4394ca1a2d325b2e95b3837646d7db9c0e1439c7d4c8f65db487b7a36ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-11T23:33:58.801 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:33:58 smithi097 bash[71320]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z 2024-09-11T23:33:59.127 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-11T23:33:59.149 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:33:58 smithi097 podman[71332]: 2024-09-11 23:33:58.895615713 +0000 UTC m=+0.698264635 container remove 907fd4394ca1a2d325b2e95b3837646d7db9c0e1439c7d4c8f65db487b7a36ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mgr-z, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-11T23:33:59.149 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:33:58 smithi097 systemd[1]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-11T23:33:59.150 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:33:59 smithi097 systemd[1]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-09-11T23:33:59.150 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:33:59 smithi097 systemd[1]: Stopped Ceph mgr.z for 25c0cd84-7093-11ef-bcea-c7b262605968. 2024-09-11T23:33:59.150 INFO:journalctl@ceph.mgr.z.smithi097.stdout:Sep 11 23:33:59 smithi097 systemd[1]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mgr.z.service: Consumed 11.467s CPU time. 2024-09-11T23:33:59.164 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:33:59.165 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-11T23:33:59.165 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-11T23:33:59.165 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.0 2024-09-11T23:33:59.495 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:33:59 smithi043 systemd[1]: Stopping Ceph osd.0 for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:33:59.860 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:33:59 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-0[44650]: 2024-09-11T23:33:59.494+0000 7f7c323ba640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:33:59.860 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:33:59 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-0[44650]: 2024-09-11T23:33:59.494+0000 7f7c323ba640 -1 osd.0 98 *** Got signal Terminated *** 2024-09-11T23:33:59.860 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:33:59 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-0[44650]: 2024-09-11T23:33:59.494+0000 7f7c323ba640 -1 osd.0 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:34:05.360 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:34:05 smithi043 podman[105654]: 2024-09-11 23:34:05.106943179 +0000 UTC m=+5.739800217 container died c2fca8036d7385042c7bd6f6d22461cdc7b0a83225e622fd33390a8b130fcbb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-09-11T23:34:05.860 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:34:05 smithi043 podman[105654]: 2024-09-11 23:34:05.60309676 +0000 UTC m=+6.235953841 container cleanup c2fca8036d7385042c7bd6f6d22461cdc7b0a83225e622fd33390a8b130fcbb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-11T23:34:05.860 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:34:05 smithi043 bash[105654]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-0 2024-09-11T23:34:06.224 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:34:05 smithi043 podman[105669]: 2024-09-11 23:34:05.896390754 +0000 UTC m=+0.780575047 container remove c2fca8036d7385042c7bd6f6d22461cdc7b0a83225e622fd33390a8b130fcbb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:34:06.507 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:34:06 smithi043 podman[105764]: 2024-09-11 23:34:06.127749876 +0000 UTC m=+0.016358314 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:34:06.507 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:34:06 smithi043 podman[105764]: 2024-09-11 23:34:06.244014531 +0000 UTC m=+0.132622994 container create 13c0cdd7976f3847c3347dad8be3358ba863ba371d8b727ffd1a1eaa8a780f2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T23:34:06.508 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:34:06 smithi043 podman[105764]: 2024-09-11 23:34:06.508399293 +0000 UTC m=+0.397007723 container init 13c0cdd7976f3847c3347dad8be3358ba863ba371d8b727ffd1a1eaa8a780f2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-11T23:34:06.799 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:34:06 smithi043 podman[105764]: 2024-09-11 23:34:06.512023473 +0000 UTC m=+0.400631897 container start 13c0cdd7976f3847c3347dad8be3358ba863ba371d8b727ffd1a1eaa8a780f2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:34:06.799 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:34:06 smithi043 podman[105764]: 2024-09-11 23:34:06.608524144 +0000 UTC m=+0.497132566 container attach 13c0cdd7976f3847c3347dad8be3358ba863ba371d8b727ffd1a1eaa8a780f2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-11T23:34:07.110 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 11 23:34:06 smithi043 podman[105764]: 2024-09-11 23:34:06.800645284 +0000 UTC m=+0.689253707 container died 13c0cdd7976f3847c3347dad8be3358ba863ba371d8b727ffd1a1eaa8a780f2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-11T23:34:07.507 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.0.service' 2024-09-11T23:34:07.554 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:34:07.554 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-11T23:34:07.554 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-11T23:34:07.555 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.1 2024-09-11T23:34:07.866 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:34:07 smithi090 systemd[1]: Stopping Ceph osd.1 for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:34:08.269 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:34:07 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-1[39555]: 2024-09-11T23:34:07.866+0000 7fbe369d3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:34:08.269 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:34:07 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-1[39555]: 2024-09-11T23:34:07.866+0000 7fbe369d3640 -1 osd.1 98 *** Got signal Terminated *** 2024-09-11T23:34:08.269 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:34:07 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-1[39555]: 2024-09-11T23:34:07.866+0000 7fbe369d3640 -1 osd.1 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:34:13.985 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:34:13 smithi090 podman[71029]: 2024-09-11 23:34:13.684858683 +0000 UTC m=+5.938538648 container died b9da261655af3d59c75fc6f911d68c1c96f4ec8946e71c6733c0baefdf626e4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-1, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True) 2024-09-11T23:34:14.269 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:34:14 smithi090 podman[71029]: 2024-09-11 23:34:14.094955732 +0000 UTC m=+6.348635731 container cleanup b9da261655af3d59c75fc6f911d68c1c96f4ec8946e71c6733c0baefdf626e4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-11T23:34:14.269 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:34:14 smithi090 bash[71029]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-1 2024-09-11T23:34:14.751 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:34:14 smithi090 podman[71044]: 2024-09-11 23:34:14.330188737 +0000 UTC m=+0.634713783 container remove b9da261655af3d59c75fc6f911d68c1c96f4ec8946e71c6733c0baefdf626e4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-1, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True) 2024-09-11T23:34:15.019 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:34:14 smithi090 podman[71134]: 2024-09-11 23:34:14.655414296 +0000 UTC m=+0.015621567 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:34:15.019 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:34:14 smithi090 podman[71134]: 2024-09-11 23:34:14.781359568 +0000 UTC m=+0.141566819 container create 7cb098ae2a42378efcc48231a0ea0f2ccadc900016a01c7ffaa70361862682eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-09-11T23:34:15.294 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:34:15 smithi090 podman[71134]: 2024-09-11 23:34:15.051323062 +0000 UTC m=+0.411530323 container init 7cb098ae2a42378efcc48231a0ea0f2ccadc900016a01c7ffaa70361862682eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:34:15.295 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:34:15 smithi090 podman[71134]: 2024-09-11 23:34:15.054676705 +0000 UTC m=+0.414883966 container start 7cb098ae2a42378efcc48231a0ea0f2ccadc900016a01c7ffaa70361862682eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:34:15.295 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:34:15 smithi090 podman[71134]: 2024-09-11 23:34:15.168306753 +0000 UTC m=+0.528514012 container attach 7cb098ae2a42378efcc48231a0ea0f2ccadc900016a01c7ffaa70361862682eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True) 2024-09-11T23:34:15.623 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 11 23:34:15 smithi090 podman[71134]: 2024-09-11 23:34:15.318832724 +0000 UTC m=+0.679039981 container died 7cb098ae2a42378efcc48231a0ea0f2ccadc900016a01c7ffaa70361862682eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:34:15.952 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.1.service' 2024-09-11T23:34:15.991 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:34:15.992 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-11T23:34:15.992 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-11T23:34:15.992 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.2 2024-09-11T23:34:16.297 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:34:16 smithi097 systemd[1]: Stopping Ceph osd.2 for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:34:16.550 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:34:16 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-2[40538]: 2024-09-11T23:34:16.296+0000 7fbc97b88640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:34:16.551 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:34:16 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-2[40538]: 2024-09-11T23:34:16.296+0000 7fbc97b88640 -1 osd.2 98 *** Got signal Terminated *** 2024-09-11T23:34:16.551 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:34:16 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-2[40538]: 2024-09-11T23:34:16.296+0000 7fbc97b88640 -1 osd.2 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:34:22.223 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:34:21 smithi097 podman[71442]: 2024-09-11 23:34:21.920338648 +0000 UTC m=+5.728435174 container died af41eb00c8095f77c30a51024a41761982e524f9741d2588bc5ff9225515356e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-11T23:34:22.548 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:34:22 smithi097 podman[71442]: 2024-09-11 23:34:22.33133946 +0000 UTC m=+6.139435981 container cleanup af41eb00c8095f77c30a51024a41761982e524f9741d2588bc5ff9225515356e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, ceph=True) 2024-09-11T23:34:22.548 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:34:22 smithi097 bash[71442]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-2 2024-09-11T23:34:22.800 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:34:22 smithi097 podman[71457]: 2024-09-11 23:34:22.549478143 +0000 UTC m=+0.618953473 container remove af41eb00c8095f77c30a51024a41761982e524f9741d2588bc5ff9225515356e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-11T23:34:23.269 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:34:22 smithi097 podman[71547]: 2024-09-11 23:34:22.861658295 +0000 UTC m=+0.015926128 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:34:23.269 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:34:23 smithi097 podman[71547]: 2024-09-11 23:34:23.000441307 +0000 UTC m=+0.154709216 container create e7e87587ccdb85295c6c436bd3e306f3b207d3d81ae0e05ff70f05cc8460b383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-11T23:34:23.551 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:34:23 smithi097 podman[71547]: 2024-09-11 23:34:23.269840979 +0000 UTC m=+0.424108812 container init e7e87587ccdb85295c6c436bd3e306f3b207d3d81ae0e05ff70f05cc8460b383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-11T23:34:23.551 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:34:23 smithi097 podman[71547]: 2024-09-11 23:34:23.274495113 +0000 UTC m=+0.428762949 container start e7e87587ccdb85295c6c436bd3e306f3b207d3d81ae0e05ff70f05cc8460b383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:34:23.551 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:34:23 smithi097 podman[71547]: 2024-09-11 23:34:23.411551232 +0000 UTC m=+0.565819067 container attach e7e87587ccdb85295c6c436bd3e306f3b207d3d81ae0e05ff70f05cc8460b383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:34:23.955 INFO:journalctl@ceph.osd.2.smithi097.stdout:Sep 11 23:34:23 smithi097 podman[71547]: 2024-09-11 23:34:23.601003742 +0000 UTC m=+0.755271579 container died e7e87587ccdb85295c6c436bd3e306f3b207d3d81ae0e05ff70f05cc8460b383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-11T23:34:24.280 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.2.service' 2024-09-11T23:34:24.328 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:34:24.329 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-11T23:34:24.329 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-11T23:34:24.329 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.3 2024-09-11T23:34:24.610 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 23:34:24 smithi043 systemd[1]: Stopping Ceph osd.3 for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:34:25.110 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 23:34:24 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-3[51410]: 2024-09-11T23:34:24.623+0000 7f8e8fa33640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:34:25.110 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 23:34:24 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-3[51410]: 2024-09-11T23:34:24.623+0000 7f8e8fa33640 -1 osd.3 98 *** Got signal Terminated *** 2024-09-11T23:34:25.110 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 23:34:24 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-3[51410]: 2024-09-11T23:34:24.623+0000 7f8e8fa33640 -1 osd.3 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:34:30.610 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 23:34:30 smithi043 podman[105890]: 2024-09-11 23:34:30.346665162 +0000 UTC m=+5.810888066 container died 26543efd1f9d8d6ec95da5645ec6fcc410d1b49d1ed1e505871752c2f57ddd0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-11T23:34:31.100 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 23:34:30 smithi043 podman[105890]: 2024-09-11 23:34:30.849328963 +0000 UTC m=+6.313551900 container cleanup 26543efd1f9d8d6ec95da5645ec6fcc410d1b49d1ed1e505871752c2f57ddd0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-11T23:34:31.100 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 23:34:30 smithi043 bash[105890]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-3 2024-09-11T23:34:31.360 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 23:34:31 smithi043 podman[105907]: 2024-09-11 23:34:31.100684508 +0000 UTC m=+0.754226930 container remove 26543efd1f9d8d6ec95da5645ec6fcc410d1b49d1ed1e505871752c2f57ddd0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-09-11T23:34:31.778 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 23:34:31 smithi043 podman[105998]: 2024-09-11 23:34:31.41176258 +0000 UTC m=+0.016259578 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:34:31.778 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 23:34:31 smithi043 podman[105998]: 2024-09-11 23:34:31.518359724 +0000 UTC m=+0.122856706 container create 7c466dc08090a2fb27409614159dc46cb94f238244a0c007d3052eb743d71f01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-11T23:34:32.110 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 23:34:31 smithi043 podman[105998]: 2024-09-11 23:34:31.779154249 +0000 UTC m=+0.383651240 container init 7c466dc08090a2fb27409614159dc46cb94f238244a0c007d3052eb743d71f01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T23:34:32.110 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 23:34:31 smithi043 podman[105998]: 2024-09-11 23:34:31.782873938 +0000 UTC m=+0.387370920 container start 7c466dc08090a2fb27409614159dc46cb94f238244a0c007d3052eb743d71f01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:34:32.110 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 23:34:31 smithi043 podman[105998]: 2024-09-11 23:34:31.887597007 +0000 UTC m=+0.492094002 container attach 7c466dc08090a2fb27409614159dc46cb94f238244a0c007d3052eb743d71f01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T23:34:32.110 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 11 23:34:31 smithi043 podman[105998]: 2024-09-11 23:34:31.982116081 +0000 UTC m=+0.586613065 container died 7c466dc08090a2fb27409614159dc46cb94f238244a0c007d3052eb743d71f01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3) 2024-09-11T23:34:32.694 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.3.service' 2024-09-11T23:34:32.742 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:34:32.742 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-11T23:34:32.742 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-11T23:34:32.742 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.4 2024-09-11T23:34:33.080 INFO:journalctl@ceph.osd.4.smithi090.stdout:Sep 11 23:34:32 smithi090 systemd[1]: Stopping Ceph osd.4 for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:34:33.519 INFO:journalctl@ceph.osd.4.smithi090.stdout:Sep 11 23:34:33 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-4[46309]: 2024-09-11T23:34:33.080+0000 7f6da2829640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:34:33.519 INFO:journalctl@ceph.osd.4.smithi090.stdout:Sep 11 23:34:33 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-4[46309]: 2024-09-11T23:34:33.080+0000 7f6da2829640 -1 osd.4 98 *** Got signal Terminated *** 2024-09-11T23:34:33.519 INFO:journalctl@ceph.osd.4.smithi090.stdout:Sep 11 23:34:33 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-4[46309]: 2024-09-11T23:34:33.080+0000 7f6da2829640 -1 osd.4 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:34:39.019 INFO:journalctl@ceph.osd.4.smithi090.stdout:Sep 11 23:34:38 smithi090 podman[71251]: 2024-09-11 23:34:38.728928639 +0000 UTC m=+5.790997952 container died b794ba6904af9749f06df61ea356b0503d5e948c870b613e4a61b2b5bbf4361b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2) 2024-09-11T23:34:39.517 INFO:journalctl@ceph.osd.4.smithi090.stdout:Sep 11 23:34:39 smithi090 podman[71251]: 2024-09-11 23:34:39.183276199 +0000 UTC m=+6.245345501 container cleanup b794ba6904af9749f06df61ea356b0503d5e948c870b613e4a61b2b5bbf4361b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-11T23:34:39.517 INFO:journalctl@ceph.osd.4.smithi090.stdout:Sep 11 23:34:39 smithi090 bash[71251]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-4 2024-09-11T23:34:39.769 INFO:journalctl@ceph.osd.4.smithi090.stdout:Sep 11 23:34:39 smithi090 podman[71269]: 2024-09-11 23:34:39.518575104 +0000 UTC m=+0.777609005 container remove b794ba6904af9749f06df61ea356b0503d5e948c870b613e4a61b2b5bbf4361b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-11T23:34:40.132 INFO:journalctl@ceph.osd.4.smithi090.stdout:Sep 11 23:34:39 smithi090 podman[71355]: 2024-09-11 23:34:39.730322592 +0000 UTC m=+0.015921631 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:34:40.132 INFO:journalctl@ceph.osd.4.smithi090.stdout:Sep 11 23:34:39 smithi090 podman[71355]: 2024-09-11 23:34:39.844765008 +0000 UTC m=+0.130364044 container create 30925e12ef38818c1fe0b834ae2ce936f5b176fb477812f0e7fd9ba674060f19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T23:34:40.520 INFO:journalctl@ceph.osd.4.smithi090.stdout:Sep 11 23:34:40 smithi090 podman[71355]: 2024-09-11 23:34:40.132487077 +0000 UTC m=+0.418086113 container init 30925e12ef38818c1fe0b834ae2ce936f5b176fb477812f0e7fd9ba674060f19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T23:34:40.520 INFO:journalctl@ceph.osd.4.smithi090.stdout:Sep 11 23:34:40 smithi090 podman[71355]: 2024-09-11 23:34:40.135793217 +0000 UTC m=+0.421392257 container start 30925e12ef38818c1fe0b834ae2ce936f5b176fb477812f0e7fd9ba674060f19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-11T23:34:40.520 INFO:journalctl@ceph.osd.4.smithi090.stdout:Sep 11 23:34:40 smithi090 podman[71355]: 2024-09-11 23:34:40.241314144 +0000 UTC m=+0.526913193 container attach 30925e12ef38818c1fe0b834ae2ce936f5b176fb477812f0e7fd9ba674060f19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:34:40.520 INFO:journalctl@ceph.osd.4.smithi090.stdout:Sep 11 23:34:40 smithi090 podman[71355]: 2024-09-11 23:34:40.325966611 +0000 UTC m=+0.611565659 container died 30925e12ef38818c1fe0b834ae2ce936f5b176fb477812f0e7fd9ba674060f19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T23:34:40.992 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.4.service' 2024-09-11T23:34:41.034 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:34:41.035 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-11T23:34:41.035 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-11T23:34:41.035 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.5 2024-09-11T23:34:41.352 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 11 23:34:41 smithi097 systemd[1]: Stopping Ceph osd.5 for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:34:41.802 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 11 23:34:41 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-5[46989]: 2024-09-11T23:34:41.351+0000 7fc9216a5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:34:41.803 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 11 23:34:41 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-5[46989]: 2024-09-11T23:34:41.351+0000 7fc9216a5640 -1 osd.5 98 *** Got signal Terminated *** 2024-09-11T23:34:41.803 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 11 23:34:41 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-5[46989]: 2024-09-11T23:34:41.351+0000 7fc9216a5640 -1 osd.5 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:34:47.210 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 11 23:34:46 smithi097 podman[71663]: 2024-09-11 23:34:46.912579162 +0000 UTC m=+5.680563342 container died aea2317b7054d2bfaa705c411587c57e8a67bcf40dda9eeb24455f0972f106d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-5, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-11T23:34:47.551 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 11 23:34:47 smithi097 podman[71663]: 2024-09-11 23:34:47.310128414 +0000 UTC m=+6.078112592 container cleanup aea2317b7054d2bfaa705c411587c57e8a67bcf40dda9eeb24455f0972f106d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-5, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:34:47.551 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 11 23:34:47 smithi097 bash[71663]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-5 2024-09-11T23:34:47.892 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 11 23:34:47 smithi097 podman[71679]: 2024-09-11 23:34:47.586636279 +0000 UTC m=+0.675456271 container remove aea2317b7054d2bfaa705c411587c57e8a67bcf40dda9eeb24455f0972f106d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-5, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-11T23:34:48.173 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 11 23:34:47 smithi097 podman[71765]: 2024-09-11 23:34:47.795939342 +0000 UTC m=+0.015623263 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:34:48.173 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 11 23:34:47 smithi097 podman[71765]: 2024-09-11 23:34:47.920822176 +0000 UTC m=+0.140506082 container create 2c79842bb652cb8f5f6005136bc292cace39dd81eb39a0e553e7bf8f4ba74da9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2) 2024-09-11T23:34:48.173 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 11 23:34:48 smithi097 podman[71765]: 2024-09-11 23:34:48.173836666 +0000 UTC m=+0.393520580 container init 2c79842bb652cb8f5f6005136bc292cace39dd81eb39a0e553e7bf8f4ba74da9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-11T23:34:48.551 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 11 23:34:48 smithi097 podman[71765]: 2024-09-11 23:34:48.177420362 +0000 UTC m=+0.397104278 container start 2c79842bb652cb8f5f6005136bc292cace39dd81eb39a0e553e7bf8f4ba74da9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-11T23:34:48.551 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 11 23:34:48 smithi097 podman[71765]: 2024-09-11 23:34:48.273401191 +0000 UTC m=+0.493085108 container attach 2c79842bb652cb8f5f6005136bc292cace39dd81eb39a0e553e7bf8f4ba74da9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:34:48.551 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 11 23:34:48 smithi097 podman[71765]: 2024-09-11 23:34:48.366766358 +0000 UTC m=+0.586450274 container died 2c79842bb652cb8f5f6005136bc292cace39dd81eb39a0e553e7bf8f4ba74da9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-11T23:34:49.041 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.5.service' 2024-09-11T23:34:49.079 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:34:49.079 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-11T23:34:49.079 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-11T23:34:49.080 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.6 2024-09-11T23:34:49.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 11 23:34:49 smithi043 systemd[1]: Stopping Ceph osd.6 for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:34:49.820 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 11 23:34:49 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-11T23:34:49.418+0000 7fb3eceaf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:34:49.820 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 11 23:34:49 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-11T23:34:49.418+0000 7fb3eceaf640 -1 osd.6 98 *** Got signal Terminated *** 2024-09-11T23:34:49.820 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 11 23:34:49 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-11T23:34:49.418+0000 7fb3eceaf640 -1 osd.6 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:34:55.252 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 11 23:34:54 smithi043 podman[106102]: 2024-09-11 23:34:54.949531584 +0000 UTC m=+5.666503272 container died 4055caad1f2a503d6001cc2b3763d28a91dac72c19a0a844588e4b4a12a4385d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-6, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-11T23:34:55.610 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 11 23:34:55 smithi043 podman[106102]: 2024-09-11 23:34:55.418440286 +0000 UTC m=+6.135411973 container cleanup 4055caad1f2a503d6001cc2b3763d28a91dac72c19a0a844588e4b4a12a4385d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-6, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:34:55.610 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 11 23:34:55 smithi043 bash[106102]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-6 2024-09-11T23:34:56.076 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 11 23:34:55 smithi043 podman[106129]: 2024-09-11 23:34:55.761623706 +0000 UTC m=+0.805724941 container remove 4055caad1f2a503d6001cc2b3763d28a91dac72c19a0a844588e4b4a12a4385d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-11T23:34:56.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 11 23:34:56 smithi043 podman[106219]: 2024-09-11 23:34:55.979956238 +0000 UTC m=+0.016451040 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:34:56.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 11 23:34:56 smithi043 podman[106219]: 2024-09-11 23:34:56.087717792 +0000 UTC m=+0.124212640 container create f9e878176a0fabaa77c4cd6168244df22623bee4e0d25927ac6250d6a37cb872 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-11T23:34:56.360 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 11 23:34:56 smithi043 podman[106219]: 2024-09-11 23:34:56.315231635 +0000 UTC m=+0.351726438 container init f9e878176a0fabaa77c4cd6168244df22623bee4e0d25927ac6250d6a37cb872 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-11T23:34:56.361 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 11 23:34:56 smithi043 podman[106219]: 2024-09-11 23:34:56.318664147 +0000 UTC m=+0.355158954 container start f9e878176a0fabaa77c4cd6168244df22623bee4e0d25927ac6250d6a37cb872 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-11T23:34:56.859 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 11 23:34:56 smithi043 podman[106219]: 2024-09-11 23:34:56.415304529 +0000 UTC m=+0.451799349 container attach f9e878176a0fabaa77c4cd6168244df22623bee4e0d25927ac6250d6a37cb872 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-11T23:34:56.859 INFO:journalctl@ceph.osd.6.smithi043.stdout:Sep 11 23:34:56 smithi043 podman[106219]: 2024-09-11 23:34:56.512861091 +0000 UTC m=+0.549355902 container died f9e878176a0fabaa77c4cd6168244df22623bee4e0d25927ac6250d6a37cb872 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3) 2024-09-11T23:34:57.300 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.6.service' 2024-09-11T23:34:57.350 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:34:57.351 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-11T23:34:57.351 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-11T23:34:57.351 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.7 2024-09-11T23:34:57.693 INFO:journalctl@ceph.osd.7.smithi090.stdout:Sep 11 23:34:57 smithi090 systemd[1]: Stopping Ceph osd.7 for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:34:57.957 INFO:journalctl@ceph.osd.7.smithi090.stdout:Sep 11 23:34:57 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-7[52823]: 2024-09-11T23:34:57.692+0000 7f2e6c59a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:34:57.957 INFO:journalctl@ceph.osd.7.smithi090.stdout:Sep 11 23:34:57 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-7[52823]: 2024-09-11T23:34:57.692+0000 7f2e6c59a640 -1 osd.7 98 *** Got signal Terminated *** 2024-09-11T23:34:57.958 INFO:journalctl@ceph.osd.7.smithi090.stdout:Sep 11 23:34:57 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-7[52823]: 2024-09-11T23:34:57.692+0000 7f2e6c59a640 -1 osd.7 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:35:03.703 INFO:journalctl@ceph.osd.7.smithi090.stdout:Sep 11 23:35:03 smithi090 podman[71457]: 2024-09-11 23:35:03.364781327 +0000 UTC m=+5.812956951 container died 30578d87addfd123bfc2b497df2eae0fd7ddd5db6d482a6bdf49c3d855e19493 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T23:35:04.019 INFO:journalctl@ceph.osd.7.smithi090.stdout:Sep 11 23:35:03 smithi090 podman[71457]: 2024-09-11 23:35:03.828327297 +0000 UTC m=+6.276502919 container cleanup 30578d87addfd123bfc2b497df2eae0fd7ddd5db6d482a6bdf49c3d855e19493 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-7, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2) 2024-09-11T23:35:04.020 INFO:journalctl@ceph.osd.7.smithi090.stdout:Sep 11 23:35:03 smithi090 bash[71457]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-7 2024-09-11T23:35:04.489 INFO:journalctl@ceph.osd.7.smithi090.stdout:Sep 11 23:35:04 smithi090 podman[71484]: 2024-09-11 23:35:04.172202831 +0000 UTC m=+0.799358537 container remove 30578d87addfd123bfc2b497df2eae0fd7ddd5db6d482a6bdf49c3d855e19493 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-11T23:35:04.769 INFO:journalctl@ceph.osd.7.smithi090.stdout:Sep 11 23:35:04 smithi090 podman[71573]: 2024-09-11 23:35:04.393444832 +0000 UTC m=+0.015635322 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:35:04.769 INFO:journalctl@ceph.osd.7.smithi090.stdout:Sep 11 23:35:04 smithi090 podman[71573]: 2024-09-11 23:35:04.51610541 +0000 UTC m=+0.138295890 container create b53974770891a8c377ae6cfbc935f65d5ae5b5e183ed169183ca4f3d56061836 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, ceph=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-11T23:35:05.269 INFO:journalctl@ceph.osd.7.smithi090.stdout:Sep 11 23:35:04 smithi090 podman[71573]: 2024-09-11 23:35:04.77727587 +0000 UTC m=+0.399466349 container init b53974770891a8c377ae6cfbc935f65d5ae5b5e183ed169183ca4f3d56061836 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-11T23:35:05.270 INFO:journalctl@ceph.osd.7.smithi090.stdout:Sep 11 23:35:04 smithi090 podman[71573]: 2024-09-11 23:35:04.780693781 +0000 UTC m=+0.402884269 container start b53974770891a8c377ae6cfbc935f65d5ae5b5e183ed169183ca4f3d56061836 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:35:05.270 INFO:journalctl@ceph.osd.7.smithi090.stdout:Sep 11 23:35:04 smithi090 podman[71573]: 2024-09-11 23:35:04.885896693 +0000 UTC m=+0.508087173 container attach b53974770891a8c377ae6cfbc935f65d5ae5b5e183ed169183ca4f3d56061836 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-11T23:35:05.270 INFO:journalctl@ceph.osd.7.smithi090.stdout:Sep 11 23:35:04 smithi090 podman[71573]: 2024-09-11 23:35:04.965097496 +0000 UTC m=+0.587287977 container died b53974770891a8c377ae6cfbc935f65d5ae5b5e183ed169183ca4f3d56061836 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-7-deactivate, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-09-11T23:35:05.651 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.7.service' 2024-09-11T23:35:05.691 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:35:05.691 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-11T23:35:05.691 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-11T23:35:05.692 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.8 2024-09-11T23:35:06.013 INFO:journalctl@ceph.osd.8.smithi097.stdout:Sep 11 23:35:05 smithi097 systemd[1]: Stopping Ceph osd.8 for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:35:06.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:05 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:05.625+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:06.271 INFO:journalctl@ceph.osd.8.smithi097.stdout:Sep 11 23:35:06 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-8[53913]: 2024-09-11T23:35:06.012+0000 7f3b7a420640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:35:06.271 INFO:journalctl@ceph.osd.8.smithi097.stdout:Sep 11 23:35:06 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-8[53913]: 2024-09-11T23:35:06.012+0000 7f3b7a420640 -1 osd.8 98 *** Got signal Terminated *** 2024-09-11T23:35:06.272 INFO:journalctl@ceph.osd.8.smithi097.stdout:Sep 11 23:35:06 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-8[53913]: 2024-09-11T23:35:06.012+0000 7f3b7a420640 -1 osd.8 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:35:06.360 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:05 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9[65131]: 2024-09-11T23:35:05.875+0000 7f05f0e0e640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:00.552692+0000 front 2024-09-11T23:34:00.553041+0000 (oldest deadline 2024-09-11T23:35:05.852760+0000) 2024-09-11T23:35:07.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:06 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:06.671+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:07.110 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:06 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9[65131]: 2024-09-11T23:35:06.850+0000 7f05f0e0e640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:00.552692+0000 front 2024-09-11T23:34:00.553041+0000 (oldest deadline 2024-09-11T23:35:05.852760+0000) 2024-09-11T23:35:08.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:07 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:07.635+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:08.301 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:07 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:07.877+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:08.360 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:07 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9[65131]: 2024-09-11T23:35:07.895+0000 7f05f0e0e640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:00.552692+0000 front 2024-09-11T23:34:00.553041+0000 (oldest deadline 2024-09-11T23:35:05.852760+0000) 2024-09-11T23:35:09.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:08 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:08.645+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:09.300 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:08 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:08.922+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:09.360 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:08 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9[65131]: 2024-09-11T23:35:08.881+0000 7f05f0e0e640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:00.552692+0000 front 2024-09-11T23:34:00.553041+0000 (oldest deadline 2024-09-11T23:35:05.852760+0000) 2024-09-11T23:35:10.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:09 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:09.620+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:10.301 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:09 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:09.951+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:10.360 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:09 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9[65131]: 2024-09-11T23:35:09.880+0000 7f05f0e0e640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:00.552692+0000 front 2024-09-11T23:34:00.553041+0000 (oldest deadline 2024-09-11T23:35:05.852760+0000) 2024-09-11T23:35:11.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:10 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:10.581+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:11.300 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:10 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:10.955+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:11.360 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:10 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9[65131]: 2024-09-11T23:35:10.924+0000 7f05f0e0e640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:00.552692+0000 front 2024-09-11T23:34:00.553041+0000 (oldest deadline 2024-09-11T23:35:05.852760+0000) 2024-09-11T23:35:11.879 INFO:journalctl@ceph.osd.8.smithi097.stdout:Sep 11 23:35:11 smithi097 podman[71868]: 2024-09-11 23:35:11.595157414 +0000 UTC m=+5.709500915 container died ebb786151e1887d4d220bb4f8b8bb987eb8c60d4b3d53b644b21be43a6c11d79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-8, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T23:35:12.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:11 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:11.580+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:12.271 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:11 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:11.981+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:12.271 INFO:journalctl@ceph.osd.8.smithi097.stdout:Sep 11 23:35:11 smithi097 podman[71868]: 2024-09-11 23:35:11.987616785 +0000 UTC m=+6.101960317 container cleanup ebb786151e1887d4d220bb4f8b8bb987eb8c60d4b3d53b644b21be43a6c11d79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-11T23:35:12.271 INFO:journalctl@ceph.osd.8.smithi097.stdout:Sep 11 23:35:11 smithi097 bash[71868]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-8 2024-09-11T23:35:12.360 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:11 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9[65131]: 2024-09-11T23:35:11.886+0000 7f05f0e0e640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:00.552692+0000 front 2024-09-11T23:34:00.553041+0000 (oldest deadline 2024-09-11T23:35:05.852760+0000) 2024-09-11T23:35:12.551 INFO:journalctl@ceph.osd.8.smithi097.stdout:Sep 11 23:35:12 smithi097 podman[71897]: 2024-09-11 23:35:12.27210851 +0000 UTC m=+0.675331020 container remove ebb786151e1887d4d220bb4f8b8bb987eb8c60d4b3d53b644b21be43a6c11d79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-8, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2) 2024-09-11T23:35:12.936 INFO:journalctl@ceph.osd.8.smithi097.stdout:Sep 11 23:35:12 smithi097 podman[71986]: 2024-09-11 23:35:12.505837549 +0000 UTC m=+0.015625864 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:35:12.936 INFO:journalctl@ceph.osd.8.smithi097.stdout:Sep 11 23:35:12 smithi097 podman[71986]: 2024-09-11 23:35:12.614759703 +0000 UTC m=+0.124548017 container create 2873f862c34d36d5b24dc7457ecce935955cf254d3777e0f55760a2190d9d2fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-11T23:35:12.937 INFO:journalctl@ceph.osd.8.smithi097.stdout:Sep 11 23:35:12 smithi097 podman[71986]: 2024-09-11 23:35:12.835975685 +0000 UTC m=+0.345764012 container init 2873f862c34d36d5b24dc7457ecce935955cf254d3777e0f55760a2190d9d2fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True) 2024-09-11T23:35:12.937 INFO:journalctl@ceph.osd.8.smithi097.stdout:Sep 11 23:35:12 smithi097 podman[71986]: 2024-09-11 23:35:12.839692152 +0000 UTC m=+0.349480469 container start 2873f862c34d36d5b24dc7457ecce935955cf254d3777e0f55760a2190d9d2fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-11T23:35:12.937 INFO:journalctl@ceph.osd.8.smithi097.stdout:Sep 11 23:35:12 smithi097 podman[71986]: 2024-09-11 23:35:12.935921376 +0000 UTC m=+0.445709705 container attach 2873f862c34d36d5b24dc7457ecce935955cf254d3777e0f55760a2190d9d2fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True) 2024-09-11T23:35:13.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:12 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:12.608+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:13.110 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:12 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9[65131]: 2024-09-11T23:35:12.837+0000 7f05f0e0e640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:00.552692+0000 front 2024-09-11T23:34:00.553041+0000 (oldest deadline 2024-09-11T23:35:05.852760+0000) 2024-09-11T23:35:13.301 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:13 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:13.013+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:13.301 INFO:journalctl@ceph.osd.8.smithi097.stdout:Sep 11 23:35:13 smithi097 podman[72009]: 2024-09-11 23:35:13.05030972 +0000 UTC m=+0.016725585 container died 2873f862c34d36d5b24dc7457ecce935955cf254d3777e0f55760a2190d9d2fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-11T23:35:13.689 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.8.service' 2024-09-11T23:35:13.731 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:35:13.731 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-11T23:35:13.732 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-11T23:35:13.732 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.9 2024-09-11T23:35:14.020 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:13 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:13.564+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:14.046 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:13 smithi043 systemd[1]: Stopping Ceph osd.9 for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:35:14.046 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:13 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9[65131]: 2024-09-11T23:35:13.845+0000 7f05f0e0e640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:00.552692+0000 front 2024-09-11T23:34:00.553041+0000 (oldest deadline 2024-09-11T23:35:05.852760+0000) 2024-09-11T23:35:14.046 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:14 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9[65131]: 2024-09-11T23:35:14.045+0000 7f05f8800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:35:14.046 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:14 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9[65131]: 2024-09-11T23:35:14.045+0000 7f05f8800640 -1 osd.9 98 *** Got signal Terminated *** 2024-09-11T23:35:14.046 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:14 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9[65131]: 2024-09-11T23:35:14.045+0000 7f05f8800640 -1 osd.9 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:35:14.300 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:14 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:14.028+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:15.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:14 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:14.596+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:15.020 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:14 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:14.596+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-09-11T23:34:08.273947+0000 front 2024-09-11T23:34:08.273916+0000 (oldest deadline 2024-09-11T23:35:14.174121+0000) 2024-09-11T23:35:15.301 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:14 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:14.991+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:15.360 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:14 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9[65131]: 2024-09-11T23:35:14.862+0000 7f05f0e0e640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:00.552692+0000 front 2024-09-11T23:34:00.553041+0000 (oldest deadline 2024-09-11T23:35:05.852760+0000) 2024-09-11T23:35:16.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:15 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:15.595+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:16.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:15 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:15.595+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-09-11T23:34:08.273947+0000 front 2024-09-11T23:34:08.273916+0000 (oldest deadline 2024-09-11T23:35:14.174121+0000) 2024-09-11T23:35:16.300 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:15 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:15.986+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:16.360 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:15 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9[65131]: 2024-09-11T23:35:15.869+0000 7f05f0e0e640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:00.552692+0000 front 2024-09-11T23:34:00.553041+0000 (oldest deadline 2024-09-11T23:35:05.852760+0000) 2024-09-11T23:35:17.021 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:16 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:16.566+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:17.022 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:16 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:16.566+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-09-11T23:34:08.273947+0000 front 2024-09-11T23:34:08.273916+0000 (oldest deadline 2024-09-11T23:35:14.174121+0000) 2024-09-11T23:35:17.051 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:16 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:16.988+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:17.360 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:16 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9[65131]: 2024-09-11T23:35:16.894+0000 7f05f0e0e640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:00.552692+0000 front 2024-09-11T23:34:00.553041+0000 (oldest deadline 2024-09-11T23:35:05.852760+0000) 2024-09-11T23:35:18.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:17 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:17.611+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:18.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:17 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:17.611+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-09-11T23:34:08.273947+0000 front 2024-09-11T23:34:08.273916+0000 (oldest deadline 2024-09-11T23:35:14.174121+0000) 2024-09-11T23:35:18.300 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:18 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:18.035+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:18.359 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:17 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9[65131]: 2024-09-11T23:35:17.870+0000 7f05f0e0e640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:00.552692+0000 front 2024-09-11T23:34:00.553041+0000 (oldest deadline 2024-09-11T23:35:05.852760+0000) 2024-09-11T23:35:19.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:18 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:18.592+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:19.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:18 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:18.592+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-09-11T23:34:08.273947+0000 front 2024-09-11T23:34:08.273916+0000 (oldest deadline 2024-09-11T23:35:14.174121+0000) 2024-09-11T23:35:19.110 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:18 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9[65131]: 2024-09-11T23:35:18.826+0000 7f05f0e0e640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:00.552692+0000 front 2024-09-11T23:34:00.553041+0000 (oldest deadline 2024-09-11T23:35:05.852760+0000) 2024-09-11T23:35:19.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:18 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9[65131]: 2024-09-11T23:35:18.826+0000 7f05f0e0e640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-09-11T23:34:12.254185+0000 front 2024-09-11T23:34:12.254217+0000 (oldest deadline 2024-09-11T23:35:18.154134+0000) 2024-09-11T23:35:19.300 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:19 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:19.024+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:20.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:19 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:19.605+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:20.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:19 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:19.605+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-09-11T23:34:08.273947+0000 front 2024-09-11T23:34:08.273916+0000 (oldest deadline 2024-09-11T23:35:14.174121+0000) 2024-09-11T23:35:20.039 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:19 smithi043 podman[106324]: 2024-09-11 23:35:19.772573716 +0000 UTC m=+5.844065092 container died 7dc9095d7a81247e2db82ac49071046a952581cd1cd6790bfb2233e891404806 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:35:20.300 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:20 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:20.015+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:20.360 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:20 smithi043 podman[106324]: 2024-09-11 23:35:20.195528769 +0000 UTC m=+6.267020137 container cleanup 7dc9095d7a81247e2db82ac49071046a952581cd1cd6790bfb2233e891404806 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3) 2024-09-11T23:35:20.360 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:20 smithi043 bash[106324]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9 2024-09-11T23:35:20.805 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:20 smithi043 podman[106339]: 2024-09-11 23:35:20.396801884 +0000 UTC m=+0.621714150 container remove 7dc9095d7a81247e2db82ac49071046a952581cd1cd6790bfb2233e891404806 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T23:35:21.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:20 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:20.642+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:21.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:20 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:20.642+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-09-11T23:34:08.273947+0000 front 2024-09-11T23:34:08.273916+0000 (oldest deadline 2024-09-11T23:35:14.174121+0000) 2024-09-11T23:35:21.112 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:20 smithi043 podman[106429]: 2024-09-11 23:35:20.708546579 +0000 UTC m=+0.016076106 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:35:21.112 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:20 smithi043 podman[106429]: 2024-09-11 23:35:20.822592331 +0000 UTC m=+0.130121887 container create 50c35e119e410b452477fa0849aba3f83bb7e990183eb6afae4a3e0685ffc37c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD) 2024-09-11T23:35:21.112 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:21 smithi043 podman[106429]: 2024-09-11 23:35:21.041450269 +0000 UTC m=+0.348979793 container init 50c35e119e410b452477fa0849aba3f83bb7e990183eb6afae4a3e0685ffc37c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-11T23:35:21.113 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:21 smithi043 podman[106429]: 2024-09-11 23:35:21.044826041 +0000 UTC m=+0.352355561 container start 50c35e119e410b452477fa0849aba3f83bb7e990183eb6afae4a3e0685ffc37c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-11T23:35:21.300 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:20 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:20.981+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:21.551 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:21 smithi043 podman[106429]: 2024-09-11 23:35:21.132947115 +0000 UTC m=+0.440476652 container attach 50c35e119e410b452477fa0849aba3f83bb7e990183eb6afae4a3e0685ffc37c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-11T23:35:21.552 INFO:journalctl@ceph.osd.9.smithi043.stdout:Sep 11 23:35:21 smithi043 podman[106429]: 2024-09-11 23:35:21.243883531 +0000 UTC m=+0.551413048 container died 50c35e119e410b452477fa0849aba3f83bb7e990183eb6afae4a3e0685ffc37c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-11T23:35:21.827 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.9.service' 2024-09-11T23:35:21.868 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:35:21.869 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-11T23:35:21.869 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-11T23:35:21.869 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.10 2024-09-11T23:35:21.895 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:21 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:21.618+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:21.896 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:21 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:21.618+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-09-11T23:34:08.273947+0000 front 2024-09-11T23:34:08.273916+0000 (oldest deadline 2024-09-11T23:35:14.174121+0000) 2024-09-11T23:35:22.187 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:21 smithi090 systemd[1]: Stopping Ceph osd.10 for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:35:22.300 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:21 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:21.985+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:22.519 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:22 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:22.187+0000 7fc85a586640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:35:22.519 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:22 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:22.187+0000 7fc85a586640 -1 osd.10 98 *** Got signal Terminated *** 2024-09-11T23:35:22.519 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:22 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:22.187+0000 7fc85a586640 -1 osd.10 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:35:23.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:22 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:22.579+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:23.020 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:22 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:22.579+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-09-11T23:34:08.273947+0000 front 2024-09-11T23:34:08.273916+0000 (oldest deadline 2024-09-11T23:35:14.174121+0000) 2024-09-11T23:35:23.020 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:22 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:22.579+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2024-09-11T23:34:20.575893+0000 front 2024-09-11T23:34:20.575788+0000 (oldest deadline 2024-09-11T23:35:22.275656+0000) 2024-09-11T23:35:23.300 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:22 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:22.978+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:24.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:23 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:23.571+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:24.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:23 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:23.571+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-09-11T23:34:08.273947+0000 front 2024-09-11T23:34:08.273916+0000 (oldest deadline 2024-09-11T23:35:14.174121+0000) 2024-09-11T23:35:24.020 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:23 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:23.571+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2024-09-11T23:34:20.575893+0000 front 2024-09-11T23:34:20.575788+0000 (oldest deadline 2024-09-11T23:35:22.275656+0000) 2024-09-11T23:35:24.300 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:24 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:24.027+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:25.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:24 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:24.582+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:25.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:24 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:24.582+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-09-11T23:34:08.273947+0000 front 2024-09-11T23:34:08.273916+0000 (oldest deadline 2024-09-11T23:35:14.174121+0000) 2024-09-11T23:35:25.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:24 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:24.582+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2024-09-11T23:34:20.575893+0000 front 2024-09-11T23:34:20.575788+0000 (oldest deadline 2024-09-11T23:35:22.275656+0000) 2024-09-11T23:35:25.550 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:25 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:25.074+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:26.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:25 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:25.603+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:26.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:25 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:25.603+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-09-11T23:34:08.273947+0000 front 2024-09-11T23:34:08.273916+0000 (oldest deadline 2024-09-11T23:35:14.174121+0000) 2024-09-11T23:35:26.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:25 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:25.603+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2024-09-11T23:34:20.575893+0000 front 2024-09-11T23:34:20.575788+0000 (oldest deadline 2024-09-11T23:35:22.275656+0000) 2024-09-11T23:35:26.550 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:26 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:26.065+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:27.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:26 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:26.568+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:03.673084+0000 front 2024-09-11T23:34:03.673054+0000 (oldest deadline 2024-09-11T23:35:05.373106+0000) 2024-09-11T23:35:27.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:26 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:26.568+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6804 osd.1 since back 2024-09-11T23:34:08.273947+0000 front 2024-09-11T23:34:08.273916+0000 (oldest deadline 2024-09-11T23:35:14.174121+0000) 2024-09-11T23:35:27.019 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:26 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10[59837]: 2024-09-11T23:35:26.568+0000 7fc856b9c640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2024-09-11T23:34:20.575893+0000 front 2024-09-11T23:34:20.575788+0000 (oldest deadline 2024-09-11T23:35:22.275656+0000) 2024-09-11T23:35:27.550 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:27 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:27.106+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:27.550 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:27 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:27.106+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2024-09-11T23:34:20.275036+0000 front 2024-09-11T23:34:20.275161+0000 (oldest deadline 2024-09-11T23:35:26.075570+0000) 2024-09-11T23:35:28.196 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:27 smithi090 podman[71675]: 2024-09-11 23:35:27.867009215 +0000 UTC m=+5.802345352 container died 2d6ef181a656606882b4f38d9169634041b2ff892f5d0dcd24d7892535aed006 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-11T23:35:28.519 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:28 smithi090 podman[71675]: 2024-09-11 23:35:28.313731696 +0000 UTC m=+6.249067831 container cleanup 2d6ef181a656606882b4f38d9169634041b2ff892f5d0dcd24d7892535aed006 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-11T23:35:28.519 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:28 smithi090 bash[71675]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10 2024-09-11T23:35:28.550 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:28 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:28.102+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:28.550 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:28 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:28.102+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2024-09-11T23:34:20.275036+0000 front 2024-09-11T23:34:20.275161+0000 (oldest deadline 2024-09-11T23:35:26.075570+0000) 2024-09-11T23:35:29.005 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:28 smithi090 podman[71690]: 2024-09-11 23:35:28.561942733 +0000 UTC m=+0.688561175 container remove 2d6ef181a656606882b4f38d9169634041b2ff892f5d0dcd24d7892535aed006 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:35:29.269 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:29 smithi090 podman[71787]: 2024-09-11 23:35:28.909230582 +0000 UTC m=+0.015988706 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:35:29.269 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:29 smithi090 podman[71787]: 2024-09-11 23:35:29.009281483 +0000 UTC m=+0.116039607 container create 66aa22d09d08023183517b8e147b974f89c479799c9cefc9aefe8e0d6ca4c412 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T23:35:29.269 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:29 smithi090 podman[71787]: 2024-09-11 23:35:29.212060779 +0000 UTC m=+0.318818897 container init 66aa22d09d08023183517b8e147b974f89c479799c9cefc9aefe8e0d6ca4c412 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:35:29.270 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:29 smithi090 podman[71787]: 2024-09-11 23:35:29.215303527 +0000 UTC m=+0.322061646 container start 66aa22d09d08023183517b8e147b974f89c479799c9cefc9aefe8e0d6ca4c412 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-11T23:35:29.551 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:29 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:29.087+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:29.551 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:29 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:29.087+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2024-09-11T23:34:20.275036+0000 front 2024-09-11T23:34:20.275161+0000 (oldest deadline 2024-09-11T23:35:26.075570+0000) 2024-09-11T23:35:29.698 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:29 smithi090 podman[71787]: 2024-09-11 23:35:29.312144698 +0000 UTC m=+0.418902840 container attach 66aa22d09d08023183517b8e147b974f89c479799c9cefc9aefe8e0d6ca4c412 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-11T23:35:29.698 INFO:journalctl@ceph.osd.10.smithi090.stdout:Sep 11 23:35:29 smithi090 podman[71787]: 2024-09-11 23:35:29.40738063 +0000 UTC m=+0.514138750 container died 66aa22d09d08023183517b8e147b974f89c479799c9cefc9aefe8e0d6ca4c412 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-11T23:35:29.988 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.10.service' 2024-09-11T23:35:30.070 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:35:30.070 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-11T23:35:30.070 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-11T23:35:30.070 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.11 2024-09-11T23:35:30.391 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:30 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:30.118+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:30.391 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:30 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:30.118+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2024-09-11T23:34:20.275036+0000 front 2024-09-11T23:34:20.275161+0000 (oldest deadline 2024-09-11T23:35:26.075570+0000) 2024-09-11T23:35:30.391 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:30 smithi097 systemd[1]: Stopping Ceph osd.11 for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:35:30.391 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:30 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:30.391+0000 7f8ab0804640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:35:30.392 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:30 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:30.391+0000 7f8ab0804640 -1 osd.11 98 *** Got signal Terminated *** 2024-09-11T23:35:30.392 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:30 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:30.391+0000 7f8ab0804640 -1 osd.11 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:35:31.553 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:31 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:31.106+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:31.553 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:31 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:31.106+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2024-09-11T23:34:20.275036+0000 front 2024-09-11T23:34:20.275161+0000 (oldest deadline 2024-09-11T23:35:26.075570+0000) 2024-09-11T23:35:32.550 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:32 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:32.088+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:32.551 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:32 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:32.088+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2024-09-11T23:34:20.275036+0000 front 2024-09-11T23:34:20.275161+0000 (oldest deadline 2024-09-11T23:35:26.075570+0000) 2024-09-11T23:35:32.551 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:32 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:32.088+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-11T23:34:26.576650+0000 front 2024-09-11T23:34:26.576512+0000 (oldest deadline 2024-09-11T23:35:31.276502+0000) 2024-09-11T23:35:33.551 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:33 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:33.085+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:33.551 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:33 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:33.085+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2024-09-11T23:34:20.275036+0000 front 2024-09-11T23:34:20.275161+0000 (oldest deadline 2024-09-11T23:35:26.075570+0000) 2024-09-11T23:35:33.551 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:33 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:33.085+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-11T23:34:26.576650+0000 front 2024-09-11T23:34:26.576512+0000 (oldest deadline 2024-09-11T23:35:31.276502+0000) 2024-09-11T23:35:34.551 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:34 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:34.061+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:34.551 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:34 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:34.061+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2024-09-11T23:34:20.275036+0000 front 2024-09-11T23:34:20.275161+0000 (oldest deadline 2024-09-11T23:35:26.075570+0000) 2024-09-11T23:35:34.551 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:34 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:34.061+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-11T23:34:26.576650+0000 front 2024-09-11T23:34:26.576512+0000 (oldest deadline 2024-09-11T23:35:31.276502+0000) 2024-09-11T23:35:35.550 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:35 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:35.107+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-09-11T23:34:01.071718+0000 front 2024-09-11T23:34:01.071756+0000 (oldest deadline 2024-09-11T23:35:06.971668+0000) 2024-09-11T23:35:35.551 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:35 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:35.107+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.97:6804 osd.2 since back 2024-09-11T23:34:20.275036+0000 front 2024-09-11T23:34:20.275161+0000 (oldest deadline 2024-09-11T23:35:26.075570+0000) 2024-09-11T23:35:35.551 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:35 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11[60630]: 2024-09-11T23:35:35.107+0000 7f8aace1a640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-09-11T23:34:26.576650+0000 front 2024-09-11T23:34:26.576512+0000 (oldest deadline 2024-09-11T23:35:31.276502+0000) 2024-09-11T23:35:36.300 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:36 smithi097 podman[72089]: 2024-09-11 23:35:36.037401212 +0000 UTC m=+5.770323021 container died cfd0a54a2a0023501643c4562def03e6625caf28ce556d244ed4b209c35abee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:35:36.726 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:36 smithi097 podman[72089]: 2024-09-11 23:35:36.424579215 +0000 UTC m=+6.157501020 container cleanup cfd0a54a2a0023501643c4562def03e6625caf28ce556d244ed4b209c35abee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-11T23:35:36.726 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:36 smithi097 bash[72089]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11 2024-09-11T23:35:37.050 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:36 smithi097 podman[72104]: 2024-09-11 23:35:36.726968101 +0000 UTC m=+0.689304687 container remove cfd0a54a2a0023501643c4562def03e6625caf28ce556d244ed4b209c35abee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-11T23:35:37.328 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:37 smithi097 podman[72195]: 2024-09-11 23:35:36.978650065 +0000 UTC m=+0.015702881 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:35:37.328 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:37 smithi097 podman[72195]: 2024-09-11 23:35:37.093385224 +0000 UTC m=+0.130438031 container create 2824fa675fe8daa7e4c521337acc1d273548033779fcf41d241a045ed950c0e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T23:35:37.328 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:37 smithi097 podman[72195]: 2024-09-11 23:35:37.328863691 +0000 UTC m=+0.365916496 container init 2824fa675fe8daa7e4c521337acc1d273548033779fcf41d241a045ed950c0e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-09-11T23:35:37.801 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:37 smithi097 podman[72195]: 2024-09-11 23:35:37.332288615 +0000 UTC m=+0.369341429 container start 2824fa675fe8daa7e4c521337acc1d273548033779fcf41d241a045ed950c0e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-11T23:35:37.801 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:37 smithi097 podman[72195]: 2024-09-11 23:35:37.420604288 +0000 UTC m=+0.457657121 container attach 2824fa675fe8daa7e4c521337acc1d273548033779fcf41d241a045ed950c0e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3) 2024-09-11T23:35:37.801 INFO:journalctl@ceph.osd.11.smithi097.stdout:Sep 11 23:35:37 smithi097 podman[72195]: 2024-09-11 23:35:37.519893734 +0000 UTC m=+0.556946553 container died 2824fa675fe8daa7e4c521337acc1d273548033779fcf41d241a045ed950c0e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-11T23:35:38.099 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@osd.11.service' 2024-09-11T23:35:38.138 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:35:38.139 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-11T23:35:38.139 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-11T23:35:38.139 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.a 2024-09-11T23:35:38.429 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 11 23:35:38 smithi043 systemd[1]: Stopping Ceph mds.a for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:35:38.860 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 11 23:35:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-a[71737]: 2024-09-11T23:35:38.429+0000 7efe97d84640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:35:38.860 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 11 23:35:38 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-a[71737]: 2024-09-11T23:35:38.429+0000 7efe97d84640 -1 mds.a *** got signal Terminated *** 2024-09-11T23:35:42.770 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 11 23:35:42 smithi043 podman[106530]: 2024-09-11 23:35:42.470160683 +0000 UTC m=+4.133126078 container died 0697bba84f7eed955a6b3a171ada4b947d183695cf5bb094fd16dd7f7cb8e995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-a, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-11T23:35:43.110 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 11 23:35:42 smithi043 podman[106530]: 2024-09-11 23:35:42.953764337 +0000 UTC m=+4.616729775 container cleanup 0697bba84f7eed955a6b3a171ada4b947d183695cf5bb094fd16dd7f7cb8e995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-a, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:35:43.110 INFO:journalctl@ceph.mds.a.smithi043.stdout:Sep 11 23:35:42 smithi043 bash[106530]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-a 2024-09-11T23:35:43.472 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.a.service' 2024-09-11T23:35:43.508 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:35:43.508 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-11T23:35:43.509 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-11T23:35:43.509 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.d 2024-09-11T23:35:43.860 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 11 23:35:43 smithi043 systemd[1]: Stopping Ceph mds.d for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:35:43.860 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 11 23:35:43 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-d[70559]: 2024-09-11T23:35:43.835+0000 7f3a56a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:35:43.860 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 11 23:35:43 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-d[70559]: 2024-09-11T23:35:43.835+0000 7f3a56a00640 -1 mds.d *** got signal Terminated *** 2024-09-11T23:35:47.610 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 11 23:35:47 smithi043 podman[106652]: 2024-09-11 23:35:47.232795079 +0000 UTC m=+3.492726965 container died 03d0b56e736e1dce4e96ca0011f57ecd8a34b64f143f0687a2d102b9b2d195a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-d, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-11T23:35:48.001 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 11 23:35:47 smithi043 podman[106652]: 2024-09-11 23:35:47.717954749 +0000 UTC m=+3.977886623 container cleanup 03d0b56e736e1dce4e96ca0011f57ecd8a34b64f143f0687a2d102b9b2d195a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-d, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-11T23:35:48.001 INFO:journalctl@ceph.mds.d.smithi043.stdout:Sep 11 23:35:47 smithi043 bash[106652]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-d 2024-09-11T23:35:48.141 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.d.service' 2024-09-11T23:35:48.175 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:35:48.176 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-11T23:35:48.176 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-11T23:35:48.176 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.g 2024-09-11T23:35:48.610 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 11 23:35:48 smithi043 systemd[1]: Stopping Ceph mds.g for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:35:48.610 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 11 23:35:48 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-g[71351]: 2024-09-11T23:35:48.504+0000 7f5441f0c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:35:48.610 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 11 23:35:48 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-g[71351]: 2024-09-11T23:35:48.504+0000 7f5441f0c640 -1 mds.g *** got signal Terminated *** 2024-09-11T23:35:51.505 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 11 23:35:51 smithi043 podman[106772]: 2024-09-11 23:35:51.224531047 +0000 UTC m=+2.825727473 container died e7f1f9ed7f3cde862ea82ce0b67471ae715698dd5477d1133d3cf51589f02e84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-g, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T23:35:51.860 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 11 23:35:51 smithi043 podman[106772]: 2024-09-11 23:35:51.612954575 +0000 UTC m=+3.214150996 container cleanup e7f1f9ed7f3cde862ea82ce0b67471ae715698dd5477d1133d3cf51589f02e84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-11T23:35:51.860 INFO:journalctl@ceph.mds.g.smithi043.stdout:Sep 11 23:35:51 smithi043 bash[106772]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-g 2024-09-11T23:35:52.044 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.g.service' 2024-09-11T23:35:52.080 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:35:52.080 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-11T23:35:52.080 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-11T23:35:52.080 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.j 2024-09-11T23:35:52.610 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 11 23:35:52 smithi043 systemd[1]: Stopping Ceph mds.j for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:35:52.610 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 11 23:35:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-j[70178]: 2024-09-11T23:35:52.384+0000 7fc988363640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:35:52.610 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 11 23:35:52 smithi043 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-j[70178]: 2024-09-11T23:35:52.384+0000 7fc988363640 -1 mds.j *** got signal Terminated *** 2024-09-11T23:35:55.832 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 11 23:35:55 smithi043 podman[106902]: 2024-09-11 23:35:55.543001874 +0000 UTC m=+3.239554383 container died fa72d4c80cdc7e4f02fa21eb47aecbaf10bae1607e1ef1f562647bbe87af75ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-11T23:35:55.833 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 11 23:35:55 smithi043 podman[106902]: 2024-09-11 23:35:55.833556369 +0000 UTC m=+3.530108879 container cleanup fa72d4c80cdc7e4f02fa21eb47aecbaf10bae1607e1ef1f562647bbe87af75ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-11T23:35:56.110 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 11 23:35:55 smithi043 bash[106902]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-j 2024-09-11T23:35:56.110 INFO:journalctl@ceph.mds.j.smithi043.stdout:Sep 11 23:35:56 smithi043 podman[106915]: 2024-09-11 23:35:56.059343429 +0000 UTC m=+0.508829718 container remove fa72d4c80cdc7e4f02fa21eb47aecbaf10bae1607e1ef1f562647bbe87af75ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-j, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:35:56.186 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.j.service' 2024-09-11T23:35:56.221 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:35:56.221 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-11T23:35:56.221 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-11T23:35:56.222 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.b 2024-09-11T23:35:56.513 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:35:56 smithi090 systemd[1]: Stopping Ceph mds.b for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:35:56.769 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:35:56 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-b[65178]: 2024-09-11T23:35:56.512+0000 7fc0f4171640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:35:56.769 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:35:56 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-b[65178]: 2024-09-11T23:35:56.512+0000 7fc0f4171640 -1 mds.b *** got signal Terminated *** 2024-09-11T23:36:00.269 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:36:00 smithi090 podman[71891]: 2024-09-11 23:36:00.016412361 +0000 UTC m=+3.596933164 container died de004e4ae010e8cf6dd3c3c0c5065f1151ba789a005747f71328c46a8290f763 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-11T23:36:00.672 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:36:00 smithi090 podman[71891]: 2024-09-11 23:36:00.371748013 +0000 UTC m=+3.952268857 container cleanup de004e4ae010e8cf6dd3c3c0c5065f1151ba789a005747f71328c46a8290f763 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-11T23:36:00.672 INFO:journalctl@ceph.mds.b.smithi090.stdout:Sep 11 23:36:00 smithi090 bash[71891]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-b 2024-09-11T23:36:00.812 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.b.service' 2024-09-11T23:36:00.847 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:36:00.848 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-11T23:36:00.848 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-11T23:36:00.848 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.e 2024-09-11T23:36:01.269 INFO:journalctl@ceph.mds.e.smithi090.stdout:Sep 11 23:36:00 smithi090 systemd[1]: Stopping Ceph mds.e for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:36:01.269 INFO:journalctl@ceph.mds.e.smithi090.stdout:Sep 11 23:36:01 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-e[64566]: 2024-09-11T23:36:01.166+0000 7ff961875640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:36:01.269 INFO:journalctl@ceph.mds.e.smithi090.stdout:Sep 11 23:36:01 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-e[64566]: 2024-09-11T23:36:01.166+0000 7ff961875640 -1 mds.e *** got signal Terminated *** 2024-09-11T23:36:04.269 INFO:journalctl@ceph.mds.e.smithi090.stdout:Sep 11 23:36:03 smithi090 podman[72013]: 2024-09-11 23:36:03.98109799 +0000 UTC m=+2.915946956 container died 53810d43153a5c8a33fb0bb1a260c2f97e6bc32eba03e8c46b9eca314bc1f2ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:36:04.620 INFO:journalctl@ceph.mds.e.smithi090.stdout:Sep 11 23:36:04 smithi090 podman[72013]: 2024-09-11 23:36:04.378295636 +0000 UTC m=+3.313144602 container cleanup 53810d43153a5c8a33fb0bb1a260c2f97e6bc32eba03e8c46b9eca314bc1f2ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-09-11T23:36:04.621 INFO:journalctl@ceph.mds.e.smithi090.stdout:Sep 11 23:36:04 smithi090 bash[72013]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-e 2024-09-11T23:36:04.743 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.e.service' 2024-09-11T23:36:04.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:36:04.780 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-11T23:36:04.780 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-11T23:36:04.781 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.h 2024-09-11T23:36:05.019 INFO:journalctl@ceph.mds.h.smithi090.stdout:Sep 11 23:36:04 smithi090 systemd[1]: Stopping Ceph mds.h for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:36:05.519 INFO:journalctl@ceph.mds.h.smithi090.stdout:Sep 11 23:36:05 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-h[64868]: 2024-09-11T23:36:05.132+0000 7fc0d9018640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:36:05.519 INFO:journalctl@ceph.mds.h.smithi090.stdout:Sep 11 23:36:05 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-h[64868]: 2024-09-11T23:36:05.132+0000 7fc0d9018640 -1 mds.h *** got signal Terminated *** 2024-09-11T23:36:08.242 INFO:journalctl@ceph.mds.h.smithi090.stdout:Sep 11 23:36:07 smithi090 podman[72146]: 2024-09-11 23:36:07.97084079 +0000 UTC m=+2.974733511 container died a55012e7c4519e7332fc5e9163b992db7b3f58367436666ef4cbe9c5ff98cc21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-11T23:36:08.519 INFO:journalctl@ceph.mds.h.smithi090.stdout:Sep 11 23:36:08 smithi090 podman[72146]: 2024-09-11 23:36:08.324863881 +0000 UTC m=+3.328756592 container cleanup a55012e7c4519e7332fc5e9163b992db7b3f58367436666ef4cbe9c5ff98cc21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-h, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-09-11T23:36:08.520 INFO:journalctl@ceph.mds.h.smithi090.stdout:Sep 11 23:36:08 smithi090 bash[72146]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-h 2024-09-11T23:36:08.755 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.h.service' 2024-09-11T23:36:08.789 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:36:08.790 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-11T23:36:08.790 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-11T23:36:08.790 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.k 2024-09-11T23:36:09.155 INFO:journalctl@ceph.mds.k.smithi090.stdout:Sep 11 23:36:08 smithi090 systemd[1]: Stopping Ceph mds.k for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:36:09.520 INFO:journalctl@ceph.mds.k.smithi090.stdout:Sep 11 23:36:09 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-k[64251]: 2024-09-11T23:36:09.155+0000 7f9cb51d9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:36:09.520 INFO:journalctl@ceph.mds.k.smithi090.stdout:Sep 11 23:36:09 smithi090 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-k[64251]: 2024-09-11T23:36:09.155+0000 7f9cb51d9640 -1 mds.k *** got signal Terminated *** 2024-09-11T23:36:12.097 INFO:journalctl@ceph.mds.k.smithi090.stdout:Sep 11 23:36:11 smithi090 podman[72268]: 2024-09-11 23:36:11.781584434 +0000 UTC m=+2.770475093 container died c753dd3ba9cff24311b537dd46ba63726ac18d892539ec701922af3337b6ebd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-k, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-11T23:36:12.097 INFO:journalctl@ceph.mds.k.smithi090.stdout:Sep 11 23:36:12 smithi090 podman[72268]: 2024-09-11 23:36:12.098122136 +0000 UTC m=+3.087012794 container cleanup c753dd3ba9cff24311b537dd46ba63726ac18d892539ec701922af3337b6ebd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T23:36:12.510 INFO:journalctl@ceph.mds.k.smithi090.stdout:Sep 11 23:36:12 smithi090 bash[72268]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-k 2024-09-11T23:36:12.510 INFO:journalctl@ceph.mds.k.smithi090.stdout:Sep 11 23:36:12 smithi090 podman[72283]: 2024-09-11 23:36:12.307976875 +0000 UTC m=+0.519381647 container remove c753dd3ba9cff24311b537dd46ba63726ac18d892539ec701922af3337b6ebd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T23:36:12.520 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.k.service' 2024-09-11T23:36:12.553 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:36:12.554 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-11T23:36:12.554 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-11T23:36:12.554 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.c 2024-09-11T23:36:12.907 INFO:journalctl@ceph.mds.c.smithi097.stdout:Sep 11 23:36:12 smithi097 systemd[1]: Stopping Ceph mds.c for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:36:13.169 INFO:journalctl@ceph.mds.c.smithi097.stdout:Sep 11 23:36:12 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-c[65456]: 2024-09-11T23:36:12.907+0000 7f7681d35640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:36:13.169 INFO:journalctl@ceph.mds.c.smithi097.stdout:Sep 11 23:36:12 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-c[65456]: 2024-09-11T23:36:12.907+0000 7f7681d35640 -1 mds.c *** got signal Terminated *** 2024-09-11T23:36:16.508 INFO:journalctl@ceph.mds.c.smithi097.stdout:Sep 11 23:36:16 smithi097 podman[72311]: 2024-09-11 23:36:16.059238747 +0000 UTC m=+3.311637939 container died 9eaae0bf15046461d7cb6a5331a3ea3198f61264d23dd43512df0392a366d5c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True) 2024-09-11T23:36:16.801 INFO:journalctl@ceph.mds.c.smithi097.stdout:Sep 11 23:36:16 smithi097 podman[72311]: 2024-09-11 23:36:16.59050297 +0000 UTC m=+3.842902161 container cleanup 9eaae0bf15046461d7cb6a5331a3ea3198f61264d23dd43512df0392a366d5c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-c, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T23:36:16.801 INFO:journalctl@ceph.mds.c.smithi097.stdout:Sep 11 23:36:16 smithi097 bash[72311]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-c 2024-09-11T23:36:17.026 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.c.service' 2024-09-11T23:36:17.064 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:36:17.064 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-11T23:36:17.064 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-11T23:36:17.064 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.f 2024-09-11T23:36:17.386 INFO:journalctl@ceph.mds.f.smithi097.stdout:Sep 11 23:36:17 smithi097 systemd[1]: Stopping Ceph mds.f for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:36:17.800 INFO:journalctl@ceph.mds.f.smithi097.stdout:Sep 11 23:36:17 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-f[65154]: 2024-09-11T23:36:17.385+0000 7fd5b6cab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:36:17.800 INFO:journalctl@ceph.mds.f.smithi097.stdout:Sep 11 23:36:17 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-f[65154]: 2024-09-11T23:36:17.385+0000 7fd5b6cab640 -1 mds.f *** got signal Terminated *** 2024-09-11T23:36:20.380 INFO:journalctl@ceph.mds.f.smithi097.stdout:Sep 11 23:36:20 smithi097 podman[72446]: 2024-09-11 23:36:20.060959857 +0000 UTC m=+2.780870250 container died 1c28dd39f10a21c7749a5e73b8aeea50859afe41cf94bb42edba1bf429c81edc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-f, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:36:20.753 INFO:journalctl@ceph.mds.f.smithi097.stdout:Sep 11 23:36:20 smithi097 podman[72446]: 2024-09-11 23:36:20.469554624 +0000 UTC m=+3.189465016 container cleanup 1c28dd39f10a21c7749a5e73b8aeea50859afe41cf94bb42edba1bf429c81edc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-f, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T23:36:20.753 INFO:journalctl@ceph.mds.f.smithi097.stdout:Sep 11 23:36:20 smithi097 bash[72446]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-f 2024-09-11T23:36:20.905 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.f.service' 2024-09-11T23:36:20.939 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:36:20.939 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-11T23:36:20.939 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-11T23:36:20.939 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.i 2024-09-11T23:36:21.301 INFO:journalctl@ceph.mds.i.smithi097.stdout:Sep 11 23:36:21 smithi097 systemd[1]: Stopping Ceph mds.i for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:36:21.301 INFO:journalctl@ceph.mds.i.smithi097.stdout:Sep 11 23:36:21 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-i[65760]: 2024-09-11T23:36:21.272+0000 7fc9aa374640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:36:21.301 INFO:journalctl@ceph.mds.i.smithi097.stdout:Sep 11 23:36:21 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-i[65760]: 2024-09-11T23:36:21.272+0000 7fc9aa374640 -1 mds.i *** got signal Terminated *** 2024-09-11T23:36:24.398 INFO:journalctl@ceph.mds.i.smithi097.stdout:Sep 11 23:36:24 smithi097 podman[72568]: 2024-09-11 23:36:24.058370478 +0000 UTC m=+2.891649315 container died 47fd2598a74d92817b10daf8ed346baa155827baa7ea36158642ce0424b2cc97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:36:24.698 INFO:journalctl@ceph.mds.i.smithi097.stdout:Sep 11 23:36:24 smithi097 podman[72568]: 2024-09-11 23:36:24.498403124 +0000 UTC m=+3.331681972 container cleanup 47fd2598a74d92817b10daf8ed346baa155827baa7ea36158642ce0424b2cc97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-i, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-11T23:36:24.699 INFO:journalctl@ceph.mds.i.smithi097.stdout:Sep 11 23:36:24 smithi097 bash[72568]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-i 2024-09-11T23:36:24.916 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.i.service' 2024-09-11T23:36:24.949 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:36:24.949 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-11T23:36:24.950 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-11T23:36:24.950 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.l 2024-09-11T23:36:25.300 INFO:journalctl@ceph.mds.l.smithi097.stdout:Sep 11 23:36:25 smithi097 systemd[1]: Stopping Ceph mds.l for 25c0cd84-7093-11ef-bcea-c7b262605968... 2024-09-11T23:36:25.800 INFO:journalctl@ceph.mds.l.smithi097.stdout:Sep 11 23:36:25 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-l[64855]: 2024-09-11T23:36:25.342+0000 7f182c344640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:36:25.801 INFO:journalctl@ceph.mds.l.smithi097.stdout:Sep 11 23:36:25 smithi097 ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-l[64855]: 2024-09-11T23:36:25.342+0000 7f182c344640 -1 mds.l *** got signal Terminated *** 2024-09-11T23:36:29.350 INFO:journalctl@ceph.mds.l.smithi097.stdout:Sep 11 23:36:29 smithi097 podman[72692]: 2024-09-11 23:36:29.070489955 +0000 UTC m=+3.904793801 container died a21dd9902d79295cb1c8c769a1528139502b09f937ca0732781e7d37fd02aec2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-l, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-11T23:36:29.719 INFO:journalctl@ceph.mds.l.smithi097.stdout:Sep 11 23:36:29 smithi097 podman[72692]: 2024-09-11 23:36:29.45242032 +0000 UTC m=+4.286724157 container cleanup a21dd9902d79295cb1c8c769a1528139502b09f937ca0732781e7d37fd02aec2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-11T23:36:29.719 INFO:journalctl@ceph.mds.l.smithi097.stdout:Sep 11 23:36:29 smithi097 bash[72692]: ceph-25c0cd84-7093-11ef-bcea-c7b262605968-mds-l 2024-09-11T23:36:29.859 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-25c0cd84-7093-11ef-bcea-c7b262605968@mds.l.service' 2024-09-11T23:36:29.892 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:36:29.892 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-11T23:36:29.892 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T23:36:30.106 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:36:32.727 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T23:36:32.929 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:36:35.474 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T23:36:35.677 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:36:38.185 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:36:38.221 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:36:38.253 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:36:38.280 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-11T23:36:38.282 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901461/remote/smithi043/crash 2024-09-11T23:36:38.283 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/crash -- . 2024-09-11T23:36:38.322 INFO:teuthology.orchestra.run.smithi043.stderr:tar: /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T23:36:38.323 INFO:teuthology.orchestra.run.smithi043.stderr:tar: Error is not recoverable: exiting now 2024-09-11T23:36:38.325 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901461/remote/smithi090/crash 2024-09-11T23:36:38.325 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/crash -- . 2024-09-11T23:36:38.380 INFO:teuthology.orchestra.run.smithi090.stderr:tar: /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T23:36:38.380 INFO:teuthology.orchestra.run.smithi090.stderr:tar: Error is not recoverable: exiting now 2024-09-11T23:36:38.382 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901461/remote/smithi097/crash 2024-09-11T23:36:38.383 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/crash -- . 2024-09-11T23:36:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:tar: /var/lib/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T23:36:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:tar: Error is not recoverable: exiting now 2024-09-11T23:36:38.417 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-11T23:36:38.417 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-11T23:36:38.514 INFO:teuthology.orchestra.run.smithi043.stdout:2024-09-11T23:20:00.000156+0000 mon.a (mon.0) 1012 : cluster [WRN] application not enabled on pool 'cephfs_metadata' 2024-09-11T23:36:38.514 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-11T23:36:38.515 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[SEC\]' /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-11T23:36:38.553 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[ERR\]' /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-11T23:36:38.631 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[WRN\]' /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-11T23:36:38.707 INFO:teuthology.orchestra.run.smithi043.stdout:2024-09-11T23:20:00.000156+0000 mon.a (mon.0) 1012 : cluster [WRN] application not enabled on pool 'cephfs_metadata' 2024-09-11T23:36:38.707 INFO:tasks.cephadm:Compressing logs... 2024-09-11T23:36:38.708 DEBUG:teuthology.orchestra.run.smithi043:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T23:36:38.750 DEBUG:teuthology.orchestra.run.smithi090:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T23:36:38.752 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T23:36:38.787 INFO:teuthology.orchestra.run.smithi097.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-11T23:36:38.788 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T23:36:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T23:36:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-11T23:36:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-11T23:36:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T23:36:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-11T23:36:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-11T23:36:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-11T23:36:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.log 2024-09-11T23:36:38.793 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-11T23:36:38.793 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-11T23:36:38.793 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.log: /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-11T23:36:38.793 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-11T23:36:38.793 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-11T23:36:38.793 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T23:36:38.794 INFO:teuthology.orchestra.run.smithi090.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-11T23:36:38.794 INFO:teuthology.orchestra.run.smithi090.stderr:: No such file or directory 2024-09-11T23:36:38.795 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-11T23:36:38.795 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-11T23:36:38.795 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.10.log: /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T23:36:38.796 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-11T23:36:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-11T23:36:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-11T23:36:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-11T23:36:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T23:36:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T23:36:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-11T23:36:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.log 2024-09-11T23:36:38.799 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-11T23:36:38.799 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-11T23:36:38.799 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-11T23:36:38.799 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T23:36:38.799 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-11T23:36:38.803 INFO:teuthology.orchestra.run.smithi043.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-11T23:36:38.803 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-11T23:36:38.803 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-11T23:36:38.803 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T23:36:38.803 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T23:36:38.803 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose --/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-11T23:36:38.803 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T23:36:38.803 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-11T23:36:38.803 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-11T23:36:38.803 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-11T23:36:38.804 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mon.a.log: /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.log 2024-09-11T23:36:38.804 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-11T23:36:38.804 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-11T23:36:38.804 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-11T23:36:38.804 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-11T23:36:38.804 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T23:36:38.804 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 86.6% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T23:36:38.804 INFO:teuthology.orchestra.run.smithi043.stderr: 94.6% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-11T23:36:38.804 INFO:teuthology.orchestra.run.smithi043.stderr: 94.7% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-11T23:36:38.804 INFO:teuthology.orchestra.run.smithi043.stderr: 94.6% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-11T23:36:38.805 INFO:teuthology.orchestra.run.smithi043.stderr: 94.6% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-11T23:36:38.849 INFO:teuthology.orchestra.run.smithi043.stderr: 94.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T23:36:38.899 INFO:teuthology.orchestra.run.smithi097.stderr: 92.1% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-11T23:36:38.913 INFO:teuthology.orchestra.run.smithi043.stderr: 84.4% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T23:36:38.941 INFO:teuthology.orchestra.run.smithi090.stderr: 81.8% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T23:36:38.975 INFO:teuthology.orchestra.run.smithi097.stderr: 91.0% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T23:36:39.005 INFO:teuthology.orchestra.run.smithi043.stderr: 91.0% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T23:36:39.071 INFO:teuthology.orchestra.run.smithi043.stderr: 90.3% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T23:36:39.085 INFO:teuthology.orchestra.run.smithi090.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T23:36:39.181 INFO:teuthology.orchestra.run.smithi090.stderr: 91.0% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T23:36:39.207 INFO:teuthology.orchestra.run.smithi097.stderr: 81.7% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T23:36:39.248 INFO:teuthology.orchestra.run.smithi090.stderr: 91.9% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-11T23:36:39.262 INFO:teuthology.orchestra.run.smithi097.stderr: 92.3% 86.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T23:36:39.262 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T23:36:39.287 INFO:teuthology.orchestra.run.smithi090.stderr: 86.6% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T23:36:39.289 INFO:teuthology.orchestra.run.smithi097.stderr: 94.0% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-11T23:36:39.347 INFO:teuthology.orchestra.run.smithi090.stderr: 90.5% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T23:36:39.450 INFO:teuthology.orchestra.run.smithi097.stderr: 94.2% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-11T23:36:39.569 INFO:teuthology.orchestra.run.smithi097.stderr: 90.5% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T23:36:40.813 INFO:teuthology.orchestra.run.smithi043.stderr: 90.3% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-11T23:36:41.838 INFO:teuthology.orchestra.run.smithi097.stderr: 93.8% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-11T23:36:41.928 INFO:teuthology.orchestra.run.smithi097.stderr: 91.7% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-11T23:36:43.723 INFO:teuthology.orchestra.run.smithi090.stderr: 94.1% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-11T23:36:44.651 INFO:teuthology.orchestra.run.smithi090.stderr: 91.7% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-11T23:36:51.599 INFO:teuthology.orchestra.run.smithi043.stderr: 90.3% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-11T23:36:55.837 INFO:teuthology.orchestra.run.smithi090.stderr: 96.0% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-11T23:37:53.213 INFO:teuthology.orchestra.run.smithi090.stderr: 94.5% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-11T23:39:40.102 INFO:teuthology.orchestra.run.smithi043.stderr: 92.4% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-11T23:46:17.053 INFO:teuthology.orchestra.run.smithi097.stderr: 92.3% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-11T23:46:30.684 INFO:teuthology.orchestra.run.smithi090.stderr: 92.5% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-11T23:49:04.023 INFO:teuthology.orchestra.run.smithi090.stderr: 92.3% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-11T23:49:38.058 INFO:teuthology.orchestra.run.smithi090.stderr: 92.0% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-11T23:49:39.914 INFO:teuthology.orchestra.run.smithi090.stderr: 92.4% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-11T23:49:39.917 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-09-11T23:49:39.917 INFO:teuthology.orchestra.run.smithi090.stderr:real 13m1.150s 2024-09-11T23:49:39.918 INFO:teuthology.orchestra.run.smithi090.stderr:user 8m24.011s 2024-09-11T23:49:39.918 INFO:teuthology.orchestra.run.smithi090.stderr:sys 0m21.447s 2024-09-11T23:50:39.349 INFO:teuthology.orchestra.run.smithi043.stderr: 92.5% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-11T23:52:49.288 INFO:teuthology.orchestra.run.smithi097.stderr: 92.4% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-11T23:53:00.647 INFO:teuthology.orchestra.run.smithi043.stderr: 92.3% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-11T23:53:15.003 INFO:teuthology.orchestra.run.smithi043.stderr: 92.3% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-11T23:53:15.006 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-09-11T23:53:15.006 INFO:teuthology.orchestra.run.smithi043.stderr:real 16m36.243s 2024-09-11T23:53:15.006 INFO:teuthology.orchestra.run.smithi043.stderr:user 8m9.956s 2024-09-11T23:53:15.006 INFO:teuthology.orchestra.run.smithi043.stderr:sys 0m19.365s 2024-09-11T23:53:51.339 INFO:teuthology.orchestra.run.smithi097.stderr: 92.4% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-11T23:54:05.598 INFO:teuthology.orchestra.run.smithi097.stderr: 92.4% -- replaced with /var/log/ceph/25c0cd84-7093-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-11T23:54:05.601 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-09-11T23:54:05.601 INFO:teuthology.orchestra.run.smithi097.stderr:real 17m26.833s 2024-09-11T23:54:05.601 INFO:teuthology.orchestra.run.smithi097.stderr:user 7m58.299s 2024-09-11T23:54:05.601 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m20.865s 2024-09-11T23:54:05.602 INFO:tasks.cephadm:Archiving logs... 2024-09-11T23:54:05.602 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901461/remote/smithi043/log 2024-09-11T23:54:05.603 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T23:55:17.469 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901461/remote/smithi090/log 2024-09-11T23:55:17.471 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T23:56:35.996 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901461/remote/smithi097/log 2024-09-11T23:56:35.997 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T23:57:53.085 INFO:tasks.cephadm:Removing cluster... 2024-09-11T23:57:53.085 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 --force 2024-09-11T23:57:53.398 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:57:54.163 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 --force 2024-09-11T23:57:54.663 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:57:55.396 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 25c0cd84-7093-11ef-bcea-c7b262605968 --force 2024-09-11T23:57:55.800 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: 25c0cd84-7093-11ef-bcea-c7b262605968 2024-09-11T23:57:56.560 INFO:tasks.cephadm:Removing cephadm ... 2024-09-11T23:57:56.560 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T23:57:56.581 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T23:57:56.601 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T23:57:56.618 INFO:tasks.cephadm:Teardown complete 2024-09-11T23:57:56.619 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T23:57:56.635 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T23:57:56.636 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T23:57:56.638 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T23:57:56.644 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T23:57:56.805 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T23:57:56.817 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T23:57:56.818 DEBUG:teuthology.orchestra.run.smithi043:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:57:56.821 DEBUG:teuthology.orchestra.run.smithi090:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:57:56.824 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:57:56.836 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:57:56.838 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:57:56.838 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:57:56.870 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:57:56.870 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-09-11T23:57:56.870 INFO:teuthology.orchestra.run.smithi090.stdout:^* hv01.front.sepia.ceph.com 3 6 377 51 +1830us[+1855us] +/- 53ms 2024-09-11T23:57:56.870 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 55 -1480us[-1455us] +/- 79ms 2024-09-11T23:57:56.870 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 -1494us[-1469us] +/- 85ms 2024-09-11T23:57:56.870 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:57:56.877 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:57:56.878 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-09-11T23:57:56.878 INFO:teuthology.orchestra.run.smithi043.stdout:^* hv01.front.sepia.ceph.com 3 6 377 61 +1913us[+1862us] +/- 53ms 2024-09-11T23:57:56.878 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 59 -1416us[-1416us] +/- 79ms 2024-09-11T23:57:56.878 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 -1424us[-1424us] +/- 85ms 2024-09-11T23:57:56.878 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:57:56.885 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:57:56.885 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-09-11T23:57:56.885 INFO:teuthology.orchestra.run.smithi097.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 +1837us[+1846us] +/- 53ms 2024-09-11T23:57:56.886 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 56 -1502us[-1493us] +/- 79ms 2024-09-11T23:57:56.887 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 -1524us[-1514us] +/- 85ms 2024-09-11T23:57:56.887 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:57:56.887 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T23:57:56.898 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T23:57:56.899 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T23:57:56.921 DEBUG:teuthology.orchestra.run.smithi043:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T23:57:57.089 DEBUG:teuthology.orchestra.run.smithi090:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T23:57:57.187 DEBUG:teuthology.orchestra.run.smithi097:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T23:57:57.303 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:57:57.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:57:57.337 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:57:57.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:57:57.370 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:57:57.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:57:57.399 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T23:57:57.411 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T23:57:57.454 INFO:teuthology.task.internal:Duration was 3151.360316 seconds 2024-09-11T23:57:57.455 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T23:57:57.488 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T23:57:57.489 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T23:57:57.491 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T23:57:57.495 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T23:57:57.546 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:57:57.547 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:57:57.556 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:57:57.855 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T23:57:57.856 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi043.front.sepia.ceph.com 2024-09-11T23:57:57.856 DEBUG:teuthology.orchestra.run.smithi043:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T23:57:57.882 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi090.front.sepia.ceph.com 2024-09-11T23:57:57.882 DEBUG:teuthology.orchestra.run.smithi090:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T23:57:57.910 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2024-09-11T23:57:57.910 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T23:57:57.944 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T23:57:57.944 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T23:57:57.947 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T23:57:57.953 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T23:57:58.013 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T23:57:58.013 DEBUG:teuthology.orchestra.run.smithi043:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T23:57:58.018 DEBUG:teuthology.orchestra.run.smithi090:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T23:57:58.025 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T23:57:58.925 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T23:57:58.940 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T23:57:58.940 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T23:57:58.978 DEBUG:teuthology.orchestra.run.smithi090:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T23:57:59.053 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T23:57:59.106 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T23:57:59.122 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:57:59.126 DEBUG:teuthology.orchestra.run.smithi090:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:57:59.128 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:57:59.156 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = core 2024-09-11T23:57:59.161 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = core 2024-09-11T23:57:59.164 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2024-09-11T23:57:59.195 DEBUG:teuthology.orchestra.run.smithi043:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:57:59.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:57:59.231 DEBUG:teuthology.orchestra.run.smithi090:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:57:59.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:57:59.246 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:57:59.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:57:59.262 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T23:57:59.274 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T23:57:59.275 DEBUG:teuthology.misc:Transferring archived files from smithi043:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901461/remote/smithi043 2024-09-11T23:57:59.275 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T23:57:59.337 DEBUG:teuthology.misc:Transferring archived files from smithi090:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901461/remote/smithi090 2024-09-11T23:57:59.337 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T23:57:59.376 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901461/remote/smithi097 2024-09-11T23:57:59.376 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T23:57:59.419 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T23:57:59.419 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T23:57:59.422 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T23:57:59.424 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T23:57:59.468 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T23:57:59.482 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T23:57:59.483 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T23:57:59.505 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T23:57:59.506 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T23:57:59.509 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T23:57:59.511 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T23:57:59.522 INFO:teuthology.orchestra.run.smithi043.stdout: 265966 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 23:57 /home/ubuntu/cephtest 2024-09-11T23:57:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 23:57 /home/ubuntu/cephtest 2024-09-11T23:57:59.526 INFO:teuthology.orchestra.run.smithi097.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 23:57 /home/ubuntu/cephtest 2024-09-11T23:57:59.528 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T23:57:59.540 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T23:57:59.609 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} duration: 3151.360315799713 failure_reason: '"2024-09-11T23:20:00.000156+0000 mon.a (mon.0) 1012 : cluster [WRN] application not enabled on pool ''cephfs_metadata''" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-09-11T23:57:59.609 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T23:57:59.704 INFO:teuthology.run:FAIL